Power scene defect small target detection method based on Gaussian mask supervision and cross-layer attention guidance

Abstract

The invention discloses an electric power scene defect small target detection method based on Gaussian mask supervision and cross-layer attention guidance, and the method comprises the steps: inputting an electric power scene image into a detection model, extracting an initial feature map through a backbone network, carrying out the multi-stage feature extraction of the initial feature map according to a convolution path, and carrying out the multi-stage feature extraction of the initial feature map; processing the multi-stage features based on a path aggregation network, and outputting a plurality of fusion feature maps with different feature levels from shallow to deep; and based on cross-scale window attention, guiding a shallow fusion feature map to carry out semantic information modeling by using a deep fusion feature map with high semantics in every two adjacent fusion feature maps, and after a plurality of output feature maps are obtained, respectively processing and outputting prediction results by using a multi-branch detection head. According to the method, shallow feature activation prediction and cross-scale window attention guidance are fused, and the detection robustness and positioning precision of a tiny fault target in an unmanned aerial vehicle inspection image can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of fault detection, and in particular to a method for detecting small defect targets in power scenarios based on Gaussian mask supervision and cross-layer attention guidance. Background Art

[0002] In the daily operation of power systems, fault detection is a crucial technical step in maintaining grid stability. However, for high-voltage transmission lines, tiny defects often occur on the surface of overhead cables, making them difficult to detect with the human eye. Furthermore, in inclement weather, the image quality of cable footage significantly degrades, significantly reducing the accuracy and efficiency of drone inspections.

[0003] A recent study has proposed a cable fault detection model, YOLOv9-SED, based on an improved YOLOv9. First, the defogging network UnfogNet is added to the original YOLOv9 model, effectively enhancing image clarity in complex and harsh high-altitude environments. The SEAM attention mechanism and Shape-IoU loss function are also introduced to improve the model's feature extraction capabilities for small objects. Finally, the original Conv convolutional layer is replaced with DualConv convolution, enhancing model performance while reducing model complexity. However, YOLOv9-SED primarily relies on the feature extraction capabilities of the deep network. While shallow feature maps (such as the output of low-level convolutions) contain rich details (such as small object edges and textures), they do not use an auxiliary head to predict target activation maps. This results in limited detection accuracy for small defects in complex mountainous scenes. Furthermore, the model relies solely on conventional bounding box and classification loss supervision, without incorporating ground-truth feature map supervision. This results in insufficient sensitivity to target location, making it prone to missed detections, particularly in complex backgrounds or low-contrast scenes.

[0004] Research is also underway to improve the Neck portion of YOLOv8, introducing the C2f-SCConv module (combining cross-stage partial links and spatial channel reconstruction convolutions) and the BiFPN structure to enhance multi-scale feature fusion capabilities. Furthermore, a global attention mechanism (GAM) is embedded at the end of the backbone network to suppress noise, and the Wise-IoU loss function is used to optimize bounding box regression accuracy. However, this solution suffers from low utilization of shallow features: while the improved model integrates multi-scale features through BiFPN, shallow features (such as low-layer convolution outputs) are not specifically enhanced, resulting in insufficient utilization of high-resolution detail information, which affects the localization accuracy of small targets. Dynamic context modeling is also lacking: although the GAM mechanism enhances global feature weight distribution, it does not explicitly establish local spatial correlations between cross-layer features. For example, deep features cannot directly guide the local area attention adjustment of shallow features, which can lead to high false detection rates in complex backgrounds. Summary of the Invention

[0005] In order to solve the technical problems of low small target detection accuracy and serious complex background interference in the current drone inspection scenes of mountain power lines, and the proneness of missed detection and false detection in complex backgrounds in the existing technology, the present invention provides a power scene defect small target detection method based on Gaussian mask supervision and cross-layer attention guidance, which integrates shallow feature activation prediction and cross-scale window attention guidance. It can effectively improve the detection robustness and positioning accuracy of tiny fault targets in drone inspection images.

[0006] In order to achieve the above technical objectives, the technical solution of the present invention is:

[0007] A method for detecting small defect targets in power scenes based on Gaussian mask supervision and cross-layer attention guidance, including:

[0008] Step 1: Input the power scene image into the detection model. The backbone network of the detection model first extracts the initial feature map, then performs multi-stage feature extraction on the initial feature map according to the convolution path, and outputs the feature map obtained in each stage respectively, thereby obtaining multiple feature maps at different scales. The feature maps formed based on the order of the stages are shallow feature maps and deep feature maps from shallow to deep.

[0009] Step 2: Based on the path aggregation network, a top-down path is first used to upsample the deep feature maps in every two adjacent stages of the feature maps obtained in step 1 and then splice them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow. Then, a bottom-up path is used to downsample the shallow intermediate feature maps in every two adjacent intermediate feature maps and then splice them with the deep intermediate feature maps, finally outputting multiple fused feature maps with different feature levels from shallow to deep.

[0010] Step 3: Based on cross-scale window attention, the deep fusion feature map with high semantics in each two adjacent fusion feature maps is used to guide the shallow fusion feature map to perform semantic information modeling, that is, the key vector Key and the value vector Value are generated based on the deep fusion feature map, and the query vector Query is generated based on the shallow fusion feature map. Then, the Key, Value and Query are divided into windows, and then split into multiple heads through the projection matrix. Then, the relative position bias is introduced to calculate the multi-head attention of Query and Key. The obtained multi-head attention result is multiplied with the multi-head of Value, and then spliced ​​and mapped by the output projection matrix to obtain the output. The output is channel-gated and added to the shallow fusion feature map as the output feature map; finally, multiple output feature maps are obtained;

[0011] In step 4, the multiple output feature maps obtained in step 3 are processed separately using a multi-branch detection head and the prediction results are output.

[0012] Furthermore, in the method, step 1 includes:

[0013] The model uses CSPDarkNet as the backbone network, and uses the starting layer composed of convolutional modules in the backbone network to compress the size of the input power grid image and increase the dimension of the channel to obtain the initial feature map;

[0014] The initial feature map is then input into the convolution path for feature extraction. The convolution path includes multiple CSP network modules formed by stacking CSP structures in series, and each CSP network module is used as a single stage for performing feature extraction. The feature map extracted in each stage is retained, thereby obtaining multiple feature maps with different resolutions or scales. Among them, the higher resolution is the shallow feature map with more texture details, and the lower resolution is the deep feature map with more semantic information.

[0015] Furthermore, in the method, in step 2, the process of obtaining multiple intermediate feature maps having different feature levels from deep to shallow includes:

[0016] The deep feature map is upsampled to adjust the resolution so that it is consistent with the resolution of the shallow feature map extracted in the adjacent stage. The two are then spliced ​​according to the channel dimension, and the number of channels is adjusted through the CSP module formed by a single CSP structure to obtain the corresponding intermediate feature map. The expression is:

[0017] ;

[0018] in is the intermediate feature map of the i-th layer formed in the top-down fusion path, represents nearest neighbor upsampling, represents splicing along the channel dimension, Indicates the CSP module, The output of the backbone network Layer feature map, The output of the backbone network Layer feature map, The number of channels is adjusted by the CSP module, and Stay consistent;

[0019] In step 2, the process of outputting multiple fused feature maps with different feature levels from shallow to deep includes:

[0020] The shallow intermediate feature maps in two adjacent intermediate feature maps are downsampled using convolution with a stride of 2. After batch normalization and activation function activation, they are concatenated with the deep intermediate feature map according to the channel dimension, and the number of channels is adjusted using the CSP module to achieve fusion to obtain the fused feature map. The expression is:

[0021] ;

[0022] in represents the fusion feature map of the i-th layer, represents the activation function, Indicates Batch Norm, which is batch normalization. Represents a 3×3 convolution.

[0023] Furthermore, in the method, step 3 includes:

[0024] The following formula is used to generate the key vector Key and value vector Value based on the deep fusion feature map with high semantics in each two adjacent fusion feature maps:

[0025] ;

[0026] in Indicates Key, Indicates Value, represents a real matrix, express and The number of channels, express and of high, express and width, represents the deep fusion feature map, , It means that the channel dimension is divided into two parts with equal number of channels. Contains upsampling and 1×1 convolution projection, Upsampling for nearest neighbor interpolation;

[0027] The following formula is used to generate the query vector Query based on the shallow fusion feature map in each of the two adjacent fusion feature maps:

[0028] ;

[0029] in Indicates Query, Represents 1×1 convolution projection; represents the shallow fusion feature map, ;

[0030] Then 、 、 Divided into Non-overlapping windows:

[0031] ;

[0032] in is the window size, and Represents the deep fusion feature map height and width;

[0033] Then according to the query matrix corresponding to each window , key matrix Sum Matrix To calculate the multi-head attention, for the m-th head:

[0034] ;

[0035] in 、 and are the query matrix, key matrix and value matrix of the mth head after being processed by the projection matrix; 、 and They are the query projection matrix, key projection matrix and value projection matrix, which are all parameters learned in the model. , is the dimension of the attention head, , Represents a deep fusion feature map The number of channels, is the number of long positions, 、 and They represent the query matrix, key matrix, and value matrix of the mth head in the window respectively;

[0036] Then introduce the relative position bias obtained through learning , to calculate and Attention Matrix ,in Used to compensate for the position insensitivity of the attention mechanism, :

[0037] ;

[0038] in represents the softmax normalization function, The superscript T in stands for matrix transpose. Indicates the position offset of the mth head;

[0039] Then and the corresponding Multiply them together to get the settlement result of the mth head , then merge the settlement results of all heads and output them through projection After mapping, we get the fused attention enhanced features :

[0040] ;

[0041] ;

[0042] in is the output projection matrix, ; Finally, the attention enhancement features are fused through the gated residual connection Fusion feature map with the original shallow layer , and obtain the output feature map obtained by fusion of every two adjacent feature maps:

[0043] ;

[0044] in is the output feature map, represents the Sigmoid nonlinear activation function, is the channel attention gating, Represents element-wise multiplication.

[0045] Furthermore, in the method, step 4 includes:

[0046] Based on the fused and enhanced multi-scale features, a multi-branch detection head is used to perform the final prediction output of the model. For the output features of N scales, N detection heads are used to process and output the prediction results respectively. All detection heads adopt a decoupled structure, including two branches: the classification branch and the regression branch. Both branch structures are composed of a standard 3×3 convolution module and a 1×1 element-by-element convolution module, and output category scores and prediction boxes respectively.

[0047] Furthermore, in the method described, in step 1, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network in step 2; in step 2, the last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fused feature map for generating the Query in step 3; in step 3, the last fused feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head in step 4.

[0048] Furthermore, in the method, the detection model, during the training process, when executing step 2, further includes generating a predicted target activation mask based on the shallowest fusion feature map output by the path aggregation network, and generating a mask of a single target area following a Gaussian distribution based on the true value, supervising the predicted target activation mask, and calculating the loss of the two to enhance the activation of small target areas in the shallow fusion feature map:

[0049] The shallowest fusion feature map After two upsampling blocks, a reconstructed image is generated by a standard 1×1 convolution; each upsampling block includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling:

[0050] ;

[0051] in Activation mask for predicted targets, is an upsampling block that processes the feature information and performs 2x upsampling once. Represents 1×1 convolution;

[0052] Then a Gaussian supervision mask is generated based on the true value: the set of true bounding boxes for the input image , where N is the total number of target samples in the image, object bounding boxes It is represented by the coordinates of the upper left and lower right corners of the box in the image, that is, , express The coordinates of the upper left corner in the image, express The coordinates of the lower right corner in the image; first set the target filtering condition of the true value, and only retain the targets with an absolute pixel area less than 32×32 in the mask:

[0053] ;

[0054] in represents a small target subset, Indicates the The area covered by the target bounding box:

[0055] ;

[0056] in Indicates the The width of the area covered by the target bounding box, Indicates the The height of the area covered by the target bounding box;

[0057] Then based on the collection Construct anisotropic Gaussian kernel: For each valid target, first establish a two-dimensional Gaussian probability density function :

[0058] ;

[0059] Where exp is the exponential function, Indicates the coordinates of a valid target; For the The center coordinates of a target are expressed as:

[0060] ;

[0061] Two-dimensional Gaussian probability function The standard deviations in the x and y directions are calculated independently to achieve anisotropic Gaussian distribution in two dimensions, matching the aspect ratio characteristics of the target. The Gaussian modeling covariance matrix of a single valid target is for:

[0062] ;

[0063] is the standard deviation, which is proportional to the target size and is calculated as follows:

[0064] ;

[0065] in , is the scale factor hyperparameter;

[0066] Finally, perform mixed Gaussian superposition and normalization to generate a mixed Gaussian response map :

[0067] ;

[0068] Then the mixed Gaussian response map is normalized by pixel-by-pixel numerical truncation to calculate the final response mask. :

[0069] ;

[0070] Finally got ,in and The length and width of the input image are used for the detection model, and the response mask is used as a Gaussian supervision mask to supervise the mask prediction based on the shallow feature map;

[0071] Then, based on the predicted target activation mask and Gaussian supervision mask, calculate the L1 loss function of the two That is, the mask prediction loss, and as part of the total loss, participates in gradient backpropagation and parameter update:

[0072] ;

[0073] in and All represent numbers, i.e. and indivual.

[0074] Furthermore, the method described above, in the model training, the total loss It includes three parts: classification loss, box regression loss and mask prediction loss, namely:

[0075] ;

[0076] in Prediction loss for masks; For classification loss, cross entropy loss is used; For the bounding box regression loss, a combination of DFL loss and CIOU loss is used; is the weight hyperparameter of the loss function;

[0077] Calculating the total loss of the prediction results Finally, the Adam optimizer is used to iteratively optimize the detection model and update the model parameters to reduce the total prediction loss of the model.

[0078] The present invention also provides an electronic device, comprising:

[0079] one or more processors;

[0080] a storage device for storing one or more programs;

[0081] The one or more processors execute the one or more programs to implement the aforementioned method.

[0082] The present invention also provides a computer-readable medium having a computer program stored thereon, wherein the computer program implements the aforementioned method when executed by a processor.

[0083] The technical effect of the present invention is that it conducts targeted research on the two major technical problems of low small target detection accuracy and severe complex background interference in mountainous power line drone inspection scenarios. In order to solve the problems of weak small target feature representation in shallow feature maps and insufficient interaction between deep semantic features and shallow detail features, the present invention proposes a cross-level window attention mechanism. This mechanism divides the deep feature map into non-overlapping local windows, generates a spatial weight matrix through multi-head attention calculation, and dynamically modulates the response intensity of the local area corresponding to the shallow feature map. Specifically, the shallow feature window serves as the query-related semantics, and the high-level feature area serves as the key and value to provide high-level semantic context information. After Softmax normalization, the attention weight is generated to enhance the feature response of the target area and suppress background noise. Compared with global attention, window division significantly reduces the computational complexity, and the local modeling characteristics are more suitable for the sparse distribution characteristics of targets in mountainous scenes. This module can significantly improve the detection accuracy of small targets in complex backgrounds and significantly reduce the false detection rate.

[0084] At the same time, the present invention addresses the problem of missed detection of small targets due to traditional methods ignoring the high-resolution details of shallow features. During the training process, the present invention adds an auxiliary prediction head to the shallow feature branch after the feature fusion layer of the model, converts it into a target activation mask, and supervises it through a Gaussian mask generated based on the true value. This design forces the model to learn the spatial distribution characteristics of small targets through the L1 loss function, rather than relying solely on bounding box regression. Compared with conventional supervision mechanisms, the Gaussian mask covers the center area of ​​the target and appropriately includes the context of the small target, effectively alleviating the positioning sensitivity problem caused by slight offset of the annotation box, thereby improving the robustness of small target detection, and to a certain extent, it can avoid the problem of insufficient optimization of shallow features due to gradient disappearance.

[0085] The present invention will be further described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a schematic diagram of the overall framework of the present invention;

[0087] Figure 2 Schematic diagram of the Gaussian supervision mechanism based on true value and shallow feature map of the present invention;

[0088] Figure 3 Schematic diagram of the cross-layer window multi-head attention mechanism based on channel gating in the present invention;

[0089] Figure 4 It is a schematic diagram of the process of the present invention.

[0090] Figure 5This figure is a schematic diagram of the results of small target detection in the same power scenario based on DINO, YOLOv8 and the present invention. DETAILED DESCRIPTION

[0091] See also Figure 1 The present embodiment discloses a method for detecting small targets of defects in power scenarios based on Gaussian mask supervision and cross-layer attention guidance. When training the detection model, a data set collected by a drone and finely labeled by professionals is input into the detection model.

[0092] The detection model first extracts the original features through the backbone network, and the backbone network outputs feature maps of 4 scales. Specifically, this embodiment uses CSPDarkNet as the backbone network to extract features from the input power grid image. The backbone network first compresses the size of the input image and increases the dimension of the channel through the starting layer composed of convolution modules to obtain the initial feature map. In this embodiment, the height of the initial feature map is 160, the width is 160, and the number of channels is 128, that is, 160×160×128. The backbone network then performs feature extraction through multiple CSP network modules formed by CSP structure stacking in series, and retains the feature map obtained by each CSP network module, that is, each feature extraction stage, and finally outputs 4 feature maps of different resolutions, which are respectively 、 、 and Compared with the deep feature maps in subsequent stages, the shallow feature maps contain richer texture detail information, but the semantic information is not sufficient and there is more noise; while the deep feature maps are fully feature extracted and contain rich semantic information, but the resolution is low and the useful high-frequency detail information in the image is ignored.

[0093] Next, the detection model of this embodiment fully fuses the enhanced multi-scale feature information based on the path aggregation network, thereby propagating the strong semantic information of the deep features to the shallow layer, and at the same time propagating the high-resolution detail features of the shallow layer to the deep layer. In order to achieve interaction between feature maps of different scales, this embodiment fuses feature maps of different scales based on the path aggregation network to enrich the feature representation of each level. The processing methods mainly include top-down and bottom-up paths. The path aggregation network PANet first adopts a top-down path to upsample the deep feature maps in the feature maps obtained in each two adjacent stages output by the backbone network and then splice them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow; then, a bottom-up path is adopted to downsample the shallow intermediate feature maps in each two adjacent intermediate feature maps and then splice them with the deep intermediate feature maps, finally outputting multiple fused feature maps with different feature levels from shallow to deep.

[0094] Specifically, the path aggregation network first processes the feature maps of the four stages output by the backbone network based on a top-down path. The deep feature maps in each of the two adjacent stages are first upsampled to adjust the resolution so that the resolution is consistent with the shallow feature maps of the adjacent stages. Figure 1 The two are then spliced ​​according to the channel dimension, and the number of channels is adjusted through the CSP module formed by a single CSP structure. The expression is:

[0095] ;

[0096] in is the intermediate feature map of the i-th layer formed in the top-down fusion path, represents nearest neighbor upsampling, represents splicing along the channel dimension, Indicates the CSP module, The output of the backbone network Layer feature map, The output of the backbone network Layer feature map, The number of channels is adjusted by the CSP module, and Stay consistent.

[0097] In the bottom-up path of the path aggregation network, the shallow intermediate feature maps of two adjacent intermediate feature maps are first downsampled using convolution with a stride of 2. Then, after batch normalization and activation function activation, they are concatenated with the deep intermediate feature map according to the channel dimension, and the CSP module is used to adjust the number of channels to achieve fusion and obtain the fused feature map. The specific expression is:

[0098] ;

[0099] in Indicates the Layer fusion feature map, represents the activation function, Indicates Batch Norm, which is batch normalization. Represents a 3×3 convolution.

[0100] See also Figure 2 In the subsequent training process, the target activation mask is predicted based on the shallowest feature map, and a mask of a single target region following a Gaussian distribution is generated based on the true value. The predicted mask is supervised, and the loss of the two is calculated to enhance the activation of small target regions in the shallow feature map. The training process here is no longer executed after the training is completed.

[0101] Since the information of small targets is mainly in the shallow high-resolution fusion feature map, the prediction effect of small targets depends on the quality of the shallow fusion feature map. Therefore, in order to ensure that the small target area in the shallow fusion feature map is fully activated and the background area maintains low response, this embodiment designs a shallow feature target reconstruction module based on the shallowest fusion feature map output by the path aggregation network. The small target information is reconstructed, and a small target Gaussian mask is established based on the true value and combined with the anisotropic Gaussian kernel to supervise the predicted mask and guide the model to focus on the central features of the target.

[0102] Specifically, the target mask is first predicted based on the shallow feature map. Input to the detection model Compared to the original image, the image has been downsampled by a factor of 4. Therefore, during feature reconstruction, the original feature information distribution in the feature map should be maintained while restoring the resolution. Specifically, this embodiment designs a reconstruction module that passes the feature map through two upsampling blocks and then generates a reconstructed image through a standard 1×1 convolution. Each upsampling block (layer) includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling:

[0103] ;

[0104] in Activation mask for predicted targets, is an upsampling block that processes the feature information and performs 2x upsampling once. Represents a 1×1 convolution.

[0105] Then a Gaussian supervision mask is generated based on the true value: the set of true bounding boxes for the input image , where N is the total number of target samples in the image, and each target bounding box The box is represented by the coordinates of its upper left and lower right corners in the image: , Indicates the coordinates of the upper left corner of the box in the image, Indicates the coordinates of the lower right corner of the box in the image. First, set the target screening conditions. Since the main function of the shallow feature map is to rely on its high resolution to identify very small targets, such as small targets (pixel area < 32×32) and tiny targets (pixel area < 16×16), only targets with an absolute pixel area less than 32×32 are retained in the mask:

[0106]

[0107] in represents a small target subset, Represents the area covered by the i-th target bounding box:

[0108] ;

[0109] in Indicates the The width of the area covered by the target bounding box, Indicates the The height of the area covered by the object bounding box.

[0110] Then based on the collection Construct anisotropic Gaussian kernel: For each valid target, establish a two-dimensional Gaussian probability density function :

[0111] ;

[0112] Where exp is the exponential function, Indicates the coordinates of a valid target; is the center coordinate of the i-th valid target:

[0113] ;

[0114] Two-dimensional Gaussian probability function The standard deviations in the x and y directions are calculated independently to achieve anisotropic Gaussian distribution in two dimensions, matching the aspect ratio characteristics of the target. The Gaussian modeling covariance matrix of a single valid target is for:

[0115] ;

[0116] in is the standard deviation, proportional to the target size:

[0117]

[0118] , is the proportional coefficient hyperparameter. In this embodiment, Set to 0.5.

[0119] Finally, perform mixed Gaussian superposition and normalization to generate a mixed Gaussian response map :

[0120] ;

[0121] To avoid numerical overflow and unstable training, the Gaussian response map is further normalized. If the standard normalization operation is used directly, the value range of the isolated small target area may be compressed to close to 0. Therefore, this embodiment uses pixel-by-pixel numerical truncation instead of the displayed normalization operation to calculate the final response map, that is, the response mask. :

[0122] ;

[0123] Finally got ,in and The length and width of the input image to the detection model. In this embodiment, , This response mask Used to supervise the prediction of masks based on shallow feature maps. This normalization is not strictly probabilistic normalization, but by limiting the value range to the range [0, 1], the mask can directly provide a stable supervision signal, ensuring that isolated target areas also have high signal strength.

[0124] After obtaining the predicted mask and Gaussian supervision mask, calculate the L1 loss function of the two That is, the mask prediction loss, which will be used as part of the total loss to participate in gradient backpropagation and parameter update:

[0125] ;

[0126] in and All represent numbers, i.e. and indivual.

[0127] The above execution predicts the target activation mask and calculates the L1 loss function The steps are all executed during training and will not be executed again after training is completed.

[0128] See also Figure 3Next, the detection model further processes the fused feature map. Based on the cross-scale window attention module, the deep high-semantic feature map is used to guide the shallow feature map for semantic information modeling, and channel attention gating is used to strengthen important features. That is, based on the cross-scale window attention, the deep fused feature map with high semantics in each two adjacent fused feature maps is used to guide the shallow fused feature map for semantic information modeling. In other words, the key vector Key and value vector Value are generated based on the deep fused feature map, and the query vector Query is generated based on the shallow fused feature map. The Key, Value, and Query are then divided into windows and split into multiple heads using the projection matrix. The relative position bias is then introduced to calculate the multi-head attention of the Query and Key. The obtained multi-head attention result is multiplied with the multi-head of the Value, and then the concatenation is mapped through the output projection matrix to obtain the output. The output is processed by channel gating and added to the shallow fused feature map as the output feature map; finally, multiple output feature maps are obtained.

[0129] Specifically, As the shallow fusion feature map in two adjacent fusion feature maps, is a deep feature map. Then generate Query based on the shallow feature map:

[0130] ;

[0131] in Indicates Query, Represents 1×1 convolution projection; represents the shallow fusion feature map, .

[0132] Generate K and V based on deep feature maps:

[0133] ;

[0134] in Indicates Key, Indicates Value, represents a real matrix, Indicates the number of channels, Indicates high, Indicates width, represents the deep fusion feature map, , represents the batch size, It means that the channel dimension is divided into two parts with equal number of channels. Contains upsampling and 1×1 convolution projection, Upsampling for nearest neighbor interpolation.

[0135] Then divide the above Q, K, and V into Non-overlapping windows:

[0136] ;

[0137] in is the window size, and Represents the deep fusion feature map The height and width of each window. , key matrix Sum Matrix , multi-head attention is calculated based on them, then for the m-th head:

[0138] ;

[0139] in 、 and are the query matrix, key matrix and value matrix of the mth head after being processed by the projection matrix; 、 and They are the query projection matrix, key projection matrix and value projection matrix, which are all parameters learned in the model. , is the dimension of the attention head, , Represents a deep fusion feature map The number of channels, is the number of long positions, 、 and They represent the query matrix, key matrix, and value matrix of the mth head in the window respectively.

[0140] Then introduce the relative position bias obtained through learning , to calculate and Attention Matrix ,in Used to compensate for the position insensitivity of the attention mechanism, :

[0141] ;

[0142] in represents the softmax normalization function, The superscript T in stands for matrix transpose. Indicates the position offset of the mth head;

[0143] Then multiply the attention by the V of the corresponding attention head, merge the multi-head results, and get the final output after output projection mapping:

[0144] ;

[0145] ;

[0146] in is the output projection matrix, ; Finally, the attention enhancement features are fused through the gated residual connection Fusion feature map with the original shallow layer , and obtain the output feature map obtained by fusion of every two adjacent feature maps:

[0147] ;

[0148] in is the output feature map, represents the Sigmoid nonlinear activation function, is the channel attention gating, Represents element-wise multiplication.

[0149] Based on the fused and enhanced multi-scale features, a multi-branch detection head is used to generate the model's final prediction output. For the N scale output features (N is 4 in this example) described above, N detection heads are used to process and output prediction results. All detection heads use a decoupled architecture, consisting of two main branches: a classification branch and a regression branch. Both branches have similar structures, consisting of a standard 3×3 convolutional module and a 1×1 element-by-element convolutional module.

[0150] It should be noted that in this embodiment, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network. The last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fused feature map for generating the query. The last fused feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head.

[0151] During the model training phase, a comprehensive weighted loss function is designed to calculate the error between the model output and the true value, and the mask prediction loss in step 4 is added to the loss. Finally, the model parameters are updated based on the back-propagation algorithm.

[0152] As shown in the following formula, the total loss consists of three parts, namely classification loss, box regression loss, and mask prediction loss:

[0153] ;

[0154] That is the mask prediction loss; For classification loss, cross entropy loss is used; is the bounding box regression loss, which combines DFL loss and CIOU loss. is the weight hyperparameter of the loss function.

[0155] After calculating the loss of the prediction results, the Adam optimizer is used to iteratively optimize the model, update the model parameters, reduce the total prediction loss of the model, and improve the prediction accuracy.

[0156] The present invention takes into account that most defective targets in power grid scene images are small-sized targets, and the information of small targets is mainly in the shallow high-resolution feature map. In order to shield the interference of background noise as much as possible while retaining the effective information of small target areas, the present invention introduces a lightweight auxiliary head after the feature fusion layer of the model. The auxiliary head takes the shallowest feature map as input and predicts and outputs a target area activation map. , use this image to represent the activation degree of the small target area in the shallow feature map, and generate a class-independent true value heat map based on the true value label , as the supervisory signal of the auxiliary head, helps the model learn the ability to retain foreground features and suppress background irrelevant information.

[0157] The present invention also takes into account some very small defect targets in the power grid scenario. Due to factors such as their extremely small proportion and lack of discriminative features, the context clues of their surrounding environment are very important. To this end, the present invention does not use a simple 0-1 mask, but instead combines an anisotropic Gaussian kernel to establish a small target distribution mask, supervises the target mask predicted based on the shallow feature map, and guides the model to focus on the central features of the small target. At the same time, thanks to the properties of the Gaussian distribution, the key contextual area around the target is also included in the supervision signal to a certain extent, which can ensure a supervision signal of sufficient strength. The loss between the true Gaussian mask and the predicted activation mask will be used as part of the final loss of the model and participate in the process of backpropagation and model parameter iteration. In addition, this detection head module can be discarded in the inference stage, and this mechanism will not increase the number of parameters and computational complexity during model inference.

[0158] Since the activation of small defective targets in the shallow feature map is not high, while the small target area in the deep feature map still maintains a high activation, but the boundary is relatively fuzzy. Taking into account this contradiction between deep and shallow features, the present invention designs a guidance module based on cross-scale window attention, that is, generating queries based on the shallow layer and generating keys and values ​​based on the deep layer. Different from the self-attention or self-based feature enhancement of existing methods, this embodiment aims to use the higher semantic information in the deep feature map to guide the shallow feature map for context modeling, thereby enhancing the feature representation and feature strength of small targets in the shallow feature map, thereby effectively improving the model's detection effect on small targets. After the multi-head attention operation, the channel attention gating mechanism is used to strengthen important channel features and suppress irrelevant channel features. In addition, in order to ensure that the module does not introduce a large amount of additional computation, this embodiment uses a windowed attention mechanism to guide the shallow layer.

[0159] After the detection model is trained, corresponding power grid scene photos can be collected and input into the detection model for processing, thereby outputting defect detection results based on the detection model.

[0160] At the same time, a typical power scene defect dataset was used to perform defect detection based on the existing CascadeRCNN, DetectoRS, VFNet, DINO, YOLOv8 and the method provided by the present invention. The detection results are shown in Table 1 below, where mAP represents the overall average precision. 50 Indicates that when the IoU between the predicted box and the real box ≥ 0.5, it is considered a looser standard for correct detection, mAP 75 Indicates that when the IoU between the predicted box and the true box is ≥ 0.75, it is considered a stricter and looser standard for correct detection. mAP t 、mAP vt 、mAP s and mAP m They represent the accuracy of tiny targets, extremely tiny targets, small targets and medium-sized targets respectively.

[0161] .

[0162] As shown in Table 1, the experimental results on a typical power scene defect dataset show that the detection accuracy of the proposed method is significantly higher than that of existing mature algorithms, especially in terms of mAP. vt (extremely small objects), mAP t (small object), mAP s The (small target) index is significantly higher than other algorithms. This result shows the feasibility and effectiveness of the present invention in the task of small target defect detection in power scenarios.

[0163] See also Figure 5 , Figure 5 The first column is the true value, that is, the real annotation. The annotation box is the actual defect, which includes the cable joint insulation cover falling off, the circuit breaker insulation cover missing, the pin missing and the tower top damaged. Figure 5 The second and third columns show the results of existing state-of-the-art object detection algorithms, including DINO and YOLOv8. The fourth column shows the results of the detection method proposed in this paper. It can be seen that DINO and YOLOv8 both experience varying degrees of missed detections and recognition errors, while the proposed method can more accurately locate small defects in power scenarios.

[0164] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0165] Electronic equipment includes:

[0166] one or more processors;

[0167] a storage device for storing one or more programs,

[0168] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0169] In practice, users can use electronic devices acting as terminal devices to interact with servers, also serving as electronic devices, over a network to receive or send messages. Terminal devices are generally various electronic devices equipped with a display and operated through a human-computer interface, including but not limited to smartphones, tablets, laptops, and desktop computers. Various specific application software can be installed on terminal devices as needed, including but not limited to web browsers, instant messaging software, social networking platforms, and shopping apps.

[0170] The server is a network service end used to provide various services. The method for detecting small targets with defects in power scenarios provided in this embodiment is generally executed by the server. In actual application, under the necessary conditions, the terminal device can also directly execute the method for detecting small targets with defects in power scenarios.

[0171] Similarly, the computer-readable medium of the present invention stores a computer program thereon, and when the computer program is executed by a processor, a method for detecting small targets with defects in power scenarios according to an embodiment of the present invention is implemented.

Claims

1. A method for detecting small defect targets in power scenes based on Gaussian mask supervision and cross-layer attention guidance, characterized in that: include: Step 1: Input the power scene image into the detection model. The backbone network of the detection model first extracts the initial feature map, then performs multi-stage feature extraction on the initial feature map according to the convolution path, and outputs the feature map obtained in each stage respectively, thereby obtaining multiple feature maps at different scales. The feature maps formed based on the order of the stages are shallow feature maps and deep feature maps from shallow to deep. Step 2: Based on the path aggregation network, a top-down path is first used to upsample the deep feature maps in every two adjacent stages of the feature maps obtained in step 1 and then splice them with the shallow feature maps to obtain multiple intermediate feature maps with different feature levels from deep to shallow. Then, a bottom-up path is used to downsample the shallow intermediate feature maps in every two adjacent intermediate feature maps and then splice them with the deep intermediate feature maps, finally outputting multiple fused feature maps with different feature levels from shallow to deep. Step 3: Based on cross-scale window attention, the deep fusion feature map with high semantics in each two adjacent fusion feature maps is used to guide the shallow fusion feature map to perform semantic information modeling, that is, the key vector Key and the value vector Value are generated based on the deep fusion feature map, and the query vector Query is generated based on the shallow fusion feature map. Then, the Key, Value and Query are divided into windows, and then split into multiple heads through the projection matrix. Then, the relative position bias is introduced to calculate the multi-head attention of Query and Key. The obtained multi-head attention result is multiplied with the multi-head of Value, and then spliced ​​and mapped by the output projection matrix to obtain the output. The output is channel-gated and added to the shallow fusion feature map as the output feature map; finally, multiple output feature maps are obtained; In step 4, the multiple output feature maps obtained in step 3 are processed separately using a multi-branch detection head and the prediction results are output.

2. The method according to claim 1, characterized in that The step 1 comprises: The model uses CSPDarkNet as the backbone network, and uses the starting layer composed of convolutional modules in the backbone network to compress the size of the input power grid image and increase the dimension of the channel to obtain the initial feature map; The initial feature map is then input into the convolution path for feature extraction. The convolution path includes multiple CSP network modules formed by stacking CSP structures in series, and each CSP network module is used as a single stage for performing feature extraction. The feature map extracted in each stage is retained, thereby obtaining multiple feature maps with different resolutions or scales. Among them, the higher resolution is the shallow feature map with more texture details, and the lower resolution is the deep feature map with more semantic information.

3. The method according to claim 1, characterized in that In step 2, the process of obtaining multiple intermediate feature maps with different feature levels from deep to shallow includes: The deep feature map is upsampled to adjust the resolution so that it is consistent with the resolution of the shallow feature map extracted in the adjacent stage. The two are then spliced ​​according to the channel dimension, and the number of channels is adjusted through the CSP module formed by a single CSP structure to obtain the corresponding intermediate feature map. The expression is: ； in is the intermediate feature map of the i-th layer formed in the top-down fusion path, represents nearest neighbor upsampling, represents splicing along the channel dimension, Indicates the CSP module, represents the i-th layer feature map output by the backbone network, The output of the backbone network Layer feature map, The number of channels is adjusted by the CSP module, and Stay consistent; In step 2, the process of outputting multiple fused feature maps with different feature levels from shallow to deep includes: The shallow intermediate feature maps in two adjacent intermediate feature maps are downsampled using convolution with a stride of 2. After batch normalization and activation function activation, they are concatenated with the deep intermediate feature map according to the channel dimension, and the number of channels is adjusted using the CSP module to achieve fusion to obtain the fused feature map. The expression is: ； in Indicates the Layer fusion feature map, represents the activation function, Indicates Batch Norm, which is batch normalization. Represents a 3×3 convolution.

4. The method according to claim 1, wherein The step 3 comprises: The following formula is used to generate the key vector Key and value vector Value based on the deep fusion feature map with high semantics in each two adjacent fusion feature maps: ； in Indicates Key, Indicates Value, represents a real matrix, express and The number of channels, express and of high, express and width, represents the deep fusion feature map, , It means that the channel dimension is divided into two parts with equal number of channels. Contains upsampling and 1×1 convolution projection, Upsampling for nearest neighbor interpolation; The following formula is used to generate the query vector Query based on the shallow fusion feature map in each of the two adjacent fusion feature maps: ； in Indicates Query, Represents 1×1 convolution projection; represents the shallow fusion feature map, ; Then 、 、 Divided into Non-overlapping windows: ； in is the window size, and Represents the deep fusion feature map height and width; Then according to the query matrix corresponding to each window , key matrix Sum Matrix To calculate the multi-head attention, for the m-th head: ； in 、 and are the query matrix, key matrix and value matrix of the mth head after being processed by the projection matrix; 、 and They are the query projection matrix, key projection matrix and value projection matrix, which are all parameters learned in the model. , is the dimension of the attention head, , Represents a deep fusion feature map The number of channels, is the number of long positions, 、 and They represent the query matrix, key matrix, and value matrix of the mth head in the window respectively; Then introduce the relative position bias obtained through learning , to calculate and Attention Matrix ,in Used to compensate for the position insensitivity of the attention mechanism, : ； in represents the softmax normalization function, The superscript T in stands for matrix transpose. Indicates the position offset of the mth head; Then and the corresponding Multiply them together to get the settlement result of the mth head , then merge the settlement results of all heads and output them through projection After mapping, we get the fused attention enhanced features : ； ； in is the output projection matrix, ; Finally, the attention enhancement features are fused through the gated residual connection Fusion feature map with the original shallow layer , and obtain the output feature map obtained by fusion of every two adjacent feature maps: ； in is the output feature map, represents the Sigmoid nonlinear activation function, is the channel attention gating, Represents element-wise multiplication.

5. The method according to claim 1, wherein The step 4 comprises: Based on the fused and enhanced multi-scale features, a multi-branch detection head is used to perform the final prediction output of the model. For the output features of N scales, N detection heads are used to process and output the prediction results respectively. All detection heads adopt a decoupled structure, including two branches: the classification branch and the regression branch. Both branch structures are composed of a standard 3×3 convolution module and a 1×1 element-by-element convolution module, and output category scores and prediction boxes respectively.

6. The method according to any one of claims 3 to 5, characterized in that: In step 1, the feature map output by the last stage of the backbone network is directly used as the last deep intermediate feature map of the bottom-up path of the path aggregation network in step 2; in step 2, the last intermediate feature map generated by the top-down path of the path aggregation network is directly used as the first shallow fused feature map for generating the Query in step 3; in step 3, the last fused feature map generated by the bottom-up path of the path aggregation network is directly used as the output feature map processed by the detection head in step 4.

7. The method according to claim 1, characterized in that During the training process, the detection model further includes, when executing step 2, generating a predicted target activation mask based on the shallowest fusion feature map output by the path aggregation network, generating a mask of a single target area following a Gaussian distribution based on the true value, supervising the predicted target activation mask, and calculating the loss of the two to enhance the activation of small target areas in the shallow fusion feature map: The shallowest fusion feature map After two upsampling blocks, a reconstructed image is generated by a standard 1×1 convolution; each upsampling block includes four steps: standard 3×3 convolution, batch normalization, SiLU activation function, and bilinear interpolation upsampling: ； in Activation mask for predicted targets, is an upsampling block that processes the feature information and performs 2x upsampling once. Represents 1×1 convolution; Then a Gaussian supervision mask is generated based on the true value: the set of true bounding boxes for the input image , where N is the total number of target samples in the image, object bounding boxes It is represented by the coordinates of the upper left and lower right corners of the box in the image, that is, , express The coordinates of the upper left corner in the image, express The coordinates of the lower right corner in the image; first set the target filtering condition of the true value, and only retain the targets with an absolute pixel area less than 32×32 in the mask: ； in represents a small target subset, Indicates the The area covered by the target bounding box: ； in Indicates the The width of the area covered by the target bounding box, Indicates the The height of the area covered by the target bounding box; Then based on the collection Construct anisotropic Gaussian kernel: For each valid target, first establish a two-dimensional Gaussian probability density function : ； Where exp is the exponential function, Indicates the coordinates of a valid target; For the The center coordinates of a target are expressed as: ； Two-dimensional Gaussian probability function The standard deviations in the x and y directions are calculated independently to achieve anisotropic Gaussian distribution in two dimensions, matching the aspect ratio characteristics of the target. The Gaussian modeling covariance matrix of a single valid target is for: ； is the standard deviation, which is proportional to the target size and is calculated as follows: ； in , is the scale factor hyperparameter; Finally, perform mixed Gaussian superposition and normalization to generate a mixed Gaussian response map : ； Then the mixed Gaussian response map is normalized by pixel-by-pixel numerical truncation to calculate the final response mask. : ； Finally got ,in and The length and width of the input image are used for the detection model, and the response mask is used as a Gaussian supervision mask to supervise the mask prediction based on the shallow feature map; Then, based on the predicted target activation mask and Gaussian supervision mask, calculate the L1 loss function of the two That is, the mask prediction loss, and as part of the total loss, participates in gradient backpropagation and parameter update: ； in and All represent numbers, i.e. and indivual.

8. The method according to claim 7, characterized in that During model training, the total loss It includes three parts: classification loss, box regression loss and mask prediction loss, namely: ； in Prediction loss for masks; For classification loss, cross entropy loss is used; For the bounding box regression loss, a combination of DFL loss and CIOU loss is used; is the weight hyperparameter of the loss function; When calculating the total loss of the prediction results Finally, the Adam optimizer is used to iteratively optimize the detection model and update the model parameters to reduce the total prediction loss of the model.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs; It is characterized in that the one or more processors execute the one or more programs to implement the method according to any one of claims 1-8.

10. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images