A transformer-based feature fusion enhanced power grid hidden danger target identification method

Abstract

The application provides a feature fusion enhancement power grid hidden danger target identification method based on a Transformer, which is based on DINO-DETR, combines a global semantic network GSNet and a fusion enhancement module FRM to extract and integrate rich backbone features; multi-scale features are extracted from an image through ResNet and globally enhanced through the GSNet, and then the features of adjacent scales are further fused through the FRM module; the fused features are input into a Transformer encoder with position embedding; in the decoder stage of the encoder, a hybrid query selection strategy is introduced, the decoder query is divided into position query and content query, a deformable attention mechanism is used to refine the features and update the query layer by layer, so that the accurate position and classification of the power grid hidden danger target can be effectively identified, and the accuracy and efficiency of the power grid hidden danger target detection are improved; the application can effectively identify the accurate position and classification of the power grid hidden danger target, and improve the accuracy and efficiency of the power grid hidden danger target detection.

Description

Technical Field

[0001] This invention relates to deep learning, computer vision, and target recognition, and in particular to a method for enhancing the identification of potential power grid hazards based on feature fusion using Transformer. Background Technology

[0002] Before the advent of the Transformer architecture, object detection primarily relied on models based on convolutional neural networks (CNNs), such as the R-CNN series, YOLO, and SSD. These models typically include a feature extractor (such as VGG, ResNet, etc.) and one or more subsequent network structures for generating object class predictions and bounding box regressions. Although these models achieved good performance on object detection tasks, they still had some limitations, such as insufficient capture of long-range dependencies and limited feature representation capabilities.

[0003] The Transformer architecture was originally designed to solve sequence-to-sequence tasks, such as machine translation. Its core is the self-attention mechanism, which allows the model to consider all other elements in the sequence while processing each element. This global attention mechanism enables the Transformer to capture complex dependencies within the sequence, resulting in breakthrough performance in object detection tasks.

[0004] In real-world target detection scenarios, this invention comprehensively considers large-scale image features and global target characteristics to overcome problems such as changes in observation perspective and target scale. It employs a Global Semantic Network (GSNet) and a Fusion Enhancement Module (FRM) to obtain rich backbone features and uses Deformable Attention to combine the output features of the Transformer encoder. Summary of the Invention

[0005] This invention proposes a feature fusion-based Transformer-based method to enhance the identification of potential power grid hazards. This method can effectively identify the precise location and classification of potential power grid hazards, thereby improving the accuracy and efficiency of potential power grid hazard detection.

[0006] The present invention adopts the following technical solution.

[0007] A feature fusion enhancement method for identifying potential power grid hazards based on Transformer is proposed. This method is based on DINO-DETR, combining a global semantic network GSNet and a fusion enhancement module FRM to extract and integrate rich backbone features. Multi-scale features are extracted from images using ResNet and globally enhanced using GSNet. Then, the FRM module further fuses features from adjacent scales. The fused features are fed into a Transformer encoder with location embedding. In the encoder's decoder stage, a hybrid query selection strategy is introduced, dividing the decoder query into location queries and content queries. A deformable attention mechanism is used to refine features and update the query layer by layer, effectively identifying the precise location and classification of potential power grid hazards, thus improving the accuracy and efficiency of potential power grid hazard detection.

[0008] A method for enhancing power grid hazard target identification based on Transformer feature fusion includes the following steps;

[0009] Step S1: Data preprocessing and dataset construction. The acquired remote sensing images are preprocessed, including radiometric correction, atmospheric correction and geographic correction, to eliminate noise and errors in the images, generate standard remote sensing image products, and perform pixel-level manual annotation of power grid hazard targets in the remote sensing images, add labels, and establish a remote sensing image power grid hazard target dataset for later network training.

[0010] Step S2: Construct the overall network model for Transformer feature fusion enhancement, such as... Figure 1 As shown, DINO-DETR is used as the base model, and a global semantic network GSNet and a fusion enhancement module FRM are further adopted to obtain rich backbone features;

[0011] Step S3: Further construct the global semantic network GSNet from step S2, such as... Figure 2 As shown, GSNet uses large convolutional kernels or global convolutions to expand the effective feature perception range. To limit the increase in the number of parameters caused by direct k×k kernel convolutions, GSNet uses a combination of 1×k+k×1 and k×1+1×k convolutions. The symmetric and depth-separable combination of convolutions contains detailed contextual information while reducing the number of parameters and computational cost.

[0012] Step S4: Further construct the Fusion Enhancement Module (FRM) from Step S2, such as... Figure 2As shown, when there are semantic information gaps between feature maps of different scales, direct concatenation is not a reasonable cross-scale fusion method. FRM introduces a residual branch from the backbone to inject various spatial context information. Deep features are superimposed on the original features using the residual structure to achieve the fusion of global and local information. Then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation ability. The enhanced feature pyramid contains more higher-level and semantic information.

[0013] Step S5: Further construct the attention network of the encoder and decoder modules in step S2, using the DeformableAttention method, that is, add a sampling mechanism to the Attention module, ignore the size of the input image, and only focus on a few sampling points near the target point, so as to reduce the amount of computation and speed up the convergence.

[0014] The specific method of step S2 is as follows: For a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are input into GSNet to obtain the corresponding scale features after global enhancement. The backbone features of adjacent scales and the global enhancement features are further input into the FRM module to obtain the fused and enhanced multi-scale features. Then these features are input into the Transformer encoder with corresponding position embeddings. Before the decoder, the decoder query is regarded as a position query and a content query. A hybrid query selection strategy is adopted to initialize the anchors as the decoder's position query. Through the initialized anchors and learnable content queries, DeformableAttention is used to combine the features input to the encoder and update the query layer by layer. Finally, the refined grid hazard target location coordinates and the classification results predicted by the refined content features are output.

[0015] In step S3, GSNet, as a fully convolutional network, applies linear operations only in the combined convolution; the calculation formula is as follows;

[0016] M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1;

[0017] Formula 2: Y = M + R(M) + X;

[0018] R(M) = conv2D(σ(conv2D(M))) Formula 3;

[0019] In Formulas 1, 2, and 3, X is the input, which is the feature map extracted from the backbone feature pyramid; Y is the global semantic network feature output; and M is the intermediate result after feature extraction through two convolutional branches. R(·) is the residual branch, and σ is the ReLU activation function.

[0020] In step S4, the calculation formula for the Fusion Enhancement Module (FRM) is as follows:

[0021] Z = f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 Formula Four;

[0022] In Formula 4, [·] represents channel-level connections, X and Y are feature maps from the backbone feature pyramid and feature maps processed by GSNet, respectively, and f 1×1 f 3×3 This represents 1x1 and 3x3 convolutions.

[0023] In step S5, the calculation formula for the sampling mechanism is as follows:

[0024]

[0025] In Formula 5, x is the input feature map, and each pixel in the feature map is a C-channel vector z. q The index of each pixel, which is its two-dimensional position coordinate, is p. q M represents the number of heads in the multi-head attention mechanism, where each head only considers z. q K nearby points, Δp mqk The position offset representing the sampling is a two-dimensional coordinate, A. mqk W represents the weight of attention. m and W m ′ represents the weight parameters of the two fully connected layers, and the two A values ​​that need to be predicted. mqk and Δp mqk Both are calculated by z q It came from there.

[0026] This invention can effectively predict the precise location and classification of potential power grid hazards, thereby improving the accuracy and efficiency of power grid hazard detection. Attached Figure Description

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0028] Appendix Figure 1This is a schematic diagram of the Transformer feature fusion enhancement overall network model in this invention;

[0029] Appendix Figure 2 This is a schematic diagram of the global semantic network and fusion enhancement module in this invention. Detailed Implementation

[0030] As shown in the figure, a feature fusion enhancement method for identifying potential power grid hazards based on Transformer is presented. This method is based on DINO-DETR, combining the global semantic network GSNet and the fusion enhancement module FRM to extract and integrate rich backbone features. Multi-scale features are extracted from the image using ResNet and globally enhanced using GSNet. Then, the FRM module further fuses features from adjacent scales. The fused features are fed into a Transformer encoder with location embedding. In the encoder's decoder stage, a hybrid query selection strategy is introduced, dividing the decoder query into location queries and content queries. A deformable attention mechanism is used to refine features and update queries layer by layer, effectively identifying the precise location and classification of potential power grid hazards, thus improving the accuracy and efficiency of potential power grid hazard detection.

[0031] A method for enhancing power grid hazard target identification based on Transformer feature fusion includes the following steps;

[0032] Step S1: Data preprocessing and dataset construction. The acquired remote sensing images are preprocessed, including radiometric correction, atmospheric correction and geographic correction, to eliminate noise and errors in the images, generate standard remote sensing image products, and perform pixel-level manual annotation of power grid hazard targets in the remote sensing images, add labels, and establish a remote sensing image power grid hazard target dataset for later network training.

[0033] Step S2: Construct the overall network model for Transformer feature fusion enhancement, such as... Figure 1 As shown, DINO-DETR is used as the base model, and a global semantic network GSNet and a fusion enhancement module FRM are further adopted to obtain rich backbone features;

[0034] Step S3: Further construct the global semantic network GSNet from step S2, such as... Figure 2 As shown, GSNet uses large convolutional kernels or global convolutions to expand the effective feature perception range. To limit the increase in the number of parameters caused by direct k×k kernel convolutions, GSNet uses a combination of 1×k+k×1 and k×1+1×k convolutions. The symmetric and depth-separable combination of convolutions contains detailed contextual information while reducing the number of parameters and computational cost.

[0035] Step S4: Further construct the Fusion Enhancement Module (FRM) from Step S2, such as... Figure 2 As shown, when there are semantic information gaps between feature maps of different scales, direct concatenation is not a reasonable cross-scale fusion method. FRM introduces a residual branch from the backbone to inject various spatial context information. Deep features are superimposed on the original features using the residual structure to achieve the fusion of global and local information. Then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation ability. The enhanced feature pyramid contains more higher-level and semantic information.

[0036] Step S5: Further construct the attention network of the encoder and decoder modules in step S2, using the DeformableAttention method, that is, add a sampling mechanism to the Attention module, ignore the size of the input image, and only focus on a few sampling points near the target point, so as to reduce the amount of computation and speed up the convergence.

[0037] The specific method of step S2 is as follows: For a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are input into GSNet to obtain the corresponding scale features after global enhancement. The backbone features of adjacent scales and the global enhancement features are further input into the FRM module to obtain the fused and enhanced multi-scale features. Then these features are input into the Transformer encoder with corresponding position embeddings. Before the decoder, the decoder query is regarded as a position query and a content query. A hybrid query selection strategy is adopted to initialize the anchors as the position query of the decoder. Through the initialized anchors and learnable content queries, Deformable Attention is used to combine the features input to the encoder and update the query layer by layer. Finally, the refined coordinates of the target location of the power grid hazard and the classification result predicted by the refined content features are output.

[0038] In step S3, GSNet, as a fully convolutional network, applies linear operations only in the combined convolution; the calculation formula is as follows;

[0039] M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1;

[0040] Formula 2: Y = M + R(M) + X;

[0041] R(M) = conv2D(σ(conv2D(M))) Formula 3;

[0042] In Formulas 1, 2, and 3, X is the input, which is the feature map extracted from the backbone feature pyramid; Y is the global semantic network feature output; and M is the intermediate result after feature extraction through two convolutional branches. R(·) is the residual branch, and σ is the ReLU activation function.

[0043] In step S4, the calculation formula for the Fusion Enhancement Module (FRM) is as follows:

[0044] Z = f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 Formula Four;

[0045] In Formula 4, [·] represents channel-level connections, X and Y are feature maps from the backbone feature pyramid and feature maps processed by GSNet, respectively, and f 1×1 f 3×3 This represents 1x1 and 3x3 convolutions.

[0046] In step S5, the calculation formula for the sampling mechanism is as follows:

[0047]

[0048] In Formula 5, x is the input feature map, and each pixel in the feature map is a C-channel vector z. q The index of each pixel, which is its two-dimensional position coordinate, is p. q M represents the number of heads in the multi-head attention mechanism, where each head only considers z. q K nearby points, Δp mqk The position offset representing the sampling is a two-dimensional coordinate, A. mqk W represents the weight of attention. m and W m ′ represents the weight parameters of the two fully connected layers, and the two A values ​​that need to be predicted. mqk and Δp mqk Both are calculated by z q It came from there.

[0049] Example:

[0050] This example presents a Transformer-based feature fusion enhancement method for identifying power grid hazard targets. Using remote sensing imagery as the data source and DINO-DETR as the base model, it further employs a Global Semantic Network (GSNet) and a Fusion Enhancement Module (FRM) to obtain rich backbone features. Specifically, given an image, ResNet is used to extract multi-scale features. These multi-scale features are then input into GSNet to obtain globally enhanced features at the corresponding scales. The backbone features from adjacent scales and the globally enhanced features are then input into the FRM module to obtain fused and enhanced multi-scale features. These features are then input into a Transformer encoder with corresponding location embeddings. Before the decoder, the decoder query is treated as both a location query and a content query. A hybrid query selection strategy is employed, initializing anchors as the decoder's location query. Using the initialized anchors and learnable content queries, Deformable Attention is used to combine the encoder's input features and update the query layer by layer. Finally, the refined coordinates of the power grid hazard targets and the classification results predicted by the refined content features are output.

[0051] Global Semantic Network (GSNet): GSNet should use the largest possible convolutional kernels, even global convolutions, to significantly expand the effective feature perception range. However, direct k×k kernel convolutions will significantly increase the number of parameters. GSNet uses combined 1×k+k×1 and k×1+1×k convolutions. These symmetric and depth-separable combined convolutions contain detailed contextual information while reducing the number of parameters and computational cost. GSNet is a fully convolutional network, applying linear operations only in combined convolutions. The calculation formula is as follows: Where X is the input, which is the feature map extracted from the backbone feature pyramid; Y is the global semantic network feature output; M is the intermediate result after feature extraction through two convolutional branches; R(·) is the residual branch; and σ is the ReLU activation function.

[0052] M = conv(conv1D(X) T )+conv(conv1D(X)) T

[0053] Y = M + R(M) + X

[0054] R(M)=conv2D(σ(conv2D(M)))

[0055] Fusion Enhancement Module (FRM): Due to the semantic information gap between feature maps of different scales, direct concatenation is not a reasonable cross-scale fusion method. FRM introduces a residual branch from the backbone to inject various spatial contextual information. The residual structure superimposes deep features on the original features, realizing the fusion of global and local information. Then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance the feature representation capability. The enhanced feature pyramid contains more higher-level and semantic information. The calculation formula is as follows. Where [·] is the channel-level connection, X and Y are the feature maps from the backbone feature pyramid and the feature maps processed by GSNet, f 1×1 f 3×3 This represents 1x1 and 3x3 convolutions.

[0056] Z = f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 ])))))

[0057] Deformable Attention: Both the encoder and decoder modules employ Deformable Attention, which incorporates a sampling mechanism within the attention module. Regardless of the input image size, it only focuses on a few pixels near the target point, significantly reducing computational cost and accelerating convergence. The calculation formula is as follows. Where x is the input feature map, and each pixel in the feature map is a C-channel vector z. q The index of each pixel, which is its two-dimensional position coordinate, is p. q M represents the number of heads in the multi-head attention mechanism, where each head only considers z. q K nearby points, Δp mqk The position offset representing the sampling is a two-dimensional coordinate, A. mqk W represents the weight of attention. m and W m ′ represents the weight parameters of the two fully connected layers, and the two A values ​​that need to be predicted. mqk and Δp mqk Both are calculated by z q It came from there.

[0058]

Claims

1. A method for enhancing power grid hazard target identification based on Transformer feature fusion, characterized in that: Includes the following steps; Step S1: Data preprocessing and dataset construction. The acquired remote sensing images are preprocessed, including radiometric correction, atmospheric correction and geographic correction, to eliminate noise and errors in the images. Pixel-level manual annotation and labeling are performed on power grid hazard targets in the remote sensing images to establish a remote sensing image power grid hazard target dataset. Step S2: Construct a feature fusion enhancement network model for Transformer, using DINO-DETR as the base model, and further employing a global semantic network GSNet and a fusion enhancement module FRM to obtain rich backbone features; Step S3: Further construct the global semantic network GSNet from step S2. GSNet uses convolutional kernels or global convolutions to expand the effective feature perception range. To limit the increase in the number of parameters caused by direct k×k kernel convolutions, GSNet uses a combination of 1×k+k×1 and k×1+1×k convolutions. The symmetric and depth-separable combination of convolutions contains detailed contextual information while reducing the number of parameters and computational cost. Step S4: Further construct the fusion enhancement module FRM from step S2. When there are semantic information gaps between feature maps of different scales, direct concatenation is not a reasonable cross-scale fusion method. FRM introduces a residual branch from the backbone to inject various spatial context information. Deep features are superimposed on the original features using the residual structure to achieve the fusion of global and local information. Then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation ability. The enhanced feature pyramid contains more higher-level and semantic information. Step S5: Further construct the attention network of the encoder and decoder modules in step S2, using the DeformableAttention method, that is, add a sampling mechanism to the Attention module to ignore the size of the input image; The specific method of step S2 is as follows: For a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are input into GSNet to obtain the corresponding scale features after global enhancement. The backbone features of adjacent scales and the global enhancement features are further input into the FRM module to obtain the fused and enhanced multi-scale features. Then these features are input into the Transformer encoder with corresponding position embeddings. Before the decoder, the decoder query is regarded as a position query and a content query. A hybrid query selection strategy is adopted to initialize the anchors as the decoder's position query. Through the initialized anchors and learnable content queries, DeformableAttention is used to combine the features input to the encoder and update the query layer by layer. Finally, the refined coordinates of the target location of the power grid hazard and the classification result predicted by the refined content features are output.

2. The method for enhancing power grid hazard target identification based on Transformer feature fusion according to claim 1, characterized in that: In step S3, GSNet, as a fully convolutional network, applies linear operations only in combined convolutions; The calculation formula is as follows; M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1; Formula 2: Y = M + R(M) + X; R(M) = conv2D(σ(conv2D(M))) Formula 3; In Formulas 1, 2, and 3, X is the input, which is the feature map extracted from the backbone feature pyramid; Y is the global semantic network feature output; M is the intermediate result after feature extraction through two convolutional branches; R(·) is the residual branch; and σ is the ReLU activation function.

3. The method for enhancing power grid hazard target identification based on Transformer feature fusion according to claim 1, characterized in that: In step S4, the calculation formula for the Fusion Enhancement Module (FRM) is as follows: Z = f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 Formula Four; In Formula 4, [·] represents channel-level connections, X and Y are feature maps from the backbone feature pyramid and feature maps processed by GSNet, respectively, and f 1×1 f 3×3 This represents 1x1 and 3x3 convolutions.

4. The method for enhancing power grid hazard target identification based on Transformer feature fusion according to claim 1, characterized in that: In step S5, the calculation formula for the sampling mechanism is as follows: In Formula 5, x is the input feature map, and each pixel in the feature map is a C-channel vector z. q The index of each pixel, which is its two-dimensional position coordinate, is p. q M represents the number of heads in the multi-head attention mechanism, where each head only considers z. q K nearby points, Δp mqk The position offset representing the sampling is a two-dimensional coordinate, A. mqk W represents the weight of attention. m and W m ′ represents the weight parameters of the two fully connected layers, and the two A values ​​that need to be predicted. mqk and Δp mqk Both are calculated by z q It came from there.

Images