Transform-based feature fusion enhanced power grid hidden danger target identification method
By adopting a Transformer-based feature fusion enhancement method in grid hidden danger target detection, combining global semantic network and fusion enhancement module, and using the DeformableAttention mechanism, the problem of long-distance dependence and insufficient feature representation capabilities in grid hidden danger target detection is solved. , achieving more efficient identification of hidden dangers in power grids.
Patent Information
- Application Number
- CN202411041181.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-31
AI Technical Summary
The existing object detection model based on convolutional neural networks has shortcomings in long-distance dependence and feature representation capabilities, and it is difficult to effectively identify the precise location and classification of hidden danger targets in the power grid, especially in the observation of changes in perspective and target scales.
Using a Transformer-based feature fusion enhancement method, combining global semantic network (GSNet) and fusion enhancement module (FRM), multi-scale features are extracted through ResNet, and a hybrid query selection strategy is performed in the decoder stage using the DeformableAttention mechanism, and the features are updated layer by layer to identify The precise location and classification of hidden danger targets in power grids.
It significantly improves the accuracy and efficiency of grid hidden danger target detection, can effectively capture complex dependencies and global features, and improves feature representation ability and detection accuracy.
Smart Images

Figure CN119027806A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to deep learning, computer vision, and target recognition, and in particular to a Transformer-based feature fusion enhanced power grid hidden danger target recognition method. Background Art
[0002] Before the emergence of the Transformer architecture, the field of object detection mainly relied on models based on convolutional neural networks (CNNs), such as the R-CNN series, YOLO, and SSD. These models usually contain a feature extractor (such as VGG, ResNet, etc.) and one or more subsequent network structures to generate category predictions and bounding box regressions for objects. Although these models have achieved good performance in object detection tasks, they still have some limitations, such as insufficient capture of long-distance dependencies and limited feature representation capabilities.
[0003] The Transformer architecture was originally designed to solve sequence-to-sequence tasks such as machine translation. At its core is the self-attention mechanism, which allows the model to consider all other elements in the sequence while processing each element of the sequence. This global attention mechanism enables the Transformer to capture complex dependencies within the sequence, thus achieving breakthrough performance in object detection tasks.
[0004] In realistic target detection scenarios, the wide range of image features and global target characteristics are comprehensively considered to overcome problems such as changes in observation perspective and changes in target scale. The patent of this invention adopts a global semantic network (GSNet) and a fusion enhancement module (FRM) to obtain rich backbone features, and uses deformable attention (DeformableAttention) to combine the output features of the Transformer encoder. Summary of the invention
[0005] The present invention proposes a Transformer-based feature fusion enhanced power grid hidden danger target identification method, which can effectively identify the precise location and classification of power grid hidden danger targets and improve the accuracy and efficiency of power grid hidden danger target detection.
[0006] The present invention adopts the following technical solutions.
[0007] A Transformer-based feature fusion enhanced power grid hidden danger target recognition method, the recognition method is based on DINO-DETR, combined with a global semantic network GSNet and a fusion enhancement module FRM to extract and integrate rich backbone features; multi-scale features are extracted from images through ResNet, and globally enhanced through GSNet, and then the features of adjacent scales are further fused using the FRM module; the fused features are sent to a Transformer encoder with position embedding; in the decoder stage of the encoder, a hybrid query selection strategy is introduced to divide the decoder query into a position query and a content query, and a deformable attention mechanism is used to refine the features and update the query layer by layer, so as to effectively identify the precise location and classification of power grid hidden danger targets, and improve the accuracy and efficiency of power grid hidden danger target detection.
[0008] The Transformer-based feature fusion enhanced power grid hidden danger target identification method includes the following steps:
[0009] Step S1: Data preprocessing and data set construction. The collected remote sensing images are preprocessed, including radiation correction, atmospheric correction, and geographic correction, to eliminate noise and errors in the images, generate standard remote sensing image products, and manually annotate the power grid hidden danger targets in the remote sensing images at the pixel level, add labels, and establish a remote sensing image power grid hidden danger target dataset for later network training;
[0010] Step S2: Construct the Transformer feature fusion enhanced overall network model, such as Figure 1 As shown in the figure, DINO-DETR is used as the basic model, and a global semantic network GSNet and a fusion enhancement module FRM are further adopted to obtain rich backbone features;
[0011] Step S3: Further construct the global semantic network GSNet in step S2, such as Figure 2 As shown in the figure, GSNet uses large convolution kernels or global convolutions to expand the effective feature perception range. In order to limit the increase in the number of parameters caused by direct k×k kernel convolution, GSNet uses 1×k+k×1 and k×1+1×k combined convolutions to contain detailed contextual information with symmetric and depth-separable combined convolutions, while reducing the number of parameters and computational costs;
[0012] Step S4: further construct the fusion enhancement module FRM in step S2, such as Figure 2As shown in the figure, when there is a semantic information gap between feature maps of different scales and direct cascading is not a reasonable cross-scale fusion method, FRM introduces a residual branch from the trunk to inject various spatial context information; the deep features are superimposed on the original features with the residual structure to achieve the fusion of global information and local information; then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation capabilities; the enhanced feature pyramid contains more higher-level and semantic information;
[0013] Step S5: Further construct the attention network of the encoder and decoder modules in step S2, in which the DeformableAttention method is adopted, that is, a sampling mechanism is added to the Attention module, the size of the input image is ignored, and only a few sampling points near the target point are focused on to reduce the amount of calculation and speed up the convergence.
[0014] The specific method of step S2 is as follows: for a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are respectively input into GSNet to obtain the corresponding scale features after global enhancement, and the backbone features and global enhancement features of adjacent scales are further input into the FRM module to obtain fused and enhanced multi-scale features, and then these features are input into the Transformer encoder with corresponding position embedding, and before the decoder, the decoder query is regarded as the position query and content query; a hybrid query selection strategy is adopted to initialize the anchors as the position query of the decoder, and the deformable attention DeformableAttention is used to combine the features of the input encoder through the initialized anchors and the learnable content query, and the query is updated layer by layer, and finally the refined power grid hidden danger target location coordinates and the classification results predicted by the refined content features are output.
[0015] In step S3, GSNet is a fully convolutional network that only applies linear operations in combined convolutions; the calculation formula is as follows;
[0016] M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1;
[0017] Y=M+R(M)+X Formula 2;
[0018] R(M)=conv2D(σ(conv2D(M)))Formula 3;
[0019] In Formula 1, Formula 2, and Formula 3, X is the input, which is the feature map extracted from the backbone feature pyramid, Y is the global semantic network feature output, and M is the intermediate result after feature extraction by two convolution branches. R(·) is the residual branch, and σ is the ReLU activation function.
[0020] In step S4, the calculation formula of the fusion enhancement module FRM is:
[0021] Z=f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 ]))))) Formula 4;
[0022] In Formula 4, [·] is the channel-level connection, X, Y are the feature maps from the backbone feature pyramid and the feature maps processed by GSNet, and f 1×1 、f 3×3 Represents 1*1 and 3*3 convolutions.
[0023] In step S5, the calculation formula of the sampling mechanism is:
[0024]
[0025] In formula 5, x is the input feature map, and each pixel in the feature map is a vector z of the C channel. q , the index of each pixel point, that is, the two-dimensional position coordinate is p q , M represents the number of heads in the multi-head attention mechanism, and each head only considers z q Nearby K points, Δp mqk Represents the sampling position offset, which is a two-dimensional coordinate. mqk Represents the attention weight, W m and W m ′ represents the weight parameters of two sets of fully connected layers, two A that need to be predicted mqk and Δp mqk All of them are calculated by z q Come from.
[0026] The present invention can effectively predict the precise location and classification of power grid hidden danger targets, thereby improving the accuracy and efficiency of power grid hidden danger target detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0028] Attached Figure 1It is a schematic diagram of the overall network model of Transformer feature fusion enhancement in the present invention;
[0029] Attached Figure 2 It is a schematic diagram of the global semantic network and fusion enhancement module in the present invention. DETAILED DESCRIPTION
[0030] As shown in the figure, a Transformer-based feature fusion enhanced power grid hidden danger target recognition method is described. The recognition method is based on DINO-DETR, and combines the global semantic network GSNet and the fusion enhancement module FRM to extract and integrate rich backbone features; multi-scale features are extracted from the image through ResNet, and globally enhanced through GSNet, and then the features of adjacent scales are further fused using the FRM module; the fused features are sent to the Transformer encoder with position embedding; in the decoder stage of the encoder, a hybrid query selection strategy is introduced to divide the decoder query into position query and content query, and the deformable attention mechanism is used to refine the features and update the query layer by layer to effectively identify the precise location and classification of power grid hidden danger targets, and improve the accuracy and efficiency of power grid hidden danger target detection.
[0031] The Transformer-based feature fusion enhanced power grid hidden danger target identification method includes the following steps:
[0032] Step S1: Data preprocessing and data set construction. The collected remote sensing images are preprocessed, including radiation correction, atmospheric correction, and geographic correction, to eliminate noise and errors in the images, generate standard remote sensing image products, and manually annotate the power grid hidden danger targets in the remote sensing images at the pixel level, add labels, and establish a remote sensing image power grid hidden danger target dataset for later network training;
[0033] Step S2: Construct the Transformer feature fusion enhanced overall network model, such as Figure 1 As shown in the figure, DINO-DETR is used as the basic model, and a global semantic network GSNet and a fusion enhancement module FRM are further adopted to obtain rich backbone features;
[0034] Step S3: Further construct the global semantic network GSNet in step S2, such as Figure 2 As shown in the figure, GSNet uses large convolution kernels or global convolutions to expand the effective feature perception range. In order to limit the increase in the number of parameters caused by direct k×k kernel convolution, GSNet uses 1×k+k×1 and k×1+1×k combined convolutions to contain detailed contextual information with symmetric and depth-separable combined convolutions, while reducing the number of parameters and computational costs;
[0035] Step S4: further construct the fusion enhancement module FRM in step S2, such as Figure 2 As shown in the figure, when there is a semantic information gap between feature maps of different scales and direct cascading is not a reasonable cross-scale fusion method, FRM introduces a residual branch from the trunk to inject various spatial context information; the deep features are superimposed on the original features with the residual structure to achieve the fusion of global information and local information; then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation capabilities; the enhanced feature pyramid contains more higher-level and semantic information;
[0036] Step S5: Further construct the attention network of the encoder and decoder modules in step S2, in which the DeformableAttention method is adopted, that is, a sampling mechanism is added to the Attention module, the size of the input image is ignored, and only a few sampling points near the target point are focused on to reduce the amount of calculation and speed up the convergence.
[0037] The specific method of step S2 is as follows: for a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are respectively input into GSNet to obtain the corresponding scale features after global enhancement, and the backbone features of adjacent scales and the global enhancement features are further input into the FRM module to obtain fused and enhanced multi-scale features, and then these features are input into the Transformer encoder with corresponding position embedding, and before the decoder, the decoder query is regarded as the position query and content query; a hybrid query selection strategy is adopted to initialize the anchors as the position query of the decoder, and the deformable attention is used to combine the features of the input encoder through the initialized anchors and the learnable content query, and the query is updated layer by layer, and finally the refined grid hidden danger target location coordinates and the classification results predicted by the refined content features are output.
[0038] In step S3, GSNet is a fully convolutional network that only applies linear operations in combined convolutions; the calculation formula is as follows;
[0039] M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1;
[0040] Y=M+R(M)+X Formula 2;
[0041] R(M)=conv2D(σ(conv2D(M)))Formula 3;
[0042] In Formula 1, Formula 2, and Formula 3, X is the input, which is the feature map extracted from the backbone feature pyramid, Y is the global semantic network feature output, and M is the intermediate result after feature extraction by two convolution branches. R(·) is the residual branch, and σ is the ReLU activation function.
[0043] In step S4, the calculation formula of the fusion enhancement module FRM is:
[0044] Z=f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 ]))))) Formula 4;
[0045] In Formula 4, [·] is the channel-level connection, X, Y are the feature maps from the backbone feature pyramid and the feature maps processed by GSNet, and f 1×1 、f 3×3 Represents 1*1 and 3*3 convolutions.
[0046] In step S5, the calculation formula of the sampling mechanism is:
[0047]
[0048] In formula 5, x is the input feature map, and each pixel in the feature map is a vector z of the C channel. q , the index of each pixel point, that is, the two-dimensional position coordinate is p q , M represents the number of heads in the multi-head attention mechanism, and each head only considers z q Nearby K points, Δp mqk Represents the sampling position offset, which is a two-dimensional coordinate. mqk Represents the attention weight, W m and W m ′ represents the weight parameters of two sets of fully connected layers, two A that need to be predicted mqk and Δp mqk All of them are calculated by z q Come from.
[0049] Example:
[0050] This example provides a Transformer-based feature fusion enhanced power grid hidden danger target recognition method, which uses remote sensing images as data sources, DINO-DETR as the basic model, and further adopts a global semantic network (GSNet) and a fusion enhancement module (FRM) to obtain rich backbone features. Specifically, given an image, ResNet is used to extract multi-scale features, and then the multi-scale features are respectively input into GSNet to obtain the corresponding scale features after global enhancement. The backbone features and global enhancement features of adjacent scales are further input into the FRM module to obtain fusion enhanced multi-scale features, and then these features are input into the Transformer encoder with corresponding position embedding. Before the decoder, the decoder query is regarded as the position query and content query. A hybrid query selection strategy is adopted to initialize the anchors as the position query of the decoder. Through the initialized anchors and the learnable content query, the deformable attention is used to combine the encoder's input features and update the query layer by layer, and finally output the refined power grid hidden danger target location coordinates and the classification results predicted by the refined content features.
[0051] Global Semantic Network (GSNet): GSNet should use the largest possible convolution kernel, or even global convolution, to significantly expand the effective feature perception range. However, direct k×k kernel convolution will significantly increase the number of parameters. GSNet uses 1×k+k×1 and k×1+1×k combined convolutions. These symmetric and depth-separable combined convolutions contain detailed contextual information while reducing the number of parameters and computational cost. GSNet is a fully convolutional network that only applies linear operations in combined convolutions. The calculation formula is as follows. Among them, X, as input, is the feature map extracted from the backbone feature pyramid, Y is the global semantic network feature output, and M is the intermediate result after feature extraction by two convolution branches. R(·) is the residual branch, and σ is the ReLU activation function.
[0052] M = conv(conv1D(X) T )+conv(conv1D(X)) T
[0053] Y=M+R(M)+X
[0054] R(M)=conv2D(σ(conv2D(M)))
[0055] Fusion Enhancement Module (FRM): Since there is a semantic information gap between feature maps of different scales, direct cascading is not a reasonable cross-scale fusion method. FRM introduces a residual branch from the backbone to inject various spatial context information. The residual structure superimposes deep features on the basis of the original features to achieve the fusion of global and local information. Then, stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce information loss in the channel, and enhance feature representation capabilities. The enhanced feature pyramid contains more higher-level and semantic information. The calculation formula is as follows. Where [·] is the channel-level connection, X, Y are the feature maps from the backbone feature pyramid and the feature maps processed by GSNet, and f 1×1 、f 3×3 Represents 1*1 and 3*3 convolutions.
[0056] Z=f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 ])))))
[0057] Deformable Attention: Both the encoder and decoder modules use the Deformable Attention method, that is, a sampling mechanism is added to the Attention module. No matter how large the input image is, it only focuses on a few points near the target point, which greatly reduces the amount of calculation and speeds up the convergence. The calculation formula is as follows. Where x is the input feature map, and each pixel in the feature map is a C-channel vector z q , the index of each pixel point, that is, the two-dimensional position coordinate is p q , M represents the number of heads in the multi-head attention mechanism, and each head only considers z q Nearby K points, Δp mqk Represents the sampling position offset, which is a two-dimensional coordinate. mqk Represents the attention weight, W m and W m ′ represents the weight parameters of two sets of fully connected layers, two A that need to be predicted mqk and Δp mqk All by calculating z q Come from.
[0058]
Claims
1. A Transformer-based feature fusion enhanced power grid hidden danger target identification method, characterized by: The recognition method is based on DINO-DETR, combining the global semantic network GSNet and the fusion enhancement module FRM to extract and integrate rich backbone features; multi-scale features are extracted from the image through ResNet, and globally enhanced through GSNet, and then the features of adjacent scales are further fused using the FRM module; The fused features are fed into a Transformer encoder with position embedding. In the decoder stage of the encoder, a hybrid query selection strategy is introduced to divide the decoder query into position query and content query. The deformable attention mechanism is used to refine the features and update the query layer by layer to effectively identify the precise location and classification of power grid hidden danger targets, thereby improving the accuracy and efficiency of power grid hidden danger target detection.
2. According to claim 1, a Transformer-based feature fusion enhanced power grid hidden danger target identification method is characterized by: The steps include: Step S1: Data preprocessing and data set construction: preprocess the collected remote sensing images, including radiation correction, atmospheric correction, and geographic correction, to eliminate noise and errors in the images, and manually annotate the power grid hidden danger targets in the remote sensing images at the pixel level, add labels, and establish a remote sensing image power grid hidden danger target dataset; Step S2: Construct the overall network model of Transformer feature fusion enhancement, take DINO-DETR as the basic model, and further adopt a global semantic network GSNet and a fusion enhancement module FRM to obtain rich backbone features; Step S3: further construct the global semantic network GSNet in step S2. GSNet uses large convolution kernels or global convolutions to expand the effective feature perception range. In order to limit the increase in the number of parameters caused by direct k×k kernel convolution, GSNet uses 1×k+k×1 and k×1+1×k combined convolutions to include detailed context information with symmetric and depth-separable combined convolutions, while reducing the number of parameters and computational cost; Step S4: further construct the fusion enhancement module FRM in step S2. When there is a semantic information gap between feature maps of different scales and direct cascading is not a reasonable cross-scale fusion method, FRM introduces a residual branch from the trunk to inject various spatial context information; the deep features are superimposed on the original features with the residual structure to achieve the fusion of global information and local information; then, the stacked convolutional layers are used to eliminate the aliasing effect caused by interpolation, reduce the information loss in the channel, and enhance the feature representation capability; the enhanced feature pyramid contains more higher-level and semantic information; Step S5: Further construct the attention network of the encoder and decoder modules in step S2, in which the Deformable Attention method is adopted, that is, a sampling mechanism is added to the Attention module, the size of the input image is ignored, and only a few sampling points near the target point are focused on to reduce the amount of calculation and speed up the convergence.
3. According to claim 2, a Transformer-based feature fusion enhanced power grid hidden danger target identification method is characterized by: The specific method of step S2 is as follows: for a given image, ResNet is used to extract multi-scale features, and then the multi-scale features are respectively input into GSNet to obtain the corresponding scale features after global enhancement, and the backbone features of adjacent scales and the global enhancement features are further input into the FRM module to obtain fused and enhanced multi-scale features, and then these features are input into the Transformer encoder with corresponding position embedding, and before the decoder, the decoder query is regarded as the position query and content query; a hybrid query selection strategy is adopted to initialize the anchors as the position query of the decoder, and the deformable attention is used to combine the features of the input encoder through the initialized anchors and the learnable content query, and the query is updated layer by layer, and finally the refined grid hidden danger target location coordinates and the classification results predicted by the refined content features are output.
4. According to claim 2, a method for identifying power grid hidden danger targets by feature fusion based on Transformer is characterized by: In step S3, GSNet acts as a fully convolutional network and only applies linear operations in combined convolutions; The calculation formula is as follows; M = conv(conv1D(X) T )+conv(conv1D(X)) T Formula 1; Y=M+R(M)+X Formula 2; R(M)=conv2D(σ(conv2D(M))) Formula 3; In Formula 1, Formula 2, and Formula 3, X is the input, which is the feature map extracted from the backbone feature pyramid, Y is the global semantic network feature output, and M is the intermediate result after feature extraction by two convolution branches. R(·) is the residual branch, and σ is the ReLU activation function.
5. According to claim 2, a method for identifying power grid hidden danger targets by feature fusion based on Transformer is characterized by: In step S4, the calculation formula of the fusion enhancement module FRM is: Z=f 1×1 (f 3×3 (f 1×1 (f 3×3 (f 1×1 ([X i ,Y i ,Y i+1 ]))))) Formula 4; In Formula 4, [·] is the channel-level connection, X, Y are the feature maps from the backbone feature pyramid and the feature maps processed by GSNet, and f 1×1 、f 3×3 Represents 1*1 and 3*3 convolutions.
6. According to claim 2, a method for identifying power grid hidden danger targets by feature fusion based on Transformer is characterized by: In step S5, the calculation formula of the sampling mechanism is: In formula 5, x is the input feature map, and each pixel in the feature map is a vector z of the C channel. q , the index of each pixel point, that is, the two-dimensional position coordinate is p q , M represents the number of heads in the multi-head attention mechanism, and each head only considers z q Nearby K points, Δp mqk Represents the sampling position offset, which is a two-dimensional coordinate. mqk Represents the attention weight, W m and W′ m Represents the weight parameters of two sets of fully connected layers, two A that need to be predicted mqk and Δp mqk All by calculating z q Come from.
Citation Information
Patent Citations
Traffic sign detection method based on dense connection attention
CN116109602A
Remote sensing image disaster detection method based on multi-scale adaptive semantic segmentation
CN116778165A
Container weak and small serial number target detection and identification method based on deep learning
CN117253154A
RGB-D image semantic segmentation method based on deep learning
CN117934849A
Method, apparatus, and device for obtaining disparity map, control system, and storage medium
WO2020182117A1
Cited By
Well lid hidden danger integrated identification method and system fusing target detection and segmentation
CN119672418A
Gradual correction medical image recognition system based on deformable attention mechanism
CN119888239A
Large model-based risk hidden danger identification method and device, electronic equipment and storage medium
CN120298836A
