A small sample target detection system and method based on an attention mechanism

Abstract

The application discloses a small sample target detection system and method based on an attention mechanism, and the system comprises a grouping space and channel attention module and an attention residual module. The grouping space and channel attention module is used for completing the attention mechanism between channels based on spatial information embedding, and loading the weight of the final spatial and channel attention on an original feature map. The attention residual module is based on the grouping space and channel attention module, and is composed of a convolution layer, a normalization layer, an activation function and a residual connection to form the attention residual module, so as to avoid attention gradient disappearance and gradient explosion.

Description

Technical Field

[0001] The present invention relates to a small sample target detection system and method based on an attention mechanism, which is applicable to the technical field of small sample target detection in computer vision. Background Art

[0002] With the development of human society, images and videos have become increasingly important sources of information. Deep learning systems based on large amounts of image and video samples are becoming increasingly mature. However, in industry, there is still a large demand for target detection in specialized, less popular categories. Applying target recognition technology to these niche areas is a challenge. The most significant challenge is the lack of datasets. Specifically, it is difficult to obtain large amounts of training data using publicly available datasets. Furthermore, even if large amounts of data are available, labeling them is not only time-consuming but also requires specialized technical expertise. Given these two challenges, deep learning-based target detection technology still has enormous potential for industrial application. If a relatively high-performance algorithm model can be trained from relatively small amounts of data, it will greatly facilitate the application of target detection in niche areas. Therefore, designing algorithms that fully utilize data for target detection with small sample sizes is an urgent need for the development of computer vision technology in industry. Object detection with small sample sizes holds great research value and has broad applications in healthcare, agriculture, marine science, and military fields.

[0003] However, due to the small size of the target category dataset, the detection accuracy of ordinary small-sample target detection models is far behind that of general target detection models with massive sample data. One of the reasons is the problem of model positioning the target. Due to the lack of prior knowledge, many model target detection result boxes deviate too far from the actual instance position or the detection model does not extract and pay enough attention to some key features. This is an important reason for the low accuracy of small-sample target detection.

[0004] In 2017, Google launched the Transformer architecture, which used the attention mechanism for translation tasks and achieved excellent results, once again propelling the attention mechanism into the spotlight. Attention models are widely used in a variety of deep learning tasks, including natural language processing, image recognition, and speech recognition. Their roots lie in research on human vision, which shows that when the ability to process information is limited, humans selectively focus on some information and ignore others. The attention mechanism enables the network to focus more on a specific set of features, which can be expressed using a general expression:

[0005] Attention(Q,,V)=softmax(QKT)V

[0006] Among them, Q is the query vector Queries, K is the key-value vector Keys and V is the content vector Values. A weight is calculated by the query vector and the key-value vector and added to the content vector, thereby achieving selective attention or ignoring of the content. Summary of the Invention

[0007] Aiming at the existing small-shot object detection problem, a small-shot object detection system based on attention mechanism is proposed. The system includes grouped spatial and channel attention modules and attention residual modules.

[0008] Specifically, the grouped spatial and channel attention module is mainly divided into two modules. The first module realizes spatial attention by grouping the feature maps in the channel dimension and generating correlation maps for the feature maps of each group in two directions of space. The second module completes the inter-channel attention mechanism based on spatial information embedding by splicing the correlation feature maps of different groups and different spatial dimensions, and then uniformly performs a series of convolution regularization operations, and then performs channel restoration separately. The final spatial and channel attention weights are loaded on the original feature maps.

[0009] Specifically, the attention residual block consists of a convolutional layer, a batch normalization layer, an activation function layer, and residual connections. As the foundational building block of the entire network, the attention residual module stacks these layers, along with a grouped spatial and channel-wise attention residual layer and residual connections. The embedding of these grouped spatial and channel-wise attention residual layers weights the feature maps after convolution operations in both spatial and channel dimensions, adaptively adjusting the importance of features across different channels, enhancing key feature information, and strengthening the network's feature extraction capabilities.

[0010] Through specific experiments, it was found that the small sample target detection system based on the attention mechanism improved the detection average precision mAP of the small sample target detection model, indicating that the present invention can effectively improve the small sample target detection accuracy of the model.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A small sample target detection system based on an attention mechanism, characterized by comprising the following steps:

[0013] Step S101: The input H×W×C feature map, where H represents image height, W represents image width, and C represents image channels, is divided into multiple groups in the channel dimension. Each group represents a specific sub-feature. By grouping, different features can be learned and noise suppressed in a targeted manner.

[0014] Step S102: Each set of features X kIt is divided into two paths, using pooling kernels of size (W, 1) and (1, H) to perform global average pooling (GAP) on each channel in the vertical and horizontal directions respectively, to obtain a row feature map with a height of H. And the column feature map with width W and The feature groups X are embedded in k Information in two spatial dimensions, and The expression of is shown in formula (1) (2):

[0015]

[0016]

[0017] where x c (i,h) represents the original feature map X k The i-th eigenvector of the h-th row, x c (w,j) represents the original feature map X k The j-th eigenvector of the w-th column;

[0018] Step S103: Divide into two branches, corresponding to the spatial attention module and the channel attention module respectively;

[0019] Step S104: In the spatial attention module, calculate the correlation mask map in two spatial directions and The expressions are shown in formula (3) and (4):

[0020]

[0021]

[0022] where · represents the dot product;

[0023] Step S105: and Normalize in space to get the normalized correlation mask map in the H direction and W direction and As shown in formulas (5)(6)(7):

[0024]

[0025]

[0026]

[0027] in Represents row dependency mask map and column correlation mask μ {h,w} express The spatial global average over each channel, express Spatial global variance on each channel;

[0028] Step S106: In the channel attention module, the row feature map and column feature maps Align and splice according to the channel to obtain the spliced ​​feature map right Use a 1×1 convolution and activation as shown in formula (8):

[0029]

[0030] Among them, δ is the nonlinear activation function, F1 is the convolutional encoding function, and the feature map of the channel after encoding is obtained. Here r represents the downsampling ratio, which serves as the bottleneck structure of the entire grouping space and channel attention module to control the size of the module;

[0031] Step S107: Next, split f back into the tensors of the previous two spatial dimensions. and Then use two 1×1 convolutions to decode the two cross-correlation feature maps to the original number of channels, and then use the activation function to obtain the following two feature map weights with values ​​in (0,1): and As shown in formula (9) and (10):

[0032]

[0033]

[0034] Among them F h is the convolution decoding function in the h direction, F w is the convolution decoding function in the w direction, σ(·) is the Sigmoid activation function;

[0035] Step S108: and Loading into the feature map weighted by the spatial attention mechanism in a row and column manner Finally, we get the kth group of feature maps after spatial and channel weighting As shown in formula (11):

[0036]

[0037] Where ⊙ represents the element-wise dot product;

[0038] Step S109: After obtaining the feature map after spatial and channel attention on each group, the different channel feature maps are spliced ​​and combined, and then the weights after spatial and channel attention are loaded onto the original feature map to obtain the final output of the grouped spatial and channel attention module.

[0039] Furthermore, the step S2 specifically includes:

[0040] Step S201: Input the feature model into the convolution layer for channel transformation;

[0041] Step S202: performing batch normalization on the feature map after channel transformation;

[0042] Step S203: Pass the batch normalized feature map through an activation function;

[0043] Step S204: Convolution the feature map after the activation function to restore the number of channels of the original feature map;

[0044] Step S205: Send the feature map with the original number of channels restored to the grouping space and channel attention module, perform attention weighting to obtain the feature map loaded with attention;

[0045] Step S206: The feature map loaded with attention is connected to the original feature map through residual connection to obtain the final output.

[0046] Compared with the existing main attention mechanism, the grouped space and channel attention module of the present invention can enhance the cross-correlation attention of the feature map channels in two spatial dimensions, and then merge all the groups after spatial enhancement into channels to enhance the attention in the channel dimension, thereby improving the detection model's attention to the instance. Especially for the small sample learning field with only a small number of samples, the present invention can improve the detection accuracy of the small sample target detection model; the attention residual block therein alleviates the problems of gradient vanishing and gradient exploding in the network backpropagation process by including the residual structure of the grouped space and channel attention modules, thereby achieving the effect of enhancing the small sample target detection network.

[0047] Compared with the current main attention mechanism, this invention can improve the detection performance accuracy of small sample targets. It is a more lightweight, more accurate attention mechanism that is more suitable for small sample target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flowchart of the small sample target detection system based on the attention mechanism in Example 1.

[0049] Figure 2 This is the structural diagram of the grouping space and channel attention module in Example 1.

[0050] Figure 3 This is the structural diagram of the attention residual block in Example 1. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0052] Example 1

[0053] See also Figure 1-Figure 3 ,This implementation provides a small sample object detection system based on ,attention mechanism.

[0054] Specifically, refer to Figure 1 , this system specifically includes:

[0055] Step S1: Construct group space and channel attention module;

[0056] Step S2: Construct an attention residual block;

[0057] Furthermore, the step S1 specifically includes:

[0058] Step S101: The input H×W×C feature map, where H represents image height, W represents image width, and C represents image channels, is divided into multiple groups in the channel dimension. Each group represents a specific sub-feature. By grouping, different features can be learned and noise suppressed in a targeted manner.

[0059] Step S102: Each set of features X k It is divided into two paths, using pooling kernels of size (W, 1) and (1, H) to perform global average pooling (GAP) on each channel in the vertical and horizontal directions respectively, to obtain a row feature map with a height of H. And the column feature map with width W and The feature groups X are embedded in k Information in two spatial dimensions, and The expression of is shown in formula (1) (2):

[0060]

[0061]

[0062] where x c (i,h) represents the original feature map X k The i-th eigenvector of the h-th row, x c (w,j) represents the original feature map X k The j-th eigenvector of the w-th column;

[0063] Step S103: Divide into two branches, corresponding to the spatial attention module and the channel attention module respectively;

[0064] Step S104: In the spatial attention module, calculate the correlation mask map in two spatial directions and The expressions are shown in formula (3) and (4):

[0065]

[0066]

[0067] where · represents the dot product;

[0068] Step S105: and Normalize in space to get the normalized correlation mask map in the H direction and W direction and As shown in formulas (5)(6)(7):

[0069]

[0070]

[0071]

[0072] in Represents row dependency mask map and column correlation mask μ {h,w} express The spatial global average over each channel, express Spatial global variance on each channel;

[0073] Step S106: In the channel attention module, the row feature map and column feature maps Align and splice according to the channel to obtain the spliced ​​feature map right Use a 1×1 convolution and activation as shown in formula (8):

[0074]

[0075] Among them, δ is the nonlinear activation function, F1 is the convolutional encoding function, and the feature map of the channel after encoding is obtained. Here r represents the downsampling ratio, which serves as the bottleneck structure of the entire grouping space and channel attention module to control the size of the module;

[0076] Step S107: Next, split f back into the tensors of the previous two spatial dimensions. and Then use two 1×1 convolutions to decode the two cross-correlation feature maps to the original number of channels, and then use the activation function to obtain the following two feature map weights with values ​​in (0,1): and As shown in formula (9) and (10):

[0077]

[0078]

[0079] Among them F h is the convolution decoding function in the h direction, F w is the convolution decoding function in the w direction, σ(·) is the Sigmoid activation function;

[0080] Step S108: and Loading into the feature map weighted by the spatial attention mechanism in a row and column manner Finally, we get the kth group of feature maps after spatial and channel weighting As shown in formula (11):

[0081]

[0082] Where ⊙ represents the element-wise dot product;

[0083] Step S109: After obtaining the feature map after spatial and channel attention on each group, the different channel feature maps are spliced ​​and combined, and then the weights after spatial and channel attention are loaded onto the original feature map to obtain the final output of the grouped spatial and channel attention module.

[0084] Furthermore, the step S2 specifically includes:

[0085] Step S201: Input the feature model into the convolution layer for channel transformation;

[0086] Feature S202: Batch normalize the feature map after channel transformation;

[0087] Step S203: Pass the batch normalized feature map through an activation function;

[0088] Step S204: Convolution the feature map after the activation function to restore the number of channels of the original feature map;

[0089] Step S205: Send the feature map with the original number of channels restored to the grouping space and channel attention module, perform attention weighting to obtain the feature map loaded with attention;

[0090] Step S206: The feature map loaded with attention is connected to the original feature map through residual connection to obtain the final output.

Claims

1. A small sample target detection system based on attention mechanism, characterized in that: The system includes: a grouped spatial and channel attention module and an attention residual module; The grouped spatial and channel attention module is used to complete the inter-channel attention mechanism based on spatial information embedding, and load the final spatial and channel attention weights on the original feature map; The attention residual module is a residual structure constructed based on the grouping space and channel attention modules; The grouped space and channel attention module is divided into three modules. The first module is the group pooling module, which groups the original feature maps and obtains the average pooling map of the grouped feature maps along the width W and height H directions; the second module is the spatial attention module, which generates spatial correlation maps for the feature maps of each group with the help of the pooling maps in two directions, and loads the correlation maps in two directions onto the source feature maps to realize spatial attention; the third module is the channel attention module, which splices the average pooling maps obtained in the W and H directions within the group, and then uniformly performs a series of convolution regularization operations to encode the spliced ​​feature maps, and then decodes them separately to realize channel restoration, and loads the final spatial and channel attention weights onto the original group feature maps to realize attention between channels.

2. The small sample target detection system based on the attention mechanism according to claim 1 is characterized in that: The attention residual module includes a convolution layer, an activation function layer, a batch normalization layer, a grouped spatial and channel attention residual layer, and a residual connection; the embedding of the grouped spatial and channel attention residual layer can weight the feature map after the convolution operation in the spatial and channel dimensions.

3. A small sample target detection method based on the system according to any one of claims 1-2, characterized in that: The steps include: Step S1: The feature maps are grouped in the channel dimension, and an average pooling map is generated for each group of feature maps in two directions of space; the pooling maps in the two directions are respectively generated into spatial correlation maps, and the correlation maps in the two directions of each group are loaded onto the source feature map to realize spatial attention; the correlation maps in the two directions of each group are spliced, and then a series of convolution regularization operations are uniformly performed to encode the spliced ​​feature maps, and then they are decoded separately to realize channel restoration, and the final spatial and channel attention weights are loaded onto the original group feature maps to realize inter-channel attention; each group of feature maps after spatial and channel attention weighting are spliced ​​and fused along the channel to obtain the output of the original feature map after the attention mechanism; Step S2: Construct an attention residual module based on each group space and channel attention module, convolution module, batch normalization module, and activation function module.

4. The small sample target detection method according to claim 3, characterized in that The step S1 specifically includes: Step S101: The input H×W×C feature map, where H represents image height, W represents image width, and C represents image channels, is divided into multiple groups in the channel dimension. Each group represents a specific sub-feature. By grouping, different features can be learned and noise suppressed in a targeted manner. Step S102: Each set of features X k It is divided into two paths, using pooling kernels of size (W, 1) and (1, H) to perform global average pooling on each channel in the vertical and horizontal directions respectively, and obtain a row feature map with a height of H. And the column feature map with width W and The feature groups X are embedded in k Information in two spatial dimensions, and The expressions of are shown in formulas (1) and (2): where x c (i,h) represents the original feature map X k The i-th eigenvector of the h-th row, x c (w,j) represents the original feature map X k The j-th eigenvector of the w-th column; Step S103: Divide into two branches, corresponding to the spatial attention module and the channel attention module respectively; Step S104: In the spatial attention module, calculate the correlation mask map in two spatial directions and The expressions are shown in formulas (3) and (4): where · represents the dot product; Step S105: and Normalize in space to get the normalized correlation mask map in the H direction and W direction and As shown in formulas (5), (6), and (7): in, Represents row dependency mask map and column correlation mask μ {h,w} express The spatial global average over each channel, express Spatial global variance on each channel; Step S106: In the channel attention module, the row feature map and column feature maps Align and splice according to the channel to obtain the spliced ​​feature map right Use a 1×1 convolution and activation as shown in formula (8): Among them, δ is the nonlinear activation function, F1 is the convolutional encoding function, and the feature map of the channel after encoding is obtained. Here r represents the downsampling ratio, which serves as the bottleneck structure of the entire grouping space and channel attention module to control the size of the module; Step S107: Next, split f back into the tensors of the previous two spatial dimensions. and Then use two 1×1 convolutions to decode the two cross-correlation feature maps to the original number of channels, and then use the activation function to obtain the following two feature map weights with values ​​in (0, 1): and As shown in formulas (9) and (10): Among them F h is the convolution decoding function in the h direction, F w is the convolution decoding function in the w direction, σ(·) is the Sigmoid activation function; Step S108: and Loading into the feature map weighted by the spatial attention mechanism in a row and column manner Finally, we get the kth group of feature maps after spatial and channel weighting As shown in formula (11): Where ⊙ represents the element-wise dot product; Step S109: After obtaining the feature map after spatial and channel attention on each group, the different channel feature maps are spliced ​​and combined, and then the weights after spatial and channel attention are loaded onto the original feature map to obtain the final output of the grouped spatial and channel attention module.

5. The small sample target detection method according to claim 3, characterized in that: The step S2 specifically includes: Step S201: Input the feature model into the convolution layer for channel transformation; Feature S202: Batch normalize the feature map after channel transformation; Step S203: Pass the batch normalized feature map through an activation function; Step S204: Convolution the feature map after the activation function to restore the number of channels of the original feature map; Step S205: Send the feature map with the original number of channels restored to the grouping space and channel attention module, perform attention weighting to obtain the feature map loaded with attention; Step S206: The feature map loaded with attention is connected to the original feature map through residual connection to obtain the final output.

Images