A method for extracting discriminative information from feature maps using convolutional neural networks based on multi-axis compression and fusion attention mechanism

By introducing a multi-axis compression fusion attention mechanism in convolutional neural network, using S-MCF and C-MCF modules to perform multi-axis compression operations on space and channels, the high computational complexity and refinement characteristics in the prior art are solved. The problem of insufficient mining is achieved, and the detailed features of feature maps are extracted efficiently, improving network performance.

CN118644762BActive Publication Date: 2024-12-03SHANDONG JIAOTONG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411103138.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-03
Estimated Expiration
2044-08-13

AI Technical Summary

Technical Problem

The existing convolutional neural network has high computational complexity when extracting discriminant information in feature maps and cannot fully explore and refine features. The self-attention mechanism calculates the weighted sum of features within the global scope and cannot effectively capture long-distance dependencies.

Method used

Design a convolutional neural network based on multi-axis compression fusion attention mechanism. By adding spatial multi-axis compression fusion S-MCF module and channel multi-axis compression fusion C-MCF module after ResNet-50 backbone network, respectively Multi-axis compression is performed on the channel, computing global context and dependencies, and fusing vertical and horizontal attention information to extract refined features.

Benefits of technology

It reduces the computational complexity, improves the computing efficiency, can fully explore the refined features of the feature map, improves the overall performance of the network, focuses on the details and structural features of the input feature map, and strengthens the areas of common concern on multiple axes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644762B_ABST
    Figure CN118644762B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of computer technology, and specifically to a method for extracting discriminative information from a feature map using a convolutional neural network based on a multi-axis compression fusion attention mechanism. The convolutional neural network combines a spatial multi-axis compression fusion S-MCF module and a channel multi-axis compression fusion C-MCF module. The multi-axis compression fusion strategy adopted can efficiently capture global and fine-grained features, reduce computational complexity, and fully mine detail information in feature maps, thereby significantly improving the overall performance of the network. While maintaining high computational efficiency, the present invention retains feature information on different axes through a multi-axis compression operation, so that the network can focus on the details and structural features of the input feature map in the horizontal direction and the appearance longitudinal features in the vertical direction, and the fusion of information from multiple axes can strengthen the areas of common concern on multiple axes to improve the effect of attention learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for extracting discriminative information in a feature graph by using a convolutional neural network based on a multi-axis compression fusion attention mechanism. Background Art

[0002] Convolutional neural networks have made significant breakthroughs in various computer vision fields, covering tasks such as image classification, semantic segmentation, and data extraction. Among them, the introduction of attention mechanisms in convolutional neural networks allows the model to pay more attention to the key information in the image, which plays an increasingly important role in various visual tasks. This dynamic and adaptive mechanism allows the model to assign different weights according to the importance of features, improving efficiency and model performance. Among the various types of attention mechanisms, the self-attention mechanism is a special form that updates the weight of each node (position / channel) by calculating the weighted sum of features based on pairwise dependencies on a global scale to model long-distance dependencies. However, this method not only has high computational complexity, but also fails to fully mine refined features. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a method for designing a convolutional neural network based on a multi-axis compression fusion attention mechanism to extract discriminative information from feature maps.

[0004] The technical solution adopted by the present invention to solve the technical problem is:

[0005] A method for extracting discriminative information from a feature map using a convolutional neural network based on a multi-axis compression fusion attention mechanism, using ResNet-50 as a backbone network, adding a spatial multi-axis compression fusion S-MCF (Spatial Multi-axis Compression Fusion) module and a channel multi-axis compression fusion C-MCF (Channel Multi-axis Compression Fusion) module in a sequential stacking manner after the res_conv5 layer to form a convolutional neural network based on a multi-axis compression fusion attention mechanism, the network is used to extract discriminative information from an input feature map;

[0006] The S-MCF module first obtains the robust feature responses of each row element and each column element on each channel through compression operations in different directions; secondly, the dependency between the same coordinates of each position is generated through self-attention calculation, and the refined features are mined while modeling the global context of different axes; finally, the feature information is aggregated by fusion of vertical and horizontal attention in space to obtain the output feature map of the S-MCF module;

[0007] The C-MCF module first obtains the robust feature responses of each row element and each column element on each channel through compression operations in different directions; secondly, channel self-attention calculations are performed on different axes to obtain the dependencies between each channel and other channels in different directions; finally, the vertical and horizontal attention of the channel are fused to obtain the output feature map of the C-MCF module.

[0008] The above technical solution reduces the computational complexity and can fully exploit the refined features of the input feature map.

[0009] Furthermore, the S-MCF module uses the output feature map F∈R of the res_conv5 layer C×H×W As input, H, W, C are the height, width and number of channels of the feature map F respectively; first, F undergoes 2D convolution with a convolution kernel of 1 and group convolution to obtain the feature map Q s ∈R C×H×W , K s ∈R C×H×W and V s ∈R C×H×W , where Q s =1DConv(F),(K s ,V s )=GConv(F), 1DConv represents 1×1 convolution, GConv represents group convolution; the average values ​​of the above three feature maps are calculated in the horizontal and vertical directions respectively to realize horizontal compression of the feature maps, and the feature maps after horizontal compression are obtained Q h s ∈R C×H×1 , K h s ∈R C×H×1 and V h s ∈R C×H×1 , after deformation, we get the matrix Q h’ s ∈R H×C , K h’ s ∈R C×H and V h’ s ∈R H×C , and the vertically compressed feature map Q v s ∈R C×1×W , K v s ∈RC×1×W and V v s ∈R C×1×W , after deformation, we get the matrix Q v’ s ∈R W×C , K v’ s ∈R C×W , V v’ s ∈R W×C ;

[0010] ( Q h’ s , K h’ s , V h’ s ) = Reshape( ( Q h s , K h s , V h s ));

[0011] ( Q v’ s , K v’ s , V v’ s ) = Reshape(mean↑( Q v s , K v s , V v s ));

[0012] The spatial vertical axis attention map is calculated by the following formula P h s ∈R H×C And the spatial horizontal axis attention map P v s ∈R W×C :

[0013] ;

[0014] ;

[0015] in, represents matrix multiplication, d c is the scaling factor.

[0016] The S-MCF module only propagates information on two compressed axial features, so the computational complexity is relatively low; the S-MCF module performs attention calculations in the horizontal and vertical directions respectively, which allows more parallel operations and improves computational efficiency.

[0017] Furthermore, the spatial vertical axis attention map and the spatial horizontal axis attention map are fused to obtain the spatial multi-axis fusion attention map A, the attention map A is masked to the input feature map F, and then added to F to obtain the feature map W s ∈R C×H×W : W s = A☉F + F, where ☉ is element-by-element multiplication; then batch normalization and GELU activation function are implemented in sequence to obtain the final output feature map R s ∈R C×H×W : R s = GELU(BN( W s )).

[0018] Furthermore, the C-MCF module is based on the output feature map of the S-MCF module. R s ∈R C×H×W As input, H, W, C are feature maps R s The height, width and number of channels; R s After 2D convolution with a convolution kernel of 1 and group convolution, the feature map Q is obtained. c ∈R C×H×W , K c ∈R C×H×W and V c ∈R C×H×W , where Q c =1DConv( R s ), (K c ,V c )=GConv( R s), 1DConv represents 1×1 convolution, GConv represents group convolution; the average values ​​of the above three feature maps are calculated in the horizontal and vertical directions respectively to realize horizontal compression of the feature maps, and the feature maps after horizontal compression are obtained Q h c ∈R C×H×1 , K h c ∈R C×H×1 and V h c ∈R C×H×1 , after deformation, we get the matrix Q h’ c ∈R C×H , K h’ c ∈R H×C and V h’ c ∈R H×C ; and the vertically compressed feature map Q v c ∈R C×1×W , K v c ∈R C×1×W and V v c ∈R C×1×W , after deformation, we get the matrix Q v’ c ∈R C×W , K v’ c ∈R W×C , V v’ c ∈R W×C ;

[0019] ( Q h’ c , K h’ c , K h’ c ) = Reshape( ( Q h c , K hc , V h c ));

[0020] ( Q v’ c , K v’ c , V v’ c ) = Reshape(mean↑( Q v c , K v c , V v c ));

[0021] The channel vertical axis attention map is calculated by the following formula P h c ∈R H×C And channel horizontal axis attention map P v c ∈R W×C :

[0022] ;

[0023] ;

[0024] in, represents matrix multiplication, d c is the scaling factor.

[0025] Furthermore, the channel vertical axis attention map and the channel horizontal axis attention map are fused to obtain the channel multi-axis fused attention map B, and the attention map B is masked to the input feature map R s , and then with R s Add to get the feature map W c ∈R C×H×W : W c =B☉ R s + R s , where ☉ is element-by-element multiplication; then batch normalization and GELU activation function are implemented in sequence to obtain the output feature map of the C-MCF module R c ∈RC×H×W : R c = GELU(BN( W c )).

[0026] Furthermore, a global average pooling operation is performed on the output features of the C-MCF module and compressed into a 2048-dimensional feature vector. The feature vector is subjected to dimensionality reduction processing by applying 1*1 convolution, batch normalization and ReLU activation function, and finally a 256-dimensional feature vector is obtained.

[0027] Technical effects of the present invention:

[0028] Compared with the prior art, the present invention proposes a convolutional neural network based on a multi-axis compression fusion attention mechanism. The network combines the spatial multi-axis compression fusion S-MCF module and the channel multi-axis compression fusion C-MCF module. The multi-axis compression fusion strategy adopted can efficiently capture global and fine-grained features, reduce computational complexity, and fully mine the detail information in the feature map, thereby significantly improving the overall performance of the network. While maintaining high computational efficiency, the present invention retains feature information on different axes through multi-axis compression operations, so that the network can focus on the details and structural features of the input feature map in the horizontal direction and the appearance longitudinal features in the vertical direction, and the fusion of information on multiple axes can strengthen the areas of common concern on multiple axes to improve the effect of attention learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a structural diagram of a convolutional neural network based on a multi-axis compression fusion attention mechanism of the present invention;

[0030] Figure 2 It is the structural block diagram of the S-MCF module of the present invention;

[0031] Figure 3 This is a structural block diagram of the C-MCF module of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention is clearly and completely described below in conjunction with the accompanying drawings. Example

[0033] like Figure 1 As shown, the present embodiment involves a method for extracting discriminative information from a feature map using a convolutional neural network based on a multi-axis compression fusion attention mechanism, including:

[0034] The convolutional neural network uses ResNet-50 as the backbone network, and adds a spatial multi-axis compression fusion S-MCF module and a channel multi-axis compression fusion C-MCF module in a sequential stacking manner after the res_conv5 layer to form a convolutional neural network based on a multi-axis compression fusion attention mechanism, and the network is used to extract discriminative information in the input feature map;

[0035] The S-MCF module first obtains the robust feature responses of each row element and each column element on each channel through compression operations in different directions; secondly, after self-attention calculation, the S-MCF module generates the dependency between the same coordinates of each position, and mines the refined features while modeling the global context of different axes; finally, the fusion and aggregation feature information of the vertical and horizontal directions of the space is used to strengthen the features of common attention on different axes to improve the effect of attention learning;

[0036] The C-MCF module first obtains the robust feature responses of each row element and each column element on each channel through compression operations in different directions; secondly, channel self-attention calculations are performed on different axes to obtain the dependencies between each channel and other channels in different directions; finally, the vertical and horizontal attention of the channel are fused to infer the importance of the channel.

[0037] 1. S-MCF module structure

[0038] like Figure 2 As shown, the output feature map F∈R of the res_conv5 layer C×H×W As input, H, W, C are the height, width and number of channels of the feature map F respectively; first, F undergoes 2D convolution with a convolution kernel of 1 and group convolution to obtain the feature map Q s ∈R C ×H×W , K s ∈R C×H×W and V s ∈R C×H×W , where Q s =1DConv(F),(K s ,V s )=GConv(F), 1DConv means 1×1 convolution, GConv means grouped convolution. Grouped convolution divides the input feature map into multiple groups, performs convolution calculations within each group, and allows weights to be shared within each group, which means that grouped convolution can effectively reduce the number of parameters and thus improve computational efficiency. The feature map is horizontally compressed by taking the average of the above three feature maps in the horizontal direction to obtain the feature map Q h s ∈R C×H×1 ,K h s ∈R C×H×1 and V h s ∈R C×H×1 , after deformation, we get the matrix Q h’ s ∈R H×C , K h’ s ∈R C×H and V h’ s ∈R H×C ,

[0039] ( Q h’ s , K h’ s , V h’ s ) = Reshape( ( Q h s , K h s , V h s ));

[0040] Similarly, the feature map Q v s ∈R C×1×W , K v s ∈R C×1×W and V v s ∈R C×1×W It is obtained by calculating the average value of the three feature maps in the vertical direction, and the matrix is ​​obtained after deformation Q v’ s ∈R W×C , K v’ s ∈R C×W , V v’ s ∈R W×C ,

[0041] (Q v’ s , K v’ s , V v’ s ) = Reshape(mean↑( Q v s , K v s , V v s )).

[0042] Horizontal and vertical compression focus on the horizontal and vertical feature information of the feature map respectively. In this way, the model can focus on important features in a specific direction, which helps to better capture the structure and pattern in the input feature map. Specifically, horizontal compression first helps the network focus on the detailed features of the input feature map, and secondly, horizontal compression can capture the horizontal structural features of the input feature map. Vertical compression can pay more attention to the vertical features of the input feature map.

[0043] Calculating the Matrix Q h’ s and K h’ s The product of and use the softmax activation function to get the spatial vertical axis relationship matrix X h s ∈R H×H , X h s Each row in represents the relationship between a certain position and other H positions, and then, with the matrix V h’ s Perform weighted summation to obtain the spatial vertical axis attention map P h s ∈R H×C , the specific calculation formula is:

[0044] ;

[0045] in, Represents matrix multiplication, the Softmax function along the matrix X h s Normalize in the column direction, d c is the scaling factor.

[0046] Similarly, the spatial horizontal axis relationship matrix X v s ∈R W×W Each row in the graph represents the relationship between a certain position and other W positions, and the horizontal axis of the space is the attention map. P v s ∈R W×C It is expressed as:

[0047] .

[0048] The traditional self-attention mechanism needs to calculate the relationship between the query vector and the key vector for all positions, resulting in a computational complexity that is proportional to the square of the number of positions. The S-MCF module only propagates information on two compressed axial features, so the computational complexity is relatively low. Performing attention calculations in the horizontal and vertical directions separately allows more parallel operations, thus improving computational efficiency to a certain extent.

[0049] Attention in a single direction may ignore or weaken information in other directions, while multi-directional fusion can alleviate the limitations of single-directional attention and make more comprehensive use of relevant information in the feature map. After obtaining the spatial vertical axis attention map and the spatial horizontal axis attention map, the two are fused to obtain the spatial multi-axis fusion attention map A. The attention map A not only contains information from multiple axes, but also by adding and fusing the attention maps of multiple axes, it can strengthen the areas of common concern on multiple axes, regard these areas as more important features, and help improve the model's extraction and utilization of key information.

[0050] Mask the attention map A to the input feature map F, and then add it to F to get the feature map W s ∈R C×H×W Then batch normalization (BN) and GELU activation function are implemented in sequence to obtain the final output feature map R s ∈R C×H×W , described by the formula:

[0051] W s = A☉F + F;

[0052] R s = GELU(BN( W s ));

[0053] Among them, ☉ is element-by-element multiplication.

[0054] 2. C-MCF module structure

[0055] like Figure 3 As shown, the output feature map of the S-MCF module R s ∈R C×H×W As input, H, W, C are feature maps R s The height, width and number of channels; R s After 2D convolution with a convolution kernel of 1 and group convolution, the feature map Q is obtained. c ∈R C ×H×W , K c ∈R C×H×W and V c ∈R C×H×W , where Q c =1DConv( R s ), (K c ,V c )=GConv( R s ), 1DConv represents 1×1 convolution, GConv represents grouped convolution; take the average of the above feature maps in the horizontal direction to get the feature map Q h c ∈R C×H×1 , K h c ∈R C×H×1 and V h c ∈R C×H×1 , after deformation, we get the matrix Q h’ c ∈R C×H , K h’ c ∈R H×C and V h’ c ∈R H×C ,

[0056] ( Q h’ c , K h’ c , K h’ c ) = Reshape( ( Qh c , K h c , V h c ));

[0057] Similarly, the three feature maps after vertical compression are deformed to obtain the matrix Q v’ c ∈R C×W , K v’ c ∈R W×C , V v’ c ∈R W×C ,

[0058] ( Q v’ c , K v’ c , V v’ c ) = Reshape(mean↑( Q v c , K v c , V v c ));

[0059] After matrix multiplication and softmax activation function, the channel vertical axis relationship matrix is ​​obtained X h c ∈R C×C and channel horizontal axis relationship matrix X v c ∈R C×C Unlike the S-MCF module, the C-MCF module aims to capture the relationship between channels in different directions. The element at position (i, j) in the relationship matrix X i , j Represents the pairwise relationship between the i-th channel and the j-th channel. The i-th row in the matrix X i , * Indicates the relationship between each channel and the ith channel. Although the two C×C matrices are the same in size, they represent different meanings. X h creflects the horizontal correlation between channels in the feature map, while X v c It reflects the vertical correlation between channels in the feature map.

[0060] X h c and X v c Respectively with the matrix V h’ c and V v’ c Perform weighted summation to obtain the channel vertical axis attention map P h c ∈R H×C And channel horizontal axis attention map P v c ∈R W×C , the specific calculation formula is:

[0061] ;

[0062] ;

[0063] in, Represents matrix multiplication, the Softmax function along the matrix X h c Normalize in the column direction, d c is the scaling factor.

[0064] After obtaining the channel vertical axis attention map and the channel horizontal axis attention map, the two are fused to obtain the channel multi-axis fusion attention map B. Mask the attention map B to the input feature map R s , and then with R s Add to get the feature map W c ∈R C×H×W Then batch normalization (BN) and GELU activation function are implemented in sequence to obtain the final output feature map R c ∈R C×H×W , described by the formula:

[0065] W c = B☉ R s + Rs ;

[0066] R c = GELU(BN( W c ));

[0067] Among them, ☉ is element-by-element multiplication.

[0068] A global average pooling operation is performed on the output features of the C-MCF module and compressed into a 2048-dimensional feature vector. The feature vector is reduced in dimension by applying 1*1 convolution, batch normalization (Batch Normalization, BN) and ReLU (Rectified Linear Unit) activation function, and finally a 256-dimensional feature vector is obtained.

[0069] The present invention adds an S-MCF module and a C-MCF module after the res_conv5 layer, uses the S-MCF module to highlight important spatial positions, and uses the C-MCF module to adaptively calibrate the weight of each channel to extract important features. In addition, the two modules proposed by the present invention perform self-attention from different dimensions, prompting the network to extract fine-grained features of the input feature map. The feature map after the S-MCF module and the C-MCF module is compressed and reduced in dimension by pooling to obtain a 256-dimensional feature vector.

[0070] The 256-dimensional feature vector obtained by the present invention is used for training of triplet loss, and is used for training of cross entropy loss after conversion by a fully connected layer. Specifically, during the training process, cross entropy loss is used to classify samples of different categories, and triplet loss is used to optimize the distance between features of each category.

[0071] The present invention proposes a multi-axis compression fusion attention mechanism. Specifically, the mechanism first obtains the robust feature response of each row element on each channel through the compression operation in the horizontal direction, while the compression operation in the vertical direction generates the robust feature response of each column element on each channel. Secondly, the weight is updated by parallel self-attention calculation in a single axis direction, which greatly reduces the complexity, and mines the refined features in different directions while modeling the global context. The present invention aggregates feature information by fusing the attention graphs of multiple axes and strengthens the features of common concern on multiple axes to improve the effect of attention learning. Based on the attention mechanism, two attention modules are designed from the channel and space aspects: the spatial multi-axis compression fusion (S-MCF) module and the channel multi-axis compression fusion (C-MCF) module. When performing attention calculation, the S-MCF module focuses on the relationship between the same coordinates of each position in different directions. When performing self-attention calculation, the C-MCF module focuses on the dependency between channels. The present invention introduces the above-mentioned two attention modules of different dimensions into a convolutional neural network, and proposes a convolutional neural network based on a multi-axis compressed fusion attention mechanism. The network uses ResNet-50 as the backbone network, and adds S-MCF modules and C-MCF modules in a sequential stacking manner after the res_conv5 layer. The spatial module and the channel module use the self-attention mechanism to extract subtle discriminative information of the input feature map, which can locate key clues and capture rich discriminative information without relying on additional labels and models, thereby improving the overall performance of the network.

[0072] The above-mentioned specific implementation modes are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementation modes. Any appropriate changes or modifications made thereto by any ordinary technician in the technical field in accordance with the claims of the present invention shall fall within the patent protection scope of the present invention.

Claims

1. A method for extracting discriminative information from feature maps using a convolutional neural network based on multi-axis compression and fusion attention mechanism. It is characterized in that Using ResNet-50 as the backbone network, adding the spatial multi-axis compression fusion S-MCF module and the channel multi-axis compression fusion C-MCF module in a sequential stacking manner after the res_conv5 layer to form a convolutional neural network based on the multi-axis compression fusion attention mechanism, and taking the input feature map as the input of the network, the network is used to extract the discriminative information in the input feature map; The S-MCF module first obtains the robust feature responses of each row element and each column element on each channel through compression operations in different directions; secondly, the dependency between the same coordinates of each position is generated through self-attention calculation, and the refined features are mined while modeling the global context of different axes; finally, the feature information is aggregated by fusion of vertical and horizontal attention in space to obtain the output feature map of the S-MCF module; The C-MCF module first obtains the robust feature response of each row element and each column element on each channel through compression operations in different directions; secondly, channel self-attention calculation is performed on different axes to obtain the dependency relationship between each channel and other channels in different directions; finally, the vertical and horizontal attention of the channel are fused to obtain the output feature map of the C-MCF module, so as to realize the extraction of discriminative information in the input feature map; The S-MCF module uses the output feature map F∈R of the res_conv5 layer. C×H×W As input, H, W, C are the height, width and number of channels of feature map F respectively; first, F undergoes 2D convolution with kernel 1 and group convolution to obtain feature map Q s ∈R C ×H×W , K s ∈R C×H×W and V s ∈R C×H×W , where Q s =1DConv(F),(K s ,V s )=GConv(F), 1DConv represents 1×1 convolution, GConv represents group convolution; the average values ​​of the above three feature maps are calculated in the horizontal and vertical directions respectively to realize horizontal compression of the feature maps, and the feature maps after horizontal compression are obtained Q h s ∈R C×H×1 , K h s ∈R C×H×1 and V h s ∈R C×H×1 , after deformation, we get the matrix Q h’ s ∈R H×C , K h’ s ∈R C×H and V h’ s ∈R H×C , and the vertically compressed feature map Q v s ∈R C×1×W , K v s ∈R C×1×W and V v s ∈R C×1×W , after deformation, we get the matrix Q v’ s ∈R W×C , K v’ s ∈R C×W , V v’ s ∈R W×C ; ( Q h’ s , K h’ s , V h’ s ) = Reshape( ( Q h s , K h s , V h s )); ( Q v’ s , K v’ s , V v’ s ) = Reshape(mean↑( Q v s , K v s , V v s )); The spatial vertical axis attention map is calculated by the following formula P h s ∈R H×C And the spatial horizontal axis attention map P v s ∈R W ×C : ; ; in, represents matrix multiplication, d c is the scaling factor; The spatial vertical axis attention map and the spatial horizontal axis attention map are fused to obtain the spatial multi-axis fusion attention map A, the attention map A is masked to the input feature map F, and then added to F to obtain the feature map W s ∈R C×H×W : W s = A☉F + F, where ☉ is element-by-element multiplication; then batch normalization and GELU activation function are implemented in sequence to obtain the output feature map of the S-MCF module R s ∈R C×H×W : R s = GELU(BN( W s )); The C-MCF module is based on the output feature map of the S-MCF module. R s ∈R C×H×W As input, H, W, C are feature maps R s The height, width and number of channels; R s After 2D convolution with a convolution kernel of 1 and group convolution, the feature map Q is obtained. c ∈R C×H×W , K c ∈R C×H×W and V c ∈R C×H×W , where Q c =1DConv( R s ), (K c ,V c )=GConv( R s ), 1DConv represents 1×1 convolution, GConv represents group convolution; the average values ​​of the above three feature maps are calculated in the horizontal and vertical directions respectively to realize horizontal compression of the feature maps, and the feature maps after horizontal compression are obtained Q h c ∈R C×H×1 , K h c ∈R C×H×1 and V h c ∈R C×H×1 , after deformation, we get the matrix Q h’ c ∈R C×H , K h’ c ∈R H×C and V h’ c ∈R H×C ; and the vertically compressed feature map Q v c ∈R C×1×W , K v c ∈R C×1×W and V v c ∈R C×1×W , after deformation, we get the matrix Q v’ c ∈R C×W 、 K v’ c ∈R W×C 、 V v’ c ∈R W×C ; ( Q h’ c , K h’ c , K h’ c ) = Reshape( ( Q h c , K h c , V h c )); ( Q v’ c , K v’ c , V v’ c ) = Reshape(mean↑( Q v c , K v c , V v c )); The channel vertical axis attention map is calculated by the following formula P h c ∈R H×C And channel horizontal axis attention map P v c ∈R W ×C : ; ; in, represents matrix multiplication, d c is the scaling factor; By retaining feature information on different axes through multi-axis compression operations, the network can focus on the details and structural features in the horizontal direction of the input feature map and the appearance longitudinal features in the vertical direction, while fusing information from multiple axes can strengthen the areas of common concern on multiple axes.

2. The method for extracting discriminative information from feature graphs using a convolutional neural network based on multi-axis compression fusion attention mechanism according to claim 1, It is characterized in that The channel vertical axis attention map and the channel horizontal axis attention map are fused to obtain the channel multi-axis fused attention map B, and the attention map B is masked to the input feature map R s , and then with R s Add to get the feature map W c ∈R C×H×W : W c = B☉ R s + R s , where ☉ is element-by-element multiplication; then batch normalization and GELU activation function are implemented in sequence to obtain the output feature map of the C-MCF module R c ∈R C×H×W : R c = GELU(BN( W c )).

3. According to claim 1, the method for extracting discriminative information from feature graphs using a convolutional neural network based on a multi-axis compression fusion attention mechanism, It is characterized in that A global average pooling operation is performed on the output features of the C-MCF module and compressed into a 2048-dimensional feature vector. The feature vector is reduced in dimension by applying 1*1 convolution, batch normalization and ReLU activation function, and finally a 256-dimensional feature vector is obtained.

Citation Information

Patent Citations

  • Thread turning direction classification method based on convolutional neural network

    CN114202502A

  • Vehicle re-identification method based on cross-context and feature response attention mechanism

    CN116152792A