A deep convolutional network target recognition method with dual-channel attention mechanism

Abstract

The present invention discloses a deep convolutional network target recognition method with a dual-channel attention mechanism, comprising the following steps: constructing a convolutional neural network, taking a single sample pair as input, and extracting a high-dimensional feature map; constructing spatial attention and channel attention mechanism modules respectively, taking the two high-dimensional feature maps extracted by the neural network as input, calculating the correlation between feature pixels in the spatial dimension and adding them element by element with the original features; stacking the outputs of the spatial and channel attention mechanism modules in the channel dimension to obtain the final feature representation of the model; constructing training sample pairs, expanding the scale of similar targets through data enhancement, and directly pairing different targets; calculating the cross entropy loss and learning the network parameters through stochastic gradient descent to obtain a neural network model with the ability to distinguish target categories. The present invention can improve the accuracy of visual target image recognition in single-sample scenarios and for target categories that have not participated in training.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to a deep convolutional network target recognition method with a dual-channel attention mechanism. Background Art

[0002] In the past decade, deep learning has achieved great success in the field of computer vision, and more and more researchers have begun to pay attention to the application of neural networks in object recognition.

[0003] While neural network models have achieved impressive results in most object recognition tasks, they still face challenges in real-world production environments, such as a wide variety of object types, insufficient training samples, fine-grained variations within classes, and an increase in categories. Neural networks are typical supervised learning algorithms that rely on large, labeled training datasets. This data cost is significant, making it difficult to collect sufficient images for each target class for training. Furthermore, when faced with frequently changing categories, typical neural network classification models are unable to effectively handle classes that were not included in training. This is another issue that needs to be addressed before this technology can be put into practical use. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a deep convolutional network target recognition method with a dual-channel attention mechanism, which can realize automatic classification of visual targets under the premise that there is only one training image sample for each type of target.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A deep convolutional network target recognition method with a dual-channel attention mechanism includes the following steps:

[0007] Step 1: Construct a convolutional neural network, take image sample pairs as input, and extract high-dimensional feature maps;

[0008] Step 2: Construct a spatial attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature pixels in the spatial dimension and adds them element-by-element with the original features;

[0009] Step 3: Construct a channel attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature channels in the channel dimension, and adds them element by element with the original features;

[0010] Step 4: Stack the outputs of the spatial attention mechanism module and the channel attention mechanism module in the channel dimension to obtain the final feature representation of the model;

[0011] Step 5: Construct training sample pairs. Targets of the same type are expanded through data augmentation, and targets of different types are directly paired.

[0012] Step 6: Calculate the cross entropy loss and learn the network parameters through stochastic gradient descent to obtain a neural network model with the ability to distinguish target categories.

[0013] Furthermore, in the present invention: said step 1 also includes,

[0014] Step 1-1: Construct a convolutional neural network, which includes 17 convolutional layers. The head convolutional layer consists of 64 7×7 convolution kernels with a stride of 2, which downsamples the input image by 0.5 times and increases the number of feature map channels to 64 dimensions. The maximum pooling layer uses a 3×3 window size and a stride of 2 to downsample the feature map by 0.5 times. Except for the head convolutional layer, every two convolutional layers using 3×3 convolution kernels form a residual module with a direct connection structure, totaling 8 residual modules. The first convolution layer in each residual block has a stride of 2, and the rest have a stride of 1. The number of convolution kernels increases with the depth of the network. Finally, a high-dimensional feature map with a size of 1 / 32 of the input image and a channel number increased to 512 dimensions is extracted. The network weights are obtained by random initialization and are continuously updated through backpropagation during training.

[0015] Step 1-2: Construct two identical convolutional neural networks as described in step 1-1. Each convolutional neural network receives one image in the image sample pair as input and outputs high-dimensional feature maps F1 and F2 respectively.

[0016] Furthermore, in the present invention: said step 2 also includes,

[0017] Step 2-1: For the original high-dimensional feature map F∈R extracted by the convolutional neural network described in step 1 C×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively, and are input into three groups of 1×1 convolutional layers to obtain three new feature maps F a 、F b 、F c , and flatten its width and height dimensions, that is, {F a ,F b ,F c}∈R C×(H×W) Then, F a The transpose of F b After multiplication, the spatial attention matrix M is obtained through the Softmax function s ∈R (H×W)×(H×W) , specifically

[0018]

[0019] in, Indicates the correlation between the feature pixels at the i-th position and the j-th position, T represents transposition, Fa 、F b It is the feature map output by the convolutional layer.

[0020] Step 2-2: Make F c With M s Multiply and combine with the original high-dimensional feature F∈R C×H×W Add element by element to obtain the output feature F s , specifically

[0021]

[0022] where η s is a trainable scale factor and is initialized to 0 to avoid Too large, j is the subscript of the spatial position.

[0023] Furthermore, in the present invention: said step 3 also includes,

[0024] Step 3-1: For the high-dimensional feature map F∈R extracted by the neural network described in step 1 C×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively. After flattening its width and height dimensions, the channel attention matrix M is obtained by multiplying it with its own transpose. t ∈R C×C , let i,j represent the i-th and j-th positions in space, T is the transpose operation, specifically

[0025]

[0026] Step 3-2: Make F and M t Multiply and combine with the original feature F∈R C×H×W Add element by element to obtain the output feature F t , specifically

[0027]

[0028] where η t is a trainable scale factor and is initialized to 0 to avoid Too big.

[0029] Furthermore, in the present invention: said step 4 also includes,

[0030] Step 4-1: For the attention mechanism modules described in step 2 and step 3, the respective F s With F t Stacked in the channel dimension to form a dual attention mechanism module;

[0031] Step 4-2: For the two high-dimensional feature maps F1, F2∈R output by the neural network described in step 1 C×H×W, respectively pass it through the dual attention mechanism module to obtain a better feature representation F′1,F′2∈R (2×C)×H×W .

[0032] Furthermore, in the present invention: said step 5 also includes,

[0033] Step 5-1: For different target category images, directly form sample pairs for training the network

[0034] Step 5-2: For images of the same target category, random scale scaling, rotation, affine transformation, and brightness, saturation, and contrast adjustments are performed to make the two images within each pair of the same sample different and then form sample pairs, so that the number of sample pairs of the same target image and the number of sample pairs of different target images are consistent. Assuming that there are E categories of visual targets to be identified, the single sample training set described in this patent contains E target images. After constructing the sample pairs according to the method described in step 5, the total number of sample pairs N is pairs for

[0035]

[0036] Where n is the independent variable.

[0037] Furthermore, in the present invention: said step 6 also includes,

[0038] Step 6-1: Let the neural network process the output of a pair of feature maps F′1, F′2∈R through the dual attention mechanism (2 ×C)×H×W After global average pooling, we get a pair of feature vectors f1,f2∈R (2×C) , we calculate the Euclidean distance between f1 and f2, and map it to the range of [0,1] through the Sigmoid function to obtain the final output y of the neural network i , then define the cross entropy loss function loss, specifically as

[0039]

[0040] Where i represents the i-th pair of output features, y i , is the output of the neural network.

[0041] Step 6-2: Using loss as the loss function and the sample pairs described in step 5 as input, the adaptive moment estimation algorithm is used to train the neural network described in steps 1 to 4. The first-order moment estimation and second-order moment estimation of the gradient are used to dynamically adjust the learning rate of each parameter. The weight decay of the adaptive moment estimation algorithm is set to 5e -5 , 32 samples are input as a small batch, and the learning rate is initialized to 4e -3, it decays to half of its original value every 40 iterations, and after 200 iterations in total, a neural network model with the ability to distinguish target categories is obtained.

[0042] The beneficial effects of the present invention are:

[0043] Under the condition that there is only one training image for each type of target, the neural network can be trained by constructing data augmentation sample pairs to expand the training data; the neural network structure enables the model to be trained with a small number of samples and to identify target categories that have not participated in the training; the dual attention mechanism improves the intra-class compactness and inter-class discrimination, thereby improving the recognition accuracy; the cross-entropy loss function avoids the imbalance of penalty caused by manually setting the margin during training, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the overall process of the method of the present invention;

[0045] Figure 2 This is a diagram of the convolutional network structure used for image feature extraction in the present invention;

[0046] Figure 3 Schematic diagram of the spatial attention mechanism module in the present invention;

[0047] Figure 4 Schematic diagram of the channel attention mechanism module in the present invention;

[0048] Figure 5 The following are the ablation test results of the method described in the present invention on the experimental dataset. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0050] As attached Figure 1 FIG. 1 is a schematic diagram of the overall process of a deep convolutional network target recognition method with a dual-channel attention mechanism proposed in the present invention. The method specifically includes the following steps:

[0051] Step 1: Construct a convolutional neural network, take image sample pairs as input, and extract high-dimensional feature maps.

[0052] Step 2: Construct a spatial attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature pixels in the spatial dimension and adds them element-by-element with the original features;

[0053] Step 3: Construct a channel attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature channels in the channel dimension, and adds them element by element with the original features;

[0054] Step 4: Stack the outputs of the spatial attention mechanism module and the channel attention mechanism module in the channel dimension to obtain the final feature representation of the model;

[0055] Step 5: Construct training sample pairs. Targets of the same type are expanded through data augmentation, and targets of different types are directly paired.

[0056] Step 6: Calculate the cross entropy loss and learn the network parameters through stochastic gradient descent to obtain a neural network model with the ability to distinguish target categories.

[0057] As attached Figure 2 As shown in the figure, two identical convolutional neural networks are constructed, each receiving one image from a pair of image samples as input and outputting high-dimensional feature maps F1 and F2, respectively. The constructed convolutional neural network contains 17 convolutional layers. The head convolutional layer consists of 64 7×7 convolutional kernels with a stride of 2, which downsamples the input image by a factor of 0.5 and increases the number of channels in the feature map to 64. The maximum pooling layer uses a 3×3 window size with a stride of 2 to downsample the feature map by a factor of 0.5. Except for the head convolutional layer, every two convolutional layers using 3×3 convolutional kernels are directly connected to form a residual module, totaling 8 residual modules. The first convolutional layer in each residual block has a stride of 2, and the remaining ones have a stride of 1. The number of convolutional kernels increases with the depth of the network, ultimately extracting a high-dimensional feature map with a size of 1 / 32 of the input image and a channel number increased to 512. The network weights are randomly initialized and continuously updated through backpropagation during training.

[0058] As attached Figure 3 As shown, the high-dimensional feature map F∈R extracted by the convolutional neural network C×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively, and are input into three groups of 1×1 convolutional layers to obtain three new feature maps F a 、F b 、F c , and flatten its width and height dimensions, that is, {F a ,F b ,F c}∈R C×(H×W) Then, F a The transpose of F b After multiplication, the spatial attention matrix M is obtained through the Softmax function s ∈R (H×W)×(H×W) , specifically

[0059]

[0060] in, It represents the correlation between the feature pixels at the i-th position and the j-th position.

[0061] Make F c With M s Multiply and combine with the original feature F∈R C×H×W Add element by element to obtain the output feature F s , specifically

[0062]

[0063] where η s is a trainable scale factor and is initialized to 0 to avoid Too big. s Selectively aggregating according to the spatial attention matrix allows strongly correlated features to promote each other, improving intra-class compactness and semantic consistency, and enabling the network to better distinguish target images of different categories.

[0064] As attached Figure 4 As shown, the channel attention mechanism module is constructed, and the high-dimensional feature map F∈R extracted by the neural network C ×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively. After flattening its width and height dimensions, the channel attention matrix M is obtained by multiplying it with its own transpose. t ∈R C×C , specifically

[0065]

[0066] Make F and M t Multiply and combine with the original feature F∈R C×H×W Add element by element to obtain the output feature F t , specifically

[0067]

[0068] where η t is a trainable scale factor and is initialized to 0 to avoid Therefore, the feature F generated at each position t It is the result of weighted addition of features at all positions and original features. It simulates the dependency between feature map channels, enhances inter-class distinguishability and feature identifiability, and enables the network to highlight the feature representation of fine-grained changes in the target image.

[0069] like Figure 5As shown, in order to verify the beneficial effects of the deep convolutional network target recognition method with a dual-channel attention mechanism proposed in this invention, the following experiments were conducted:

[0070] An ablation experiment was conducted on the target recognition dataset for a deep convolutional network target recognition method with a dual-channel attention mechanism proposed in the present invention. Specifically, four structures, namely, a single-channel convolutional network, a convolutional neural network + sample pair construction, a convolutional neural network + sample pair construction + dual attention mechanism, and a convolutional neural network + sample pair construction + dual attention mechanism + loss, were used to perform single-sample visual target recognition experiments. Top-1 accuracy, Top-5 accuracy, and average F1 score were selected as evaluation indicators. The Top-1 accuracy is the accuracy rate that the first-ranked category is consistent with the actual result, and the Top-5 accuracy is the accuracy rate that the top five categories are consistent with the actual result. The definition of F1 score is specifically as follows:

[0071]

[0072] Among them, TP, FP, and FN are the number of positive samples judged as positive samples, negative samples judged as positive samples, and positive samples judged as negative samples, respectively, and N is the number of categories.

[0073] It can be observed that the convolutional neural network + sample pair construction + dual attention mechanism + loss of the present invention achieved the best recognition effect on both data sets. The convolutional neural network structure and the large-scale training samples required by it played a decisive role in improving the performance. The dual attention mechanism and loss also improved the results to a certain extent.

[0074] It should be noted that the above content merely illustrates the technical idea of ​​the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.

Claims

1. A deep convolutional network target recognition method with a dual-channel attention mechanism, characterized by: The following steps are included: Step 1: Construct a convolutional neural network, take image sample pairs as input, and extract high-dimensional feature maps; Step 2: Construct a spatial attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature pixels in the spatial dimension and adds them element-by-element with the original features; Step 3: Construct a channel attention mechanism module, which takes the two high-dimensional feature maps extracted by the neural network as input, calculates the correlation between feature channels in the channel dimension, and adds them element by element with the original features; Step 3-1: For the high-dimensional feature map F∈R extracted by the neural network described in step 1 C×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively. After flattening its width and height dimensions, the channel attention matrix M is obtained by multiplying it with its own transpose. t ∈R C×C , let i, j represent the i-th position and j-th position in space, T is the transposition operation, specifically Step 3-2: Make F and M t Multiply and combine with the original feature F∈R C×H×W Add element by element to obtain the output feature F t , specifically where η t is a trainable scale factor and is initialized to 0 to avoid Too large; Step 4: Stack the outputs of the spatial attention mechanism module and the channel attention mechanism module in the channel dimension to obtain the final feature representation of the model; Step 5: Construct training sample pairs. Targets of the same type are expanded through data augmentation, and targets of different types are directly paired. Step 6: Calculate the cross entropy loss and learn the network parameters through stochastic gradient descent to obtain a neural network model with the ability to distinguish target categories; Step 6-1: Let the neural network process the output of a pair of feature maps F1′, F2′∈R through the dual attention mechanism (2×C)×H×W After global average pooling, we get a pair of feature vectors f1,f2∈R (2×C) , we calculate the Euclidean distance between f1 and f2, and map it to the range of [0,1] through the Sigmoid function to obtain the final output y of the neural network i , then define the cross entropy loss function loss, specifically as Where i represents the i-th pair of output features; y i , is the output of the neural network; Step 6-2: Using loss as the loss function and the sample pairs described in step 5 as input, the adaptive moment estimation algorithm is used to train the neural network described in steps 1 to 4. The first-order moment estimation and second-order moment estimation of the gradient are used to dynamically adjust the learning rate of each parameter. The weight decay of the adaptive moment estimation algorithm is set to 5e -5 , 32 samples are input as a small batch, and the learning rate is initialized to 4e -3 , it decays to half of its original value every 40 iterations, and a total of 200 iterations are performed to obtain a neural network model with the ability to distinguish target categories.

2. The method for object recognition using a deep convolutional network with a dual-channel attention mechanism according to claim 1, characterized in that: The step 1 specifically includes: Step 1-1: Construct a convolutional neural network, which includes 17 convolutional layers. The head convolutional layer consists of 64 7×7 convolution kernels with a stride of 2, which downsamples the input image by 0.5 times and increases the number of feature map channels to 64 dimensions. The maximum pooling layer uses a 3×3 window size and a stride of 2 to downsample the feature map by 0.5 times. Except for the head convolutional layer, every two convolutional layers using 3×3 convolution kernels form a residual module with a direct connection structure, totaling 8 residual modules. The first convolution layer in each residual block has a stride of 2, and the rest have a stride of 1. The number of convolution kernels increases with the depth of the network. Finally, a high-dimensional feature map with a size of 1 / 32 of the input image and a channel number increased to 512 dimensions is extracted. The network weights are obtained by random initialization and are continuously updated through backpropagation during training. Step 1-2: Construct two identical convolutional neural networks as described in step 1-1. Each convolutional neural network receives one image in the image sample pair as input and outputs high-dimensional feature maps F1 and F2 respectively.

3. The method for object recognition using a deep convolutional network with a dual-channel attention mechanism according to claim 2, wherein: The step 2 specifically includes: Step 2-1: For the original high-dimensional feature map F∈R extracted by the convolutional neural network described in step 1 C×H×W , where H, W, and C represent the height, width, and number of channels of the feature map, respectively, and are input into three groups of 1×1 convolutional layers to obtain three new feature maps F a 、F b 、F c , and flatten its width and height dimensions, that is, {F a ,F b ,F c }∈R C×(H×W) ; Then, F a The transpose of F b After multiplication, the spatial attention matrix M is obtained through the Softmax function s ∈R (H×W)×(H×W) , specifically in, Indicates the correlation between the feature pixels at the i-th position and the j-th position; T represents transposition, F a 、F b It is the feature map output by the convolutional layer; Step 2-2: Make F c With M s Multiply and combine with the original high-dimensional feature F∈R C×H×W Add element by element to obtain the output feature F s , specifically where η s is a trainable scale factor and is initialized to 0 to avoid Too large, j is the subscript of the spatial position.

4. The method for object recognition using a deep convolutional network with a dual-channel attention mechanism according to claim 3, wherein: Said step 4 also includes, Step 4-1: For the attention mechanism modules described in step 2 and step 3, the F s With F t Stacked in the channel dimension to form a dual attention mechanism module; Step 4-2: For the two high-dimensional feature maps F1, F2∈R output by the neural network described in step 1 C×H×W , respectively pass it through the dual attention mechanism module to obtain better feature representation F1′,F2′∈R (2×C)×H×W .

5. The method for object recognition using a deep convolutional network with a dual-channel attention mechanism according to claim 4, wherein: Said step 5 further comprises, Step 5-1: For different target category images, directly form sample pairs for training the network; Step 5-2: For images of the same target category, random scale scaling, rotation, affine transformation, and brightness, saturation, and contrast adjustments are performed to make the two images in each pair of the same sample different and then form sample pairs, so that the number of sample pairs of target images of the same category and the number of sample pairs of target images of different categories are consistent; there are a total of E categories of visual targets to be identified, and the single sample training set contains E target images. After constructing sample pairs according to the method described in step 5, the total number of sample pairs N pairs for Where n is the independent variable.

Images