Residual network image classification method based on convolution axis attention mechanism

Abstract

The invention discloses a residual network image classification method based on a convolutional axis attention mechanism, and the method comprises the following steps: 1, selecting a training sample set and a test sample set, carrying out the preprocessing of images in the training sample set and the test sample set, H representing the height of the image, and W representing the width of the image; step 2, constructing a CANet network, creating a randomly initialized input image M belonging to R3 * H * W, and transmitting the input image M to the CANet network to execute forward propagation to obtain a classification result; step 3, inputting the training set image obtained in the step 1.2 into the CANet network constructed in the step 2 for training; and step 4, classifying the test set images by using the trained CANet network to obtain image classification accuracy. According to the method, the parameter quantity can be effectively reduced on the premise of ensuring the image classification precision of the existing residual network ResNet.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing methods, and specifically relates to a residual network image classification method based on a convolutional axis attention mechanism (CAA). Background Art

[0002] Image classification aims to automatically classify images into predefined categories. It is a fundamental problem in the field of computer vision and is crucial to advanced visual tasks such as image detection and behavior analysis.

[0003] With the development of computer technology, Convolutional Neural Network (CNN) and attention mechanism have become key technologies to improve image classification performance. ResNet network is a very famous network structure in the field of image classification. It effectively solves the gradient vanishing problem in deep network training by introducing residual connections, and achieves remarkable results in image classification tasks by stacking more convolutional layers. However, the limited receptive field of convolution limits the performance of image classification networks, which makes it impossible for the network to capture enough image context information and detail information. In addition, with the increase in the number of parameters, ResNet needs to pay a higher computing resource cost in practical applications to achieve better image classification accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a residual network image classification method based on a convolutional axis attention mechanism, which solves the problem of a relatively high number of ResNet network parameters.

[0005] The technical solution adopted by the present invention is a residual network image classification method based on a convolution axis attention mechanism, comprising the following steps:

[0006] Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0007] Step 2: Construct CANet (Convolutional Axis Network) and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0008] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0009] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

[0010] The present invention is also characterized in that:

[0011] Step 1 is as follows:

[0012] Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively;

[0013] Step 1.2, preprocess the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels.

[0014] In step 2, the CANet network contains four modules, namely: low-level feature extraction module, feature enhancement module, feature fusion module and convolution axis attention mechanism CAA module.

[0015] Step 2 is as follows:

[0016] Step 2.1, the first module of the CANet network is a low-level feature extraction module. A low-level feature extraction module is constructed. The low-level feature extraction module contains a MaxPooling layer (maximum pooling layer), a residual connection and four convolutional layers. Each convolutional layer performs preliminary feature extraction, such as edges, corners and basic texture information, through convolution and normalization. The input image M is input into the low-level feature extraction module of the CANet network. First, the input image M passes through a 7×7 convolutional layer A, and then uses a MaxPooling layer with a size of 3×3 and a step size of 2 for downsampling. Then, it passes through a 1×1 convolutional layer B to reduce the dimension of the feature map to C, passes through a 3×3 convolutional layer D, and then passes through a 1×1 convolutional layer E to increase the dimension to 4C. The output feature map of the MaxPooling layer is residually connected with the output feature map of the convolutional layer E to obtain the feature map M. 1 ∈R 4C×H / 4×W / 4 , where C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map;

[0017] Step 2.2, the second module of the CANet network is the feature enhancement module. The feature enhancement module contains three convolutional layers and a residual connection. Each convolutional layer contains deep convolution and a larger convolution kernel. On the basis of low-level features, the expressive ability of features is further enhanced, so that the CANet network can deepen its understanding of image features. The feature map M obtained in step 2.1 is constructed. 1The input is fed into the first convolutional layer F of the feature enhancement module, and a 1×1 convolutional layer F is used to reduce the dimension of the input feature map to 2C, and then downsampled by a 3×3 convolutional layer G with a step size of 2, and finally a 1×1 convolutional layer H is used to increase the dimension of the feature map to 8C, and the input feature map M is converted into 1 Perform residual connection with the output feature map of convolution layer H to obtain feature map M 2 ∈R 8C×H / 8×W / 8 ;

[0018] Step 2.3, the third module of the CANet network is the feature fusion module. The feature fusion module is constructed. The feature fusion module includes three convolutional layers and a residual connection. Through convolution kernels of different sizes and residual connections, features of different scales are fused to provide richer information for the CANet network. The feature map M obtained in step 2.2 is 2 The input feature map is firstly reduced to 4C by the first 1×1 convolutional layer I, and the size remains unchanged. Then, it is downsampled by a 3×3 convolutional layer J with a step size of 2, and then it is increased to 16C by the second 1×1 convolutional layer N. 2 Perform residual connection with the output feature map of convolution layer N to obtain feature map M 3 ∈R 16C×H / 16×W / 16 ;

[0019] Step 2.4, the fourth module of the CANet network is the convolution axis attention mechanism module. The convolution axis attention mechanism module is constructed. The convolution axis attention mechanism module includes a convolution layer O, a CAA layer (convolution axis attention mechanism layer), a convolution layer P, an AvgPooling layer (average pooling layer) and a fully connected layer; among them, the CAA layer includes a CBAM layer and an AA layer; the CBAM layer includes a maximum pooling layer, an average pooling layer, two channel convolution layers, a spatial convolution layer and two Sigmoid layers; the AA layer includes a convolution layer W Q , a convolutional layer W K , a convolutional layer W V And two Softmax layers;

[0020] The convolution axis attention mechanism module applies to the feature map M 3 The processing process is as follows:

[0021] For a given feature map M 3 First, the feature map is reduced in dimension through a 1×1 convolution layer O to obtain the feature map M 4 ∈R 8C×H / 32×W / 32 , and then the feature map M 4Input into the CAA layer of the convolution axis attention mechanism module to perform attention operation on the feature map;

[0022] Feature map M 4 The CBAM layer in the CAA layer combines channel attention and spatial attention. The CBAM layer first performs spatial attention on the input feature map M. 4 Through the global maximum pooling layer and the global average pooling layer, two feature maps with 8C channels and a size of 1×1 are obtained. The two feature maps then pass through two channel convolution layers respectively. The first channel convolution layer reduces the number of channels from 8C to C / 2, and the second channel convolution layer increases the number of channels from C / 2 back to 8C. The two feature maps that pass through the two channel convolution layers are added in the channel dimension and passed through the Sigmoid layer to obtain a one-dimensional channel attention weight map Mc∈R 8C×1×1 , Mc and the input feature map M 4 Perform element-by-element multiplication to obtain the feature map X′∈R 8C×H / 32×W / 32 ; The CBAM layer then calculates the maximum and average values ​​of the feature map X′ in the channel dimension, and obtains two feature maps with 1 channel and a size of H / 32×W / 32. These two feature maps are concatenated in the channel dimension and passed through a spatial convolution layer to obtain a two-dimensional spatial attention weight map Ms∈R 1×H / 32×W / 32 , Ms is multiplied element-by-element with the feature map X′ through the Sigmoid activation function to obtain the final refined feature map X″∈R 8C×H / 32×W / 32 ;

[0023] Secondly, the feature map X″ is input into the AA layer as the input of the next stage. The AA layer decomposes the feature map X″ into horizontal axis features and vertical axis features; the convolution operation of Q (query), K (key) and V (value) is used to build attention. The feature map X″ passes through a convolution layer W Q , a convolutional layer W K and a convolutional layer W V To generate Q∈R 8C×H / 32×W / 32 , K∈R 8C ×H / 32×W / 32 and V∈R 8C×H / 32×W / 32, convert Q from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), b represents the batch size, convert K from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), convert V from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), then perform matrix multiplication on Q and K to obtain the row attention score, and normalize it through the Softmax layer, then perform matrix multiplication on the normalized row attention score and V to obtain the row attention weight map, add the row attention weight map to the input feature map X″, and obtain the row attention enhanced feature map Row∈R 8C ×H / 32×W / 32 ; Similar to row attention, the operation is performed on the column dimension, converting Q from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), where b represents the batch size, converting K from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), and converting V from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32). Calculate the dot product of Q and K by matrix multiplication to obtain the column attention score, normalize the column attention score by Softmax, and then perform matrix multiplication with V to obtain the column attention weight map, add the column attention weight map to the input feature map X″, and obtain the column attention enhanced feature map Col∈R 8C×H / 32×W / 32 ;

[0024] In this way, the model will pay more attention to the positions with higher weights when processing features. Then use bilinear interpolation to adjust the feature maps Row and Col of different axes back to their original sizes, merge the two feature maps, and weight them by parameters to obtain the final feature map M. 5 ∈R 8C×H / 32×W / 32 ; After a 1×1 convolution layer P, the feature map M is obtained 6 ∈R 32C×H / 32×W / 32 ;

[0025] Finally, the feature map M 6 Input to the AvgPooling layer, feature map M 6 The spatial dimensions (height and width) of each channel of will be reduced to 1×1, and the feature map M is obtained. 7 ∈R 32C×1×1 , then, the feature map M 7 Flatten it into a one-dimensional vector, and map the flattened one-dimensional feature vector to the final output category through a fully connected layer. Classify it according to the output of the fully connected layer to get the classification result.

[0026] In step 2.4, the attention process of the CBAM layer is summarized as formula (1) and formula (2) respectively:

[0027]

[0028] in Represents element-wise multiplication.

[0029] In step 2.4, the attention process of the AA layer is summarized as shown in formula (3) and formula (4):

[0030] Row=RowAttention(X″) (3)

[0031] Col=ColAttention(X″) (4).

[0032] In step 2.4, the feature map M 5 As shown in formula (5):

[0033] M 5 =Row+Col (5).

[0034] The specific operations of step 3 are:

[0035] The diversified training set images obtained in step 1.2 are input into the CANet network designed in step 2 for training. The standard cross entropy loss function L is used to calculate the difference between the probability distribution of the CANet network output category and the true label (i.e., the true label corresponding to each image in the training set image). Then, the CANet network is updated by the back propagation algorithm to minimize the average loss over the entire training. The standard cross entropy loss function L is shown in formula (6):

[0036]

[0037] Among them, c represents the category, K represents the total number of categories, and y c is the one-hot encoding of the true label of category c, p c is the predicted probability of the CANet network for category c.

[0038] The specific operations of step 4 are:

[0039] Step 4.1, input the test set images of the diversified images obtained in step 1.2 into the CANet network trained in step 3. These data are usually processed in batches to improve the efficiency of the prediction process;

[0040] Step 4.2: During the test in step 4.1, the accuracy function is used to calculate the Top-1 accuracy between the CANet network output and the target label after each iteration, as shown in formula (7):

[0041]

[0042] Among them, NCP is the number of images predicted correctly by the CANet network, and TNP is the total number of images predicted by the CANet network;

[0043] Compare the Top-1 accuracy obtained in the current iteration with the Top-1 highest accuracy recorded in the previous iteration. If the Top-1 accuracy of the current iteration is higher than the Top-1 highest accuracy recorded previously, update the Top-1 highest accuracy and save the state of the CANet network at this time. This state includes the CANet network architecture, CANet network parameters, the Top-1 highest accuracy, iteration, data loading settings, optimizer state and learning rate scheduler state to ensure that the CANet network has good generalization performance in practical applications. Otherwise, proceed directly to the next step.

[0044] Step 4.3, after obtaining the current highest accuracy Top-1 in step 4.2, if the specified number of training cycles is not reached, return to step 3 to continue iterating. When the CANet network completes the specified number of training cycles, the iteration stops and the final accuracy Top-1 is obtained.

[0045] The beneficial effects of the present invention are:

[0046] (1) Compared with other image classification methods based on ResNet network, the residual network image classification method (CANet) based on convolutional axis attention mechanism (CAA) of the present invention has significantly reduced network parameters when the classification accuracy is comparable to that of other image classification methods based on ResNet network, which effectively solves the problem of relatively high number of ResNet network parameters and improves the feasibility and efficiency of ResNet network in image classification applications.

[0047] (2) The method of the present invention firstly improves the learning ability and generalization ability of the CANet network by using three residual blocks of different sizes; and by fusing the CBAM mechanism and the AA mechanism to form a CAA mechanism, and embedding the convolution axis attention mechanism module containing the CAA mechanism into the CANet network, so that the CANet network can capture richer and more detailed features of the image from multiple angles, and concentrate computing resources to capture the key features of the image, thereby reducing the number of parameters of the CANet network while ensuring the feature extraction capability of the CANet network. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a framework diagram of the residual network image classification method based on the convolution axis attention mechanism of the present invention;

[0049] Figure 2 The CANet network structure diagram of the residual network image classification method based on the convolution axis attention mechanism of the present invention;

[0050] Figure 3 This is a structural diagram of the CAA layer in the residual network image classification method based on the convolutional axis attention mechanism of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The present invention provides a residual network image classification method based on a convolutional axis attention mechanism, comprising the following steps:

[0053] Step 1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively, and perform preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0054] The specific process of step 1 is:

[0055] Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively;

[0056] Step 1.2, preprocessing operations are performed on the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization operations, and random cropping to ensure the consistency of the size and number of channels of the input image, thereby obtaining diversified training set images and test set images.

[0057] Step 2: Build the CANet network. The CANet network consists of four modules: low-level feature extraction module, feature enhancement module, feature fusion module, and convolution axis attention mechanism (CAA) module. Create a randomly initialized input image M∈R 3 ×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0058] The specific process of step 2 is:

[0059] Step 2.1, the first module of the CANet network is a low-level feature extraction module. A low-level feature extraction module is constructed. The low-level feature extraction module contains a MaxPooling layer (maximum pooling layer), a residual connection and four convolutional layers. Each convolutional layer performs preliminary feature extraction, such as edges, corners and basic texture information, through convolution and normalization. The input image M is input into the low-level feature extraction module of the CANet network. First, the input image M passes through a 7×7 convolutional layer A, and then uses a MaxPooling layer with a size of 3×3 and a step size of 2 for downsampling. Then, it passes through a 1×1 convolutional layer B to reduce the dimension of the feature map to C, passes through a 3×3 convolutional layer D, and then passes through a 1×1 convolutional layer E to increase the dimension to 4C. The output feature map of the MaxPooling layer is residually connected with the output feature map of the convolutional layer E to obtain the feature map M. 1 ∈R 4C×H / 4×W / 4 , where C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map;

[0060] Step 2.2, the second module of the CANet network is the feature enhancement module. The feature enhancement module contains three convolutional layers and a residual connection. Each convolutional layer contains deep convolution and a larger convolution kernel. On the basis of low-level features, the expressive ability of features is further enhanced, so that the CANet network can deepen its understanding of image features. The feature map M obtained in step 2.1 is constructed. 1 The input is fed into the first convolutional layer F of the feature enhancement module, and a 1×1 convolutional layer F is used to reduce the dimension of the input feature map to 2C, and then downsampled by a 3×3 convolutional layer G with a step size of 2, and finally a 1×1 convolutional layer H is used to increase the dimension of the feature map to 8C, and the input feature map M is converted into 1 Perform residual connection with the output feature map of convolution layer H to obtain feature map M 2 ∈R 8C×H / 8×W / 8 ;

[0061] Step 2.3, the third module of the CANet network is the feature fusion module. The feature fusion module is constructed. The feature fusion module includes three convolutional layers and a residual connection. Through convolution kernels of different sizes and residual connections, features of different scales are fused to provide richer information for the CANet network. The feature map M obtained in step 2.2 is 2 The input feature map is firstly reduced to 4C by the first 1×1 convolutional layer I, and the size remains unchanged. Then, it is downsampled by a 3×3 convolutional layer J with a step size of 2, and then it is increased to 16C by the second 1×1 convolutional layer N. 2Perform residual connection with the output feature map of convolution layer N to obtain feature map M 3 ∈R 16C×H / 16×W / 16 ;

[0062] Step 2.4, the fourth module of the CANet network is the convolution axis attention mechanism module. The convolution axis attention mechanism module is constructed. The convolution axis attention mechanism module includes a convolution layer O, a CAA layer (convolution axis attention mechanism layer), a convolution layer P, an AvgPooling layer (average pooling layer) and a fully connected layer; among them, the CAA layer includes a CBAM layer and an AA layer; the CBAM layer includes a maximum pooling layer, an average pooling layer, two channel convolution layers, a spatial convolution layer and two Sigmoid layers; the AA layer includes a convolution layer W Q , a convolutional layer W K , a convolutional layer W V And two Softmax layers;

[0063] The first layer of the convolution axis attention mechanism module is the convolution layer O, and the output of the convolution layer O is input into the CAA layer. The CAA layer includes the CBAM layer and the AA layer. In the CBAM layer, the feature map passes through a maximum pooling layer and an average pooling layer to obtain two feature maps, and then passes through two channel convolution layers and a Sigmoid layer to obtain a one-dimensional channel attention weight map, which is multiplied element-by-element with the input feature map to obtain a channel attention feature map; the CBAM layer then calculates the maximum value and average value of the channel attention feature map in the channel dimension to obtain two feature maps, which are concatenated in the channel dimension, and pass through a spatial convolution layer to obtain a two-dimensional spatial attention weight map, which is multiplied element-by-element with the feature map through a Sigmoid layer to obtain the final refined feature map; the output of the CBAM layer is used as the input of the AA layer, and the feature map will pass through a convolution layer W respectively. Q , a convolutional layer W K and a convolutional layer W VGenerate Q, K and V accordingly, perform matrix multiplication on Q and K to get the row attention score, pass through the Softmax layer, perform matrix multiplication with V to get the row attention weight map, add it to the input feature map, and get the feature map enhanced by row attention. Similar to row attention, the operation is performed on the column dimension, and the dot product of Q and K is calculated by matrix multiplication to get the column attention score. The column attention score is normalized by Softmax, and then matrix multiplication is performed with V to get the column attention weight map. The column attention weight map is added to the input feature map X″ to get the feature map enhanced by column attention. Then use bilinear interpolation to adjust the feature maps of different axes back to the original size, merge the two feature maps, and get the final feature map; then input the feature map into the convolutional layer P, and finally pass the AvgPooling layer and the fully connected layer to get the classification result. The convolutional axis attention mechanism module uses the CAA mechanism, which is the convolutional attention mechanism (ConvolutionalBlock Attention The fusion of Convolutional Block Module (CBAM) and Axial-attention (AA) mechanism performs convolution and attention operations, enabling the network to more effectively focus on the most important features in the image and obtain richer and more detailed feature representations.

[0064] The convolution axis attention mechanism module applies to the feature map M 3 The processing process is as follows:

[0065] For a given feature map M 3 First, the feature map is reduced in dimension through a 1×1 convolution layer O to obtain the feature map M 4 ∈R 8C×H / 32×W / 32 , and then the feature map M 4 Input into the CAA layer of the convolution axis attention mechanism module to perform attention operation on the feature map;

[0066] Feature map M 4 The CBAM layer in the CAA layer combines channel attention and spatial attention. The CBAM layer first performs spatial attention on the input feature map M. 4 Through the global maximum pooling layer and the global average pooling layer, two feature maps with 8C channels and a size of 1×1 are obtained. The two feature maps then pass through two channel convolution layers respectively. The first channel convolution layer reduces the number of channels from 8C to C / 2, and the second channel convolution layer increases the number of channels from C / 2 back to 8C. The two feature maps that pass through the two channel convolution layers are added in the channel dimension and passed through the Sigmoid layer to obtain a one-dimensional channel attention weight map Mc∈R 8C×1×1 , Mc and the input feature map M 4 Perform element-by-element multiplication to obtain the feature map X′∈R 8C×H / 32×W / 32; The CBAM layer then calculates the maximum and average values ​​of the feature map X′ in the channel dimension, and obtains two feature maps with 1 channel and a size of H / 32×W / 32. These two feature maps are concatenated in the channel dimension and passed through a spatial convolution layer to obtain a two-dimensional spatial attention weight map Ms∈R 1×H / 32×W / 32 , Ms is multiplied element-by-element with the feature map X′ through the Sigmoid activation function to obtain the final refined feature map X″∈R 8C×H / 32×W / 32 ; Among them, the attention process of the CBAM layer is summarized as formula (1) and formula (2) respectively:

[0067]

[0068] in represents element-wise multiplication;

[0069] Secondly, the feature map X″ is input into the AA layer as the input of the next stage. The AA layer decomposes the feature map X″ into horizontal axis features and vertical axis features; the convolution operation of Q (query), K (key) and V (value) is used to build attention. The feature map X″ passes through a convolution layer W Q , a convolutional layer W K and a convolutional layer W V To generate Q∈R 8C×H / 32×W / 32 , K∈R 8C ×H / 32×W / 32 and V∈R 8C×H / 32×W / 32 , convert Q from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), b represents the batch size, convert K from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), convert V from (b, 8C, H / 32, W / 32) to (b× H / 32, -1, W / 32), then perform matrix multiplication on Q and K to obtain the row attention score, and normalize it through the Softmax layer, then perform matrix multiplication on the normalized row attention score and V to obtain the row attention weight map, add the row attention weight map to the input feature map X″, and obtain the row attention enhanced feature map Row∈R 8C ×H / 32×W / 32; Similar to row attention, the operation is performed on the column dimension, converting Q from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), where b represents the batch size, converting K from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), and converting V from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32). Calculate the dot product of Q and K by matrix multiplication to obtain the column attention score, normalize the column attention score by Softmax, and then perform matrix multiplication with V to obtain the column attention weight map, add the column attention weight map to the input feature map X″, and obtain the column attention enhanced feature map Col∈R 8C×H / 32×W / 32 ;

[0070] The attention process of the AA layer is summarized as shown in formula (3) and formula (4) respectively:

[0071] Row=RowAttention(X″) (3)

[0072] Col=ColAttention(X″) (4)

[0073] In this way, the model will pay more attention to the positions with higher weights when processing features. Then use bilinear interpolation to adjust the feature maps Row and Col of different axes back to their original sizes, merge the two feature maps, and weight them by parameters to obtain the final feature map M. 5 ∈R 8C×H / 32× W / 32 , as shown in formula (5):

[0074] M 5 =Row+Col (5)

[0075] After a 1×1 convolution layer P, the feature map M is obtained. 6 ∈R 32C×H / 32×W / 32 ;

[0076] Finally, the feature map M 6 Input to the AvgPooling layer, feature map M 6 The spatial dimensions (height and width) of each channel of will be reduced to 1×1, and the feature map M is obtained. 7 ∈R 32C×1×1 , then, the feature map M 7 Flatten it into a one-dimensional vector, and map the flattened one-dimensional feature vector to the final output category through a fully connected layer. Classify it according to the output of the fully connected layer to get the classification result.

[0077] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0078] The specific operations of step 3 are:

[0079] The diversified training set images obtained in step 1.2 are input into the CANet network designed in step 2 for training. The standard cross entropy loss function L is used to calculate the difference between the probability distribution of the CANet network output category and the true label (i.e., the true label corresponding to each image in the training set image). Then, the CANet network is updated by the back propagation algorithm to minimize the average loss over the entire training. The standard cross entropy loss function L is shown in formula (6):

[0080]

[0081] Among them, c represents the category, K represents the total number of categories, and y c is the one-hot encoding of the true label of category c (if the image prediction result belongs to category c, then y c =1, otherwise y c =0), p c is the predicted probability of the CANet network for category c.

[0082] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy;

[0083] The specific operations of step 4 are:

[0084] Step 4.1, input the test set images of the diversified images obtained in step 1.2 into the CANet network trained in step 3. These data are usually processed in batches to improve the efficiency of the prediction process;

[0085] Step 4.2: During the test in step 4.1, the accuracy function is used to calculate the Top-1 accuracy between the CANet network output and the target label after each iteration, as shown in formula (7):

[0086]

[0087] Among them, NCP is the number of images predicted correctly by the CANet network, and TNP is the total number of images predicted by the CANet network;

[0088] Compare the Top-1 accuracy obtained in the current iteration with the Top-1 highest accuracy recorded in the previous iteration. If the Top-1 accuracy of the current iteration is higher than the Top-1 highest accuracy recorded previously, update the Top-1 highest accuracy and save the state of the CANet network at this time. This state includes the CANet network architecture, CANet network parameters, the Top-1 highest accuracy, iteration, data loading settings, optimizer state and learning rate scheduler state to ensure that the CANet network has good generalization performance in practical applications. Otherwise, proceed directly to the next step.

[0089] Step 4.3, after obtaining the current highest accuracy Top-1 in step 4.2, if the specified number of training cycles is not reached, return to step 3 to continue iterating. When the CANet network completes the specified number of training cycles, the iteration stops and the final accuracy Top-1 is obtained.

[0090] Example 1

[0091] CIFAR 10 is a commonly used image classification dataset. Each image is a color picture of size 32×32 and 3 channels. During training, each image is resized to 224×224. The dataset contains 10 different categories, each with 60,000 images, 50,000 of which are used for training and 10,000 for testing. The experiment uses a 3090 graphics card for training, and the batch size is set to 128. This method is implemented using the Pytorch library and the Pytorch visual model library (timm). The initial learning rate is set to 5e-4, the minimum learning rate is set to 5e-6, and the weight decay is set to 0.05. The AdamW optimizer is used, and the learning rate is adjusted using cosine annealing. A total of 320 epochs are trained.

[0092] The residual network image classification method based on the convolution axis attention mechanism is shown in the following figure. Figure 1 As shown, the specific steps include:

[0093] Step 1, select 50,000 and 10,000 images of size 32×32×3 as the training sample set and the test sample set respectively, and perform preprocessing operations on the images in the training sample set and the test sample set;

[0094] The specific process of step 1 is:

[0095] Step 1.1, select 50,000 and 10,000 images of size 32×32×3 as the training sample set and the test sample set respectively;

[0096] Step 1.2, preprocess the images in the training sample set and test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels, thereby obtaining diversified training set and test set images.

[0097] Step 2: Build CANet (Convolutional Axis Network). The structure of CANet network is as follows: Figure 2 As shown in Figure 2, the CANet network consists of four modules: low-level feature extraction module, feature enhancement module, feature fusion module, and convolution axis attention mechanism (CAA) module. It creates a randomly initialized input image M∈R 3×224×224 , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0098] The specific process of step 2 is:

[0099] Step 2.1, the first module of the CANet network is a low-level feature extraction module. The low-level feature extraction module is constructed. The low-level feature extraction module contains a maximum pooling layer (MaxPooling layer), a residual connection and four convolutional layers. Each convolutional layer performs preliminary feature extraction, such as edges, corners and basic texture information, through convolution and normalization. The input image M is input into the low-level feature extraction module of the CANet network. First, the input image M passes through a 7×7 convolutional layer A, and then uses a MaxPooling layer with a size of 3×3 and a step size of 2 for downsampling. Then, it passes through a 1×1 convolutional layer B to reduce the dimension of the feature map to 64, passes through a 3×3 convolutional layer D, and then passes through a 1×1 convolutional layer E to increase the dimension to 256. The output feature map of the MaxPooling layer is residually connected with the output feature map of the convolutional layer E to obtain the feature map M. 1 ∈R 256×56×56 ;

[0100] Step 2.2, the second module of the CANet network is the feature enhancement module. The feature enhancement module contains three convolutional layers and a residual connection. Each convolutional layer contains deep convolution and a larger convolution kernel. On the basis of low-level features, the expressive ability of features is further enhanced, so that the CANet network can deepen its understanding of image features. The feature map M obtained in step 2.1 is constructed. 1The input is fed into the first convolutional layer of the feature enhancement module, and a 1×1 convolutional layer F is used to reduce the dimension of the input feature map to 128 and a size of 56×56. Then, a 3×3 convolutional layer G with a step size of 2 is used for downsampling. Finally, a 1×1 convolutional layer H is used to increase the dimension of the feature map to 512, and the input feature map M is used. 1 Perform residual connection with the output feature map of convolution layer H to obtain feature map M 2 ∈R 512×28×28 ;

[0101] Step 2.3, the third module of the CANet network is the feature fusion module. The feature fusion module is constructed. The feature fusion module includes three convolutional layers and a residual connection. Through convolution kernels of different sizes and residual connections, features of different scales are fused to provide richer information for the CANet network. The feature map M obtained in step 2.2 is 2 The input feature map is firstly reduced to 256 and 28×28 by the first 1×1 convolution layer I. Then, it is downsampled by a 3×3 convolution layer J with a step size of 2. Then, it is increased to 1024 by the second 1×1 convolution layer N. 2 The output feature map of the last convolutional layer N is residually connected to obtain the feature map M 3 ∈R 1024×14×14 ;

[0102] Step 2.4, the fourth module of the CANet network is the convolution axis attention mechanism module. The convolution axis attention mechanism module is constructed. The convolution axis attention mechanism module includes a convolution layer O, a CAA layer (convolution axis attention mechanism layer), a convolution layer P, an AvgPooling layer (average pooling layer) and a fully connected layer; among them, the CAA layer includes a CBAM layer and an AA layer; the CBAM layer includes a maximum pooling layer, an average pooling layer, two channel convolution layers, a spatial convolution layer and two Sigmoid layers; the AA layer includes a convolution layer W Q , a convolutional layer W K , a convolutional layer W V And two Softmax layers.

[0103] First, the first layer of the convolution axis attention mechanism module is the convolution layer O, and then the output of the convolution layer O is input into the CAA layer, such as Figure 3As shown in the figure, the CAA layer includes the CBAM layer and the AA layer. In the CBAM layer, the feature map passes through a maximum pooling layer and an average pooling layer to obtain two feature maps, and then passes through two channel convolution layers and a Sigmoid layer to obtain a one-dimensional channel attention weight map, which is multiplied element-by-element with the input feature map to obtain a channel attention feature map; the CBAM layer then calculates the maximum value and average value of the channel attention feature map in the channel dimension to obtain two feature maps, which are concatenated in the channel dimension, and pass through a spatial convolution layer to obtain a two-dimensional spatial attention weight map, which is multiplied element-by-element with the feature map through a Sigmoid layer to obtain the final refined feature map. The output of the CBAM layer is used as the input of the AA layer, and the feature map passes through a convolution layer W. Q , a convolutional layer W K and a convolutional layer W V Generate Q, K and V, perform matrix multiplication on Q and K to obtain the row attention score, pass through the Softmax layer, perform matrix multiplication with V to obtain the row attention weight map, add it to the input feature map, and obtain the feature map enhanced by row attention. Similar to row attention, the operation is performed on the column dimension, and the dot product of Q and K is calculated by matrix multiplication to obtain the column attention score. The column attention score is normalized by Softmax, and then matrix multiplication is performed with V to obtain the column attention weight map. The column attention weight map is added to the input feature map X″ to obtain the feature map enhanced by column attention. Then use bilinear interpolation to adjust the feature maps of different axes back to the original size, and merge the two feature maps to obtain the final feature map. Then input the feature map into the convolutional layer P, and finally pass the AvgPooling layer and the fully connected layer to obtain the classification result. The convolution axis attention mechanism module uses the CAA mechanism, which is the convolutional attention mechanism (Convolutional Block Attention The fusion of Convolutional Block Module (CBAM) and Axial-attention (AA) mechanism performs convolution and attention operations, enabling the network to more effectively focus on the most important features in the image and obtain richer and more detailed feature representations.

[0104] The convolution axis attention mechanism module applies to the feature map M 3 The processing process is as follows:

[0105] For a given feature map M 3 First, the feature map is reduced in dimension through a 1×1 convolution layer O to obtain the feature map M 4 ∈R 512×7×7 , and then the feature map M 4 Input into the CAA layer of the convolution axis attention mechanism module to perform attention operation on the feature map.

[0106] Feature map M4 The CBAM layer in the CAA layer combines channel attention and spatial attention. The CBAM layer first performs spatial attention on the input feature map M. 4 Through global maximum pooling and global average pooling, two feature maps with a channel number of 512 and a size of 1×1 are obtained. The two feature maps are then passed through two channel convolution layers respectively. The first channel convolution layer reduces the number of channels from 512 to 32, and the second channel convolution layer increases the number of channels from 32 back to 512. The two feature maps that have passed through the two channel convolution layers are added in the channel dimension and passed through the Sigmoid layer to obtain a one-dimensional channel attention weight map Mc∈R 512×1×1 , Mc and the input feature map M 4 Perform element-by-element multiplication to obtain the feature map X′∈R 512×7×7 ; The CBAM layer then calculates the maximum and average values ​​of the feature map X′ in the channel dimension, and obtains two feature maps with a channel number of 1 and a size of 7×7. These two feature maps are concatenated in the channel dimension and passed through a spatial convolution layer to obtain a two-dimensional spatial attention weight map Ms∈R 1×7×7 , Ms is multiplied element-by-element with the feature map X′ through the Sigmoid activation function to obtain the final refined feature map X″∈R 512×7×7 The attention process of the CBAM layer can be summarized as formula (1) and formula (2) respectively:

[0107]

[0108] in Represents element-wise multiplication.

[0109] Secondly, the feature map X″ is input into the AA layer as the input of the next stage, and the AA layer decomposes the feature map X″ into horizontal axis features and vertical axis features. Using the convolution operation of query (Q), key (K), and value (V) to build attention, the feature map X″ will pass through three convolutional layers to generate Q∈R 512×7×7 , K∈R 512×7×7 and V∈R 512×7×7 , convert Q from (128, 512, 7, 7) to (128×7, -1, 7) shape, 128 represents the batch size, convert K from (128, 512, 7, 7) to (128×7, -1, 7) shape, convert V from (128, 512, 7, 7) to (128×7, -1, 7) shape, then perform matrix multiplication on Q and K to obtain the row attention score, and normalize it through the Softmax layer, then perform matrix multiplication on the normalized row attention score and V to obtain the row attention weight map, the row attention weight map is added to the input feature map X″, and the row attention enhanced feature map Row∈R is obtained. 512×7×7; Similar to row attention, the operation is performed on the column dimension, converting Q from (128, 512, 7, 7) to (128×7, -1, 7) shape, b represents the batch size, converting K from (128, 512, 7, 7) to (128×7, -1, 7) shape, converting V from (128, 512, 7, 7) to (128×7, -1, 7) shape, using matrix multiplication to calculate the dot product of Q and K to obtain the column attention score, normalizing the column attention score through Softmax, and then performing matrix multiplication with V to obtain the column attention weight map, adding the column attention weight map to the input feature map X″ to obtain the column attention enhanced feature map Col∈R 512×7×7 The attention process of the AA layer can be summarized as shown in formula (3) and formula (4) respectively:

[0110] Row=RowAttention(X″) (3)

[0111] Col=ColAttention(X″) (4)

[0112] In this way, the model will pay more attention to the positions with higher weights when processing features. Then use bilinear interpolation to adjust the feature maps Row and Col of different axes back to their original sizes, merge the two feature maps, and weight them by parameters to obtain the final feature map M. 5 ∈R 512×7×7 , as shown in formula (5):

[0113] M 5 =Row+Col (5)

[0114] After a 1×1 convolution layer P, the feature map M is obtained. 6 ∈R 2048×7×7 .

[0115] Finally, the feature map M 6 Input to the AvgPooling layer, feature map M 6 The spatial dimensions (height and width) of each channel of will be reduced to 1×1, and the feature map M is obtained. 7 ∈R 2048×1×1 , then, the feature map M 7 Flatten it into a one-dimensional vector, and map the flattened one-dimensional feature vector to the final output category through a fully connected layer. Classify it according to the output of the fully connected layer to get the classification result.

[0116] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0117] The specific operations of step 3 are:

[0118] The diversified training set images obtained in step 1.2 are input into the CANet network designed in step 2 for training. The standard cross entropy loss function L is used to calculate the difference between the probability distribution of the CANet network output category and the true label (i.e., the true label corresponding to each image in the training set image). Then, the CANet network is updated through the back propagation algorithm to minimize the average loss over the entire training. The standard cross entropy loss function L is shown in formula (6):

[0119]

[0120] Where c represents the category, K represents the total number of categories, and yc is the one-hot encoding of the true label of category c (if the image prediction result belongs to category c, then y c =1, otherwise y c =0), pc is the predicted probability of the CANet network for category c.

[0121] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy;

[0122] The specific operations of step 4 are:

[0123] Step 4.1, input the test set images of the diversified images obtained in step 1.2 into the CANet network trained in step 3. These data are usually processed in batches to improve the efficiency of the prediction process;

[0124] Step 4.2: During the test in step 4.1, the accuracy function is used to calculate the Top-1 accuracy between the CANet network output and the target label after each iteration, as shown in formula (7):

[0125]

[0126] Among them, NCP is the number of images predicted correctly by the CANet network, and TNP is the total number of images predicted by the CANet network.

[0127] Compare the Top-1 accuracy obtained in the current iteration with the Top-1 highest accuracy recorded in the previous iteration. If the Top-1 accuracy of the current iteration is higher than the Top-1 highest accuracy recorded previously, update the Top-1 highest accuracy and save the state of the CANet network at this time. This state includes the CANet network architecture, CANet network parameters, the Top-1 highest accuracy, iteration, data loading settings, optimizer state, and learning rate scheduler state to ensure that the CANet network has good generalization performance in practical applications. Otherwise, proceed directly to the next step.

[0128] Step 4.3, after obtaining the current highest accuracy Top-1 in step 4.2, if the specified number of training cycles is not reached, return to step 3 to continue iterating. When the CANet network completes the specified number of training cycles, the iteration stops and the final accuracy Top-1 is obtained.

[0129] Performance comparison:

[0130] In order to verify the feasibility of the CANet network of the method of the present invention, image classification methods with ResNet network as the backbone network are selected for comparison, including ResNet-50, ResNet-101, ResNet-152 and the CANet method proposed in this paper. The accuracy of these methods on the CIFAR 10 validation set is sorted out, and the parameter indicators include: parameter quantity (Params (M)), computational complexity (FLOPs (G)) and evaluation accuracy (Top-1 Acc. (%)).

[0131] It can be seen from Table 1 that the CANet network of the method of the present invention has a classification accuracy (96.37%) comparable to that of ResNet and its variant networks, but the number of parameters is 21.28M, which is lower than that of ResNet and its variant networks.

[0132] Table 1 Experimental results of various methods on CIFAR 10 dataset

[0133] Model Backbone ImageSize Params(M) FLOPs(G) Top-1 Acc.(%) ResNet-50 ResNet <![CDATA[224 2 ]]> 23.53 4.13 96.17 ResNet-101 ResNet <![CDATA[224 2 ]]> 42.52 7.86 96.43 ResNet-152 ResNet <![CDATA[224 2 ]]> 58.16 11.6 97.21 CANet ResNet <![CDATA[224 2 ]]> 21.28 4.25 96.37

[0134] The residual network image classification method (CANet) based on the convolutional axis attention mechanism (CAA) of the present invention has significantly reduced network parameters compared with other image classification methods based on the ResNet network while achieving comparable classification accuracy, which effectively solves the problem of the relatively high number of ResNet network parameters and improves the feasibility and efficiency of the ResNet network in image classification applications.

[0135] Example 2

[0136] The residual network image classification method based on the convolution axis attention mechanism includes the following steps:

[0137] Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0138] Step 2: Build the CANet network and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0139] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0140] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

[0141] Example 3

[0142] The residual network image classification method based on the convolution axis attention mechanism includes the following steps:

[0143] Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0144] Step 1 is as follows:

[0145] Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively;

[0146] Step 1.2, preprocess the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels.

[0147] Step 2: Build the CANet network and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0148] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0149] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

[0150] Example 4

[0151] The residual network image classification method based on the convolution axis attention mechanism includes the following steps:

[0152] Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0153] Step 1 is as follows:

[0154] Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively;

[0155] Step 1.2, preprocess the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels.

[0156] Step 2: Build the CANet network and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0157] In step 2, the CANet network contains four modules, namely: low-level feature extraction module, feature enhancement module, feature fusion module and convolution axis attention mechanism CAA module.

[0158] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0159] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

[0160] Example 5

[0161] The residual network image classification method based on the convolution axis attention mechanism includes the following steps:

[0162] Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image;

[0163] Step 1 is as follows:

[0164] Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively;

[0165] Step 1.2, preprocess the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels.

[0166] Step 2: Build the CANet network and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result;

[0167] In step 2, the CANet network contains four modules, namely: low-level feature extraction module, feature enhancement module, feature fusion module and convolution axis attention mechanism CAA module.

[0168] Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training;

[0169] The specific operations of step 3 are:

[0170] The diversified training set images obtained in step 1.2 are input into the CANet network designed in step 2 for training. The standard cross entropy loss function L is used to calculate the difference between the probability distribution of the CANet network output category and the true label. Then, the CANet network is updated through the back propagation algorithm to minimize the average loss over the entire training. The standard cross entropy loss function L is shown in formula (6):

[0171]

[0172] Among them, c represents the category, K represents the total number of categories, and y c is the one-hot encoding of the true label of category c, p c is the predicted probability of the CANet network for category c.

[0173] Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

[0174] Example 6

[0175] Except for the following contents, other contents are the same as those in Example 5;

[0176] The specific operations of step 4 are:

[0177] Step 4.1, input the test set images of the diversified images obtained in step 1.2 into the CANet network trained in step 3;

[0178] Step 4.2: During the test in step 4.1, the accuracy function is used to calculate the Top-1 accuracy between the CANet network output and the target label after each iteration, as shown in formula (7):

[0179]

[0180] Among them, NCP is the number of images predicted correctly by the CANet network, and TNP is the total number of images predicted by the CANet network;

[0181] Compare the Top-1 accuracy obtained in the current iteration with the Top-1 highest accuracy recorded in the previous iteration. If the Top-1 accuracy of the current iteration is higher than the Top-1 highest accuracy recorded previously, update the Top-1 highest accuracy and save the state of the CANet network at this time. This state includes the CANet network architecture, CANet network parameters, the Top-1 highest accuracy, iteration, data loading settings, optimizer state and learning rate scheduler state to ensure that the CANet network has good generalization performance in practical applications. Otherwise, proceed directly to the next step.

[0182] Step 4.3, after obtaining the current highest accuracy Top-1 in step 4.2, if the specified number of training cycles is not reached, return to step 3 to continue iterating. When the CANet network completes the specified number of training cycles, the iteration stops and the final accuracy Top-1 is obtained.

Claims

1. A residual network image classification method based on convolutional axis attention mechanism, characterized in that: The steps include: Step 1, selecting a training sample set and a test sample set, and performing preprocessing operations on the images in the training sample set and the test sample set, where H represents the height of the image and W represents the width of the image; Step 2: Build the CANet network and create a randomly initialized input image M∈R 3×H×W , forward propagation is performed by passing the input image M to the CANet network to obtain the classification result; Step 3, input the training set images obtained in step 1.2 into the CANet network constructed in step 2 for training; Step 4: Use the trained CANet network to classify the test set images to obtain the image classification accuracy.

2. The residual network image classification method based on convolutional axis attention mechanism according to claim 1 is characterized in that: Step 1 is as follows: Step 1.1, select m and n images of size 3×H×W as the training sample set and the test sample set respectively; Step 1.2, preprocess the images in the training sample set and the test sample set in step 1.1, including adjusting the image size to 224×224, random horizontal flipping, normalization, and random cropping to ensure the consistency of the input image size and number of channels.

3. The residual network image classification method based on the convolution axis attention mechanism according to claim 2, characterized in that: In step 2, the CANet network contains four modules, namely: low-level feature extraction module, feature enhancement module, feature fusion module and convolution axis attention mechanism CAA module.

4. The residual network image classification method based on the convolution axis attention mechanism according to claim 2, characterized in that: Step 2 is as follows: Step 2.1, build a low-level feature extraction module, which includes a MaxPooling layer, a residual connection and four convolutional layers; input the input image M into the low-level feature extraction module of the CANet network, first, the input image M passes through a 7×7 convolutional layer A, and then uses a MaxPooling layer with a size of 3×3 and a step size of 2 for downsampling, and then passes through a 1×1 convolutional layer B to reduce the dimension of the feature map to C, passes through a 3×3 convolutional layer D, and then passes through a 1×1 convolutional layer E to increase the dimension to 4C, and the output feature map of the MaxPooling layer is residually connected with the output feature map of the convolutional layer E to obtain the feature map M1∈R 4C×H / 4×W / 4 , where C represents the number of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map; Step 2.2, construct a feature enhancement module, which contains three convolutional layers and a residual connection; input the feature map M1 obtained in step 2.1 into the first convolutional layer F of the feature enhancement module, use a 1×1 convolutional layer F to reduce the dimension of the input feature map to 2C, and then pass it through a 3×3 convolutional layer G with a step size of 2 for downsampling, and finally pass it through a 1×1 convolutional layer H to increase the dimension of the feature map to 8C, and perform a residual connection between the input feature map M1 and the output feature map of the convolutional layer H to obtain the feature map M2∈R 8C×H / 8×W / 8 ; Step 2.3, construct a feature fusion module, which includes three convolutional layers and a residual connection; input the feature map M2 obtained in step 2.2 into the convolutional layer of the feature fusion module, first pass through the first 1×1 convolutional layer I to reduce the dimension of the input feature map to 4C, and keep the size unchanged. Secondly, pass through a convolutional layer J with a size of 3×3 and a step size of 2 for downsampling, and then pass through the second 1×1 convolutional layer N to increase the dimension of the feature map to 16C, and perform a residual connection between the input feature map M2 and the output feature map of the convolutional layer N to obtain the feature map M3∈R 16C×H / 16×W / 16 ; Step 2.4, the fourth module of the CANet network is the convolutional axis attention mechanism module. The convolutional axis attention mechanism module is constructed. The convolutional axis attention mechanism module includes a convolutional layer O, a CAA layer, a convolutional layer P, an AvgPooling layer and a fully connected layer; the CAA layer includes a CBAM layer and an AA layer; the CBAM layer includes a maximum pooling layer, an average pooling layer, two channel convolutional layers, a spatial convolutional layer and two Sigmoid layers; the AA layer includes a convolutional layer W Q , a convolutional layer W K , a convolutional layer W V And two Softmax layers; The convolution axis attention mechanism module processes the feature map M3 as follows: For a given feature map M3, we first reduce the dimension of the feature map through a 1×1 convolution layer O to obtain the feature map M4∈R 8C×H / 32×W / 32 ; The feature map M4 is input to the CBAM layer in the CAA layer. The CBAM layer first passes the input feature map M4 through the global maximum pooling layer and the global average pooling layer in the spatial dimension to obtain two feature maps with 8C channels and a size of 1×1. The two feature maps then pass through two channel convolution layers respectively. The first channel convolution layer reduces the number of channels from 8C to C / 2, and the second channel convolution layer increases the number of channels from C / 2 back to 8C. The two feature maps that pass through the two channel convolution layers are added in the channel dimension and passed through the Sigmoid layer to obtain a one-dimensional channel attention weight map Mc∈R 8C×1×1 , perform element-by-element multiplication of Mc and the input feature map M4 to obtain the feature map X′∈R 8C×H / 32×W / 32 ; The CBAM layer then calculates the maximum and average values ​​of the feature map X′ in the channel dimension, and obtains two feature maps with 1 channel and a size of H / 32×W / 32. These two feature maps are concatenated in the channel dimension and passed through a spatial convolution layer to obtain a two-dimensional spatial attention weight map Ms∈R 1×H / 32×W / 32 , Ms is multiplied element-by-element with the feature map X′ through the Sigmoid activation function to obtain the final refined feature map X″∈R 8C×H / 32×W / 32 ; Secondly, the feature map X″ is input to the AA layer as the input of the next stage. The AA layer decomposes the feature map X″ into horizontal axis features and vertical axis features; the convolution operations of Q, K and V are used to build attention. The feature map X″ will pass through a convolution layer W Q , a convolutional layer W K and a convolutional layer W V To generate Q∈R 8C×H / 32×W / 32 , K∈R 8C×H / 32×W / 32 and V∈R 8C×H / 32×W / 32 , convert Q from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), b represents the batch size, convert K from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), convert V from (b, 8C, H / 32, W / 32) to (b×H / 32, -1, W / 32), then perform matrix multiplication on Q and K to obtain the row attention score, and normalize it through the Softmax layer, then perform matrix multiplication on the normalized row attention score and V to obtain the row attention weight map, add the row attention weight map to the input feature map X″, and obtain the row attention enhanced feature map Row∈R 8C×H / 32×W / 32 ; Similar to row attention, the operation is performed on the column dimension, converting Q from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), where b represents the batch size, converting K from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32), and converting V from (b, 8C, H / 32, W / 32) to the shape of (b×H / 32, -1, W / 32). Calculate the dot product of Q and K by matrix multiplication to obtain the column attention score, normalize the column attention score by Softmax, and then perform matrix multiplication with V to obtain the column attention weight map, add the column attention weight map to the input feature map X″, and obtain the column attention enhanced feature map Col∈R 8C×H / 32×W / 32 ; Then use bilinear interpolation to adjust the feature maps Row and Col of different axes back to their original sizes, merge the two feature maps, and weight them by parameters to obtain the final feature map M5∈R 8C×H / 32×W / 32 ; After a 1×1 convolution layer P, the feature map M6∈R is obtained 32C×H / 32×W / 32 ; Finally, the feature map M6 is input into the AvgPooling layer, and the spatial dimension of each channel of the feature map M6 is reduced to 1×1, and the feature map M7∈R is obtained. 32C×1×1 ,Then, the feature map M7 is flattened into a one-dimensional vector, and the flattened one-dimensional feature vector is mapped to the final output category through a fully connected layer, and classification is performed according to the output of the fully connected layer to obtain the classification result.

5. The residual network image classification method based on the convolution axis attention mechanism according to claim 4, characterized in that: In step 2.4, the attention process of the CBAM layer is summarized as formula (1) and formula (2) respectively: in Represents element-wise multiplication.

6. The residual network image classification method based on the convolution axis attention mechanism according to claim 4, characterized in that: In step 2.4, the attention process of the AA layer is summarized as shown in formula (3) and formula (4): Row = RowAttention(X″)(3) Col = ColAttention(X″)(4) 7. The residual network image classification method based on the convolution axis attention mechanism according to claim 4, characterized in that: In step 2.4, the feature map M5 is as shown in formula (5): M5=Row+Col(5).

8. The residual network image classification method based on the convolution axis attention mechanism according to claim 4, characterized in that: The specific operations of step 3 are: The diversified training set images obtained in step 1.2 are input into the CANet network designed in step 2 for training. The standard cross entropy loss function L is used to calculate the difference between the probability distribution of the CANet network output category and the true label. Then, the CANet network is updated through the back propagation algorithm to minimize the average loss over the entire training. The standard cross entropy loss function L is shown in formula (6): Among them, c represents the category, K represents the total number of categories, and y c is the one-hot encoding of the true label of category c, p c is the predicted probability of the CANet network for category c.

9. The residual network image classification method based on the convolution axis attention mechanism according to claim 4, characterized in that: The specific operations of step 4 are: Step 4.1, input the test set images of the diversified images obtained in step 1.2 into the CANet network trained in step 3; Step 4.2: During the test in step 4.1, the accuracy function is used to calculate the Top-1 accuracy between the CANet network output and the target label after each iteration, as shown in formula (7): Among them, NCP is the number of images predicted correctly by the CANet network, and TNP is the total number of images predicted by the CANet network; Compare the Top-1 accuracy obtained in the current iteration with the Top-1 highest accuracy recorded in the previous iteration. If the Top-1 accuracy of the current iteration is higher than the Top-1 highest accuracy recorded previously, update the Top-1 highest accuracy and save the state of the CANet network at this time. This state includes the CANet network architecture, CANet network parameters, the Top-1 highest accuracy, iteration, data loading settings, optimizer state and learning rate scheduler state to ensure that the CANet network has good generalization performance in practical applications. Otherwise, proceed directly to the next step. Step 4.3, after obtaining the current highest accuracy Top-1 in step 4.2, if the specified number of training cycles is not reached, return to step 3 to continue iterating. When the CANet network completes the specified number of training cycles, the iteration stops and the final accuracy Top-1 is obtained.

Images