Remote sensing scene image classification method based on mixed attention of groups

Abstract

The application relates to a remote sensing scene image classification method based on a grouping attention mixed attention, and relates to a remote sensing scene image classification method.The application aims to solve the problem that although the existing method can improve the performance of a model by introducing an attention module in remote sensing scene image classification, directly introducing the attention module increases the parameters and calculation amount of the model, and causes the model to operate slowly.The process is as follows: I. obtaining a hyperspectral image;II. establishing a lightweight convolutional neural network model based on a mixed attention, including an input layer, a first convolutional layer, a second convolutional layer, a first to sixth channel grouping mixed attention module, a global average pooling, a full connection layer and a SoftMax function;each channel grouping mixed attention module comprises a channel attention module and a spatial attention module;III. obtaining a trained neural network model;IV. inputting an image to be measured into the trained neural network model to obtain a classification result.The application is used in the field of image classification.

Description

Technical Field

[0001] This invention relates to a method for classifying remote sensing scene images. Background Technology

[0002] In recent years, convolutional neural networks have achieved excellent performance in many fields [1-4]. Especially in the field of image classification [5], convolutional neural networks (CNN) have become the most commonly used method in this field. The core construction of convolutional neural networks is the convolutional layer. For each convolutional layer, a set of filters is learned along the input channel to express the local spatial connection pattern, and information features are extracted by fusing spatial information and channel information in the local receptive field. Improving the spatial encoding quality of the entire feature level of the convolutional neural network to enhance the network's representation ability is an effective means to improve network performance. VGGNets [6] shows that increasing the network depth can significantly improve the network performance. ResNet [7] solves the problem of performance degradation caused by increasing network depth. Based on VGGNets, it extends the network depth to 150 layers or even thousands of layers by adding skip connections and achieves good performance. InceptionNet [8-11] divides the input features into four channels and uses different convolutional kernels on each channel to adapt to features of different scales. Finally, the extracted features are fused along the channel direction, and finally the feature representation ability of the network is improved by increasing the width of the network. Subsequently, a series of lightweight convolutional neural networks were proposed. These networks have good representation quality while reducing complexity. Xception

[12] and MobilenetV1

[13] introduced depthwise separable convolution to replace traditional convolution to achieve lightweight networks. Depthwise separable convolution divides ordinary convolution into depthwise convolution and 1×1 convolution to reduce the number of parameters in the model. The subsequently proposed MobilenetV2

[14] proved the effectiveness of depthwise separable convolution. Grouped convolution is also a way to improve the representation ability of the network by increasing the width of the network, and grouped convolution can reduce the computational cost of the network while increasing the width of the network. Assuming that G represents the number of groups, compared with ordinary convolution, the number of parameters and the computational cost are the same as those of ordinary convolution. Grouped convolution was first used in AlexNet to reduce computational costs due to hardware limitations. ResNeXts

[15] used grouped convolution to increase the depth and width of the model, resulting in a significant improvement in accuracy. ShuffleNet[16,17] proposed channel shuffling, which can alleviate the problem of information loss caused by the lack of information exchange between channels due to grouping.

[0003] The improvement of network performance by attention mechanism has been verified in many tasks. SENet

[18] improves network performance by explicitly modeling the interdependencies between channels. SENet includes two operations: squeezing and excitation. The squeezing operation squeezes the features in space through global average pooling to obtain a value with a global receptive field. The excitation operation passes the obtained value through two consecutive fully connected layers, uses the correlation between channels to obtain a channel attention map, and finally recalibrates the features through the channel attention map. SKNet

[19] adds two operations: separation and fusion on the basis of SENet. The separation operation uses convolutional kernels with different receptive field sizes to capture multi-scale semantic information, and the fusion operation fuses multi-scale semantic information to enhance feature diversity. Finally, the SE module aggregates the feature maps of convolutional kernels of different sizes according to the weights. Improving network performance by explicitly modeling the interdependencies between channels and space and fusing the two is also an effective method. CBAM

[20] extracts channel attention and spatial attention by combining global average pooling and max pooling. The input features are enhanced spatially and channel-wise using spatial attention and channel attention respectively, and the enhanced features are then fused to improve model performance. Wang et al.

[21] designed a recurrent attention structure to compress high-level semantic and spatial features to reduce the number of learning parameters. Tong et al.

[22] introduced an attention mechanism in DenseNet to adaptively enhance the weights of important feature channels. Yu et al.

[23] improved channel attention and combined the improved channel attention with ResNet to propose a hierarchical attention mechanism. Alhichri et al.

[24] proposed a deep attention convolutional neural network for learning feature maps from large scene regions. We note that although introducing an attention module into the network helps improve network performance, directly adding the attention mechanism to the network increases the number of network parameters and computation, which reduces the computation speed of the model. Summary of the Invention

[0004] The purpose of this invention is to address the problem that while introducing attention modules into remote sensing scene image classification can improve model performance, directly introducing attention modules increases the model's parameters and computational load, resulting in slower model processing speed. Therefore, this invention proposes a remote sensing scene image classification method based on grouped hybrid attention.

[0005] The specific process of the remote sensing scene image classification method based on grouped hybrid attention is as follows:

[0006] Step 1: Acquire hyperspectral images;

[0007] Step 2: Establish a lightweight convolutional neural network model based on hybrid attention; the specific process is as follows:

[0008] The lightweight convolutional neural network model based on hybrid attention includes an input layer, a first convolutional layer, a second convolutional layer, a first-channel grouped hybrid attention module, a second-channel grouped hybrid attention module, a third-channel grouped hybrid attention module, a fourth-channel grouped hybrid attention module, a fifth-channel grouped hybrid attention module, a sixth-channel grouped hybrid attention module, global average pooling, a fully connected (FC) layer, and a SoftMax function classifier.

[0009] Each of the first, second, third, fourth, fifth, and sixth channel grouped hybrid attention modules includes a channel attention module and a spatial attention module.

[0010] The channel attention module includes an input layer, a global average pooling layer, a third convolutional layer, a ReLU activation function layer, a fourth convolutional layer, and a Sigmoid activation function ζ(·) layer;

[0011] The kernel size of the third convolutional layer is 1×1; the kernel size of the fourth convolutional layer is 1×1;

[0012] The spatial attention module includes an input layer, a fifth convolutional layer, and a Sigmoid activation function ζ(·) layer;

[0013] The kernel size of the fifth convolutional layer is 5×5;

[0014] The features output by the channel attention module and the features output by the spatial attention module are fused to obtain the output of the channel grouping hybrid attention module;

[0015] Step 3: Input the hyperspectral image into the established lightweight convolutional neural network model based on hybrid attention for training, and obtain the trained lightweight convolutional neural network model based on hybrid attention.

[0016] Step 4: Input the hyperspectral image to be tested into the trained lightweight convolutional neural network model based on hybrid attention to obtain the classification result.

[0017] The beneficial effects of this invention are as follows:

[0018] First, this invention constructs novel channel attention and spatial attention modules to recalibrate features, and then fuses these two attention modules to obtain a hybrid attention module. To achieve model lightweighting, a group-wise hybrid attention strategy is further proposed. Finally, a lightweight convolutional neural network (LCNN-GWHA) for remote sensing scene image classification is constructed based on group-wise hybrid attention. The group-wise hybrid attention module groups the input features. Each group of features is recalibrated using spatial attention and channel attention respectively, and then the recalibrated features are fused. By using group-wise hybrid attention, the network parameters and computational cost are significantly reduced, improving the network's computational speed. Experiments on four open and challenging remote sensing scene datasets show that this method still has a significant advantage in classification accuracy with a very low parameter count.

[0019] Channel attention is implemented by setting the channel compression ratio to 4 based on the SE module. This invention uses a 5×5×1 convolutional kernel to compress the input features through channels, and then applies the compressed features to a Sigmoid activation function to obtain spatial attention. Next, channel attention and spatial attention are fused to obtain hybrid attention. To achieve a lightweight model, we further propose a grouped hybrid attention method. Grouped hybrid attention groups the input features and introduces hybrid attention to each group. Each group is recalibrated using both spatial and channel attention, and then the recalibrated features are fused.

[0020] This invention proposes an end-to-end lightweight neural network for remote sensing scene image classification, combining the advantages of channel attention, spatial attention, and channel grouping. We introduce channel attention to enhance important features, spatial attention to strengthen regions of interest, and fuse the two attention methods to obtain a hybrid attention with higher activation and the ability to learn more meaningful features. To achieve lightweightness, this invention further proposes grouped hybrid attention, introducing hybrid attention into each group, which significantly improves model efficiency while maintaining classification accuracy. Experiments were conducted on four datasets with various training ratios, and the results demonstrate the robustness of the proposed method, achieving higher accuracy compared to state-of-the-art methods. The method proposed in this invention provides a novel approach for remote sensing scene classification.

[0021] The main contributions of this invention are as follows:

[0022] (1) This invention proposes a channel attention module that is more suitable for remote sensing scene image classification based on the SE module. The proposed method sets the channel compression ratio to 4 and uses a 1×1 convolution kernel to replace the fully connected layer. The 1×1 convolution does not destroy the spatial structure of the features and the size of the input features can be arbitrary.

[0023] (2) This invention proposes a spatial attention module with a simpler implementation process. A 5×5×1 convolutional kernel is directly used to compress the channels, and then the spatial attention features are obtained through the Sigmoid activation function. The use of a 5×5 convolutional kernel can increase the receptive field and add spatial location information.

[0024] (3) By combining channel attention and spatial attention in parallel, a hybrid attention is formed. Hybrid attention has higher activation and can learn more meaningful features.

[0025] (4) To construct a lightweight convolutional network, this invention further proposes a grouped hybrid attention module. This module first divides the input features into 4 groups along the channel dimension, and then introduces hybrid attention into each group. During grouped convolution, each group is recalibrated using spatial attention and channel attention respectively, and finally the recalibrated features are fused by channel.

[0026] (5) A lightweight convolutional neural network (LCNN-GWHA) for remote sensing scene image classification is proposed based on grouped hybrid attention, and the superiority of the method is demonstrated through a series of experiments. Attached Figure Description

[0027] Figure 1 This is a diagram showing the overall structure of the method of the present invention. Channels represents the number of channels, and Output size represents the size of the output feature.

[0028] Figure 2 This is a structural diagram of the channel attention module of the present invention;

[0029] Figure 3 This is a structural diagram of the spatial attention module of the present invention;

[0030] Figure 4 This is a structural diagram of the hybrid attention module (C&S) of the present invention;

[0031] Figure 5 This is a structural diagram of the Channel Grouping Hybrid Attention Module (GWHAM) of the present invention;

[0032] Figure 6 This is the feature structure map obtained from traditional convolution;

[0033] Figure 7A diagram of the confusion matrix on the RSSCN dataset;

[0034] Figure 8 Here is a confusion matrix diagram on the UCM dataset;

[0035] Figure 9 The confusion matrix is ​​plotted on the AID dataset where the training ratio is 50%.

[0036] Figure 10 The confusion matrix is ​​plotted on the NWPU dataset where the training ratio is 20%.

[0037] Figure 11a Figure 1 shows the visualization results of the three attention methods; Figure 11b Two figures show the visualization results of the three attention methods;

[0038] Figure 12a Figure 1 shows the visualization results of the heatmap on the UCM21 dataset;

[0039] Figure 12b Two figures show the visualization results of the heatmap on the UCM21 dataset;

[0040] Figure 12c Three figures show the visualization results of the heatmap on the UCM21 dataset;

[0041] Figure 12d Four figures show the visualization results of the heatmap on the UCM21 dataset;

[0042] Figure 12e Five figures show the visualization results of the heatmap on the UCM21 dataset;

[0043] Figure 13a Figure 1 shows the results of random classification prediction.

[0044] Figure 13b Two figures show the results of random classification prediction;

[0045] Figure 13c Three figures show the results of random classification prediction;

[0046] Figure 13d Four figures show the random classification prediction results;

[0047] Figure 13e Five figures show the results of random classification prediction;

[0048] Figure 13f Six figures show the results of random classification prediction;

[0049] Figure 13g The results of random classification prediction are shown in 7 figures. Detailed Implementation

[0050] Specific Implementation Method 1: The specific process of the remote sensing scene image classification method based on grouped hybrid attention in this implementation method is as follows:

[0051] Step 1: Acquire hyperspectral images;

[0052] Step 2: Establish a lightweight convolutional neural network model based on hybrid attention; the specific process is as follows:

[0053] The lightweight convolutional neural network model based on hybrid attention includes an input layer, a first convolutional layer, a second convolutional layer, a first-channel grouped hybrid attention module, a second-channel grouped hybrid attention module, a third-channel grouped hybrid attention module, a fourth-channel grouped hybrid attention module, a fifth-channel grouped hybrid attention module, a sixth-channel grouped hybrid attention module, global average pooling (GAP), a fully connected layer (FC), and a SoftMax function classifier.

[0054] Each of the first, second, third, fourth, fifth, and sixth channel grouped hybrid attention modules includes a channel attention module and a spatial attention module.

[0055] The channel attention module includes an input layer, a global average pooling (GAP) layer, a third convolutional layer, a ReLU activation function layer, a fourth convolutional layer, and a Sigmoid activation function ζ(·) layer.

[0056] The kernel size of the third convolutional layer is 1×1; the kernel size of the fourth convolutional layer is 1×1;

[0057] The spatial attention module includes an input layer, a fifth convolutional layer, and a Sigmoid activation function ζ(·) layer;

[0058] The kernel size of the fifth convolutional layer is 5×5;

[0059] The features output by the channel attention module and the features output by the spatial attention module are fused to obtain the output of the channel grouping hybrid attention module;

[0060] Step 3: Input the hyperspectral image into the established lightweight convolutional neural network model based on hybrid attention for training, and obtain the trained lightweight convolutional neural network model based on hybrid attention.

[0061] Step 4: Input the hyperspectral image to be tested into the trained lightweight convolutional neural network model based on hybrid attention to obtain the classification result.

[0062] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the connection relationships of the lightweight convolutional neural network model based on hybrid attention are as follows:

[0063] The input layer is connected to the first convolutional layer, the first convolutional layer is connected to the second convolutional layer, the second convolutional layer is connected to the first channel grouped hybrid attention module, the first channel grouped hybrid attention module is connected to the second channel grouped hybrid attention module, the second channel grouped hybrid attention module is connected to the third channel grouped hybrid attention module, the third channel grouped hybrid attention module is connected to the fourth channel grouped hybrid attention module, the fourth channel grouped hybrid attention module is connected to the fifth channel grouped hybrid attention module, the fifth channel grouped hybrid attention module is connected to the sixth channel grouped hybrid attention module, the sixth channel grouped hybrid attention module is connected to global average pooling (GAP), global average pooling (GAP) is connected to a fully connected FC layer, and the fully connected FC layer is connected to a SoftMax function classifier.

[0064] The other steps and parameters are the same as in Specific Implementation Method 1.

[0065] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that the connection relationship of the channel attention module is as follows:

[0066] Channel attention first obtains features by spatially compressing the input features. Then compress the features The correlation between different channels is modeled through convolution operations, and different weight coefficients are assigned to each channel to strengthen important features and suppress unimportant features. The specific process is as follows: Figure 2 As shown.

[0067] Let the input features be X = [x1, x2, ..., x...]. C ],in Represent the characteristics of the i-th channel; 1≤i≤C;

[0068] The input feature X is spatially compressed using global average pooling to obtain the feature. Features after global average pooling compression It has global spatial information;

[0069] Where the feature x of the i-th channel i The result M after global average pooling i It is expressed as follows:

[0070]

[0071] In equation (2), H and W represent the characteristic x, respectively. i Height and width; x i(m,n) represents the feature of the i-th channel at the corresponding spatial location (m,n);

[0072] Features with global receptive field The input is to the third convolutional layer, the output of the third convolutional layer is input to the ReLU activation function layer, the output of the ReLU activation function layer is input to the fourth convolutional layer, and the output of the fourth convolutional layer is the feature.

[0073] in Indicates the weights of the third convolutional layer. R represents the weights of the fourth convolutional layer, and R(·) represents the ReLU activation function.

[0074] The third convolutional layer performs dimensionality reduction, decreasing the number of channels in the features to half the original number. Next, the ReLU activation function is used to increase the non-linear relationship between the channels;

[0075] The fourth convolutional layer restores the number of channels;

[0076] Features The channel attention features are obtained by normalizing the learned activation values ​​of each channel to the range [0,1] using the Sigmoid activation function. ζ(·) represents the activation function Sigmoid;

[0077] Finally, channel attention features are used. For input features X = [x1, x2, ..., x C Recalibrate to obtain

[0078]

[0079] In equation (3), This indicates the importance of the i-th channel. These activation values ​​can be adaptively adjusted through learning by a convolutional neural network. The channel attention module can strengthen important features and suppress unimportant features.

[0080] Other steps and parameters are the same as in specific implementation method one or two.

[0081] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the connection relationship of the spatial attention module is as follows:

[0082] Spatial attention first obtains features by squeezing the channels. Then compress the features By assigning different weight coefficients to each location through an activation function, the specific target region of interest is strengthened while irrelevant regions are suppressed. The specific process is as follows: Figure 3 As shown.

[0083] Let the input features be X = [x 1,1 ,x 1,2 ,...,x m,n ,...,x H,W ], The features represent the corresponding spatial locations (m, n), where m ∈ {1, 2, ..., H} and n ∈ {1, 2, ..., W}; H and W represent the features x, respectively. m,n Height and width;

[0084] The input features pass through the fifth convolutional layer (the fifth convolutional layer kernel). Features after channel compression Output features feature The calculation process is as follows:

[0085]

[0086] In formula (4), * represents convolution, f sq This indicates that the kernel size of the fifth convolutional layer is 5×5 and the number of channels is 1;

[0087] This represents a linear combination of all channels at spatial location (m,n);

[0088] Features Features are processed by the Sigmoid activation function ζ(·). Normalizing to the [0,1] range yields spatial attention features.

[0089] Finally, spatial attention characteristics were analyzed. For input features X = [x 1,1 ,x 1,2 ,...,x m,n ,...,x H,W Recalibrate to obtain

[0090]

[0091] This indicates the importance of spatial coordinates (m, n) in the feature map. It emphasizes relevant locations and suppresses less important ones.

[0092] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0093] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the connection relationship of each channel grouping attention module in the first channel grouping attention module, the second channel grouping attention module, the third channel grouping attention module, the fourth channel grouping attention module, the fifth channel grouping attention module, and the sixth channel grouping attention module is as follows:

[0094] Hybrid attention module

[0095] Spatial attention ignores information exchange between channels, while channel attention ignores information exchange within space. This invention combines channel attention and spatial attention to form a more efficient hybrid attention mechanism, such as... Figure 4 As shown.

[0096] Combine spatial attention module and channel attention module Simultaneously, we calibrate the input feature X in both space and channel, using... This indicates the importance of the i-th channel at spatial coordinates (m, n).

[0097] When an input feature X undergoes a mixed attention calibration operation When · indicates element-wise multiplication, the channel features and spatial features of the input features at position (m,n,i) are enhanced respectively. The enhanced features have higher activation and can learn more meaningful features.

[0098] Grouped Hybrid Attention Module

[0099] While incorporating hybrid attention into the network improves performance, it inevitably increases the network's computational cost, leading to a decrease in model speed. Therefore, we use grouped convolutions on top of the attention mechanism to extract features more efficiently, such as... Figure 5 As shown.

[0100] Let the input features be... First, the input features are grouped along the channel dimension to obtain...

[0101] The four grouped features X1, X2, X3, and X4 are input channels grouped and then enhanced in the hybrid attention module, focusing on both the channel and spatial dimensions of the features.

[0102] Finally, the obtained features The output feature Y is obtained by fusing along the channel direction;

[0103] The specific process is as follows:

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] In the formula, This indicates the importance of the i-th channel at spatial coordinate position (m,n), and Θ represents the fusion of features according to channel dimensions.

[0110] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0111] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the Global Average Pooling (GAP) connection to the FC fully connected layer and the output result of the FC fully connected layer on the Global Average Pooling are described. We obtain Q by classification.

[0112] The number of categories in the FC fully connected layer is N;

[0113] The fully connected (FC) layer connects to the SoftMax function classifier. The output of the SoftMax function classifier is...

[0114] The output result S is represented as follows:

[0115]

[0116] In the formula, In Q, the first 1 element (indexed starting from 0), Q[ k ] represents the k-th element in Q (index starts from 0).

[0117] Lightweight Convolutional Neural Network Based on Hybrid Attention

[0118] The structural diagram of the modular network proposed in this invention is as follows: Figure 1 As shown, this structure mainly consists of convolutions, group attention modules, global average pooling layers, and a classifier. The input features first undergo two consecutive convolutional operations to extract shallow feature information, then pass through six consecutive group attention modules to extract deeper features. The output features of the final convolutional layer are then directly mapped to each class using global average pooling (GAP). Using global average pooling does not increase the weight parameters, effectively avoiding overfitting that may occur during training.

[0119] Finally, the SoftMax classifier is used to classify the features. If a fully connected layer (FC) with N classes is used to classify the output of average pooling... The result of the classification is The output of SoftMax is The output S of SoftMax is represented as follows:

[0120]

[0121] In the formula, In Q, the first 1 element (indexed starting from 0), Q[ k ] represents the k-th element in Q (index starts from 0).

[0122] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0123] Specific Implementation Method Seven: This implementation method differs from one of Specific Implementation Methods One to Six in that the lightweight convolutional neural network model based on hybrid attention established in step two uses cross-entropy as the loss function.

[0124] Assumption If the encoding result of the input sample label is given, then the loss function is expressed as:

[0125]

[0126] In the formula, N represents the number of categories. This represents the output of Softmax, where the input sample labels use one-hot encoding.

[0127] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0128] Ordinary convolution process

[0129] Assume the input features are The output features are obtained through the convolution operation F(·). like Figure 6 As shown, using U = [u1, u2, ..., u C ] represents the set of convolution kernels in the convolution operation F(·), where u C Let represent the c-th convolutional kernel. Then the output Y = [y1, y2, ..., y]. C The c-th channel feature in the image can be represented as:

[0130]

[0131] In the above formula, * represents convolution. X = [x 1 ,x 2 ,...,x C′ ], x represents the i-th channel of the C-th convolutional kernel. i This represents the i-th channel of the input feature. This represents the spatial features learned on the i-th channel. This represents the summation operation performed on the spatial features learned through convolution across all channels, so the final output feature y of the convolution is... c It includes both channel features and spatial features. By explicitly modeling the correlations between different channels and the correlations between different spaces, channel attention and spatial attention can be obtained.

[0132] The beneficial effects of the present invention are verified using the following embodiments:

[0133] Example 1:

[0134] The proposed LCNN-GWHA method was comprehensively evaluated using multiple metrics. Experiments were conducted on four challenging datasets to compare the proposed LCNN-GWHA method with state-of-the-art methods. To ensure fairness, both the proposed and comparative methods used the same hyperparameters and were tested under the same experimental conditions. Experimental results show that the proposed method can classify remote sensing scene images more accurately and has significant advantages in terms of parameter size and running speed.

[0135] Dataset A

[0136] This experiment was conducted on four commonly used datasets. The datasets are UCM21

[25] , RSCCN7

[26] , AID

[27] , and NWPU-RESISC45

[28] . Table 1 compares the number of images, number of scene categories, total number of images, spatial resolution of images, and image size of the four datasets. To avoid memory overflow during training, bilinear interpolation was used to adjust the size of the input images to 256×256.

[0137] Table 1 Comparison of the four datasets

[0138]

[0139] B. Experimental Setup

[0140] Stratified sampling was used to partition the dataset to avoid the risk of sampling bias. In addition, to ensure that the proposed method and the comparison method use the same training samples, a random seed was set when partitioning the samples. For the UCM21

[25] dataset, the training ratio was set to 80%; for the RSCCN7

[26] dataset, the training ratio was set to 50%; for the AID

[27] dataset, the training ratios were set to 20% and 50%; and for the NWPU-RESI SC45

[28] dataset, the training ratios were 10% and 20%, respectively. The parameters and equipment used in the experiment are listed in Table 2.

[0141] Table 2 Experimental Environment Parameter Settings

[0142]

[0143] C. Comparison with advanced methods

[0144] Table 3 lists the performance of the proposed method on four datasets with various training ratios. To verify the performance of the proposed method, we used overall accuracy (OA), average accuracy (AA), kappa coefficient (Kappa), and F1 score (F1) as evaluation metrics in the experiments. OA represents the percentage of correctly classified samples in the total test set. AA represents the ratio of the number of correctly predicted samples in each class to the total number of samples in that class. The F1 score is a weighted average of precision and recall, used to measure the robustness of the model. The Kappa coefficient is used for consistency evaluation, assessing whether the predicted results are consistent with the actual classification results. As can be seen from Table 3, the proposed method achieves overall accuracy (OA) and average accuracy (AA) of over 90% on the four datasets, and the difference between overall accuracy (OA) and average accuracy (AA) is within 1%, demonstrating that the proposed method has strong generalization ability. The Kappa coefficient of over 90% indicates that the predicted values ​​obtained by the proposed method are almost completely consistent with the true values. The F1 score also demonstrates the strong robustness of the proposed method.

[0145] Table 3 shows the performance of the proposed model on four datasets.

[0146]

[0147]

[0148] Next, the proposed method and advanced methods were compared on different datasets using different evaluation metrics. First, an experiment was conducted on the RSCN dataset with a training:test ratio of 5:5, and the results are shown in Table 4. The proposed method has 0.3M parameters and achieves a classification accuracy of 97.78%, the highest accuracy and fewest parameters compared to all the compared methods. The proposed method outperforms Coutourlet CNN

[34] by 2.24%, ADFF

[33] by 2.57%, and SE-MDPMNet

[35] by 3.07%. Figure 7 The image shows the confusion matrix on the RSCN dataset. Figure 7 The results show that the method achieved a classification accuracy of 99% for the forest, parking lot, and river scenes, indicating that these scenes have high inter-class differences and intra-class similarities. The classification accuracy for the grassland scene was the lowest at 95%, with some scenes being misclassified as forest or field scenes. This is because these three scenes are similar, with only minor intra-class differences leading to incorrect classification of the grassland scene.

[0149] Table 4 compares the performance of the proposed model with state-of-the-art methods on the RSCCN dataset.

[0150]

[0151] Table 5 shows the experimental comparison between the proposed method and existing methods on the UCM dataset with a training:test ratio of 8:2. On this dataset, the accuracy of various methods can reach over 99%, so the number of parameters is a very important evaluation metric. The proposed method has 0.31M parameters and a classification accuracy of 99.76%, which is 5.89M fewer than the LCNN-BFF Method

[32] with an accuracy of 99.29%, and 21.69M fewer than the Inceptionv3+CapsNet Method

[43] with an accuracy of 99.05%. The proposed method greatly reduces the number of model parameters while achieving high accuracy.

[0152] exist Figure 8 The diagram shows the confusion matrix of the proposed method on the training:test = 8:2 UCM dataset. Figure 8 As can be seen, all scenes except the medium-sized residential area scene were completely identified. This is because the medium-sized residential area and mobile home scenes are very similar in appearance, leading to significant confusion during classification.

[0153] Table 5 compares the performance of the proposed model with state-of-the-art methods on the UCM dataset.

[0154]

[0155]

[0156] Table 6 shows the experimental comparison between the proposed method and existing methods on the AID dataset with training:test = 2:8 and training:test = 5:5. As can be seen from Table 6, when the training ratio is 20%, the proposed method achieves the best performance among all the compared methods with the fewest parameters, with a classification accuracy of 93.85%, which is 0.58% higher than the InceptionV3

[52] method and 1.46% higher than the ResNet50

[52] method. When the training ratio is 50%, compared with the Discriminative CNN

[40] method, the InceptionV3

[52] method and the Skip-Connected CNN

[39] method, it achieves a significant advantage in classification accuracy while the model parameter count is only 0.2%, 0.6% and 1.6% of theirs, respectively.

[0157] exist Figure 9 The image shows the confusion matrix of the proposed method on the training:test = 5:5 AID dataset. From... Figure 9 As can be seen, the grassland, overpass, and sparsely populated residential areas were classified completely correctly. The factory scene had the lowest recognition accuracy at 94%, because the similar architectural styles of factories and commercial areas led to some factories being incorrectly classified as commercial areas. Deserts and wastelands were also easily confused due to their similar surface appearances, resulting in lower classification accuracy for deserts. Nevertheless, compared to other newer classification methods, the method proposed in this invention still achieves high classification accuracy.

[0158] Table 6 compares the performance of the proposed model with state-of-the-art methods on the AID dataset.

[0159]

[0160] Table 7 shows the experimental comparison between the proposed method and existing methods on the NWPU45 dataset with training:test = 2:8 and training:test = 1:9. When the training ratio is 10%, the proposed method achieves an accuracy of 92.24% with 0.3M parameters, which is 2.01% higher than LiG with RBF kernel

[53] with 2.07M parameters, 7.91% higher than Skip-Connected CNN

[39] with 6M parameters, and 5.71% higher than LCNN-BFF Method

[32] with 6.2M parameters. When the training ratio is 20%, the proposed method achieves a classification accuracy of 94.26%, which is 1.01% higher than LiG with RBF kernel

[53] , 2.37% higher than Discriminative with VGG16

[40] , and 2.53% higher than LCNN-BFFMethod

[32] . Experimental results show that the method proposed in this invention can obtain more significant features with fewer parameters in datasets with rich image variations and high inter-class similarity and intra-class dissimilarity.

[0161] exist Figure 10 The diagram shows the confusion matrix of the proposed method on the NWPU45 dataset with a training:test ratio of 2:8. Due to the high intra-class dissimilarity and inter-class similarity of the NWPU45 dataset, no single class was classified completely correctly. However, classification accuracy exceeded 90% for 44 scenes, achieving satisfactory results. Figure 10 As shown, the scenarios with the worst recognition accuracy are churches and palaces, with accuracies of 90% and 89% respectively. This is because these two scenarios have very similar buildings, leading to confusion during classification. Furthermore, the classification accuracy for roundabouts is low at 91%, as irregular intersections are easily misclassified as roundabouts. Nevertheless, the proposed method still provides good classification results for all scenarios.

[0162] Table 7 compares the performance of the proposed model with state-of-the-art methods on the NWPU dataset.

[0163]

[0164] D. Comparison of model running speed

[0165] To verify the speed advantage of our method, we conducted comparative experiments with state-of-the-art methods on the UCM21 dataset using the ATT evaluation metric. The experimental results of our method and state-of-the-art methods are shown in Table 8.

[0166] As can be seen from Table 8, under the same experimental equipment, the method of the present invention takes 0.010s to process a remote sensing image, which is the shortest among all the comparative methods. It is 0.018s faster than the SiameseAlexNet

[48] method and 0.019s faster than the LCNN-BFF

[32] method.

[0167] Table 8 compares the proposed model with state-of-the-art methods in ATT on the UCM21 dataset.

[0168]

[0169]

[0170] E. Model Size Assessment

[0171] Experiments were conducted on the AID dataset with a training ratio of 50% using the FLOPs evaluation metric. FLOPs measures the complexity of the model, and the smaller the FLOPs value, the more lightweight the network. The experimental comparison results between the proposed method and state-of-the-art methods are shown in Table 9. It is clear that the method of the present invention is superior to all the compared methods in terms of parameter quantity and complexity. Compared with the lightweight network models MobileNetV2

[35] and SE-MDPMNet

[35] , the proposed method has a significant advantage in FLOPs value, and the classification accuracy is 1.68% and 0.5% higher than them, respectively.

[0172] Table 9 compares the model size of the proposed model with that of state-of-the-art methods on the AID dataset.

[0173]

[0174] F. Model Visualization Analysis

[0175] To more intuitively demonstrate the feature extraction capabilities of the proposed method, we employ a visualization approach. As shown in Figure 13, we select images from the UCM21 dataset and visualize the feature maps of three attention methods: channel attention, spatial attention, and hybrid attention. Figure 11a , 11b The visualization results show that the three attention methods focus on different points of interest and strengthen different regions. Channel attention strengthens a specific feature point of interest, spatial attention strengthens a background region of interest, and hybrid attention combines the two, strengthening both the background region and the feature point simultaneously. By strengthening both the feature point of interest and the background region, hybrid attention achieves higher activation and can learn more meaningful features.

[0176] To comprehensively evaluate the model of this invention from different perspectives, a gradient-based localization method (Grad-CAM) was employed to perform visualization analysis on different network models. This method can use the gradient of any target and then generate a coarse attention map from the last layer of the convolutional network to display the important regions in the image predicted by the model. A selection of images from the UCM21 dataset were randomly selected for visualization analysis. Figure 12a , 12b As can be seen from 12c, 12d, and 12e, the model of this invention can better cover semantic objects and has a wide highlight range. We believe this is because the proposed group attention enhances the features, giving the proposed method a strong ability for object localization and recognition.

[0177] In addition, the trained network model was used to conduct random tests on scene images in the UCM21 dataset. For example... Figure 13a , 13b As shown in 13c, 13d, 13e, 13f, and 13g, the prediction confidence of this model is above 99%, with some individual cases reaching 100%. This proves that the method can extract image features more effectively.

[0178] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

[0179] References

[0180] 1. Krizhevsky, A.; Sutskever, I.; Hinton, GEImagenet classification with deep convolutional neural networks. In NeurIPS, pages 1097–1105, 2012.

[0181] 2. Toshev, A.; Szegedy, C. DeepPose: Human pose estimation via deep neural networks. pp. 1653-1660, In CVPR, 2014.

[0182] 3.Long,J.;Shelhamer,E.;Darrell,T.Fully convolutional networks forsemantic segmentation.pp.3431-3440,In CVPR,2015.

[0183] 4.Ren,S.;He,K.;Girshick,R.;Sun,J.Faster R-CNN:Towards real-timeobject detection with region proposal networks.vol.39,no.6,pp.1137-1149,InConference on Neural Information Processing Systems,2017.

[0184] 5.He,K.;Zhang,X.;Ren,S.;Sun,J.Deep residual learning for imagerecognition.In: Proceedings of the IEEE conference on computer vision andpattern recongnition.pp. 770-778(2016)1,3,4,5,8,18.

[0185] 6.Simonyan,K.;Zisserman,A.Very deep convolutional networks for large-scale image recognition.In arXiv:1409.1556[cs.CV]10Apr 2015.

[0186] 7.He,K.;Zhang,X.;Ren,S.;Sun,J.Identity mappings in deep residualnetworks.pp. 630-645,In ECCV,2016.

[0187] 8.Szegedy,C.;Ioffe,S.;Vanhoucke,V.;Alemi,A.A.Inception-v4,inception-resnet and the impact ofresidual connections on learning.InAAAI,2017.

[0188] 9.Szegedy,C.;Liu,W.;Jia,Y.;Sermanet,P.;Reed,S.;Anguelov,D.;Erhan,D.;Vanhoucke, V.;Rabinovich,A.Going deeper with convolutions.pp.1-9,In CVPR,2015.

[0189] 10.Szegedy,C.;Vanhoucke,V.;Ioffe,S.;Shlens,J.;Wojna,Z.Rethinking theinception architecture for computer vision.pp.2818-2826,In CVPR,2016.

[0190] 11.Ioffe,S.;Szegedy,C.Batch normalization:Accelerating deep networktraining by reducing internal covariate shift.In arXiv:1502.03167,2015.

[0191] 12.Carreira,J.;Madeira,H.;Silva,J.G.Xception:A technique for theexperimental evaluation of dependability in modern computers.Transactions onSoftware Engineering,vol.24,no. 2,pp.125-136,1998.

[0192] 13.Howard,A.G.;Zhu,M.;Chen,B.;Kalenichenko,D.;Wang,W.;Weyand,M.;Andreetto, M.;Adam,H.Mobilenets:Effifi-cient convolutional neural networksfor mobile vision applications.arXiv:1704.04861,17Apr 2017.

[0193] 14.Sandler,M.;Howard,A.;Zhu,M.;Zhmoginov,A.;Chen,L.C.Mobilenetv2:Inverted residuals and linear bottlenecks.pp.4510-4520,In CVPR,2018.

[0194] 15.Xie,S.;Girshick,R.;Dollar,P.;Tu,Z.;He,K.Aggregated residualtransformations for deep neural networks.pp.1492-1500,In CVPR,2017.

[0195] 16.Zhang,X.;Zhou,X.;Lin,M.;Sun,J.Shufflflenet:An extremelyeffificient convolutional neural network for mobile devices.arxiv2017.pp.6848-6856,arXiv:1707.01083.

[0196] 17.Ma,N.;Zhang,X.;Zheng,H.T.;Sun,J.Shufflflenet v2:Practicalguidelines for effificient cnn architecture design.In arXiv:1807.11164,2018.

[0197] 18.Hu,J.;Shen,L.;Sun,G.Squeeze-and-excitation networks.In CVPR 2018,pp.7132-7141.

[0198] 19.Li,X.;Wang,W.;Hu,X.;Yang,J.Selective Kernel Networks.In CVPR2019,pp.510-519.

[0199] 20.Woo,S.;Park,J.;Lee,J.Y.;Kweon,I.S.CBAM:ConvolutionalBlockAttention Module.In ECCV2018,pp.3-19.

[0200] 21.Wang,Q.;Liu,S.;Chanussot,J.;Li,X.Scene Classification WithRecurrent Attention of VHR Remote Sensing Images.In IEEE Transactions onGeoscience and Remote Sensing, vol.57,no.2,pp.1155-1167,Feb.2019,doi:10.1109 / TGRS.2018.2864987.

[0201] 22.Tong,W.;Chen,W.;Han,W.;Li,X.;Wang,L.Channel-Attention-BasedDenseNet Network for Remote Sensing Image Scene Classification.In IEEEJournal of Selected Topics in Applied Earth Observations and Remote Sensing,vol.13,pp.4121-4132,2020, doi:10.1109 / JSTARS.2020.3009352.

[0202] 23.Yu,D.;Guo,H.;Xu,Q.;Lu,J.;Zhao,C.;Lin,Y.Hierarchical Attention andBilinear Fusion for Remote Sensing Image Scene Classification,”IEEE Journalof Selected Topics in Applied Earth Observations and Remote Sensing,vol.13,pp.6372–6383,2020,doi: 10.1109 / JSTARS.2020.3030257.

[0203] 24.Alhichri,H.;Alswayed,A.S.;Bazi,Y.;Ammour,N.;Alajlan,N.A.Classification ofRemote Sensing Images Using EfficientNet-B3 CNN ModelWithAttention.In IEEE Access,vol.9, pp.14078-14094,2021,doi:10.1109 / ACCESS.2021.3051085.

[0204] 25.Yang,Y.;Newsam,S.Bag-of-visual-words and spatial extensions forland-use classifification.in Proc.18th SIGSPATIALInt.Conf.Adv.Geograph.Inf.Syst.,2010,pp. 270–279.

[0205] 26.Zou,Q.;Ni,L.;Zhang,T.;Wang,Q.Deep learning based feature selectionfor remote sensing scene classifification.IEEE Geosci.Remote Sens.Lett.,vol.12,no.11,pp. 2321–2325,Nov.2015.

[0206] 27.Xia,G.S.;Hu,J.;Hu,F.;Shi,B.;Bai,X.;Zhong,Y.;Zhang,L.AID:Abenchmarkdata set for performance evaluation of aerial scene classifification.IEEETrans.Geosci.Remote Sens.,vol.55,no.7,pp.3965–3981,Jul.2017.

[0207] 28.Cheng,G.;Han,J.;Lu,X.Remote sensing image scene classifification:Benchmark and state ofthe art.Proc.IEEE,vol.105,no.10,pp.1865–1883,Oct.2017.

[0208] 29.F.Zhao et al.,“A novel two-stage scene classification model basedon Feature var iablesignificancein high-resolution remote sensing,”GeocartoInt.,to be published,doi: 10.1080 / 10106049.2019.1583772.

[0209] 30.Y.Liu,Y.Liu,and L.Ding,“Scene classification based on two-stagedeep feature fusion,” IEEE Geosci.Remote Sens.Lett.,vol.15,no.2,pp.183–186,Feb.2018.

[0210] 31.B.Liu et al.,“Weighted spatial pyramid matching collaborativerepresentation for remote-sensing-image scene classification,”Remote Sens.,vol.11,no.5,2019,Art.no. 518.

[0211] 32.C.Shi,T.Wang and L.Wang,"Branch Feature Fusion Convolution Networkfor Remote Sensing Scene Classification,"IEEE Journal of Selected Topics inApplied Earth Observations and Remote Sensing,vol.13,pp.5194-5210,2020,doi:10.1109 / JSTARS.2020.3018307.

[0212] 33.B.Lietal.,“Aggregated deep fisher feature for VHR remote sensingscene classification,” IEEE J.Sel.Topics Appl.Earth Observ.Remote Sens.,vol.12,no.9,pp.3508–3523,Sep. 2019.

[0213] 34.M.Liu,L.Jiao,X.Liu,L.Li,F.Liu,and S.Yang,“C-CNN:Contourletconvolutional neural networks,”IEEE Trans.Neural Netw.Learn.Syst.,earlyaccess,Jul.21,2020, doi:10.1109 / TNNLS.2020.3007412.

[0214] 35.B.Zhang,Y.Zhang,and S.Wang,“A lightweight and discriminative modelfor remote sensing scene classifification with multidilation pooling module,”IEEE J.Sel.Topics Appl. Earth Observ.Remote Sens.,vol.12,no.8,pp.2636–2653,Aug.2019.

[0215] 36.N.He,L.Fang,S.Li,A.Plaza,and J.Plaza,“Remote sensing sceneclassification using multilayer stacked covariance pooling,”IEEETrans.Geosci.Remote Sens.,vol.56,no.12, pp.6899–6910,Dec.2018.

[0216] 37.H.Sun,S.Li,X.Zheng,and X.Lu,“Remote sensing scene classificationby gated bidirectional network,”IEEE Trans.Geosci.Remote Sens.,vol.58,no.1,pp.82–96,Jan. 2020.

[0217] 38.X.Lu,H.Sun,and X.Zheng,“A feature aggregation convolutional neuralnetwork for remote sensing scene classification,”IEEE Trans.Geosci.RemoteSens.,vol.57,no.10,pp. 7894–7906,Oct.2019.

[0218] 39.N.He,L.Fang,S.Li,J.Plaza,and A.Plaza,“Skip-connected covariancenetwork for remote sensing scene classification,”IEEE Trans.NeuralNetw.Learn.Syst.,vol.31,no.5, pp.1461–1474,May 2020.

[0219] 40.G.Cheng,C.Yang,X.Yao,L.Guo,and J.Han,“When deep learning meetsmetric learning:Remote sensing image scene classification via learningdiscriminative CNNs,” IEEE Trans.Geosci.Remote Sens.,vol.56,no.5,pp.2811–2821,May 2018.

[0220] 41.Y.Boualleg,M.Farah,and I.R.Farah,“Remote sensing sceneclassification using convolutional features and deep forest classifier,”IEEEGeosci.Remote Sens.Lett.,vol.16, no.12,pp.1944–1948,Dec.2019.

[0221] 42.J.Xie,N.He,L.Fang,and A.Plaza,“Scale-free convolutional neuralnetwork for remote sensing scene classification,”IEEE Trans.Geosci.RemoteSens.,vol.57,no.9,pp. 6916–6928,Sep.2019.

[0222] 43.W.Zhang et al.,“Remote sensing image scene classification usingCNN-CapsNet,”Remote Sens.,vol.11,no.5,2019,Art.no.494.

[0223] 44.J.Li et al.,"Deep discriminative representation learning withattention map for scene classification",Remote Sens.,vol.12,no.9,2020.

[0224] 45.P.Yan,F.He,Y.Yang,andF.Hu,“Semi-supervised representation learningfor remote sensing image classification based on generative adversarialnetworks,”IEEE Access,vol.8,pp. 54135–54144,Mar.2020.

[0225] 46.C.Wang et al.,“Multiple resolution block feature for remote-sensing scene classification,”Int.J.Remote Sens.,vol.40,no.18,pp.6884–6904,2019.

[0226] 47.X.Liu,Y.Zhou,J.Zhao,R.Yao,B.Liu,and Y.Zheng,“Siamese convolutionalneural networks for remote sensing scene classification,”IEEE Geosci.RemoteSens.Lett.,vol. 16,no.8,pp.1200–1204,Aug.2019.

[0227] 48.Y.Zhou et al.,“Remote sensing scene classification based onrotation-invariant feature learning and joint decision making,”EURASIPJ.Image Video Process.,vol.2019,no.1, pp.1–11,2019

[0228] 49.X.Lu et al.,“Bidirectional adaptive feature fusion for remotesensing scene classification, Neurocomputing,vol.328,pp.135–146,2019.

[0229] 50.Y.Liu et al.,“Scene classification based on multiscaleconvolutional neural network,”IEEE Trans.Geosci.Remote Sens.,vol.56,no.12,pp.7109–7121,2018.

[0230] 51.R.Cao,L.Fang,T.Lu,and N.He,“Self-attention-based deep featurefusion for remote sensing scene classification,”IEEE Geosci.RemoteSens.Lett.,to be published,doi: 10.1109 / LGRS.2020.2968550.

[0231] 52.W.Li et al.,“Classifification ofhigh-spatial-resolution remotesensing scenes method using transfer learning and deep convolutional neuralnetwork,”IEEE J.Sel.Topics Appl.Earth Observ.Remote Sens.,vol.13,pp.1986–1995,May 2020,doi: 10.1109 / JSTARS.2020.2988477.

[0232] 53.C.Xu,G.Zhu,and J.Shu,“A lightweight intrinsic mean for remotesensing classifification with lie group kernel function,”IEEE Geosci.RemoteSens.Lett.,early access,Jul.17, 2020,doi:10.1109 / LGRS.2020.3007775。

Claims

1. A method for remote sensing scene image classification based on mixed attention of packets, characterized in that: The method specifically comprises the following steps: Step 1: obtaining a hyperspectral image; Step 2: establishing a lightweight convolutional neural network model based on mixed attention; the specific process is as follows: The lightweight convolutional neural network model based on mixed attention comprises an input layer, a first convolutional layer, a second convolutional layer, a first channel grouping mixed attention module, a second channel grouping mixed attention module, a third channel grouping mixed attention module, a fourth channel grouping mixed attention module, a fifth channel grouping mixed attention module, a sixth channel grouping mixed attention module, a global average pooling, an FC full connection layer, and a SoftMax function classifier; Each channel grouping mixed attention module in the first channel grouping mixed attention module, the second channel grouping mixed attention module, the third channel grouping mixed attention module, the fourth channel grouping mixed attention module, the fifth channel grouping mixed attention module and the sixth channel grouping mixed attention module comprises a channel attention module and a spatial attention module; The channel attention module comprises an input layer, a global average pooling, a third convolutional layer, a Relu activation function layer, a fourth convolutional layer, and a Sigmoid activation function ζ(·) layer; The third convolutional layer has a kernel size of 1×1; and the fourth convolutional layer has a kernel size of 1×1; The spatial attention module comprises an input layer, a fifth convolutional layer, and a Sigmoid activation function ζ(·) layer; The fifth convolutional layer has a kernel size of 5×5; The features output by the channel attention module and the features output by the spatial attention module are fused to obtain the output of the channel grouping mixed attention module; Step 3: inputting the hyperspectral image into the established lightweight convolutional neural network model based on mixed attention to obtain a trained lightweight convolutional neural network model based on mixed attention; Step 4: inputting a to-be-detected hyperspectral image into the trained lightweight convolutional neural network model based on mixed attention to obtain a classification result; The connection relationship of the lightweight convolutional neural network model based on mixed attention is as follows: The input layer is connected with the first convolutional layer, the first convolutional layer is connected with the second convolutional layer, the second convolutional layer is connected with the first channel grouping mixed attention module, the first channel grouping mixed attention module is connected with the second channel grouping mixed attention module, the second channel grouping mixed attention module is connected with the third channel grouping mixed attention module, the third channel grouping mixed attention module is connected with the fourth channel grouping mixed attention module, the fourth channel grouping mixed attention module is connected with the fifth channel grouping mixed attention module, the fifth channel grouping mixed attention module is connected with the sixth channel grouping mixed attention module, the sixth channel grouping mixed attention module is connected with the global average pooling, the global average pooling is connected with the FC full connection layer, and the FC full connection layer is connected with the SoftMax function classifier. 2.The method of claim 1, wherein: The connection relationship of the channel attention module is as follows: Let the input features be X = [x1, x2,..., x C ], where xi represents the feature of the i-th channel; 1≤i≤C; The input feature X is subjected to a global average pooling operation for spatial compression to obtain a feature where x is a feature of the i-th channel i the result M after global average pooling i is represented as follows: In formula (2), H, W respectively represent height and width of the feature x i i (m, n) represents the feature of the i-th channel at the corresponding spatial position (m, n).​ features inputting a third convolutional layer, the third convolutional layer outputting an inputting a Relu activation function layer, the Relu activation function layer outputting an inputting a fourth convolutional layer, the fourth convolutional layer outputting features ; wherein denote weights of the third convolutional layer, denote weights of the fourth convolutional layer, R(·) denotes an activation function Relu; characteristics channel attention features ζ(·) represents the activation function Sigmoid; Finally pass through channel attention feature The input feature X = [x1, x2,..., x C ] is calibrated to obtain ; 3.The method of claim 2, wherein: The connection relationship of the spatial attention module is as follows: Let the input features be X = [x 1,1 ,x 1,2 ,...,x m,n ,...,x H,W ], The features represent the corresponding spatial locations (m, n), where m ∈ {1, 2, ..., H} and n ∈ {1, 2, ..., W}; H and W represent the features x, respectively. m,n Height and width; The input features pass through a fifth convolutional layer, and output features characteristics The calculation process is as follows: In formula (4), * represents convolution, f sq represents that the convolution kernel size of the fifth convolution layer is 5x5, and the channel is 1; represents a linear combination of all channels at spatial location (m, n); characteristics spatial attention features through a sigmoid activation function ζ(·) Finally through spatial attention features The input features X = [x 1,1 ,x 1,2 ,...,x m,n ,...,x H,W ] are calibrated to obtain 4. The method of claim 3, wherein the method further comprises: The connection relationship of each channel grouping mixed attention module in the first channel grouping mixed attention module, the second channel grouping mixed attention module, the third channel grouping mixed attention module, the fourth channel grouping mixed attention module, the fifth channel grouping mixed attention module and the sixth channel grouping mixed attention module is as follows: Let input features First group the input features along the channel dimension to get The four grouped features X1, X2, X3, X4 are input into the channel grouping mixed attention module to respectively strengthen the channel and spatial dimensions of the features to obtain Finally, the resulting features Fusion along the channel direction to obtain output features Y; The specific process is as follows: In the formula, represents the importance of the i-th channel at the spatial coordinate position (m, n), and Θ represents the fusion of features in the channel dimension.

5. The method of claim 4, wherein the method further comprises: The global average pooling connects a FC full connection layer, and the FC full connection layer is connected to an output result of the global average pooling Classification is performed to obtain Q, The classification number of the FC full connection layer is N; The FC full connection layer connects a SoftMax function classifier, and an output result of the SoftMax function classifier is The output result S is expressed as follows: wherein represents the kth element in Q represents the kth element in Q [k] represents the kth element in Q 6. The method of claim 5, wherein the method further comprises: The step two establishes the light convolutional neural network model based on mixed attention, and cross entropy is used as a loss function; Assume where y is the encoding of the input sample label, and the loss function is given by In the formula, N represents the number of categories of classification, The output result of Softmax is represented, and the input sample label adopts one-hot coding rule.

Images