A Remote Sensing Image Scene Classification Method Integrating Head-to-Tail Attention and GRU Network

Abstract

This invention discloses a remote sensing image scene classification method integrating head-and-tail attention and GUR networks. The steps are as follows: The input remote sensing image is weighted according to its channel dimension; the high-order features extracted by CNNs are then weighted according to their spatial dimension; the features, after global joint correction by the head channel attention module and the tail spatial attention module, are processed by global average pooling to obtain a 2048-dimensional feature vector, which is then connected to a fully connected layer with 512 nodes. Finally, the softmax function is used to predict the class probability of the output image. This remote sensing image scene classification method integrating head-and-tail attention and GUR networks, through two attention modules with a joint global structure and the GRU gated unit algorithm within the modules, enables the entire network to more effectively identify remote sensing image features, thereby generating importance weights for different channels and different spatial regions. It has high practicality and application potential in improving the discriminative power of feature representation.

Description

Technical Field

[0001] This invention belongs to the technical field of image scene classification, and particularly relates to a remote sensing image scene classification method that integrates head-tail attention and GRU network. Background Technology

[0002] The rapid development of high-resolution remote sensing satellite technology has created a wealth of diverse remote sensing data. The increasing volume and variety of high-resolution and multi-channel remote sensing image data have propelled remote sensing data processing into the era of big data. Because high-resolution and multi-channel remote sensing images store a wealth of information on the spatial structure and channel layout of ground features, effectively identifying and interpreting these images has become a popular research area. Compared to medium- and low-resolution and three-channel natural images, high-resolution and multi-channel remote sensing images contain richer textures, details, and spatial features of ground features. This not only provides abundant feature information for the automatic interpretation of remote sensing images but also presents more requirements and challenges.

[0003] In recent years, remote sensing image scene classification has become a significant and dynamic research topic. Remote Sensing Image Scene Classification (RSISC) is a task in remote sensing image interpretation that aims to automatically classify the scene content of remote sensing images by mapping each image to a pre-defined category label based on the high-level semantic information contained within the image. Currently, research methods for remote sensing image classification can be mainly divided into two categories: those based on handcrafted features and those based on learned features. These two methods differ significantly in their representation of image features. Especially after the introduction of the AlexNet convolutional neural network, feature learning-based methods have gradually emerged. These methods, represented by Convolutional Neural Networks (CNNs), excel in extracting high-order semantic features and have thus gradually become the mainstream approach for remote sensing image classification. In contrast, methods based on handcrafted feature extraction are gradually being replaced.

[0004] Although CNN-based neural network architectures have become indispensable methods for classification and recognition tasks in computer vision, the significant differences in imaging structure and characteristics between natural scene images and remote sensing images mean that deep network architectures designed for natural scene images cannot be directly applied to the Relative Scene-Sensitive Image Classification (RSISC) problem. In RSISC, high-altitude overhead imaging of remote sensing images is characterized by its wide imaging range, large spatial scale variations, and complex semantic content, which pose challenges to the application of CNN-based neural network architectures in RSISC. In remote sensing image scene classification, the richness of imaging targets and the diversity of semantic content lead to significant differences within the same category and high similarity between different categories, thus posing a considerable challenge to scene classification tasks. Summary of the Invention

[0005] Based on the shortcomings of the existing technology, the technical problem solved by the present invention is to provide a remote sensing image scene classification method that integrates head-tail attention and GRU network. By using two attention modules with a joint global structure and the GRU gated unit algorithm inside the module, the entire network can more effectively identify remote sensing image features, thereby generating importance weights for different channels and different spatial regions. It has high practicality and application potential in improving the discriminative ability of feature representation.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0007] The remote sensing image scene classification method of the present invention, which integrates head-and-tail attention and GRU network, includes the following steps:

[0008] Step S1: Adjust the channel-dimensional weights of the input remote sensing image in the channel dimension;

[0009] Step S2: After extracting high-order features from CNNs, perform spatial dimension reweighting;

[0010] Step S3: The features after global joint correction by the first channel attention module and the last spatial attention module are used to obtain a 2048-dimensional feature vector by global average pooling. After being connected to a fully connected layer with 512 nodes, the class probability of the output image is predicted by the softmax function.

[0011] Preferably, in step S1, the image data corrected by the first channel attention module is further input into the VGG-16 convolutional neural network model to obtain the high-order features extracted by convolution.

[0012] Optionally, in step 2, the high-order feature 7×7×2048 is represented as a prior and input into a recurrent neural network with a gated recurrent unit (GRU) as the basic unit. The output is the importance weights of different spaces in the tail, thereby enhancing the significant spatial regions and suppressing the background regions.

[0013] Based on the above, the remote sensing image scene classification method integrating head-tail attention and GRU network of the present invention has at least the following beneficial effects:

[0014] 1. The method of this invention is based on the beginning and end of high-order features extracted by CNNs, combined with the context information extraction capability of the GRU gating unit module in a recurrent neural network. It designs a channel-level attention module before the CNNs and a spatial-level attention module after the CNNs extract high-order features. Through two attention modules with a joint global structure and the GRU gating unit algorithm within each module, the entire network can more effectively identify remote sensing image features, thereby generating importance weights for different channels and different spatial regions. The network structure of this module ensures that important salient regions receive more attention, while less important features in the background are weakened. This method has high practicality and application potential in improving the discriminative power of feature representation.

[0015] 2. By leveraging the different levels of importance represented by the channel and spatial dimensions, a channel-dimensional attention module (CAMB) and a spatial-dimensional attention module (SAMA) are designed before and after the high-order features extracted by the standard convolutional neural network. Utilizing the context information extraction capability of the recurrent neural network gated unit (GRU), the importance weights of different channel and spatial dimension features are learned and output sequentially. This approach focuses more on salient features in remote sensing images while ignoring non-salient features, thereby improving the discriminative power of feature representations. The proposed remote sensing image scene classification method module, which integrates head-and-tail attention and GRU networks, can be connected to any standard convolutional neural network, and the entire network structure can be trained end-to-end. Comprehensive comparative experiments were conducted on three challenging public datasets: AID, UC-Merced, and HSRS-SC. The highest classification accuracies obtained were 97.68%, 99.41%, and 98.38%, respectively. The results demonstrate that the algorithm of this invention can effectively identify remote sensing scenes and achieve competitive classification results.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, the following detailed description is provided in conjunction with preferred embodiments and accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below.

[0018] Figure 1 This is an overall framework diagram of the remote sensing image scene classification method that integrates head-tail attention and GRU network of the present invention;

[0019] Figure 2 This is a structural diagram of the GRU channel attention module;

[0020] Figure 3 This is a structural diagram of the GRU spatial attention module;

[0021] Figure 4 It is the AID dataset;

[0022] Figure 5 It is the UCMerced dataset;

[0023] Figure 6 It is the HSRS-SC dataset. Detailed Implementation

[0024] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this specification. These embodiments illustrate the principles of the invention, and other aspects, features, and advantages of the invention will become apparent from this detailed description. In the accompanying drawings, the same or similar parts in different figures are indicated by the same reference numerals.

[0025] The following reference Figure 1-6 The remote sensing image scene classification method integrating head-tail attention and GRU network of the present invention is described in detail.

[0026] like Figure 1 As shown, taking the HSRS-SC dataset and the classic VGG-16 convolutional model as examples, the input remote sensing image size is 224×224×48, and the high-level feature size extracted by VGG-16 is 7×7×2048. The channel dimension of the input remote sensing image can be regarded as 48 different feature types, and the feature types they rely on are different for different remote sensing image classification tasks. Therefore, the feature representation of the channel dimension plays an important role in accurately characterizing the identity information of remote sensing images. For the 7×7×2048 high-order features extracted by VGG-16, the physical meaning of the spatial dimension can be regarded as 7×7 different types of spatial features.

[0027] In summary, to further improve the discriminative power of remote sensing image feature representation, it is necessary to further modify the standard CNN in two dimensions: the channel dimension of the input remote sensing image and the spatial dimension after the high-order features extracted by CNNs. This involves increasing the weight of important features and decreasing the weight of unimportant features. A head-and-tail attention module is then developed based on both the input remote sensing image and the high-order features extracted by CNNs.

[0028] First, the input remote sensing image is weighted according to the channel dimension. Then, the high-order features extracted by CNNs are weighted again according to the spatial dimension. The framework diagram of the method is as follows: Figure 1 As shown. Taking the VGG-16 convolutional neural network model and the HSRS-SC dataset as examples, the last high-order feature layer is the convolutional layer with an input dimension of 224×224×48 and the last convolutional layer with a size of 7×7×2048 extracted by VGG-16, which is connected to the channel dimension attention module and the spatial dimension attention module in sequence.

[0029] The first channel attention module takes 48 224×224-dimensional vectors (424×224-dimensional) obtained by expanding each channel of the input remote sensing image and performs global average pooling (GAP) to obtain 1×1×48 channel vectors. These vectors are then sequentially input into a recurrent neural network (RNN) using Gated Recurrent Units (GRUs) as basic units to extract the contextual relationships between channels and generate importance weights for each channel. The channel importance of the input remote sensing image is then weighted and corrected. The image data corrected by the first channel attention module is further input into a VGG-16 convolutional neural network model to obtain high-order features extracted by convolution. Using the 7×7×2048 high-order feature representation as a priori, this is input into the RNN using GRUs as basic units, outputting the importance weights for different spatial regions. This achieves enhancement of salient spatial regions and suppression of background regions. Finally, the features, after global joint correction by the first channel attention module and the last spatial attention module, are used to obtain a 2048-dimensional feature vector through global average pooling. This vector is then connected to a fully connected layer with 512 nodes, and the class probability of the output image is predicted using the softmax function.

[0030] In the input remote sensing feature image, different channels represent different types of visual features. Taking VGG-16 as an example, the input remote sensing feature image contains 48 channels, which can be regarded as 48 different visual features. Different features play different roles in expressing the semantic content of the remote sensing image. The purpose of the first channel attention module is to take the input remote sensing feature image as a basis and, according to the importance of different channels, further generate importance weights for each channel, thereby enhancing salient features and ignoring non-salient features. The specific network structure of the first channel attention module is as follows: Figure 2 As shown.

[0031] The input remote sensing image F (224×224×48) is expanded according to the channel dimension to obtain 48 channel feature representations with a dimension of 224×224: F1={f1,f2,…,f 48}, f i ∈d 224×224 The weight parameters of the channel attention module are generated through random initialization, which learns to generate importance weights for each channel. While this random initialization achieves the goal of correcting the importance of different channels, it neglects the interrelationships between channels and underutilizes the prior information of each channel. Recurrent neural networks (RNNs), as an important means of processing sequence information, can effectively extract contextual relationships from sequence information, providing a superior solution for further determining the interrelationships and importance weights between different channels. Therefore, to more accurately generate importance weights between different channels, this invention uses the feature representations of each channel as prior information, which are sequentially input into a RNN with GRU as the basic unit. GRU is a type of RNN with fewer parameters and good performance in many sequence relationship extraction tasks. By extracting the contextual relationships between different channels, modified feature representations F2 = {f′1, f′2, ..., f′} are generated for each channel. 48}, f′ i ∈d 224×224 The original 48 channel features are represented by a global average pooling (GAP) to obtain a 1×48 vector. The sequence input to the GRU unit is then downsampled by a factor of 2 and activated by the ReLU function. If the number of channels is less than 12, the process is performed by upsampling by a factor of 2 and 4 of the original number of channels. This paper takes upsampling and a number of channels of 3 as an example. The process can be seen in equation (1) and formula (2):

[0032] F gru1 =Relu{GRU[GAP(F1)×2]} (1)

[0033] F gru2 =Relu[GRU(F gru1 (2)×2]

[0034] Among them, feature F gru1 ∈d 6×1 Feature F gru2 ∈d 12×1 .

[0035] The Fgru2 obtained from the upsampling by the GRU is input into the downsampling process using the GRU as the unit. This process can be seen in equation (3) and formula (4):

[0036] F gru3 =Relu[GRU(F gru2 )÷2] (3)

[0037] F gru4 =Relu[GRU(F gru3 (4) ÷ 2

[0038] Among them, feature F gru3 ∈d 6×1 Feature F gru4 ∈d 3×1 .

[0039] The obtained Fgru4 feature is further processed with a fully connected layer and a sigmoid activation function, and the value of the sigmoid function output is used as the importance weight of each channel. This process can be seen in equation (5):

[0040] W HC_Attention =Sigmoid*(F gru4 (5)

[0041] Among them, the weights W of the fully connected layer HC_Attention ∈d 3×1 .

[0042] The feature representations of each channel are mapped to 1-dimensional vectors. Then, these 48 1-dimensional vectors are multiplied by the input original remote sensing image. The result is used as the importance weight for each channel. The importance weight W for each channel is then obtained. HC_Attention Then, borrowing from the residual connections in ResNet, the features with corrected channel weights are added to the channels of the original remote sensing image. This ensures that even if the channel attention module is inactive, the entire network retains its original feature information without loss, thus correcting the importance of each channel. The entire calculation process of the first-level channel attention module can be represented as follows:

[0043] F2 = F1 + F1 * W HC_Attention (6)

[0044] Among them, W HC_AttentionThe dimensions have been adjusted to 1×1×3, representing element-wise multiplication.

[0045] The wide imaging range and complex spatial dimensions of remote sensing images make it crucial to effectively represent salient spatial regions while ignoring background areas for accurate feature identification. The purpose of the tail spatial attention module is to further extract the contextual relationships between different spatial region features mapped from the original hyperspectral image after the high-order features extracted by CNNs, generating importance weights for each region to focus on salient areas and ignore background areas. Similar to the network structure of the head channel-dimensional attention module, the network structure of the tail spatial-dimensional attention module is as follows: Figure 3 As shown.

[0046] The features F2 (224×224×48) corrected by the first channel dimension attention module are input into CNNs for convolutional feature extraction, finally obtaining the high-order features F3 (7×7×48), which are then input into the last spatial dimension attention module. The spatial feature representation (1×1) of the image in the high-level feature F3 can be regarded as the local feature representation of the original image (32×32). Therefore, the high-order feature representations F4 = {f1, f2, ..., f4} of 49 (7×7) regions in the image can be obtained. 49}, f i ∈d 1×1×2048 To improve the discriminative power of feature representations, a tail-level spatial attention module was designed, with a structure similar to the head-level channel-level attention module. To better explore the interrelationships and weight allocations between different image regions, this invention introduces a GRU recurrent neural network as a contextualizer for extracting sequence information from different regions. The feature representations of 49 different regions are sequentially input into the recurrent neural network composed of GRUs, and connected to a shared fully connected layer to obtain the weight parameters of the spatial attention module. Finally, it is connected to a classification fully connected layer, and the importance weights W of different regions are output through a softmax function. TS_Attention Unlike the initial channel-dimensional attention module, the higher-order features of F3 obtained from CNNs are processed in the channel dimension using max pooling and average pooling to obtain the corrected F3. AM The input is then fed into the GRU module and downsampled once to obtain F. gru5 and F gru6 This calculation is as shown in equations (7) and (8):

[0047] F gru5 =Relu[GRU(F AM )÷4] (7)

[0048] F gru6 =Relu[GRU(F gru5[ )×4] (8)

[0049] Among them, feature F gru5 ∈d 12×1 Feature F gru6 ∈d 49×1 .

[0050] The final features F extracted from the obtained GRU unit modules gru6 Connected to a shared fully connected layer, the importance weights W for different regions are output through the softmax function. TS_Attention The calculation for this process is as shown in equation (9):

[0051] W TS_Attention =Sigmoid*(F gru6 (8)

[0052] Among them, W TS_Attention The size has been adjusted to (7×7×1), where * represents element-wise multiplication.

[0053] The importance weights of the tail spatial dimension attention module are element-wise multiplied with the input high-order features to obtain the final high-level features F5 after the tail spatial attention module correction. The entire calculation process is shown in Equation (10):

[0054] F5 = F4 + F4 * W TS_Attention (10)

[0055] Among them, W TS_Attention The reshape is adjusted to (7×7×1) to represent element-wise multiplication.

[0056] To verify the effectiveness of the proposed dual attention module, experiments were conducted on three remote sensing scene classification datasets of different sizes (HSRS-SC, AID, and UC-Merced).

[0057] AID (Aerial Image Dataset) is a large-scale aerial image dataset constructed by Wuhan University. The data, sourced from Google Earth, contains 10,000 aerial scene images, each 600×600 pixels in size, with a spatial resolution ranging from 1 to 8 meters. The dataset is divided into 30 categories, each containing between 220 and 420 images. The images were acquired at different times and in different seasons, thus increasing the intra-class variability of the AID dataset. Figure 4 As shown.

[0058] The UC-Merced (UC Merced Land Use Dataset) dataset, sourced from the U.S. Geological Survey, includes 21 categories such as aircraft and golf courses. Each category contains 100 images, with a size of 256×256 pixels and a resolution of 0.3m. Examples of images for each category are shown below. Figure 5 As shown.

[0059] HSRS-SC (hyperspectral remote sensing dataset for scene classification) comes from aerial data of the Heihe River eco-hydrological remote sensing experiment. It is the largest known hyperspectral scene classification dataset, formed through calibration coefficient correction and atmospheric correction. HSRS-SC consists of 1385 images in 5 categories, with high spatial resolution (1m), a wide wavelength range (380-1050nm), and rich spatial and spectral information on ground features. The typical topography of the Heihe River basin includes mountains, rivers, farmland, villages, and cities. Therefore, this invention selects the five most representative scene categories (farmland, buildings, cities, unused areas, and water bodies) as semantic categories. To verify the applicability of the proposed model and maintain consistency with the original paper, experiments were conducted using 3-band and 48-band data, respectively. Figure 6 As shown.

[0060] To facilitate experimental comparison with existing methods, a portion of data was randomly selected from three datasets for training and testing. In dataset AID, 20% and 50% of the data were randomly selected as training sets, and the remaining 80% and 50% were selected as testing sets. In UCMerced, 20% and 80% of the data were randomly selected as training sets, and the remaining 80% and 20% were selected as testing sets. In dataset HSRS-SC, 50 and 100 samples were selected for training, respectively, and the remainder were selected as testing sets.

[0061] In this invention, the scene classification model was tested on a personal computer equipped with a Windows 10 operating system, an Intel i10-8500 CPU, 32GB of RAM, and an NVIDIA RTX 3060 graphics card. The algorithm proposed in this invention is implemented using the Tensorflow framework. The parameters of the two pre-trained CNNs (VGG-16 and Xception) models were obtained through training on the large-scale ImageNet dataset. All parameters were downloaded from https: / / storage.googleapis.com / tensorflow / keras-applications.

[0062] In this invention, the Adam stochastic gradient descent optimizer is used to train the entire network, and experiments are conducted on three datasets. The resizing resolution and number of channels of these datasets are listed in Table 1, while the batch size for data training is given in Table 2. The training iterations for all three datasets are 50 epochs, with a learning rate of 1e-4. Notably, on the HSRS-SC dataset, to ensure consistency with the original cited paper, experiments were conducted using 3 channels (see light range 19 / 13 / 7) and 48 full channels, respectively. For the acquisition of high-order features, this experiment mainly uses VGG-16 and Xception as baseline convolutional neural networks, combined with the proposed dual attention module. During the testing phase, the parameters of the first 13 layers of the VGG-16 network structure are used as pre-training parameters, and the last 3 layers are self-training parameter layers, thus serving as the final high-order feature representation of the remote sensing image; the parameters of the first 102 layers of the Xception network structure are used as pre-training parameters, and the last 30 layers are self-training parameter layers, thus serving as the final high-order feature representation of the remote sensing image.

[0063] The evaluation metrics used in this invention mainly include overall classification accuracy (OA) and confusion matrix (CM). OA is defined as the percentage of correctly classified samples in the test set out of the total test set, and its calculation method is shown in equation (11):

[0064]

[0065] Where T represents the number of samples correctly classified by the model in the test set, and N represents the total number of samples in the test set. In this experiment, each test result was repeated 10 times, and the average accuracy and standard deviation of the 10 tests were used as the final experimental result. CM is a more intuitive way to represent the performance of an algorithm in classification tasks. Its diagonal elements represent the classification accuracy of each category, and the remaining elements amn represent the proportion of the m-th class being misclassified as the n-th class. CM can better illustrate the classification situation between similar and confused categories.

[0066] Table 1. Image size and number of channels for each dataset.

[0067]

[0068] Table 2 Training batches for each dataset

[0069]

[0070] To fully verify the effectiveness of the proposed dual attention modules, this invention designed four sets of comparative experiments to verify the effectiveness of the first channel attention module and the last spatial attention module, respectively. CAMB (Channel Attention Mechanism Before CNN) represents the first channel attention module, and SAMA (Spatial Attention Mechanism After CNN) represents the last spatial attention module. The two types of attention modules were connected to the benchmark networks VGG-16 and Xception, respectively. The experimental results on three datasets are compared in Tables 3 and 4.

[0071] The experimental results comparing the proposed method with the standard model demonstrate the effectiveness of the two attention modules, as analyzed below: First, the two attention modules were combined with two benchmark convolutional neural networks, VGG-16 and Xception, respectively. Experimental results show that both attention modules significantly improve the representational ability of features extracted by deep convolutional neural networks. The experimental results also show that the proposed method achieves significant improvements on all three datasets. This significant performance improvement is primarily attributed to the wide imaging range and complex multi-channel structure of remote sensing images. Therefore, in addition to focusing on salient regions through the dual attention modules at the beginning and end, the GRU units within the attention modules also better understand the correlations between semantic content in remote sensing images. Secondly, Tables 3 and 4 show that a reasonable combination of the positional structure of the dual attention modules at the beginning and end further enhances the discriminative power of feature representation, significantly improving the classification accuracy of the baseline network. Particularly on the UC Merced dataset, the proposed method increased the accuracy of the original baseline network to 99.41%. Finally, in the HSRS-SC dataset, experiments comparing 3-channel and 48-channel models show that the 48-channel model performs better, achieving an accuracy of 98.38%, thanks to the multi-channel imaging characteristics of remote sensing images. In summary, this demonstrates that the beginning and end attention modules have different focuses in feature representation. The first channel attention module focuses on extracting channel features from the original remote sensing image, while the last spatial attention module focuses on extracting features from the wide imaging range of the remote sensing image. The two modules complement and promote each other. A comparison of the three datasets reveals a significant performance gap between the two benchmark networks. Xception's feature representation capability significantly surpasses VGG-16. However, both proposed attention modules are well-suited for use with the two benchmark convolutional neural networks. This demonstrates the applicability of the proposed dual attention module and its ability to be combined with any convolutional neural network. Table 3 compares the OA (%) results of the AID and UC Merced models on the datasets.

[0072]

[0073] Table 4 Comparison of OA (%) results for the HSRS-SC model test on the dataset

[0074]

[0075] To verify the effectiveness of the proposed method on different datasets, comparative experiments were conducted with relevant benchmark methods proposed in recent years on the datasets AID, UC-Merced, and HSRS-SC. The proposed method achieved optimal performance at both 20% and 50% training ratios, with accuracy exceeding existing benchmark methods, reaching approximately 96.20% and 97.68% overall classification accuracy, respectively. Notably, the method AFFJL(res101) proposed in 2022 improved the original best classification performance by approximately 0.46% at a 50% training ratio, achieving state-of-the-art (SOTA). The proposed method achieved accuracies of 92.72% and 99.41% at both training ratios, approaching 100%, significantly improving upon the best performance on these datasets. To verify the impact of channel dimension on remote sensing images, experiments were conducted comparing the model of this invention with 3-channel and 48-channel models. The highest accuracy results for the two channels were 97.68% and 98.36%, respectively, with the 48-channel model showing a 1.32% improvement in accuracy compared to the 3-channel model. This verifies the effectiveness of the method of the present invention, thereby achieving optimal implementation.

[0076] The proposed method achieves significant accuracy improvements on three remote sensing image scene classification datasets: HSRS-SC, AID, and UC-Merced. Specifically, on the HSRS-SC dataset, the method improves accuracy by 1.07% compared to the current state-of-the-art implementation; on the AID dataset, the accuracy improvement is 0.27%; and on the UC Merced dataset, the accuracy improvement is 0.97%. These results demonstrate that the proposed method effectively improves the accuracy of remote sensing image scene classification across different datasets and exhibits good universality and transferability. Combining the proposed dual attention module with the Xception standard convolutional neural network achieves high classification accuracy with the best confusion matrices on the AID and UC-Merced datasets, especially surpassing the best accuracy achieved on the 3-channel dataset in the 48-channel dataset test.

[0077] While the proposed method significantly improves the discriminative power of feature representations on three challenging datasets and achieves good overall classification accuracy, its overemphasis on local features and regions can lead to confusion between images of similar categories. The proposed method achieves high classification accuracy for each category, especially with a 50% training ratio, where the accuracy for each category approaches 1. However, the most significant error is the widespread misclassification of images with a true classification of category 25 (school) as category 8 (Commercial) and category 23 (Resort). This is because the method further emphasizes local regions and features on top of the original feature representation. This leads to an overemphasis on the local similarities between resorts and commercial areas after training on this dataset, resulting in confusion regarding specific information.

[0078] This invention aims to address the problem of low classification accuracy caused by large intra-class differences and high inter-class similarities in remote sensing image scene classification. To this end, a feature representation method integrating a dual attention module (head and tail) and a GRU network is proposed. This method utilizes the high-order features extracted by CNNs to design an attention module at both the head and tail, and combines it with the GRU unit module of a recurrent neural network to extract contextual information, thereby capturing the feature correlations between different channels and the importance between different spatial regions. Experiments demonstrate that this method effectively improves the discriminative power of feature representation, surpassing existing benchmark methods. However, it is also recognized that this method introduces a certain amount of training parameters and requires a certain amount of training data. Furthermore, experiments revealed that classification confusion easily occurs in regions with similar local features. Therefore, future research will explore the use of graph neural networks to enhance the feature extraction capabilities of the model.

[0079] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. It should be noted that this description is intended for those skilled in the art.

[0080] For those who are skilled in the art, several improvements and modifications can be made without departing from the principles of this invention.

[0081] Changes and modifications are also considered to be within the scope of protection of this invention.

Claims

1. A remote sensing image scene classification method integrating head-tail attention and GRU network, characterized in that, Includes the following steps: Step S1: Adjust the channel-dimensional weights of the input remote sensing image in the channel dimension; Step S2: After extracting high-order features from CNNs, perform spatial dimension reweighting; Step S3: The features after global joint correction by the first channel attention module and the last spatial attention module are used to obtain a 2048-dimensional feature vector by global average pooling. After being connected to a fully connected layer with 512 nodes, the class probability of the output image is predicted by the softmax function. By leveraging the different levels of importance represented by the channel dimension and the spatial dimension, channel attention modules and spatial attention modules are designed before and after the high-order features extracted by the standard convolutional neural network. By utilizing the context information extraction capability of the recurrent neural network gating unit, the importance weights of different channel dimension features and spatial dimension features are learned and output sequentially.

2. The remote sensing image scene classification method integrating head-tail attention and GRU network as described in claim 1, characterized in that, In step S1, the image data corrected by the first channel attention module is further input into the VGG-16 convolutional neural network model to obtain the high-order features extracted by convolution.

3. The remote sensing image scene classification method integrating head-and-tail attention and GRU network as described in claim 2, characterized in that, In step 2, the high-order feature 7×7×2048 is represented as a priori and input into a recurrent neural network with a gated recurrent unit (GRU) as the basic unit. The output is the importance weight of different spaces in the tail, thereby enhancing the significant spatial regions and suppressing the background regions.

Images