Remote sensing image classification method and device based on spectral attention network and medium

Abstract

The application provides a remote sensing image classification method based on a spectral-spatial attention network, and relates to the technical field of remote sensing image processing, which comprises the following steps: acquiring a hyperspectral remote sensing satellite image and performing data preprocessing to construct a hyperspectral remote sensing image classification dataset; constructing a classification model DSAFNet based on a double-branch spectral-spatial attention fusion network and taking the model as an initial hyperspectral classification model; training the initial hyperspectral classification model through the hyperspectral remote sensing image classification dataset to obtain a trained classification network model; acquiring a hyperspectral remote sensing image to be classified; inputting the hyperspectral remote sensing image to be classified into the trained classification network model to obtain an image classification result. The technical scheme of the application realizes generation of a hyperspectral remote sensing image classification model with high recognition accuracy and efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a remote sensing image classification method based on a spectral attention network, a device and a medium. BACKGROUND

[0002] Hyperspectral remote sensing images can provide rich spectral and spatial information simultaneously due to their "graph-spectrum integration" characteristics, and have important application value in precision agriculture, environmental monitoring, national defense and soil assessment. Hyperspectral image classification is a key part of remote sensing information processing, and its recognition accuracy directly affects the credibility of subsequent applications. Traditional classification methods, such as support vector machines, K-nearest neighbors, random forests, logistic regression and sparse representation, have shown good classification performance in processing low-dimensional data. However, when dealing with high-dimensional nonlinear characteristics of hyperspectral data, these methods show certain limitations. With the breakthrough progress of deep learning methods in the field of computer vision, this technology has been widely applied in the classification research of hyperspectral remote sensing images. Convolutional neural networks, recurrent neural networks, generative adversarial networks, graph convolutional networks, and stacked autoencoders have been widely used and have achieved remarkable classification results. In particular, CNN technology has made breakthrough progress in the field of HSI classification due to its superiority in spectral and spatial features of hyperspectral remote sensing images. In particular, 2D-CNN, as an effective spatial feature extraction method, has been widely used in hyperspectral remote sensing image classification. For example, Ding et al. introduced an adaptive kernel method combined with 2D-CNN, which improved the classification accuracy by learning the convolution kernel.

[0003] 2D-CNN performs well in the use of spatial information, but it is difficult to fully exploit the discriminative features of the spectral dimension. To overcome this limitation, researchers have proposed 3D-CNN, such as He et al. who further proposed a multi-scale 3S-CNN model aimed at realizing end-to-end integration of spatial and spectral feature learning. Although 3D-CNN has shown strong ability in processing hyperspectral data, it also faces problems such as high computational complexity, sensitivity to noisy bands, and difficulty in capturing long-distance spatial dependencies. With the increasing depth of convolutional neural networks, problems such as gradient vanishing and gradient explosion gradually appear, affecting the training stability and convergence speed of the model. Therefore, researchers have begun to introduce attention mechanisms into hyperspectral classification, highlighting the bands and pixels that contribute more to the classification results through attention mechanisms. For example, the end-to-end multi-level hybrid attention network DMCN and the collaborative attention CNN integrate CNN and attention mechanisms, providing a promising solution to improve the classification results in hyperspectral image analysis.

[0004] In summary, the existing hyperspectral image classification methods based on convolutional neural network and attention mechanism have achieved certain results, but still face problems such as insufficient feature extraction, high computational complexity, sensitivity to spectral redundancy and noise, and the like, which makes the classification effect of the model not ideal. SUMMARY

[0005] The present application aims at: in order to solve the existing hyperspectral image classification method based on convolutional neural network and attention mechanism, there are problems such as insufficient feature extraction, high computational complexity, sensitivity to spectral redundancy and noise, etc., and provides a remote sensing image classification method based on space-spectrum attention network.

[0006] The above-mentioned purpose of the present application is realized by the following technical scheme:

[0007] S1: Obtain hyperspectral remote sensing satellite image and perform data preprocessing, and construct a hyperspectral remote sensing image classification dataset;

[0008] S2: Construct a classification model DSAFNet based on a double-branch spectral-spatial attention fusion network and use it as an initial hyperspectral classification model;

[0009] S3: Train the initial hyperspectral classification model through the hyperspectral remote sensing image classification dataset to obtain a trained classification network model;

[0010] S4: Obtain a hyperspectral remote sensing image to be classified; input the hyperspectral remote sensing image to be classified into the trained classification network model to obtain an image classification result.

[0011] Optionally, step S1 comprises:

[0012] S11: Obtain hyperspectral remote sensing satellite images of different ground object categories, different spatial resolutions, and covering various ground object combinations;

[0013] S12: Perform principal component analysis on the hyperspectral remote sensing satellite images Perform principal component analysis along the spectral dimension to reduce the dimensionality of the original B-band hyperspectral data to M bands to obtain a reduced hyperspectral image, wherein H, W, B and M represent height, width, original spectral band number and reduced spectral band number, respectively;

[0014] S13: Perform zero padding operation and edge expansion on the reduced hyperspectral image, and the edge expansion size is , is the neighborhood window size;

[0015] S14: Take each valid pixel of the hyperspectral image as the center to extract generating N three-dimensional data blocks corresponding to the N spatial positions, where N=H×W, N represents the number of spatial pixels in the hyperspectral image;

[0016] S15: constructing a hyperspectral remote sensing image classification dataset through the three-dimensional data blocks.

[0017] Optionally, the step S2 comprises:

[0018] The initial hyperspectral classification model comprises: an input layer, a spatial feature branch, a spectral feature branch, a global average pooling unit, an adaptive weighted fusion unit, and a fully connected layer;

[0019] The spatial feature branch and the spectral feature branch are both connected to the input layer; the spatial feature branch and the spectral feature branch are both connected to the global average pooling unit; the global average pooling unit, the adaptive weighted fusion unit, and the fully connected layer are sequentially connected in order;

[0020] The spatial feature branch comprises: a first position attention module PAM, a first channel attention module CAM, a depth separable convolution layer with a convolution kernel size of 3×3, a second position attention module PAM, a second channel attention module CAM, and a depth separable convolution layer with a convolution kernel size of 5×5; the first position attention module PAM, the first channel attention module CAM, the depth separable convolution layer with a convolution kernel size of 3×3, the second position attention module PAM, the second channel attention module CAM, and the depth separable convolution layer with a convolution kernel size of 5×5 are sequentially connected in order;

[0021] The spatial information in the hyperspectral image is extracted through the spatial feature branch;

[0022] The spectral feature branch comprises: a convolution module, a bottleneck residual module BRB, and a spectral attention module SAM;

[0023] The convolution module, the bottleneck residual module BRB, and the spectral attention module SAM are sequentially connected in order;

[0024] The spectral information in the hyperspectral image is extracted through the spectral feature branch;

[0025] The global average pooling unit is used for globally and averagely pooling the spatial information and the spectral information to compress the spatial dimension;

[0026] The adaptive weighted fusion unit is used for integrating the features of the pooled spatial information and the pooled spectral information through an adaptive weighted fusion strategy to obtain a combined feature map;

[0027] The fully connected layer is used for dynamically adjusting the output weights of the spatial feature branch and the spectral feature branch in the combined feature map to obtain a final fusion map.

[0028] Optionally, step S3 comprises:

[0029] The processing steps of the position attention module PAM are as follows:

[0030] Let the local features of the hyperspectral image in the hyperspectral remote sensing image classification dataset be ;

[0031] The local features are subjected to nonlinear transformation using convolution by the position attention module PAM to generate three feature maps A, B and C;

[0032] The dimensions of the feature maps A, B and C are reshaped; matrix multiplication is performed between the transposed feature map A and the feature map B, and a position attention map is calculated through a softmax layer; ;

[0033] Matrix multiplication is performed between the feature map C and the position attention map, and the result is reshaped into to obtain a result feature map;

[0034] The result feature map is multiplied by a learnable scale parameter, and an element-wise summation operation is performed with the input feature to obtain the final output of the position attention module PAM .

[0035] Optionally, the convolution module is a 3D convolution module with a convolution kernel size of 3x3x3.

[0036] Optionally, step S3 further comprises:

[0037] The feature maps output by the spatial feature branch and the spectral feature branch are and respectively, and the feature map X and the feature map are spliced along the last dimension to obtain a combined feature map containing double-branch information ;

[0038] The combined feature map is combined through a fully connected layer to generate attention weights ;

[0039] According to the attention weights w, the input features and are weighted and fused to obtain a final fused map .

[0040] Optionally, step S3 further comprises: training the initial hyperspectral classification model using a classification cross-entropy loss function, expressed as:

[0041]

[0042] wherein, denotes the total number of samples in the training sample set, denotes the serial number of the sample in the training sample set, denotes the true label of the nth sample, denotes the predicted label of the nth sample, denotes the loss value between the predicted label and the true label.

[0043] An electronic device comprising a processor, a memory, a user interface and a network interface, the memory being configured to store instructions, the user interface and the network interface being configured to communicate with other devices, and the processor being configured to execute the instructions stored in the memory to cause the electronic device to perform a remote sensing image classification method based on a spectral attention network.

[0044] A computer-readable storage medium storing instructions that, when executed, perform a remote sensing image classification method based on a spectral attention network.

[0045] The technical scheme provided by the present application has the beneficial effects that:

[0046] The two parallel feature extraction branches of the classification network model provided by the present application are respectively used for extracting spatial features and spectral features in the image, thereby improving the discrimination ability of the model; the spatial feature branch is composed of a position attention module, a channel attention module and a depth separable convolution layer in an alternating manner to form a two-layer structure, which effectively enhances the expression ability of image spatial details; the spectral feature branch adopts a spectral attention module based on 3D convolution combined with a bottleneck residual block, and focuses on extracting image spectral information; the spatial and spectral features extracted by the two branches are adaptively integrated through a weighted feature fusion strategy; through the construction of the classification network model, the problems of insufficient utilization of spectral information and difficulty in capturing long-distance spatial dependency in current hyperspectral classification methods are alleviated, and a hyperspectral remote sensing image classification model with high recognition accuracy and efficiency is generated. BRIEF DESCRIPTION OF DRAWINGS

[0047] The present application will be further described below in conjunction with the drawings and examples, wherein:

[0048] Figure 1 is a step diagram in the embodiment of the present application;

[0049] Figure 2 is a network structure framework diagram in the embodiment of the present application;

[0050] Figure 3 is a position attention module PAM structure schematic diagram in the embodiment of the present application;

[0051] Figure 4 Fig. 1 is a schematic diagram of a channel attention module CAM structure in an embodiment of the present application;

[0052] Figure 5 Fig. 2 is a schematic diagram of a bottleneck residual block BRB structure in an embodiment of the present application;

[0053] Figure 6 Fig. 3 is a schematic diagram of a spectral attention module SAM structure in an embodiment of the present application;

[0054] Figure 7 Fig. 4 is a schematic diagram of an electronic device structure in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to have a clearer understanding of the technical features, objectives and effects of the present application, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0056] An embodiment of the present application provides a hyperspectral remote sensing satellite image classification method based on a spatial-spectral attention network.

[0057] Please refer to Figure 1 , Figure 1 Fig. 5 is a flowchart of a hyperspectral remote sensing satellite image classification method based on a spatial-spectral attention network in an embodiment of the present application, which comprises:

[0058] S1: Obtain a hyperspectral remote sensing satellite image and perform data preprocessing to construct a hyperspectral remote sensing image classification dataset;

[0059] As an embodiment, the hyperspectral classification dataset is divided into a training set and a test set, and both the training set and the test set contain different ground object categories and different spatial resolutions to increase the representativeness of the dataset.

[0060] S2: Construct a classification model DSAFNet based on a double-branch spectral-spatial attention fusion network and use it as an initial hyperspectral classification model;

[0061] S3: Train the initial hyperspectral classification model through the hyperspectral remote sensing image classification dataset to obtain a trained classification network model;

[0062] S4: Obtain a hyperspectral remote sensing image to be classified; input the hyperspectral remote sensing image to be classified into the trained classification network model to obtain an image classification result.

[0063] Step S1 comprises:

[0064] S11: Obtain a hyperspectral remote sensing satellite image covering multiple ground object combinations with different ground object categories and different spatial resolutions;

[0065] S12: Perform data preprocessing on the hyperspectral remote sensing satellite image perform principal component analysis processing along the spectral dimension to reduce the dimensionality of the original hyperspectral data of B bands to M bands, to obtain a reduced-dimension hyperspectral image, wherein H, W, B, and M represent height, width, original spectral band number, and reduced-dimension spectral band number, respectively;

[0066] S13: performing zero padding operation and edge extension on the reduced-dimension hyperspectral image, and the edge extension size is , is a neighborhood window size;

[0067] S14: extracting a three-dimensional neighborhood cube of centered on each valid pixel of the hyperspectral image, to generate a three-dimensional data block corresponding to N spatial positions, wherein N = H x W, and N represents the number of spatial pixels in the hyperspectral image;

[0068] S15: constructing a hyperspectral remote sensing image classification data set through the three-dimensional data block.

[0069] As an embodiment, the acquired hyperspectral remote sensing image is subjected to PCA dimension reduction and spatial block processing to obtain a final hyperspectral classification data set.

[0070] Step S2 includes:

[0071] The initial hyperspectral classification model includes: an input layer, a spatial feature branch, a spectral feature branch, a global average pooling unit, an adaptive weighted fusion unit, and a fully connected layer;

[0072] The spatial feature branch and the spectral feature branch are both connected to the input layer; the spatial feature branch and the spectral feature branch are both connected to the global average pooling unit; the global average pooling unit, the adaptive weighted fusion unit, and the fully connected layer are sequentially connected in order;

[0073] The spatial feature branch includes: a first position attention module PAM, a first channel attention module CAM, a depth separable convolution layer with a convolution kernel size of 3x3, a second position attention module PAM, a second channel attention module CAM, and a depth separable convolution layer with a convolution kernel size of 5x5; the first position attention module PAM, the first channel attention module CAM, the depth separable convolution layer with a convolution kernel size of 3x3, the second position attention module PAM, the second channel attention module CAM, and the depth separable convolution layer with a convolution kernel size of 5x5 are sequentially connected in order;

[0074] The spatial information in the hyperspectral image is extracted through the spatial feature branch;

[0075] The spectral feature branch includes: a convolution module, a bottleneck residual module BRB, and a spectral attention module SAM;

[0076] The convolution module, the bottleneck residual module BRB and the spectral attention module SAM are sequentially and orderly connected;

[0077] The spectral feature branch is used for extracting spectral information in the hyperspectral image;

[0078] The global average pooling unit is used for globally and averagely pooling the spatial information and the spectral information to compress the spatial dimension;

[0079] The adaptive weighted fusion unit is used for integrating the features of the pooled spatial information and the pooled spectral information through an adaptive weighted fusion strategy to obtain a combined feature map;

[0080] The fully connected layer is used for dynamically adjusting the output weights of the spatial feature branch and the spectral feature branch in the combined feature map to obtain a final fusion map.

[0081] As an embodiment, the initial hyperspectral classification model comprises: a spatial feature branch, which is composed of a position attention module PAM, a channel attention module CAM and a depth separable convolution layer in an alternating manner to form a two-layer structure, and which effectively enhances the expression ability of image spatial details; a spectral feature branch, which adopts a spectral attention module SAM based on a 3D convolution, and combines a bottleneck residual block to focus on extracting image spectral information; and a weighted feature fusion strategy is used to adaptively integrate the spatial and spectral features extracted by the two branches, so as to effectively improve the discrimination ability of the model.

[0082] As an embodiment, the design of the spatial-spectral dual-branch feature extraction is mainly based on the structure and limitations of the 2D convolution and the 3D convolution, and combines different attention modules to simultaneously extract spatial features and spectral features in the hyperspectral image, so as to fully utilize the complementarity of the spatial and spectral information, thereby effectively improving the discrimination ability and classification performance of the model.

[0083] Step S3 comprises:

[0084] The processing steps of the position attention module PAM are as follows:

[0085] Supposing that the local feature of the hyperspectral image in the hyperspectral remote sensing image classification dataset is ;

[0086] The local feature is subjected to nonlinear transformation by using convolution through the position attention module PAM to generate three feature maps A, B and C;

[0087] The dimensions of the feature maps A, B and C are reshaped; matrix multiplication is performed between the transposed feature map A and the feature map B, and a position attention map is calculated through a softmax layer; ​

[0088] performing matrix multiplication between the feature map C and the location attention map, and reshaping the result into ;

[0089] multiplying the result by a learnable scale parameter a, and performing element-wise summation operation with the input feature to obtain the final output of the location attention module PAM .

[0090] In an embodiment of the present application, the output is sent to the channel attention module CAM, which calculates the channel attention map from the original local feature , and other processing procedures are the same as those of the location attention module PAM.

[0091] In an embodiment of the present application, in order to obtain higher-level abstract information, the serial combination structure of PAM module and CAM module is adopted herein to sequentially perform location attention enhancement and channel attention optimization on the input feature. Subsequently, a 2D depth separable convolution layer with a convolution kernel size of 3x3 is connected, which can effectively extract spatial features while reducing the number of model parameters and relieving the overfitting phenomenon. This process is repeated once, and the second operation still maintains the serial attention mechanism of PAM and CAM, but the convolution kernel size of the last connected 2D depth separable convolution layer is adjusted from 3x3 to 5x5 to expand the receptive field and enhance the model's perception ability of multi-scale features.

[0092] The convolution module is a 3D convolution module with a convolution kernel size of 3x3x3.

[0093] As an embodiment, a hyperspectral classification method based on a dual-branch spectral-spatial attention fusion network is constructed, which introduces a spectral feature module SAM based on 3D convolution for extracting spectral information of the image, and combines a bottleneck residual module to relieve the gradient disappearance problem in the network and improve the efficiency of feature extraction and information transmission.

[0094] As an embodiment, in the spectral feature branch, a 3D convolution with a convolution kernel size of 3x3 is first performed for preliminary feature mapping, and the output feature map of the convolution is transmitted to the bottleneck residual module BRB. The BRB module first reduces the number of channels of the input feature map through a 1x1x1 convolution layer, and then enters a 3x3x3 convolution layer, and batch normalization and ReLU activation function are applied after each convolution operation; finally, the number of channels of the feature map is restored to the original size or adjusted to a suitable dimension through a 1x1x1 convolution layer, and the input data is directly accumulated to the output.

[0095] The input feature map is processed by the spectral attention module SAM where B is batch size, D is spectral dimension, and C is the number of channels; a dynamic frequency domain position encoding process is performed, high spectral data is converted to the frequency domain through fast Fourier transform, then different frequency components are weighted using learnable complex weights, finally, the weighted frequency domain data is converted back to the spatial domain through inverse fast Fourier transform, and is added to the original data.

[0096] Subsequently, the SAM module adopts a lightweight global attention mechanism, first performs global average pooling operation, and then adopts a shared multilayer perceptron (MLP), which is composed of two fully connected layers. The first layer introduces non-linear characteristics through a ReLU activation function, and the second layer uses a Sigmoid activation function to generate normalized attention weights.

[0097] Step S3 further includes:

[0098] In an embodiment of the present application, a hyperspectral classification method based on a dual-branch spectral-spatial attention fusion network is constructed. Since the two branches are affected by different neural network models, their feature distributions are significantly different. An adaptive weighted fusion strategy is adopted, and weights w are dynamically allocated to the two branches.

[0099] The feature maps output by the spatial feature branch and the spectral feature branch are and respectively. The feature map X and the feature map are spliced along the last dimension to obtain a combined feature map containing dual-branch information .

[0100] Through a fully connected layer, the combined feature map is combined to generate attention weights .

[0101] According to the attention weights w, the input features and are weighted and fused to obtain a final fused map .

[0102] Step S3 further includes: using a classification cross-entropy loss function to train the initial hyperspectral classification model, which is expressed as follows:

[0103]

[0104] wherein, N represents the total number of samples in the training sample set, n represents the serial number of the sample in the training sample set, y n represents the true label of the nth sample, denotes the predicted label of the n-th sample, denotes the loss value between the predicted label and the real label.

[0105] As an embodiment, the network initial learning rate is set, the learning rate decay strategy is selected, the training parameters and the optimizer are adjusted, the precision verification is performed in the training process, the precision of each training of the model is evaluated by using the verification sample data set, the network model parameters are adjusted according to the evaluation precision and the loss value of the verification set, and the accuracy and the loss value of each training of the network model are recorded, so as to obtain the best network parameter model.

[0106] As an embodiment, the network initial learning rate is set, the learning rate decay strategy is selected, the training parameters and the optimizer are adjusted, the precision verification is performed in the training process, the precision of each training of the model is evaluated by using the verification sample data set, the network model parameters are adjusted according to the evaluation precision and the loss value of the verification set, and the accuracy and the loss value of each training of the network model are recorded, so as to obtain the best network parameter model. Figure 2 is a network structure framework diagram in the embodiment of the application; Figure 3 is a position attention module PAM structure schematic diagram in the embodiment of the application; Figure 4 is a channel attention module CAM structure schematic diagram in the embodiment of the application; Figure 5 is a bottleneck residual module BRB structure schematic diagram in the embodiment of the application; Figure 6 is a spectrum attention module SAM structure schematic diagram in the embodiment of the application.

[0107] The application provides an embodiment as follows:

[0108] 1. Experimental conditions: three hyperspectral image data sets are used, wherein:

[0109] The first hyperspectral data set is the Indian Pines data set collected by the AVIRIS sensor at the Indian Pines test site in the Indian northwest. It uses an image with a spatial resolution of 20m x 20m, covers 145x145 pixels, and contains 16 types of ground objects.

[0110] The second data set is the Pavia university data set obtained by using ROSIS, which covers 610x340 pixels and contains 9 types of ground objects.

[0111] The third data set is the Botswana data set obtained by the American space agency EO-1 satellite from 2001 to 2004, which has a size of 1476x256 pixels and contains 14 types of ground objects.

[0112] 2. Experimental content and result analysis: three quantitative evaluation indexes are used in this experiment: overall accuracy (OA), average accuracy (AA), and Kappa coefficient (Kappa).

[0113] 2.1, the present application and the existing seven methods SVM, 2D-CNN, 3D-CNN, CDCNN, SSRN, HybridSN, RSSAN are used for classification of Indian Pines data set, and three quantitative evaluation indexes are recorded, and the results are shown in Table 1.

[0114] Table 1

[0115]

[0116] 2.2, the present application and the existing seven methods SVM, 2D-CNN, 3D-CNN, CDCNN, SSRN, M3D-DCNN, MCCNN are used for classification of Pavia university data set, and three quantitative evaluation indexes are recorded, and the results are shown in Table 2.

[0117] Table 2

[0118]

[0119] 2.3, the present application and the existing seven methods SVM, 2D-CNN, 3D-CNN, CDCNN, SSRN, HybridSN, RSSAN are used for classification of Botswana data set, and three quantitative evaluation indexes are recorded, and the results are shown in Table 3.

[0120] Table 3

[0121]

[0122] From the results of Tables 1-3, it can be seen that on the three data sets, the overall classification accuracy OA and Kappa coefficient of the method of the present application are higher than those of the existing seven methods, the average classification accuracy AA of Indian Pines data set and Botswana data set is also higher than that of the existing seven methods, and the average classification accuracy AA of Pavia university data set is slightly lower than that of CDCNN, but still higher than that of the other six existing methods. It is proved that the method of the present application is effective and superior.

[0123] The technical scheme of the embodiment, by collecting hyperspectral remote sensing image, and pre-processing the image data to obtain hyperspectral remote sensing image classification data set; the hyperspectral remote sensing image classification data set is divided into training set and test set; a classification model based on double-branch spectral-spatial attention fusion network is constructed; the training samples are input into the classification model in batches, the network parameters are trained and optimized, and 50% verification set is divided for verification, after multiple training, the network finally converges, and the trained model is obtained; the test set is input into the model obtained by training, and the class prediction probability image is obtained through testing.

[0124] The application also discloses an electronic device. Referring to Figure 7 , Figure 7 is a structural schematic diagram of an electronic device disclosed by the embodiment of the application. The electronic device 500 can comprise at least one processor 501, at least one network interface 504, a user interface 503, a memory 505 and at least one communication bus 502.

[0125] The communication bus 502 is used to realize the connection communication between the components.

[0126] The user interface 503 can comprise a display screen, and the optional user interface 503 can further comprise a standard wired interface and a wireless interface.

[0127] The network interface 504 can optionally comprise a standard wired interface and a wireless interface (such as a WI-FI interface).

[0128] The application also discloses a computer readable storage medium, which stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the remote sensing image classification method based on the empty spectrum attention network.

[0129] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure.

[0130] The present application is intended to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field of the present disclosure not recorded in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for remote sensing image classification based on spectral-spatial attention network, characterized in that, The method comprises the following steps: S1: acquiring hyperspectral remote sensing satellite images and performing data preprocessing to construct a hyperspectral remote sensing image classification dataset; S2: constructing a classification model DSAFNet based on a double-branch spectral-spatial attention fusion network and taking the model as an initial hyperspectral classification model; Step S2 comprises: The initial hyperspectral classification model comprises an input layer, a spatial feature branch, a spectral feature branch, a global average pooling unit, an adaptive weighted fusion unit, and a fully connected layer; The spatial feature branch and the spectral feature branch are both connected to the input layer; the spatial feature branch and the spectral feature branch are both connected to the global average pooling unit; the global average pooling unit, the adaptive weighted fusion unit, and the fully connected layer are sequentially connected in order; The spatial feature branch comprises a first position attention module PAM, a first channel attention module CAM, a depth separable convolution layer with a convolution kernel size of 3x3, a second position attention module PAM, a second channel attention module CAM, and a depth separable convolution layer with a convolution kernel size of 5x5; the first position attention module PAM, the first channel attention module CAM, the depth separable convolution layer with a convolution kernel size of 3x3, the second position attention module PAM, the second channel attention module CAM, and the depth separable convolution layer with a convolution kernel size of 5x5 are sequentially connected in order; The spatial information in the hyperspectral image is extracted through the spatial feature branch; The spectral feature branch comprises a convolution module, a bottleneck residual module BRB, and a spectral attention module SAM; The convolution module, the bottleneck residual module BRB, and the spectral attention module SAM are sequentially connected in order; The spectral information in the hyperspectral image is extracted through the spectral feature branch; The global average pooling unit is used for globally average pooling the spatial information and the spectral information to compress the spatial dimension; The adaptive weighted fusion unit is used for integrating the features of the pooled spatial information and spectral information through an adaptive weighted fusion strategy to obtain a combined feature map; The fully connected layer is used for dynamically adjusting the output weight of the spatial feature branch and the spectral feature branch in the combined feature map to obtain a final fusion map; S3: training the initial hyperspectral classification model through the hyperspectral remote sensing image classification dataset to obtain a trained classification network model; S4: acquiring hyperspectral remote sensing images to be classified; inputting the hyperspectral remote sensing images to be classified into the trained classification network model to obtain an image classification result. 2.The method of claim 1, wherein, Step S1 comprises: S11: acquiring hyperspectral remote sensing satellite images of different ground object categories, different spatial resolutions, and covering multiple ground object combinations; S12: Hyperspectral remote sensing satellite imagery Principal component analysis is performed along the spectral dimension to reduce the original B-band hyperspectral data to M-bands, resulting in a dimensionality-reduced hyperspectral image. Here, H, W, B, and M represent the height, width, number of original spectral bands, and number of dimensionality-reduced spectral bands, respectively. S13: Perform zero padding operation and edge extension on the dimension-reduced hyperspectral image, and the edge extension size is , is a neighborhood window size; S14: Extract from each effective pixel of the hyperspectral image. The three-dimensional neighborhood cube is used to generate N three-dimensional data blocks corresponding to spatial locations, where N = H × W, and N represents the number of spatial pixels in the hyperspectral image; S15: constructing a hyperspectral remote sensing image classification dataset through a three-dimensional data block. 3.The method of claim 1, wherein, Step S3 comprises: The processing steps of the position attention module PAM are as follows: Let the local feature of the hyperspectral image in the hyperspectral remote sensing image classification dataset be ; The local features After the position attention module PAM, a nonlinear transformation is performed using convolution to generate three feature maps A, B and C; The dimensions of the feature maps A, B and C are reduced by Reshaping; performing matrix multiplication between the transpose of the feature map A and the feature map B, and passing through a softmax layer to obtain a position attention map ; performing a matrix multiplication between the feature map C and the position attention map, reshaping the result into , to obtain a result feature map; The resulting feature map is multiplied by a learnable scale parameter and summed element-wise with the input feature .​ 4.The method of claim 1, wherein, The convolution module is a 3D convolution module with a convolution kernel size of 3x3x3. 5.The method of claim 1, wherein, Step S3 further comprises: The feature maps output by the spatial feature branch and the spectral feature branch are respectively and , combine feature map X and feature map By concatenating along the last dimension, a combined feature map containing information from both branches is obtained. ; Through a fully connected layer, the combined feature maps are combined to generate attention weights ;​ Based on the attention weight w, the input features and Weighted fusion is performed to obtain the final fused graph. . 6.The method of claim 1, wherein, Step S3 further comprises training the initial hyperspectral classification model using a classification cross-entropy loss function, which is expressed as follows: wherein, denotes the total number of samples in the training sample set, denotes the serial number of the sample in the training sample set, denotes the true label of the nth sample, denotes the predicted label of the nth sample, denotes the loss value between the predicted label and the true label.

7. An electronic device, comprising: The electronic device comprises a processor, a memory, a user interface and a network interface, the memory is used for storing instructions, the user interface and the network interface are used for communication with other devices, and the processor is used for executing the instructions stored in the memory to enable the electronic device to perform the remote sensing image classification method based on the spectral attention network according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, when the instructions are executed by a computer, the remote sensing image classification method based on the spectral attention network according to any one of claims 1-6 is executed.

Images