Hyperspectral image classification method and system based on multi-scale spectral band learning

Abstract

The application provides a hyperspectral image classification method and system based on multi-scale spectral band learning, and belongs to the technical field of remote sensing image processing. Firstly, a series of hyperspectral data blocks are obtained by performing a block operation on an original hyperspectral image with labeled pixel points in the original hyperspectral image as centers. Then, a spectral weight learning module and a multi-scale spectral-spatial convolutional neural network are trained by using a spectral weight center loss function and a cross-entropy loss function. Key spectral information of the hyperspectral data blocks is extracted by using the spectral weight learning module, and the hyperspectral data blocks after spectral reweighting are obtained. Multi-scale spectral-spatial fusion feature extraction is performed on the hyperspectral data blocks after spectral reweighting by using the multi-scale spectral-spatial convolutional neural network, the features are sent into the multi-scale spectral-spatial convolutional neural network for classification prediction, and finally, the classification results are obtained. The application provides a new solution for key spectral information extraction in a spectral image.

Description

Technical Field

[0001] This invention relates to a method and system for hyperspectral image classification, specifically a method and system for hyperspectral image classification based on multi-scale spectral band learning, belonging to the field of remote sensing image processing. Background Technology

[0002] Hyperspectral imaging technology is based on image data in many narrow bands. It combines imaging and spectral techniques to detect the two-dimensional geometric space and one-dimensional spectral information of a target, acquiring continuous, narrow-band image data with high spectral resolution. Hyperspectral imaging technology is developing rapidly, and common methods include grating beam splitting, acousto-optic tunable filter beam splitting, prism beam splitting, and chip coating.

[0003] Unlike traditional imaging techniques that capture information from only a single or three spectral bands, hyperspectral imaging technology performs a much finer division of the spectral dimensions. During the imaging process, for each pixel, information from hundreds of wavelength bands across the entire electromagnetic spectrum is typically acquired, resulting in continuous, narrow-band image data with high spectral resolution, thus producing a hyperspectral image rich in spectral information. Therefore, hyperspectral imaging technology has been widely used in fields such as mineral exploration, agricultural regional planning, target detection, and land cover surveying.

[0004] With technological advancements and cost reductions, many researchers have recently explored its application in tasks such as facial recognition. Hyperspectral image classification, a branch of remote sensing image processing, aims to assign a unique category label to each pixel in a hyperspectral image, marking its class. This plays a crucial role in hyperspectral remote sensing image analysis. However, due to the presence of hundreds of spectral bands in hyperspectral images, their data dimensionality is typically tens or even hundreds of times greater than that of RGB images. Furthermore, the cost and complexity of hyperspectral image acquisition limit the amount of data. The redundant information in high-dimensional data and the limited number of samples pose significant challenges to hyperspectral image classification.

[0005] Therefore, how to deal with the data redundancy problem brought about by hyperspectral resolution, and how to obtain stable and accurate classification information from limited sample data, has always been a difficult problem in hyperspectral image classification tasks.

[0006] To address the data redundancy problem caused by high data dimensionality, traditional hyperspectral image classification methods typically employ principal component analysis, factor analysis, and independent component analysis to extract key band information from hyperspectral images and remove non-key bands, thereby reducing data redundancy. While these methods can reduce the data dimensionality of hyperspectral images, they mostly differentiate between key and non-key information based on data correlation, without seriously considering the discriminative information provided by each band in the specific classification process. Therefore, many researchers have begun to explore other hyperspectral data dimensionality reduction methods.

[0007] Band selection, as another method for dimensionality reduction of hyperspectral data, aims to select a subset of bands containing the most spectral information for subsequent classification, based on the importance of the information contained in each band. Common band selection methods include search-based, clustering-based, and ranking-based methods. In recent years, with the development of deep learning, some network search-based band selection methods have emerged. Although band selection-based data dimensionality reduction methods can reduce data redundancy while preserving key information as much as possible, they inevitably cause some information loss. Furthermore, these methods mostly utilize linear relationships between local bands during band selection, lacking consideration for nonlinear relationships and global interactions.

[0008] In traditional hyperspectral image classification, after dimensionality reduction to obtain the reduced spectral data, the next step is to further extract spectral features from the samples using manual feature operators. Then, a series of machine learning classification models, such as support vector machines, nearest neighbor classifiers, logistic regression classifiers, Bayesian classifiers, and decision trees, are used to classify the extracted features and obtain the classification results. These methods can achieve considerable results when the data dimensionality is small and the sample data is abundant. However, when the number of samples is limited or the data dimensionality reaches a certain level, the effectiveness of manual feature operators and the representational power of traditional machine learning models are severely tested.

[0009] The rise and rapid development of deep learning has brought new opportunities to hyperspectral image classification, leading to the emergence of several deep learning-based hyperspectral image classification methods. Examples include using LSTM to extract long-term and short-term correlations of spectral features to distinguish similar spectral features of land cover types, and combining traditional handcrafted feature operators with CNNs to extract additional spectral features. These methods achieve good results in most classification categories; however, when faced with categories with very few samples, their classification results are often unsatisfactory.

[0010] Therefore, in order to better address the data redundancy problem caused by hyperspectral resolution and the performance bottleneck caused by a limited number of samples, there is an urgent need for a hyperspectral image classification method that can effectively extract key spectral information from hyperspectral data and extract more discernible, stable and accurate classification information from limited sample data. Summary of the Invention

[0011] The purpose of this invention is to address the problems and shortcomings of existing hyperspectral data dimensionality reduction methods, such as information loss, lack of consideration for nonlinear relationships and global interactions between spectra, and insufficient stability and accuracy of extracted classification information due to the lack of utilization of multi-scale spectral and spatial information in hyperspectral images. This invention creatively proposes a hyperspectral image classification method and system based on multi-scale spectral band learning.

[0012] To achieve the above objectives, the present invention employs the following technical solutions.

[0013] A hyperspectral image classification method based on multi-scale spectral band learning includes the following steps:

[0014] Step 1: Using the labeled pixels in the original hyperspectral image as the center, perform a block operation with repeating pixels on the original hyperspectral image to obtain a series of hyperspectral data blocks.

[0015] In this case, the label of each hyperspectral data block is consistent with the label of its center pixel.

[0016] Step 2: Train the spectral weight learning module and the multi-scale spectral-spatial convolutional neural network using the spectral weight center loss function and the cross-entropy loss function.

[0017] Step 3: Using the trained spectral weight learning module, perform key spectral information extraction on the hyperspectral data blocks obtained in Step 1 to obtain the spectrally reweighted hyperspectral data blocks.

[0018] Step 4: Using the trained multi-scale spectral-spatial convolutional neural network, multi-scale spectral-spatial fusion feature extraction is performed on the spectrally reweighted hyperspectral data block. The obtained features are then fed into the fully connected layer of the multi-scale spectral-spatial convolutional neural network for classification prediction to obtain the final classification result.

[0019] To achieve the objectives described in this invention, this invention further proposes a hyperspectral image classification system based on multi-scale spectral band learning to implement the method, including a spectral weight learning module, a multi-scale spectral-spatial feature extraction module, a multi-scale spectral-spatial feature fusion module, and a feature classification module.

[0020] Beneficial effects

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] This invention effectively solves the problems of information loss and lack of consideration for nonlinear relationships and global interactions between spectra in existing hyperspectral data dimensionality reduction methods, as well as the unstable and inaccurate classification information extracted due to the lack of utilization of multi-scale spectral and spatial information in hyperspectral images. It provides accurate and reliable coverage detection and regional division results for tasks such as land cover detection and agricultural regional planning. At the same time, it brings a new solution to the data redundancy problem caused by hyperspectral resolution and provides a new means for extracting key spectral information from spectral images. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0024] Figure 2 This is a detailed schematic diagram of the spectral weight learning module proposed in the method of this invention;

[0025] Figure 3 This is a schematic diagram of the multi-scale spectral-spatial convolutional neural network described in the method of the present invention;

[0026] Figure 4 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0028] like Figure 1 As shown, a hyperspectral image classification method based on multi-scale spectral band learning includes the following steps:

[0029] Step 1: Using labeled pixels as the center, divide the original hyperspectral image into blocks with repeating pixels to obtain a series of hyperspectral data blocks. The label of each hyperspectral data block is consistent with the label of its center pixel.

[0030] Hyperspectral image classification differs from traditional image classification. While traditional image classification assigns a unique category label to each image, hyperspectral image classification assigns a unique category label to each pixel, making it a pixel-level classification task. The most direct approach for pixel-level classification is to extract all spectral information from the original hyperspectral image and use this as input to train the network to obtain the classification result. This method is simple and seems logically sound for hyperspectral image classification, but the results are often unsatisfactory. This is because it ignores a crucial aspect of image processing—neighboring pixel information. The information contained in a single pixel is always incomplete and insufficient. Neighboring pixels, being the closest pixels to the target pixel in the image, typically contain information with similarities and correlations to the target pixel. By supplementing this information with neighboring pixels, the resulting hyperspectral data block contains more complete and sufficient information, better meeting the needs of the network's learning.

[0031] Therefore, before performing spectral weight learning and multi-scale feature extraction, it is necessary to first divide all labeled pixels in the original hyperspectral image. For each pixel, a square region with a side length of W is selected as the center of the pixel for block division, resulting in a series of hyperspectral data blocks. Where R represents the real number field, C is the number of spectral bands in the hyperspectral data block, and the label of each hyperspectral data block is determined by the label of its center pixel.

[0032] Step 2: Use the spectral weight center loss function and the cross-entropy loss function to perform constrained training on the spectral weight learning module and the multi-scale spectral-spatial convolutional neural network.

[0033] In previous spectral weight learning modules, a specific weight distribution was typically learned for the entire dataset or for each individual sample. This weight distribution was then used to reweight the entire dataset or each individual sample to extract and utilize key spectral information. However, in reality, the spectral response curve of an object is closely related to its material. Objects of the same material have consistent spectral response curves. In the context of hyperspectral image classification, this means that samples of the same type have identical spectral response curves. Therefore, the previous approach of using the same spectral weight distribution for the entire dataset or using a separate spectral weight distribution for each sample has certain problems.

[0034] This invention learns spectral weights for each type of sample and uses the spectral weight center loss function for constraint to reduce the spectral weight gap between samples of the same type, so as to achieve the purpose of using similar spectral weight distributions for samples of the same type.

[0035] Specifically, the spectral weighting center loss function Represented as:

[0036]

[0037] Where, N c Let represent the number of samples belonging to class c, and M represent the total number of classes in the sample. N represents the total number of samples; The spectral weight center of the c-th class sample is a learnable parameter. This represents the spectral weight of the i-th sample belonging to class c.

[0038] By using the learnable spectral weight center of each class of samples as a guide, the spectral weight learning module can learn the spectral weight distribution of samples belonging to the current class, thereby reducing the difference in spectral weight distribution between samples of the same class and facilitating the learning of key spectral band information.

[0039] Furthermore, to enable multi-scale spectral-spatial convolutional neural networks to learn more information helpful for classification, the cross-entropy loss function can be utilized. To capture the differences in the relative magnitudes of predicted probabilities, and further standardize the network's learning of correct classification knowledge, its formal expression is as follows:

[0040]

[0041] Where N represents the total number of samples, and M represents the total number of categories in the samples. i,c Let y be the sign function, and when the true class of sample i is c, y i,c Select 1 if the value is 1, otherwise select 0. i,c This represents the predicted probability that sample i belongs to class c. `log()` represents taking the logarithm.

[0042] Then the total loss function as follows:

[0043]

[0044] Where τ represents the weight of the loss function.

[0045] Step 3: Using the trained spectral weight learning module, perform key spectral information extraction operations on the hyperspectral data blocks obtained by segmentation to obtain the spectrally reweighted hyperspectral data blocks.

[0046] To address the information loss and lack of consideration for nonlinear relationships and global interactions between spectra in existing hyperspectral data dimensionality reduction methods, this invention proposes a novel spectral weight learning module. By using global convolution, it considers the nonlinear relationships and similarities between all spectral bands, obtains the weight information of each band in the sample, and achieves the extraction of key spectral band information.

[0047] Specifically, such as Figure 2 As shown, for a hyperspectral data block C represents the number of spectral bands in the hyperspectral data block, W represents the length and width of the hyperspectral data block, and R represents the real number field.

[0048] First, The samples are fed into a global average pooling layer and a global max pooling layer respectively, and the results are obtained. Mean distribution in spatial dimension and maximum value distribution

[0049]

[0050]

[0051] GAP() represents global average pooling operation, and GMP() represents global max pooling operation.

[0052] Subsequently, to further obtain the weight information between each channel, a two-layer shared fully connected layer is used to learn the nonlinear relationship between each channel.

[0053] Specifically, for Mean distribution in spatial dimension and maximum value distribution They are sequentially fed into the shared fully connected layer to obtain... and Channel dependency

[0054]

[0055]

[0056] Where * denotes matrix multiplication, ReLU() is a non-linear activation function, and W1 and W2 represent the weights of two shared fully connected layers, respectively. γ is the scaling factor, and C represents the number of spectral bands in the hyperspectral data block.

[0057] In one embodiment of the invention, γ is set to 8.

[0058] For the two-layer shared fully connected layer, two convolutional layers with a kernel size of 1×1 are used respectively. Then, the channel dependencies under these two distributions are summed, and the Sigmoid function is used to obtain... spectral weights

[0059]

[0060] Where σ(·) represents the Sigmoid function.

[0061] Furthermore, for each hyperspectral data block By sequentially following the above process, the corresponding spectral weights are obtained. Hyperspectral data block after spectral reweighting Obtained through the following formula:

[0062]

[0063] Here, ⊙ represents element-wise dot product.

[0064] Step 4: Using the trained multi-scale spectral-spatial convolutional neural network, multi-scale spectral-spatial fusion feature extraction is performed on the spectrally reweighted hyperspectral data block, and the obtained features are fed into the fully connected layer of the network for classification prediction to obtain the final classification result.

[0065] Specifically, to address the problem of unstable and inaccurate classification information resulting from the lack of utilization of multi-scale spectral and spatial information in hyperspectral images, this invention proposes an attention-based multi-scale spectral-spatial convolutional neural network to maximize the utilization of these features, considering the feature similarity between adjacent regions and the spectral-spatial correlation between them. This network uses multi-scale convolutional kernels to extract spectral-spatial features under different receptive fields, and further refines the obtained features using an external attention mechanism to obtain more discriminative multi-scale spectral-spatial fusion features.

[0066] like Figure 3 As shown, for the hyperspectral data block after spectral weight learning and reweighting... First, shallow spectral-spatial features are extracted through a two-dimensional convolution with a kernel size of 3×3 and a batch normalization operation.

[0067] Subsequently, multi-scale spectral-spatial features were extracted by compressing attention residual blocks using four pyramids.

[0068] Pyramid-compressed attention residual blocks, such as Figure 3 As shown, specifically, for the input feature map C′ represents the number of channels in the feature map, and H′ and W′ represent the length and width of the feature map, respectively. First, a 1×1 2D convolution is used to extract the channel dependencies of the input feature map. Second, an SPC module (Split and Concat) is used to obtain multi-scale spatial feature maps under different spatial receptive fields. Then, a grouped SEWeight module (Squeeze-and-Excitation Weight) is used to extract grouped channel attention from the multi-scale spatial feature maps. In the grouped SEWeight module, the input feature map is first grouped according to the grouping method in the SPC module, and the SEWeight module is used to obtain the channel attention weights for each group. Subsequently, the channel attention of different groups is concatenated, and the channel attention vectors under different channel groups are recalibrated using the Softmax function, thereby obtaining the grouped channel attention in the spectral dimension. Finally, the obtained grouped channel attention and the corresponding feature map are multiplied, and then refined by a two-dimensional convolution with a kernel size of 1×1. The refined features are then added to the input feature map to obtain multi-scale spectral-spatial features with more discriminative information.

[0069] Subsequently, the multi-scale spectral-spatial features obtained by the external attention mechanism were used to extract the spectral-spatial correlation in the hyperspectral data block, thus obtaining the multi-scale spectral-spatial fusion features.

[0070] Specifically, an external attention mechanism is used to fuse the obtained multi-scale spectral-spatial features. In traditional self-attention computation, the calculation of the sample query vector Q, key vector K, and value vector V only considers the relationships between elements in a single sample, ignoring the potential relationships between elements of different samples, which limits the flexibility of self-attention. Compared with the traditional self-attention mechanism, the external attention mechanism described in this invention uses two learnable memory units to replace K and V in the self-attention mechanism. Utilizing network training methods, the memory units no longer consider a single sample, but rather the relationships between samples, and utilize the potential knowledge learned from them, thus improving the flexibility and capability of the attention mechanism. The corresponding expression is as follows:

[0071]

[0072] Where ExternalAttention() represents the external attention operation, Norm() represents normalization, and D K and D V These are the memory units that replace keys K and values ​​V in the external attention mechanism, d k Represents memory unit D K Dimensions.

[0073] Furthermore, by utilizing a multilayer perceptron (MLP), the features obtained through fusion via an external attention mechanism are further refined to obtain multi-scale spectral-spatial fusion features.

[0074] Finally, the obtained multi-scale spectral-spatial fusion features are fed into a fully connected classification layer for classification to obtain the classification results of the hyperspectral image.

[0075] Figure 4 This is a schematic diagram of the system of the present invention.

[0076] like Figure 4 As shown, a hyperspectral image classification system based on multi-scale spectral band learning includes a spectral weight learning module M100, a multi-scale spectral-spatial feature extraction module M200, a multi-scale spectral-spatial feature fusion module M300, and a feature classification module M400.

[0077] Among them, the spectral weight learning module M100 is used to extract key spectral information from the hyperspectral data block and obtain the hyperspectral data block after spectral reweighting.

[0078] The M200 multi-scale spectral-spatial feature extraction module is used to extract multi-scale spectral and spatial information from spectrally reweighted hyperspectral data blocks to obtain multi-scale spectral-spatial features with more discriminative information.

[0079] Furthermore, the multi-scale spectral-spatial feature extraction module M200 includes an SPC submodule M210, a grouped SEWeight submodule M220, and an SEWeight submodule M221. The SPC submodule is used to acquire multi-scale spatial feature maps under different spatial receptive fields; the grouped SEWeight submodule is used to extract grouped channel attention from the multi-scale spatial feature maps; and the SEWeight submodule is used to obtain the channel attention weights for each group.

[0080] The M300 multi-scale spectral-spatial feature fusion module utilizes the attention weights extracted from the multi-scale spectral-spatial features to fuse the multi-scale spectral-spatial features, and further extracts and refines the fused features to obtain multi-scale spectral-spatial fused features.

[0081] Furthermore, the multi-scale spectral-spatial feature fusion module M300 includes an external attention submodule M310 and a multilayer perceptron submodule M320. The external attention submodule M310 is used to fuse multi-scale spectral-spatial features and extract spectral-spatial correlations from hyperspectral data blocks. The multilayer perceptron submodule M320 is used to further refine the fused features to obtain multi-scale spectral-spatial fused features.

[0082] The feature classification module M400 uses multi-scale spectral-spatial fusion features obtained from hyperspectral data blocks as the basis for category division to obtain the classification results of hyperspectral images.

[0083] The connection relationships between the above modules are as follows:

[0084] The output of the spectral weight learning module M100 is connected to the input of the multi-scale spectral-spatial feature extraction module M200.

[0085] The output of the multi-scale spectral-spatial feature extraction module M200 is connected to the input of the multi-scale spectral-spatial feature fusion module M300. Inside the multi-scale spectral-spatial feature extraction module M200, the output of the SPC submodule M210, which is used to acquire multi-scale spatial feature maps, is connected to the input of the grouped SEWeight submodule M220, which is used to acquire grouped attention weights.

[0086] The output of the multi-scale spectral-spatial feature fusion module M300 is connected to the input of the feature classification module M400. Inside the multi-scale spectral-spatial feature fusion module M300, the output of the external attention submodule M310 is connected to the input of the multilayer perceptron submodule M320.

Claims

1. A hyperspectral image classification method based on multi-scale spectral band learning, characterized in that, Includes the following steps: Step 1: Using the labeled pixels in the original hyperspectral image as the center, perform a block operation with repeating pixels on the original hyperspectral image to obtain a series of hyperspectral data blocks; In this process, the label of each hyperspectral data block is consistent with the label of its center pixel. Step 2: Train the spectral weight learning module and the multi-scale spectral-spatial convolutional neural network using the spectral weight center loss function and the cross-entropy loss function; Step 3: Using the trained spectral weight learning module, perform key spectral information extraction on the hyperspectral data blocks obtained by segmentation in Step 1 to obtain the spectrally reweighted hyperspectral data blocks. Among them, the spectral weight learning module uses global convolution to consider the nonlinear relationship and similarity between all spectral bands, obtain the weight information of each band of the sample, and realize the extraction of key spectral band information. For a hyperspectral data block C represents the number of spectral bands in the hyperspectral data block, W represents the length and width of the hyperspectral data block, and R represents the real number field. First, The samples are fed into a global average pooling layer and a global max pooling layer respectively, and the results are obtained. Mean distribution in spatial dimension and maximum value distribution Wherein, GAP() represents global average pooling operation, and GMP() represents global max pooling operation; Subsequently, to further obtain the weight information between each channel, a two-layer shared fully connected layer is used to learn the nonlinear relationship between each channel; for Mean distribution in spatial dimension and maximum value distribution They are sequentially fed into the shared fully connected layer to obtain... and Channel dependency Where * denotes matrix multiplication, ReLU() is a non-linear activation function, and W1 and W2 represent the weights of two shared fully connected layers, respectively. γ is the scaling factor, and C represents the number of spectral bands in the hyperspectral data block; For the two-layer shared fully connected layer, two convolutional layers with a kernel size of 1×1 are used respectively; then, the channel dependencies under the two distributions are summed, and the Sigmoid function is used to obtain... spectral weights Where σ(·) represents the Sigmoid function; For each hyperspectral data block By sequentially following the above process, the corresponding spectral weights are obtained. Hyperspectral data block after spectral reweighting Obtained through the following formula: Where ⊙ represents element-wise dot product; Step 4: Using the trained multi-scale spectral-spatial convolutional neural network, multi-scale spectral-spatial fusion feature extraction is performed on the spectrally reweighted hyperspectral data block. The obtained features are then fed into the fully connected layer of the multi-scale spectral-spatial convolutional neural network for classification prediction to obtain the final classification result.

2. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 1, characterized in that, In step 1, all labeled pixels in the original hyperspectral image are divided. For each pixel, a square region with a side length of W is selected as the center of the pixel for further partitioning, resulting in a series of hyperspectral data blocks. Where R represents the real number field, C is the number of spectral bands in the hyperspectral data block, and the label of each hyperspectral data block is determined by the label of its center pixel.

3. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 1, characterized in that, In step 2, the spectral weights of each class of samples are learned, and the spectral weight center loss function is used as a constraint to reduce the spectral weight gap between samples of the same class. Among them, the spectral weight center loss function Represented as: Where, N c Let represent the number of samples belonging to class c, and M represent the total number of classes in the sample. N represents the total number of samples; The spectral weight center of the c-th class sample is a learnable parameter. This represents the spectral weight of the i-th sample belonging to class c in the sample. By using the learnable spectral weight center of each class of samples as a guide, the spectral weight learning module learns the spectral weight distribution of samples belonging to the current class, thereby reducing the difference in spectral weight distribution between samples of the same class.

4. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 3, characterized in that, In step 2, the cross-entropy loss function is used. To capture the differences in the relative magnitudes of predicted probabilities and to regulate the network's learning of correct classification knowledge, it can be expressed as follows: Where N represents the total number of samples, and M represents the total number of categories in the samples; y i,c Let y be the sign function, and when the true class of sample i is c, y i,c Select 1 otherwise select 0; p i,c This represents the predicted probability that sample i belongs to class c; log() represents taking the logarithm. Then the total loss function as follows: Where τ represents the weight of the loss function.

5. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 1, characterized in that, γ is set to 8.

6. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 1, characterized in that, For the hyperspectral data block after spectral weight learning and reweighting First, shallow spectral-spatial features are extracted through two-dimensional convolution and batch normalization operations; Subsequently, multi-scale spectral-spatial features are extracted by pyramid compression of attention residual blocks; then, the spectral-spatial correlation in the hyperspectral data blocks is extracted by fusing the multi-scale spectral-spatial features obtained by external attention mechanism. Using a multilayer perceptron module, the features obtained through fusion via an external attention mechanism are further refined to obtain multi-scale spectral-spatial fusion features. Finally, the obtained multi-scale spectral-spatial fusion features are fed into a fully connected classification layer for classification to obtain the classification results of the hyperspectral image.

7. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 6, characterized in that, In pyramid-compressed attention residual blocks, for the input feature map C ′ H represents the number of channels in the feature map. ′ and W ′ These represent the length and width of the feature map, respectively. First, the channel dependency of the input feature map is extracted through a two-dimensional convolution; Secondly, the SPC module is used to obtain multi-scale spatial feature maps under different spatial receptive fields; Then, the grouped SEWeight module is used to extract grouped channel attention from the multi-scale spatial feature map; In the SEWeight grouping module, the input feature map is first grouped according to the grouping method in the SPC module, and the SEWeight module is used to obtain the channel attention weights of each group. Then, the channel attention of different groups is connected, and the channel attention vectors under different channel groups are recalibrated using the Softmax function, thereby obtaining the grouped channel attention in the spectral dimension. Finally, the obtained grouped channel attention and the corresponding feature map are multiplied, and then the feature information is refined by two-dimensional convolution. The refined feature is then added to the input feature map to obtain multi-scale spectral-spatial features with more discriminative information.

8. The hyperspectral image classification method based on multi-scale spectral band learning as described in claim 6, characterized in that, The obtained multi-scale spectral-spatial features are fused using an external attention mechanism; Two learnable memory units are used to replace K and V in the self-attention mechanism. Using network training methods, these memory units consider the relationships between samples and utilize the latent knowledge learned from them, thus improving the flexibility and capability of the attention mechanism. The corresponding expression is as follows: Where ExternalAttention() represents the external attention operation, Norm() represents normalization, and D K and D V These are the memory units that replace keys K and values ​​V in the external attention mechanism, d k Represents memory unit D K Dimensions.

9. A hyperspectral image classification system based on multi-scale spectral band learning, as described in claim 1, characterized in that, It includes a spectral weight learning module M100, a multi-scale spectral-spatial feature extraction module M200, a multi-scale spectral-spatial feature fusion module M300, and a feature classification module M400, among which... The M100 spectral weighting learning module is used to extract key spectral information from hyperspectral data blocks to obtain hyperspectral data blocks that have undergone spectral reweighting. The M200 multi-scale spectral-spatial feature extraction module is used to extract multi-scale spectral and spatial information from spectrally reweighted hyperspectral data blocks to obtain multi-scale spectral-spatial features with more discriminative information. The multi-scale spectral-spatial feature extraction module M200 includes an SPC submodule M210, a grouped SEWeight submodule M220, and an SEWeight submodule M221. The SPC submodule is used to acquire multi-scale spatial feature maps under different spatial receptive fields. The grouped SEWeight submodule is used to extract grouped channel attention from the multi-scale spatial feature maps. The SEWeight submodule is used to obtain the channel attention weights for each group. The M300 multi-scale spectral-spatial feature fusion module utilizes the attention weights extracted from the multi-scale spectral-spatial features to fuse the multi-scale spectral-spatial features, and further extracts and refines the fused features to obtain multi-scale spectral-spatial fused features. The multi-scale spectral-spatial feature fusion module M300 includes an external attention submodule M310 and a multilayer perceptron submodule M320. The external attention submodule M310 is used to fuse multi-scale spectral-spatial features and extract spectral-spatial correlations from hyperspectral data blocks. The multilayer perceptron submodule M320 is used to further refine the fused features to obtain multi-scale spectral-spatial fusion features. The feature classification module M400 uses multi-scale spectral-spatial fusion features obtained from hyperspectral data blocks as the basis for category division to obtain the classification results of hyperspectral images; The connection relationships between the above modules are as follows: The output of the spectral weight learning module M100 is connected to the input of the multi-scale spectral-spatial feature extraction module M200; The output of the multi-scale spectral-spatial feature extraction module M200 is connected to the input of the multi-scale spectral-spatial feature fusion module M300; inside the multi-scale spectral-spatial feature extraction module M200, the output of the SPC submodule M210, which is used to obtain multi-scale spatial feature maps, is connected to the input of the grouped SEWeight submodule M220, which is used to obtain grouped attention weights. The output of the multi-scale spectral-spatial feature fusion module M300 is connected to the input of the feature classification module M400; inside the multi-scale spectral-spatial feature fusion module M300, the output of the external attention submodule M310 is connected to the input of the multilayer perceptron submodule M320.

Images