A Hyperspectral Image Classification Method Based on Spectral Channel Fusion and Cross-Scale Global Aggregation

Abstract

This invention belongs to the field of image processing technology and discloses a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation. The method includes: acquiring a hyperspectral image to be processed; inputting the hyperspectral image to be processed into a hyperspectral image classification model for classification to obtain an image classification result; wherein the hyperspectral image classification model includes a frequency domain spectral feature extraction module, a multi-scale spectral channel fusion attention module, and a cross-scale global aggregation module connected in sequence. This invention possesses strong portability, can efficiently handle image classification tasks in various complex scenarios, and significantly improves classification accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation. Background Technology

[0002] Hyperspectral image classification is crucial in remote sensing applications because it contains rich spectral and spatial information. Convolutional neural networks (CNNs) can automatically extract features, but they struggle to capture deep semantic information, and the computational cost increases exponentially with the number of network layers. While Transformers excel at extracting high-level semantic information, they face significant computational challenges in hyperspectral image classification, particularly when dealing with high-dimensional data. Furthermore, during model training, the limited number of labeled samples makes it difficult to accurately identify different land cover types, thus impacting classification accuracy.

[0003] In recent years, various solutions have been proposed to address the challenges of hyperspectral image classification. Traditional classification methods, such as support vector machines, k-nearest neighbors, and decision trees, have shown potential in classifying hyperspectral data. However, these methods often fail to adaptively extract abstract features from the scene, leading to reduced accuracy in classifying high-dimensional data. To overcome these limitations, deep learning methods, especially convolutional neural networks and recurrent neural networks, have been widely applied to hyperspectral image classification. They have demonstrated significant advantages in learning hierarchical representations of hyperspectral data and improving classification performance, particularly when labeled data is abundant.

[0004] However, among current hyperspectral image classification models, some are able to effectively extract discriminative spectral-spatial information, while others may extract too many redundant features. This not only increases the computational burden but also reduces classification accuracy. Furthermore, how to enhance the representation of these spectral-spatial features during extraction is also a crucial factor affecting classification performance. Meanwhile, traditional Transformer models typically consume significant computational resources, which not only slows down processing speed but also makes high-precision classification tasks more complex. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation to solve the problems existing in the prior art.

[0006] To achieve the above objectives, this invention provides a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation, comprising:

[0007] Acquire the hyperspectral image to be processed;

[0008] The hyperspectral image to be processed is input into a hyperspectral image classification model for classification to obtain the image classification result; wherein, the hyperspectral image classification model includes a frequency domain spectral feature extraction module, a multi-scale spectral channel fusion attention module, and a cross-scale global aggregation module connected in sequence.

[0009] Optionally, the training process of the hyperspectral image classification model specifically includes:

[0010] Acquire training data, which includes hyperspectral training images and corresponding image classification results;

[0011] An initial hyperspectral image classification model is constructed. The training data is input into the initial hyperspectral image classification model for classification. The model is trained with the goal of minimizing the loss between the initial training result and the image classification result corresponding to the hyperspectral training image. The trained hyperspectral image classification model is then obtained.

[0012] Optionally, the processing procedure of the hyperspectral image classification model specifically includes:

[0013] The hyperspectral image to be processed is input into the frequency domain spectral feature extraction module, and multi-scale and multi-frequency information is extracted from the hyperspectral image to be processed by combining the fast Fourier transform method to obtain spectral information at different scales.

[0014] Spectral information at different scales is input into the multi-scale spectral channel fusion attention module for feature fusion, and the importance of key features is adaptively enhanced through the attention mechanism to obtain feature information at different scales;

[0015] Input feature information at different scales into the cross-scale global aggregation module to aggregate feature information at different scales and output classification results.

[0016] Optionally, the processing flow of the frequency domain spectral feature extraction module specifically includes:

[0017] Input the hyperspectral image to be processed Where B represents the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0018] The input image is processed using a 1×1 convolutional layer, Conv2D:

[0019]

[0020] Process feature map F using depthwise separable convolution (DWConv):

[0021]

[0022] After applying the GELU activation function:

[0023] F GELU =GELU(F dw )

[0024] In the formula, F dw F is the feature map after depthwise separable convolution. GELU The feature map after applying the activation function;

[0025] For feature map F GELU Rearrange:

[0026] Q,K,V=F GELU ·chunkk(3,dim=1)

[0027] In the formula, Q, K, and V represent the query, key, and value, respectively, and chunk represents the partitioning along a given dimension;

[0028] Perform Fourier transforms Q and K to transform the features from the spatial domain to the frequency domain, and then perform a two-dimensional fast Fourier transform:

[0029]

[0030] in, This represents the two-dimensional Fast Fourier Transform, and the Q-value after the Fourier Transform is... patch and K patch These are the corresponding frequency domain components;

[0031] In the frequency domain, the module calculates Q by element-wise multiplication. patch and K patch Similarity:

[0032] Out fft =Q fft ×K fft

[0033] In the formula, Out fft The result of the similarity calculation;

[0034] The frequency domain information is converted back to the spatial domain using the inverse Fourier transform:

[0035]

[0036] in, This represents the inverse two-dimensional Fourier transform, where Out is the spatial domain output.

[0037] The output feature map after the inverse transformation is rearranged to restore the dimensions of the original inputs Q and K, and LayerNorm is used to normalize the feature map so that each channel of the feature map has a similar distribution.

[0038] Nonlinear transformation using the GELU activation function:

[0039] Out = GELU(LayerNorm(Out))

[0040] Finally, V is multiplied element-wise with the processed Out to obtain the final output feature map. Then, a 1×1 convolutional layer is used to restore the number of output channels to the original C, resulting in the final output OUt. FDSFM for:

[0041]

[0042] Optionally, the processing flow of the multi-scale spectral channel fusion attention module specifically includes:

[0043] Input spectral information at different scales, and then use global average pooling to obtain a feature map of size [B,C,1,1], where B is the batch size and C is the number of channels;

[0044] By applying convolutional kernels of different sizes s1, s2, ..., s to the pooled feature maps... k The 1D convolution is used to obtain the response at each scale. Then, the channel dimension is transformed by channel convolution to obtain the channel attention at each scale.

[0045] Channel attention at each scale can be expressed by the following formula:

[0046]

[0047] Among them, Conv i (·) denotes the convolution operation at the i-th scale, where x is the input after average pooling. This is a channel convolutional layer used to adjust the channel dimensions; σ is the sigmoid activation function. Channel attention at each scale;

[0048] Averaging the channel attention across all scales yields multi-scale channel attention:

[0049]

[0050] Spatial attention is extracted in the spatial dimension through convolution operations, given input features. Figure X Spatial attention is calculated as follows:

[0051]

[0052] in, It is a 1×1 convolution operation used to generate a spatial attention map;

[0053] Define a mixing module Mix, which generates the final output by weighted fusion of two input feature maps F1 and F2.

[0054] Optionally, the processing procedure of the Mix module specifically includes:

[0055] Given input feature maps F1 and F2, the fusion process can be represented by the following formula:

[0056]

[0057] in, It is the Sigmoid activation function, F Mix It outputs a feature map;

[0058] Multi-scale channel attention and spatial attention are weighted and fused through the Mix module, and the final fused output F MSFAM It can be represented as:

[0059] F MSFAM =X·Mix(F MSA ,F SA )

[0060] In the formula, X is the input image, and F MSA For the results of multi-scale channel attention operations, F SA This is the result of spatial attention calculation.

[0061] Optionally, the processing procedure of the cross-scale global aggregation module specifically includes:

[0062] Let the input feature maps be the left feature map L and the right feature map R. Determine whether feature maps L and R have the same spatial dimension. If the dimensions are different, then adjust the size of feature map R to the size of L through an upsampling operation.

[0063]

[0064] in, This is the r feature map after upsampling. upsample is the upsampling operation, shape is the shape of the feature map, and if represents a judgment.

[0065] After dimensional alignment, the cross-scale global aggregation module performs feature fusion on the input L and the adjusted R. First, it performs a convolution operation on the R feature map. CBR Then, the convolutional R and L are concatenated to form a new fused feature map. At this point, the convolution operation Conv is used. cat The stitched feature maps are further processed to generate a fused feature map O:

[0066]

[0067] Where L is the original feature map, Represents the feature map after convolution; Conv CBR (·) refers to a sequence containing a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function;

[0068] After performing an extended convolution operation on the fused feature map O, it is added to the original feature map to generate a context-enhanced feature map O. g :

[0069] O g =O + DilatedConv(O)

[0070] Global average pooling is used to process feature map O to extract global information. The global features are then compressed using a 1×1 convolution to obtain scaled global features. Bilinear interpolation is then used to sample the global feature map to the size of the original input feature map.

[0071] O g =O g +upsample(Conv2D 1×1 (GAP(O)))

[0072] In the formula, GAP represents global average pooling;

[0073] After obtaining the context-enhanced feature maps, the cross-scale global aggregation module performs convolution operations on feature maps L and R respectively, and uses the context-enhanced feature map O. g We weight feature maps L and R, concatenate the weighted feature maps L and R with the previous feature map O, and then further fuse them into a new feature map O2 through a convolution operation.

[0074]

[0075] Where L2=Conv2D 3×3 (L)⊙O, It is the feature map after convolution and weighting, and ⊙ represents element-wise multiplication;

[0076] Weighted coefficients are generated through local mean pooling and local max pooling operations. The sigmoid activation function is used to normalize the weighted coefficients to the [0,1] interval, and then they are multiplied element-wise with the feature map O2 to obtain the final output feature map O. f :

[0077]

[0078] Among them, Og2 These are the dynamic weighting coefficients obtained through the above weighting process. Represents matrix multiplication;

[0079] Three 1×1 kernel convolutional layers are applied to enhance the non-linearity of the previously obtained feature maps, generating three distinct feature matrices:

[0080]

[0081] In the formula, reshape is the transformation matrix;

[0082] Calculate the similarity between Q and K, then multiply the weighted matrix by the matrix containing V to obtain the weighted feature map:

[0083]

[0084] Where G represents the enhanced feature obtained through adaptive weighting;

[0085] Add the enhanced feature G to the original feature map O. f Then, it is normalized by convolution and using the Sigmoid activation function:

[0086]

[0087] Here, δ is a learnable parameter that controls the weights of the enhanced features;

[0088] For input features O f Perform average pooling and max pooling operations, concatenate the results, and then pass them through a convolutional layer to obtain the final output F. CGAM :

[0089] F CGAM =Conv CBR (Conv cat (Avg(O f ),Max(O f ))).

[0090] The technical effects of this invention are as follows:

[0091] This invention proposes a novel frequency domain spectral feature extraction module that extracts multi-scale and multi-frequency information from hyperspectral images by utilizing Fast Fourier Transform. This method significantly enhances the ability to process high-dimensional data, especially in capturing local structural features and global patterns in hyperspectral images, thereby improving the discrimination capability.

[0092] The multi-scale spectral channel fusion attention module of this invention integrates spectral information at different scales and dynamically enhances the importance of key features through an attention mechanism, enabling the model to adaptively focus on the most critical spectral channels to complete the classification task. This fusion and weighting mechanism effectively improves classification accuracy and robustness.

[0093] The cross-scale global aggregation module of this invention further enhances feature representation by aggregating information from multiple scales. This module ensures that features from different scales are fully utilized in the final network output, optimizing the classification task from a global perspective. The aggregation of cross-scale information effectively reduces pixel misclassification and improves classification accuracy, especially in complex hyperspectral image classification tasks. Attached Figure Description

[0094] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0095] Figure 1 This is a flowchart of the hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation in an embodiment of the present invention;

[0096] Figure 2 This is a framework diagram of the hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation in an embodiment of the present invention;

[0097] Figure 3 This is a structural diagram of the frequency domain spectral feature extraction module in an embodiment of the present invention;

[0098] Figure 4 This is a structural diagram of the multi-scale spectral channel fusion attention module in an embodiment of the present invention;

[0099] Figure 5 This is a structural diagram of the cross-scale global aggregation module in an embodiment of the present invention;

[0100] Figure 6 The figures show the classification accuracy of different algorithms in the Pavia University dataset in this embodiment of the invention. Figure 6 (a) shows the actual distribution of nine land features in the Pavia University dataset in this embodiment of the invention; Figure 6 (b) is a graph showing the overall classification accuracy of the Pavia University dataset based on the SSFTT comparison algorithm in this embodiment of the invention. Figure 6 (c) is a graph showing the overall classification accuracy of the Pavia University dataset based on the morphFormer comparison algorithm in this embodiment of the invention. Figure 6(d) is the overall classification accuracy of the Pavia University dataset based on the DBCT comparison algorithm in this embodiment of the invention; Figure 6 (e) is a graph showing the overall classification accuracy of the Pavia University dataset based on the GAHT comparison algorithm in this embodiment of the invention. Figure 6 In the middle (f), the overall classification accuracy of the Pavia University dataset based on the MASSFormer comparison algorithm in this embodiment of the invention is shown. Figure 6 In the middle (g), the overall classification accuracy of the Pavia University dataset based on the contrast algorithm DCTN in this embodiment of the invention is shown. Figure 6 In the middle (h), the overall classification accuracy of the Pavia University dataset based on the LSFAT comparison algorithm in this embodiment of the invention is shown. Figure 6 In the middle (i), the overall classification accuracy of the Pavia University dataset based on the MambaHSI comparison algorithm in this embodiment of the invention is shown. Figure 6 In the middle (j), the overall classification accuracy of the Pavia University dataset based on the 3DSS-Mamba contrast algorithm in this embodiment of the invention is shown. Figure 6 In the middle (k), the overall classification accuracy of the Pavia University dataset based on the contrast algorithm FS-CGNet in this embodiment of the invention is shown.

[0101] Figure 7 These are classification accuracy graphs of different algorithms in the Salinas dataset from embodiments of the present invention; wherein, Figure 7 (a) shows the actual distribution of 16 land features in the Salinas dataset in this embodiment of the invention; Figure 7 (b) is a graph showing the overall classification accuracy of the Salinas dataset based on the SSFTT comparison algorithm in this embodiment of the invention. Figure 7 (c) is a graph showing the overall classification accuracy of the Salinas dataset based on the morphFormer comparison algorithm in this embodiment of the invention. Figure 7 (d) is the overall classification accuracy of the Salinas dataset based on the DBCT comparison algorithm in this embodiment of the invention; Figure 7 (e) is a graph showing the overall classification accuracy of the Salinas dataset based on the GAHT comparison algorithm in this embodiment of the invention. Figure 7 In the middle (f), the overall classification accuracy of the Salinas dataset based on the MASSFormer comparison algorithm in the embodiment of the present invention is shown. Figure 7 In the middle (g), the overall classification accuracy of the Salinas dataset based on the DCTN comparison algorithm in this embodiment of the invention is shown. Figure 7 In the middle (h), the overall classification accuracy of the Salinas dataset based on the LSFAT comparison algorithm in the embodiment of the present invention is shown. Figure 7 In the middle (i), the overall classification accuracy of the Salinas dataset based on the MambaHSI comparison algorithm in the embodiment of the present invention is shown. Figure 7 In the middle (j), the overall classification accuracy of the Salinas dataset based on the 3DSS-Mamba comparison algorithm in this embodiment of the invention is shown. Figure 7 In the middle (k), the overall classification accuracy of the Salinas dataset based on the contrast algorithm FS-CGNet in this embodiment of the invention is shown.

[0102] Figure 8 These are classification accuracy maps of different algorithms in this invention's embodiments on the WHU-Hi-LongKou dataset; wherein, Figure 8 (a) shows the actual distribution of nine land features in the WHU-Hi-LongKou dataset in this embodiment of the invention; Figure 8 (b) is a graph showing the overall classification accuracy of the WHU-Hi-LongKou dataset based on the SSFTT comparison algorithm in this embodiment of the invention. Figure 8 (c) is a graph showing the overall classification accuracy of the WHU-Hi-LongKou dataset based on the contrast algorithm morphFormer in this embodiment of the invention. Figure 8 (d) is the overall classification accuracy of the WHU-Hi-LongKou dataset based on the DBCT comparison algorithm in this embodiment of the invention; Figure 8 (e) is a graph showing the overall classification accuracy of the WHU-Hi-LongKou dataset based on the GAHT comparison algorithm in this embodiment of the invention. Figure 8 In the middle (f), the overall classification accuracy of the WHU-Hi-LongKou dataset based on the MASSFormer comparison algorithm in the embodiment of the present invention is shown. Figure 8 In the middle (g), the overall classification accuracy of the WHU-Hi-LongKou dataset based on the contrast algorithm DCTN in the embodiment of the present invention is shown. Figure 8 (h) is the overall classification accuracy of the WHU-Hi-LongKou dataset based on the LSFAT comparison algorithm in this embodiment of the invention; Figure 8 In the middle (i), the overall classification accuracy of the WHU-Hi-LongKou dataset based on the MambaHSI contrast algorithm in this embodiment of the invention is shown. Figure 8 In the middle (j), the overall classification accuracy of the WHU-Hi-LongKou dataset based on the 3DSS-Mamba contrast algorithm in this embodiment of the invention is shown. Figure 8In the middle (k), the overall classification accuracy of the WHU-Hi-LongKou dataset based on the contrast algorithm FS-CGNet in this embodiment of the invention is shown.

[0103] Figure 9 These are classification accuracy graphs of different algorithms in the XiongAn dataset from embodiments of the present invention; wherein, Figure 9 (a) shows the actual distribution of 20 land features in the XiongAn dataset in this embodiment of the invention; Figure 9 (b) is a graph showing the overall classification accuracy of the XiongAn dataset based on the SSFTT comparison algorithm in this embodiment of the invention. Figure 9 (c) is a graph showing the overall classification accuracy of the XiongAn dataset based on the morphFormer comparison algorithm in this embodiment of the invention. Figure 9 (d) is the overall classification accuracy of the XiongAn dataset based on the DBCT comparison algorithm in this embodiment of the invention; Figure 9 (e) is a graph showing the overall classification accuracy of the XiongAn dataset based on the GAHT comparison algorithm in this embodiment of the invention. Figure 9 In the middle (f), the overall classification accuracy of the XiongAn dataset based on the MASSFormer comparison algorithm in the embodiment of the present invention is shown. Figure 9 (g) is the overall classification accuracy map of the XiongAn dataset based on the DCTN comparison algorithm in this embodiment of the invention; Figure 9 (h) is the overall classification accuracy graph of the XiongAn dataset based on the LSFAT comparison algorithm in this embodiment of the invention; Figure 9 In the middle (i), the overall classification accuracy of the XiongAn dataset based on the MambaHSI comparison algorithm in this embodiment of the invention is shown. Figure 9 In the middle (j), the overall classification accuracy of the XiongAn dataset based on the 3DSS-Mamba comparison algorithm in this embodiment of the invention is shown. Figure 9 In the middle (k), the overall classification accuracy of the XiongAn dataset based on the contrast algorithm FS-CGNet in this embodiment of the invention is shown.

[0104] Figure 10 The ten algorithms in this embodiment of the invention use OA (Operational Access) with a training sample ratio of 2%-10% in different datasets; wherein, Figure 10 In (a), the OA of 10 algorithms in the embodiments of the present invention is shown in the Pavia University dataset with a training sample ratio of 2%-10%; Figure 10 (b) shows the OA of 10 algorithms in this embodiment of the invention using a training sample ratio of 2%-10% on the Salinas dataset; Figure 10(c) shows the OA of the 10 algorithms in this embodiment of the invention using a training sample ratio of 2%-10% on the WHU-Hi-LongKou dataset; Figure 10 In the middle (d), the OA of the 10 algorithms in the embodiments of the present invention is shown in the XiongAn dataset with a training sample ratio of 2%-10%;

[0105] Figure 11 To illustrate the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention using different block sizes on different datasets, wherein... Figure 11 In the middle (a), the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention is shown under different block sizes on the Pavia University dataset; Figure 11 (b) shows the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention using different block sizes on the Salinas dataset; Figure 11 (c) represents the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention using different block sizes on the WHU-Hi-LongKou dataset; Figure 11 In the middle (d), the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention is shown under different block sizes on the XiongAn dataset.

[0106] Figure 12 The overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention is shown under different learning rates on different datasets. Figure 12 In the middle (a), the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention is shown under different learning rates on the Pavia University dataset. Figure 12 (b) shows the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention on the Salinas dataset using different learning rates; Figure 12 (c) represents the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention on the WHU-Hi-LongKou dataset using different learning rates; Figure 12 In the middle (d), the overall classification accuracy of the FS-CGNet algorithm in this embodiment of the invention is shown under different learning rates on the XiongAn dataset. Detailed Implementation

[0107] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0108] like Figure 1 - Figure 12As shown, this embodiment provides a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation, including: acquiring a hyperspectral image to be processed; inputting the hyperspectral image to be processed into a hyperspectral image classification model for classification to obtain an image classification result; wherein, the hyperspectral image classification model includes a frequency domain spectral feature extraction module, a multi-scale spectral channel fusion attention module, and a cross-scale global aggregation module connected in sequence.

[0109] First, this embodiment uses a frequency domain spectral feature extraction module. By utilizing the Fast Fourier Transform, this module can extract multi-scale and multi-frequency information from hyperspectral images. This method significantly improves the ability to process high-dimensional data, especially in distinguishing local structural features from global patterns, thus enhancing the model's discriminative ability.

[0110] Secondly, a multi-scale spectral channel fusion attention module is used. This module can fuse spectral information from different scales and adaptively enhance the importance of key features through an attention mechanism, enabling the model to focus on the most critical spectral channels for classification. This mechanism effectively improves classification accuracy and model robustness, ensuring efficient feature fusion.

[0111] Finally, a cross-scale global aggregation module is used, which further enhances the feature representation capability of hyperspectral images by aggregating feature information from multiple scales. This method ensures that information from different scales is fully utilized, optimizes the overall classification task performance, and significantly reduces pixel misclassification and improves classification accuracy, especially in complex hyperspectral image classification tasks.

[0112] The method described in this embodiment is applicable to the classification of hyperspectral image datasets both domestically and internationally, and has strong portability, thus better meeting the needs of image classification.

[0113] The specific implementation process of this embodiment includes:

[0114] 1. The frequency domain spectral feature extraction module specifically includes:

[0115] Assume the input feature map is Where B represents the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively. This module first processes the input feature map through a 1×1 convolutional layer, Conv2D.

[0116]

[0117] Next, the module processes the F feature map using depthwise separable convolution (DWConv), which convolves each channel individually. This convolution operates by convolving each channel while maintaining the independence of each channel. The convolution operation is as follows:

[0118]

[0119] Then, after applying the activation function GELU, the result is:

[0120] F GELU =GELU(F dw )

[0121] Before frequency domain processing, the module rearranges F according to the block size (9 in this case). GELU The feature map is transformed into a structure composed of multiple patches. Assuming each patch is 9x9 in size, the rearranged shape is as follows:

[0122] Q,K,V=F GELU ·chunk(3,dim=1)

[0123] In the formula, Q, K, and V represent the query, key, and value, respectively, and chunk represents the partitioning along a given dimension;

[0124] Next, Fourier transforms Q and K are performed to transform the features in the spatial domain to the frequency domain, and their two-dimensional fast Fourier transforms are calculated respectively:

[0125]

[0126] in, This represents a two-dimensional Fast Fourier Transform. The Q-value after the Fourier Transform is... patch and K patch It corresponds to the frequency domain component.

[0127] In the frequency domain, the module calculates Q by element-wise multiplication. patch and K patch The similarity is calculated using the following formula:

[0128] Out fft =Q fft ×K fft

[0129] Then, the frequency domain information is converted back to the spatial domain through the inverse Fourier transform:

[0130]

[0131] in, This represents the two-dimensional inverse Fourier transform. The result of the inverse Fourier transform is a 9×9 feature map, which can be restored to the spatial domain. Out represents the spatial domain output.

[0132] The output feature map after the inverse transform is rearranged to restore the dimensions of the original inputs Q and K. Next, it is normalized using LayerNorm to ensure that each channel of the feature map has a similar distribution. Then, a non-linear transformation is performed using the GELU activation function.

[0133] Out = GELU(LayerNorm(Out))

[0134] Finally, V is multiplied element-wise with the processed Out to obtain the final output feature map. A 1×1 convolutional layer is then used to restore the number of output channels to the original C, resulting in the final output Out. FDSFM It can be represented as:

[0135]

[0136] 2. The multi-scale spectral channel fusion attention module specifically includes:

[0137] A multi-scale spectral channel fusion attention module enhances the representation of input image features by calculating multi-scale channel attention and spatial attention. Channel attention is calculated through multiple 1D convolution operations with different kernel sizes. The specific steps are as follows: First, the input is subjected to global average pooling to obtain a feature map of size [B,C,1,1], where B is the batch size and C is the number of channels. Then, different convolution kernel sizes s1,s2,…,s are applied to the pooled feature map. k 1D convolutions are used to obtain the response at each scale. Then, through... Transform the channel dimensions to obtain the channel attention for each scale.

[0138] Channel attention at each scale can be expressed by the following formula:

[0139]

[0140] Among them, Conv i (·) represents the convolution operation at the i-th scale, where x is the input after average pooling. This is a channel convolutional layer used to adjust the channel dimensions; σ is the sigmoid activation function. Channel attention at each scale;

[0141] Finally, the channel attention at all scales is averaged to obtain multi-scale channel attention:

[0142]

[0143] Spatial attention is extracted in the spatial dimension through convolution operations. Given input features... Figure XSpatial attention is calculated as follows:

[0144]

[0145] in, It is a 1×1 convolution operation used to generate a spatial attention map. F MSA For the results of multi-scale channel attention operations, F SA This is the result of spatial attention calculation.

[0146] A mixing module, Mix, is defined to generate the final output by weighted fusion of two input feature maps, F1 and F2. Specifically, the input to the Mix module is two feature maps, and the weights are determined by learnable parameters. The control and blending mode uses the Sigmoid activation function to adjust the blending ratio.

[0147] Given input feature maps F1 and F2, the fusion process can be represented by the following formula:

[0148]

[0149] in, It is the Sigmoid activation function: F Mix It is the output feature map.

[0150] Multi-scale channel attention and spatial attention are weighted and fused using the Mix module. The final fused output F MSFAM It can be represented as:

[0151] F MSFAM =X·Mix(F MSA ,F SA )

[0152] 3. The cross-scale global aggregation module specifically includes:

[0153] A cross-scale global aggregation module is used, which further enhances the feature representation capability of hyperspectral images by aggregating feature information from multiple scales. Assume two input feature maps, L (left feature map) and R (right feature map), both with shapes B×C×H×W. First, ensure that L and R feature maps have the same spatial dimensions. If their spatial dimensions differ, then upsampling is used to resize the R feature map to the size of L.

[0154]

[0155] in, This is the R feature map after upsampling, making it the same size as the L feature map. `upsample` is the upsampling operation, `shape` is the feature map shape, and `if` represents a conditional statement.

[0156] After dimensional alignment, the cross-scale global aggregation module performs feature fusion on the input L and the adjusted R. First, a convolution operation Conv is performed on the R feature map. CBR Then, the convolutional R and L are concatenated to form a new fused feature map. At this point, the convolution operation Conv... cat The stitched feature maps are further processed to generate a fused feature map O:

[0157]

[0158] Where L is the original feature map, Represents the feature map R after the convolution operation. Conv CBR (·) refers to a sequence containing 3×3 convolutional layers, batch normalization layers, and ReLU activation functions.

[0159] To capture global contextual information, the cross-scale global aggregation module uses dilated convolutions to expand the receptive field of the feature maps. Dilated convolutions increase the extent of the receptive field, thereby capturing a wider range of contextual information. After performing the expanded convolution operation on the fused feature map O, it is added to the original feature map to generate a context-enhanced feature map O. g :

[0160] O g =O + DilatedConv(O)

[0161] Secondly, Global Average Pooling (GAP) is used to process feature map O to extract global information. The global features are compressed using a 1×1 convolution to obtain scaled global features. Then, bilinear interpolation is used to sample the global feature map to the spatial size of the original input feature map.

[0162] O g =O g +upsample(Conv2d 1×1 (GAp(O)))

[0163] In the formula, GAP represents global average pooling;

[0164] After obtaining the context-enhanced feature maps, the cross-scale global aggregation module performs convolution operations on the L and r feature maps respectively, and uses the context-enhanced feature map O... gThey are weighted. The weighted L and R feature maps are concatenated with the previous feature map O, and then further fused into a new feature map O2 through a convolution operation:

[0165]

[0166] Where L2=Conv2D 3×3 (L)⊙O and It is the feature map after convolution and weighting, and ⊙ represents element-wise multiplication.

[0167] Then, the cross-scale global aggregation module generates weighted coefficients through local mean pooling and local max pooling operations. The pooling result is subtracted from the local mean pooling result, and a convolution operation is used to generate the weighted coefficients. Then, the weighted coefficients are normalized to the [0,1] interval using the Sigmoid activation function and multiplied element-wise with the O2 feature map to obtain the final output feature map O. f :

[0168]

[0169] Among them, O g2 These are the dynamic weighting coefficients obtained through the above weighting process. This represents matrix multiplication.

[0170] Three 1×1 kernel convolutional layers are applied to enhance the non-linearity of the previously obtained feature maps, generating three distinct feature matrices:

[0171]

[0172] In the formula, reshape is the transformation matrix;

[0173] Calculate the similarity between Q and K (by multiplying the matrix to obtain a weighted matrix), then multiply the weighted matrix by a matrix containing V to obtain a weighted feature map:

[0174]

[0175] Where G represents the enhanced feature obtained through adaptive weighting.

[0176] Add the enhanced feature G to the original feature map O. f Then, it is normalized by convolution and using the Sigmoid activation function:

[0177]

[0178] Here, δ is a learnable parameter that controls the weights of the enhanced features.

[0179] Finally, for the input feature O fPerform average pooling and max pooling operations, concatenate the results, and then pass them through a convolutional layer to obtain the final output F. CGAM :

[0180] F CGAM =Conv CBR (Conv cat (Avg(O f ),Max(O f )))

[0181] Figure 1 This is a flowchart of a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation. The process includes: First, a frequency domain spectral feature extraction module is proposed, which captures multi-scale and multi-frequency information using techniques such as Fast Fourier Transform, enhancing the ability to distinguish between local and global features. Second, a multi-scale spectral channel fusion attention module is introduced, which adaptively focuses on key spectral channels, significantly improving classification accuracy and robustness. Finally, a cross-scale global aggregation module is designed to aggregate features from multiple scales, ensuring optimal utilization of cross-scale information and further improving overall classification performance.

[0182] Figure 2 This is a framework diagram of a hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation. The basic framework includes a frequency domain spectral feature extraction module, a multi-scale spectral channel fusion attention module, and a cross-scale global aggregation module.

[0183] Figure 3 This is a structural diagram of the frequency domain spectral feature extraction module. By utilizing the Fast Fourier Transform, this module can extract multi-scale and multi-frequency information from hyperspectral images. This method significantly improves the ability to process high-dimensional data, especially in distinguishing local structural features from global patterns, thus enhancing the model's discriminative ability.

[0184] Figure 4 This is a diagram of the multi-scale spectral channel fusion attention module. This module can fuse spectral information from different scales and adaptively enhance the importance of key features through an attention mechanism, enabling the model to focus on the most critical spectral channels for classification tasks. This mechanism effectively improves classification accuracy and model robustness, ensuring efficient feature fusion.

[0185] Figure 5 This is a structural diagram of the cross-scale global aggregation module. This module further enhances the feature representation capability of hyperspectral images by aggregating feature information from multiple scales. This method ensures that information from different scales is fully utilized, optimizing the overall performance of the classification task. Particularly in complex hyperspectral image classification tasks, it significantly reduces pixel misclassification and improves classification accuracy.

[0186] Figure 6 (a) is a real hyperspectral image dataset from the Pavia University dataset, sourced from the University of Pavia in northern Italy. The data was captured on July 8, 2002, by the Reflectance Optical Systems Imaging Spectroradiometer (ROSIS). This dataset contains 115 spectral bands with a spatial size of 610 × 340 pixels. The spectral resolution is 4 nanometers, covering a range from 0.43 to 0.86 micrometers, with a spatial resolution of 3.7 meters per pixel. The ground truth data for this dataset includes nine different urban land cover categories, totaling 42,776 labeled pixels. After removing 12 noisy bands, the remaining 103 bands were used for classification tasks. Figure 6 (b)- Figure 6 (k) shows the classification results of 10 other algorithms on the Pavia University dataset, including SSFTT, morphFormer, DBCT, GAHT, MASSFormer, DCTN, LSFAT, MambaHSI, 3DSS-Mamba, and FS-CGNet. By comparing these classification results, it can be seen that the proposed FS-CGNet algorithm (i.e., Figure 6 (k) shows the best classification performance.

[0187] Figure 7 (a) is a real hyperspectral image dataset from the Salinas dataset, acquired in 1998 using the AVIRIS sensor in Salinas Valley, California, USA. This dataset covers the agricultural area of ​​Salinas Valley and contains a single image with a spatial size of 512 × 217 pixels and 224 spectral channels. The spatial resolution of the spectral channels is 3.7 meters per pixel. For the purposes of this example, 20 water absorption bands were excluded, leaving 204 bands for experimental analysis. The Salinas dataset contains 54,129 pixels representing 16 real-world land cover categories, including vegetables, vineyards, and bare soil. Figure 7 (b)- Figure 7 (k) shows the classification results of 10 other algorithms on the Salinas dataset, including SSFTT, morphFormer, DBCT, GAHT, MASSFormer, DCTN, LSFAT, MambaHSI, 3DSS-Mamba, and FS-CGNet. By comparing these classification results, it can be seen that the proposed FS-CGNet algorithm (i.e., Figure 7 (k) shows the best classification performance.

[0188] Figure 8(a) is a real hyperspectral image dataset. The WHU-Hi-LongKou dataset was captured between 13:49 and 14:37 on July 17, 2018, in Longkou Town, Hubei Province, China, using an 8mm Headwall Nano-Hyperspec imaging sensor and a DJI Matrice 600Pro (DJI M600Pro) drone platform. Data acquisition was conducted under clear, cloudless weather conditions with an air temperature of approximately 36°C and a relative humidity of approximately 65%. The study area is a typical agricultural landscape, encompassing six crop species: maize, cotton, sesame, broadleaf soybean, narrowleaf soybean, and rice. The drone flew at an altitude of 500 meters, capturing images with a size of 550 × 400 pixels and 270 spectral bands ranging from 400 to 1000 nanometers. The spatial resolution of the hyperspectral imagery carried by the drone is approximately 0.463 meters per pixel. Figure 8 (b)- Figure 8 (k) shows the classification results of 10 other algorithms on the WHU-Hi-LongKou dataset, including SSFTT, morphFormer, DBCT, GAHT, MASSFormer, DCTN, LSFAT, MambaHSI, 3DSS-Mamba, and FS-CGNet. By comparing these classification results, it can be seen that the proposed FS-CGNet algorithm (i.e., Figure 8 (k) shows the best classification performance.

[0189] Figure 9 (a) is a real hyperspectral image dataset. The XiongAn dataset was acquired using the Full Spectrum Multimodal Imaging Spectroradiometer developed by the Shanghai Institute of Technical Physics, Chinese Academy of Sciences, as part of the High Resolution Earth Observation Program. This spectroradiometer covers a spectral range of 400 to 1000 nanometers, with a total of 250 spectral bands. The image size is 3750 × 1580 pixels, with a spatial resolution of 0.5 meters, providing high-precision data for detailed remote sensing analysis. Figure 9 (b)- Figure 9 (k) shows the classification results of 10 other algorithms on the XiongAn dataset, including SSFTT, morphFormer, DBCT, GAHT, MASSFormer, DCTN, LSFAT, MambaHSI, 3DSS-Mamba, and FS-CGNet. By comparing these classification results, it can be seen that the proposed FS-CGNet algorithm (i.e., Figure 9 (k) shows the best classification performance.

[0190] Figure 10 (a)- Figure 10(d) shows the changes in classification accuracy of the ten algorithms on four different datasets using different proportions of training samples.

[0191] Reference Figure 11 (a)- Figure 11 (d) This study investigated the variation of overall classification accuracy with different block sizes. Figure 11 In (a), for the Pavia University dataset, the best overall classification accuracy was achieved when the block size was 15×15. Figure 11 (b) shows that for the Salinas dataset, the optimal classification performance was achieved when the block size was 17×17. Figure 11 In (c), for the WHU-Hi-LongKou dataset, a block size of 15×15 showed the best classification accuracy. Finally, in Figure 11 In (d), for the XiongAn dataset, the optimal classification accuracy was achieved when the block size was 17×17.

[0192] Reference Figure 12 (a)- Figure 12 (d) This study investigated the variation of overall classification accuracy under different learning rates. Figure 12 In (a), for the PaviaUniversity dataset, the best overall classification accuracy was achieved with a learning rate of 0.0002. Figure 12 In (b), for the Salinas dataset, the best classification accuracy is achieved with a learning rate of 0.0004. Figure 12 (c) shows that the highest overall classification accuracy was achieved with a learning rate of 0.0003 on the WHU-Hi-LongKou dataset. Figure 12 In (d), the best classification performance was also achieved with a learning rate of 0.0003 for the XiongAn dataset.

[0193] First, this embodiment proposes a frequency domain spectral feature extraction module, which captures multi-scale and multi-frequency information by utilizing techniques such as Fast Fourier Transform, enhancing the ability to distinguish between local and global features. Second, this embodiment introduces a multi-scale spectral channel fusion attention module, which significantly improves classification accuracy and robustness by adaptively focusing on key spectral channels. Finally, this embodiment designs a cross-scale global aggregation module, aggregating features from multiple scales to ensure optimal utilization of cross-scale information, thereby further improving overall classification performance. The method in this embodiment is based on a deep learning framework, fully leveraging the deep capabilities of convolutional neural networks in feature extraction, the advantages of multi-branch attention mechanisms in focusing on key information, and the enhanced effect of cross-scale feature fusion. By designing hyperspectral remote sensing image datasets adapted to different spatial resolutions, this method possesses strong portability and can efficiently handle image classification tasks in various complex scenarios, significantly improving classification accuracy and robustness.

[0194] The above are specific embodiments of this example and are not intended to limit this example. The hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation provided in this example is also applicable to classifying other non-hyperspectral images. Some adjustments and optimizations may be made without departing from the essence and scope of this example, and the scope of protection of this example shall be determined by the claims.

[0195] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation, characterized in that, include: Acquire the hyperspectral image to be processed; The hyperspectral image to be processed is input into a hyperspectral image classification model for classification to obtain the image classification result; wherein, the hyperspectral image classification model includes a frequency domain spectral feature extraction module, a multi-scale spectral channel fusion attention module, and a cross-scale global aggregation module connected in sequence; The processing steps of the hyperspectral image classification model specifically include: The hyperspectral image to be processed is input into the frequency domain spectral feature extraction module, and multi-scale and multi-frequency information is extracted from the hyperspectral image to be processed by combining the fast Fourier transform method to obtain spectral information at different scales. Spectral information at different scales is input into the multi-scale spectral channel fusion attention module for feature fusion, and the importance of key features is adaptively enhanced through the attention mechanism to obtain feature information at different scales; Input feature information at different scales into the cross-scale global aggregation module to aggregate feature information at different scales and output classification results; The processing procedure of the cross-scale global aggregation module specifically includes: Let the input feature maps be the left feature map. and right feature map ,judge and Does the feature map have the same spatial dimension? If the dimensions are different, then an upsampling operation is performed. The size of the feature map is adjusted to Size: ； in, It is after upsampling processing Feature map For upsampling operation, For the shape of the feature map, Represents judgment; After dimensional alignment, the cross-scale global aggregation module performs cross-scale global aggregation on the input. and the adjusted To perform feature fusion, firstly, for Convolution operation on feature maps Then convolutional and The features are then concatenated to form a new fused feature map. At this point, a convolution operation is used. The stitched feature maps are further processed to generate a fused feature map. : ； in, It is the original feature map. This represents the feature map after the convolution operation; Refers to a collection A sequence of convolutional layers, batch normalization layers, and ReLU activation functions; In the fused feature map After performing the extended convolution operation, it is added to the original feature map to generate a context-enhanced feature map. : ； Feature maps are processed using global average pooling. Extract global information, through Convolution compresses the global features to obtain scaled global features, and bilinear interpolation is used to sample the global feature map to the size of the original input feature map. ； In the formula, For global average pooling; After obtaining the context-enhanced feature maps, the cross-scale global aggregation module processes the feature maps respectively. and feature map Perform convolution operations and use context-enhanced feature maps. For feature maps and feature map Weighted feature map and feature map Compared with previous feature maps The features are concatenated and further fused into a new feature map through convolution. : ； in, , It is the feature map after convolution and weighting. This indicates element-wise multiplication; Weighted coefficients are generated through local mean pooling and local max pooling operations, and then normalized using the Sigmoid activation function. Intervals, and feature maps Element-wise multiplication yields the final output feature map. : ； in, These are the dynamic weighting coefficients obtained through the above weighting process. Represents matrix multiplication; Three applications The kernel convolutional layer, to enhance the nonlinearity of the previously obtained feature maps, generates three distinct feature matrices: ； In the formula, This is the transformation matrix; calculate and The similarity between them is then used to calculate the weighted matrix and include... Multiply the matrices to obtain the weighted feature map: ； in, This represents the enhanced features obtained through adaptive weighting; Enhance features Add to the original feature map Then, it is normalized by convolution and using the Sigmoid activation function: ； in, It is a learnable parameter that controls the weights of the enhanced features; For input features Perform average pooling and max pooling operations, concatenate the results, and then pass them through a convolutional layer to obtain the final output. : 。 2. The hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation according to claim 1, characterized in that, The training process of the hyperspectral image classification model specifically includes: Acquire training data, which includes hyperspectral training images and corresponding image classification results; An initial hyperspectral image classification model is constructed. The training data is input into the initial hyperspectral image classification model for classification. The model is trained with the goal of minimizing the loss between the initial training result and the image classification result corresponding to the hyperspectral training image. The trained hyperspectral image classification model is then obtained.

3. The hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation according to claim 1, characterized in that, The processing flow of the frequency domain spectral feature extraction module specifically includes: Input the hyperspectral image to be processed ,in Indicates batch size. It is the number of channels. and These are the height and width of the feature map, respectively. Through a Conv2D convolutional layers process the input image: ； Feature maps are processed using depthwise separable convolution (DWConv). : ； After applying the GELU activation function: ； In the formula, The feature map is the result of depthwise separable convolution. The feature map after applying the activation function; For feature maps Rearrange: ； In the formula, These are queries, keys, and values, respectively. To divide into blocks on a given dimension; Perform Fourier transform and The features in the spatial domain are transformed to the frequency domain, and then a two-dimensional fast Fourier transform is performed: ； ； in, This represents the two-dimensional Fast Fourier Transform, and the result of the Fourier Transform. and These are the corresponding frequency domain components; In the frequency domain, the module performs calculations through element-wise multiplication. and Similarity: ； In the formula, The result of the similarity calculation; The frequency domain information is converted back to the spatial domain using the inverse Fourier transform: ； in, This represents the two-dimensional inverse Fourier transform. Output results for the spatial domain; The output feature maps after the inverse transform are rearranged to recover the original input. and In terms of dimensions, LayerNorm is used to normalize the feature map so that each channel of the feature map has a similar distribution; Nonlinear transformation using the GELU activation function: ； Finally, Compared with the processed Element-wise multiplication yields the final output feature map, which is then processed... The convolutional layer restores the number of output channels to the original value. Final output for: 。 4. The hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation according to claim 1, characterized in that, The processing flow of the multi-scale spectral channel fusion attention module specifically includes: Input spectral information at different scales, and then perform global average pooling to obtain a pool of size [value missing]. The feature map, where It refers to the batch size. It is the number of channels; By applying convolutional kernels of different sizes to the pooled feature maps The 1D convolution is used to obtain the response at each scale. Then, the channel dimension is transformed by channel convolution to obtain the channel attention at each scale. Channel attention at each scale can be expressed by the following formula: ； in, Indicates the first Scale-based convolution operations, It is the input after average pooling. It is a channel convolutional layer, used to adjust the channel dimensions. It is the Sigmoid activation function. Channel attention at each scale; Averaging the channel attention across all scales yields multi-scale channel attention: ； Spatial attention is extracted in the spatial dimension through convolution operations, given an input feature map. Spatial attention is calculated as follows: ； in, It is Convolution operations are used to generate spatial attention maps; Define a mixing module Mix, which fuses two input feature maps using weighted methods. and Generate the final output.

5. The hyperspectral image classification method based on spectral channel fusion and cross-scale global aggregation according to claim 4, characterized in that, The processing procedure of the Mix module specifically includes: Given input feature map and The fusion process can be represented by the following formula: ； in, It is the Sigmoid activation function. It outputs a feature map; Multi-scale channel attention and spatial attention are weighted and fused through the Mix module, and the final fused output is... It can be represented as: ； In the formula, For the input image, This is the result of multi-scale channel attention computation. This is the result of spatial attention calculation.

Images