Hyperspectral classification method based on residual attention and swin transformer fusion

Abstract

The invention provides a hyperspectral classification method based on fusion of residual attention and swin transform, and belongs to the field of hyperspectral classification, and the method comprises the steps: constructing a plurality of modules, including a residual attention module and a swin transform module; the method comprises the following steps: preprocessing input data through a 2D CNN, and generating a patch image through a data enhancement technology (such as random flipping and random cutting); the patches are processed through an embedded layer and then enter a swin transfer module, and feature extraction is carried out through a multi-head self-attention mechanism (W-MSA). Then, the features are further processed by a multi-layer perceptron (MLP), and patch merging is performed. And finally, outputting a classification result through an average pooling layer and a Softmax classifier. According to the invention, through cooperative work of multiple modules, the classification precision of the hyperspectral image and the classification effect in a few-sample scene are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and remote sensing hyperspectral image processing, and in particular to a hyperspectral classification method based on the fusion of residual attention and swin transformer. Background Art

[0002] Hyperspectral technology has shown broad application prospects in many fields by virtue of its accurate capture of spectral information of objects. In agriculture, hyperspectral technology can be used for crop health monitoring, pest and disease detection, and yield prediction, and can accurately judge the condition of plants by analyzing spectral characteristics. Secondly, in environmental monitoring, it can help identify pollution sources, monitor water and air quality, and provide important data support for ecological protection. In the field of mineral exploration, hyperspectral technology can effectively identify mineral components and improve resource detection efficiency. In addition, in medical diagnosis and food safety testing, hyperspectral imaging technology can non-destructively detect material components to ensure food quality and early diagnosis of diseases. Therefore, hyperspectral technology has important development potential in many fields such as smart agriculture, resource exploration, and environmental protection in the future.

[0003] The development of hyperspectral image classification can be roughly divided into three stages. In the early stage, traditional machine learning methods such as support vector machine (SVM) [Melgani F, Bruzzone L. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on geoscience and remote sensing. 2004 Aug 16; 42 (8): 1778-90.] and k-nearest neighbor (k-NN) [Ma L, Crawford MM, Tian J. Local manifold learning-based $k$-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing. 2010 Aug 23; 48 (11): 4099-109.] were mainly used to classify images based on spectral features. However, these methods are difficult to fully utilize spatial information and have limited classification accuracy. Subsequently, researchers began to combine principal component analysis (PCA) [Rodarmel C, Shan J.Principal component analysis for hyperspectral image classification. Surveying and Land Information Science. 2002Jun 1; 62(2): 115-22.] and other dimensionality reduction techniques to extract spectral-spatial features and improve classification results. However, most of these feature extraction methods rely on manual design and have poor adaptability. With the rise of deep learning, convolutional neural networks (CNN) [Lee H, Kwon H. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing. 2017Jul11; 26(10): 4843-55.] and 3D convolutional networks have gradually become the mainstream methods for hyperspectral classification. They can automatically extract spectral and spatial features, greatly improving classification accuracy. In order to further optimize the deep model, residual networks (ResNet) and attention mechanisms were introduced to solve the gradient vanishing problem in deep networks and enhance the model's ability to capture key information.Recently, self-attention based models such as swin transformer [Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE / CVF international conference on computer vision 2021 (pp. 10012-10022).] have begun to be applied to hyperspectral classification. Through multi-scale perception, the global feature perception ability of the model is further improved, becoming the latest development direction of hyperspectral classification.

[0004] Existing hyperspectral classification technologies still face challenges when dealing with small sample data. Since hyperspectral images contain rich spectral and spatial information, but sample annotation is expensive and time-consuming, it is difficult to obtain manually labeled classification samples of a large number of hyperspectral images. In the case of few samples, it is difficult for existing models to fully mine the potential information in the data, resulting in reduced classification accuracy. How to effectively utilize unlabeled data with limited labeled samples and mine the spatial and spectral connections between unlabeled data and labeled samples has become a key issue in improving classification accuracy. In addition, how to design a model so that it can still maintain high generalization ability and universality with a small number of training samples to cope with complex hyperspectral data environments is one of the core problems that need to be solved urgently in the field of hyperspectral classification. Summary of the invention

[0005] In order to solve the problems mentioned above, the present invention proposes a hyperspectral classification method based on residual attention and swintransformer fusion to solve the problem of inaccurate image classification and low precision in a small number of sample scenarios. The contents of the present invention are as follows:

[0006] The purpose of the present invention is to provide a hyperspectral classification method based on residual attention and swin transformer fusion, which is characterized by comprising the following steps:

[0007] Step 1: Obtain a hyperspectral remote sensing image with labeled categories and preprocess it into multiple training image blocks, wherein the preprocessing includes spectral dimension reduction, edge filling and image slicing;

[0008] Step 2: Process the spectral and spatial features of the input trained image patches using a residual attention module to enhance the spectral-spatial feature expression ability and improve the feature extraction effect of the network. The residual attention module includes convolutional layers (Conv3D, Conv2D), residual connections, a spectral attention module, and a spatial attention module. The differential attention module uses a spectral-spatial residual fusion method, utilizes residual connections to improve the gradient transfer ability in deep networks, and at the same time focuses on key features in the classification task through the attention mechanism to suppress redundant features.

[0009] Step 3: Generate patches through data augmentation techniques for the hyperspectral data processed in Step 2. The data augmentation techniques include random cropping and random flipping.

[0010] Step 4: Use the swin transformer module to perform multi-scale feature extraction on the patches in Step 3 to enhance the model's multi-scale perception of hyperspectral images.

[0011] Step 5: Use a global average pooling layer to extract global features from the data processed by the swin transformer module in Step 4.

[0012] Step 6: After the global feature extraction in Step 5, perform image classification through a classifier (Dense layer).

[0013] To better implement the above technical solution, in Step 1 of the hyperspectral classification method based on the fusion of residual attention and swin transformer of the present invention, the dimensionality reduction in the spectral dimension is to perform spectral dimensionality reduction on the hyperspectral image through principal component analysis (PCA), reduce the dimension of the spectral channels from the original K bands to L bands (L < K), and generate the hyperspectral data after dimensionality reduction.

[0014] To better implement the above technical solution, in Step 1 of the hyperspectral classification method based on the fusion of residual attention and swin transformer of the present invention, the edge padding is to perform edge padding on the data after dimensionality reduction, and the padding width is P to ensure that each pixel has complete surrounding information when extracting image patches.

[0015] To better implement the above technical solution, in Step 1 of the hyperspectral classification method based on the fusion of residual attention and swin transformer of the present invention, the image slicing is to divide the data into N three-dimensional cube blocks, where N = H × W, where M and N are the number of rows and columns of the image, representing the number of spatial pixels of the hyperspectral image, and use them as independent training samples for subsequent model training.

[0016] In order to better implement the above technical solution, the swin transformer module in the hyperspectral classification method based on residual attention and swin transformer fusion of the present invention realizes multi-scale perception of spectral and spatial features through a multi-head self-attention mechanism (W-MSA), and the multi-head self-attention calculation formula is:

[0017]

[0018] Among them, Q, K and V represent the query matrix, key matrix and value matrix respectively, and d is the scaling factor; through the windowed multi-head self-attention mechanism (W-MSA), the module can calculate self-attention within a local window and capture features of different scales; combined with the patch merging layer, the spatial resolution is downsampled layer by layer to further enhance the classification and discrimination ability of the model.

[0019] In order to better implement the above technical solution, the hyperspectral classification method based on residual attention and swin transformer fusion in the present invention is based on the residual attention and swin transformer model for multiple rounds of training, and the process is as follows:

[0020] Step S1, preparing a hyperspectral dataset for training;

[0021] Step S2, use loadData to load the hyperspectral dataset;

[0022] Step S3, preprocessing the hyperspectral data set in step S2 and inputting the data into the model;

[0023] Step S4: Input the label and the prediction result of the model into the loss function to calculate the loss;

[0024] Step S5: Back propagate according to the calculated loss value and update the weight of the model;

[0025] Step S6: Iterate steps S3 to S5 epochs to obtain the final trained model.

[0026] In order to better implement the above technical solution, the hyperspectral dataset in step S1 of the hyperspectral classification method based on residual attention and swintransformer fusion of the present invention includes the Pavia University dataset and the Chikusei dataset.

[0027] The beneficial effects of the present invention are:

[0028] 1. This paper combines the residual attention module with the swin transformer to propose a multi-level, multi-module efficient processing solution for the dimensionality curse, few sample problems and feature extraction difficulties in hyperspectral image classification.

[0029] 2. The present invention effectively reduces the dimension of hyperspectral data through the PCA dimensionality reduction step, reduces the computational complexity and removes redundant information, ensuring that the model is more efficient when processing high-dimensional data. At the same time, combined with the residual attention module, it adaptively captures important features in the spectrum and space, avoids the gradient vanishing problem in the deep network through residual connection, and ensures the effective transmission and learning of features.

[0030] 3. The present invention adopts channel attention and spatial attention modules to focus on key information from the spectral and spatial dimensions, respectively, further enhancing the model's discrimination ability. Through the combination of 3D CNN and 2D CNN, the model can realize multi-level feature extraction of spectral-spatial information, where 3D CNN is used to process spectral and spatial information simultaneously, and 2D CNN refines the feature extraction of spatial dimensions. At the same time, the present invention designs data enhancement operations such as random flip (RandomFlip) and random cropping (RandomCrop), which effectively improves the generalization ability of the model with a small number of training samples.

[0031] 4. The present invention combines the Swin transformer module and realizes multi-scale perception of hyperspectral images through the multi-head self-attention mechanism (W-MSA). The Swin transformer uses the local windowed attention mechanism to gradually integrate local and global information, thereby enhancing the model's ability to perceive the global features of the image. This design effectively solves the problem in the few-sample scenario. The model can fully explore the spatial connection between unlabeled data and labeled samples, thereby improving the classification accuracy.

[0032] 5. The present invention solves the problems of excessively high dimensions, insufficient sample labeling, and difficulty in global feature extraction in hyperspectral classification through the organic combination of multiple modules. Under the condition of a small number of labeled samples, the classification accuracy and the universality of the model are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A model diagram of the method of the present invention;

[0034] Figure 2 is a flow chart of the method of the present invention;

[0035] Figure 3Classification results for the Pavia University dataset: (a) false color image; (b) ground truth image; (c) FullyContNet; (d) GhostNet; (e) HybridSN; (f) M3DCNN; (g) SSRN; (h) UNet; (i) VIT; (j) Ours;

[0036] Figure 4 The classification diagram of the Pavia University dataset: (a) FullyContNet; (b) GhostNet; (c) HybridSN; (d) M3DCNN; (e) SSRN; (f) UNet; (g) VIT; (h) Ours;

[0037] Figure 5 Classification results for the Chikusei dataset: (a) false color image; (b) ground truth image; (c) FullyContNet; (d) GhostNet; (e) HybridSN; (f) M3DCNN; (g) SSRN; (h) UNet; (i) VIT; (j) Ours; Figure 6 The classification map of the Chikusei dataset: (a) FullyContNet; (b) GhostNet; (c) HybridSN; (d) M3DCNN; (e) SSRN; (f) UNet; (g) VIT; (h) Ours. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the embodiments.

[0039] The following examples are used to illustrate the present invention, but they cannot be used to limit the scope of protection of the present invention. The conditions in the examples can be further adjusted according to specific conditions. Simple improvements to the method of the present invention under the premise of the concept of the present invention belong to the scope of protection claimed in the present invention.

[0040] The present invention provides a hyperspectral classification method based on residual attention and swin transformer fusion, such as Figure 1 As shown, the following steps are included:

[0041] Step S1: Hyperspectral data preprocessing

[0042] Two public datasets are used, namely the Pavia University dataset and the Chikusei dataset.

[0043] (1) The Pavia University data is part of the hyperspectral data of the city of Pavia, Italy, taken by the German airborne reflective optics spectral imager (Reflective Optics Spectrographic Imaging System, ROSIS-03) in 2003. The spectral imager continuously images 115 bands in the wavelength range of 0.43-0.86μm, and the spatial resolution of the image is 1.3m. Among them, 12 bands are eliminated due to noise, so the images formed by the remaining 103 spectral bands are generally used. The size of the data is 610×340, so it contains a total of 2207400 pixels, but it contains a large number of background pixels. There are only 42776 pixels containing ground objects, and these pixels contain 9 types of ground objects, including trees, asphalt roads (Asphalt), bricks (Bricks), meadows (Meadows), etc.

[0044] (2) The Chikusei data was taken by the Headwall Hyperspec-VNIR-C sensor in Chikusei, Japan on July 29, 2014. The data contains 128 bands ranging from 343-1018nm, a size of 2517*2335, and a spatial resolution of 2.5m. There are 19 types of features, including urban and rural areas. The data was made public by Dr. Naoto Yokoya and Prof. Akira Iwasaki of the University of Tokyo.

[0045] The present invention aims at the problem that the Chikusei hyperspectral dataset cannot be directly read, and proposes a special processing method. Since the file format and structure of the Chikusei dataset are different from those of the conventional hyperspectral dataset, the traditional loading method cannot correctly extract the data and labels therein. In order to solve this problem, the present invention uses the load_hdf5_mat function to load the hyperspectral data, which can correctly read the spectral information in the Chikusei dataset. At the same time, the ground truth label is extracted by the load_old_mat function. Since the label data has a deeply nested structure, it needs to be indexed to obtain the correct classification label.

[0046] In addition, since the dimensional order of the Chikusei dataset is (bands, height, width), which is different from the usual hyperspectral data format, the present invention transposes the data and adjusts it to the format of (height, width, bands) to ensure that it is consistent with the input requirements of the classification model. Through this special processing method of data and labels, the present invention successfully solves the problem that the Chikusei dataset cannot be directly read, and provides standardized input data for subsequent hyperspectral classification tasks.

[0047] PCA dimensionality reduction: Project the data from the original space to a new space through linear transformation to maximize the variance of the data. The formula is as follows:

[0048] X pca =XW

[0049] Where X is the original data matrix, W is the eigenvector matrix, X pca It is the data after dimensionality reduction. PCA reduces the dimensionality of the data by extracting the principal components (eigenvectors) of the data while retaining as much information as possible.

[0050] Patches extraction: Patches are extracted from hyperspectral image data. First, the image edges are filled with a padding width of margin to ensure that there is no missing data in the extraction window. Then, patches of a certain window are extracted from the padded image to generate image patches and corresponding labels. The center pixel position of each patch is (r-margin, c-margin), where margin is half the size of the window. Since the edge pixels cannot completely extract the window, the padding width is set to (windowSize-1) / 2. Finally, the image patches with zero labels are removed, and the final number of patches is M×N, where M and N are the number of rows and columns of the image.

[0051] Step S2: Hyperspectral feature extraction

[0052] In step S2, the specific implementation process is as follows:

[0053] 1. Channel attention module: reshape the input three-dimensional data conv_layer1_1, convert it into a two-dimensional shape, and perform global average pooling on it to obtain global features. Generate attention weights through the fully connected layer and apply these weights to the original data to enhance important features on the channel.

[0054] X ca =Flatten(GlobalAveragePooling2D(Reshape(conv layer11_1 )))

[0055] 2. Spatial attention module: Spatial pooling is performed on the feature conv_layer11resc that has been processed by channel attention. The results of average pooling and maximum pooling are calculated, and they are concatenated and passed through the convolution layer to generate the spatial attention feature. This feature is multiplied with the original feature to obtain the feature after spatial attention weighting.

[0056] X sa =Conv2D(Concat(ReduceMean(conv layer11resc ),ReduceMax(conv layer11resc )))

[0057] 3. Module integration and residual connection: Combine the output of the channel attention and spatial attention modules, perform convolution processing and discard layer operations on the features, and merge the processed features with the features of the jump connection. The features of the jump connection are generated through the convolution layer and finally added to the processed features to achieve residual connection. This process ensures the effectiveness of the features after the attention mechanism while retaining the information of the original features.

[0058] X = Conv3D(Dropout(Conv3D(X sa )))+Conv3D(input layer )

[0059] Step S3: Random cropping and random flipping

[0060] Random cropping: By randomly selecting a part of the image for training, overfitting in certain areas is reduced, allowing the model to focus more on overall features rather than local details.

[0061] Random flipping: By randomly flipping the image horizontally or vertically, we increase the variability of the data and enable the model to learn to ignore irrelevant directional features.

[0062] Step S4: swin transformer processing

[0063] 1. PatchExtract: Use the tf.image.extract_patches function to split the input image into blocks, specify the block size (patch_size_x and patch_size_y) and the step size (consistent with the block size to ensure that the blocks do not overlap). Finally, use tf.reshape to convert the extracted image blocks into a shape suitable for model input, where batch_size is the batch size, patch_num*patch_num is the number of blocks extracted from each image, and patch_dim is the dimension of each block.

[0064] 2. PatchEmbedding: Use the Dense layer (self.proj) to linearly project the extracted image patches and map each patch to the specified embedding dimension (embed_dim). Use position embedding (self.pos_embed) to add position encoding to each patch so that the model can distinguish the positions of different image patches. Return the embedded patch with the corresponding position information.

[0065] 3. LN (layer normalization): used to improve the stability and convergence speed of the model. The final output formula of LN is:

[0066]

[0067] Where: x i is the input feature, μ is the feature mean, σ is the feature standard deviation, γ is the scaling parameter, β is the offset parameter, and ò is a small constant to prevent division by zero.

[0068] 4. W-MSA (window-based multi-head self-attention mechanism): Reduce the computational complexity of global self-attention by dividing local windows while maintaining efficient feature extraction.

[0069] 5. MLP (Multi-layer Perceptron): Further processes the features, increases the nonlinear capabilities of the model, and helps capture more complex features.

[0070] 6. PatchMergin: The process of downsampling image features. The input features are reshaped into the shape of the image, and then the image is divided into 2x2 small blocks. Each small block is spliced ​​together to form a new feature map, in which the number of channels at each position is quadrupled. Then, the number of channels is mapped from 4×C to 2×embed_dim through a linear transformation, thereby reducing the spatial dimension and expanding the dimension of the feature.

[0071] Step S5: Compress the feature map into a feature vector, and then generate the classification result through the fully connected layer and softmax activation function.

[0072] Figure 2This is a complete training flow chart for hyperspectral classification. The proposed model runs on a Linux system, using Python 3.9.18 and Pytorch 1.12.0 to implement hyperspectral classification. The hardware conditions are CPU: Intel Gold 6240x 2 (2.6Ghz, 18Cores), RAM: 768GB, HD: 64TB, SSD: 1920GB, Graphic: NVIDIA RTX6000 x 2. The learning rate of this experiment is set to 1e^-5, the weight is 0.0001, the ratio of the training set to the test set is 1:9, the epoch of the Pavia University dataset is 50 times, and the epoch of the Chikusei dataset is 100 times. The division of the dataset and the total number of samples in each category are shown in Tables 1 and 2. In order to compare the performance of various classification algorithms, three evaluation criteria commonly used in hyperspectral image classification tasks are used: overall classification accuracy (OA), average accuracy (AA) and kappa coefficient.

[0073] Table 1 Categories and training and test sample sizes of the Pavia University dataset

[0074] NO. class train test all 1 Asphalt 663 5968 6631 2 Meadows 1865 16784 18649 3 Gravel 210 1889 2099 4 Trees 306 2758 3064 5 Painted metal sheets 135 1210 1345 6 BareSoil 503 4526 5029 7 Bitumen 133 1197 1330 8 Self-BlockingBricks 368 3314 3682 9 Shadows 95 852 947

[0075] Table 2 Categories and training and testing sample sizes of the Chikusei dataset

[0076] NO. class train test all 1 Water 285 2560 2845 2 Baresoil(school) 286 2573 2859 3 Baresoil(park) 29 257 286 4 Baresoil(farmland) 485 4367 4852 5 Naturalplants 430 3867 4297 6 Weeds in farmland 111 997 1108 7 Forest 2052 18464 20516 8 Grass 652 5863 6515 9 Ricefeld (grown) 1337 12032 13369 10 Ricefield (first stage) 127 1141 1268 11 Rowcrops 596 5365 5961 12 Plastichouse 219 1974 2193 13 Manmade (non-dark) 122 1098 1220 14 Manmade(dark) 766 6898 7664 15 Manmade(blue) 43 388 431 16 Manmade(red) 22 200 222 17 Manmadegrass 104 936 1040 18 Asphalt 80 721 801 19 Pavedground 15 130 145

[0077] The hyperspectral classification method based on residual attention and swin transformer fusion proposed in the present invention is also compared with advanced deep learning models, including FullyContNet, GhostNet, HybridSN, M3DCNN, SSRN, UNet, and VIT. For a fair comparison, all initial parameter settings are kept consistent with those of the present invention.

[0078] Table 3 Results of different deep learning methods on the Pavia University dataset

[0079]

[0080] Table 4 Results of different deep learning methods on the Chikusei dataset

[0081]

[0082]

[0083] The experimental data of different deep learning methods are shown in Tables 3 and 4, where the black parts are the optimal results. First of all, from the perspective of overall accuracy (OA), the present invention has achieved a classification accuracy of more than 99.16% in multiple land object categories such as Asphalt, Meadows, and Trees, which is significantly better than the traditional FullyContNet, GhostNet, and HybridSN models. On more complex land objects such as Bitumen, Self-Blocking Bricks, and Shadows, the present invention also showed higher accuracy, especially in the Shadows category, achieving 100% accuracy. In terms of average category accuracy (AA), it also leads all comparison models with an accuracy of 99.94%, reflecting a stronger generalization ability between different categories. In addition, the Kappa coefficient reached 98.93%, further proving the robustness and superiority of the method. For the Pavia University dataset, experimental results show that the present invention has achieved a classification accuracy close to or equal to 100% in almost all categories, especially when dealing with complex objects such as Weeds in farmland and Rice field (first stage), which is significantly improved compared with other methods.

[0084] In order to evaluate the classification effect, Figure 3 , Figure 4 , Figure 5 and Figure 6 The classification results of the labels and the whole map of different deep learning methods for the Pavia University dataset and the Chikusei dataset are shown respectively. In the Pavia University dataset, this model accurately predicts each category, and there is no large-scale prediction category error. The predicted whole map is more accurate and clear than other classification methods. In the classification results of some complex terrain areas (such as buildings, roads, and vegetation) in the Chikusei dataset, other models (b)-(i) often have category aliasing and blurred boundaries. However, this model can better distinguish these fine-grained terrain categories, with clear boundaries, and the classification results are more interpretable.

[0085] In order to evaluate the impact of channel attention (CA) and spatial attention (SA) on model performance in detail, four ablation experiments were designed. The experimental results are shown in Table 5, and the black part is the best result. When only CA is used, the model achieves significant improvement in classification tasks by enhancing the feature weights of different channels, especially in overall accuracy (OA) and Kappa coefficient; when only SA is used, the model focuses on the extraction of spatial features. Although it has a good effect in extracting spatial information, its overall performance is slightly lower than CA; in the baseline model that does not use CA and SA at all, the performance of the model is significantly reduced, which verifies the importance of the attention mechanism to the model; when CA and SA are used in combination, the model makes full use of channel and spatial information to obtain the best classification effect, showing the complementary advantages of the two. The overall experimental results show that the attention mechanism is crucial to improving the accuracy of the model, especially when CA and SA are combined, the feature expression ability of the model can be significantly enhanced.

[0086] Table 5 Ablation experiment

[0087]

[0088] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A hyperspectral classification method based on residual attention and swin transformer fusion, characterized in that: It includes the following steps: Step 1: Obtain the hyperspectral remote sensing image with labeled categories and preprocess it into multiple training image patches. The preprocessing includes dimensionality reduction in the spectral dimension, edge padding, and image chunking; Step 2: Use the residual attention module to process the spectral and spatial features of the input training image patches. The residual attention module includes a convolutional layer, a residual connection, a spectral attention module, and a spatial attention module; Step 3: Generate patches through data augmentation techniques for the hyperspectral data processed in Step 2. The data augmentation techniques include random cropping and random flipping; Step 4: Use the swin transformer module to perform multi-scale feature extraction on the patches in Step 3; Step 5: Use the global average pooling layer to extract global features from the data processed by the swin transformer module in Step 4; Step 6: After the global feature extraction in Step 5, perform image classification through the Dense layer.

2. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 1 is characterized in that: The dimensionality reduction in the spectral dimension in Step 1 is to perform dimensionality reduction on the hyperspectral image through principal component analysis (PCA), reducing the dimension of the spectral channels from the original K bands to L bands (L < K), generating the dimensionality-reduced hyperspectral data.

3. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 2 is characterized in that: The edge padding in Step 1 is to perform edge padding on the dimensionality-reduced data, with a padding width of P.

4. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 3 is characterized by: The image chunking in Step 1 is to divide the data into N three-dimensional cube blocks, where N = H × W, where M and N are the number of rows and columns of the image, representing the number of spatial pixels of the hyperspectral image, and use them as independent training samples for subsequent model training.

5. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 1, characterized in that: In Step 4, the swin transformer module realizes multi-scale perception of spectral and spatial features through the multi-head self-attention mechanism. The formula for the multi-head self-attention is: where, and respectively represent the query matrix, the key matrix, and the value matrix, and is the scaling factor.

6. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 1, characterized in that: Based on the residual attention and swin transformer models, perform multiple rounds of training. The process is as follows: Step S1: Prepare the hyperspectral dataset for training; Step S2: Use loadData to load the hyperspectral dataset; Step S3: Preprocess the hyperspectral dataset in Step S2 and input it into the model; Step S4: Input the label and the prediction result of the model into the loss function to calculate the loss; Step S5: Backpropagate according to the calculated loss value to update the weights of the model; Step S6: Loop and iterate Steps S3 to S5 for epoch times to obtain the finally trained model.

7. The hyperspectral classification method based on residual attention and swin transformer fusion according to claim 6 is characterized by: The hyperspectral dataset in Step S1 includes the Pavia University dataset and the Chikusei dataset.

Images