A hyperspectral image classification method combining multi-attention and Transformer

Abstract

The present invention discloses a network that combines multi-attention and Transformer Encoder structures to fully extract the spatial spectral features of HSI. The multi-attention module includes channel attention and spatial attention to further focus on the information of important frequency bands and regions. In addition, the present invention also proposes a multi-layer dense adaptive fusion module to fully integrate shallow, mid-level and deep information. The feature representation and the learned mixed information contain multi-level features. In addition, we propose an Lpoly loss adjusted based on the label smoothed cross entropy to dynamically change the probability of the true predicted label of the ground object category. The network was tested on three public hyperspectral classification datasets and showed good classification performance. The results show that this method is very effective in extracting deep semantic information in hyperspectral images.

Description

Technical Field

[0001] The present invention belongs to the field of hyperspectral image classification, and specifically relates to a hyperspectral image classification method combining multi-attention and Transformer. Background Art

[0002] Imaging and spectroscopy are two key areas of traditional optical technology. Imaging can capture images of objects and obtain spatial information, while spectroscopy can obtain optical information about objects, allowing the study of their material properties. Before the 1970s, imaging and spectroscopy were independent disciplines. With the development of remote sensing, imaging spectroscopy has rapidly gained momentum. It is a rapid, nondestructive detection technique featuring high spectral resolution, multi-band, and integrated image processing. It can identify the Earth's surface over large scales and conduct in-depth research on the composition and structure of surface materials. Currently, imaging spectroscopy has become a key development trend in remote sensing technology, finding increasing application in military reconnaissance, marine remote sensing, geological exploration, vegetation analysis, and other fields.

[0003] As imaging spectroscopy technology becomes increasingly mature, people urgently need to utilize this rich spatial spectral information to meet various needs. In order to further distinguish the subtle spatial spectral differences between similar categories in the same scene, researchers have applied deep learning methods to hyperspectral image classification tasks and achieved good results. Convolutional neural networks, as a representative deep learning method, mainly include: using one-dimensional convolution and two-dimensional convolution to extract spectral information and spatial information respectively, and fusing the extracted information for classification; using three-dimensional convolution to directly extract local spatial spectral information for classification. In recent years, deep learning methods represented by the Transformer architecture have received increasing attention. Using this network framework with a self-attention mechanism to achieve hyperspectral classification has also become a good choice.

[0004] Although the above deep learning methods have largely met people's needs for using hyperspectral images for ground object classification, they still have two shortcomings:

[0005] 1) Although traditional 3D convolutional neural networks generally perform better than 1D and 2D convolutional neural networks, they also incur huge computational overhead;

[0006] 2) Hyperspectral image classification tasks can usually be regarded as small sample classification tasks. Whether it is a traditional convolutional neural network or a Transformer network architecture, there is a risk of overfitting in the process of feature learning and extraction. Summary of the Invention

[0007] The purpose of this invention is to address these shortcomings and propose a hyperspectral image classification method that combines multi-attention and Transformer. By using multiple attention mechanisms, more attention resources are focused on more important areas, improving computing resources while ensuring that network training is as fast as possible. At the same time, the Transformer Encoder combination structure is used to extract rich semantic information from hyperspectral images. For the encoder combination framework, a multi-layer dense adaptive fusion connection method is designed to alleviate the potential overfitting problem. In addition, an improved loss function based on the label smoothed cross entropy loss function is designed to improve classification performance by further adjusting the probabilities between predicted labels.

[0008] The present invention addresses this problem through a technical solution: a hyperspectral image classification network that combines multi-attention and a Transformer. The present invention first designs a multi-attention module that combines channel attention, a two-dimensional convolutional layer, and spatial attention. This module first uses channel attention to select channels. Different channels may have different weights, with channels that have a greater impact on classification accuracy receiving higher weights, and vice versa. Spatial attention further focuses attention on important regions within the two-dimensional space and deemphasizes regions that are irrelevant to the classification. A Gaussian-weighted feature representor is then used to transform the features into deep semantic features and generate semantic tokens, enabling the subsequent Transformer-Encoder architecture to fully learn and extract these features. Regarding the encoder architecture, a multi-layer dense adaptive fusion connection is employed, with the number of encoder layers set to three. Finally, the fused multi-level features are passed through a linear representation layer and classified using a softmax function. Regarding the loss function, label-smoothed cross-entropy, compared to cross-entropy, also considers the probability of incorrect classifications, as cross-entropy loss can easily overfit to classes with fewer samples in hyperspectral images. Therefore, we chose to add a polynomial based on the label smoothed cross entropy to dynamically adjust the probability of the true prediction of the ground object category to further prevent overfitting, making the network more tolerant and robust to the categories with fewer samples in hyperspectral images. The specific steps include the following:

[0009] Step 1: Divide the training data set and the test data set;

[0010] Step 2: reduce the dimensionality of the original hyperspectral data in the training dataset;

[0011] Step 3: Divide the reduced-dimensional data into several data cube blocks and perform a three-dimensional convolution layer to obtain a three-dimensional feature map;

[0012] Step 4: Execute a multi-attention module consisting of channel attention, 2D convolutional layer, and spatial attention to obtain a 2D feature map.

[0013] Step 5: Flatten each two-dimensional feature map into a one-dimensional feature vector;

[0014] Step 6: Generate semantic tags through the Gaussian weighted feature representor module;

[0015] Step 7: Connect an all-zero vector as a learnable classification tag to the semantic tag generated in step 6 to form a fused semantic tag, and embed position information into the fused semantic tag;

[0016] Step 8: executing a multi-layer dense adaptive fusion module to extract features from the fused semantic tags embedded with the position information;

[0017] Step 9: Input the first classification label into the last linear layer;

[0018] Step 10, use the softmax function to identify the label;

[0019] Step 11: Design a loss function to train the entire network consisting of steps 3 to 10, and then test it on the test dataset.

[0020] Furthermore, in step 2, a dimensionality reduction operation is performed on the hyperspectral image using principal component analysis, and the number of frequency bands after dimensionality reduction is set to 40.

[0021] Furthermore, in step 3, the size of each extracted cube block is set to 13×13×40, and the three-dimensional convolution layer uses 16 convolution kernels of size 3×3×3 to generate 16 feature maps of size 11×11×38.

[0022] Furthermore, the channel attention processing in step 4 is as follows;

[0023] For the input features, aggregate the average pooling layer and the maximum pooling layer, and then the two different spatial context descriptors F avg and F max Input into the same shared multi-layer perceptron MLP, where the hidden activation size of MLP is set to R, and then the final output feature is obtained by element-by-element summation. The process is expressed mathematically as follows:

[0024]

[0025] Where Sig is the sigmoid function, W0∈R C / r×C , the MLP weights W0 and W1 are shared, and the ReLU activation function is followed by W0;

[0026] Furthermore, in step 4, 64 3×3 convolution kernels are used in the two-dimensional convolution layer to obtain 64 9×9 feature maps.

[0027] Furthermore, for the spatial attention module, the input feature F is firstly subjected to average pooling and maximum pooling operations to obtain the features and Concatenate the resulting features and finally use a convolutional layer to generate a spatial attention feature map;

[0028]

[0029] Where Sig is the sigmoid function, f 3×3 Represents a convolution operation with a convolution kernel size of 3×3.

[0030] Furthermore, the process of generating semantic tags in step 6 is as follows;

[0031] The flattened feature map of its input is defined as X∈R hw×c , where h is the height, w is the width, and c is the number of channels. The input feature X is dot-producted with the weight W of the initialized Gaussian distribution. Then, it is transposed and the softmax function is used to focus on the relatively important semantic parts to obtain A. Finally, A is multiplied by X to generate a semantic tag, that is, T in formula (3). The size of the obtained semantic group is T∈R t×c , where t represents the number of tags; A is represented by softmax(X*W) T The formula is as follows:

[0032] T=softmax(X*W) T X (3)

[0033] Among them, X is the input feature, W represents the weight matrix of the initialized Gaussian distribution, and the size is W∈R c×t , * indicates a 1×1 dot product operation.

[0034] Furthermore, the multi-layer dense adaptive fusion module in step 8 contains a 3-layer TransformerEncoder structure. In addition, the cross-layer connection method adopted is: first, the features obtained in the first layer are adaptively fused with the third layer, and then the fused information is adaptively fused with the features in the second layer.

[0035] Furthermore, the loss function used in step 11 is to adjust the first polynomial based on the label smoothed cross entropy, and the improved loss is named Lpoly loss. Let P t is the model’s predicted probability for the target ground truth category. The detailed reasoning steps are as follows:

[0036] First, (1-P t ) i The Taylor expansion of the cross-entropy (CE) loss is:

[0037]

[0038] Optimizing cross entropy loss using gradient descent requires using P t Gradient:

[0039]

[0040] Based on the label smooth cross entropy, the first polynomial coefficient of the cross entropy is adjusted so that the probability of the true predicted label of the object category can be dynamically adjusted according to the differences in the task and the dataset itself. The formula is as follows:

[0041]

[0042] where CE(i) represents the standard cross entropy loss for i, λ is a small positive number, ε is a number greater than -1, i is the correct class, N is the number of classes, and P t is the model’s predicted probability of the target ground truth class.

[0043] Furthermore, when training the entire network, the Adam algorithm is used for optimization, and the learning rate is set to 10 -3 , the batch size is set to 64, and the training is repeated for 300 rounds. The network weight parameter W and bias parameter B are obtained by training the model.

[0044] The beneficial effects of the present invention are as follows: compared with the existing network structure for hyperspectral image classification tasks, the network framework proposed in the present invention combines multiple attention mechanisms, including channel and spatial attention, as well as the multi-head self-attention mechanism in the encoder architecture; in order to integrate and alleviate the huge differences in the degree of feature extraction of encoders at different layers, we adopt a cross-layer connection structure, combining long connections and short connections, to fuse multi-level information for subsequent classification; in addition, we also design a new loss function Lpoly loss to further prevent the network from overfitting to categories with a small number of samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a diagram of the network framework structure of the present invention.

[0046] Figure 2 Schematic diagram of the channel attention mechanism in this invention.

[0047] Figure 3 Schematic diagram of the spatial attention mechanism in this invention.

[0048] Figure 4 This is a diagram of the Gaussian weighted feature representation mechanism in the present invention.

[0049] Figure 5 This is a connection structure diagram of multi-layer dense adaptive fusion in the present invention.

[0050] Figure 6 This is a diagram of the adaptive fusion structure of multi-layer dense adaptive fusion in the present invention.

[0051] Figure 7 Figures comparing the number of semantic tokens in the Gaussian-weighted feature representation used in this paper. Figures (a), (b), and (c) show the experimental results on the Indian Pines, Pavia University, and Houston 2013 datasets, respectively, with varying numbers of semantic tokens. We tested the results on the three datasets with different numbers of tokens, 50, 100, 150, 200, and 250, respectively. The results show that setting the number of semantic tokens to 200 yields better classification results for all three datasets.

[0052] Figure 8 This paper shows the impact of the two adjustable parameters λ and ε on the overall classification accuracy when applying the proposed Lpoly loss to different datasets. Figures (a), (b), and (c) show the experimental results on the Indian Pines, Pavia University, and Houston 2013 datasets, respectively, under different loss function parameters. The results show that we set λ to 0.1 for all three datasets, ε to 1 for the Indian Pines dataset, and ε to 0.5 for the other two datasets. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings.

[0054] like Figure 1 As shown in FIG, the present invention provides a hyperspectral image classification method combining multi-attention and Transformer, and the specific steps are as follows:

[0055] Step 1: Divide the training data set and the test data set;

[0056] Step 2: Dimensionality reduction is performed on the input raw hyperspectral data using principal component analysis.

[0057] Step 3: Divide the reduced-dimensional data into several data cube blocks and perform a three-dimensional convolution layer to obtain a three-dimensional feature map;

[0058] Step 4: Execute the CSA module consisting of channel attention (CA), 2D convolutional layer and spatial attention (SA) to obtain a 2D feature map;

[0059] Step 5: Flatten each two-dimensional feature map into a one-dimensional feature vector (Flatten Feature);

[0060] Step 6: Generate semantic tokens (i.e., tokens) through the Gaussian weighted feature representation (Tokenizer) module;

[0061] In step 7, an all-zero vector is used as a learnable classification token (T0) and concatenated with the semantic tag generated in step 6 to form a fused semantic token. Position information (PE) is then embedded in the semantic token.

[0062] Step 8: Execute the Encoder Multi-layer Dense Adaptive Fusion (MDAF) module to extract features from the fused semantic tokens (i.e., the semantic tokens embedded with location information);

[0063] Step 9: Input the first classification token into the last linear layer;

[0064] Step 10, use the softmax function to identify the label;

[0065] In step 11, the loss function used during network training is a modified Lpoly loss based on label smoothed cross entropy with an adjusted first polynomial. Furthermore, the Adam algorithm is used for optimization when training the overall network model. This method is tested on three common hyperspectral classification datasets. The Pavia University dataset uses 5% of the total number of test samples, while the other two datasets use 10% of the total number of test samples.

[0066] Furthermore, in step 2, for the dimensionality reduction operation performed on the hyperspectral image using principal component analysis, we set the number of frequency bands after dimensionality reduction to 40.

[0067] Furthermore, in step 3, we set the size of each extracted cube block to 13×13×40, and in the 3D convolution layer we used 16 convolution kernels of size 3×3×3 to generate 16 feature maps of size 11×11×38.

[0068] Furthermore, for the design of channel attention in step 4, the specific process we adopted is as follows Figure 2As shown in Figure 2, for the input feature F, we aggregate the average pooling layer and the maximum pooling layer. Then we get two different spatial context descriptors and The input is fed into the same shared multilayer perceptron (MLP), where the hidden activation size of the MLP is set to R C / r×1×1 , r is the reduction ratio, C is the number of channels, and then the final output features are obtained by element-by-element summation. The process is expressed mathematically as:

[0069]

[0070] Where Sig is the sigmoid function, W0 and W1 are the weights of MLP, W0∈R C / r×C , W1∈R C×C / r Note that W0 and W1 are shared, and the ReLU activation function is followed by W0.

[0071] In the two-dimensional convolution layer, 64 3×3 convolution kernels are used to obtain 64 9×9 feature maps. For the spatial attention module, the input feature F is firstly average pooled and max pooled to obtain the features. and The resulting features are connected and finally a convolutional layer is used to generate a spatial attention feature map. The specific process is shown in the attached figure. Figure 3 shown.

[0072]

[0073] Where Sig is the sigmoid function, F is the input feature, f 3×3 Represents a convolution operation with a convolution kernel size of 3×3.

[0074] Furthermore, the semantic token generation process in step 6 is as follows:

[0075] The flattened feature map of its input is defined as X∈R hw×c , where h is the height, w is the width, and c is the number of channels. The input feature X is dot-producted with the weight W of the initialized Gaussian distribution, then transposed and the softmax function is used to focus on the relatively important semantic parts to obtain A. Finally, A is multiplied by X to generate semantic tokens, that is, T in formula (3). The size of the obtained semantic group is T∈R t×c , where t represents the number of tokens. A uses softmax(X*W) T The formula is as follows:

[0076] T=softmax(X*W) T X (3)

[0077] Among them, X is the input feature, W represents the weight matrix of the initialized Gaussian distribution, and the size is W∈R c×t , * indicates a 1×1 dot product operation.

[0078] Finally, we set the number of generated semantic tokens t to 200;

[0079] Furthermore, the multi-layer dense adaptive fusion module in step 8 contains a 3-layer TransformerEncoder structure. In addition, the cross-layer connection method we adopt is: first, the features obtained from the first layer are adaptively fused with the third layer, and then the fused information is adaptively fused with the second layer. The specific connection structure is shown in the attached figure. Figure 1 As shown in the figure; the adaptive fusion (AF) method used in this process is shown in the figure. Figure 5 As shown;

[0080] Furthermore, the loss function used in this network is to adjust the first polynomial based on the label smoothed cross entropy. We name the improved loss Lpoly loss. Let P t is the model’s predicted probability for the target ground truth category. The detailed reasoning steps are as follows:

[0081] First, (1-P t ) i The Taylor expansion of the cross-entropy (CE) loss is:

[0082]

[0083] Optimizing cross entropy loss using gradient descent requires using P t Gradient:

[0084]

[0085] Based on Label Smoothing Cross-Entropy (LSCE), we adjust the first polynomial coefficient of the cross-entropy. This allows us to dynamically adjust the probability of the true predicted label of the object category based on the differences in tasks and datasets. The formula is as follows:

[0086]

[0087] where CE(i) represents the standard cross entropy loss for i, λ is a small positive number, ε is a number greater than -1, i is the correct class, N is the number of classes, and P t is the model’s predicted probability of the target ground truth class.

[0088] Furthermore, when training the overall network model, the Adam algorithm is used for optimization, and the learning rate is set to 10 -3 , the batch size is set to 64, and the training is repeated for 300 rounds. The network weight parameter W and bias parameter B are obtained by training the model.

[0089] This example uses an Intel Xeon(R) Silver 4210R CPU, 64GB RAM, and an NVIDIA Quadro RTX 5000 16GB GPU, a Linux operating system, and is developed using Python and the open source library PyTorch.

[0090] Next, we take the Indian Pines dataset as an example to illustrate the hyperspectral classification network framework we designed.

[0091] Use the Indian Pines dataset and select 10% of the dataset as the training dataset I train , the remaining 90% is used as the test data set I test ;

[0092] After performing PCA dimensionality reduction, each patch is extracted to a size of 13 × 13 × 40. In the first 3D convolutional layer, 16 feature cubes of 13 × 13 × 38 are generated using 16 3 × 3 × 3 convolution kernels. This step utilizes 3D convolution because each patch contains rich spectral information. These 16 feature cubes are rearranged to generate a single feature cube of 11 × 11 × 608.

[0093] The cube then performs a channel attention module and a two-dimensional convolutional layer using 64 3×3 layers to obtain 64 feature maps, each of size 9×9. Each feature map is then flattened into a one-dimensional feature vector, resulting in 64 vectors of size 1×81. At this point, the features obtained are equivalent to X∈R in this paper. 81×64 .

[0094] Next, use Xavier standard normal distribution to get the initial weight matrix W a ∈R 64×200 , to guide the feature distribution to be more regular. The weight matrix W initialized a ∈R 64×200 Multiply by the feature vector group to get the semantic group A∈R 81×200 .

[0095] Then, multiply the transpose of A by X to get the final semantic Tokens (ie, T, and T∈R 200×64). Add an all-zero vector to T as a learnable classification token, and then embed the position information to obtain T in ∈R 201×64 , and finally input to the MDAF layer.

[0096] Process T through the MDAF module to represent the semantic features. This module has the same input and output size. Take the first classification token (i.e. ) as the classification vector. This vector is input into the softmax linear classifier to obtain the judgment label.

[0097] Specifically, the Adam algorithm is used for optimization in this embodiment, and the learning rate is set to 10 -3 , the batch size is set to 64, the values ​​of λ and ε in the loss function are set to 0.1 and 1 respectively, and 300 rounds of training are performed. The network weight parameter W and bias parameter B are obtained by training the model. At this point, the implementation process of this case is completed.

[0098] Calculate retrieval accuracy and test:

[0099] The trained network is used to calculate the samples in the test dataset and three evaluation indicators are calculated: overall accuracy (OA), average accuracy (AA), and Kappa coefficient (κ). The larger the values ​​of these three indicators, the better the classification results.

[0100] To evaluate the effectiveness of our method, we conducted comparative experiments on three public hyperspectral classification datasets (Indian Pines, Pavia University, and Houston 2013). We selected several representative methods: Support Vector Machine (SVM), EMAP, 1D-CNN, 2D-CNN, 3D-CNN, SSRN, Cubic-CNN, HybridSN, SSFTT, and our proposed method. We also maintained consistency in the division of training and test sets across the different methods to ensure fairness in the comparative experiments.

[0101] Table 1

[0102]

[0103]

[0104] Table 1 shows the comparative experimental results of the present invention and other methods on the Indian Pines dataset.

[0105] Table 2

[0106]

[0107] Table 2 shows the comparative experimental results of the present invention and other methods on the Pavia University dataset.

[0108] Table 3

[0109]

[0110] Table 3 shows the comparative experimental results of the present invention and other methods on the Houston 2013 dataset.

[0111] Experiments show that our proposed hyperspectral image classification method achieves the best performance on three common hyperspectral classification datasets. In addition, the overall classification accuracy (AA) metric in the experimental results shows that our proposed method is robust across different categories in different datasets.

[0112] The embodiments of the present invention are merely examples for illustrating the present invention clearly and are not limited to the above examples. Those skilled in the art may make various changes or equivalent substitutions based on this method, and any changes similar to the present invention shall fall within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image classification method combining multi-attention and Transformer, characterized by: The steps include: Step 1: Divide the training data set and the test data set; Step 2: reduce the dimensionality of the original hyperspectral data in the training dataset; Step 3: Divide the reduced-dimensional data into several data cube blocks and perform a three-dimensional convolution layer to obtain a three-dimensional feature map; Step 4: Execute a multi-attention module consisting of channel attention, 2D convolutional layer, and spatial attention to obtain a 2D feature map. Step 5: Flatten each two-dimensional feature map into a one-dimensional feature vector; Step 6: Generate semantic tags through the Gaussian weighted feature representor module; Step 7: Concatenate an all-zero vector as a learnable classification marker with the semantic marker generated in step 6 to form a fused semantic marker, and embed position information into the fused semantic marker. Step 8: executing a multi-layer dense adaptive fusion module to extract features from the fused semantic tags embedded with the position information; The multi-layer dense adaptive fusion module in step 8 contains a 3-layer Transformer Encoder structure. In addition, the cross-layer connection method adopted is: the features obtained in the first layer are first adaptively fused with the third layer, and then the fused information is adaptively fused with the features in the second layer. Step 9: Input the first classification label into the last linear layer; Step 10, use the softmax function to identify the label; Step 11: Design a loss function to train the entire network consisting of steps 3 to 10, and then test it on the test dataset; The loss function used in step 11 is to adjust the first polynomial based on the label smoothed cross entropy. The improved loss is named Lpoly loss. Let P t is the model’s predicted probability for the target ground truth category. The detailed reasoning steps are as follows: First, (1-P t ) i The Taylor expansion of the cross-entropy (CE) loss is: Optimizing cross entropy loss using gradient descent requires using P t Gradient: Based on the label smooth cross entropy, the first polynomial coefficient of the cross entropy is adjusted so that the probability of the true predicted label of the object category can be dynamically adjusted according to the differences in the task and the dataset itself. The formula is as follows: where CE(i) represents the standard cross entropy loss for i, λ is a small positive number, ε is a number greater than -1, i is the correct class, N is the number of classes, and P t is the model’s predicted probability of the target ground truth class.

2. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: In step 2, the dimensionality reduction operation is performed on the hyperspectral image using principal component analysis, and the number of frequency bands after dimensionality reduction is set to 40.

3. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: In step 3, the size of each extracted cube block is set to 13×13×40, and the 3D convolution layer uses 16 convolution kernels of size 3×3×3 to generate 16 feature maps of size 11×11×38.

4. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: The processing of channel attention in step 4 is as follows; For the input features, aggregate the average pooling layer and the maximum pooling layer, and then the two different spatial context descriptors F avg and F max Input into the same shared multi-layer perceptron MLP, where the hidden activation size of MLP is set to R, and then the final output feature is obtained by element-by-element summation. The process is expressed mathematically as follows: Where Sig is the sigmoid function, W0∈R C / r×C , the MLP weights W0 and W1 are shared, and the ReLU activation function is followed by W0.

5. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: In step 4, 64 3×3 convolution kernels are used in the two-dimensional convolution layer to obtain 64 9×9 feature maps.

6. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: For the spatial attention module, firstly, the input feature F is obtained by average pooling and maximum pooling operations respectively. and The resulting features are concatenated and finally a convolutional layer is used to generate a spatial attention feature map; Where Sig is the sigmoid function, f 3×3 Represents a convolution operation with a convolution kernel size of 3×3.

7. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: The process of generating semantic tags in step 6 is as follows; The flattened feature map of its input is defined as X∈R hw×c , where h is the height, w is the width, and c is the number of channels. The input feature X is dot-producted with the weight W of the initialized Gaussian distribution. Then, it is transposed and the softmax function is used to focus on the relatively important semantic parts to obtain A. Finally, A is multiplied by X to generate a semantic tag, that is, T in formula (3). The size of the obtained semantic group is T∈R t×c , where t represents the number of tags; A is represented by softmax(X*W) T The formula is as follows: T=softmax(X*W) T X (3) Among them, X is the input feature, W represents the weight matrix of the initialized Gaussian distribution, and the size is W∈R c×t , * indicates a 1×1 dot product operation.

8. The hyperspectral image classification method combining multi-attention and Transformer as claimed in claim 1, characterized in that: When training the entire network, the Adam algorithm is used for optimization, and the learning rate is set to 10 -3 , the batch size is set to 64, and the training is repeated for 300 rounds. The network weight parameter W and bias parameter B are obtained by training the model.

Images