A Hyperspectral Image Classification Method and System Based on the Center-Focus Transformer Model

Abstract

This invention discloses a hyperspectral image classification method and system based on a center-focused Transformer model. The method includes: selecting and cropping hyperspectral scene images; constructing a hyperspectral cube from the cropped hyperspectral scene images and randomly selecting a portion of the cropped hyperspectral scene images as training and validation sets; constructing a Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer; the preprocessing module is used to extract features from the input image and reduce the number of channels in the input image; each global self-information calculation layer includes a center pixel self-focused module, a cross-channel global fusion module, and a feedforward network layer; training and validating the Transformer model using cross-entropy classification loss on the training set; and classifying the hyperspectral image to be classified based on the validated Transformer model. The proposed method reduces computational complexity and cost, and improves the classification accuracy of hyperspectral remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image classification technology, and in particular to a hyperspectral image classification method and system based on the central attention Transformer model. Background Technology

[0002] Hyperspectral remote sensing achieves the integration of image and spectral data, containing rich spectral and spatial information, and has wide applications in precision agriculture, military reconnaissance, geological exploration, and other fields. Hyperspectral image classification is a hot topic in hyperspectral image research, and with the in-depth exploration and application of deep learning, the Transformer model has achieved widespread use in hyperspectral image classification technology. However, the computational cost of the self-attention mechanism is a significant factor limiting the development of this model in hyperspectral image classification. Existing network models and methods have not fully recognized this issue, preventing further improvements in classification performance. Summary of the Invention

[0003] This invention provides a hyperspectral image classification method and system based on the center-of-interest Transformer model to overcome the above-mentioned technical problems.

[0004] A hyperspectral image classification method based on the center-focus Transformer model includes,

[0005] S1. Select a hyperspectral scene image, crop the hyperspectral scene image with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene image, and randomly select a portion of the cropped hyperspectral scene image from the hyperspectral cube as the training set and validation set.

[0006] S2. Construct a Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input sample, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers include Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. The four global self-information calculation layers have the same structure, but the number of channels decreases layer by layer. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module is used to extract the spatial self-similarity features and global spatial information metric space of the input sample and feed them back to the feedforward network layer. The cross-channel global fusion module is used to calculate the uniform global channel information of the input sample and feed it back to the feedforward network layer. The feedforward network layer is used to extract features based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information.

[0007] S3. Train the Transformer model using cross-entropy classification loss based on the training set, and validate the trained Transformer model using the validation set. Then, classify the hyperspectral image to be classified based on the validated Transformer model.

[0008] Preferably, the processing procedure of the center pixel self-attention module is to obtain the feature map x of the input sample. f x f ∈R h ×w×b h, w, and b represent the height, width, and number of channels of the hyperspectral cube, respectively. The three-dimensional matrices k and v are obtained according to formulas (1) and (2).

[0009] v = ConvTrans2d(x f (1)

[0010] k = ConvTrans2d(x f (2)

[0011] Extract the spectral vector at the center coordinates (h / 2, w / 2) of the three-dimensional matrix k as the measurement reference vector q∈R. 1 ×1×b Express the three-dimensional matrix k in two-dimensional form, where k (i,j) ∈R 1×1×b Let (i,j,u) represent the elements at spatial coordinates (i,j)∈(h,w), where each element represents a spectral vector of dimension b. Let (i,j,u)∈(h,w,b) be the three-dimensional spatial coordinates, and k be the vectors in the vectors. (i,j,u) Let k be a certain element inside the spectral vector. According to formula (3), all elements in matrix k are normalized.

[0012]

[0013] According to formula (4), the similarity matrix d is obtained by performing a vector dot product operation between the measurement reference vector q and the adjacent spectral vectors in the normalized k. Then, the similarity matrix d is activated according to formula (5). Finally, the output x is obtained by performing an element-wise product operation between matrix v and matrix d according to formula (6). out And x out ∈R h×w×b ,

[0014] d (i,j) =k i,j ×q (4)

[0015]

[0016]

[0017] According to formula (7), the feature map x f After processing, local spatial information is obtained, and x is calculated according to formula (8). out with res(x) f The pixels are then fused to obtain the final output of the center pixel self-focus module.

[0018] res(x f ) = BN(ConvTrans2d(x f (7)

[0019]

[0020] Here, res(·) represents the residual mapping unit, which includes a 3x3 transposed convolution and BatchNorm. The Drop(·) operation is used to randomly discard some redundant information to obtain the final output.

[0021] Preferably, the processing procedure of the cross-channel global fusion module is to process the original features x of the input sample according to formula (9). c The feature matrix x is obtained by performing global average pooling and normalization operations. qkv The feature matrix x is obtained according to formula (10). qkv Local channel information v c ,

[0022] x qkv =LN(GAP(x) c (9)

[0023] v c =ConvTrans1d(x qkv (10)

[0024] Where, x c As the original feature, x c ∈R b×h×w×c b represents the size of the batch data, and c, h, and w represent the number of channels, height, and width of the feature, respectively. qkv ∈R b×c ,

[0025] The normalized vector k is calculated according to formula (11). vec The VectorNorm operation is a vector normalization operation. The calculation process of this operation is given by formula (12). Let (i,j)∈(b,c) be the coordinates in two-dimensional space, where vector k vec Each element on It is obtained by taking the square root of the sum of the squares of the elements in each row.

[0026] k vec =VectorNorm(k) (11)

[0027]

[0028] The similarity matrix r is calculated according to formula (13), where r∈R b×b q c With k c For x qkv Copying a matrix using a normalized vector k vec Based on row-corresponding element division, the similarity matrix r is processed, that is, assuming (u,v)∈(b,b) are two-dimensional spatial coordinates, then any element r in r (u,v) The calculation is performed using formula (14).

[0029] r = q c ×k c (13)

[0030]

[0031] r * =softmax(r) =softmax(r)v c (15)

[0032] Using the similarity matrix r from formula (15), the final channel self-attention matrix r can be obtained by performing softmax activation and matrix multiplication with the v matrix. * , and r * ∈R b×c The channel self-attention matrix represents the globally relevant information across different channels for different samples, i.e., the channel context attention information.

[0033] x out =x c +x c ⊙r * (16)

[0034] r * and x c Perform element-wise product and then residual operation to obtain the final output x. out .

[0035] Preferably, the feedforward network layer comprises three layers, each consisting of a two-dimensional LayerNorm, Gelu, and a ConvTrans2d operation. A group convolution ConvTrans2d operation is added to the second layer. * This is used to increase the diversity of self-similar features in the learning space.

[0036] Feed(·)=LayerNorm(Gelu(ConvTrans2d(·))) (17)

[0037] Assume the input feature is x r x r ∈R h×w×c Let h, w, and c be the length, width, and number of channels of the input features. The feature extraction behavior of each network layer is denoted as Feed(·), where t∈{1,2,3}. The feature extraction formula is as follows:

[0038] x t =Feed(x t-1 (18)

[0039] x out =Drop(x) 3 +x r (19)

[0040] The final output x is obtained after performing the Drop operation according to formula (19). out .

[0041] A hyperspectral image classification system based on the classification method of claim 1 includes an image acquisition and processing module, a Transformer model construction module, and a model training and application module.

[0042] The image acquisition and processing module is used to select hyperspectral scene images, crop the hyperspectral scene images with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene images, and randomly select a portion of the cropped hyperspectral scene images from the hyperspectral cube as training and validation sets.

[0043] The Transformer model building module is used to construct the Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers are Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. These four layers have the same structure but the number of channels decreases progressively. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module extracts the spatial self-similarity features and global spatial information metric space of the input sample and feeds them back to the feedforward network layer. The cross-channel global fusion module calculates the uniform global channel information of the input sample and feeds it back to the feedforward network layer. The feedforward network layer performs feature extraction based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information.

[0044] The model training and application module is used to train the Transformer model using cross-entropy classification loss based on the training set, and to validate the trained Transformer model using the validation set. The validated Transformer model is then used to classify the hyperspectral image to be classified.

[0045] This invention provides a hyperspectral image classification method and system based on a center-focused Transformer model. By acquiring global spatial correlation information of the image through a center pixel self-focused module, the computational complexity and cost are reduced. Homogeneous spectral information is extracted based on a cross-channel global fusion module and fused with spatial information. This information is then fed into a feedforward network layer for sufficient training and learning, thereby improving classification accuracy. This method has significant application value in the fine classification of land surfaces in hyperspectral images. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the method of the present invention;

[0048] Figure 2 This is the network structure diagram of the Transformer model of this invention;

[0049] Figure 3 This is a diagram of the calculation model for the center pixel self-attention mechanism of the present invention;

[0050] Figure 4 This is a diagram of the cross-channel global information fusion module of the present invention;

[0051] Figure 5 This is a model diagram of the feedforward network layer of the present invention;

[0052] Figure 6 These are test images from the test dataset in this embodiment of the invention;

[0053] Figure 7 This is a classification result diagram of the test dataset in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Figure 1 This is a flowchart of the method of the present invention, as shown below. Figure 1 As shown, the method in this embodiment may include:

[0056] S1. Select a hyperspectral scene image, crop the hyperspectral scene image with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene image, and randomly select a portion of the cropped hyperspectral scene image from the hyperspectral cube as the training set and validation set.

[0057] S2. Construct a Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input sample, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers include Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. The four global self-information calculation layers have the same structure, but the number of channels decreases layer by layer. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module is used to extract the spatial self-similarity features and global spatial information metric space of the input sample and feed them back to the feedforward network layer. The cross-channel global fusion module is used to calculate the uniform global channel information of the input sample and feed it back to the feedforward network layer. The feedforward network layer is used to extract features based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information.

[0058] S3. Train the Transformer model using cross-entropy classification loss based on the training set, and validate the trained Transformer model using the validation set. Then, classify the hyperspectral image to be classified based on the validated Transformer model.

[0059] Based on the above scheme, the global spatial correlation information of the image is obtained through the center pixel self-focus module, which reduces the computational complexity and cost. The homogeneous spectral information is extracted based on the cross-channel global fusion module and fused with the spatial information. It is then fed into the feedforward network layer for sufficient training and learning, which improves the classification accuracy. It has important application value in the fine classification of land surface in hyperspectral images.

[0060] Specifically, this embodiment provides detailed information on the classification method, including:

[0061] S1. Select a hyperspectral scene image, crop the hyperspectral scene image with a certain window size centered on the pixel, and make a hyperspectral cube based on the cropped hyperspectral scene image. Randomly select a portion of the cropped hyperspectral scene images from the hyperspectral cube as the training set and validation set, and use all samples as the test set. The center pixel refers to the feature element at the center position in the feature map.

[0062] S2. Construct the Transformer model. The network structure of the Transformer model is as follows: Figure 2As shown, the Transformer model includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input sample, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers include global self-information calculation layer 1, global self-information calculation layer 2, global self-information calculation layer 3, and global self-information calculation layer 4. The four global self-information calculation layers have the same structure, but the number of channels decreases layer by layer, so that the final number of channels is limited to the required number of classifications. Each global self-information calculation layer includes a center pixel self-attention module, a cross-channel global fusion module, and a feedforward network layer.

[0063] The structure of each module is as follows:

[0064] The preprocessing module consists of a transposed convolutional layer, a normalization layer, a ReLU layer, and a max pooling layer.

[0065] The center-cell self-attention module, also known as the center-cell self-attention mechanism computation layer, consists of a projection computation layer, a local residual layer, and a Dropout layer. The projection computation layer consists of a 2D transposed convolutional layer. The local residual layer consists of a 2D transposed convolutional layer and a BatchNorm layer.

[0066] The cross-channel fusion module consists of a global average pooling layer, a one-dimensional transposed convolutional layer, a one-dimensional transposed convolutional projection layer, a LayerNorm normalization layer, and a Dropout layer.

[0067] The feedforward network layer consists of three basic modules and an additional residual module. The basic modules consist of a two-dimensional LayerNorm layer, a GELU layer, and a two-dimensional transposed convolutional layer;

[0068] The preprocessing module converts hyperspectral image cubes t Perform initial mapping, remove redundant information and retain relatively important information, and save the features and the initial feature map x. 0 x 0 ∈R h×w×c Among them:

[0069] x * =BN(ConvTrans2d(s) t )) (1)

[0070] x 0 =Maxpool(max(x) * ,0)) (2)

[0071] The central pixel self-focus module is used to extract spatial self-similarity features and global spatial information measurement space from the input samples, and feeds them back to the feedforward network layer. The model diagram of the central pixel self-focus module is shown below. Figure 3 As shown, the processing procedure of the center pixel self-attention module is to obtain the feature map x of the input sample. f x f ∈R h×w×b h, w, and b represent the height, width, and number of channels of the hyperspectral cube, respectively. The three-dimensional matrices k and v are obtained according to formulas (3) and (4).

[0072] v = ConvTrans2d(x f (3)

[0073] k = ConvTrans2d(x f (4)

[0074] Extract the spectral vector at the center coordinates (h / 2, w / 2) of the three-dimensional matrix k as the measurement reference vector q∈R. 1 ×1×b Express the three-dimensional matrix k in two-dimensional form, where k (i,j) ∈R 1×1×b Let (i,j,u) represent the elements at spatial coordinates (i,j)∈(h,w), where each element represents a spectral vector of dimension b. Let (i,j,u)∈(h,w,b) be the three-dimensional spatial coordinates, and k be the vectors in the vectors. (i,j,u) Let k be a certain element inside the spectral vector. According to formula (5), all elements in matrix k are normalized.

[0075]

[0076] According to formula (6), the similarity matrix d is obtained by performing a vector dot product operation between the measurement reference vector q and the adjacent spectral vectors in the normalized k. Then, the similarity matrix d is activated according to formula (7). Finally, the output x is obtained by performing an element-wise product operation between matrix v and matrix d according to formula (8). out And x out ∈R h×w×b ,

[0077] d (i,j) =k i,j ×q (6)

[0078]

[0079]

[0080] According to formula (9), the feature map x f After processing, local spatial information is obtained, and x is calculated according to formula (10). out with res(x) f The pixels are then fused to obtain the final output of the center pixel self-focus module.

[0081] res(x f ) = BN(ConvTrans2d(x f (9)

[0082]

[0083] Here, res(·) represents the residual mapping unit, which includes a 3x3 transposed convolution and BatchNorm. The Drop(·) operation is used to randomly discard some redundant information to obtain the final output.

[0084] The processing procedure of the cross-channel global fusion module is as follows: Figure 4 As shown, specifically, the original features x of the input sample are processed according to formula (11). c The feature matrix x is obtained by performing global average pooling and normalization operations. qkv The feature matrix x is obtained according to formula (12). qkv Local channel information v c ,

[0085] x qkv =LN(GAP(x) c (11)

[0086] v c =ConvTrans1d(x qkv (12)

[0087] Where, x c As the original feature, x c ∈R b×h×w×c b represents the size of the batch data, and c, h, and w represent the number of channels, height, and width of the feature, respectively. qkv ∈R b×c ,

[0088] The normalized vector k is calculated according to formula (13). vec The VectorNorm operation is a vector normalization operation, and its calculation process is given by formula (14). Let (i,j)∈(b,c) be the coordinates in two-dimensional space, where vector k vec Each element k on i vec It is obtained by taking the square root of the sum of the squares of the elements in each row.

[0089] k vec =VectorNorm(k) (13)

[0090]

[0091] The similarity matrix r is calculated according to formula (15), where r∈R b×b q c With k c For x qkv Copying a matrix.

[0092] Using normalized vector k vec Based on row-corresponding element division, the similarity matrix r is processed, that is, assuming (u,v)∈(b,b) are two-dimensional spatial coordinates, then any element r in r (u,v) The calculation is performed using formula (16).

[0093] r = q c ×k c (15)

[0094]

[0095] r * =softmax(r) =softmax(r)v c (17)

[0096] Using the similarity matrix r from formula (17), the final channel self-attention matrix r can be obtained by performing softmax activation and matrix multiplication with the v matrix. * , and r * ∈R b×c The channel self-attention matrix represents the globally relevant information across different channels for different samples, i.e., the channel context attention information.

[0097] x out =x c +x c ⊙r * (18)

[0098] r * and x c Perform element-wise product and then residual operation to obtain the final output x. out .

[0099] The structure of the feedforward network layer is as follows: Figure 5 As shown, it specifically consists of three layers, each composed of a two-dimensional LayerNorm, Gelu, and ConvTrans2d operations. A group convolution ConvTrans2d is added to the second layer. * This is used to increase the diversity of self-similar features in the learning space.

[0100] Feed(·)=LayerNorm(Gelu(ConvTrans2d(·))) (19)

[0101] Assume the input feature is x r x r ∈R h×w×c Let h, w, and c be the length, width, and number of channels of the input features. The feature extraction behavior of each network layer is denoted as Feed(·), where t∈{1,2,3}. The feature extraction formula is as follows:

[0102] x t =Feed(x t-1 (20)

[0103] x out =Drop(x) 3 +x r ) (twenty one)

[0104] The final output x is obtained after performing the Drop operation according to formula (21). out After feature extraction via a three-layer network, residual operations are performed to reduce the potential gradient vanishing problem. Finally, the output x is obtained after a Drop operation. out .

[0105] S3. Train the Transformer model using cross-entropy classification loss based on the training set, and validate the trained Transformer model using the validation set. Then, classify the hyperspectral image to be classified based on the validated Transformer model.

[0106] A hyperspectral image classification system based on the same inventive concept includes an image acquisition and processing module, a Transformer model construction module, and a model training and application module.

[0107] The image acquisition and processing module is used to select hyperspectral scene images, crop the hyperspectral scene images with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene images, and randomly select a portion of the cropped hyperspectral scene images from the hyperspectral cube as training and validation sets.

[0108] The Transformer model building module is used to construct the Transformer model. The Transformer model includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module extracts features from the input image and reduces the number of channels. The four global self-information calculation layers are Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. These four layers have the same structure but gradually decrease the number of channels. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module extracts spatial self-similarity features and a global spatial information metric space from the input sample and feeds them back to the feedforward network layer. The cross-channel global fusion module calculates the uniform global channel information of the input sample and feeds it back to the feedforward network layer. The feedforward network layer performs feature extraction based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information.

[0109] The model training and application module is used to train the Transformer model using cross-entropy classification loss based on the training set, and to validate the trained Transformer model using the validation set. The validated Transformer model is then used to classify the hyperspectral image to be classified.

[0110] This embodiment provides a specific implementation process, including the following steps:

[0111] S1: Select hyperspectral scene images and take a portion of them as the training set; specifically, the hyperspectral scene images in this embodiment are the Indian Pine dataset, acquired by the AVIRIS sensor in 1992 over the Indian Pine Test Site in northwestern Indiana. This image captures 200 spectral bands, with a pixel size of 145x145 and a spatial resolution of 20 meters. After removing the water absorption band, there are 16 different levels. As shown in Table 1:

[0112] Table 1 shows the number of samples for target land cover categories in the Indian pines dataset.

[0113]

[0114]

[0115] In this embodiment, an experiment was conducted on the Indian Pines dataset using a hyperspectral image classification method based on a central pixel self-focus mechanism and a cross-channel global fusion module of the Transformer network according to the present invention. The experimental results are shown in Table 2.

[0116] Table 2. Classification accuracy of Indian Pines (%)

[0117]

[0118]

[0119] OA (Overall Accuracy) represents the overall classification accuracy, AA (Average Accuracy) represents the average classification accuracy, and Kappa represents the Kappa coefficient. The Kappa coefficient is a multivariate discrete method for evaluating the classification accuracy and error matrix of remote sensing images. It takes into account various missed and misclassified pixels outside the diagonal, and can penalize the bias of the model, thus providing a more comprehensive evaluation of the classification effect. Figure 6 Test images for the test dataset, Figure 7 This is a diagram showing the classification results for the test dataset.

[0120] To more objectively evaluate the role of each module in the hyperspectral image classification method based on the center-of-interest Transformer model in this invention, existing ablation experiments were added for illustration. The specific experimental results are shown in Table 3.

[0121] Table 3. Classification accuracy (%) of different modules

[0122]

[0123] The above experiments lead to the following conclusions:

[0124] (1) The experimental results in Table 2 show that the proposed image classification method based on the center attention Transformer model has good classification effect, which proves that the method has superior performance in hyperspectral image classification.

[0125] (2) The ablation experiment data in Table 3 shows that the classification results with the addition of the cross-channel global information fusion module are significantly better than those with the center pixel self-focus calculation module alone, which proves that the addition of the cross-channel global information fusion module shows more robust performance in hyperspectral image classification.

[0126] Overall beneficial effects:

[0127] This invention provides a hyperspectral image classification method and system based on a center-focused Transformer model. By acquiring global spatial correlation information of the image through a center pixel self-focused module, the computational complexity and cost are reduced. Homogeneous spectral information is extracted based on a cross-channel global fusion module and fused with spatial information. This information is then fed into a feedforward network layer for sufficient training and learning, thereby improving classification accuracy. This method has significant application value in the fine classification of land surfaces in hyperspectral images.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral image classification method based on a center-of-interest Transformer model, characterized in that, include, S1. Select a hyperspectral scene image, crop the hyperspectral scene image with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene image, and randomly select a portion of the cropped hyperspectral scene image from the hyperspectral cube as the training set and validation set. S2. Construct a Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input sample, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers include Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. The four global self-information calculation layers have the same structure, but the number of channels decreases layer by layer. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module is used to extract the spatial self-similarity features and global spatial information metric space of the input sample and feed them back to the feedforward network layer. The cross-channel global fusion module is used to calculate the uniform global channel information of the input sample and feed it back to the feedforward network layer. The feedforward network layer is used to extract features based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information. S3. Train the Transformer model using cross-entropy classification loss based on the training set, and validate the trained Transformer model using the validation set. Then, classify the hyperspectral image to be classified based on the validated Transformer model.

2. The hyperspectral image classification method based on the center-of-interest Transformer model according to claim 1, characterized in that, The processing procedure of the center pixel self-attention module is to obtain the feature map x of the input sample. f x f ∈R h×w×b h, w, and b represent the height, width, and number of channels of the hyperspectral cube, respectively. The three-dimensional matrices k and v are obtained according to formulas (1) and (2). v=ConvTrans2d(x f ) (1) k=ConvTrans2d(x f ) (2) Extract the spectral vector at the center coordinates (h / 2, w / 2) of the three-dimensional matrix k as the measurement reference vector q∈R. 1×1×b Express the three-dimensional matrix k in two-dimensional form, where k (i,j) ∈R 1×1×b Let (i,j,u) represent the elements at spatial coordinates (i,j)∈(h,w), where each element represents a spectral vector of dimension b. Let (i,j,u)∈(h,w,b) be the three-dimensional spatial coordinates, and k be the vectors in the vectors. (i,j,u) Let k be a certain element inside the spectral vector. According to formula (3), all elements in matrix k are normalized. According to formula (4), the similarity matrix d is obtained by performing a vector dot product operation between the measurement reference vector q and the adjacent spectral vectors in the normalized k. Then, the similarity matrix d is activated according to formula (5). Finally, the output x is obtained by performing an element-wise product operation between matrix v and matrix d according to formula (6). out And x out ∈R h×w×b , d (i,j) =k i,j ×q (4) According to formula (7), the feature map x f After processing, local spatial information is obtained, and x is calculated according to formula (8). out with res(x) f The pixels are then fused to obtain the final output of the center pixel self-focus module. res(x f )=BN(ConvTrans2d(x f )) (7) Here, res(·) represents the residual mapping unit, which includes a 3x3 transposed convolution and BatchNorm. The Drop(·) operation is used to randomly discard some redundant information to obtain the final output.

3. The hyperspectral image classification method based on the center-of-interest Transformer model according to claim 1, characterized in that, The processing procedure of the cross-channel global fusion module is as follows: based on formula (9), the original features x of the input sample are processed. c The feature matrix x is obtained by performing global average pooling and normalization operations. qkv The feature matrix x is obtained according to formula (10). qkv Local channel information v c , x qkv =LN(GAP(x c )) (9) v c =ConvTrans1d(x qkv ) (10) Where, x c As the original feature, x c ∈R b×h×w×c b represents the size of the batch data, and c, h, and w represent the number of channels, height, and width of the feature, respectively. qkv ∈R b×c , The normalized vector k is calculated according to formula (11). vec The VectorNorm operation is a vector normalization operation. The calculation process of this operation is given by formula (12). Let (i,j)∈(b,c) be the coordinates in two-dimensional space, where vector k vec Each element on It is obtained by taking the square root of the sum of the squares of the elements in each row. k vec =VectorNorm(k) (11) The similarity matrix r is calculated according to formula (13), where r∈R b×b q c With k c For x qkv Copying a matrix using a normalized vector k vec Based on row-corresponding element division, the similarity matrix r is processed, that is, assuming (u,v)∈(b,b) are two-dimensional spatial coordinates, then any element r in r (u,v) The calculation is performed using formula (14). r=q c ×k c (13) r * =softmax(r)=softmax(r)v c (15) Using the similarity matrix r from formula (15), the final channel self-attention matrix r can be obtained by performing softmax activation and matrix multiplication with the v matrix. * , and r * ∈R b×c The channel self-attention matrix represents the globally relevant information across different channels for different samples, i.e., the channel context attention information. x out =x c +x c ⊙r * (16) r * and x c Perform element-wise product and then residual operation to obtain the final output x. out .

4. The hyperspectral image classification method based on the center-of-interest Transformer model according to claim 1, characterized in that, The feedforward network layer consists of three layers, each composed of a two-dimensional LayerNorm, Gelu, and ConvTrans2d operations. A group convolution ConvTrans2d is added to the second layer. * This is used to increase the diversity of self-similar features in the learning space. Feed(·)=LayerNorm(Gelu(ConvTrans2d(·))) (17) Assume the input feature is x r x r ∈R h×w×c Let h, w, and c be the length, width, and number of channels of the input features. The feature extraction behavior of each network layer is denoted as Feed(·), where t∈{1,2,3}. The feature extraction formula is as follows: x t =Feed(x t-1 ) (18) x out =Drop(x 3 +x r ) (19) The final output x is obtained after performing the Drop operation according to formula (19). out .

5. A hyperspectral image classification system based on the classification method of claim 1, characterized in that, It includes an image acquisition and processing module, a Transformer model building module, and a model training and application module. The image acquisition and processing module is used to select hyperspectral scene images, crop the hyperspectral scene images with a certain window size centered on a pixel, construct a hyperspectral cube based on the cropped hyperspectral scene images, and randomly select a portion of the cropped hyperspectral scene images from the hyperspectral cube as training and validation sets. The Transformer model building module is used to construct the Transformer model, which includes a preprocessing module, four global self-information calculation layers, and a global average pooling layer. The preprocessing module uses the cropped hyperspectral image as input, extracts features from the input sample, and reduces the number of channels in the input image. The four global self-information calculation layers are Global Self-Information Calculation Layer 1, Global Self-Information Calculation Layer 2, Global Self-Information Calculation Layer 3, and Global Self-Information Calculation Layer 4. These four layers have the same structure but the number of channels decreases progressively. Each global self-information calculation layer includes a center pixel self-focus module, a cross-channel global fusion module, and a feedforward network layer. The center pixel self-focus module extracts the spatial self-similarity features and global spatial information metric space of the input sample and feeds them back to the feedforward network layer. The cross-channel global fusion module calculates the uniform global channel information of the input sample and feeds it back to the feedforward network layer. The feedforward network layer performs feature extraction based on the spatial self-similarity features, the global spatial information metric space, and the uniform global channel information. The model training and application module is used to train the Transformer model using cross-entropy classification loss based on the training set, and to validate the trained Transformer model using the validation set. The validated Transformer model is then used to classify the hyperspectral image to be classified.

Images