A hyperspectral image classification method, system, electronic device and medium

Abstract

The application discloses a hyperspectral image classification method and system, an electronic device and a medium, and relates to the field of hyperspectral image processing. The method comprises the following steps: applying factor analysis to dimension reduction on a hyperspectral image to be classified, and extracting a plurality of three-dimensional pixel blocks from the dimension-reduced hyperspectral image to be classified with each pixel point as a center point in a preset area; and inputting the plurality of three-dimensional pixel blocks into a classification model to obtain a classification result of the hyperspectral image to be classified. The application can improve the accuracy of the classification result of the hyperspectral image.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image processing, and in particular to a hyperspectral image classification method, system, electronic device, and medium. Background Technology

[0002] Currently, there are two main types of pixel-level classification methods for hyperspectral images: one is based on manually generated features, and the other is based on deep learning features. The paper "Ramachandran P, Parmar N, Vaswani A, et al. Stand-alone self-attention in vision models[J]. Advances in neural information processing systems, 2019, 32." discloses a model architecture based on a self-attention mechanism applied to the vision domain. This model utilizes an independent attention layer to build a fully attentive vision model, which outperforms convolutional baselines in both image classification and object detection, while also achieving high parameter and computational efficiency. This paper proposes the design of an independent local self-attention module, improving classification performance. However, the above method mainly relies on the interaction of independent query key pairs to compute the attention matrix, thus ignoring the rich contextual information of adjacent keys. Summary of the Invention

[0003] To overcome the problem that traditional Transformer-based structures ignore the rich contextual information of adjacent bonds, this invention provides a hyperspectral image classification method, system, electronic device, and medium that can improve the accuracy of hyperspectral image classification results.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A hyperspectral image classification method, the classification method comprising:

[0006] Factor analysis is applied to reduce the dimensionality of the hyperspectral image to be classified, and multiple three-dimensional pixel blocks are extracted from the dimensionality-reduced hyperspectral image within a preset region, with each pixel as the midpoint.

[0007] Multiple three-dimensional pixel blocks are input into a classification model to obtain the classification result of the hyperspectral image to be classified. The classification model is obtained by training a deep learning network on a training set. The deep learning network includes multiple feature fusion-permutation attention mechanism models, sampling layers, and fully connected layers connected in sequence. The feature fusion-permutation attention mechanism model includes a feature fusion model and a permutation attention mechanism connected in sequence. The feature fusion model includes a two-dimensional convolution, a multi-scale residual context Transformer network, and a stitching module connected in sequence. The training set includes multiple three-dimensional pixel blocks of hyperspectral image samples and the classification results corresponding to the hyperspectral image samples.

[0008] Optionally, the step of extracting multiple three-dimensional pixel blocks from the dimensionality-reduced hyperspectral image to be classified within a preset area, using each pixel as the midpoint, specifically includes:

[0009] Based on the coordinates of each pixel within the preset region, and according to the preset spatial sliding window size, multiple three-dimensional pixel blocks are extracted from the dimensionality-reduced hyperspectral image to be classified.

[0010] Optionally, when the coordinates of a pixel within the preset region are (a, β) and the preset spatial sliding window size is S×S, the width of the three-dimensional pixel block centered on the pixel with coordinates (a, β) is [a-(S-1) / 2, a+(S-1) / 2], and the height is [β-(S-1) / 2, β+(S-1) / 2]. The channel dimension of the three-dimensional pixel block centered on the pixel with coordinates (a, β) is consistent with the channel dimension of the dimensionality-reduced hyperspectral image to be classified.

[0011] Optionally, the deep learning network includes two connected feature fusion-permutation attention mechanism models;

[0012] The multiple three-dimensional pixel blocks are input into the feature fusion model in the first feature fusion-permutation attention mechanism model, and the first fused feature map is output.

[0013] By applying a permutation attention mechanism, the feature information of the spatial dimension and the feature information of the channel dimension of the first fused feature map are weighted to obtain the feature map after initial calibration.

[0014] The initially calibrated feature map is input into the feature fusion model in the second feature fusion-permutation attention mechanism model, and the second fused feature map is output.

[0015] By applying a permutation attention mechanism, the feature information of the spatial dimension and the feature information of the channel dimension of the second fused feature map are weighted to obtain the calibrated feature map.

[0016] Optionally, the application of the permutation attention mechanism weights the spatial dimension feature information and the channel dimension feature information of the second fused feature map to obtain a calibrated feature map, specifically including:

[0017] The second fused feature map is grouped along the channel dimension to obtain multiple groups of sub-feature maps;

[0018] Each group of sub-feature maps is divided into two parts along the channel dimension to obtain two branch feature maps; the two branch feature maps include a first branch feature map and a second branch feature map.

[0019] The first branch feature map is compressed along the channel dimension to compress the spatial dimension feature information, and a gating mechanism and activation function are applied to the compressed spatial dimension feature information to obtain the calibrated first branch feature map.

[0020] For the second branch feature map, group normalization is applied to obtain the normalized spatial dimension feature information;

[0021] A gating mechanism and activation function are applied to the normalized spatial dimension feature information to obtain an attention weight map;

[0022] The second branch feature map is multiplied by the attention weight map to obtain the calibrated second branch feature map;

[0023] The calibrated first branch feature map and the calibrated second branch feature map are connected along the channel dimension to obtain the calibrated sub-feature map;

[0024] The calibrated sub-feature maps from each group are aggregated to obtain the aggregated feature map.

[0025] A channel permutation operation is applied to the aggregated feature map to obtain a calibrated feature map.

[0026] Optionally, the multi-scale residual context Transformer network includes multiple parallel branches of different scales; the parallel branches include a first two-dimensional convolution, a CoT block, and a second two-dimensional convolution connected in sequence; the kernel size of the first two-dimensional convolution and the second two-dimensional convolution are the same.

[0027] A hyperspectral image classification system, applying the above-mentioned hyperspectral image classification method, the classification system comprising:

[0028] The extraction module is used to perform dimensionality reduction on the hyperspectral image to be classified by factor analysis, and extract multiple three-dimensional pixel blocks from the dimensionality-reduced hyperspectral image to be classified within a preset area, with each pixel as the midpoint.

[0029] A classification module is used to input multiple three-dimensional pixel blocks into a classification model to obtain the classification result of the hyperspectral image to be classified. The classification model is obtained by training a deep learning network on a training set. The deep learning network includes multiple feature fusion-permutation attention mechanism models, sampling layers, and fully connected layers connected in sequence. The feature fusion-permutation attention mechanism model includes a feature fusion model and a permutation attention mechanism connected in sequence. The feature fusion model includes a two-dimensional convolution, a multi-scale residual context Transformer network, and a stitching module connected in sequence. The training set includes multiple three-dimensional pixel blocks of hyperspectral image samples and the classification results corresponding to the hyperspectral image samples.

[0030] An electronic device includes a memory and a processor, the memory storing a computer program and the processor running the computer program to enable the electronic device to perform the hyperspectral image classification method described above.

[0031] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the hyperspectral image classification method described above.

[0032] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] The feature fusion model applied in this invention takes into account the rich contextual information of adjacent keys, enabling it to fully capture this information and improve classification performance. Furthermore, to better reflect the feature information required for hyperspectral image classification, a multi-scale residual context Transformer network is applied for feature fusion to obtain a fused feature map. Additionally, a permutation attention mechanism reduces the loss of spatial-spectral features caused by the combination of attention mechanisms. This invention overcomes the problem of traditional Transformer-based structures neglecting the rich contextual information of adjacent keys, thus improving the accuracy of hyperspectral image classification results. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the hyperspectral image classification method of the present invention;

[0036] Figure 2 This is a flowchart illustrating the practical application of the small-sample hyperspectral image classification method of the present invention.

[0037] Figure 3 This is a schematic diagram of the multi-scale residual context Transformer network structure of the present invention;

[0038] Figure 4 This is a schematic diagram of the displacement attention mechanism of the present invention;

[0039] Figure 5 This is a schematic diagram illustrating the structure of the small-sample hyperspectral image classification method of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The purpose of this invention is to provide a hyperspectral image classification method, system, electronic device, and medium that can improve the accuracy of hyperspectral image classification results.

[0042] This invention discloses a multi-scale residual context Transformer network based on a permutation attention mechanism to solve the classification task of small-sample hyperspectral images. The data samples in this invention remain unchanged after sampling, and a network architecture design is employed. First, CoT is a context Transformer block, and ResCoT is a residual context Transformer block, designed to capture contextual information. The multi-scale ResCoT module, a multi-scale residual context Transformer block, can extract and fuse multi-scale features and embed them into a convolutional neural network, thereby improving feature representation capabilities. Finally, a permutation attention mechanism, consisting of a spatial attention module and a channel attention module, is used to reduce the loss of spatial-spectral features caused by the combination of the two attention mechanisms. Simultaneously, the permutation attention mechanism can enhance the extraction of contextual features, thus improving classification performance.

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] like Figure 1 and Figure 2 As shown, the present invention provides a hyperspectral image classification method, the classification method comprising:

[0046] Step S1: Apply factor analysis to reduce the dimensionality of the hyperspectral image to be classified, and extract multiple three-dimensional pixel blocks from the dimensionality-reduced hyperspectral image within a preset region, using each pixel as the midpoint.

[0047] Specifically, based on the coordinates of each pixel within the preset region, and according to a preset spatial sliding window size, multiple three-dimensional pixel blocks are extracted from the dimensionality-reduced hyperspectral image to be classified.

[0048] Furthermore, when the coordinates of a pixel within the preset region are (a, β) and the preset spatial sliding window size is S×S, the width of the three-dimensional pixel block centered on the pixel with coordinates (a, β) is [a-(S-1) / 2, a+(S-1) / 2], and the height is [β-(S-1) / 2, β+(S-1) / 2]. The channel dimension of the three-dimensional pixel block centered on the pixel with coordinates (a, β) is consistent with the channel dimension of the dimensionality-reduced hyperspectral image to be classified.

[0049] In practical applications, hyperspectral images are factor-analyzed to reduce their channel dimension to D, minimizing spectral information redundancy. The reduced-dimensional hyperspectral image requires the extraction of three-dimensional pixel blocks with a spatial dimension of B×B. Unlike the center pixel, edge pixels require image filling before extraction. Each three-dimensional pixel block is extracted around the center pixel, and the block's category is the same as the center pixel's category. Five pixels from each category are used for training, while the remaining pixels are used for the test dataset. The final extracted three-dimensional pixel block has a dimension of B×B×D and serves as input for the next part.

[0050] Specifically, assume that the input hyperspectral image is defined as I∈R H×W×C The initial input hyperspectral image is I, with width H, height W, and channel dimension C. First, the hyperspectral image I is reduced to P∈R through factor analysis. H×W×D P is the image after dimensionality reduction of the channel dimension through factor analysis, and the channel dimension is D. After the Lomway step of factor analysis, we extract the three-dimensional pixel block X∈R from the data P. S×S×D Centered on each marked pixel (a, β), extract data with a width ranging from a-(S-1) / 2 to a+(S-1) / 2 and a height ranging from β-(S-1) / 2 to β+(S-1) / 2, and a channel dimension of D, as input to form a three-dimensional pixel block X. (a,β) Where S×S is the size of the spatial sliding window for extracting the 3D pixel blocks. These 3D pixel blocks X∈R S ×S×D It will be used as input to the model.

[0051] Step S2: Input multiple three-dimensional pixel blocks into the classification model to obtain the classification result of the hyperspectral image to be classified; the classification model is obtained by training a deep learning network on a training set; the deep learning network includes multiple feature fusion-permutation attention mechanism models, sampling layers, and fully connected layers connected in sequence; the feature fusion-permutation attention mechanism model includes a feature fusion model and a permutation attention mechanism connected in sequence; the feature fusion model includes a two-dimensional convolution, a multi-scale residual context Transformer network, and a stitching module connected in sequence; the training set includes multiple three-dimensional pixel blocks of hyperspectral image samples and the classification results corresponding to the hyperspectral image samples.

[0052] The process of determining the classification model includes: taking hyperspectral image samples as input, taking the classification results corresponding to the hyperspectral image samples as output, training the deep learning network to obtain a trained deep learning network, and using the trained deep learning network as the classification model.

[0053] In one specific implementation, the multi-scale residual context Transformer network includes multiple parallel branches of different scales; the parallel branches include a first two-dimensional convolution, a CoT block, and a second two-dimensional convolution connected in sequence; the kernel size of the first two-dimensional convolution and the second two-dimensional convolution are the same.

[0054] Multi-scale ResCoT is applied to capture contextual information and extract and fuse multi-scale features. In practical applications, the 3D pixel blocks extracted during data preprocessing are passed through a 2D convolution before being input into Multi-scale ResCoT. Multi-scale ResCoT consists of three parallel branches at different scales. Each parallel branch has two 2D convolutions and one CoT block. The 2D convolution kernels of different parallel branches have different sizes. The CoT block is used to extract rich contextual information. The 3D pixel blocks are passed through 2D convolutions and then input into the three branches of Multi-scale ResCoT at different scales. The results of each branch are then concatenated.

[0055] Specifically, such as Figure 3 As shown, x is the output of a three-dimensional pixel block after a two-dimensional convolution, and Conv consists of two two-dimensional convolutions in ResCoT. First, x undergoes a two-dimensional convolution, and the output is input into CoT. Then, it is input into another two-dimensional convolution to obtain the output. The calculation formula is shown below:

[0056] ResCoT=Conv(CoT(Conv(x))) (1).

[0057] Then, the kernel sizes of the convolutional layers in the ResCoT module are adjusted to multiple sizes, namely 1×1, 3×3, and 5×5, and combined. ResCoT(1) is the result of setting the two-dimensional convolutional kernel to 1 in equation (1), ResCoT(3) is the result of setting the two-dimensional convolutional kernel to 3 in equation (1), and ResCoT(5) is the result of setting the two-dimensional convolutional kernel to 5 in equation (1). Concat is the summation operation, and X is the summation of the three multi-scale output results in ResCoT. The calculation formula is shown in (2):

[0058] X=Concat(ResCoT(1),ResCoT(3),ResCoT(5)) (2).

[0059] In one specific implementation, the deep learning network includes two connected feature fusion-permutation attention mechanism models; multiple three-dimensional pixel blocks are input into the feature fusion model in the first feature fusion-permutation attention mechanism model to output a first fused feature map; a permutation attention mechanism is applied to weight the spatial dimension feature information and the channel dimension feature information of the first fused feature map to obtain an initially calibrated feature map; the initially calibrated feature map is input into the feature fusion model in the second feature fusion-permutation attention mechanism model to output a second fused feature map; a permutation attention mechanism is applied to weight the spatial dimension feature information and the channel dimension feature information of the second fused feature map to obtain a calibrated feature map.

[0060] The permutation attention mechanism reduces the spatial-spectral feature loss caused by the combination of two attention mechanisms. In practical applications, the multi-scale data output from the Multi-scale ResCoT module is concatenated and then input into the permutation attention mechanism module of the model. This module groups the input data along the channel dimension, and each feature group is further divided into two branches. One branch processes the input data using the channel attention mechanism, and the other branch processes the input data using the spatial attention mechanism. The results from the two branches are concatenated to obtain new sub-features. Each feature group will obtain a new sub-feature, and finally, all new sub-features are concatenated along the channel dimension.

[0061] Specifically, a permutation attention mechanism is applied to weight the spatial and channel-dimensional feature information of the second fused feature map to obtain a calibrated feature map, which includes:

[0062] Step 1: Group the second fused feature map along the channel dimension to obtain multiple groups of sub-feature maps.

[0063] In practical applications, the input feature maps are grouped along the channel dimension. Assume the input feature maps are X∈R. H ×W×C Divide X into i groups along the channel dimension, i.e., {x1, x2, x3, ..., x i}, where x i ∈R H×W×C / i Each sub-feature map xi captures specific semantic information during training, and an attention mechanism is used to process each sub-feature map x. i The feature information of the spatial dimension and channel dimension in the data is weighted.

[0064] Step 2: Divide each group of sub-feature maps into two parts along the channel dimension to obtain two branch feature maps; the two branch feature maps include a first branch feature map and a second branch feature map.

[0065] In practical applications, before weighting each sub-feature map using an attention mechanism, it is necessary to calculate the weight of each sub-feature map x. i Divided into two branches: x k1 ∈R H×W×C / 2i and x k2 ∈R H×W×C / 2i .

[0066] Step 3: Compress the spatial dimension feature information of the first branch feature map along the channel direction, and apply a gating mechanism and activation function to the compressed spatial dimension feature information to obtain the calibrated first branch feature map.

[0067] In practical applications, the feature maps x of the two branches are... k1 and x k2 Weighting is performed using both channel and spatial attention mechanisms. Input x k1 (i,j) represents the input x with width i and height j. k1 The image, where (i,j) represents x with different widths (W) and heights (H) of the input. k1 After global average pooling, GAP(x) k1 The spatial information of spatial dimension H×W is compressed along the channel direction to obtain g, which represents the global distribution of feature information in the channel dimension. The calculation formula is shown in (3):

[0068]

[0069] Then g is passed through a simple gating mechanism F c And the sigmoid activation function σ guides the model to learn the importance of feature maps in the channel dimension, and finally with x k1 Multiplication yields the calibrated first branch feature map x' k1This enables the model to adaptively weight the channel information on the feature map, and the calculation formula is shown in (4):

[0070] x' k1 =σ(F c (g))×x k1 (4).

[0071] Step 4: Apply group normalization to the second branch feature map to obtain the normalized spatial dimension feature information.

[0072] Step 5: Apply a gating mechanism and activation function to the normalized spatial dimension feature information to obtain the attention weight map.

[0073] In practical applications, within the spatial attention module, by focusing on x... k2 Group normalization is used to obtain information in the spatial dimension, and then processed by F... c The sigmoid activation function σ is used to obtain an attention weight map representing the importance of different spatial information.

[0074] Step 6: Multiply the second branch feature map with the attention weight map to obtain the calibrated second branch feature map.

[0075] In practical applications, the attention weight map is compared with x. k2 Multiplication yields the calibrated second branch feature map x' k2 This guides the model to adaptively weight the feature information in the spatial dimension. The calculation formula is shown in (5):

[0076] x' k2 =σ(F c (GN(x k2 )))×x k2 (5).

[0077] Step 7: Connect the calibrated first branch feature map and the calibrated second branch feature map along the channel dimension to obtain the calibrated sub-feature map.

[0078] In practical applications, the feature map x' of the two branches is weighted. k1 and x' k2 By concatenating them along the channel dimension, making the channel dimension the same as that of the input feature map, a new sub-feature map x' is obtained. i ∈R H×W×C / i The calculation formula is shown in (6):

[0079] x' i =Concat(x' k1 ,x'k2 (6).

[0080] Step 8: Aggregate the calibrated sub-feature maps from each group to obtain the aggregated feature map.

[0081] In practical applications, all sub-features x' after being weighted by the attention mechanism will be... i Gather together.

[0082] Step 9: Apply a channel permutation operation to the aggregated feature map to obtain a calibrated feature map.

[0083] Specifically, the permutation attention mechanism is applied to weight the feature information of the spatial dimension and the feature information of the channel dimension of the first fused feature map to obtain the initially calibrated feature map. The specific process is the same as the calculation process of "applying the permutation attention mechanism to weight the feature information of the spatial dimension and the feature information of the channel dimension of the second fused feature map to obtain the calibrated feature map", except that the input data is different and the corresponding output data is also different.

[0084] In practical applications, such as Figure 4 As shown, channel permutation shuffles the channels of different sub-features in the aggregated feature map. This enhances the model's feature representation ability under limited sample training conditions and improves classification performance. Figure 4 In this context, σ represents the Sigmoid activation function, G represents the global average pooling layer, S represents channel transformation, GN represents group normalization, and F represents... c This indicates a gating mechanism.

[0085] The permutational attention mechanism reduces the spatial-spectral feature loss caused by the combination of two attention mechanisms. In practical applications, the multi-scale data output from the Multi-scale ResCoT module is concatenated and then input into the permutational attention mechanism module of the model. This module groups the input data along the channel dimension, and each feature group is further divided into two branches. One branch processes the input data using the channel attention mechanism, and the other branch processes the input data using the spatial attention mechanism. The results from the two branches are concatenated to obtain new sub-features. Each feature group will obtain a new sub-feature, and finally, all new sub-features are concatenated along the channel dimension.

[0086] In practical applications, the output of the network model is first pooled to reshape the result into a one-dimensional vector. The dimension of the one-dimensional vector is then transformed into the dimension represented by the number of categories through a fully connected layer.

[0087] Specifically, such as Figure 5As shown, in the hyperspectral image classification task, a convolutional neural network classification model is used to extract features from the input hyperspectral image. However, the feature information extracted by several convolution and pooling operations cannot be directly used for classification. Instead, it needs to be weighted and summed through the weight matrix of the fully connected layer to map the feature space to the sample label space, thereby integrating all the features to form a discriminative one-dimensional vector, and finally obtaining the probability of which class each target belongs to.

[0088] exist Figure 5 In this process, after the hyperspectral image undergoes dimensionality reduction through factor analysis, adjacent key extraction is performed to obtain multiple three-dimensional pixel blocks. After two-dimensional convolution, the blocks enter a multi-scale residual context Transformer network, followed by summation. After summation, a permutation attention mechanism is applied. After the permutation attention mechanism, the process of "after two-dimensional convolution, the blocks enter a multi-scale residual context Transformer network, followed by summation, and after summation, a permutation attention mechanism is applied" is repeated. The resulting image passes through a sampling layer and a fully connected layer, and the classification result is output.

[0089] Example 2

[0090] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a hyperspectral image classification system is provided below, the classification system comprising:

[0091] The extraction module is used to perform dimensionality reduction on the hyperspectral image to be classified using factor analysis, and to extract multiple three-dimensional pixel blocks from the dimensionality-reduced hyperspectral image within a preset region, with each pixel as the midpoint.

[0092] A classification module is used to input multiple three-dimensional pixel blocks into a classification model to obtain the classification result of the hyperspectral image to be classified. The classification model is obtained by training a deep learning network on a training set. The deep learning network includes multiple feature fusion-permutation attention mechanism models, sampling layers, and fully connected layers connected in sequence. The feature fusion-permutation attention mechanism model includes a feature fusion model and a permutation attention mechanism connected in sequence. The feature fusion model includes a two-dimensional convolution, a multi-scale residual context Transformer network, and a stitching module connected in sequence. The training set includes multiple three-dimensional pixel blocks of hyperspectral image samples and the classification results corresponding to the hyperspectral image samples.

[0093] Example 3

[0094] This invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the hyperspectral image classification method of Embodiment 1.

[0095] Alternatively, the aforementioned electronic device may be a server.

[0096] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the hyperspectral image classification method of Embodiment 1.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0098] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of hyperspectral image classification, characterized in that, The classification method comprises: dimensionality reduction is performed on the hyperspectral image to be classified by using factor analysis, and a plurality of three-dimensional pixel blocks are extracted from the dimensionally reduced hyperspectral image to be classified with each pixel point as a center point in a preset region; inputting the plurality of three-dimensional pixel blocks into a classification model to obtain a classification result of the hyperspectral image to be classified; the classification model is obtained by training a deep learning network using a training set; the deep learning network comprises a plurality of feature fusion-permutation attention mechanism models, a sampling layer and a full connection layer connected in sequence; the feature fusion-permutation attention mechanism model comprises a feature fusion model and a permutation attention mechanism connected in sequence; the feature fusion model comprises a two-dimensional convolution, a multi-scale residual context Transformer network and a splicing module connected in sequence; the training set comprises a plurality of three-dimensional pixel blocks of a hyperspectral image sample and a classification result corresponding to the hyperspectral image sample; the deep learning network comprises two feature fusion-permutation attention mechanism models connected in sequence; inputting the plurality of three-dimensional pixel blocks into the feature fusion model in the first feature fusion-permutation attention mechanism model to output a first fused feature map; applying the permutation attention mechanism to weight the spatial dimension feature information and the channel dimension feature information of the first fused feature map to obtain an initial calibrated feature map; inputting the initial calibrated feature map into the feature fusion model in the second feature fusion-permutation attention mechanism model to output a second fused feature map; applying the permutation attention mechanism to weight the spatial dimension feature information and the channel dimension feature information of the second fused feature map to obtain a calibrated feature map, specifically comprising: grouping the second fused feature map along the channel dimension to obtain a plurality of sub-feature maps; dividing each group of sub-feature maps into two parts along the channel dimension to obtain two branch feature maps; the two branch feature maps comprise a first branch feature map and a second branch feature map; compressing the spatial dimension feature information of the first branch feature map along the channel dimension, and applying a gating mechanism and an activation function to the compressed spatial dimension feature information to obtain a calibrated first branch feature map; applying group normalization to the second branch feature map to obtain normalized spatial dimension feature information; applying a gating mechanism and an activation function to the normalized spatial dimension feature information to obtain an attention weight map; multiplying the second branch feature map and the attention weight map corresponding to each other to obtain a calibrated second branch feature map; connecting the calibrated first branch feature map and the calibrated second branch feature map in the channel dimension to obtain a calibrated sub-feature map; aggregating each group of calibrated sub-feature maps to obtain an aggregated feature map; applying a channel permutation operation to the aggregated feature map to obtain a calibrated feature map.

2. The method of claim 1, wherein, the plurality of three-dimensional pixel blocks are extracted from the dimensionally reduced hyperspectral image to be classified with each pixel point as a center point in a preset region, specifically comprising: According to the coordinates of each pixel point in the preset region, a plurality of three-dimensional pixel blocks are extracted from the dimension-reduced hyperspectral image to be classified according to a preset spatial sliding window size.

3. The method of claim 2, wherein, When the coordinates of one of the pixel points in the preset region are (a, β) and the preset spatial sliding window size is , the width of the three-dimensional pixel block with the pixel point with coordinates (a, β) as the center is[ , ], the height is[ , ], and the channel dimension of the three-dimensional pixel block with the pixel point with coordinates (a, β) as the center is consistent with the channel dimension of the reduced dimension hyperspectral image to be classified.

4. The method of claim 1, wherein, The multi-scale residual context Transformer network comprises a plurality of parallel branches of different scales; the parallel branches comprise a first two-dimensional convolution, a CoT block and a second two-dimensional convolution connected in sequence; the convolution kernel size of the first two-dimensional convolution is consistent with that of the second two-dimensional convolution.

5. A hyperspectral image classification system, characterized by, The classification system comprises: The extraction module is configured to apply factor analysis for dimension reduction on the hyperspectral image to be classified, and extract a plurality of three-dimensional pixel blocks from the dimension-reduced hyperspectral image to be classified with each pixel point as a center point in a preset region; The classification module is configured to input the plurality of three-dimensional pixel blocks into a classification model to obtain a classification result of the hyperspectral image to be classified; the classification model is obtained by training a deep learning network using a training set; the deep learning network comprises a plurality of feature fusion-permutation attention mechanism models, a sampling layer and a full connection layer connected in sequence; the feature fusion-permutation attention mechanism model comprises a feature fusion model and a permutation attention mechanism connected in sequence; the feature fusion model comprises a two-dimensional convolution, a multi-scale residual context Transformer network and a splicing module connected in sequence; the training set comprises a plurality of three-dimensional pixel blocks of a hyperspectral image sample and a classification result corresponding to the hyperspectral image sample; The deep learning network comprises two feature fusion-permutation attention mechanism models connected in sequence; The plurality of three-dimensional pixel blocks are input into the feature fusion model in the first feature fusion-permutation attention mechanism model to output a first fused feature map; The spatial dimension feature information and the channel dimension feature information of the first fused feature map are weighted by applying the permutation attention mechanism to obtain an initial calibrated feature map; The initial calibrated feature map is input into the feature fusion model in the second feature fusion-permutation attention mechanism model to output a second fused feature map; The initial calibrated feature map is input into the feature fusion model in the second feature fusion-permutation attention mechanism model to output a second fused feature map; The attention mechanism is applied to weight the feature information of the spatial dimension and the feature information of the channel dimension of the second fused feature map to obtain a calibrated feature map, specifically including: grouping the second fused feature map along the channel dimension to obtain a plurality of groups of sub-feature maps; respectively dividing each group of sub-feature maps into two parts along the channel dimension to obtain two branch feature maps; the two branch feature maps include a first branch feature map and a second branch feature map; the spatial dimension feature information of the first branch feature map is compressed along the channel dimension, and a gating mechanism and an activation function are applied to the compressed spatial dimension feature information to obtain a calibrated first branch feature map; group normalization is applied to the second branch feature map to obtain normalized spatial dimension feature information; a gating mechanism and an activation function are applied to the normalized spatial dimension feature information to obtain an attention weight map; the second branch feature map is multiplied with the corresponding attention weight map to obtain a calibrated second branch feature map; the calibrated first branch feature map and the calibrated second branch feature map are connected along the channel dimension to obtain a calibrated sub-feature map; each group of calibrated sub-feature maps is aggregated to obtain an aggregated feature map; a channel permutation operation is applied to the aggregated feature map to obtain a calibrated feature map.

6. An electronic device, comprising: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to make the electronic device execute the hyperspectral image classification method according to any one of claims 1-4.

7. A computer readable storage medium characterized by, The computer program is stored in the memory and is executed by the processor to realize the hyperspectral image classification method according to any one of claims 1-4.

Images