Hyperspectral image classification method and device based on spectral refinement, equipment and medium

Abstract

The application discloses a hyperspectral image classification method and device based on spectral refinement, equipment and medium, the method comprises the following steps: acquiring a pixel block of a hyperspectral image, extracting shallow features of different scales of the pixel block through multi-scale convolution operation to form a multi-scale feature map, processing the multi-scale feature map through a spectral refinement block, capturing discriminative spectral features of different semantics of the hyperspectral image to form a spectral refinement feature map, processing the spectral refinement feature map through a global perception block, capturing long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global perception feature map, and processing the global perception feature map through a pooling layer and a softmax layer to form a classification result of the hyperspectral image. The hyperspectral classification method effectively represents fine spectral correlation and global spectral-spatial features of the hyperspectral image.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, and in particular relates to a hyperspectral image classification method, apparatus, device and medium based on spectral refinement. Background Technology

[0002] Hyperspectral remote sensing is an imaging spectral remote sensing technology that organically combines imaging techniques with subdivided spectral techniques. Hyperspectral images contain hundreds or thousands of narrow and continuous spectral bands reflected from the observed scene, and are widely used in many applications such as agricultural assessment, urban development, military, and environmental management. Because different components absorb different amounts of light at different wavelengths, hyperspectral images will show a significant reflection of a particular defect at a specific wavelength. This allows hyperspectral images to reflect not only the external quality characteristics of a sample, such as size, shape, and defects, but also the differences in the sample's internal physical structure and chemical composition.

[0003] Hyperspectral image classification is one of the core tasks in hyperspectral image processing, aiming to assign a class label to each pixel. It is currently one of the most active research areas in academia. However, the high dimensionality, strong inter-band correlation, and spectral mixing characteristics of hyperspectral images pose significant challenges to hyperspectral image classification. Early hyperspectral image classification models were based on traditional machine learning methods, whose performance was limited by handcrafted features. Deep learning, with its powerful feature extraction capabilities, has significantly improved the performance of hyperspectral image classification, especially Convolutional Neural Networks (CNNs) and Transformer models. CNNs can extract deep features through a set of hierarchical filters and have been widely used for hyperspectral image classification. For example, one-dimensional CNNs (1D CNNs) and two-dimensional CNNs (2D CNNs) process the spectral and spatial information of hyperspectral images, respectively, while three-dimensional CNNs (3D CNNs) can jointly process spectral and spatial information for hyperspectral image classification. However, the representational ability of CNNs is limited by local receptive fields, making it difficult to capture global semantic information, while Transformer models can calculate the global correlation of features. Therefore, Transformer models have been applied to hyperspectral image classification and have achieved high classification accuracy. However, most existing works explore spectral features through 1×1 convolutions or simple attention mechanisms, failing to fully utilize the rich spectral information. Furthermore, the computational complexity of self-attention mechanisms is O(n^2). 2 The efficiency of Transformer in processing hyperspectral images is limited. Summary of the Invention

[0004] The purpose of this invention is to provide a hyperspectral classification method, apparatus, device, and medium based on spectral refinement. By using a spectral refinement block composed of a hierarchical block unit and a spectral self-attention unit, as well as a global perception block, the invention solves the technical problems of existing hyperspectral image classification methods, such as difficulty in capturing global semantic information and limited efficiency in processing hyperspectral images.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0006] This invention provides a hyperspectral image classification method based on spectral refinement, comprising:

[0007] Obtain pixel blocks from a hyperspectral image;

[0008] The shallow features of the pixel blocks at different scales are extracted by multi-scale convolution operations to form a multi-scale feature map;

[0009] The multi-scale feature map is processed by spectral thinning blocks to capture discriminative spectral features with different semantics in the hyperspectral image, so as to form a spectral thinning feature map.

[0010] The spectral refinement feature map is processed by a global sensing block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image in order to form a global sensing feature map.

[0011] The global perception feature map is processed by pooling layers and softmax layers to form the classification result of the hyperspectral image.

[0012] In one embodiment of the present invention, the pixel blocks for acquiring the hyperspectral image include:

[0013] Obtain pseudo-color images from a hyperspectral dataset;

[0014] Select one pixel from the pseudo-color image and form a cube with the surrounding pixels to form the pixel block.

[0015] In one embodiment of the present invention, the step of extracting shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map includes:

[0016] The pixel blocks are subjected to three-dimensional convolution operations with different kernel sizes to form multiple corresponding output feature maps;

[0017] The multiple corresponding output feature maps are concatenated along the channel dimension to form the multi-scale feature map.

[0018] In one embodiment of the present invention, the spectral thinning block includes a hierarchical block unit and a spectral self-attention unit. The process of processing the multi-scale feature map using the spectral thinning block to capture discriminative spectral features with different semantics in the hyperspectral image to form a spectral thinning feature map includes:

[0019] The multi-scale feature map is grouped by channel dimension using the hierarchical block unit to form multiple input feature maps;

[0020] The spectral self-attention unit processes the multiple input feature maps to form a spectral refinement feature map.

[0021] In one embodiment of the present invention, the step of performing channel-dimensional feature grouping on the multi-scale feature map through the hierarchical block unit to form multiple input feature maps includes:

[0022] The multi-scale feature map is divided into n groups of feature maps with the same channel dimension;

[0023] After performing a three-dimensional convolution operation on the first group of feature maps, it is divided into two subgroup feature maps with the same channel dimension. One subgroup feature map is used as the first input feature map of the spectral self-attention unit, and the other subgroup feature map is connected with the second group of feature maps to form the first combined feature map.

[0024] After performing a 3D convolution operation on the first combined feature map, it is further divided into two subgroup feature maps with the same channel dimension. One subgroup feature map serves as the second input feature map of the spectral self-attention unit, and the other subgroup feature map is concatenated with the third group feature map to form the second combined feature map; and so on.

[0025] After performing a three-dimensional convolution operation on the (n-2)th combined feature map, it is divided into two subgroup feature maps with the same channel dimension. One subgroup feature map is used as the (n-1)th input feature map of the spectral self-attention unit, and the other subgroup feature map is connected with the nth group feature map to form the (n-1)th combined feature map.

[0026] The (n-1)th combined feature map is subjected to a three-dimensional convolution operation and used as the nth input feature map of the spectral self-attention unit.

[0027] In one embodiment of the present invention, the step of processing the spectral refinement feature map through a global sensing block to capture the long-range spatial dependence and spectral adaptability of the hyperspectral image to form a global sensing feature map includes:

[0028] The spectral refinement feature map is subjected to a two-dimensional convolution operation with a kernel size of 1×1 to form a mapping feature map of the spectral refinement feature map;

[0029] The mapping feature map is processed using convolutional kernel attention units to form an attention feature map;

[0030] The mapping feature map and the attention feature map are multiplied element-wise and added to the mapping feature map. Then, a two-dimensional convolution operation with a kernel size of 1×1 is performed. The result is added to the spectral refinement feature map through residual connection and batch normalization is performed to form a global perception feature map.

[0031] In one embodiment of the present invention, the convolutional kernel attention unit includes a depthwise convolution, a depthwise dilated convolution, a ReLU activation function, and a two-dimensional convolution with a kernel size of 1×1.

[0032] Based on the same inventive concept, the present invention also provides a hyperspectral image classification device, comprising:

[0033] The data acquisition module is used to acquire pixel blocks of hyperspectral images;

[0034] The data processing module is used to extract shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map;

[0035] The spectral refinement module is used to process the multi-scale feature map through the spectral refinement block to capture the discriminative spectral features with different semantics of the hyperspectral image, so as to form a spectral refinement feature map;

[0036] The global perception module is used to process the spectral refinement feature map through the global perception block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global perception feature map.

[0037] The post-processing module is used to process the global perception feature map through pooling layers and softmax layers to form the classification result of the hyperspectral image.

[0038] Based on the same inventive concept, the present invention also provides an electronic device, including a processor and a memory;

[0039] The memory is used to store computer programs;

[0040] The processor is configured to execute, according to the computer program, any of the above-described hyperspectral image classification methods based on spectral refinement.

[0041] Based on the same inventive concept, the present invention also provides a computer-readable storage medium for storing a computer program for the hyperspectral image classification method based on spectral refinement described in any of the preceding claims.

[0042] This invention provides a hyperspectral image classification method based on spectral refinement, comprising: acquiring pixel blocks of a hyperspectral image; extracting shallow features of the pixel blocks at different scales through multi-scale convolution operations to form a multi-scale feature map; processing the multi-scale feature map through a spectral refinement block to capture discriminative spectral features of the hyperspectral image with different semantics to form a spectral refinement feature map; processing the spectral refinement feature map through a global perception block to capture the long-range spatial dependence and spectral adaptability of the hyperspectral image to form a global perception feature map; and processing the global perception feature map through a pooling layer and a softmax layer to form a classification result of the hyperspectral image. This hyperspectral image classification method based on spectral refinement effectively characterizes fine spectral correlations and global spectral-spatial features of the hyperspectral image. The designed spectral refinement block includes hierarchical block units and spectral self-attention units, which can capture more discriminative spectral features with different semantics. Furthermore, the global perception block is designed to capture the long-range correlations of spectral-spatial features while avoiding damage to the spatial structure. Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a hyperspectral image classification method based on spectral refinement provided for an exemplary embodiment of this application.

[0045] Figure 2 A system block diagram of a hyperspectral image classification method based on spectral refinement provided for an exemplary embodiment of this application.

[0046] Figure 3 This is a schematic diagram of the structure of the spectral refinement block in a hyperspectral image classification method based on spectral refinement, provided as an exemplary embodiment of this application.

[0047] Figure 4 The first test image of the hyperspectral image classification method based on spectral refinement provided in an exemplary embodiment of this application is a pseudo-color image.

[0048] Figure 5 The ground truth image of the first test image provided for a hyperspectral image classification method based on spectral refinement, as an exemplary embodiment of this application.

[0049] Figure 6 The image shows the classification result of a first test image of the hyperspectral image classification method based on spectral refinement provided as an exemplary embodiment of this application.

[0050] Figure 7 The second test image is a pseudo-color image of the hyperspectral image classification method based on spectral refinement provided as an exemplary embodiment of this application.

[0051] Figure 8 The second test image is a ground truth map of the hyperspectral image classification method based on spectral refinement provided as an exemplary embodiment of this application.

[0052] Figure 9 The image shows the second test image classification result of the hyperspectral image classification method based on spectral refinement provided as an exemplary embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] To address the technical problems of existing hyperspectral image classification methods, such as difficulty in capturing global semantic information and limited efficiency in processing hyperspectral images, this invention provides a hyperspectral image classification method based on spectral refinement. It employs three-dimensional convolution operations with different kernels to obtain multi-scale shallow features. A spectral refinement block, composed of hierarchical block units and spectral self-attention units, is designed to extract refined and effective spectral information. Simultaneously, a global perception block is designed, which utilizes convolutional kernel attention units with a computational complexity of O(n) to effectively capture the long-distance correlation of spectral-spatial features. The hyperspectral image classification method based on spectral refinement includes the following steps:

[0055] S100: Acquire pixel blocks of hyperspectral image;

[0056] S200: Extract shallow features of the pixel block at different scales through multi-scale convolution operations to form a multi-scale feature map;

[0057] S300: The multi-scale feature map is processed by spectral refinement blocks to capture discriminative spectral features with different semantics in the hyperspectral image, so as to form a spectral refinement feature map;

[0058] S400: The spectral refinement feature map is processed by a global sensing block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global sensing feature map.

[0059] S500: The global perception feature map is processed by pooling layers and softmax layers to form the classification result of the hyperspectral image.

[0060] The following will combine Figure 1 and Figure 2 The steps of the hyperspectral image classification method based on spectral refinement will be discussed in detail below.

[0061] First, step S100 is executed to obtain pixel blocks of the hyperspectral image.

[0062] In an exemplary embodiment of this application, the step of acquiring the pixel block of the hyperspectral image further includes the following steps:

[0063] S101: Obtain pseudo-color images from the hyperspectral dataset;

[0064] S102: Select a pixel in the pseudo-color image and form a cube with the surrounding pixels to form the pixel block.

[0065] For details, please refer to Figure 2 As shown, pseudo-color images from the hyperspectral dataset are obtained as input to the target hyperspectral image classification model. In this embodiment, the hyperspectral dataset includes the Indian Pines dataset and the University of Pavia dataset. Please refer to [link to relevant documentation]. Figure 4 and Figure 7 As shown, Figure 4 and Figure 7 The images shown are pseudo-color images of the first test image from the Indian Pines dataset and the second test image from the University of Pavia dataset. Of course, other hyperspectral datasets can also be used in other embodiments. Image I∈R H×W×B The image I is a pseudo-color image of the hyperspectral dataset, where H, W, and B correspond to height, width, and channel, respectively. A cube X∈R is formed by selecting one pixel from the image I and its surrounding pixels. P×P×B As input to the hyperspectral classification model, P represents the input pixel block size. In this embodiment, the pixel block is 9×9 in size. Of course, in other embodiments, the pixel block size can be customized according to actual application requirements.

[0066] Next, step S200 is executed, in which shallow features of different scales of the pixel block are extracted through multi-scale convolution operation to form a multi-scale feature map.

[0067] In an exemplary embodiment of this application, the step of extracting shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map further includes the following steps:

[0068] S201: Perform three-dimensional convolution operations on the pixel blocks with different kernel sizes to form multiple corresponding output feature maps;

[0069] S202: The multiple corresponding output feature maps are concatenated along the channel dimension to form the multi-scale feature map.

[0070] Please see Figure 2 As shown in an exemplary embodiment of this application, to extract shallow features at different scales, multi-scale convolution operations are performed on the pixel block X. Specifically, the pixel block X is upscaled and then input into three-dimensional convolution operations with kernel sizes of 1×1×7, 3×3×7, and 5×5×7, respectively, to obtain three output feature maps of size P×P×S,24, where S represents the depth of the feature map after the corresponding convolution operation. It should be noted that the padding of the upscaled pixel block X is different when performing three-dimensional convolution operations with kernels of different sizes to ensure that the output and input sizes are consistent. Then, the three feature maps of size P×P×S,24 are concatenated along the channel dimension to form a multi-scale feature map x∈R. P×P×S×72 .

[0071] Next, step S300 is executed, in which the multi-scale feature map is processed by the spectral refinement block to capture the discriminative spectral features of the hyperspectral image with different semantics, so as to form a spectral refinement feature map.

[0072] In an exemplary embodiment of this application, the spectral refinement block includes a hierarchical block unit and a spectral self-attention unit. The step of processing the multi-scale feature map through the spectral refinement block to capture discriminative spectral features with different semantics in the hyperspectral image to form a spectral refinement feature map further includes the following steps:

[0073] S310: The multi-scale feature map is grouped by channel dimension through the hierarchical block unit to form multiple input feature maps;

[0074] S320: The multiple input feature maps are processed by the spectral self-attention unit to form a spectral refinement feature map.

[0075] In an exemplary embodiment of this application, the step of performing channel-dimensional feature grouping on the multi-scale feature map through the hierarchical block unit to form multiple input feature maps further includes the following steps:

[0076] S311: Divide the multi-scale feature map into n groups of feature maps with the same channel dimension;

[0077] S312: After performing a three-dimensional convolution operation on the first group of feature maps, it is divided into two subgroup feature maps with the same channel dimension. One subgroup feature map is used as the first input feature map of the spectral self-attention unit, and the other subgroup feature map is connected with the second group of feature maps to form a first combined feature map.

[0078] S313: After performing a 3D convolution operation on the first combined feature map, it is further divided into two sub-group feature maps with the same channel dimension. One sub-group feature map serves as the second input feature map of the spectral self-attention unit, and the other sub-group feature map is concatenated with the third group feature map to form the second combined feature map; and so on.

[0079] S314: After performing a three-dimensional convolution operation on the (n-2)th combined feature map, it is divided into two sub-group feature maps with the same channel dimension. One sub-group feature map is used as the (n-1)th input feature map of the spectral self-attention unit, and the other sub-group feature map is connected with the nth group feature map to form the (n-1)th combined feature map.

[0080] S315: The (n-1)th combined feature map is subjected to a three-dimensional convolution operation and used as the nth input feature map of the spectral self-attention unit.

[0081] Specifically, because hyperspectral images have similar adjacent spectral information, targets can be precisely identified by capturing subtle spectral differences. In this embodiment, the spectral refinement block includes a hierarchical segmentation unit and a spectral self-attention unit, through which more refined spectral features can be extracted. Please refer to [link to relevant documentation]. Figure 3 As shown, Figure 3 A schematic diagram of the structure of the spectral refinement block is shown. In this embodiment, firstly, a three-branch structure of the multi-scale feature map is constructed, and the multi-scale feature map x∈R is... P×P×S×72 The feature maps are divided into three groups with the same channel dimension, denoted as x. i Let i = 1, 2, 3. Next, perform a 3D convolution operation with a kernel size of 1×1×7.24 on the first group of feature maps of the first branch to obtain feature map y1. Then, further divide feature map y1 into two subgroups of feature maps y1 with the same channel dimension. 1,1 and y 1,2 , the first subgroup feature map y1,1 The data is sent to the spectral self-attention unit, while another subgroup of feature maps y... 1,2 Connect it to x2 in the next branch. Then, connect y... 1,2 The feature map concatenated with x2 is subjected to another 3D convolution operation with a kernel size of 1×1×7.36 to obtain feature map y2. Feature map y2 is further divided into two subgroup feature maps with the same channel dimension, denoted as y2. 2,1 and y 2,2 The first subgroup feature map y 2,1 The data is sent to the spectral self-attention unit, while another subgroup feature map y... 2,2 Connect it to x3 in the next branch. Finally, connect y... 2,2 The feature map concatenated with x3 is subjected to another 3D convolution operation with a kernel size of 1×1×7.42 to obtain feature map y3, which is then sent to the spectral self-attention unit. The overall process of the hierarchical block unit can be represented by the following formula:

[0082]

[0083] Where, Φ i This represents a 3D convolution operation with a kernel size of 1×1×7 performed on the i-th branch, y i y represents the output feature map after the i-th branch performs a 3D convolution operation with a kernel size of 1×1×7. i,1 and y i,2 Indicates from y i The two subgroups, y i,1 y represents the feature map fed into the spectral self-attention unit after the i-th branch is grouped. i,2 ⊕ represents the feature map sent to the next branch after the i-th branch is grouped, and ⊕ represents the connection operation on the channel dimension.

[0084] It should be noted that hierarchical connections allow the current group to utilize the feature information of the previous group. Therefore, rich information from shallow to deep layers can be combined through hierarchical cross-branch residual connections, avoiding information loss and channel splitting in deep convolutional layers. Furthermore, in this embodiment, a three-branch structure is constructed for the multi-scale feature map. Of course, in other embodiments, other numbers of branch structures for the multi-scale feature map can also be constructed.

[0085] In an exemplary embodiment of this application, the spectral self-attention unit enhances the distinguishability of spectral bands, taking into account the noise and redundancy of multiple bands. The spectral self-attention unit captures more complex dependencies between spectra by encoding each band of the hyperspectral image and adaptively recalibrates spectral information in different dimensions.

[0086] For details, please continue reading. Figure 3 As shown, firstly, the input feature map y∈R is... P×P×S,C A 3D convolution operation with a kernel size of 1×1×S,C is performed, where C represents the channel dimension, and the kernel is reshaped into y'∈R. P×P×C Then, perform three 2D convolution operations with a kernel size of 1×1 on y' (keeping the channel dimension unchanged) and flatten it into one dimension to obtain the query tensor. Key tensor Sum tensor By transposing the query tensor Q and the key tensor K T The dot product calculation results are used to calculate the spectral correlation kernel A∈R using the softmax function. C×C Finally, the attention feature map is obtained by multiplying the spectral correlation kernel A and the value tensor V. The overall process of the spectral self-attention unit can be represented by the following formula:

[0087]

[0088] Here, d is a learnable scaling parameter that controls the size of the dot product between the query tensor Q and the key tensor K before applying the softmax function. The attention graph Out is then reshaped into Out'∈R. P×P×C The desired output feature map is obtained by summing the residuals with y'. The spectral self-attention unit weights the spectral features of the hyperspectral image, reducing the impact of redundant spectra on classification. Feature map y 1,1 y 2,1 And y3 are sent to the spectral self-attention unit to generate and Then the outputs of the three spectral self-attention units Connecting i = 1, 2, 3, we obtain the spectral refinement feature map Y ∈ R. P×P×72 It should be noted that the spectral refinement feature map contains optimized spectral information.

[0089] Then, step S400 is executed to process the spectral refinement feature map through a global sensing block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global sensing feature map.

[0090] In an exemplary embodiment of this application, the step of processing the spectral refinement feature map through a global sensing block to capture the long-range spatial dependence and spectral adaptability of the hyperspectral image to form a global sensing feature map further includes:

[0091] S401: Perform a two-dimensional convolution operation with a kernel size of 1×1 on the spectral refinement feature map to form a mapping feature map of the spectral refinement feature map;

[0092] S402: The mapping feature map is processed using a convolutional kernel attention unit to form an attention feature map;

[0093] S403: Element-wise multiply the mapping feature map and the attention feature map and add them to the mapping feature map, then perform a two-dimensional convolution operation with a kernel size of 1×1, add it to the spectral refinement feature map through residual connection, and perform batch normalization to form a global perception feature map.

[0094] In one exemplary embodiment of this application, please refer to Figure 2 As shown, the convolutional kernel attention unit includes depthwise convolution, depthwise dilated convolution, ReLU activation function, and two-dimensional convolution with a kernel size of 1×1.

[0095] Specifically, the spectral refinement feature map Y∈R P×P×72 The image is sent to the global sensing block, where a 1×1 two-dimensional convolution operation is first performed to obtain the mapped feature map E∈R of the spectral refinement feature map. P×P×36 Then, the mapped feature map E is processed by a convolutional kernel attention unit, and the global perceptual block integrates the advantages of self-attention and large kernel convolution. Since deep convolution (DW-Conv) can utilize the local contextual information of hyperspectral images, and deep dilated convolution (DW-D-Conv) can capture the long-term dependencies of hyperspectral images, in this embodiment, the 9×9 large kernel convolution in the convolutional kernel attention unit is divided into a 3×3 kernel deep convolution (DW-Conv), a 3×3 kernel deep dilated convolution (DW-D-Conv) with a dilation factor of 2, and a 1×1 kernel two-dimensional convolution. Through the decomposition of the 9×9 large kernel convolution, the deep convolution (DW-Conv) and the deep dilated convolution (DW-D-Conv) can perceive global spatial information, and the 1×1 kernel two-dimensional convolution can adaptively learn the spectral channel. Meanwhile, in this embodiment, a ReLU activation function is added between the depthwise convolution (DW-Conv) and the depthwise dilated convolution (DW-D-Conv). This decomposition of the large kernel convolution significantly reduces computational workload and generates an attention feature map T∈R. P×P×36 Then, the attention feature map T is multiplied element-wise with the mapping feature map E to obtain the feature map O∈R. P×P×36 The above process can be represented by the following formula:

[0096] T = Conv 1×1(DW-D-Conv(ReLU(DW-Conv(E))))

[0097]

[0098] Where DW-Conv represents depthwise convolution, DW-D-Conv represents depthwise dilated convolution, and Conv... 1×1 Represents a 1×1 convolution, ReLU represents the ReLU activation function, and T represents the attention feature map. This represents element-wise multiplication. Then, feature map O is added to the mapped feature map E, followed by a 1×1 two-dimensional convolution operation to obtain a new feature representation. Finally, the new feature representation is added to the spectral refinement feature map Y through residual connections, and batch normalization is performed to obtain the globally perceived feature map Y'∈R. P×P×72 It should be noted that the global sensing block can effectively capture long-distance spatial context structure information and adaptively learn the spectrum channel.

[0099] Finally, step S500 is executed, where the global sensing feature map is processed through a pooling layer and a softmax layer to form the classification result of the hyperspectral image. It should be noted that the softmax function is a normalized exponential function.

[0100] In summary, this invention provides a hyperspectral image classification method based on spectral refinement. The method includes acquiring pixel blocks of a hyperspectral image, extracting shallow features of the pixel blocks at different scales through multi-scale convolution operations to form a multi-scale feature map, processing the multi-scale feature map using a spectral refinement block to capture discriminative spectral features with different semantics from the hyperspectral image, forming a spectral refinement feature map, processing the spectral refinement feature map using a global perception block to capture the long-range spatial dependence and spectral adaptability of the hyperspectral image, forming a global perception feature map, and processing the global perception feature map using pooling and softmax layers to form the classification result of the hyperspectral image. This hyperspectral image classification method based on spectral refinement effectively characterizes fine-grained spectral correlations and global spectral-spatial features of the hyperspectral image. The designed spectral refinement block includes hierarchical block units and spectral self-attention units, which can capture more discriminative spectral features with different semantics. Furthermore, the global perception block is designed to capture the long-range correlations of spectral-spatial features while avoiding damage to the spatial structure. Figures 4-6 The pseudo-color image, ground truth image, and classification result image of the first test image from the Indian Pines dataset are shown respectively. Figures 7-9The images show the pseudo-color image, ground truth image, and classification result image of the second test image from the University of Pavia dataset. Please refer to [link / reference]. Figures 4-9 As shown, after the first test image and the second test image are processed by the hyperspectral image classification model, the resulting first classification image and second classification image have very few misclassified pixels and are very close to the ground real map of the corresponding hyperspectral dataset, thus realizing the classification of objects on the hyperspectral image based on the hyperspectral image information.

[0101] Based on the same inventive concept, another embodiment of the present invention provides a hyperspectral image classification device based on spectral refinement, comprising:

[0102] The data acquisition module is used to acquire pixel blocks of hyperspectral images;

[0103] The data processing module is used to extract shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map;

[0104] The spectral refinement module is used to process the multi-scale feature map through the spectral refinement block to capture the discriminative spectral features with different semantics of the hyperspectral image, so as to form a spectral refinement feature map;

[0105] The global perception module is used to process the spectral refinement feature map through the global perception block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global perception feature map.

[0106] The post-processing module is used to process the global perception feature map through pooling layers and softmax layers to form the classification result of the hyperspectral image.

[0107] Since the hyperspectral image classification device based on spectral refinement provided in this embodiment belongs to the same inventive concept as the hyperspectral image classification method based on spectral refinement provided in any of the above embodiments, it has at least the same beneficial effects as them, and will not be described in detail here.

[0108] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the hyperspectral image classification method based on spectral refinement described in any of the above embodiments according to the computer program.

[0109] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium for storing a computer program for the hyperspectral image classification method based on spectral refinement described in any of the above embodiments.

[0110] The above description is merely a preferred embodiment of this application and an explanation of the technical principles used. Those skilled in the art should understand that the scope involved in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the inventive concept. For example, technical solutions formed by replacing the above-mentioned features with technical features with similar functions disclosed in this application (but not limited to) each other.

[0111] Apart from the technical features described in the specification, the other technical features are known to those skilled in the art. To highlight the innovative features of this invention, the other technical features will not be described in detail here.

Claims

1. A hyperspectral image classification method based on spectral refinement, characterized in that, include: Obtain pixel blocks from a hyperspectral image; The shallow features of the pixel blocks at different scales are extracted by multi-scale convolution operations to form a multi-scale feature map; The multi-scale feature map is processed by spectral thinning blocks to capture discriminative spectral features with different semantics in the hyperspectral image, so as to form a spectral thinning feature map. The spectral refinement feature map is processed by a global sensing block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image in order to form a global sensing feature map. The global perception feature map is processed by pooling layers and softmax layers to form the classification result of the hyperspectral image; The spectral refinement block includes a hierarchical block unit and a spectral self-attention unit; The process of processing the multi-scale feature map through spectral refinement blocks to capture discriminative spectral features with different semantics in the hyperspectral image to form a spectral refinement feature map includes: The multi-scale feature map is grouped by channel dimension through the hierarchical block unit to form multiple input feature maps; The spectral self-attention unit processes the multiple input feature maps to capture the dependencies between spectral bands and form a refined spectral feature map.

2. The hyperspectral image classification method based on spectral refinement according to claim 1, characterized in that, The pixel blocks used to acquire the hyperspectral image include: Obtain pseudo-color images from a hyperspectral dataset; Select one pixel from the pseudo-color image and form a cube with the surrounding pixels to form the pixel block.

3. The hyperspectral image classification method based on spectral refinement according to claim 1, characterized in that, The step of extracting shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map includes: The pixel blocks are subjected to three-dimensional convolution operations with different kernel sizes to form multiple corresponding output feature maps; The multiple corresponding output feature maps are concatenated along the channel dimension to form the multi-scale feature map.

4. The hyperspectral image classification method based on spectral refinement according to claim 1, characterized in that, The step of grouping the multi-scale feature maps by channel dimension using the hierarchical block unit to form multiple input feature maps includes: The multi-scale feature map is divided into n groups of feature maps with the same channel dimension; After performing a three-dimensional convolution operation on the first group of feature maps, it is divided into two subgroup feature maps with the same channel dimension. One subgroup feature map is used as the first input feature map of the spectral self-attention unit, and the other subgroup feature map is connected with the second group of feature maps to form the first combined feature map. After performing a 3D convolution operation on the first combined feature map, it is further divided into two subgroup feature maps with the same channel dimension. One subgroup feature map serves as the second input feature map of the spectral self-attention unit, and the other subgroup feature map is concatenated with the third group feature map to form the second combined feature map; and so on. After performing a three-dimensional convolution operation on the (n-2)th combined feature map, it is divided into two subgroup feature maps with the same channel dimension. One subgroup feature map is used as the (n-1)th input feature map of the spectral self-attention unit, and the other subgroup feature map is connected with the nth group feature map to form the (n-1)th combined feature map. The (n-1)th combined feature map is subjected to a three-dimensional convolution operation and used as the nth input feature map of the spectral self-attention unit.

5. The hyperspectral image classification method based on spectral refinement according to claim 1, characterized in that, The process of processing the spectral refinement feature map through a global sensing block to capture the long-range spatial dependence and spectral adaptability of the hyperspectral image to form a global sensing feature map includes: The spectral refinement feature map is subjected to a two-dimensional convolution operation with a kernel size of 1×1 to form a mapping feature map of the spectral refinement feature map; The mapping feature map is processed using convolutional kernel attention units to form an attention feature map; The mapping feature map and the attention feature map are multiplied element-wise and added to the mapping feature map. Then, a two-dimensional convolution operation with a kernel size of 1×1 is performed. The result is added to the spectral refinement feature map through residual connection and batch normalization is performed to form a global perception feature map.

6. The hyperspectral image classification method based on spectral refinement according to claim 5, characterized in that, The convolutional kernel attention unit includes depthwise convolution, depthwise dilated convolution, ReLU activation function, and two-dimensional convolution with a kernel size of 1×1.

7. A hyperspectral image classification device based on spectral refinement, characterized in that, include: The data acquisition module is used to acquire pixel blocks of hyperspectral images; The data processing module is used to extract shallow features of different scales of the pixel block through multi-scale convolution operations to form a multi-scale feature map; A spectral refinement module is used to process the multi-scale feature map through spectral refinement blocks to capture discriminative spectral features with different semantics in the hyperspectral image, thereby forming a spectral refinement feature map. The process of processing the multi-scale feature map through spectral refinement blocks to capture discriminative spectral features with different semantics in the hyperspectral image, thereby forming a spectral refinement feature map, includes: grouping the multi-scale feature map into channel-dimensional features using a hierarchical block unit to form multiple input feature maps; and processing the multiple input feature maps using a spectral self-attention unit to capture the dependencies between spectral bands and form a spectral refinement feature map. The global perception module is used to process the spectral refinement feature map through the global perception block to capture the long-distance spatial dependence and spectral adaptability of the hyperspectral image to form a global perception feature map. The post-processing module is used to process the global perception feature map through pooling layers and softmax layers to form the classification result of the hyperspectral image.

8. An electronic device, characterized in that, Including processor and memory; The memory is used to store computer programs; The processor is configured to execute the hyperspectral image classification method based on spectral refinement as described in any one of claims 1-6 according to the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for the hyperspectral image classification method based on spectral refinement as described in any one of claims 1-6.

Images