Hyperspectral image classification method, device and storage medium

Abstract

The application discloses a hyperspectral image classification method and device and a storage medium, and relates to the technical field of remote sensing image processing and artificial intelligence. The method comprises the following steps: processing input feature blocks by using a gated large kernel convolution selection module, adopting double-branch large kernel convolution decomposition to expand the receptive field, multiplying the input after gate modulation and attention selection fusion, and outputting multi-scale spatial fusion features; processing the multi-scale spatial fusion features by using a multi-scale wavelet convolution module, and outputting frequency domain enhancement features; processing the frequency domain enhancement features by using a prior attention module, sequentially passing through spatial prior channel attention and channel prior spatial attention, combining layer normalization, a residual and a feedforward network, and outputting prior enhancement features; and finally outputting the ground object category classification result through global average pooling and a linear layer. The application can expand the receptive field, utilize frequency domain information and retain prior information, and improve the classification accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and artificial intelligence technology, specifically relating to a frequency domain enhanced hyperspectral image classification method based on deep learning, and in particular a hyperspectral image classification method that integrates gated big kernel selection and prior attention mechanism, which can be applied to fields such as agricultural resource survey, ecological environment monitoring, mineral exploration, urban planning and disaster assessment. Background Technology

[0002] Hyperspectral images are acquired by imaging dozens to hundreds of consecutive narrow bands, combining high spectral resolution with spatial structure information. The differences in spectral response and spatial distribution among different land features give hyperspectral images a significant advantage in fine-grained land feature identification and classification tasks. The goal of hyperspectral image classification is to assign accurate land feature category labels to each pixel, a task that requires simultaneously mining spectral features and spatial contextual information.

[0003] However, hyperspectral data presents challenges such as numerous bands, high dimensionality, complex category distribution, and high annotation costs. Traditional methods (such as support vector machines and nearest neighbor classifiers) rely on manually generated spectral features, making it difficult to model nonlinear relationships and limiting their generalization ability. In recent years, methods based on convolutional neural networks have become mainstream by jointly modeling spectral and spatial features through two-dimensional / three-dimensional convolution. However, a comprehensive analysis of existing deep learning methods reveals the following three key and interdependent technical shortcomings: First, the receptive field is limited, resulting in insufficient ability to model large-scale spatial contexts. Conventional convolutional kernels have limited sizes, leading to a small effective receptive field. In hyperspectral images, the spatial distribution of ground features is complex and varies in scale. Relying solely on local receptive fields makes it difficult to capture long-distance dependencies, resulting in unstable classification results, especially in areas with large-scale homogeneous features or fragmented ground features.

[0004] Second, the lack of utilization of frequency domain information makes it difficult to coordinate structural and detailed information. Existing methods are mostly limited to spatial domain feature extraction, neglecting the rich information in the frequency domain of hyperspectral images. In the frequency domain, low-frequency components reflect the overall structure, while high-frequency components represent details such as edges and textures. Ignoring frequency domain features weakens the model's ability to perceive differences in ground features and structural changes, resulting in a significant drop in classification performance in complex backgrounds or scenes with blurred class boundaries.

[0005] Third, prior information is lost in the attention mechanism, limiting the weight discrimination ability. Although attention mechanisms are widely used to enhance feature representation, existing channel or spatial attention often relies on compression operations such as global pooling, resulting in the loss of prior information on spatial structure or channel distribution in the original features. This makes the generated attention weights lack specificity and difficult to adapt to the characteristics of strong heterogeneity and multi-scale features in hyperspectral images.

[0006] The three problems mentioned above are often addressed in isolation in existing methods, lacking a collaborative solution strategy within a unified framework. Therefore, there is an urgent need to propose a classification method that can simultaneously expand the spatial receptive field, introduce frequency domain features, and preserve prior information, in order to significantly improve the accuracy and robustness of hyperspectral image classification. Summary of the Invention

[0007] To address the aforementioned technical problems in the prior art, the present invention aims to provide a hyperspectral image classification method, device, and storage medium to solve the problems of limited receptive field, insufficient utilization of frequency domain information, and loss of attention prior information in existing methods. By employing gated large kernel selection, multi-scale wavelet frequency domain enhancement, and prior attention mechanisms, the invention achieves accurate classification through large-scale spatial context modeling, frequency domain feature enhancement, and prior guidance, thereby improving the accuracy and robustness of hyperspectral image classification.

[0008] This invention solves the above-mentioned technical problems through the following technical solution: a hyperspectral image classification method, comprising: The hyperspectral image is dimensionality reduced, and a spatial window is extracted centered on each pixel to be classified as the input feature block. The input feature block is processed using a gated large kernel convolution selection module. The gated large kernel convolution selection module uses two large kernel convolution branches with different dilation rates. Each branch expands the receptive field through the decomposition and combination of depthwise convolution, depthwise dilated convolution, and point convolution. The gated signal is used to perform element-wise multiplication modulation with the outputs of the two branches respectively. Multi-scale spatial features are adaptively fused through an attention selection mechanism and then multiplied element-wise with the input feature block to output multi-scale spatial fused features. Multi-scale spatial fusion features are processed using a multi-scale wavelet convolution module. The multi-scale wavelet convolution module decomposes frequency domain sub-bands through wavelet transform, extracts frequency domain features for each sub-band using three convolution branches, and adds the reconstructed features after inverse wavelet transform to the residual features of the multi-scale spatial fusion features after deep convolution processing, and outputs frequency domain enhanced features. The frequency domain enhancement features are processed using a prior attention module. The prior attention module includes a spatial prior channel attention submodule and a channel prior spatial attention submodule, and employs layer normalization, residual connections, and a feedforward network. Specifically, the spatial prior channel attention submodule generates dynamic spatial weights using the relative positions of pixels to constrain channel attention, and the channel prior spatial attention submodule retains channel prior information through channel duplication, rearrangement, and group convolution to generate spatial attention and output the prior enhancement features. After global average pooling of the prior enhancement features, the land cover classification results are output through a linear layer.

[0009] To address the limitation of existing methods in modeling large-scale spatial context information due to convolution kernel size constraints, this invention constructs a gated large-kernel convolution selection module. It employs two large-kernel convolution branches with different dilation rates. Each branch decomposes the large-kernel convolution into a combination of depthwise convolution, depthwise dilated convolution, and pointwise convolution, significantly expanding the network's receptive field while effectively controlling computational load. Simultaneously, a gating signal is used to perform element-wise multiplication modulation with the outputs of the two branches, and an attention selection mechanism adaptively fuses multi-scale spatial features before multiplying them element-wise with the input feature block. This mechanism enables the model to dynamically adjust its receptive field based on changes in the scale of ground features in hyperspectral images, fully capturing long-distance spatial dependencies. This effectively improves the classification stability of large-scale homogeneous or fragmented ground feature interleaving areas, enhancing the accuracy and robustness of the classification results.

[0010] To address the limitations of existing methods, which are mainly confined to spatial domain feature extraction and neglect frequency domain information in hyperspectral images, this invention introduces a multi-scale wavelet convolution module. This module decomposes spatial features into one low-frequency sub-band and three high-frequency sub-bands using wavelet transform, corresponding to the overall structural information and detailed information such as edges and textures of ground features, respectively. Then, it extracts frequency domain features for each sub-band using three convolutional branches to capture frequency domain information from different directions. The enhanced features of the four sub-bands are reconstructed into spatial domain features using inverse wavelet transform, and then added to the residual features obtained after deep convolution of the multi-scale spatial fusion features. This frequency domain enhancement mechanism can explicitly distinguish and collaboratively model low-frequency and high-frequency information, significantly improving the model's ability to perceive differences in ground feature edges, textures, and structures, and exhibiting superior discrimination performance in complex backgrounds or scenes with blurred category boundaries.

[0011] To address the problem that traditional attention mechanisms rely on global compression operations during weight generation, which can easily lead to the loss of prior information about spatial structure or channel distribution, this invention designs a prior attention module. This module sequentially includes a spatial prior channel attention submodule and a channel prior spatial attention submodule, employing layer normalization, residual connections, and a feedforward network. Specifically, the spatial prior channel attention submodule generates dynamic spatial weights using the relative positions of pixels to constrain channel attention, avoiding excessive compression of the spatial dimension. The channel prior spatial attention submodule retains prior channel information through channel duplication, rearrangement, and group convolution to generate spatial attention, avoiding information loss in the channel dimension. This mechanism achieves refined attention allocation with bidirectional spatial and channel constraints, enhancing the adaptability and discriminative ability of feature representation, and effectively addressing the characteristics of strong heterogeneity and the coexistence of multi-scale features in hyperspectral images.

[0012] This invention sequentially connects a gated large-kernel convolution selection module, a multi-scale wavelet convolution module, and a prior attention module to form a complete end-to-end hyperspectral image classification model. Through the synergistic effect of these three modules, this model simultaneously addresses the shortcomings of existing technologies in large-scale spatial context modeling, frequency domain information utilization, and prior attention. Comparative experiments on multiple publicly available hyperspectral image datasets demonstrate that the method of this invention significantly outperforms existing mainstream classification methods in terms of overall classification accuracy, average classification accuracy, and Kappa coefficient. It is particularly suitable for scenarios with complex ground cover distribution, subtle differences between categories, and limited training samples, fully demonstrating the advancement, effectiveness, and robustness of this invention in hyperspectral image classification tasks.

[0013] Furthermore, the dimensionality reduction process employs principal component analysis.

[0014] This invention uses principal component analysis (PCA) to reduce the dimensionality of the original hyperspectral image. While preserving the main spectral information of the hyperspectral image, it can compress high-dimensional band data to a lower dimension, significantly reducing channel redundancy and the computational load of subsequent modules. At the same time, PCA can effectively remove the correlation between bands, reduce noise interference, and provide a more compact and robust input feature representation for subsequent spatial-spectral feature extraction.

[0015] Furthermore, the gated large kernel convolution selection module includes: Two large kernel convolutional branches with different dilation rates, each branch consisting of sequentially connected depthwise convolution, depthwise dilated convolution and pointwise convolution, are used to extract features from the input feature block and output the first branch features and the second branch features respectively. The gated convolution branch, consisting of a depthwise convolution, is used to convolve the input feature block and output a gated feature map. The first modulation unit is used to multiply the gated feature map element by element with the first branch feature to output the first modulation feature; The second modulation unit is used to multiply the gated feature map element by element with the second branch feature to output the second modulation feature; The attention selection fusion unit is used to concatenate the first modulation feature and the second modulation feature along the channel dimension, and then sequentially pass them through global average pooling, softmax activation and channel segmentation to generate a first selection weight and a second selection weight. The first selection weight is then weighted with the first modulation feature channel by channel, and the second selection weight is then weighted with the second modulation feature channel by channel and summed to obtain a fused feature map. The output unit is used to multiply the fused feature map element-wise with the input feature block to output multi-scale spatial fused features.

[0016] This invention defines the specific internal structure of the gated large kernel convolution selection module, including two large kernel convolution branches with different dilation rates, a gated convolution branch, a first / second modulation unit, an attention selection fusion unit, and an output unit. Each large kernel convolution branch employs a decomposition structure of depthwise convolution, depthwise dilated convolution, and pointwise convolution, expanding the receptive field while maintaining low computational cost. The gated convolution branch generates a gated feature map, and the modulation unit modulates the outputs of the two branches element-wise, enabling the network to dynamically adjust the response intensity of different branches based on input features. The attention selection fusion unit concatenates the modulated features, then performs pooling and Softmax to generate selection weights, and performs channel-wise weighted summation of the two modulated features to achieve adaptive fusion of the two-branch features. The output unit multiplies the fused features element-wise with the input feature block to form element-wise weighted enhancement. This structure collectively ensures that the module achieves adaptive modeling of a large spatial context with low computational cost.

[0017] Furthermore, the multi-scale wavelet convolution module includes: The wavelet transform unit is used to perform wavelet transform on the multi-scale spatial fusion features, decomposing them into one low-frequency sub-band and three high-frequency sub-bands. The multi-scale frequency domain feature extraction unit is used to extract frequency domain features for each sub-band using three convolutional branches. Two of the branches are composed of vertical convolution and horizontal convolution in series, and the other branch is a point convolution. The outputs of the three branches are added together to obtain the enhanced sub-band features of that sub-band. The inverse wavelet transform unit is used to perform inverse wavelet transform on all enhanced subband features and reconstruct them into spatial domain features. The residual path unit consists of a deep convolutional layer, which is used to perform deep convolution on the multi-scale spatial fusion features and output residual features. The addition unit is used to add the reconstructed spatial domain features to the residual features and output the frequency domain enhancement features.

[0018] This invention defines the specific internal structure of a multi-scale wavelet convolution module, including a wavelet transform unit, a multi-scale frequency domain feature extraction unit, an inverse wavelet transform unit, a residual path unit, and an addition unit. The wavelet transform unit explicitly decomposes spatial features into one low-frequency sub-band and three high-frequency sub-bands, corresponding to the overall structure and edges and texture details in different directions, respectively. The multi-scale frequency domain feature extraction unit uses three convolutional branches for each sub-band, two of which are composed of cascaded vertical and horizontal convolutions to capture anisotropic features, and the other branch is a point convolution to preserve the original feature expression. The outputs of the three branches are added to achieve multi-scale, multi-directional feature fusion for the same frequency component. The inverse wavelet transform unit reconstructs the enhanced frequency domain features back to the spatial domain. The residual path unit adds the original input, processed by depth convolution, to the reconstructed features, ensuring lossless transmission of frequency domain enhancement information. This structure achieves explicit collaborative modeling of low-frequency and high-frequency information, significantly improving the perception of differences in ground object edges, textures, and structures.

[0019] Furthermore, the vertical convolution is a 1×k convolution, and the horizontal convolution is a k×1 convolution, where k is a preset convolution kernel size.

[0020] This invention decomposes the standard k×k convolution into a concatenation of two one-dimensional convolutions, reducing the number of parameters from k to k while maintaining the same receptive field (k×k). 2 Reducing the computational complexity to 2k significantly lowers the computational complexity. At the same time, this decomposition method enables the network to extract features in the vertical and horizontal directions separately, better adapting to the anisotropic spatial distribution characteristics of ground objects in hyperspectral images, and further enhancing the efficiency and discrimination ability of frequency domain feature extraction.

[0021] Furthermore, the prior attention module includes: The first normalization unit is used to perform layer normalization on the frequency domain enhancement features; The spatial prior channel attention submodule is used to receive the output of the first normalization unit, generate dynamic spatial weights using the relative positions of pixels to constrain channel attention, and output the first attention feature. The channel prior spatial attention submodule is used to receive the first attention feature, retain channel prior information through channel copying, rearrangement and group convolution to generate spatial attention, and output the second attention feature; The first residual connection unit is used to add the second attention feature to the frequency domain enhancement feature and output the first residual feature; The second normalization unit is used to perform layer normalization on the first residual feature; A feedforward network is used to perform nonlinear transformation and feature mapping on the output of the second normalization unit; The second residual connection unit is used to add the output of the feedforward network to the first residual feature and output the prior enhancement feature.

[0022] This invention defines the overall architecture of the prior attention module, including a first normalization unit, a spatial prior channel attention submodule, a channel prior spatial attention submodule, a first residual connection unit, a second normalization unit, a feedforward network, and a second residual connection unit. This architecture connects the modules in the following sequence: "layer normalization → spatial prior channel attention → channel prior spatial attention → residual plus original input → layer normalization → feedforward network → residual," achieving the gradual injection of bidirectional priors in both spatial and channel dimensions, avoiding information loss caused by simultaneously compressing two dimensions. Layer normalization stabilizes the training process, residual connections alleviate the gradient vanishing problem in deep networks, and the feedforward network enhances nonlinear expressive power. This architecture allows the generation of attention weights to be guided by both spatial structure and channel distribution, significantly improving the precision and adaptability of attention allocation.

[0023] Furthermore, the spatial prior channel attention submodule includes: The channel attention calculation unit is used to perform global average pooling and global max pooling on the input features respectively. The two pooling results are input into the multilayer perceptron. The outputs of the two multilayer perceptrons are added together and then activated by Sigmoid to output the preliminary channel attention vector. The dynamic spatial weight generation unit is used to query a learnable relative position table based on the relative positions between pixels in the input features and generate a dynamic spatial weight matrix. The multiplication unit is used to multiply the dynamic spatial weight matrix with the preliminary channel attention vector to output the spatial prior channel attention weights; The weighting unit is used to multiply the spatial prior channel attention weights with the input features channel by channel to output the first attention feature.

[0024] This invention specifies the concrete implementation of the spatial prior channel attention submodule, including a channel attention calculation unit, a dynamic spatial weight generation unit, a multiplication unit, and a weighting unit. The channel attention calculation unit employs a dual-branch structure of average pooling and max pooling to capture global distribution information and the most salient response features, respectively. After passing through a multilayer perceptron and a sigmoid function, preliminary channel attention weights are obtained. The dynamic spatial weight generation unit queries a learnable relative position table based on the relative positions of pixels in the feature map to generate a dynamic spatial weight matrix. The multiplication unit multiplies this weight matrix with the preliminary channel attention weights to obtain the spatial prior channel attention weights. The weighting unit multiplies these spatial prior channel attention weights with the input features channel by channel. This implementation injects spatial location prior information into the channel attention weights, making channel attention no longer a globally uniform scalar but an adaptive weight that changes with spatial location, thus avoiding the complete loss of spatial information caused by global pooling in traditional channel attention.

[0025] Furthermore, the channel prior spatial attention submodule includes: The pooling and copying unit is used to perform global average pooling and global max pooling on the input features respectively, and to copy the channels of the two pooling results so that the number of channels is the same as that of the input features. The concatenation and rearrangement unit is used to concatenate the copied average pooling features, the copied max pooling features, and the input features along the channel dimension, and then rearrange the channels. The group convolution unit is used to perform group convolution on the rearranged features, and after Sigmoid activation, it outputs channel prior space attention weights. The weighting unit is used to multiply the channel prior space attention weights element-wise with the input features to output the second attention feature.

[0026] This invention specifies the concrete implementation of the channel prior spatial attention submodule, including a pooling and copying unit, a concatenation and rearrangement unit, a group convolution unit, and a weighting unit. The pooling and copying unit performs global average pooling and global max pooling on the input features, and then copies the two pooling results by channel, expanding the number of channels to be the same as the original input features. The concatenation and rearrangement unit concatenates the copied average pooling and max pooling features with the original input features along the channel dimension and then rearranges the channels, allowing the pooled features and original features to be interleaved and blended channel by channel. The group convolution unit performs group convolution on the rearranged features, and outputs the channel prior spatial attention weights after sigmoid activation. The weighting unit multiplies these weights element-wise with the input features. This implementation injects prior information about channel distribution into the spatial attention weights, fully preserving the unique information of each channel when generating spatial attention, and avoiding the loss of channel information caused by channel compression in traditional spatial attention.

[0027] Based on the same concept, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program or instructions stored in the memory, wherein the processor executes the computer program or instructions to implement the hyperspectral image classification method as described above.

[0028] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implements the hyperspectral image classification method as described above. Attached Figure Description

[0029] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the hyperspectral image classification method in an embodiment of the present invention; Figure 2 This is an architecture diagram of the gated large kernel convolution selection module in an embodiment of the present invention; Figure 3 This is a decomposition diagram of the large kernel convolution in an embodiment of the present invention; Figure 4 This is an architecture diagram of the multi-scale wavelet convolution module in an embodiment of the present invention; Figure 5 This is a diagram of the prior attention module architecture in an embodiment of the present invention; Figure 6 This is a schematic diagram of the spatial prior channel attention submodule in an embodiment of the present invention; Figure 7 This is a schematic diagram of the channel prior space attention submodule in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1

[0032] This invention proposes a hyperspectral image classification method that integrates gated large kernel selection and frequency domain prior attention. For example... Figure 1As shown, the overall process includes: dimensionality reduction and extraction processing, gated large kernel convolution selection module processing, multi-scale wavelet convolution module processing, prior attention module processing, and classification output.

[0033] Step S1: Perform dimensionality reduction processing on the hyperspectral image and extract a spatial window centered on each pixel to be classified as the input feature block.

[0034] First, acquire the raw hyperspectral image data. Assume the hyperspectral image has a spatial dimension H×W (height×width) and B continuous spectral bands; that is, the raw data tensor can be represented as I. raw ∈R H×W×B .

[0035] Since the number of original bands B is typically very large (tens to hundreds), containing a significant amount of redundant information, direct processing would incur a high computational burden. Therefore, this invention first performs dimensionality reduction on the hyperspectral image. Specifically, Principal Component Analysis (PCA) is used to reduce the dimensionality of the hyperspectral image. PCA linearly transforms the original bands into several uncorrelated principal components, sorted in descending order of variance contribution rate. This invention retains the first D principal components, ensuring that the information retention exceeds 99%, while simultaneously reducing the number of bands from B to D. The dimensionality-reduced image is denoted as I∈R. H×W×D .

[0036] Then, for each pixel to be classified in the image, a fixed-size square spatial window is extracted centered on that pixel. This window is used to incorporate spatial context information about the pixel's neighborhood. The window side length S is a hyperparameter, and its optimal value depends on the specific application scenario and sensor characteristics. Empirically, for different hyperspectral datasets, S can be set between 11 and 25 (e.g., S=15).

[0037] The window's boundaries are handled using symmetrical padding or repeating edge pixel values ​​to ensure that all pixels can be extracted within the complete window. This window actually constitutes a small image centered on the target pixel, with dimensions of S×S×D.

[0038] The extracted window is used as the input feature block for that pixel, denoted as X. Thus, the input feature block X for each pixel to be classified contains dimensionality-reduced spectral information and local spatial neighborhood information, which can be used by the subsequent gated large kernel convolution selection module.

[0039] Step S2: Process the input feature blocks using the gated large kernel convolution selection module.

[0040] The input feature block obtained in step S1 is fed into the introductory large kernel convolution selection module. The core objective of this module is to expand the receptive field and adaptively fuse multi-scale spatial context information while controlling the computational cost. Figure 2 As shown, the gated large kernel convolution selection module consists of two large kernel convolution branches with different dilation rates, one gated convolution branch, a modulation unit, an attention selection fusion unit, and an output unit.

[0041] To avoid the huge computational overhead of directly using large kernel convolution, this invention adopts... Figure 3 The large kernel convolution decomposition strategy is shown below. Traditional large kernel convolution is a full-space, full-channel convolution kernel. Each kernel performs convolution calculations on all channels of the input features and the entire spatial region simultaneously. The number of parameters and computational cost increases with the square of the kernel size. This invention decomposes the large kernel convolution into a series of three operations: depthwise convolution, depth dilation convolution, and pointwise convolution.

[0042] Depthwise convolution operates channel-wise, with each kernel performing spatial convolution on only one channel of the input feature, without cross-channel fusion. This makes the computational cost of depthwise convolution linearly related to the number of channels, much less than that of ordinary convolution.

[0043] Depth-dilated convolution: This introduces a dilation rate (dilation rate) into depthwise convolution, expanding the receptive field by inserting holes between kernel elements without increasing the actual size of the kernel. For example, using a 3×3 depthwise dilated convolution with a dilation rate of 3 can achieve an effective receptive field of 7×7. By adjusting the dilation rate, different receptive fields can be obtained with the same computational cost.

[0044] Pointwise convolution: also known as 1×1 convolution, it lacks spatial integration capabilities and is mainly used for cross-channel feature fusion. It linearly combines the channel-wise spatial features extracted by depthwise convolution and depthwise dilated convolution to achieve information interaction along the channel dimension.

[0045] The three operations described above are chained together to simulate the large receptive field effect of traditional large kernel convolution with low computational cost.

[0046] like Figure 2 As shown, the gated large kernel convolution selection module of the present invention sets up two parallel large kernel convolution branches, denoted as the first large kernel convolution branch and the second large kernel convolution branch, respectively. Both large kernel convolution branches adopt the above-mentioned decomposition structure (depth convolution → depth dilation convolution → point convolution), but the depth dilation convolution of the two large kernel convolution branches uses different dilation rate parameters, thereby obtaining receptive field features of different scales.

[0047] The specific processing procedure is as follows: The input feature blocks are first fed into the first and second large kernel convolution branches respectively. Within each large kernel convolution branch, the feature blocks sequentially undergo depthwise convolution, depth dilation convolution, and point convolution, outputting the first branch features and the second branch features respectively.

[0048] Meanwhile, to dynamically adjust the feature intensity of the two large-kernel convolution branches, a gated convolution branch is introduced. This gated convolution branch consists of a 3×3 depthwise convolution, which directly convolves the input feature block and outputs a gated feature map.

[0049] Then, the modulation units (i.e., the first modulation unit and the second modulation unit) are used to perform element-wise multiplication (i.e., multiplying corresponding elements) on the gated feature map with the first branch feature and the second branch feature, respectively, to obtain the first modulation feature and the second modulation feature. This operation enables the output of the two large-kernel convolution branches to be dynamically scaled according to the gating signal, enhancing the model's adaptability to the input content.

[0050] After obtaining the two modulation features, attention-selective fusion units are needed to selectively fuse information from different receptive fields. The specific steps are as follows: The first and second modulation features are concatenated along the channel dimension to form a concatenated feature with double the number of channels. Then, global average pooling is performed on this concatenated feature to compress the spatial information of each channel into a scalar, resulting in a global description vector. This global description vector is input into the Softmax activation function and channel segmentation is performed to obtain the first selection weight and the second selection weight. The first and second selection weights correspond to the first and second largest kernel convolution branches, respectively, and the sum of the weights of their corresponding channels is 1. The selection weights are then weighted channel-wise with the corresponding modulation features: that is, the first selection weight is multiplied by each channel of the first modulation feature, and the second selection weight is multiplied by each channel of the second modulation feature. Finally, the two weighted features are added element-wise to obtain a fused feature map.

[0051] The output unit multiplies the fused feature map element-wise with the original input feature block to obtain the final output of the gated large kernel convolution selection module, which is the multi-scale spatial fusion feature. This output feature retains the basic information of the original input while injecting multi-scale spatial context information after gating modulation and attention selection.

[0052] Step S3: Use a multi-scale wavelet convolution module to process the multi-scale spatial fusion features.

[0053] The multi-scale spatial fusion features output from step S2 are input into the multi-scale wavelet convolution module. For example... Figure 4As shown, this module aims to transform features to the frequency domain, explicitly separate low-frequency structural information from high-frequency detail information, perform targeted enhancement through multi-scale convolutional branches, and finally fuse the frequency domain enhancement information with the original spatial features through residual connections.

[0054] First, a two-dimensional discrete wavelet transform is performed on the input multi-scale spatial fusion features using a wavelet transform unit. The Haar wavelet basis is preferred in this invention, but other wavelet bases (such as the Daubechies wavelet) can also be used. Through wavelet transform, the spatial domain features are decomposed into four frequency domain sub-bands: A low-frequency sub-band (denoted as LL) corresponds to the overall structural information and approximate components of ground features; The three high-frequency sub-bands are denoted as LH (horizontal high frequency), HL (vertical high frequency), and HH (diagonal high frequency), respectively, corresponding to the edge, texture, and other detailed information of ground features.

[0055] The spatial size of each subband is half that of the original feature (if Haar wavelets are used), while the number of channels remains unchanged. This decomposition allows the model to process different frequency components separately.

[0056] After obtaining four sub-bands, a multi-scale frequency domain feature extraction unit is used to extract multi-scale frequency domain features for each sub-band. Specifically, three parallel convolutional branches are designed for each sub-band, and the outputs of the three convolutional branches are added together to obtain the enhanced features for that sub-band.

[0057] The three convolutional branches are structured as follows: The first convolutional branch consists of a series of vertical and horizontal convolutions. The vertical convolution uses a 1×k kernel (k is a preset size, such as 3 or 5), and the horizontal convolution uses a k×1 kernel. The vertical convolution is performed first, followed by the horizontal convolution; the combined result is equivalent to a k×k receptive field, but with a smaller number of parameters (k...). 2 Reduced to 2k. This convolutional branch excels at capturing structural features in the vertical-horizontal direction.

[0058] The second convolutional branch is also composed of vertical and horizontal convolutions concatenated, but it can use a different k value or a different concatenation order (e.g., horizontal first, then vertical) to capture anisotropic features at different scales.

[0059] The third convolutional branch uses point convolution (i.e., 1×1 convolution) to preserve the channel information of the original subband features without changing the spatial structure.

[0060] The outputs of the three convolutional branches are summed element-wise to obtain the enhanced subband features of that subband. This operation is then performed on the low-frequency subband LL and the three high-frequency subbands LH, HL, and HH respectively, resulting in four enhanced subband features.

[0061] The four enhanced sub-band features are reconstructed back to the spatial domain using an inverse wavelet transform unit. The inverse wavelet transform is the inverse process of the wavelet transform, which synthesizes the frequency domain sub-bands into spatial domain features, and its output size is restored to the same spatial size as the module input.

[0062] To enable lossless transmission of frequency domain enhancement information and mitigate the gradient vanishing problem, this invention introduces a residual path into the module. Specifically: The input of the module (i.e., the multi-scale spatial fusion features output in step S2) is simultaneously fed into a separate deep convolutional layer (residual path unit). This deep convolutional layer uses simple deep convolution (e.g., 3×3) to perform a lightweight transformation on the input features and output residual features. The spatial domain features reconstructed by inverse wavelet transform are added element-wise to the above residual features to obtain the final frequency domain enhanced features.

[0063] This residual connection ensures that information from the original spatial features can be directly transmitted to the output, while also superimposing frequency domain enhancement information to avoid potential information loss during wavelet transform.

[0064] The output of the multi-scale wavelet convolution module is the frequency domain enhancement feature. This feature simultaneously contains a collaborative representation of low-frequency structural information and high-frequency detail information, significantly enhancing the ability to perceive differences in ground feature edges, textures, and structures.

[0065] Step S4: Process the frequency domain enhancement features using the prior attention module.

[0066] The frequency domain enhancement features output from step S3 are fed into the prior attention module. The prior attention module adopts a Transformer-like architecture, but replaces the multi-head self-attention mechanism with a Spatial Prior Channel Attention Submodule (SPCA) and a Channel Prior Spatial Attention Submodule (CPSA). For example... Figure 5 As shown, the prior attention module sequentially includes: a first normalization unit, a spatial prior channel attention submodule, a channel prior spatial attention submodule, a first residual connection unit, a second normalization unit, a feedforward network, and a second residual connection unit.

[0067] The first normalization unit performs layer normalization on the input frequency domain enhanced features. Layer normalization normalizes the channel dimension of each sample to stabilize the feature distribution and accelerate training convergence. The normalized features are then output to the spatial prior channel attention submodule.

[0068] The spatial prior channel attention submodule receives the output of the first normalization unit. This submodule uses the relative positions of pixels to generate dynamic spatial weights to constrain channel attention, avoiding the loss of spatial structure information caused by global pooling in traditional channel attention. Its specific implementation is as follows (see...). Figure 6 ): First, global average pooling and global max pooling are performed on the input features to obtain two vectors describing global information. These two vectors are then input into a multilayer perceptron (MLP, with two MLPs sharing parameters). The outputs of the two MLPs are then summed and passed through a sigmoid activation function to obtain a preliminary channel attention vector.

[0069] Simultaneously, based on the relative positions of each pixel in the input features, a learnable relative position table is queried to generate a dynamic spatial weight matrix. This matrix has the same spatial dimensions as the input features, and the weight of each position depends on the pixel's offset relative to a reference point.

[0070] Multiplying the dynamic spatial weight matrix with the initial channel attention vector (i.e., multiplying each spatial location by the corresponding channel value of the initial channel attention vector) yields a spatial prior channel attention weight tensor, thereby injecting spatial prior information into the channel attention weights.

[0071] Finally, the spatial prior channel attention weights are multiplied channel by channel with the input features of the submodule to output the first attention feature.

[0072] The channel-prior spatial attention submodule receives the first attention feature. This submodule preserves channel prior information to generate spatial attention through channel duplication, rearrangement, and group convolution, avoiding the loss of channel information caused by channel compression in traditional spatial attention. Its specific implementation is as follows (see...). Figure 7 ): First, global average pooling and global max pooling are performed on the input features to obtain two low-dimensional feature vectors. Each vector is then copied along the channel dimension to expand it into a feature map with the same spatial size as the input features, resulting in copied average pooling features and copied max pooling features.

[0073] Then, the copied average pooling feature, the copied max pooling feature, and the original input feature (i.e., the input of this submodule) are concatenated along the channel dimension to obtain a channel-expanded feature. This feature is then subjected to channel shuffle, which interleaves channels from three different sources to enhance information fusion.

[0074] Next, group convolution is performed on the rearranged features to output a single-channel spatial attention map. This spatial attention map is then activated by the sigmoid function to obtain the channel prior spatial attention weights.

[0075] Finally, the prior spatial attention weights of the channel are multiplied element-wise with the input features of the submodule to output the second attention features.

[0076] The first residual connection unit adds the second attention feature output by the channel prior spatial attention submodule to the module's original input (i.e., the frequency domain enhancement feature output in step S3) element-wise, outputting the first residual feature. This residual connection allows the original frequency domain enhancement information to be directly passed to subsequent layers, avoiding gradient vanishing and preserving the basic information that has not undergone attention processing.

[0077] The second normalization unit performs layer normalization on the first residual features to obtain normalized features, which are then output to the feedforward network.

[0078] The feedforward network receives the output of the second normalization unit and performs a nonlinear transformation and feature mapping on it. A feedforward network typically consists of two fully connected layers and a nonlinear activation function (such as GELU), processing each location independently without altering the spatial size of the features.

[0079] The second residual connection unit adds the output of the feedforward network to the first residual feature element-wise, outputting the prior enhanced feature. This residual connection further enhances information flow and ensures feature richness.

[0080] Thus, the prior enhancement features integrate the dual constraints of spatial prior channel attention and channel prior spatial attention, resulting in stronger discriminative ability and robustness.

[0081] Step S5: After performing global average pooling on the prior augmentation features, output the land cover category classification results through a linear layer.

[0082] The prior augmentation feature output in step S4 is a three-dimensional feature tensor (height, width, number of channels) with spatial and channel dimensions. To convert this feature into a class prediction for each pixel, the spatial dimension needs to be further compressed and a classification mapping performed.

[0083] First, global average pooling is performed on the prior augmented features. Global average pooling calculates the arithmetic mean of all pixel values ​​across the spatial dimensions (height and width) of each feature channel, thus compressing the spatial information of each channel into a single scalar value. After global average pooling, the spatial dimension of the features is eliminated, resulting in a one-dimensional feature vector of the same length as the number of channels. Each element in this vector represents the global response intensity of the corresponding channel, reflecting the activation level of that channel across the entire spatial region.

[0084] The one-dimensional feature vector obtained above is input into a linear layer (i.e., a fully connected layer). The linear layer maps the dimension of the feature vector to the number of land cover categories through a linear transformation (multiplying the weight matrix with the input vector and adding a bias). Specifically, if there are K land cover categories in the dataset, the output dimension of the linear layer is K. Each element in the output vector represents the unnormalized score (logit) of the pixel belonging to the corresponding category.

[0085] In practical applications, for each pixel to be classified, the category index with the highest score in the linear layer output vector is typically taken as the final land cover category label for that pixel. If a probability distribution is required, the scores can be further converted into probability values ​​using the Softmax function. At this point, the entire hyperspectral image classification method is complete, and each pixel has obtained its corresponding land cover category classification result.

[0086] It should be noted that the combination of global average pooling and linear layers replaces the flattening operation and multiple fully connected layers in traditional methods, which can significantly reduce the number of parameters and effectively prevent overfitting, making it particularly suitable for hyperspectral classification tasks with limited training samples.

[0087] To verify the effectiveness and advancement of the hyperspectral image classification method proposed in this invention, comparative experiments were conducted on several publicly available hyperspectral image datasets, as detailed below: The experiment used three representative hyperspectral image datasets: Indian Paces: A classic hyperspectral classification benchmark dataset that mainly covers agricultural and forestry areas. It features a wide variety of land cover categories and high spectral similarity between categories, and is used to test the model's ability to distinguish under complex spectral confusion.

[0088] WHU-Hi-HanChuan and WHU-Hi-HongHu: These two datasets contain complex land cover distributions and significant spatial structure differences, which place higher demands on the model's ability to jointly model spectral and spatial data, as well as its generalization performance.

[0089] The datasets mentioned above comprehensively cover the typical challenges of hyperspectral image classification from different scenarios and complexity perspectives, and can objectively evaluate the overall performance of the methods.

[0090] The evaluation indicators used in the experiment included: Overall Accuracy (OA): The proportion of correctly classified pixels out of the total number of pixels, which measures the overall classification accuracy.

[0091] Average Accuracy (AA): The arithmetic mean of the classification accuracy for each category, reflecting the average level of classification performance for each category.

[0092] Kappa coefficient (κ): Used to comprehensively evaluate the consistency between classification results and random classification results, and can reflect the performance of classification methods in a relatively comprehensive way.

[0093] The comparison methods cover a variety of representative hyperspectral image classification techniques, including traditional machine learning methods (Support Vector Machine SVM), recurrent neural networks (LSTM), convolutional neural networks (TriCNN, HybridSN), Transformer-type methods (SpectralFormer, GSCViT, SimPoolFormer), and the recently proposed spectral-space joint classification method (FSCGNet). These comparison methods are highly representative and comparable.

[0094] The overall classification accuracy (OA), average classification accuracy (AA), and Kappa coefficient (κ) of each method on the three datasets are shown in Table 1.

[0095] Table 1 Summary of Experimental Results

[0096] As shown in Table 1, the method of this invention significantly outperforms all comparative methods on all evaluation metrics across the three datasets. Specifically: On the Indian Paces dataset, the method of this invention achieves an OA of 98.24%, an AA of 98.35%, and a Kappa coefficient of 98.14%, which are approximately 3.5, 4.6, and 4.1 percentage points higher than the best comparison method HybridSN (OA 94.76%), respectively.

[0097] On the WHU-Hi-HanChuan dataset, the method of this invention achieves an OA of 98.11%, an AA of 96.65%, and a Kappa coefficient of 98.00%, which is significantly better than other methods.

[0098] On the WHU-Hi-HongHu dataset, the method of this invention achieves an OA of 98.70%, an AA of 97.40%, and a Kappa coefficient of 98.65%, demonstrating a particularly outstanding performance advantage.

[0099] The experimental results above demonstrate that the method of this invention can effectively extract discriminative spectral and spatial features in scenarios with complex ground cover distribution, subtle differences between categories, and a limited number of training samples, exhibiting higher classification accuracy and stronger stability. The combined results on the three datasets fully demonstrate the effectiveness and advancement of the method of this invention in hyperspectral image classification tasks. Example 2

[0100] This invention also provides an electronic device, which includes: a memory, a processor, and a computer program or instructions stored in the memory. The processor executes the computer program or instructions to implement the hyperspectral image classification method of this invention.

[0101] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) or loaded from a storage portion into random access memory (RAM). The processor can be a multi-core processor or may contain multiple processors. In some embodiments, the processor may include a general-purpose main processor and one or more specialized coprocessors, such as a central processing unit, graphics processing unit (GPU), neural network processor (NPU), digital signal processor (DSP), etc. Various programs and data required for device operation are also stored in RAM. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0102] The processor and memory described above are used together to execute programs / instructions stored in the memory. When the program / instructions are executed by the computer, they can implement the methods, steps, or functions described in the above embodiments.

[0103] Although not shown, embodiments of the present invention also provide a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the hyperspectral image classification method of the present invention.

[0104] Readable storage media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0105] The above description only discloses specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or modifications that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method, characterized in that, The classification method includes: The hyperspectral image is dimensionality reduced, and a spatial window is extracted centered on each pixel to be classified as the input feature block. The input feature block is processed using a gated large kernel convolution selection module. The gated large kernel convolution selection module uses two large kernel convolution branches with different dilation rates. Each branch expands the receptive field through the decomposition and combination of depthwise convolution, depthwise dilated convolution, and point convolution. The gated signal is used to perform element-wise multiplication modulation with the outputs of the two branches respectively. Multi-scale spatial features are adaptively fused through an attention selection mechanism and then multiplied element-wise with the input feature block to output multi-scale spatial fused features. Multi-scale spatial fusion features are processed using a multi-scale wavelet convolution module. The multi-scale wavelet convolution module decomposes frequency domain sub-bands through wavelet transform, extracts frequency domain features for each sub-band using three convolution branches, and adds the reconstructed features after inverse wavelet transform to the residual features of the multi-scale spatial fusion features after deep convolution processing, and outputs frequency domain enhanced features. The frequency domain enhancement features are processed using a prior attention module. The prior attention module includes a spatial prior channel attention submodule and a channel prior spatial attention submodule, and employs layer normalization, residual connections, and a feedforward network. Specifically, the spatial prior channel attention submodule generates dynamic spatial weights using the relative positions of pixels to constrain channel attention, and the channel prior spatial attention submodule retains channel prior information through channel duplication, rearrangement, and group convolution to generate spatial attention and output the prior enhancement features. After global average pooling of the prior enhancement features, the land cover classification results are output through a linear layer.

2. The hyperspectral image classification method according to claim 1, characterized in that, The dimensionality reduction process employs principal component analysis.

3. The hyperspectral image classification method according to claim 1, characterized in that, The gated large kernel convolution selection module includes: Two large kernel convolutional branches with different dilation rates, each branch consisting of sequentially connected depthwise convolution, depthwise dilated convolution and pointwise convolution, are used to extract features from the input feature block and output the first branch features and the second branch features respectively. The gated convolution branch, consisting of a depthwise convolution, is used to convolve the input feature block and output a gated feature map. The first modulation unit is used to multiply the gated feature map element by element with the first branch feature to output the first modulation feature; The second modulation unit is used to multiply the gated feature map element by element with the second branch feature to output the second modulation feature; The attention selection fusion unit is used to concatenate the first modulation feature and the second modulation feature along the channel dimension, and then sequentially pass them through global average pooling, softmax activation and channel segmentation to generate a first selection weight and a second selection weight. The first selection weight is then weighted with the first modulation feature channel by channel, and the second selection weight is then weighted with the second modulation feature channel by channel and summed to obtain a fused feature map. The output unit is used to multiply the fused feature map element-wise with the input feature block to output multi-scale spatial fused features.

4. The hyperspectral image classification method according to claim 1, characterized in that, The multi-scale wavelet convolution module includes: The wavelet transform unit is used to perform wavelet transform on the multi-scale spatial fusion features, decomposing them into one low-frequency sub-band and three high-frequency sub-bands. The multi-scale frequency domain feature extraction unit is used to extract frequency domain features for each sub-band using three convolutional branches. Two of the branches are composed of vertical convolution and horizontal convolution in series, and the other branch is a point convolution. The outputs of the three branches are added together to obtain the enhanced sub-band features of that sub-band. The inverse wavelet transform unit is used to perform inverse wavelet transform on all enhanced subband features and reconstruct them into spatial domain features. The residual path unit consists of a deep convolutional layer, which is used to perform deep convolution on the multi-scale spatial fusion features and output residual features. The addition unit is used to add the reconstructed spatial domain features to the residual features and output the frequency domain enhancement features.

5. The hyperspectral image classification method according to claim 4, characterized in that, The vertical convolution is a 1×k convolution, and the horizontal convolution is a k×1 convolution, where k is a preset convolution kernel size.

6. The hyperspectral image classification method according to claim 1, characterized in that, The prior attention module includes: The first normalization unit is used to perform layer normalization on the frequency domain enhancement features; The spatial prior channel attention submodule is used to receive the output of the first normalization unit, generate dynamic spatial weights using the relative positions of pixels to constrain channel attention, and output the first attention feature. The channel prior spatial attention submodule is used to receive the first attention feature, retain channel prior information through channel copying, rearrangement and group convolution to generate spatial attention, and output the second attention feature; The first residual connection unit is used to add the second attention feature to the frequency domain enhancement feature and output the first residual feature; The second normalization unit is used to perform layer normalization on the first residual feature; A feedforward network is used to perform nonlinear transformation and feature mapping on the output of the second normalization unit; The second residual connection unit is used to add the output of the feedforward network to the first residual feature and output the prior enhancement feature.

7. The hyperspectral image classification method according to claim 6, characterized in that, The spatial prior channel attention submodule includes: The channel attention calculation unit is used to perform global average pooling and global max pooling on the input features respectively. The two pooling results are input into the multilayer perceptron. The outputs of the two multilayer perceptrons are added together and then activated by Sigmoid to output the preliminary channel attention vector. The dynamic spatial weight generation unit is used to query a learnable relative position table based on the relative positions between pixels in the input features and generate a dynamic spatial weight matrix. The multiplication unit is used to multiply the dynamic spatial weight matrix with the preliminary channel attention vector to output the spatial prior channel attention weights; The weighting unit is used to multiply the spatial prior channel attention weights with the input features channel by channel to output the first attention feature.

8. The hyperspectral image classification method according to claim 6, characterized in that, The channel prior spatial attention submodule includes: The pooling and copying unit is used to perform global average pooling and global max pooling on the input features respectively, and to copy the channels of the two pooling results so that the number of channels is the same as that of the input features. The concatenation and rearrangement unit is used to concatenate the copied average pooling features, the copied max pooling features, and the input features along the channel dimension, and then rearrange the channels. The group convolution unit is used to perform group convolution on the rearranged features, and after Sigmoid activation, it outputs channel prior space attention weights. The weighting unit is used to multiply the channel prior space attention weights element-wise with the input features to output the second attention features.

9. An electronic device, comprising: A memory, a processor, and a computer program or instructions stored in the memory, characterized in that the processor executes the computer program or instructions to implement the hyperspectral image classification method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the hyperspectral image classification method as described in any one of claims 1 to 8.

Images