A hyperspectral image classification method

Abstract

The application belongs to the technical field of image classification, and discloses a hyperspectral image classification method, comprising the following steps: acquiring training data, wherein the training data comprises hyperspectral image training data and corresponding image classification labels; constructing an initial image classification model, wherein the initial image classification model comprises a spectral-spatial feature extraction module, a local feature extraction module and a KANLinear module connected in sequence; training the initial image classification model based on the training data to obtain a trained image classification model; and performing an image classification task of a hyperspectral image to be classified based on the trained image classification model. The technical solution disclosed by the application can efficiently process a hyperspectral image classification task, improve classification performance and optimize computing resources.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image classification, and particularly relates to a hyperspectral image classification method. BACKGROUND

[0002] Hyperspectral imaging technology has shown important value in agricultural monitoring, urban planning and other fields by improving spatial resolution and spectral coverage (400-2500 nanometers hundreds of bands), fusing spatial-spectral multi-dimensional data. The core challenge of the classification task is to analyze complex spectral features. Traditional methods such as LDA, PCA and other spectral analysis techniques are limited because they ignore spatial correlation. Although spatial-spectral joint methods improve the effect, shallow models (such as SVM) are difficult to adaptively extract deep features. In recent years, the Transformer architecture has made breakthroughs with its long-range modeling capabilities, but the high complexity of its self-attention mechanism restricts the computing efficiency. In summary, hyperspectral image classification is of great significance in remote sensing applications, but traditional methods and existing technologies still have deficiencies. Future research should focus on developing more efficient and adaptive feature extraction methods to fully utilize the rich information of hyperspectral images and overcome the limitations of existing technologies. SUMMARY

[0003] The purpose of the application is to provide a hyperspectral image classification method to solve the problems existing in the prior art.

[0004] To achieve the above purpose, the application provides a hyperspectral image classification method, comprising: obtaining training data, the training data comprising hyperspectral image training data and corresponding image classification labels; constructing an initial image classification model, the initial image classification model comprising a spectral-spatial feature extraction module, a local feature extraction module and a KANLinear module connected in turn; training the initial image classification model based on the training data to obtain a trained image classification model; and performing an image classification task of a hyperspectral image to be classified based on the trained image classification model.

[0005] Optionally, the training process of the image classification model specifically comprises: inputting the training data into the image classification model for classification prediction, and training according to a target loss function to obtain the trained image classification model.

[0006] Optionally, the image classification model's processing steps specifically include: dividing the hyperspectral image data into multiple overlapping 3-D blocks; extracting features from each 3-D block using a spectral-spatial feature extraction module to obtain spectral-spatial features; inputting the extracted spatial features into a local feature extraction module, where features are extracted and fused using convolutional additive self-attention and a visual state space sequence model to obtain fused features; and inputting the fused features into a KANLinear module, where the features are transformed using a B-spline parameterized learnable activation function to output the final image classification result.

[0007] Optionally, the step of using the spectral-spatial feature extraction module to extract features from each 3-D patch specifically includes: inputting the 3-D patch into a 3-D convolutional layer, located at the i-th... th Layer j th The activation value at spatial location (x, y, z) in the feature map is calculated as follows:

[0008]

[0009] Where φ represents the activation function, B i,j The bias term is represented by H. The dimension of the 3D convolution kernel is determined by H. i W i and R i The specified values ​​represent its height, width, and depth, respectively. Represents the θth th The weights corresponding to positions (h', w', r') in the feature map; ω is the weight parameter of the convolution kernel, and v represents the activation value of the previous layer;

[0010] Let k0 be the number of 3D convolutional kernels used in the 3D convolutional layer, with each kernel having a size of k1×k2×k3. The output feature cube size is k0@(s-k1+1)×(s-k2+1)×(B-k3+1). Further, the output of the 3D convolution is input into a 2D convolutional layer, where the input size is (s-k1+1)×(s-k2+1)×k0(B-k3+1). In the 2D convolutional layer, the activation value at spatial location (x,y) is calculated using the following formula:

[0011]

[0012] Among them, H' i and W' i These represent the height and width of the 2D convolution kernel, respectively. Represents θ th The weights corresponding to the positions (h', w') in the feature map, where B represents the spectral dimension and s is the window length;

[0013] Furthermore, for the input X', a series of deep convolutions with different receptive fields are used to extract rich contextual features spanning different spatial ranges, as shown in the following formula:

[0014]

[0015] in, This represents a depthwise convolution operation, assuming there are D deconstructed convolutional kernels, each of which is then processed through a 1×1 convolutional layer, denoted as .

[0016]

[0017] Features obtained from multiple convolutional kernels with different receptive field ranges are merged and then... Perform channel max pooling and average pooling operations, denoted as P respectively. max (·) and P avg (·), in order to effectively capture spatial relationships, the specific calculation formulas include:

[0018]

[0019] Among them, SA avg and SA max These represent the average and maximum values ​​of the spatial descriptors after pooling, respectively. This represents the concatenated feature tensor obtained by merging multi-scale features extracted by convolutional kernels with different receptive fields, and further processed by convolutional layers. This is used to connect and transform the pooled features, generating N spatial attention maps to enhance information interaction.

[0020]

[0021] In the formula, Spatial attention map;

[0022] For each spatial attention map, the sigmoid activation function σ(·) is applied, i.e. By weighting the deconstructed convolutional kernel features with a spatial mask and then integrating them through a convolutional layer F(·), attention features S are obtained:

[0023]

[0024] Element-wise multiplication of X' and S yields the expression: Y' = X'·S. Here, Y' represents the spectral-spatial characteristic.

[0025] Optionally, the fusion feature acquisition process specifically comprises: dividing the input hyperspectral image data into a plurality of non-overlapping image patches through a patch embedding layer to obtain embedded representations; inputting the embedded representations into parallel convolution addition self-attention branch and visual state space sequence module branch for feature extraction; and performing element-wise multiplication and residual connection processing on the features of the convolution addition self-attention branch and the visual state space sequence module branch to fuse local features and long-range dependency features to obtain fusion features.

[0026] Optionally, the processing process of the convolution addition self-attention branch specifically comprises: extracting local features of the embedded representations through convolution operation, and combining additive self-attention to capture global context information.

[0027] Optionally, the processing process of the visual state space sequence module branch specifically comprises: performing layer normalization processing on the input embedded representations, performing double-path feature extraction and state space modeling on the normalized data to capture long-range dependency relationships; wherein the first path captures linear patterns in the data through linear transformation and a nonlinear activation function, and the second path combines linear transformation, depth separable convolution and activation function to improve computational efficiency and capture fine-grained spatial features; and the outputs of the first path and the second path are merged along the channel dimension to obtain long-range dependency features.

[0028] Optionally, the transformation of the features by the B-spline parameterized learnable activation function specifically comprises: defining a regular grid, calculating a grid step h:

[0029]

[0030] In the formula, G r is the range of the grid, which is [-1, 1] by default, and the grid G is generated as G s r is the dimension index of the input feature, G is a parameter for controlling the size of the grid, D is the dimension of the input feature, and S is the order of the B-spline function.

[0031] For each input feature , the relationship between it and the grid points is calculated, and the function value is calculated according to the recursive definition of the B-spline function; for high-order splines, the B-spline function is recursively calculated as:

[0032]

[0033] wherein, is the k-th order B-spline function, g is a tensor composed of grid points for defining discrete points in each feature space, and g k is the k-th node value in the grid point sequence, which belongs to the discrete grid point tensor g that defines the support domain of the B-spline basis function. is an input feature value;

[0034] W is a weight represented by a piecewise polynomial spline Further transposed transformation is performed on the input Final output y is obtained out :

[0035]

[0036] wherein, is an output calculated by a base weight, is an output obtained by piecewise polynomial weighting, W base is a weight matrix of a base linear transformation, is a transposition of W base Final output is y out ∈R N×O O represents an output feature dimension.

[0037] The technical effects of the present application are:

[0038] The present application combines a 3-D-2-D hybrid convolutional neural network with a large selection kernel LSK network, which can gradually construct feature representation and ensure gradual feature abstraction from shallow to deep layers, thus effectively handling the complexity of hyperspectral images. The present application proposes a CASMamba module, which cooperatively integrates the local feature extraction capability of the convolutional layer, the focusing key information capability of the self-attention mechanism, and the long-range dependence modeling capability of the visual state space sequence model VSSM. The combination of the three provides more comprehensive and accurate feature extraction capability, thereby improving the classification performance. In the present application, the KANLinear module is used to replace the traditional linear layer, and provides stronger interpretability and flexibility. Through this replacement scheme, the model can better capture the potential spatial regularity and spectral features, which helps to improve the performance of the model in hyperspectral images, especially in handling high-dimensional data, which can effectively deal with high complexity. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a hyperspectral image classification method flow chart based on large selection kernel and convolution addition self-attention Mamba in the embodiment of the present application;

[0040] Figure 2 is a hyperspectral image classification method framework diagram based on large selection kernel and convolution addition self-attention Mamba in the embodiment of the present application;

[0041] Figure 3 is a CAS-VSSM module framework diagram in the embodiment of the present application;

[0042] Figure 4 Fig. 1 is a plot of overall classification accuracy of various algorithms in the Botswana dataset of the embodiments of the present application; wherein, Figure 4 Fig. 1(a) is a plot of overall classification accuracy of the comparative algorithm HybridSN in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(b) is a plot of overall classification accuracy of the comparative algorithm LSFAT in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(c) is a plot of overall classification accuracy of the comparative algorithm DBCT in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(d) is a plot of overall classification accuracy of the comparative algorithm DCTN in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(e) is a plot of overall classification accuracy of the comparative algorithm LSGA in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(f) is a plot of overall classification accuracy of the comparative algorithm SSFTT in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(g) is a plot of overall classification accuracy of the comparative algorithm MASSFormer in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(h) is a plot of overall classification accuracy of the comparative algorithm CVSSN in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(i) is a plot of overall classification accuracy of the comparative algorithm SSmamba in the Botswana dataset of the embodiments of the present application; Figure 4 Fig. 1(j) is a plot of overall classification accuracy of the HLSK-CAS mamba of the embodiments of the present application in the Botswana dataset;

[0043] Figure 5 Fig. 2 is a plot of overall classification accuracy of various algorithms in the Houston2013 dataset of the embodiments of the present application; wherein, Figure 5 Fig. 2(a) is a plot of overall classification accuracy of the comparative algorithm HybridSN in the Houston2013 dataset of the embodiments of the present application; Figure 5 Fig. 2(b) is a plot of overall classification accuracy of the comparative algorithm LSFAT in the Houston2013 dataset of the embodiments of the present application; Figure 5 Fig. 2(c) is a plot of overall classification accuracy of the comparative algorithm DBCT in the Houston2013 dataset of the embodiments of the present application; Figure 5 Fig. 2(d) is a plot of overall classification accuracy of the comparative algorithm DCTN in the Houston2013 dataset of the embodiments of the present application; Figure 5Fig. 6 is a diagram of overall classification accuracy of Houston 2013 dataset based on the comparative algorithm LSGA in the embodiment of the present application; Figure 5 Fig. 7 is a diagram of overall classification accuracy of Houston 2013 dataset based on the comparative algorithm SSFTT in the embodiment of the present application; Figure 5 Fig. 8 is a diagram of overall classification accuracy of Houston 2013 dataset based on the comparative algorithm MASSFormer in the embodiment of the present application; Figure 5 Fig. 9 is a diagram of overall classification accuracy of Houston 2013 dataset based on the comparative algorithm CVSSN in the embodiment of the present application; Figure 5 Fig. 10 is a diagram of overall classification accuracy of Houston 2013 dataset based on the comparative algorithm SSmamba in the embodiment of the present application; Figure 5 Fig. 11 is a diagram of overall classification accuracy of Houston 2013 dataset based on HLSK-CASmamba proposed in the embodiment in the embodiment of the present application;

[0044] Figure 6 Fig. 12 is a diagram of overall classification accuracy of Pavia University dataset of each algorithm in the embodiment of the present application; wherein, Figure 6 Fig. 13 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm HybridSN in the embodiment of the present application; Figure 6 Fig. 14 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm LSFAT in the embodiment of the present application; Figure 6 Fig. 15 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm DBCT in the embodiment of the present application; Figure 6 Fig. 16 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm DCTN in the embodiment of the present application; Figure 6 Fig. 17 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm LSGA in the embodiment of the present application; Figure 6 Fig. 18 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm SSFTT in the embodiment of the present application; Figure 6 Fig. 19 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm MASSFormer in the embodiment of the present application; Figure 6 Fig. 20 is a diagram of overall classification accuracy of Pavia University dataset based on the comparative algorithm CVSSN in the embodiment of the present application; Figure 6Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 6 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application;

[0045] Figure 7 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 7 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 7 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 7 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application;

[0046] Figure 8 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 8 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 8 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 8 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application;

[0047] Figure 9 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 9 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 9 Fig. 2 is a diagram of overall classification accuracy of Pavia University dataset based on comparative algorithm SSmamba in the embodiment of the present application; Figure 9(c) is the classification accuracy of the 10 algorithms in the embodiments of the application using different learning rates on the Pavia University dataset.

[0048] Figure 10 (c) is the classification accuracy of the 10 algorithms in the embodiments of the application using different learning rates on the Pavia University dataset. Figure 10 (a) is the classification accuracy of the 10 algorithms in the embodiments of the application using different learning rates on the Botswana dataset. Figure 10 (b) is the classification accuracy of the 10 algorithms in the embodiments of the application using different learning rates on the Houston2013 dataset. Figure 10 (c) is the classification accuracy of the 10 algorithms in the embodiments of the application using different learning rates on the Pavia University dataset. DETAILED DESCRIPTION

[0049] As shown in Figure 1 - Figure 10 In this embodiment, a hyperspectral image classification method is provided, including: obtaining training data, the training data including hyperspectral image training data and corresponding image classification labels; constructing an initial image classification model, the initial image classification model including a spectral-spatial feature extraction module, a local feature extraction module and a KANLinear module connected in sequence; training the initial image classification model based on the training data to obtain a trained image classification model; and performing an image classification task of a hyperspectral image to be classified based on the trained image classification model.

[0050] The embodiment provides a hyperspectral image classification method based on a large selective kernel and a convolution addition self-attention Mamba. First, a spectral-spatial feature extraction module is constructed, which integrates a 3-D convolution layer, a 2-D convolution layer and a large selective kernel network, and is used for efficiently extracting abstract spectral and spatial features of the hyperspectral image. Second, a new CASMamba model (Conv Additive Self-Attention Mamba Module) is proposed, and a core module CAS-VSSM combines a convolution addition self-attention and a visual state space sequence model. The combination can not only extract local features by using the convolution layer, but also model spatial dependency by using the self-attention mechanism and capture long-range dependency by using the VSSM model. Finally, the KANLinear module (Kolmogorov-Arnold Network Linear Module) is introduced to replace the traditional linear layer, so that the model can better cope with the high dimension and complexity of the hyperspectral image, thereby obtaining higher classification precision and robustness. The method provided in the embodiment is based on a deep learning framework, fully utilizes the large selective kernel and the convolution addition self-attention Mamba module, effectively captures spectral-spatial features and deep semantic features of the hyperspectral image, constructs a hyperspectral remote sensing image dataset suitable for different spatial resolutions, has strong portability and can better meet the image classification requirements.

[0051] First, the embodiment constructs a spectral-spatial feature extraction module, which integrates a 3-D convolution layer, a 2-D convolution layer and a large selective kernel network, and is used for efficiently extracting abstract spectral and spatial features of the hyperspectral image. Second, a new CASMamba model is proposed, and a core module CAS-VSSM combines a convolution addition self-attention and a visual state space sequence model. The combination can not only extract local features by using the convolution layer, but also model spatial dependency by using the self-attention mechanism and capture long-range dependency by using the VSSM model. Finally, the KANLinear module is introduced to replace the traditional linear layer, and the acquisition process of the sample label is further optimized.

[0052] The spectral-spatial feature extraction module specifically includes: a hyperspectral data cube is represented as I ∈ R M×N×B , wherein M represents the width of the image, N represents the height of the image, and B represents the spectral dimension. Each pixel in I corresponds to a label vector Y = (y1, y2, …, y C), where C represents the ground object classification category. Hyperspectral images contain B bands, which can capture rich spatial-spectral features, but also bring significant processing complexity and redundant information. To solve this problem, Principal Component Analysis (PCA) is usually applied in the preprocessing stage to reduce the number of bands from B to K while preserving the original spatial structure. Thus, the adjusted input is denoted as X ∈ R M×N×K .

[0053] The input data is divided into a number of overlapping 3-D patches P ∈ R s×s×K , which are centered at spatial coordinates (m, n), where s x s represents the size of the window. The resulting 3-D patches are organized into a grid of size (M-s+1) x (N-s+1). Each 3-D patch P m,n , centered at (m, n), has a width range of m-(s-1) / 2 to m+(s-1) / 2 and a width range of n-(s-1) / 2 to n+(s-1) / 2.

[0054] To fully exploit the abstract spatial-spectral features of each sample patch, a hybrid convolutional layer is designed, which consists of a 3-D convolutional layer and a 2-D convolutional layer. Each training sample patch, with size s x s x K, is input into the 3-D convolutional layer. Based on this, the activation value at spatial position (x, y, z) in the j th layer of the i th feature map is calculated as:

[0055]

[0056] where φ represents the activation function, B i,j denotes the bias term. The dimensions of the 3-D convolutional kernel are specified by H i , W i , and R i , which represent its height, width, and depth, respectively. In addition, denotes the weight corresponding to position (h', w', r') in the θ th feature map, ω is the weight parameter of the convolutional kernel, and v represents the activation value of the previous layer.

[0057] Let k0 be the number of 3-D convolutional kernels used in the 3-D convolutional layer. The size of each 3-D kernel is k1 x k2 x k3, and the output feature cube size is k0 x (s-k1+1) x (s-k2+1) x (B-k3+1). Further, the output of the 3-D convolution is input into the 2-D convolutional layer, where the input size is (s-k1+1) x (s-k2+1) x k0 x (B-k3+1). In the 2-D convolutional layer, the activation value at spatial position (x, y) is calculated as:

[0058]

[0059] where H' and W' represent the height and width of the 2-D convolution kernel, respectively. The parameters i and W' i represent the height and width of the 2-D convolution kernel, respectively. The parameters denote the weights corresponding to the position (h', w') of the feature map. th

[0060] Further, for the input X', a series of deep convolutions with different receptive fields are adopted to extract rich contextual features across different spatial ranges, as follows: where denotes the deep convolution operation. It is assumed that there are D deconstructed convolution kernels, each of which is then processed by a 1x1 convolution layer, denoted as In this part, the feature maps generated by multiple scale large convolution kernels are used to enhance the classification performance by focusing on the most relevant spatial context to distinguish different land cover categories. The specific implementation process is as follows:

[0061] First, the features obtained from multiple convolution kernels with different receptive field ranges are merged. Then, the is subjected to channel max-pooling and average-pooling operations, denoted as P max (·) and P avg (·), respectively, in order to effectively capture spatial relationships.

[0062]

[0063] where SA avg and SA max represent the average and maximum values of the pooled spatial descriptors, respectively, represents the spliced feature tensor obtained after merging the multi-scale features extracted by convolution kernels with different receptive fields. In addition, a convolution layer is further adopted to connect and convert the pooled features, generating N spatial attention maps to enhance information interaction.

[0064]

[0065] In order to generate different spatial selection masks for the deconstructed large convolution kernels, a sigmoid activation function σ(·) is adopted for each spatial attention map, i.e. In addition, the attention feature S is obtained by weighting the deconstructed convolution kernel features with the spatial mask and then integrating them through the convolution layer F(·).

[0066]

[0067] ​Finally, element-wise multiplication is performed between X' and S, denoted as Y' = X' · S, where Y' is the spectral-spatial feature.

[0068] The proposed spectral-spatial feature extraction module includes the following stages: first, a 3-D convolutional layer is used to extract shallow spectral feature representations from the principal components. Then, a 2-D convolutional layer is applied to capture more abstract spectral and spatial features. Finally, by dynamically adjusting a larger spatial receptive field, the large selection kernel module enhances the extraction of deep context information, thereby adapting to changes in different land cover classes.

[0069] The convolutional additive self-attention Mamba module specifically includes: the CASMamba network first passes the input x ∈ R H×W×3 into non-overlapping 4 × 4 image patches to obtain an embedding representation Then, the embedding data is processed by layer normalization to standardize the feature distribution, and then processed by the CASMamba backbone network. The CAS-VSSM module is the core module of the CASMamba network, which adopts a double-branch structure, combining the convolutional additive self-attention module and the visual state space sequence module.

[0070] In the convolutional additive self-attention branch, the CAS mechanism extracts local features from the input by combining convolutional operations and additive self-attention, and innovatively captures both local spatial dependencies and global context information. This mechanism optimizes efficiency by reducing computational complexity, with lower computational overhead than traditional attention mechanisms. The CAS mechanism dynamically weights local features, enhancing the network's focusing ability in key areas and improving feature extraction, while avoiding significant increases in computational cost.

[0071] In the visual state space sequence module (VSSM) branch, the input is first processed by layer normalization to stabilize the learning process, and then divided into two processing paths, each focusing on a different feature extraction strategy. The first path captures linear patterns in the data through linear transformation and nonlinear activation functions. The second path combines linear transformation, depth separable convolution, and activation functions, using depth separable convolution to improve computational efficiency and reduce parameter quantity while maintaining the capture of fine-grained spatial features. VSSM dynamically models state space sequences to promote high-level feature extraction and enhance the network's understanding of multi-scale spatio-temporal dependencies. The integration of the two paths balances computational efficiency and the extraction ability of low-level and high-level features.

[0072] The extracted features are normalized across layers and multiplied element-wise with the output of the first branch to achieve feature fusion. The fused features are processed through a linear layer and a residual connection is introduced to generate the final output of the CAS-VSS M module. In the VSSM branch, the default activation function is SiLU. In the last stage of the network, the outputs from the two branches are merged along the channel dimension and fused through a 1x1 convolutional layer to facilitate information exchange across channels.

[0073] Finally, by combining the CAS mechanism in the CAS branch with the state space modeling in the VSSM branch, the architecture can effectively model complex interactions in the input, thereby improving performance in tasks that require fine-grained recognition and temporal perception while maintaining high computational efficiency. The specific implementation process is as follows:

[0074] In the CAS module, the similarity function is calculated as:

[0075] Sim(Q, K) = φ(Q) + φ(K) s.t. φ(Q) = C(S(Q))

[0076] where the function φ(Q) represents the context mapping function, and φ(K) includes channel attention C(·) ∈ R N×d and spatial attention S(·) ∈ R N×d Based on this, the output of CAS is represented as:

[0077] O = τ(φ(Q) + φ(K)) x V

[0078] where τ(·) ∈ R N×d represents a linear transformation that integrates context information, and O(N) represents the complexity. Traditional state space models can be conceptualized as linear time-invariant systems that convert input sequences x(t) ∈ R d×L into output responses y(t) ∈ R through hidden states h(t) ∈ R. d represents the dimension of the state and L represents the length of the sequence.

[0079] The transformation is calculated as:

[0080]

[0081] where t represents the time step of the current input, h'(t) represents the hidden state corresponding to the current input x(t), and h(t) represents the hidden state of the previous time step. The matrix A ∈ R d×d represents the evolution parameter, B ∈ R d×L and C ∈ R d×L represent the projection parameters, respectively.

[0082] Traditional state-space models have limited ability to adapt to different inputs when training parameters, which limits their ability to effectively model dynamic systems. To address this limitation, the VSSM model is introduced as a discretized version of continuous SSM. In this module, continuous parameters A and B are converted to discrete parameters and Using the step size parameter △. Zero-order hold (ZOH) is the most commonly used discretization method, and the calculation formula is as follows:

[0083]

[0084] Where I represents the identity matrix. The discretized state-space model equation is as follows:

[0085]

[0086] Where h p represents the hidden state at time step p, X p and y p represent the input sequence and output sequence at time step p, respectively.

[0087] The KANLinear module specifically includes: when determining the label for each pixel category, the embodiment replaces the traditional linear layer with the KANLinear layer. Unlike traditional neural network architectures, which use fixed activation functions at the neuron level, the KANLinear layer applies learnable activation functions on the edges (connections) of the network. These activation functions are parameterized by B-splines, which provide a flexible and stable method for modeling complex relationships between features. This replacement not only enhances the model's representation ability, but also improves computational efficiency by reducing memory usage. This flexibility and efficiency make the KANLinear layer more advantageous in hyperspectral image classification, as hyperspectral image classification requires effective processing and classification of complex high-dimensional data.

[0088] The input tensor is Where N is the batch size and D is the dimension of the input features, calculated as:

[0089]

[0090] Where, is the feature vector of the i-th sample. This module uses a regular grid for interpolation, and the size of the grid is controlled by the G s parameter, and the interval of the grid is controlled by the G r parameter. For each feature in the input data, a grid will be established within the defined interval of that feature.

[0091] First, calculate the grid step size h:

[0092]

[0093] where G r is the range of the grid, which is [-1,1] by default. Then, the grid r is the dimension index of the input feature, S is the order of the B-spline function, which means there are a certain number of piecewise spline functions on each feature used for interpolation.

[0094] For each input feature , the relationship between it and the grid points is calculated, and the function value is calculated according to the recursive definition of the B-spline function. B(X) is the B-spline function, which is first defined as an exponential function on each grid to mark the grid interval where the input feature is located. That is, if X i is between two consecutive grid points, then B(X) is 1, otherwise it is 0.

[0095] For high-order splines, the B-spline function is recursively calculated as:

[0096]

[0097] where, is the k-th order B-spline function, g is a tensor composed of grid points, which is used to define the discrete points of each feature space. g k is the k-th node value in the grid point sequence, which belongs to the discrete grid point tensor g that defines the support domain of the B-spline basis function, is the input feature value;

[0098] Finally, for the accurate modeling of hyperspectral images, a piecewise polynomial representation of the weight W spline is used to further transform the input to obtain the final output y out :

[0099]

[0100] where, is the output calculated by the basic weight, is the output obtained by weighting the piecewise polynomial. W base is a weight matrix of a basic linear transformation, is the transpose of W base , and the final output is y out ∈R N×O , O represents the output feature dimension.

[0101] The embodiment proposes a new algorithm, a hyperspectral image classification method based on large selective kernel and convolution addition self-attention Mam ba, for the hyperspectral image classification task. Although convolutional neural network CNN performs effectively in the hyperspectral image classification task, they are often difficult to fully capture complex semantic features, and with the increase of network depth, the calculation cost increases significantly. In contrast, the Transformer has good effect in modeling spectral-spatial dependence, but due to its complexity, it also brings significant computational overhead. The Mamba model uses a state space model to provide an effective alternative, which efficiently captures long-range dependencies in HSI while ensuring computational efficiency. This feature not only improves classification performance, but also optimizes computing resources, making it a promising method for hyperspectral image applications. The embodiment proposes a HLSK-Mamba model, which introduces three key modules to improve performance and efficiency. First, a spectral-spatial feature extraction module is constructed, which combines 3-D convolution layers, 2-D convolution layers and large selective kernel networks to effectively extract abstract spectral and spatial features in hyperspectral images. Second, the embodiment proposes a new CASMamba model, whose core module CAS-VSSM combines convolution addition self-attention and visual state space sequence model. This integrated method effectively utilizes the local feature extraction capability of convolution layer, the ability of self-attention mechanism to focus on key information, and the long-range dependence modeling capability of VSSM. Finally, the KANLinear module is used to better cope with the high dimensionality and complexity of hyperspectral images, thereby enhancing the ability to obtain accurate sample labels.

[0102] In order to effectively capture abstract spatial-spectral feature representations and inherent unique prior knowledge in hyperspectral images, the embodiment proposes an architecture that combines 3-D-2-D hybrid convolutional neural network and large selective kernel LSK network. Such architecture can gradually build feature representations, ensuring gradual feature abstraction from shallow to deep layers, thus effectively handling the complexity of hyperspectral images.

[0103] The embodiment proposes a CASMamba module that cooperatively integrates the local feature extraction capability of convolution layer, the ability of self-attention mechanism to focus on key information, and the long-range dependence modeling capability of visual state space sequence model VSSM. The combination of the three provides more comprehensive and accurate feature extraction capability, thereby improving classification performance.

[0104] In this embodiment, KANLinear modules are used to replace traditional linear layers, providing stronger interpretability and flexibility. Through this replacement, the model can better capture potential spatial regularity and spectral features, which helps to improve the performance of the model in hyperspectral images, especially when dealing with high-dimensional data, effectively dealing with high complexity.

[0105] A series of extensive experiments were conducted on three publicly available benchmark hyperspectral image datasets: Houston2013, Botswana, and Pavia University. The experimental results show that the network proposed in this embodiment outperforms some of the most advanced methods in the field in terms of performance.

[0106] This embodiment proposes a new algorithm, a hyperspectral image classification method based on large selective kernel and convolution addition self-attention Mamba, for the hyperspectral image classification task. Although convolutional neural networks (CNN) perform effectively in hyperspectral image classification tasks, they often struggle to fully capture complex semantic features, and as the network depth increases, the computational cost increases significantly. In contrast, the Transformer has better results in modeling spectral-spatial dependencies, but due to its complexity, it also brings significant computational overhead. The Mamba model uses a state space model to provide an effective alternative that efficiently captures long-range dependencies in HSI with linear complexity while ensuring computational efficiency. This feature not only improves classification performance but also optimizes computing resources, making it a promising method for hyperspectral image applications. This embodiment proposes the HLSK-Mamba model, which introduces three key modules to improve performance and efficiency. First, a spectral-spatial feature extraction module is constructed, combining 3-D convolution layers, 2-D convolution layers, and large selective kernel networks to effectively extract abstract spectral and spatial features in hyperspectral images. Second, this embodiment proposes a new CASMamba model, whose core module CAS-VSSM combines convolution addition self-attention and visual state space sequence model. This integrated approach effectively utilizes the local feature extraction capabilities of convolution layers, the ability to focus on key information of self-attention mechanisms, and the long-range dependency modeling capabilities of VSSM. Finally, the KANLinear module is used to better cope with the high dimensionality and complexity of hyperspectral images, thereby enhancing the ability to obtain accurate sample labels.

[0107] Figure 1is a hyperspectral image classification method flowchart based on large kernel and convolution additive self-attention Mamba. The flow mainly includes: first, a spectral-spatial feature extraction module is constructed, which integrates 3-D convolution layer, 2-D convolution layer and large kernel network, to efficiently extract abstract spectral and spatial features of hyperspectral images; second, a novel CASMamba module is proposed, the core module CAS-VSSM combines convolution additive self-attention and visual state space sequence model. This combination not only can utilize convolution layer to extract local features, but also can utilize self-attention mechanism to focus on key information, and can utilize VSSM model to capture long-range dependencies; finally, the KANLinear module is used to further optimize the sample label acquisition process.

[0108] Figure 2 is a hyperspectral image classification method framework based on large kernel and convolution additive self-attention Mamba. The basic framework includes spectral-spatial feature extraction module, CASMamba module and KANLinear module.

[0109] Figure 3 is a CAS-VSSM module framework diagram, which includes CAS branch module and VSSM branch module.

[0110] Figure 4 (a)- Figure 4 (j) shows the classification results of 10 algorithms on the Botswana dataset, including HybridSN, LSFAT, DBCT, DCTN, LSGA, SSFTT, MASSFormer, CVSSN, SSmamba and HLSK-CASMamba. By comparing these classification results, it can be seen that the proposed HLSK-CASMamba algorithm (i.e. Figure 4 (j)) shows the best classification effect.

[0111] Figure 5 (a)- Figure 5 (j) shows the classification results of 10 algorithms on the Houston2013 dataset, including HybridSN, LSFAT, DBCT, DCTN, LSGA, SSFTT, MASSFormer, CVSSN, SSmamba and HLSK-CASMamba. By comparing these classification results, it can be seen that the proposed HLSK-CASMamba algorithm (i.e. Figure 5 (j)) shows the best classification effect.

[0112] Figure 6 (a)- Figure 6(j) shows the classification results of 10 algorithms on the Pavia University dataset, including HybridSN, LSFAT, DBCT, DCTN, LSGA, SSFTT, MASSFormer, CVSSN, SSmamba and HLSK-CASMamba. By comparing these classification results, it can be seen that the proposed HLSK-CASMamba algorithm (i.e. Figure 6 (j)) shows the best classification effect.

[0113] Figure 7 (a)- Figure 7 (c) shows the change of classification accuracy of ten algorithms when using different proportions of training samples on three different datasets.

[0114] Referring to Figure 8 (a)- Figure 8 (c) shows the change of classification accuracy under different Patch Size sizes. In Figure 8 (a), for the Botswana dataset, the best classification effect is obtained when the Patch Size size is 11x11. In Figure 8 (b), for the Houston2013 dataset, the best classification effect is obtained when the Patch Size size is 9x9. In Figure 9 (c), for the Pavia University dataset, the best classification effect is obtained when the Patch Size size is 13x13.

[0115] Referring to Figure 9 (a)- Figure 9 (c) shows the change of classification accuracy under different Batch Size sizes. In Figure 9 (a), for the Botswana dataset, the best classification effect is obtained when the Batch Size size is 64. In Figure 9 (b), for the Houston2013 dataset, the best classification effect is obtained when the Batch Size size is 64. In Figure 10 (c), for the Pavia University dataset, the best classification effect is obtained when the Batch Size size is 64.

[0116] Referring to Figure 10 (a)- Figure 10 (c) shows the change of classification accuracy under different learning rates. In Figure 10 (a), for the Botswana dataset, the best classification effect is obtained when the learning rate is 2 e-3 .Figure 10 (b) for the Houston 2013 dataset, a learning rate of 2 e-3 obtained the best classification results. ​ (c) for the Pavia University dataset, a learning rate of 2 e-3 obtained the best classification results.

[0117] The above is a specific example of the present embodiment, and is not intended to limit the present embodiment. The hyperspectral image classification method based on the large selection kernel and convolution addition self-attention Mamba provided by the present embodiment is also applicable to the classification of other non-hyperspectral images. Without departing from the essence and scope of the present embodiment, some adjustments and optimizations can be made, and the protection scope of the present embodiment is subject to the claims.

Claims

1. A method of hyperspectral image classification, characterized in that, The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device.

2. The hyperspectral image classification method of claim 1, wherein, The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device.

3. The hyperspectral image classification method of claim 1, wherein, The application relates to a hyperspectral image classification method and device. The input 3-D patch is fed into a 3-D convolutional layer, located at the i th layer, j th The activation value at spatial location (x, y, z) in the feature map is computed as: where φ denotes an activation function, B i,j denotes a bias term, the dimensions of the 3-D convolution kernel are specified by H i ,W i and R i , respectively, representing its height, width and depth, respectively, and denotes the weight corresponding to the position (h', w', r') in the θ th feature map; ω is the weight parameter of the convolution kernel, and v represents the activation value of the previous layer. The application relates to a hyperspectral image classification method and device. Among them, H' i and W' i These represent the height and width of the 2D convolution kernel, respectively. Represents θ th The weights corresponding to the positions (h', w') in the feature map, where B represents the spectral dimension and s is the window length; The application relates to a hyperspectral image classification method and device. U0 = X', wherein, denotes a depthwise convolution operation, assuming D deconstructed convolutional kernels, each of which is subsequently processed through a 1 x 1 convolutional layer, denoted as The features obtained from multiple convolution kernels with different receptive field ranges are merged, and channel max-pooling and average-pooling operations are performed on P max (·) and P avg (·), respectively, in order to effectively capture spatial relationships, and the specific calculation formula includes: wherein SA avg and SA max respectively represent the average and maximum values of the pooled spatial descriptors, represent the concatenation feature tensor obtained after merging the multi-scale features extracted by the convolution kernels of different receptive fields, and further using a convolution layer to connect and convert the pooled features to generate N spatial attention maps, thereby enhancing the interaction of information: In the formula, For spatial attention map; The sigmoid activation function σ(·) is applied to each spatial attention map, i.e. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device.

4. The hyperspectral image classification method of claim 1, wherein, The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device.

5. The hyperspectral image classification method of claim 4, wherein, The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral image classification method and device. The application relates to a hyperspectral Local features of the embedding representation are extracted by convolution operation, and global context information is captured by combining additive self-attention.

6. The hyperspectral image classification method of claim 4, wherein, The processing process of the visual state space sequence module branch specifically comprises the following steps: Layer normalization processing is performed on the input embedding representation, and double-path feature extraction and state space modeling are performed on the normalized data to capture long-range dependencies; wherein the first path captures linear patterns in the data through linear transformation and a nonlinear activation function, and the second path combines linear transformation, deep separable convolution, and an activation function to improve computational efficiency and capture fine-grained spatial features; The outputs of the first path and the second path are merged along the channel dimension to obtain long-range dependency features.

7. The hyperspectral image classification method of claim 1, wherein, The B-spline parameterized learnable activation function transforms the features, specifically comprising the following steps: A regular grid is defined, and the grid step h is calculated: In the formula, G r is the range of the grid, which is [-1, 1] by default, and the grid is generated r is the dimension index of the input feature, G s is a parameter for controlling the size of the grid, D is the dimension of the input feature , and S is the order of the B-spline function; For each input feature Compute its relation to the grid points and compute the function value according to the recursive definition of the B-spline function; For high-order splines, the B-spline function is recursively calculated as: wherein, is the kth order B-spline function, g is a tensor of grid points used to define the discrete points of each feature space, g k is the kth node value in the sequence of grid points, belonging to the discrete grid point tensor g that defines the support domain of the B-spline basis function, is the input feature value; The weights W are represented by a piecewise polynomial spline Further transposition transform on the input Obtaining the final output y out : wherein, is the output calculated by the base weight, is the output obtained by the segmented polynomial weighting, W base is the weight matrix of a base linear transformation, is the transpose of W base , and the final output is y out ∈R N×O , O represents the output feature dimension.

Images