A hyperspectral image classification model

Abstract

The application relates to the technical field of hyperspectral image processing, and more particularly to a hyperspectral image classification model. The model aims to solve the calculation complexity and performance bottleneck in hyperspectral image processing. The model can be applied to the analysis of hyperspectral images in the fields of remote sensing, environmental monitoring, resource exploration and the like. Efficient dimension reduction processing is carried out by adopting random Fourier feature nonlinear principal component analysis (RFF-NLPCA), so that redundant information is effectively reduced and the calculation speed is accelerated. The model introduces light-weight spatial SSM Block and spectral SSM Block modules, and the feature modeling of the spatial dimension and the spectral dimension improves the image classification accuracy and robustness. The gating fusion module (GFM) optimizes the fusion capability of the spatial and spectral features, and improves the discriminant capability of the feature expression. Finally, the model can efficiently process high-dimensional hyperspectral data, provide more accurate classification results, has lower calculation cost and higher performance, and is particularly suitable for processing large-scale hyperspectral image data.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of hyperspectral image processing, and more particularly to a hyperspectral image classification model. BACKGROUND

[0002] Hyperspectral images contain rich spectral information and can provide more details than ordinary RGB images, and have become a key technology in multiple fields, especially in geographic information systems (GIS), agricultural monitoring, mineral exploration and environmental monitoring. Unlike traditional RGB images, hyperspectral images can provide detailed material and substance composition information by capturing a wider spectral band, making them play an important role in resource management, land use, crop health assessment and mineral resource exploration. However, with the increase in the amount and dimension of hyperspectral image data, traditional image processing methods face great challenges in processing these complex data.

[0003] Traditional hyperspectral image classification methods usually rely on manual feature extraction and classic machine learning algorithms such as support vector machines (SVM), decision trees, K-nearest neighbors (KNN), etc. These methods can process hyperspectral data to some extent, but due to the manual intervention in the feature extraction stage, they are difficult to fully exploit the deep information in the hyperspectral data, and have limited modeling ability for the complex spatial-spectral relationship in the hyperspectral image. In addition, these traditional methods have large computational requirements for high-dimensional data and are sensitive to data noise, resulting in their performance often failing to meet the actual demand when facing complex and dynamic hyperspectral images. In recent years, deep learning methods, especially in image classification tasks, have made significant breakthroughs. Convolutional neural networks (CNN) and Transformer models have performed well in image feature extraction and classification tasks, and can automatically learn meaningful features from data, greatly improving classification accuracy. However, although deep learning models have achieved success in image processing, the high dimensionality and complexity of hyperspectral images remain a major challenge. Deep learning models often struggle to handle high-dimensional data, relationships between different bands, and noise in hyperspectral images. Therefore, how to improve the processing capability of deep learning models for hyperspectral images while ensuring computational efficiency has become an urgent research topic. SUMMARY

[0004] The application provides a hyperspectral image classification model that can effectively alleviate the computational complexity and performance bottleneck problems faced by traditional methods in processing hyperspectral images.

[0005] The above object is achieved by the following technical solutions:

[0006] A hyperspectral image classification model comprises a dimension reduction module, a space-spectrum joint preliminary feature extraction module, a space / spectrum dimension feature modeling module and a gated fusion module.

[0007] The dimension reduction module performs hyperspectral image dimension reduction based on nonlinear principal component analysis of random Fourier features.

[0008] The space-spectrum joint preliminary feature extraction module extracts spatial features from the original image in parallel by two-dimensional convolution of two different receptive fields, and models the mutual relationship between spectral channels by using a spectral attention mechanism that fuses different convolution kernel scales.

[0009] The space / spectrum dimension feature modeling module comprises a lightweight spatial VSS module for processing spatial information, and a lightweight spectral VSS module for processing spectral information.

[0010] The gated fusion module fuses spatial and spectral information.

[0011] A hyperspectral image classification method adopts the above-mentioned hyperspectral image classification model and comprises the following steps:

[0012] Step one: the dimension reduction module performs dimension reduction processing on the hyperspectral image by random Fourier feature nonlinear principal component analysis, and the reduced image is cropped into a block region;

[0013] Step two: the block region is sent to the space-spectrum joint preliminary feature extraction module to extract spatial information and spectral information by parallel multi-scale spatial convolution and spectral channel attention mechanism;

[0014] Step three: the space / spectrum dimension feature modeling module suppresses gradient disappearance by a double residual path and realizes space-spectrum feature modeling by combining multi-directional dynamic scanning;

[0015] Step four: the gated fusion module fuses the information of the spatial dimension and the spectral dimension, and the fused features are mapped to the category space by a GAP and MLP module to output probability values of each category.

[0016] The hyperspectral image classification model has the following advantages:

[0017] The application adopts random Fourier feature-based nonlinear principal component analysis (RFF-NLPCA) for efficient high-spectral image dimension reduction, effectively solves the calculation complexity problem of traditional methods in processing nonlinear high-dimensional data, and maintains the spectral discriminant information of the image. Secondly, the model designs a lightweight spatial VSS module and a spectral VSS module, and through the feature modeling of spatial dimension and spectral dimension, the accuracy and robustness of the hyperspectral image classification are greatly improved. Finally, the gating fusion module (GFM) enables the spatial and spectral features to be adaptively fused, significantly enhancing the discriminant ability of the final feature representation and avoiding information loss. These innovative designs enable the model to perform outstandingly in hyperspectral image classification, especially suitable for large-scale hyperspectral data analysis tasks in the fields of remote sensing and environmental monitoring, and have extremely high practical value. The calculation complexity and performance bottleneck encountered by traditional methods in processing hyperspectral images can be effectively overcome. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A hyperspectral image classification model architecture diagram of the application is shown;

[0019] Figure 2 A preliminary feature extraction module schematic diagram is shown;

[0020] Figure 3 A lightweight spatial / spectral VSS module schematic diagram is shown;

[0021] Figure 4 A gating fusion module schematic diagram is shown;

[0022] Figure 5 A training result schematic diagram is shown. DETAILED DESCRIPTION

[0023] A hyperspectral image classification model, combining Figure 1The input hyperspectral image is first processed by random Fourier features-nonlinear principal component analysis (RFF-NLPCA) dimensionality reduction. This dimensionality reduction process combines the nonlinear mapping of random Fourier features and the principal component analysis (PCA) dimensionality reduction method. The reduced image is further cropped into a local P-size block region. By focusing on the local information of the image, the model can more accurately capture the detailed features related to the classification task within the block, effectively reducing redundant information and speeding up the calculation. Subsequently, this block region is sent to the primary feature extraction module (PFE), which is responsible for extracting basic features from the original image and laying the foundation for subsequent spatial and spectral feature extraction.

[0024] To extract more rich spatial and spectral features, the model introduces two core modules: lightweight spatial VSS and lightweight spectral VSS. Lightweight spatial VSS effectively extracts spatial information through spatial dimension state space model, thereby improving the perception ability of spatial structure. At the same time, lightweight spectral VSS focuses on the processing of spectral information, and uses spectral dimension state space model to further enhance the feature expression of spectral dimension.

[0025] To effectively fuse spatial and spectral information, the model uses a gated fusion module (GFM) that finely fuses features from both dimensions, improving the complementarity between them and enhancing the discriminability of the final feature representation. Finally, the fused features are mapped to the class space through the Global Average Pooling (GAP) and Multilayer Perceptron (MLP) modules, outputting the probability value of each class.

[0026] Specifically: 1. Dimensionality reduction: Hyperspectral images usually have tens to hundreds of continuous spectral channels, and their original data can be represented as a three-dimensional tensor with shape (H, W, C), where H and W are the height and width of the image, respectively, and C represents the spectral dimension of each pixel. In the dimensionality reduction process, the hyperspectral image is first flattened into a two-dimensional matrix along the spatial dimension where N = H x W is the total number of pixels, and then RFF-NLPCA maps the spectral vector of each pixel from the original C-dimensional space to a D-dimensional nonlinear feature space through random Fourier features, obtaining

[0027]

[0028] where, is normally distributed is the frequency matrix of the samples, is a bias vector sampled from uniform distribution U(0,2π), which approximates a Radial Basis Function (RBF). According to Bochner's theorem, any positive definite kernel function with shift invariance can be constructed into an equivalent feature mapping through its Fourier transform, so this method can bypass the kernel matrix in computation, explicitly approximating the effect of kernel PCA, i.e.,

[0029] k(x,y)≈φ(x) T φ(y)

[0030] In this explicit mapping space, the centered feature matrix is performed traditional PCA, i.e. by eigen-decomposition of the covariance matrix to get the principal component matrix and project the original data as:

[0031]

[0032] Finally, the reduced matrix Y is reconstructed into three-dimensional structure (H,W,C1), thus completing the nonlinear dimensionality reduction process from the original C channel to the target C1 channel. Compared with traditional PCA which can only make orthogonal projection in the original linear space, or kernel PCA which needs to construct and store a huge kernel matrix, RFF-NLPCA, based on the nonlinear ability of the approximate kernel method, replaces the kernel function with an approximate explicit mapping, reducing the computational complexity from O(N3) to O(ND). It not only can effectively compress high-dimensional spectral data, but also can maintain rich semantic and spectral discriminant information, especially suitable for high-dimensional large-scale data structure such as hyperspectral images.

[0033] 2. To fully exploit the spatial structure information and the correlation between spectral channels contained in the hyperspectral image, the present application designs a space-spectrum joint preliminary feature extraction module before the main deep modeling structure, as shown in Figure 2 . This module uses parallel multi-scale spatial convolution and spectral channel attention mechanism to enhance feature expression ability and maintain the continuity of original information.

[0034] Specifically, let the input feature be First, two-dimensional convolution with two different receptive fields is used to extract spatial features in parallel, and the channel dimension is spliced to get multi-scale features Then, to model the relationship between spectral channels, a spectral attention mechanism that integrates different convolution kernel scales is used, and the multi-scale features F catz is obtained by compressing the spatial dimension through global average pooling, and then a 3x1 and 5x1 one-dimensional convolution is used to extract the local correlation of the spectrum to generate channel attention weight and acts on the feature map:

[0035] F cat =BN(Concat(Conv 3×3 (X),Conv 5×5 (X)))

[0036] z=AvgPool P×P (F cat )

[0037] α=σ(GELU(Conv 3×1 (z)+Conv 5×1 (z)))

[0038] F attn =F cat ⊙α

[0039] Finally, a 1x1 convolution is introduced to form a residual connection, which fuses the results after attention enhancement with the original features to obtain the final output:

[0040] F out =F attn +Conv 1×1 (X)

[0041] 3. Lightweight spatial / spectral VSS module: a spatial / spectral dimension feature modeling module suitable for hyperspectral images, i.e., a lightweight spatial / spectral VSS module, as shown in Figure 3 , which is composed of a spatial / spectral state space model and a double-path residual optimization. Let the input feature of the module be First, it is mapped to the hidden space by the normalization layer (LN) and linear projection (Linear) Then, the local space-spectrum correlation feature is extracted by depthwise separable convolution (DWConv) and activation function SiLU

[0042] X″=SiLU(DWConv(X′))

[0043] For the lightweight spatial VSS module and the lightweight spectral VSS module, after obtaining X″, they are further processed by the state space model respectively to obtain the cross-region long-range dependence and then obtain Y spa and Y spe , after the second layer normalization (LN) and linear dimension reduction, the spectral information integrity is preserved through the residual connection:

[0044]

[0045] Meanwhile, the feed-forward network (FFN) enhances the nonlinear representation ability through channel expansion and contraction operations, and the final output results are:

[0046]

[0047] This module suppresses gradient vanishing through dual residual paths (main residual and FFN residual), and combines multi-directional dynamic scanning to model spatial and spectral features respectively, significantly improving the accuracy and robustness of hyperspectral image classification.

[0048] 4. Gated fusion module: As shown in Figure 4 , the spatial feature and the spectral feature are concatenated in the channel to calculate two gating weights G spa and G spe , which represent the importance of spatial and spectral features in the final fusion. The calculation of the gating weight uses two 1x1 convolution layers, which are applied to the concatenated feature map, and the Sigmoid activation function is used to obtain:

[0049]

[0050] Next, we weight the two branch features according to the gating weights to obtain the fused features:

[0051]

[0052] To further enhance information flow and gradient transmission, we introduce residual connection to the fused features, i.e. by calculating and passing through a 1x1 convolution and ReLU activation function to obtain additional residual features. Finally, the fused features and residual features are added to obtain the final output:

[0053]

[0054] This process effectively avoids information loss and adjusts different features in an adaptive manner. This method not only preserves the complementary information of spatial and spectral features, but also enhances the representation ability of the fused features.

[0055] 5.In the training process of the hyperspectral image classification model of the present application, the Adam optimizer is used, with an initial learning rate of 0.001 and a dynamic adjustment strategy learning rate decay to ensure the stability and convergence of the training. The loss function uses the cross-entropy loss function, aiming to measure the difference between the predicted results and the actual labels, and to promote the effective optimization of the model. In addition, L2 regularization (weight decay) and Dropout are applied in the training process to prevent overfitting and improve the generalization ability of the model. The training batch size is set to 64 and adjusted according to the memory usage during the training process, with ten-fold cross-validation and 120 rounds per training round. For the Indian Pines, Pavia University, Salinas data sets, the block size P is set to 6, and the OA, AA, Kappa coefficient are used to measure the learning effect of the model, as shown in Figure 5 the final training results.

Claims

1. A method of hyperspectral image classification, characterized in that, The method comprises the following steps: Step 1: a dimension reduction module reduces the hyperspectral image through random Fourier feature nonlinear principal component analysis, and the reduced image is cropped into a block region; Step 2: the block region is sent to a space-spectrum joint preliminary feature extraction module, spatial information and spectral information are extracted through parallel multi-scale spatial convolution and spectral channel attention mechanism; Step 3: a space / spectrum dimension feature modeling module suppresses gradient disappearance through a double residual path, and combines multi-directional dynamic scanning to respectively realize space / spectrum feature modeling; Step 4: a gating fusion module fuses the space dimension and the spectrum dimension information, and maps the fused features to a category space through a GAP and an MLP module to output probability values of each category. The original data of the hyperspectral image is represented as a three-dimensional tensor with a shape of (H, W, C), wherein H and W are the height and width of the image respectively, and C represents the spectral dimension of each pixel point. A1. Flattening the hyperspectral image along the spatial dimension into a two-dimensional matrix wherein is the total number of pixels, and then the random Fourier feature nonlinear principal component analysis maps the spectral vector of each pixel from the original C-dimensional space to a D-dimensional nonlinear feature space, resulting in: x = x1, x2,..., xC where, is normally distributed a frequency matrix of samples, is uniformly distributed a bias vector sampled from a uniform distribution, computationally bypasses the kernel matrix, explicitly approximates the effect of kernel PCA, i.e.: A2, centering the feature matrix in this explicit mapping space by eigen-decomposition of the covariance matrix to obtain the principal component matrix and project the original data to: A3, the reduced matrix Y is reconstructed into a three-dimensional structure (H, W, C1).

2. The hyperspectral image classification method according to claim 1, in step 2: B1, the input feature is Two-dimensional convolution with two different receptive fields is adopted to extract spatial features in parallel, and the multi-scale features are obtained by concatenating in the channel dimension ; B2, multi-scale features The spatial dimension is compressed by global average pooling to obtain z, and then a 3x1 and 5x1 one-dimensional convolution is used to extract the spectral local correlation to generate channel attention weights , and acts on the feature map: B3, a 1x1 convolution is introduced to form a residual connection, the attention-enhanced result is fused with the original feature to obtain a final output: 。 3. The hyperspectral image classification method according to claim 2, in step 3: The spatial / spectral dimension feature modeling module is respectively composed of a lightweight spatial VSS module and a lightweight spectral VSS module, and the module input features , first mapped to a hidden space through a normalization layer and a linear projection , then local space-spectrum correlation features are extracted through a depth separable convolution and an activation function SiLU: After obtaining , both are further processed by state space model to obtain cross-area long-range dependencies, and then and , after the second layer normalization (LN) and linear dimension reduction, the spectral information integrity is preserved through residual connection: Meanwhile, a feedforward network (FFN) enhances the nonlinear expression ability through channel expansion and contraction operations, and finally outputs results respectively: 。 4. The hyperspectral image classification method of claim 3, in step three, spatial features and spectral features are concatenated, and two gating weights 、 are calculated, representing the importance of spatial and spectral features in the final fusion respectively; the gating weights are calculated using two 1x1 convolution layers, which are applied to the concatenated feature maps, and are passed through a Sigmoid activation function to obtain: The two branch features are weighted and fused according to the gating weight to obtain fused features: 。 5. The hyperspectral image classification method of claim 4, by calculating and passing through a 1x1 convolution and a ReLU activation function, obtaining additional residual features, finally, fusing the features and the residual features and adding them to obtain the final output: 。 6. A hyperspectral image classification model, characterized in that, The hyperspectral image classification method according to any one of claims 1 to 5 comprises a dimension reduction module, a space-spectrum joint preliminary feature extraction module, a space / spectrum dimension feature modeling module and a gating fusion module.

7. The hyperspectral image classification model according to claim 6, wherein the dimension reduction module reduces the hyperspectral image based on random Fourier feature nonlinear principal component analysis.

8. The hyperspectral image classification model according to claim 7, wherein the space-spectrum joint preliminary feature extraction module extracts spatial features from the original image in parallel through two-dimensional convolutions with different receptive fields, and models the mutual relationship between spectral channels through a spectral attention mechanism that fuses different convolution kernel scales.

9. The hyperspectral image classification model according to claim 8, wherein the space / spectrum dimension feature modeling module comprises a lightweight space VSS module for processing spatial information, and a lightweight spectrum VSS module for processing spectral information.

10. The hyperspectral image classification model according to claim 9, wherein the gating fusion module fuses the spatial and spectral information.

Images