Hyperspectral image mineral fine classification method based on spatial-spectral component representation

Abstract

The invention discloses a hyperspectral image mineral fine classification method based on spatial-spectral component representation, and relates to the technical field of image classification in computer vision, and the method comprises the following steps: obtaining a Gaofen-5 remote sensing image, and carrying out data preprocessing to obtain hyperspectral data; inputting the hyperspectral data into a demixing module based on an auto-encoder to generate abundance features; the hyperspectral data are input into a classifier composed of a multi-scale cavity convolution module and a Transform module based on a double-flow attention mechanism, and deep classification features are generated; fusing the abundance features and the deep classification features, inputting the fused features into a spatial-spectral adaptive fusion module, modeling dependence between spectral channels, and capturing local spatial context information; and finally, obtaining a classification result by using a full connection layer. According to the invention, the hyperspectral image mineral fine classification can be carried out in a mineral scene with serious mixed pixels.

Description

Technical Field

[0001] This invention relates to the field of image classification technology in computer vision, specifically to a method for fine classification of minerals in hyperspectral images based on spatial spectral component representation. Background Technology

[0002] Hyperspectral remote sensing technology can acquire continuous and high-resolution spectral information. Mineral mapping is an important research direction in the field of hyperspectral remote sensing, with the main goal of fully utilizing spectral information to characterize the compositional properties of surface minerals. To gain a deeper understanding of mineral distribution within a study area, fine classification of hyperspectral remote sensing images has become a key research topic. However, mining areas often exhibit severe pixel mixing and mineral co-occurrence, making it difficult to characterize the mineral composition of the study area solely through classification networks, thus limiting the development of refined mineral classification.

[0003] In the field of hyperspectral mineral classification, the most widely used methods are based on the physicochemical properties of minerals, primarily relying on spectral curve features for classification. Examples include Spectral Angle Mapping (SAM) and Mixture Tuned Matched Filtering (MTMF). Oskyay et al. used the SAM algorithm on HyMap hyperspectral data to identify and extract hematite in the southeastern Colorado Plateau region of Utah. Neleto et al. applied the MTMF algorithm to the Pedra Branca gold mine in Borborema Province, Brazil, successfully extracting non-tremolite and kaolinite. These methods based on mineral physicochemical properties are simple to operate and computationally efficient, suitable for rapid identification of minerals over large areas; however, their ability to handle severe pixel mixing and distinguish spectrally similar minerals is limited. (Pedra Branca, Borborema Province, Brazil)

[0004] In recent years, many scholars have proposed machine learning-based methods for hyperspectral mineral identification and classification, such as Support Vector Machines (SVM) and Random Forest (RF). While these algorithms have achieved promising results in mineral identification and classification, most focus solely on spectral information, neglecting spatial location features. Their performance is less than satisfactory in scenarios where minerals are scattered and complex. With the development of deep learning, methods based on autoencoders, convolutional neural networks (CNNs), graph convolutional networks (GCNs), generative adversarial networks (GANs), and Transformer models are increasingly widely used in mineral identification and classification. Deep learning models can automatically learn high-level and highly discriminative features from raw hyperspectral data through multi-layered nonlinear structures, effectively extracting the absorption and reflection characteristics of minerals, thereby reducing reliance on prior knowledge.

[0005] While deep learning-based methods can achieve high accuracy in mineral identification and classification, they primarily output discrete labels for minerals and tend to predict the dominant mineral in mixed pixels, making it difficult to identify secondary minerals. Furthermore, these methods require a large number of labeled samples, which can lead to overfitting and decreased generalization ability. Abundance features describe the relative contributions of different endmember components within a pixel, thus accurately representing the proportion of each mineral in a mixed mineral assemblage. This quantitative information not only supplements the discrete output of traditional classification methods but also provides guidance for identifying spectrally mixed minerals. Researchers have developed various spectral unmixing methods for mineral identification and classification, including sparse autoencoders and deep autoencoders with multiple hidden layers. Numerous studies have shown that deep autoencoders can effectively extract spectral information of minerals, suppress noise, and mitigate the impact of outliers on mineral classification, demonstrating significant advantages over traditional methods in comparative analyses. Fang et al. used PRISMA hyperspectral imagery and a Bayesian convolutional unmixing network to classify minerals in a chalcopyrite mining area in Nevada. They also designed an ablation experiment to verify that the abundance features generated by the unmixing network can effectively improve the accuracy of mineral classification. Abundance characteristics obtained through unmixing can characterize the relative proportions of minerals; however, their performance is highly dependent on the selection of end-member spectra and the validity of model assumptions. When mineral end-members are incomplete or spectral variations are significant, abundance estimations are prone to bias.

[0006] In summary, relying solely on classification networks is insufficient to characterize the mineral composition within pixels, while abundance features generated using only unmixing networks lack high-level semantic constraints. Achieving refined mineral classification in mineral study areas with mixed spectra and fragmented mineral distributions remains a challenge. Summary of the Invention

[0007] This invention addresses the aforementioned challenges by proposing a high-resolution mineral classification method based on spatial-spectral component representation from hyperspectral images. The proposed model integrates an autoencoder-structured unmixing network and a deep classification network, achieving a complementary fusion of physically interpretable abundance information and deep spectral-spatial classification information. Furthermore, traditional convolutional structures commonly used in existing classification models often struggle to capture fine mineral textures and large-scale geological structural information, resulting in poor performance in complex surface environments. This invention proposes a dilated convolution employing multi-scale void ratios, aiming to simultaneously capture both local detailed textures and large-scale mineral distribution. This design enables the network to learn multi-scale contextual information more efficiently, thereby improving the model's classification performance and robustness. Extensive experiments on a prepared mineral dataset demonstrate that the proposed method significantly outperforms existing state-of-the-art methods in challenging mineral study areas, achieving higher classification accuracy.

[0008] The technical solution of the present invention is as follows:

[0009] A method for fine mineral classification based on hyperspectral imagery using spatial spectral component representation includes the following steps:

[0010] Step 1: Acquire Gaofen-5 remote sensing images of the study area. After data preprocessing such as radiometric correction, atmospheric correction, bad band removal and orthorectification, hyperspectral data is produced based on the SAM algorithm and with reference to existing mapping products in the study area. The hyperspectral data is then used to extract image blocks. The label of each image block is determined by the category label of the center pixel. Then, training sample set and test sample set are randomly divided according to a fixed number of each category.

[0011] Step 2: Construct an autoencoder-based demixing module, using multi-layer convolution, batch normalization, and ReLU activation function to capture the dependencies between pixels and extract abundance features;

[0012] Step 3: Construct a multi-scale dilated convolution module. By introducing dilated convolution with different dilation rates, the local texture details and large-scale spatial distribution information of the minerals can be extracted simultaneously.

[0013] Step 4: Construct a Transformer module based on a two-stream attention mechanism. By introducing a two-stream attention mechanism, while maintaining the high-quality contextual semantic information extracted by dilated convolution, long-distance dependencies covering the entire image are established.

[0014] Step 5: Construct a spatial-spectral adaptive fusion module to fuse the abundance features extracted in Step 2 with the deep classification features output by the dual-stream attention Transformer extracted in Step 5, model the spectral dependence between spectral channels and capture local spatial context information.

[0015] Step 6: Construct a fine mineral classification model based on spatial spectral component representation of hyperspectral images:

[0016] The modules described in steps 3 and 4 are combined to form a classifier. The unmixing module from step 2 is combined with the classifier to form a two-stage classification model. The image patch data is input into the model, and through a stitching operation, the abundance features generated by the unmixing module and the deep classification features generated by the classifier are effectively fused to obtain the features. , will feature The data is input into the spatial spectrum adaptive fusion module to obtain the final classification features, and a linear layer is used to obtain the classification result.

[0017] Step 7: Train the model using reconstruction loss and cross-entropy loss until the model converges to obtain a trained model. Input the test sample set into the trained model to obtain the mineral classification results.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] First, this invention uses a two-stage classification architecture, which effectively integrates abundance features and deep powder features, avoiding the limitation of relying solely on classification networks to characterize the mineral composition inside pixels, and enabling the network to obtain higher-level semantic information constraints.

[0020] Secondly, by introducing a multi-scale dilated convolution module and using dilated convolution with different dilation rates, this invention can take into account both the texture details of minerals and the spatial distribution information at a large scale, effectively improving the classification of hyperspectral images of mineral scenes with severe mineral mixing and mixed pixel phenomena.

[0021] Third, this invention uses a spatial-spectral adaptive fusion module to further express the features after the fusion of abundance features and classification features, which can model the spectral dependencies between spectral channels and capture local spatial context information. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the architecture of a hyperspectral image mineral fine classification model based on spatial spectral component representation as described in this embodiment of the invention.

[0023] Figure 2 This is a schematic diagram of the two-stream attention mechanism described in an embodiment of the present invention.

[0024] Figure 3The images shown are false-color composite images and real ground feature images from the TAMSAT dataset used in the embodiments of the present invention.

[0025] Figure 4 The image shows the classification results of various classification methods on the TAMSAT dataset in the embodiments of the present invention. Detailed Implementation

[0026] To better understand the above technical solutions, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0027] Specifically, Figure 1 This is a schematic diagram of a hyperspectral image mineral fine classification model based on spatial spectral component representation, as described in an embodiment of the present invention. Figure 1 As shown, the method for fine mineral classification based on spatial spectral composition representation in hyperspectral images includes the following steps:

[0028] Step 1: Acquire Gaofen-5 remote sensing images of the study area. After data preprocessing such as radiometric correction, atmospheric correction, bad band removal and orthorectification, create hyperspectral data TAMSAT based on the SAM algorithm and referencing existing mapping products of the study area. Extract image patches from the hyperspectral data. The label of each image patch is determined by the category label of the center pixel. Then, randomly divide the training sample set and test sample set according to a fixed number of each category.

[0029] The hyperspectral data used in this embodiment is GF-5 hyperspectral imagery acquired on September 30, 2019, in the Tamusu region of Inner Mongolia. The wavelength range is 400nm-2500nm, the spatial resolution is 30m, the spectral resolution is better than 5nm, and the spatial size is 252×772. After data preprocessing to remove bands affected by atmospheric conditions and those with low signal-to-noise ratios, 283 bands were retained for subsequent classification. The study area is roughly distributed with four minerals: high-alumina sericite, medium-alumina sericite, low-alumina sericite, and chlorite. After SAM mapping and comparison with existing mapped data products for correction, the true land cover labels of the hyperspectral data were obtained to evaluate classification performance.

[0030] Step 2: Input the hyperspectral dataset into an autoencoder-based demixing module to extract abundance vectors and generate abundance features. This module consists of stacked multi-layer convolutions, batch normalization, and ReLU activation functions, designed to capture spectral mixing relationships between pixels and generate interpretable abundance features. Specifically, given a hyperspectral vector... The formula is as follows:

[0031]

[0032] in: It is a set of weights and bias matrices. This represents the ReLU activation function. To ensure the abundance has practical significance, a non-negative sum constraint is added. The generated abundance features are then used for subsequent processing.

[0033] Step 3: Construct a multi-scale void convolution module. In the mineral study area, minerals are not isolated point distributions, but rather form patchy or banded spatial continuums influenced by topographic undulations. Furthermore, the mining area's topography is fragmented, resulting in scattered ore bodies. To address these issues, void convolution with different void ratios is introduced, which can capture both small-scale ore body texture information and large-scale geological structural information.

[0034] In the dilated convolution module, a 3D dilated convolution with a 3×3×3 kernel and a dilation rate of 2 is first used. This operation expands the receptive field to 5×5×5 without loss of resolution. Subsequently, a 2D dilated convolution with a 3×3 kernel and a dilation rate of 3 is performed on the output feature map of the 3D convolution to further capture long-range spatial dependencies and output features. The formula is as follows:

[0035]

[0036] in: and These represent a 3D dilated convolution with a dilation rate of 2 and a 2D dilated convolution with a dilation rate of 3, respectively.

[0037] Step 4: Construct a Transformer module based on a two-stream attention mechanism. By introducing a two-stream attention mechanism, while maintaining the high-quality contextual semantic information extracted by dilated convolution, long-distance dependencies covering the entire image are established.

[0038] Although dilated convolution significantly expands the receptive field, enabling the capture of long-distance mineral spatial distribution within a mining area, it remains limited by local inductive bias, which restricts its ability to model global spectral dependencies. To overcome this limitation, a Transformer module based on a two-stream attention mechanism is proposed. The framework diagram of the two-stream attention mechanism is shown below. Figure 2 As shown, the dual-stream attention Transformer module takes the features output by the dilated convolution as input.

[0039] First, average pooling and linear projection together generate vectors. At the same time, the features After average pooling, 1×1 convolution, layer normalization, and GELU activation function, a vector is generated. and For vectors Attention mechanism computation is performed to obtain global features. global features With features Element-wise multiplication is performed to obtain the output characteristics of the module. This approach preserves the high-quality contextual semantic information extracted by dilated convolution while establishing long-range dependencies covering the entire image. The formula is as follows:

[0040]

[0041]

[0042]

[0043]

[0044] in: Indicates linear projection. This indicates the average pooling operation. Representation layer normalization, This indicates element-wise multiplication.

[0045] Step 5: Construct a spectral adaptive fusion module to fuse the abundance features extracted in Step 2 with the deep classification features output by the Transformer dual-stream attention mechanism in Step 5, model the spectral dependencies between spectral channels and capture local spatial context information.

[0046] The spectral adaptive fusion module is designed as the final feature recalibration stage of the network. The module consists of two parallel paths: a 1×1 convolution and a 3×3 convolution. The 1×1 convolution path focuses on modeling the dependencies between spectral channels, while the 3×3 convolution path captures local spatial context information. Both paths are processed using batch normalization and activation functions. The module's input data is features. Features are generated through two parallel paths. and The two features are concatenated to obtain the final output features of the module. This design enables the model to dynamically balance global spectral discrimination and local spatial detail while avoiding the vanishing gradient phenomenon found in deep networks. The formula is as follows:

[0047]

[0048]

[0049]

[0050] in: Represents a 1×1 convolution. This represents a 3×3 convolution. This indicates batch normalization.

[0051] Step 6: Construct a fine mineral classification model based on spatial spectral component representation of hyperspectral images:

[0052] The modules described in steps 3 and 4 are combined to form a classifier. The unmixing module from step 2 is combined with the classifier to form a two-stage classification model. The image patch data is input into the model, and through a stitching operation, the abundance features generated by the unmixing module and the deep classification features generated by the classifier are effectively fused to obtain the features. , will feature The data is input into the spatial spectrum adaptive fusion module to obtain the final classification features, and a linear layer is used to obtain the classification result.

[0053] Step 7: Train the model using reconstruction loss and cross-entropy loss until the model converges, obtaining a trained model. The loss function is composed of reconstruction loss L. RE With cross-entropy loss L CE The composition, formula is as follows:

[0054]

[0055]

[0056]

[0057] in: Indicates the number of training samples. and These represent the first and second parts of the original input hyperspectral data and the reconstructed hyperspectral data, respectively. 1 pixel, and They represent the first The true value and predicted probability of each pixel It is a hyperparameter used to balance the importance of different loss functions, and its value range is [0,1].

[0058] The test sample set is input into the trained model to obtain the mineral classification results.

[0059] To quantitatively evaluate the classification performance of the proposed model, three evaluation metrics were used: overall classification accuracy, average classification accuracy, and Kappa coefficient. Higher values ​​indicate better classification accuracy. These three metrics were calculated using a confusion matrix, a format for representing the classification accuracy of hyperspectral images. The specific form of the confusion matrix is ​​as follows:

[0060]

[0061] In the matrix This represents the number of pixels in class i that are misclassified as class j, n represents the total number of classes in the sample, and N is the total number of samples. This represents the number of correctly classified pixels; a larger value indicates higher classification accuracy. The confusion matrix provides the overall classification accuracy, average classification accuracy, and Kappa coefficient.

[0062] Overall classification accuracy refers to the proportion of samples correctly classified by the model, which can be derived from the confusion matrix using the following formula:

[0063]

[0064] Average classification accuracy refers to the average accuracy of a classification model across all categories. It is mainly used to evaluate the model's average performance across each category. A higher value indicates a higher classification accuracy. The formula is as follows:

[0065]

[0066] The Kappa coefficient is a method used to measure consistency. In practical classification problems, the Kappa coefficient usually falls between [0,1]. A higher Kappa coefficient indicates a higher classification accuracy of the model. The formula is as follows:

[0067]

[0068] The experimental GPU hardware was an NVIDIA RTX 5060Ti, and the PyTorch 3.7 environment was used. The network was optimized using the Adam optimizer, with 200 iterations, an initial learning rate of 0.001, and a batch size of 32. All experiments were repeated 10 times. The training and test sample settings of the dataset are shown in Table 1.

[0069] Table 1. Number of training and test samples in the TAMSAT dataset

[0070]

[0071] To verify the classification performance of the model proposed in this invention, comparative experiments were conducted using traditional classification methods as well as advanced convolutional neural networks and Transformer methods. The comparative methods included SVM, HybridSN, SSFTT, GAHT, DSNet, and DSFormer. The code for all the comparative methods has been open-sourced on relevant websites.

[0072] Table 2. Classification results of different methods on the TAMSAT dataset

[0073]

[0074] Figure 4 This document presents the classification results of various methods on the TAMSAT dataset, including SVM, HybridSN, SSFTT, GAHT, DSNet, DSFormer, and the present invention. Visually, traditional support vector machine methods suffer from blurred boundaries and significant noise in the classification images due to their inability to effectively model spatial-spectral joint information, and severe confusion among the three types of sericite. HybridSN and SSFTT algorithms significantly improve performance compared to support vector machine algorithms through 3D convolution or Transformer-based feature extraction; however, mineral fragmentation remains, particularly noticeable in medium-aluminum and low-aluminum sericite. The GAHT algorithm further enhances spatial consistency, but local classification errors still exist in mineral-mixed regions. DSNet and DSFormer models more effectively maintain spatial continuity and have relatively clear boundaries, but limitations remain in transitional mineralization zones with subtle spectral differences. The model proposed in this invention yields classification images with clearer boundaries and better mineral separability. This is thanks to the introduction of an autoencoder-based demixing module, which introduces physically meaningful spectral composition representations, effectively mitigating interference from mixed pixels. The classification boundaries of the three types of sericite are clearer, the overall texture is smoother, and the salt-and-pepper noise phenomenon is effectively mitigated, demonstrating significant spatial consistency and spectral discriminability.

[0075] Table 2 presents the quantitative results of different model methods, with the best results highlighted in bold. The model proposed in this invention performs best across all evaluation metrics. Specifically, the overall classification accuracy reaches 97.92%, an improvement of 0.68% compared to the DSFormer method and 9.53% compared to the traditional support vector machine algorithm. The average classification accuracy reaches 96.26%, surpassing all compared methods, indicating stable performance across different categories. Furthermore, the Kappa coefficient reaches 97.25%, confirming the robustness of the proposed model. In summary, the model proposed in this invention demonstrates significant advantages in both qualitative and quantitative aspects.

[0076] This demonstrates that by effectively integrating physically interpretable abundance information with data-driven deep classification features, the model proposed in this invention exhibits superior performance in mineral scenes with severe pixel mixing, providing a more robust and reliable solution for refined hyperspectral image mineral classification.

Claims

1. A hyperspectral image mineral fine classification method based on spectral-spatial component representation, characterized in that, The method comprises the following steps: Step 1: Obtain the high-resolution remote sensing image of the study area, and after radiation correction, atmospheric correction, bad wave band elimination and orthorectification and other data preprocessing, according to the spectral angle mapping (Spectral Angle Mapping, SAM) algorithm and referring to the existing mapping product of the study area, a hyperspectral data TAMSAT is made, the hyperspectral data is extracted in image blocks, the label of each image block is determined by the class label of the center pixel, and then a fixed number of each class is randomly divided into a training sample set and a test sample set; Step 2: Construct a demixing module based on an autoencoder, use multi-layer convolution, batch normalization and ReLU activation function to capture the dependency between pixels and extract abundance features; Step 3: Construct a multi-scale dilated convolution module, introduce dilated convolution with different hole rates, and simultaneously extract local texture details and large-scale spatial distribution information of minerals; Step 4: Construct a Transformer module based on a dual-flow attention mechanism, introduce a dual-flow attention mechanism to maintain high-quality context semantic information extracted by the dilated convolution, and establish long-distance dependency covering the entire image; Step 5: Construct a spectral-spatial adaptive fusion module, fuse the abundance features extracted in step 2 and the deep classification features output by the dual-flow attention Transformer in step 5, model the dependency between spectral channels and capture local spatial context information; Step 6: Construct a hyperspectral image mineral fine classification model based on spectral-spatial component representation: The modules in steps 3 and 4 are combined to form a classifier, the unmixing module in step 2 and the classifier are combined to form a two-stage classification model, the image block data is input into the model, the abundance features generated by the unmixing module are effectively fused with the deep classification features generated by the classifier through a splicing operation, and the feature is obtained is input into a spectral self-adaptive fusion module to obtain a final classification feature, and a linear layer is used to obtain a classification result; Step 7: Train the model using reconstruction loss and cross-entropy loss until the model converges, obtain the trained model, input the test sample set into the trained model, and obtain the mineral classification result.

2. The method according to claim 1, wherein, Hyperspectral data wherein represents the spatial dimension, C represents the number of bands, the multi-scale dilated convolution module input into the step 3, the process is as follows: First, a three-dimensional dilated convolution is used with a convolution kernel of 3x3x3 and a dilation rate of 2 to expand the receptive field without losing resolution, obtaining three-dimensional dilated convolution output features, and then, two-dimensional dilated convolution is performed on the three-dimensional dilated convolution output features with a convolution kernel size of 3x3 and a dilation rate of 3 to further capture long-distance spatial dependencies, obtaining output features , and the formula is as follows: wherein: and respectively denote a 3D dilated convolution with a dilation rate of 2 and a 2D dilated convolution with a dilation rate of 3.

3. The hyperspectral mineral fine classification method based on the spectral feature representation of claim 1, wherein, In step 4, the Transformer module based on the dual-flow attention mechanism is constructed as follows: First, the average pooling and linear projection are used to generate vector Q together, and the features After average pooling, 1x1 convolution, layer normalization and GELU activation function, vectors K and V are generated, and the attention mechanism operation is performed on vectors Q, K and V to obtain global feature The global feature is multiplied element by element with the feature to obtain the output feature of the module, and the formula is as follows: wherein: represents a linear projection, represents a mean pooling operation, represents layer normalization, represents an element-wise multiplication.

4. The hyperspectral mineral fine classification method based on the spectral feature representation of claim 1, wherein, In step 5, the spectral-spatial adaptive fusion module is constructed as follows: The module is composed of two parallel paths of 1x1 convolution and 3x3 convolution, the 1x1 convolution path focuses on modeling the dependency between spectral channels, and the 3x3 convolution path is used to capture local spatial context information, both paths are processed using batch normalization and activation functions, and the input data of the module is the feature , the feature and are generated through the two parallel paths respectively, and the final output feature of the module is obtained by splicing the two features , and the formula is as follows: where: denotes a 1 x 1 convolution, denotes a 3 x 3 convolution, denotes batch normalization.

Images