A hyperspectral image reconstruction method and system

Abstract

The application discloses a hyperspectral image reconstruction method and system, and the method comprises the following steps: acquiring a hyperspectral image, obtaining a spectral matrix of the hyperspectral image, and any row vector of the spectral matrix being a spectral vector of a corresponding pixel in the hyperspectral image; inputting the spectral matrix into a preset encoder to output a low-dimensional representation of each row vector of the spectral matrix; constructing a first preset image, the data of any pixel of the first preset image being formed by the low-dimensional representation of a corresponding row vector in the spectral matrix, inputting the first preset image into a preset model to output a second preset image, and the resolution of the second preset image being higher than that of the first preset image; inputting the second preset image into a preset decoder to output a third preset image, and the number of channels of the data of any pixel of the third preset image being greater than that of the low-dimensional representation of a corresponding row vector in the spectral matrix, so as to obtain a reconstructed hyperspectral image according to the third preset image. The application can obtain a hyperspectral image with improved spatial resolution.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a hyperspectral image reconstruction method and system. Background Technology

[0002] The spatial resolution of mineral grain images in rock thin sections directly affects the accuracy of fine structure analysis and composition identification of mineral grains. Limited by factors such as the physical resolution, noise control, and data acquisition speed of hyperspectral imaging instruments, actual acquired hyperspectral images of rock thin sections often suffer from insufficient spatial resolution and blurred details, making it difficult to accurately identify the boundaries and microstructures of mineral grains. This not only limits the application depth of hyperspectral image processing in mineral composition analysis, quantitative analysis of grain morphology, and reservoir microstructure research, but also significantly impacts subsequent tasks such as grain segmentation and mineral classification. Summary of the Invention

[0003] The purpose of this invention is to provide a hyperspectral image reconstruction method and system that can reconstruct hyperspectral images with improved spatial resolution.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A hyperspectral image reconstruction method, comprising:

[0006] A hyperspectral image is acquired, and a spectral matrix of the hyperspectral image is obtained. Any row vector of the spectral matrix is ​​the spectral vector of the corresponding pixel in the hyperspectral image. Any pixel of the hyperspectral image includes data from multiple spectral channels. The spectral vector of a pixel in the hyperspectral image is formed by the data from the multiple spectral channels of the pixel in the hyperspectral image.

[0007] The spectral matrix is ​​input into a preset encoder, and a low-dimensional representation of each row vector of the spectral matrix is ​​output. The number of channels in the low-dimensional representation of any row vector is less than the number of channels in the spectral vector of the pixel corresponding to any row vector.

[0008] A first preset image is constructed, the resolution of the first preset image is the same as that of the hyperspectral image, the data of any pixel of the first preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix, and the first preset image is input into a preset model to output a second preset image, the resolution of the second preset image is higher than that of the first preset image;

[0009] The second preset image is input into a preset decoder, and a third preset image is output. The resolution of the third preset image is the same as that of the second preset image. The number of channels of the data of any pixel in the third preset image is greater than the number of channels of the low-dimensional representation of the corresponding row vector in the spectral matrix, so as to obtain a reconstructed hyperspectral image based on the third preset image.

[0010] Optionally, training the preset encoder and the preset decoder includes:

[0011] Acquire hyperspectral image samples, obtain the spectral matrix of the hyperspectral image samples, and perform a noise addition operation on the spectral matrix of the hyperspectral image samples to obtain a noisy spectral matrix of the hyperspectral image samples;

[0012] The preset encoder and the preset decoder are trained based on the spectral matrix and the noisy spectral matrix of the hyperspectral image sample. The preset encoder takes the noisy spectral matrix as input and outputs a low-dimensional representation of each row vector of the noisy spectral matrix. The preset decoder takes the low-dimensional representation of each row vector of the noisy spectral matrix as input and outputs the reconstructed spectral vector of each pixel of the hyperspectral image sample. The number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix is ​​less than the number of channels in the row vector of the noisy spectral matrix, and the number of channels in the reconstructed spectral vector of the pixel of the hyperspectral image sample is greater than the number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix.

[0013] Optionally, the goal of training the preset encoder and the preset decoder is to minimize the difference between the reconstructed spectral vector of the pixels of the hyperspectral image sample and the original spectral vector of the pixels of the hyperspectral image sample.

[0014] Optionally, training the preset model includes:

[0015] Acquire hyperspectral image samples, obtain the spectral matrix of the hyperspectral image samples, and input the spectral matrix of the hyperspectral image samples into the preset encoder to output a low-dimensional representation of each row vector of the spectral matrix of the hyperspectral image samples;

[0016] A fourth preset image is constructed, the resolution of which is the same as that of the hyperspectral image sample, and the data of any pixel of the fourth preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix of the hyperspectral image sample;

[0017] The fourth preset image is downsampled to obtain a fifth preset image. The resolution of the fifth preset image is lower than that of the fourth preset image. The preset model is trained based on the fourth preset image and the fifth preset image. The preset model takes the fifth preset image as input and the reconstructed image as output.

[0018] Optionally, the loss function used to train the preset model represents the difference between the reconstructed image and the fourth preset image.

[0019] Optionally, the preset model includes a first convolutional layer, a convolutional + residual network, an upsampling layer, and a second convolutional layer connected in sequence. The first preset image input to the preset model is first convolved by the first convolutional layer, then processed by the convolutional + residual network, further processed by the upsampling layer, and then processed by the second convolutional layer.

[0020] Optionally, obtaining the reconstructed hyperspectral image based on the third preset image includes:

[0021] For any spectral channel of any pixel in the third preset image, the data of the spectral channel of the pixel in the third preset image is convolved with a preset Laplacian convolution kernel to obtain the detail enhancement component of the spectral channel of the pixel in the third preset image. The detail enhancement component is then superimposed with the data of the spectral channel of the pixel in the third preset image according to a preset weight to obtain the enhanced data of the spectral channel of the pixel in the third preset image.

[0022] Optionally, obtaining the reconstructed hyperspectral image based on the third preset image further includes:

[0023] For any spectral channel of any pixel in the third preset image, if the enhanced data of this spectral channel of this pixel in the third preset image is less than the minimum value of a preset interval, then the minimum value of the preset interval is taken as the data of this spectral channel of this pixel; if the enhanced data of this spectral channel of this pixel in the third preset image is within the preset interval, then the enhanced data of this spectral channel of this pixel is taken as the data of this spectral channel of this pixel; if the enhanced data of this spectral channel of this pixel in the third preset image is greater than the maximum value of the preset interval, then the maximum value of the preset interval is taken as the data of this spectral channel of this pixel.

[0024] Optionally, before obtaining the spectral matrix of the hyperspectral image, the method further includes:

[0025] For any pixel in the hyperspectral image, the data of the multiple spectral channels in the spectral vector of this pixel are arranged in ascending order to obtain a sorted vector; the quantile of the data of each spectral channel in the sorted vector is calculated, and the quantile corresponding to each spectral channel is mapped to the interval [0, 1]; the mapped values ​​corresponding to the multiple spectral channels are arranged in the original spectral channel order to obtain a normalized spectral vector.

[0026] A hyperspectral image reconstruction system, comprising:

[0027] Memory, used to store computer programs;

[0028] A processor for executing the computer program to implement the steps of the hyperspectral image reconstruction method as described in any of the preceding claims.

[0029] As can be seen from the above technical solution, the hyperspectral image reconstruction method and system provided by the present invention includes: acquiring a hyperspectral image, obtaining a spectral matrix of the hyperspectral image, wherein any row vector of the spectral matrix is ​​the spectral vector of the corresponding pixel in the hyperspectral image, any pixel of the hyperspectral image includes data of multiple spectral channels, and the spectral vector of the pixel of the hyperspectral image is formed by the data of the multiple spectral channels of the pixel of the hyperspectral image; inputting the spectral matrix into a preset encoder, outputting a low-dimensional representation of each row vector of the spectral matrix, wherein the number of channels in the low-dimensional representation of any row vector is less than the number of channels in the spectral vector of the corresponding pixel of any row vector; constructing a first pre-encoder; Given an image, a first preset image has the same resolution as a hyperspectral image. The data of any pixel in the first preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix. The first preset image is input into a preset model, and a second preset image is output. The resolution of the second preset image is higher than that of the first preset image. The second preset image is input into a preset decoder, and a third preset image is output. The resolution of the third preset image is the same as that of the second preset image. The number of channels of the data of any pixel in the third preset image is greater than the number of channels of the low-dimensional representation of the corresponding row vector in the spectral matrix, so as to obtain a reconstructed hyperspectral image based on the third preset image.

[0030] The hyperspectral image reconstruction method and system of the present invention encodes the high-dimensional spectral data of the hyperspectral image into low-dimensional data using a preset encoder, improves the resolution of the image after the low-dimensional encoding of the spectral data using a preset model, and further reconstructs the high-dimensional spectral data of the image after the resolution improvement using a preset decoder, thereby obtaining a hyperspectral image with improved spatial resolution. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating a hyperspectral image reconstruction method as provided in one embodiment;

[0033] Figure 2 This is a schematic diagram of the structure of a preset encoder, a preset model, and a preset decoder in a hyperspectral image reconstruction method according to an embodiment.

[0034] Figure 3 This is a schematic diagram illustrating how a hyperspectral image is processed sequentially by a preset encoder, a preset model, and a preset decoder in a hyperspectral image reconstruction method according to an embodiment.

[0035] Figure 4-1 Low-resolution image of a thin section of quartz sandstone for a specific example;

[0036] Figure 4-2 High-resolution image of a thin section of quartz sandstone for a specific example;

[0037] Figure 4-3 In a specific instance, according to Figure 4-1 The image shown is a pseudo-color image obtained using a preset encoder;

[0038] Figure 4-4 In a specific instance, according to Figure 4-1 The reconstructed hyperspectral image obtained from the image shown;

[0039] Figure 5-1 Low-resolution image of a thin section of feldspathic quartz sandstone for a specific example;

[0040] Figure 5-2 High-resolution image of a thin section of feldspathic quartz sandstone, taken for a specific example;

[0041] Figure 5-3 In a specific instance, according to Figure 5-1 The image shown is a pseudo-color image obtained using a preset encoder;

[0042] Figure 5-4 In a specific instance, according to Figure 5-1 The image shown is a reconstructed hyperspectral image obtained from the image shown. Detailed Implementation

[0043] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0044] For reference Figure 1 , Figure 1 A flowchart of a hyperspectral image reconstruction method is provided as an embodiment. The hyperspectral image reconstruction method includes the following steps:

[0045] S11: Acquire a hyperspectral image and obtain the spectral matrix of the hyperspectral image. Any row vector of the spectral matrix is ​​the spectral vector of the corresponding pixel in the hyperspectral image. Any pixel of the hyperspectral image includes data from multiple spectral channels. The spectral vector of the pixel of the hyperspectral image is formed by the data from the multiple spectral channels of the pixel of the hyperspectral image.

[0046] Hyperspectral images include spatial dimension data and spectral dimension data. The data of multiple spectral channels included in any pixel form spectral dimension data, while the data included in each pixel of a hyperspectral image form spatial dimension data.

[0047] For any pixel in a hyperspectral image, the pixel's spectral vector is formed by the data from the pixel's multiple spectral channels. Each row of the spectral matrix corresponds one-to-one with a pixel in the hyperspectral image, and any row vector of the spectral matrix is ​​the spectral vector of the corresponding pixel in the hyperspectral image.

[0048] S12: Input the spectral matrix into a preset encoder and output a low-dimensional representation of each row vector of the spectral matrix. The number of channels in the low-dimensional representation of any row vector is less than the number of channels in the spectral vector of the pixel corresponding to any row vector.

[0049] The spectral matrix is ​​input into a preset encoder, which outputs a low-dimensional representation of each row vector of the spectral matrix. The number of channels in the low-dimensional representation of any row vector is less than the number of channels in the spectral vector of the corresponding pixel in that row vector. The number of channels is the number of spectral channels. The preset encoder performs dimensionality reduction processing on the spectral vector of each pixel in the hyperspectral image, thus performing low-dimensional encoding.

[0050] S13: Construct a first preset image, the resolution of which is the same as that of the hyperspectral image. The data of any pixel in the first preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix. Input the first preset image into a preset model and output a second preset image, the resolution of which is higher than that of the first preset image.

[0051] The image is reconstructed based on the low-dimensional representation of each row vector of the obtained spectral matrix, thus constructing the first preset image.

[0052] Image resolution refers to the number of pixels per inch in an image. The first preset image has the same resolution as the hyperspectral image, while the second preset image output by the preset model has a higher resolution than the first preset image. Therefore, the resolution of the second preset image output by the preset model is higher than that of the hyperspectral image. The preset model is used to process a low-resolution input image and output a higher-resolution image, thus improving the spatial resolution of the image. The spatial resolution of an image is its overall resolution.

[0053] S14: Input the second preset image into a preset decoder and output a third preset image. The resolution of the third preset image is the same as that of the second preset image. The number of channels of the data of any pixel in the third preset image is greater than the number of channels of the low-dimensional representation of the corresponding row vector in the spectral matrix, so as to obtain a reconstructed image based on the third preset image.

[0054] The second preset image is input into a preset decoder, and the number of channels in the data of any pixel in the output third preset image is greater than the number of channels in the low-dimensional representation of the row vectors in the spectral matrix, thus reconstructing an image with a large number of spectral channels. The resolution of the third preset image is the same as that of the second preset image, but higher than that of the hyperspectral image. Therefore, the obtained third preset image has a large number of spectral channels and a high resolution.

[0055] The hyperspectral image reconstruction method in this embodiment encodes the high-dimensional spectral data of the hyperspectral image into low-dimensional data using a preset encoder, improves the resolution of the image after the low-dimensional encoding of the spectral data using a preset model, and further reconstructs the high-dimensional spectral data of the image after the resolution improvement using a preset decoder, thus obtaining a hyperspectral image with improved spatial resolution.

[0056] A hyperspectral image can be considered a three-dimensional data cube comprising spatial and spectral data. Obtaining the spectral matrix of a hyperspectral image involves flattening the three-dimensional hyperspectral image data along its spatial dimension, converting the data of each pixel at a spatial location into a row, thus obtaining a two-dimensional matrix with rows corresponding to pixels and columns corresponding to spectral channels. For example, the three-dimensional data of a hyperspectral image can be represented as follows: It consists of H rows and W columns of pixels, with each pixel containing C spectral channels, and is represented as a two-dimensional matrix when flattened in spatial dimensions. Where N = H × W, and the spectral vector of pixel (i,j) As a matrix The spectral matrix of the hyperspectral image is obtained from the image itself. This allows the hyperspectral image to be reconstructed into a data structure suitable for deep learning, simplifying the data format and enabling the complete spectral information of each pixel to be independently input into the encoder and decoder. This facilitates the encoder and decoder to compactly encode and reconstruct the high-dimensional spectral features of each pixel.

[0057] The spectral matrix of a hyperspectral image is input into a preset encoder, which outputs a low-dimensional representation of each row vector of the spectral matrix. The preset encoder can be considered to map each row vector of the spectral matrix to a pseudo-color space, and the low-dimensional representation of any row vector of the spectral matrix can be considered a pseudo-color space representation. In some embodiments, the low-dimensional representation output by the preset encoder can have three channels, meaning it includes three spectral channels. For example, the low-dimensional representation of the nth pixel of the hyperspectral image is z. n , .

[0058] Constructing a first preset image based on the low-dimensional representation of each row vector of the spectral matrix of a hyperspectral image can be considered as reconstructing the pseudo-color space representation of each row vector of the spectral matrix into a pseudo-color image. For example, the low-dimensional representation output by the preset encoder includes three spectral channels, and the constructed first preset image, i.e., the pseudo-color image, can be represented as follows: It consists of H rows and W columns of pixels, and each pixel includes three spectral channels, which can be considered as R channel, G channel and B channel.

[0059] In some implementations, a first preset image is input into a preset model to output a second preset image, the second preset image is input into a preset decoder to output a third preset image, and the number of channels of the data of any pixel in the third preset image is the same as the number of channels of the spectral vector of the corresponding pixel in the hyperspectral image.

[0060] In some implementations, training a preset encoder and a preset decoder includes the following steps:

[0061] S21: Acquire a hyperspectral image sample, obtain the spectral matrix of the hyperspectral image sample, and perform a noise addition operation on the spectral matrix of the hyperspectral image sample to obtain the noise-added spectral matrix of the hyperspectral image sample;

[0062] S22: The preset encoder and the preset decoder are trained based on the spectral matrix and the noisy spectral matrix of the hyperspectral image sample. The preset encoder takes the noisy spectral matrix as input and outputs a low-dimensional representation of each row vector of the noisy spectral matrix. The preset decoder takes the low-dimensional representation of each row vector of the noisy spectral matrix as input and outputs the reconstructed spectral vector of each pixel of the hyperspectral image sample. The number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix is ​​less than the number of channels in the row vector of the noisy spectral matrix, and the number of channels in the reconstructed spectral vector of the pixel of the hyperspectral image sample is greater than the number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix.

[0063] Hyperspectral image samples are sample images used to train the preset encoder and preset decoder.

[0064] Noise is added to the spectral matrix of the hyperspectral image samples to simulate observation noise that may exist during actual hyperspectral image acquisition. In this embodiment, the method of adding noise to the spectral matrix of the hyperspectral image samples is not limited. In some implementations, Gaussian noise perturbation can be superimposed on the spectral matrix of the hyperspectral image samples. For example, the spectral matrix of the hyperspectral image samples is represented as follows: The noisy spectral matrix of a hyperspectral image sample is represented as: ,Right now:

[0065] , ;

[0066] Where ε represents noise, N(0,σ) 2 ) indicates that the mean is 0 and the variance is σ. 2 Gaussian noise.

[0067] For example, the preset encoder is represented as f θ (·), the preset decoder is represented as g φ (·), This represents the reconstructed spectral vector of the nth pixel. This represents the noisy spectral vector of the nth pixel. Let represent the original spectral vector of the nth pixel. Then we have:

[0068] , ;

[0069] Among them, zn This represents the low-dimensional representation of the nth pixel output by the preset encoder. The preset decoder then encodes the low-dimensional representation z. n Reconstruction is performed to obtain the reconstructed hyperspectral features. For example, the number of channels in the reconstructed spectral vector can be the same as the number of channels in the original spectral vector. If the number of channels in the original spectral vector is C, and the preset encoder output has three channels in its low-dimensional representation, then... , .

[0070] Based on the above process, a pre-defined encoder and decoder are trained. The spectral matrix of the hyperspectral image samples is noise-added before training, enabling the trained pre-defined encoder and decoder to learn robust low-dimensional representations of spectral features under noise interference. This achieves effective compression and high-fidelity reconstruction of high-dimensional spectral data. The pre-defined encoder and decoder not only improve the compactness of feature representation but also enhance adaptability to actual noise perturbations, providing high-quality spectral input for subsequent spatial super-resolution reconstruction. The pre-defined encoder and decoder can be collectively referred to as a denoising autoencoder.

[0071] In some implementations, the goal of training the preset encoder and preset decoder is to minimize the difference between the reconstructed spectral vector of a pixel in the hyperspectral image sample and the original spectral vector of the pixel in the hyperspectral image sample.

[0072] In some implementations, the difference between the reconstructed spectral vector of a pixel in a hyperspectral image sample and the original spectral vector of the pixel in the hyperspectral image sample is represented as the average of a first preset difference among the pixels of the hyperspectral image sample. The first preset difference of a pixel represents the squared norm of the difference between the reconstructed spectral vector of that pixel and the original spectral vector of that pixel. For example, the difference between the reconstructed spectral vector of a pixel in a hyperspectral image sample and the original spectral vector of the pixel in the hyperspectral image sample can be called the reconstruction error L. rec , is represented as:

[0073] .

[0074] Where N represents that the hyperspectral image sample includes N pixels.

[0075] In some implementations, training the preset model includes the following steps:

[0076] S31: Acquire hyperspectral image samples, obtain the spectral matrix of the hyperspectral image samples, input the spectral matrix of the hyperspectral image samples into the preset encoder, and output a low-dimensional representation of each row vector of the spectral matrix of the hyperspectral image samples.

[0077] Hyperspectral image samples are sample images used to train the preset encoder and preset decoder, and high-resolution hyperspectral images can be used.

[0078] S32: Construct a fourth preset image, the resolution of which is the same as that of the hyperspectral image sample, and the data of any pixel of the fourth preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix of the hyperspectral image sample.

[0079] The image is reconstructed from the low-dimensional representation of each row vector of the spectral matrix of the hyperspectral image sample, and a fourth preset image is constructed.

[0080] S33: Downsample the fourth preset image to obtain a fifth preset image. The resolution of the fifth preset image is lower than that of the fourth preset image. Train the preset model based on the fourth preset image and the fifth preset image. The preset model takes the fifth preset image as input and the reconstructed image as output.

[0081] The fourth preset image is downsampled to obtain a fifth preset image with reduced resolution. This fifth preset image is then input into a preset model, which outputs a reconstructed image. The reconstructed image has a higher resolution than the fifth preset image, thus training the preset model. In some implementations, the resolution of the reconstructed image can be the same as the resolution of the fourth preset image, i.e., the resolution of the reconstructed image is the same as the resolution of the hyperspectral image sample.

[0082] The preset model is trained according to the above process. The trained preset model can reconstruct an image with improved resolution spectral data as a low-dimensional representation of low-resolution spectral data, thereby enhancing the spatial details of hyperspectral images.

[0083] In some embodiments, the loss function for training the preset model represents the amount of difference between the reconstructed image and the fourth preset image. The preset model is trained by minimizing the loss function. In some embodiments, the loss function for training the preset model represents the average of a second preset difference among multiple hyperspectral image samples, where the second preset difference represents the square of the F-norm of the amount of difference between the reconstructed image and the fourth preset image of the hyperspectral image sample.

[0084] For example, the loss function is expressed as:

[0085] ;

[0086] Where M represents M hyperspectral image samples, F θ (·) indicates a preset model. The fourth preset image representing a hyperspectral image sample. This represents the fifth preset image obtained by downsampling the fourth preset image. The loss represented by this loss function can be called the Frobenius loss. In some implementations, the network model includes parameters θ, and the loss function is minimized by fine-tuning θ during training of the preset model.

[0087] In this embodiment, the network structure of the preset model is not limited. In some implementations, the preset model includes a first convolutional layer, a convolutional + residual network, an upsampling layer, and a second convolutional layer connected in sequence. The image input to the preset model first undergoes convolution in the first convolutional layer, then processing through the convolutional + residual network, further upsampling, and then passing through the second convolutional layer. In some implementations, after the image is input to the preset model, mean standardization is performed first to adjust the data, and then deep feature extraction is performed through multiple residual blocks. Mean standardization of the image includes calculating the mean and standard deviation of the image's pixel data; standardization is achieved by subtracting the mean from the data of each pixel and then dividing by the standard deviation. For example, see [reference needed]. Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the structure of a preset encoder, a preset model, and a preset decoder in a hyperspectral image reconstruction method according to an embodiment. Figure 3 This is a schematic diagram illustrating the sequential processing of a hyperspectral image by a preset encoder, a preset model, and a preset decoder in a hyperspectral image reconstruction method according to an embodiment. Exemplarily, the preset model is represented as F... θ (·),but:

[0088] ;

[0089] in, This represents the first preset image, i.e., the low-dimensional representation z of each row vector of the spectral matrix of the hyperspectral image. n The reconstructed image, This represents the second preset image. If the low-dimensional representation includes three spectral channels, then... The first preset image can be considered a pseudo-color image, and the second preset image can be considered a pseudo-color image with increased resolution, which can be considered a super-resolution image. In this embodiment, the preset model is based on a deep residual structure, which is a highly efficient single-image super-resolution deep neural network. Its core advantage is that it utilizes a deep residual learning structure to significantly enhance the spatial detail representation capability of low-resolution images.

[0090] In some implementations, a Laplacian enhancement operation is applied to each spectral channel of the third preset image output by the preset decoder. If this hyperspectral image reconstruction method is applied to a rock thin section hyperspectral image, it can make the hyperspectral image highlight the boundaries and fine-grained structure of mineral grains.

[0091] In some implementations, obtaining a reconstructed hyperspectral image based on a third preset image includes: for any spectral channel of any pixel in the third preset image, performing a convolution operation on the data of the spectral channel of the pixel in the third preset image with a preset Laplacian convolution kernel to obtain a detail enhancement component of the spectral channel of the pixel in the third preset image, and superimposing the detail enhancement component with the data of the spectral channel of the pixel in the third preset image according to a preset weight to obtain the enhanced data of the spectral channel of the pixel in the third preset image.

[0092] For example, for any spectral channel of any pixel in the third preset image, the Laplace transform is expressed as:

[0093] ΔZ c =Z´ c *K;

[0094] Where * denotes the convolution operation, ΔZ c Z' represents the detail enhancement component of the c-th channel. c represents the data of the c-th channel of the third preset image, and K represents the Laplacian convolution kernel.

[0095] Then, the detail enhancement components are superimposed back onto the original image according to the weight λ to obtain the enhanced output: ;

[0096] in, Let λ represent the enhanced data in channel c, and λ also represent the detail enhancement intensity hyperparameter, typically ranging from 0.2 to 0.7, which needs to be selected based on experimental experience. In practical applications, a suitable Laplacian kernel can be selected; for example, a 3×3 Laplacian kernel K can be chosen. A common form is:

[0097] .

[0098] In some embodiments, obtaining the reconstructed hyperspectral image based on the third preset image further includes: for any spectral channel of any pixel in the third preset image, if the enhanced data of this spectral channel of the pixel in the third preset image is less than the minimum value of a preset interval, then the minimum value of the preset interval is used as the data of this spectral channel of the pixel; if the enhanced data of this spectral channel of the pixel in the third preset image is within the preset interval, then the enhanced data of this spectral channel of the pixel is used as the data of this spectral channel of the pixel; if the enhanced data of this spectral channel of the pixel in the third preset image is greater than the maximum value of the preset interval, then the maximum value of the preset interval is used as the data of this spectral channel of the pixel. The enhanced data of any spectral channel of any pixel in the third preset image is numerically truncated to ensure that the pixel value remains within the preset interval, avoiding artifacts and outliers caused by over-enhancement. The preset interval can be [0, 255].

[0099] In some embodiments, before obtaining the spectral matrix of the hyperspectral image, the method further includes: for any pixel of the hyperspectral image, arranging the data of the plurality of spectral channels in the spectral vector of this pixel in ascending order to obtain a sorted vector; calculating the quantile of the data of each of the plurality of spectral channels in the sorted vector, and mapping the quantile corresponding to each spectral channel to the interval [0, 1]; arranging the mapped values ​​corresponding to the plurality of spectral channels in the original spectral channel order to obtain a normalized spectral vector. Thus, for any pixel of the hyperspectral image, the quantile normalization of the data of the plurality of spectral channels of this pixel is performed.

[0100] For any pixel in a hyperspectral image, the data of the multiple spectral channels of that pixel are normalized using quantiles, so that they are distributed within a normalized range of 0 to 1. Normalizing the data of the multiple spectral channels of a pixel in a hyperspectral image not only eliminates numerical scale differences caused by different sampling batches, different sample types, or different imaging instruments, but also effectively improves the numerical stability and convergence speed of subsequent model training, providing favorable conditions for efficient representation of spectral features and subsequent learning.

[0101] This embodiment also provides a hyperspectral image reconstruction system, including:

[0102] Memory, used to store computer programs;

[0103] A processor is configured to execute the computer program to implement the steps of the hyperspectral image reconstruction method as described in any of the preceding embodiments.

[0104] In this embodiment, the hyperspectral image reconstruction system encodes the high-dimensional spectral data of a hyperspectral image into low-dimensional data using a preset encoder. Then, it uses a preset model to improve the resolution of the image after the low-dimensional encoding of the spectral data. Finally, it uses a preset decoder to reconstruct the high-dimensional spectral data from the image after the resolution improvement, thus obtaining a hyperspectral image with improved spatial resolution.

[0105] In some implementations, hyperspectral images can be acquired by building a microscopic hyperspectral imaging system, which can connect a polarizing microscope to a hyperspectral camera and configure data acquisition software to build a high spatial resolution and hyperspectral resolution imaging platform.

[0106] In one specific instance, an acquisition system was built using a CX40P polarizing microscope and a FigSpec® FS-23 hyperspectral camera, with a spectral range of 400-1000 nm and a resolution of 2.5 nm. Data processing was performed on a PC platform with an Intel i7-13900K CPU and 128GB RAM. Thin sections of quartz sandstone and feldspathic quartz sandstone were selected. The microscope's halogen lamp was turned on as a transmitted light source. The field of view containing typical minerals in the rock thin section was selected, and a three-dimensional data cube, combining micron-level spatial dimensions and nanometer-level spectral dimensions, was acquired using a line scan method with the imaging hyperspectral camera. This resulted in a 480×480×300 three-dimensional data cube obtained through line scanning in transmitted light mode. Low-resolution images were acquired by cropping the central region using a 5x objective lens, while high-resolution images were acquired using a 10x objective lens.

[0107] For reference Figures 4-1 to 4-4 , Figure 4-1 This is a low-resolution image of a thin section of quartz sandstone, taken for a specific example. Figure 4-2 High-resolution images of thin sections of quartz sandstone, taken for a specific example. Figure 4-2 The resolution of the image shown is Figure 4-1 Twice the resolution of the image shown. Figure 4-3 In a specific instance, according to Figure 4-1 The image shown is a pseudo-color image obtained using a preset encoder. Figure 4-4 In a specific instance, according to Figure 4-1 The image shown is a reconstructed hyperspectral image obtained from the image shown.

[0108] For reference Figures 5-1 to 5-4 , Figure 5-1 This is a low-resolution image of a thin section of feldspathic quartz sandstone, taken for a specific example. Figure 5-2 High-resolution images of thin sections of feldspathic quartz sandstone, taken for a specific example. Figure 5-2 The resolution of the image shown is Figure 5-1 Twice the resolution of the image shown. Figure 5-3 In a specific instance, according to Figure 5-1The image shown is a pseudo-color image obtained using a preset encoder. Figure 5-4 In a specific instance, according to Figure 5-1 The image shown is a reconstructed hyperspectral image obtained from the image shown.

[0109] Hyperspectral image super-resolution reconstruction aims to restore low-resolution hyperspectral images to high-resolution, high-fidelity images, effectively improving the discernibility of fine-grained structures and the integrity of spectral information. Through super-resolution reconstruction, not only can the spatial details and boundary clarity of mineral grains be enhanced, but the ability to identify small-scale minerals can also be improved, providing a higher-quality data foundation for precise quantitative analysis of rock composition and the exploration of new materials. However, super-resolution reconstruction of rock thin-section hyperspectral images faces many challenges. First, hyperspectral images of rock thin sections have high dimensionality and many channels, with tight coupling between spatial and spectral information. Maintaining spectral consistency and physical authenticity while improving spatial resolution is a core challenge. Second, mineral grains in rock thin sections exhibit diverse morphologies and complex boundaries, and traditional methods based on interpolation or single spatial information often fail to effectively reconstruct the fine-grained boundaries and complex structures of minerals.

[0110] The hyperspectral image reconstruction method and system proposed in this embodiment can perform two-stage super-resolution reconstruction of hyperspectral images of rock thin sections. First, high-dimensional spectral data is encoded and reconstructed in a low-dimensional manner based on a pixel-level autoencoder, improving the compactness of spectral signal representation and reconstruction quality. Subsequently, a spatial super-resolution model based on a deep residual network is used to enhance the spatial details of the low-resolution hyperspectral pseudo-color image, achieving high-fidelity restoration of the microstructure of mineral grains. This method effectively combines spectral information compression and spatial resolution enhancement, significantly improving the spatial details and mineral boundary resolution of rock thin section images while maintaining spectral consistency.

[0111] To address the issue of insufficient mineral identification and segmentation accuracy in hyperspectral images of rock thin sections due to limited spatial resolution, blurred fine-grained structures, and noise interference, this embodiment proposes a super-resolution reconstruction method and system for mineral particles based on spectral spatial co-characterization. This method combines pixel-level autoencoded spectral compression, depth residual super-resolution reconstruction, and a dedicated hyperspectral image data acquisition platform to systematically improve the spatial detail and spectral fidelity of hyperspectral images, significantly enhancing the clarity of mineral particle boundaries and compositional resolution.

[0112] a) This embodiment proposes a two-stage mineral grain image enhancement method combining spectral feature autoencoder compression and spatial depth residual super-resolution. This method systematically achieves spectral information compression, low-dimensional pseudo-color reconstruction, and simultaneous enhancement of spatial details in the processing of hyperspectral images of rock thin sections, solving the problem of limited spatial resolution in high-dimensional hyperspectral data. b) This embodiment introduces a pixel-level autoencoder with noise perturbation, which can learn robust pseudo-color spatial representations of mineral grains even in the presence of observational noise. This enables efficient compression and high-fidelity reconstruction of high-dimensional spectral data, improving the compactness of spectral feature representation and noise resistance. c) For the compressed pseudo-color hyperspectral image, a pre-trained depth residual super-resolution network is used for target domain fine-tuning, achieving spatial resolution enhancement and structural clarity enhancement of the mineral grain image. Simultaneously, a Laplacian spatial enhancement operator is applied to the output high-resolution pseudo-color channel, effectively highlighting grain boundaries and micro-fine-grained structures. Controllable parameters suppress over-enhancement and artifacts, achieving adaptive enhancement of spatial details. d) The entire process covers high-resolution hyperspectral imaging, quantile normalization, noise reduction autoencoder compression, spatial super-resolution reconstruction and spatial detail enhancement, forming a dedicated integrated acquisition-processing-enhancement system for mineral grain analysis of rock thin sections.

[0113] This invention effectively solves the challenges of mineral particle identification and structural reconstruction in hyperspectral images of rock thin sections caused by limited spatial resolution, blurred fine-grained structures, and noise interference by constructing a two-stage super-resolution reconstruction method based on spectral spatial co-representation. Utilizing quantile normalization and pixel-level unfolding techniques, the invention achieves standardized and efficient input of hyperspectral data, providing a solid data foundation for high-dimensional spectral compression and reconstruction using a denoising autoencoder. The denoising autoencoder significantly improves the compact representation and noise resistance of mineral spectral features, ensuring high fidelity in spectral reconstruction. A spatial super-resolution model based on a deep residual network, after targeted fine-tuning, can efficiently improve the spatial resolution of the compressed pseudo-color image, restoring the microstructure and details of mineral particles. Furthermore, the Laplacian enhancement operator achieves adaptive enhancement of particle boundaries and fine-grained structures, effectively highlighting the spatial features of multi-scale particles. The end-to-end hyperspectral acquisition and integrated processing system enables real-time high-fidelity reconstruction of three-dimensional rock thin section images with micron-level spatial resolution and nanometer-level spectral resolution. This invention provides an efficient and reliable technical solution for the fine identification and structural analysis of multi-scale mineral particles in complex lithofacies environments, and has important application value in fields such as geological exploration, quantitative composition, and micro-area structural evolution.

[0114] The hyperspectral image reconstruction method and system provided by this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make several improvements and modifications to this invention without departing from the principles of this invention, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A hyperspectral image reconstruction method, characterized in that, include: A hyperspectral image is acquired, and a spectral matrix of the hyperspectral image is obtained. Any row vector of the spectral matrix is ​​the spectral vector of the corresponding pixel in the hyperspectral image. Any pixel of the hyperspectral image includes data from multiple spectral channels. The spectral vector of a pixel in the hyperspectral image is formed by the data from the multiple spectral channels of the pixel in the hyperspectral image. The spectral matrix is ​​input into a preset encoder, and a low-dimensional representation of each row vector of the spectral matrix is ​​output. The number of channels in the low-dimensional representation of any row vector is less than the number of channels in the spectral vector of the pixel corresponding to any row vector. A first preset image is constructed, the resolution of the first preset image is the same as that of the hyperspectral image, the data of any pixel of the first preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix, and the first preset image is input into a preset model to output a second preset image, the resolution of the second preset image is higher than that of the first preset image; The second preset image is input into a preset decoder, and a third preset image is output. The resolution of the third preset image is the same as that of the second preset image. The number of channels of the data of any pixel in the third preset image is greater than the number of channels of the low-dimensional representation of the corresponding row vector in the spectral matrix, so as to obtain a reconstructed hyperspectral image based on the third preset image. The preset model is used to reconstruct an image with improved resolution and low-dimensional representation of spectral data from an image where the spectral data is represented in a low-dimensional form. Training the preset model includes: Acquire hyperspectral image samples, obtain the spectral matrix of the hyperspectral image samples, and input the spectral matrix of the hyperspectral image samples into the preset encoder to output a low-dimensional representation of each row vector of the spectral matrix of the hyperspectral image samples; A fourth preset image is constructed, the resolution of which is the same as that of the hyperspectral image sample, and the data of any pixel of the fourth preset image is formed by the low-dimensional representation of the corresponding row vector in the spectral matrix of the hyperspectral image sample; The fourth preset image is downsampled to obtain a fifth preset image. The resolution of the fifth preset image is lower than that of the fourth preset image. The preset model is trained based on the fourth preset image and the fifth preset image. The preset model takes the fifth preset image as input and the reconstructed image as output.

2. The hyperspectral image reconstruction method according to claim 1, characterized in that, Training the preset encoder and the preset decoder includes: Acquire hyperspectral image samples, obtain the spectral matrix of the hyperspectral image samples, and perform a noise addition operation on the spectral matrix of the hyperspectral image samples to obtain a noisy spectral matrix of the hyperspectral image samples; The preset encoder and the preset decoder are trained based on the spectral matrix and the noisy spectral matrix of the hyperspectral image sample. The preset encoder takes the noisy spectral matrix as input and outputs a low-dimensional representation of each row vector of the noisy spectral matrix. The preset decoder takes the low-dimensional representation of each row vector of the noisy spectral matrix as input and outputs the reconstructed spectral vector of each pixel of the hyperspectral image sample. The number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix is ​​less than the number of channels in the row vector of the noisy spectral matrix, and the number of channels in the reconstructed spectral vector of the pixel of the hyperspectral image sample is greater than the number of channels in the low-dimensional representation of the row vector of the noisy spectral matrix.

3. The hyperspectral image reconstruction method according to claim 2, characterized in that, The goal of training the preset encoder and the preset decoder is to minimize the difference between the reconstructed spectral vector of the pixels of the hyperspectral image sample and the original spectral vector of the pixels of the hyperspectral image sample.

4. The hyperspectral image reconstruction method according to claim 1, characterized in that, The loss function for training the preset model represents the difference between the reconstructed image and the fourth preset image.

5. The hyperspectral image reconstruction method according to claim 1, characterized in that, The preset model includes a first convolutional layer, a convolutional + residual network, an upsampling layer, and a second convolutional layer connected in sequence. The first preset image input to the preset model is first convolved by the first convolutional layer, then processed by the convolutional + residual network, further processed by the upsampling layer, and then processed by the second convolutional layer.

6. The hyperspectral image reconstruction method according to claim 1, characterized in that, Obtaining the reconstructed hyperspectral image based on the third preset image includes: For any spectral channel of any pixel in the third preset image, the data of the spectral channel of the pixel in the third preset image is convolved with a preset Laplacian convolution kernel to obtain the detail enhancement component of the spectral channel of the pixel in the third preset image. The detail enhancement component is then superimposed with the data of the spectral channel of the pixel in the third preset image according to a preset weight to obtain the enhanced data of the spectral channel of the pixel in the third preset image.

7. The hyperspectral image reconstruction method according to claim 6, characterized in that, Obtaining the reconstructed hyperspectral image based on the third preset image also includes: For any spectral channel of any pixel in the third preset image, if the enhanced data of this spectral channel of this pixel in the third preset image is less than the minimum value of a preset interval, then the minimum value of the preset interval is taken as the data of this spectral channel of this pixel; if the enhanced data of this spectral channel of this pixel in the third preset image is within the preset interval, then the enhanced data of this spectral channel of this pixel is taken as the data of this spectral channel of this pixel; if the enhanced data of this spectral channel of this pixel in the third preset image is greater than the maximum value of the preset interval, then the maximum value of the preset interval is taken as the data of this spectral channel of this pixel.

8. The hyperspectral image reconstruction method according to any one of claims 1 to 7, characterized in that, Before obtaining the spectral matrix of the hyperspectral image, the method further includes: For any pixel in the hyperspectral image, the data of the multiple spectral channels in the spectral vector of this pixel are arranged in ascending order to obtain a sorted vector; the quantile of the data of each spectral channel in the sorted vector is calculated, and the quantile corresponding to each spectral channel is mapped to the interval [0, 1]; the mapped values ​​corresponding to the multiple spectral channels are arranged in the original spectral channel order to obtain a normalized spectral vector.

9. A hyperspectral image reconstruction system, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the hyperspectral image reconstruction method as described in any one of claims 1 to 8.

Images