Model-Driven Deep Generative Network Method for Estimating Mineral Abundance in Lunar Hyperspectral Imagery

Abstract

This invention discloses a model-driven deep generative network method for estimating mineral abundance in lunar hyperspectral images. It leverages the ability of an autoencoder network to learn a low-dimensional representation of the input image to obtain abundance. A recurrent generative adversarial network (GAN) implicitly learns a nonlinear mixing mechanism, and a recurrent consistency loss is introduced to achieve unsupervised unmixing using the abundance obtained from the autoencoder. Furthermore, considering the radiative transfer process, the Hapke model is simplified by converting reflectance into single-scattering albedo, establishing a correlation between the unmixing mechanism of the generative network and the physical mechanism of spectral mixing, and designing a model-driven autoencoder GAN. This invention enables nonlinear unmixing and considers the radiative transfer process, making it more suitable for mineral abundance estimation in densely mixed lunar surface scenes.

Description

Technical Field

[0001] This invention relates to a hyperspectral remote sensing image processing method, and more particularly to a model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images. Background Technology

[0002] Remote sensing of the Moon began in the 1950s. Lunar rocks are mainly composed of minerals such as clinopyroxene, orthopyroxene, plagioclase, olivine, ilmenite, and karst glass. Before the launch of the American Clementine, lunar mineral detection was primarily conducted using ground-based telescopes. Ground-based telescopes offer the advantage of high spectral resolution, but their spatial resolution is relatively low, generally between 2 and 10 km. Research is mainly concentrated on localized areas of the lunar surface, making it impossible to observe the far side of the Moon, and their spectra are easily affected by Earth's atmosphere.

[0003] Before the widespread application of hyperspectral data, multispectral and visible-near-infrared reflectance spectroscopy were effective methods for studying the composition of the Earth's surface and the Moon. Currently, the most studied multispectral data include the Clementine dataset from the United States and the Changdrayann-1 dataset from India. Lunar mineral inversion from multispectral data primarily employs methods such as band ratio analysis, lookup table analysis, spectral feature analysis, and hybrid spectral decomposition. However, due to the limited number of bands in multispectral data, the accuracy and precision of the mineral distribution obtained by these methods are relatively low. With the rise of lunar exploration, a batch of lunar hyperspectral data has been acquired, making the use of continuous spectral analysis for minerals a research trend. In addition to classic spectral matching, the Hapke radiative transfer model, and the Gaussian correction model, machine learning and neural network algorithms can construct nonlinear relationships between mineral composition and spectral features, and have been widely adopted in lunar hyperspectral image mineral research.

[0004] The main reason lunar surface minerals can be detected using remote sensing data is that Fe and Ti ions in lunar surface minerals exhibit spectral absorption characteristics in the ultraviolet, visible, and near-infrared ranges. These characteristics are primarily based on electronic processes. Therefore, the intensity, energy, and width of the spectral features acquired from remote sensing data can serve as criteria for identifying these transition metal cations and their host minerals. Furthermore, only metals with sufficient abundance can be detected by remote sensing. Based on the relationship between lunar surface mineral elements, abundance, and spectral characteristics, mathematical models can be established to invert the abundance of lunar surface minerals.

[0005] Some studies have introduced data-driven deep learning techniques into lunar surface mineral abundance inversion and achieved certain results. These methods use networks to establish models between spectral reflectance and abundance. Korokhin et al. first constructed neural network ANNs in 2008 to estimate the abundance of lunar surface TiO2 using Clementine data; Wang et al. used Chang'e-1 IIM data to estimate the abundance of lunar surface TiO2 using the DT-SVM method; Wang et al. used two support vector machines (SVMs) to infer the abundance of lunar surface TiO2 using IIM data; and Xia et al. established a neural network model to depict the linear and complex nonlinear relationships between chemical composition and spectral features, obtaining high-resolution maps of six oxides on the lunar surface.

[0006] The aforementioned hyperspectral images provide continuous spectral data with a wide spectral range and high spectral resolution for estimating lunar surface mineral abundance. Current research on mineral abundance inversion based on lunar hyperspectral data provides quantitative information for detailed studies of lunar mineral formation and evolution history. However, the inversion results still have certain limitations and remain a major challenge and focus in lunar surface mineral remote sensing detection and inversion. On the one hand, the lunar surface is covered with impact craters, gullies, dunes, etc., making the spectrometer imaging environment extremely complex. Furthermore, lunar regolith is tightly mixed, with photons interacting with multiple components during transmission, making the development of specific information extraction methods for this complex environment a significant challenge. On the other hand, nonlinear Hapke models are widely used in quantitative mineral analysis, but many variables included in these models are difficult to accurately estimate from hyperspectral images. Deep networks have the advantage of constructing nonlinear models, but the results of deep networks lack physical interpretability. How to construct model-driven deep networks that establish a clear correlation with the physical mechanisms of spectral mixing is the key issue addressed in this invention. Summary of the Invention

[0007] Purpose of the invention: In order to overcome the shortcomings of the prior art, the present invention provides a model-driven deep generative network method for estimating mineral abundance in lunar hyperspectral images, which can effectively solve the problem of weak physical interpretability of deep learning network results and improve the accuracy of lunar mineral abundance inversion in the scenario of close mixing of lunar surface and lunar soil.

[0008] Technical Solution: To achieve the above-mentioned objectives, the technical solution adopted in this invention is a model-driven deep generative network-based method for estimating mineral abundance in lunar surface hyperspectral images, comprising the following steps:

[0009] Step 1: Preprocess the lunar surface hyperspectral remote sensing image to obtain the pixel matrix of the lunar surface hyperspectral remote sensing image, and calculate the reflectance of each pixel in the image based on the pixel matrix.

[0010] Step 2: Construct a two-way reflectance model of lunar surface soil particles, and obtain the single albedo function by inverse solution;

[0011] Step 3: Based on the bidirectional reflectance of each pixel in the image, calculate the single albedo matrix of the lunar surface hyperspectral remote sensing image using the single albedo function.

[0012] Step 4: Construct a Recurrent Generative Adversarial Network (RGAN). Use the single-shot albedo matrix of the lunar hyperspectral remote sensing image as input to the RGAN to perform abundance inversion on the hyperspectral image, outputting the mineral abundance matrix of the lunar hyperspectral image. As an unsupervised network, the RGAN overcomes the problem of scarce lunar surface mineral abundance samples and improves the accuracy of mineral abundance estimation from lunar hyperspectral images.

[0013] The bidirectional reflectance model for lunar surface soil particles is calculated using the following formula:

[0014]

[0015] Where y is the bidirectional reflectance of lunar surface soil particles, w is the single albedo, and μ and μ0 are the cosine values ​​of the incident angle and the exit angle, respectively.

[0016] The single albedo function is calculated as follows:

[0017]

[0018] Where w is the single albedo, y is the bidirectional reflectance of lunar surface soil particles, and μ and μ0 are the cosine values ​​of the incident angle and the exit angle, respectively.

[0019] The basic architecture for constructing a recurrent generative adversarial network (GAN) consists of two GANs and a recurrent consistency loss function. Each GAN contains a generator and a discriminator, and each generator comprises an encoder and a decoder. The generator's initial part consists of several convolutional layers, which fully utilize the spatial-spectral joint information of hyperspectral pixel blocks, while the latter part is a softmax constraint layer, ensuring that the output abundance matrix... It satisfies the physical constraints of summation being one and non-negativity. The recurrent adversarial network consists of two generators G1 and G2 and two discriminators D1 and D2. G1 is used to obtain a low-dimensional representation of the input matrix, G2 is used to reconstruct the input matrix, D1 is used to judge the error between the low-dimensional matrix and the reference abundance matrix, and D2 is used to judge the error between the reconstructed input and the original input. The network can be regarded as a demixing process from image to abundance and a reconstruction process from abundance to image, realizing a loop.

[0020] The construction of the recurrent generative adversarial network includes:

[0021] (a) Input the single albedo matrix Y of the lunar surface hyperspectral remote sensing image into generator G1 to generate the corresponding abundance matrix. Achieve demixing;

[0022] (b) The abundance matrix generated by generator G1 is used as the input to generator G2. Generator G2 reconstructs the corresponding single albedo matrix and outputs the reconstructed single albedo matrix.

[0023] (c) Connect generator G2 and generator G1 to form a loop;

[0024] (d) Generators G1 and G2 correspond to discriminators D1 and D2, respectively. Discriminator D1 is used to determine whether the input data of generator G1 is the reconstructed single albedo matrix output by generator G2. Using the same training data Y of the single albedo matrix, the discriminator D2 uses the abundance matrix output by generator G1 as its input data to discriminate generator G2. The training samples S of the abundance matrix;

[0025] (e) The objective function is calculated as follows:

[0026]

[0027] Among them, L Re L represents the cycle-consistent loss function. GAN Let Y represent the adversarial loss function, Y represent the single albedo matrix of the image input to G1, and S represent the reference abundance matrix. This is the abundance matrix output by G1.

[0028] The activation functions of the convolutional layers in the generator G1 are all ReLU.

[0029] Working Principle: Considering the differences between Earth and the Moon's environment, and the fact that the lunar regolith consists of tightly mixed particles, the Hapke radiative transfer model is used to represent bidirectional reflectance as a nonlinear function of scene-related particle density and size, as well as single-scattering albedo, thus describing the multiple scattering of light between different particles at a microscopic scale. To simplify the Hapke model for modeling tightly mixed scenes, it is assumed that particles are spherical from a microscopic perspective, exhibit isotropic scattering, and have a sufficiently large phase angle (the angle between incident and outgoing radiation). The bidirectional reflectance of particles is modeled using optical and physical parameters, converting the pixel's reflectance into single-scattering albedo, making the model applicable to tightly mixed scenes. Then, the abundance is obtained by utilizing the low-dimensional representation of the input image through an autoencoder network, and a recurrent generative adversarial network is constructed to implicitly learn the nonlinear mixing mechanism. Introducing a recurrent consistency loss enables unsupervised unmixing using the abundance obtained from the autoencoder. The method implemented in this invention not only improves the accuracy of hyperspectral image abundance estimation but also has significant theoretical and practical implications for advancing research on mineral abundance estimation in lunar hyperspectral remote sensing images.

[0030] This invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0031] This invention proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0032] This invention proposes a computer program product, including a computer program and / or instructions, which, when executed by a processor, implements the model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0033] Beneficial Effects: Compared with existing technologies, this invention has the following beneficial effects: 1. This invention considers the dense mixing of lunar surface soil layers due to weathering and other processes, resulting in multiple scattering interactions between particles. It converts the reflectance of hyperspectral images into single-pass albedo and introduces it into the deep network, establishing a connection between the physical mechanisms of deep network unmixing and spectral mixing, thus improving the physical interpretability of the results. 2. This invention employs a recurrent generative adversarial network implicit learning nonlinear mixing mechanism. The introduction of a recurrent consistency loss enables unsupervised unmixing using the abundance obtained from autoencoding, effectively overcoming the problem of limited lunar surface mineral abundance samples and improving the accuracy of lunar surface mineral abundance inversion. Attached Figure Description

[0034] Figure 1 This is a flowchart of the model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images as described in this invention.

[0035] Figure 2 This is a false-color composite image of hyperspectral remote sensing imagery used in an example.

[0036] Figure 3 This is an abundance diagram of five minerals obtained by the method of this invention. Detailed Implementation

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0038] Example 1

[0039] The model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images described in this invention has the following process: Figure 1 As shown, it includes the following steps:

[0040] Step 1: Preprocess the lunar surface hyperspectral remote sensing image to obtain the pixel matrix of the lunar surface hyperspectral remote sensing image, and calculate the reflectance of each pixel in the image based on the pixel matrix.

[0041] Step 2: Construct a two-way reflectance model of lunar surface soil particles, and obtain the single albedo function by inverse solution;

[0042] Step 3: Based on the bidirectional reflectance of each pixel in the image, calculate the single albedo matrix of the lunar surface hyperspectral remote sensing image using the single albedo function.

[0043] Step 4: Construct a recurrent generative adversarial network (RGAN). Use the single albedo matrix of the lunar surface hyperspectral remote sensing image as the input of the RGAN to perform abundance inversion on the hyperspectral image and output the mineral abundance matrix of the lunar surface hyperspectral image.

[0044] The experimental data in this embodiment are hyperspectral remote sensing images acquired by the visible-near-infrared imaging spectrometer carried by the Chang'e-3 Yutu lunar rover. The images are in-situ detection data, containing 256×256 pixels, 100 bands, with a spectral resolution of approximately 2-6 nm, a spectral range of 450-950 nm, and a signal-to-noise ratio of 31 dB. Figure 2 The image is a hyperspectral image. In the experiment, the number of endmembers in the image was set to 5 using an endmember estimation algorithm.

[0045] The method for estimating mineral abundance in lunar hyperspectral images using the above experimental data is as follows:

[0046] (1) Perform data preprocessing on the hyperspectral remote sensing image and convert it into a two-dimensional matrix Y1 of size m×n, where m is 100 and n is 65536;

[0047] (2) The bidirectional reflectance y of lunar regolith particles is modeled using optical and physical parameters, and the formula is: Where w is the albedo, R is the conversion function between the albedo and the reflectivity of the pixel itself, and μ and μ0 are the cosine values ​​of the incident angle and the exit angle, respectively.

[0048] (3) Rewrite the formula in step 2 as an inverse problem so that the single albedo w can be expressed in terms of reflectivity y:

[0049]

[0050] (4) Obtain the single albedo matrix Y of the image, and use it as the input matrix of the network;

[0051] (5) Construct a nonlinear spectral mixing mechanism for the implicit learning table of a recurrent generative adversarial network;

[0052] (6) First, the generator G1 is used to generate the corresponding abundance matrix for unmixing. The generator has four convolutional layers with kernel sizes of 3×3×16, 3×3×32, 1×1×64, and 1×1×128, and the activation function is ReLU. This fully utilizes the spatial-spectral joint information of the hyperspectral pixel block. The latter part is a softmax constraint layer, which makes the output abundance satisfy the physical constraints of summing to one and non-negativity.

[0053] (7) Then the generated abundance matrix is ​​used as the input of generator G2 to reconstruct the single albedo matrix of the corresponding hyperspectral image. That is, G2 learns the forward process of spectral mixing and G1 learns the reverse process of mixing decomposition.

[0054] (8) A loop is formed by connecting G1 and G2;

[0055] (9) Corresponding to generators G1 and G2, the network uses two discriminators D1 and D2. The discriminator is a fully connected network consisting of three fully connected layers. Discriminator D1 is used to discriminate the input data of generator G1, which comes from the reconstructed single albedo matrix output by generator G2. Using the same training data Y of the single albedo matrix, the discriminator D2 uses the abundance matrix output by generator G1 as its input data to discriminate generator G2. The training samples S of the abundance matrix;

[0056] (10) Calculate the objective function:

[0057]

[0058] (11) Output the mineral abundance matrix of the lunar surface hyperspectral image.

[0059] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0060] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0061] In one embodiment, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images.

[0062] The lunar surface mineral abundance estimation algorithm (Hapke-cycleGAN) of this invention was experimentally analyzed on hyperspectral remote sensing image data, and compared with related mineral abundance estimation algorithms. The comparison methods mainly included the classic VCA-FCLS algorithm, the deep network method uDAS, and the 3D-CNN algorithm. Due to the lack of reference endmembers and abundance, the abundance estimation result was measured by calculating the signal reconstruction error between the original image and the reconstructed data obtained from the estimated abundance. A larger signal reconstruction error indicates better accuracy. As shown in Table 1, the algorithm of this invention has better mineral abundance estimation accuracy. Furthermore, Figure 3 The diagram shows the mineral abundance maps of five minerals obtained by estimating the mineral abundance of hyperspectral images from the in-situ exploration of the Chang'e-3 Yutu lunar rover.

[0063] Table 1. Reconstruction error values ​​of different algorithms on the dataset.

[0064]

Claims

1. A model-driven deep generative network method for estimating mineral abundance in lunar surface hyperspectral images, characterized in that: Includes the following steps: Step 1: Preprocess the lunar surface hyperspectral remote sensing image to obtain the pixel matrix of the lunar surface hyperspectral remote sensing image, and calculate the reflectance of each pixel in the image based on the pixel matrix. Step 2: Construct a two-way reflectance model of lunar surface soil particles, and obtain the single albedo function by inverse solving; Step 3: Based on the bidirectional reflectance of each pixel in the image, calculate the single albedo matrix of the lunar surface hyperspectral remote sensing image using the single albedo function. Step 4: Construct a recurrent generative adversarial network (RGAN). Use the single albedo matrix of the lunar surface hyperspectral remote sensing image as the input of the RGAN to perform abundance inversion on the hyperspectral image and output the mineral abundance matrix of the lunar surface hyperspectral image.

2. The method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network according to claim 1, characterized in that: The bidirectional reflectance model for lunar surface soil particles is calculated using the following formula: Where y is the bidirectional reflectance of lunar surface soil particles, w is the single albedo, and μ and μ0 are the cosine values ​​of the incident angle and the exit angle, respectively.

3. The method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network according to claim 1, characterized in that: The single albedo function is calculated as follows: Where w is the single albedo, y is the bidirectional reflectance of lunar surface soil particles, and μ and μ0 are the cosine values ​​of the incident angle and the exit angle, respectively.

4. The method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network according to claim 1, characterized in that: The basic architecture of a recurrent generative adversarial network (GAN) includes the GAN and a recurrent consistency loss function. The GAN consists of two generators and two discriminators; each generator comprises an encoder and a decoder. Each generator includes several sequentially connected convolutional layers and SoftMax constraint layers. The SoftMax constraint layers set the output abundance matrix... Satisfy physical constraints; The discriminator is a fully connected network.

5. The method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network according to claim 1, characterized in that, The construction of the recurrent generative adversarial network includes: (a) Input the single albedo matrix Y of the lunar surface hyperspectral remote sensing image into generator G1 to generate the corresponding abundance matrix. Achieve demixing; (b) The abundance matrix generated by generator G1 is used as the input to generator G2. Generator G2 reconstructs the corresponding single albedo matrix and outputs the reconstructed single albedo matrix. (c) Connect generator G2 and generator G1 to form a loop; (d) Generators G1 and G2 correspond to discriminators D1 and D2, respectively. Discriminator D1 is used to determine whether the input data of generator G1 is the reconstructed single albedo matrix output by generator G2. Using the same training data Y of the single albedo matrix, the discriminator D2 uses the abundance matrix output by generator G1 as its input data to discriminate generator G2. The training samples S of the abundance matrix; (e) The objective function is calculated as follows: Where L represents the objective function, G1 represents generator G1, G2 represents generator G2, D1 represents the discriminator of generator G1, D2 represents the discriminator of generator G2, and L Re L represents the cycle-consistent loss function. GAN Let Y represent the adversarial loss function, Y represent the single albedo matrix of the image input to generator G1, and S represent the reference abundance matrix. This is the abundance matrix output by generator G1.

6. The method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network according to claim 5, characterized in that: The activation functions of the convolutional layers in the generator G1 are all ReLU.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven depth generation network as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for estimating mineral abundance in lunar surface hyperspectral images using a model-driven deep generative network as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program and / or instructions, characterized in that, When executed by a processor, the computer program / instruction implements the model-driven depth generation network method for estimating mineral abundance in lunar surface hyperspectral images according to any one of claims 1 to 6.

Images