Total variation regularization depth expansion network method for hyperspectral image unmixing
By employing a total variation regularized deep unfolding network method, combined with sparse constraints and total variation space constraints, a deep learnable network is constructed. This addresses the issues of slow iteration, low efficiency, and insufficient stability in existing hyperspectral unmixing models, achieving high-precision, high-efficiency, and robust hyperspectral data unmixing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-17
- Publication Date
- 2026-04-03
AI Technical Summary
Existing hyperspectral unmixing models suffer from slow iterative convergence, low computational efficiency, hyperparameters that depend on empirical settings and cannot be adaptive, lack interpretability and stability, and do not fully incorporate the spatial structure information of hyperspectral images, thus limiting the improvement of unmixing accuracy.
A deep learnable network is constructed by adopting a total variation regularized deep unfolded network method, combining sparse constraints and total variation space constraints. The network is solved iteratively by alternating direction multiplier method, and the iterative process is unfolded into a deep neural network. Learnable parameters are introduced, and an unsupervised learning mode is adopted.
It significantly improves demixing accuracy and robustness, greatly enhances computational efficiency, is suitable for real-time processing of large-scale hyperspectral data, has a network structure with clear physical meaning, strong adaptability, and lowers the application threshold.
Smart Images
Figure CN121789043A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of hyperspectral image processing technology, specifically a total variation regularized deep unfolding network method for hyperspectral image unmixing. Background Technology
[0002] Hyperspectral images (HSI) possess the characteristic of "map-spectrum integration," containing rich spatial and spectral information about ground features, and are widely used in fields such as geological exploration, environmental monitoring, and precision agriculture. However, limited by the spatial resolution of the imaging system, hyperspectral images contain a large number of mixed pixels composed of various ground features, severely restricting the accuracy of subsequent analysis. Hyperspectral unmixing technology aims to decompose mixed pixels into a series of pure substances (called "endmembers") and their corresponding proportions (called "abundance"), which is a key step in improving the application value of hyperspectral images.
[0003] Currently, hyperspectral unmixing models can be mainly divided into two categories: linear mixing models and nonlinear mixing models. Nonlinear mixing models consider the multiple scattering effect between endmembers, while linear mixing models ignore the influence of multiple scattering, assuming that the mixed pixel spectrum is a linear mixture of the endmember spectra. Therefore, linear mixing models have a simple structure, clear physical meaning, and are the most widely used. Unmixing methods based on linear mixing models are mainly divided into geometric methods, statistical methods, and sparse regression methods. Among them, sparse unmixing methods transform the unmixing problem into a sparse regression problem by introducing a spectral library containing a large number of endmembers, that is, selecting a few active endmembers from a large-scale spectral library to reconstruct the observed signal.
[0004] To obtain good unmixing results, it is usually necessary to introduce prior constraints during the solution process, such as the nonnegativity, sparsity, and spatial smoothness of abundance. Traditional model-driven methods, such as SUnSAL, design optimization models that include these priors and use the Alternating Direction Method of Multipliers (ADMM) for solution [Iordache MD, Bioucas-Dias JM, Plaza A. Sparse Unmixing of Hyperspectral Data[J].IEEE Transactions on Geoscience & Remote Sensing, 2011,49(6):2014-2039.]. Although these methods are highly interpretable, they have the following inherent defects: (1) slow iterative convergence speed and low computational efficiency; (2) hyperparameters in the model (such as regularization coefficients) need to be manually set based on experience, making it difficult to adapt to different data; (3) the solution algorithm is fixed and cannot learn a better solution path from the data.
[0005] In recent years, data-driven deep learning methods have been introduced into the field of unmixing due to their powerful feature learning capabilities and efficient forward computation speed. However, these methods are often regarded as "black boxes," lacking interpretability and being sensitive to network initialization and the amount of training data, resulting in insufficient stability.
[0006] The algorithmic unfolding network maps each iteration of a traditional iterative optimization algorithm to a layer of a neural network, thereby constructing a well-structured and interpretable deep model. It successfully combines model-driven theoretical guarantees with data-driven adaptive learning capabilities. However, most existing unfolding networks fail to fully and flexibly incorporate the unique spatial structural information of hyperspectral images, such as the piecewise smoothness of abundance maps, thus limiting further improvements in their unmixing accuracy. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of the prior art and to propose a high-precision, high-efficiency, and high-robust method for hyperspectral data unmixing in mineral exploration, precision agriculture, environmental monitoring, and disaster assessment.
[0008] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a total variation regularized depth unrolling network method for hyperspectral image unmixing, comprising the following steps:
[0009] Step 1: Input hyperspectral image data and endmember spectral library;
[0010] Step 2: Based on the linear mixture model, construct a hyperspectral data unmixing model that integrates sparse constraints and total variation spatial constraints to optimize the solution function;
[0011] Step 3, Alternating Iterative Solution: Transform the optimization function into an equivalent augmented Lagrangian form, and use the alternating direction multiplier method to iteratively solve for each variable;
[0012] Step 4, Construct a deep unfolded network: Expand the iterative process into a deep learnable network structure with several layers, and parameterize the hyperparameters and several approximate operators in the iteration into learnable parameters;
[0013] Step 5, train the deep unfolded network: based on the hyperspectral data and endmember spectral library, given a loss function, continuously optimize the parameters in the deep unfolded network through backpropagation and gradient descent algorithms, and obtain the reconstructed spectrum until the loss function is stable;
[0014] Step 6: After training is complete, forward propagate the deep unrolled network to the new input to obtain a fast abundance coefficient estimate.
[0015] Preferably, the input hyperspectral data and endmember spectral library mentioned in step 1 are as follows:
[0016] Input the hyperspectral image data to be unmixed and endmember spectral library , It is the number of bands. It is the number of pixels in the image. It is an end-member number.
[0017] Preferably, the optimization solution function for constructing the hyperspectral data unmixing model that includes sparse priors and spatial total variation priors described in step 2 is as follows:
[0018] In hyperspectral unmixing, this model assumes that the spectral response of each pixel can be represented by a linear combination of the spectra of its constituent endmembers, and its mathematical expression can be written as:
[0019] (1),
[0020] In formula (1) , where is the abundance coefficient, and the elements in each column represent the proportion of the corresponding endmember spectrum in a single mixed pixel; It is Gaussian noise;
[0021] By introducing sparse constraints and total variation space constraints, the optimization solution function of the model described in equation (1) is obtained:
[0022] (2),
[0023] In equation (2), and For regularization parameters, ; Denotes the Frobenius norm of a matrix; Representing a matrix Norm; , Represents the abundance coefficient matrix No. Line 1 The elements of the column, when When it is a non-negative value, It equals 0, otherwise it equals positive infinity; This is the total variation regularization term.
[0024] Preferably, the alternating iterative solution in step 3 is specifically as follows:
[0025] Introducing auxiliary variables and ,in express Spatial gradient, The spatial gradient operator has discrete gradients in the horizontal and vertical directions, respectively. and , express No. Line 1 In vectorized form, the spatial gradient of the matrix is represented as a block matrix, i.e. ;
[0026] Equation (2) can be transformed into:
[0027] (3),
[0028] In equation (3), Let the norm of total variation with different species be defined as follows: , and They represent in No. Line 1 The gradient components in the horizontal and vertical directions at column elements. express No. Line 1 Column elements;
[0029] Define function Equation (3) can be transformed into the method of augmented Lagrange multipliers:
[0030] (4),
[0031] In equation (4), For Lagrange multipliers, The penalty factor decomposes equation (4) into three subproblems:
[0032] Keep other variables fixed and update the abundance coefficients. :
[0033] (5),
[0034] In equation (5), Represents the identity matrix. This is the transpose of the spatial gradient operator. The discrete Laplace operator is defined as follows: , express No. Line 1 Column elements;
[0035] Fix other variables and update auxiliary variables. :
[0036] (6),
[0037] In equation (6), It is a soft threshold operator, and ,in For non-negative parameters, Represents a symbolic function;
[0038] Fix other variables and update auxiliary variables. :
[0039] (7),
[0040] With other variables fixed, update the Lagrange multipliers. :
[0041] (8).
[0042] Preferably, the construction of the depth unfolded network in step 4 is as follows:
[0043] The alternating direction multiplier method The iteration process unfolds into a process with A deep neural network with several stages, where equations (5), (6), (7), and (8) are calculated sequentially in each stage.
[0044] Invert the matrix in equation (5) Fixed as based on endmember spectral library The pre-calculated terms, and the penalty factor and Replace with learnable parameters and Equation (5) can be expressed as:
[0045] (9),
[0046] The threshold of the soft threshold operator in equations (6) and (7) , Replace with learnable parameters , Subtraction is used to approximate the soft threshold operator, and a linear rectified function is employed. limit and The nonnegativity of can be expressed as:
[0047] (10)
[0048] Equation (7) can be expressed as:
[0049] (11),
[0050] Introduce learnable parameters into equation (8) Equation (8) can be expressed as:
[0051] (12)
[0052] The set of learnable parameters is as follows: .
[0053] Preferably, the given loss function in step 5 is a comprehensive loss function. , is the spectral angular distance loss Mean square error loss and spectral information divergence loss The weighted sum is defined as follows:
[0054] (13)
[0055] In equation (13), , These are preset weighting coefficients; , , Based on the first Abundance estimate obtained in the second iteration Hyperspectral data for network reconstruction With input hyperspectral data The calculation formula is as follows:
[0056] (14)
[0057] In equation (14), and These represent the input hyperspectral data. and reconstructed hyperspectral data The The spectral vector of each pixel; Indicates the total number of pixels; express Norm.
[0058] Compared with existing technologies, this invention has the following significant advantages: (1) It explicitly introduces total variation space constraints, forcing the network to learn a piecewise smooth abundance map in space, effectively suppressing noise interference and significantly improving the unmixing accuracy and robustness to complex mixed noise; (2) It transforms the traditional optimization algorithm that requires hundreds of iterations into a neural network with one forward propagation, thereby improving the computational efficiency by orders of magnitude and making it more suitable for real-time or near-real-time processing of large-scale hyperspectral data; (3) The network structure is based on a rigorous mathematical model, and each layer has a clear mathematical and physical meaning, overcoming the shortcomings of the black box model of deep learning. At the same time, the introduction of learnable parameters enables the model to adaptively learn the optimal solution strategy from the data, avoiding tedious and inaccurate manual parameter tuning; (4) It adopts an unsupervised learning mode, requiring only input of hyperspectral images and public spectral libraries, without any manually labeled ground truth, reducing the application threshold and making it highly universal. Attached Figure Description
[0059] Figure 1 This is a flowchart of the total variation regularized depth unrolling network method for hyperspectral image unmixing according to the present invention;
[0060] Figure 2 These are abundance maps obtained from simulated hyperspectral data using different unmixing algorithms; where:
[0061] Figure 2 (a) is the true abundance map of the generated simulated hyperspectral data;
[0062] Figure 2 (b) is the abundance map obtained from simulated hyperspectral data using the SUnSAL algorithm;
[0063] Figure 2 (c) is the abundance map obtained from simulated hyperspectral data using the RSU algorithm;
[0064] Figure 2 (d) is the abundance map obtained from simulated hyperspectral data using the DAE algorithm;
[0065] Figure 2 (e) is the abundance map obtained by simulating hyperspectral data using a total variation regularized depth unfolding network method for hyperspectral image demixing according to the present invention. Detailed Implementation
[0066] The specific steps to achieve the content of this invention are as follows:
[0067] Step 1, Input hyperspectral image data and endmember spectral library , It is the number of bands. It is the number of pixels in the image. It is an end-member number.
[0068] Step 2, based on the linear mixture model (1) Construct a hyperspectral data unmixing model that integrates sparse constraints and total variation spatial constraints to optimize the solution function. (2), of which For abundance coefficients, the elements in each column represent the proportion of the corresponding endmember spectrum in a single mixed pixel; It is Gaussian noise; and For regularization parameters, ; Denotes the Frobenius norm of a matrix; Representing a matrix Norm; , Representation matrix No. Line 1 The elements of the column, when When it is a non-negative value, It equals 0, otherwise it equals positive infinity; This is the total variation regularization term.
[0069] Step 3, solve the problem iteratively, as follows:
[0070] Step 3.1, first introduce auxiliary variables. and The optimization solution function is transformed into:
[0071] (3),
[0072] in The equivalent augmented Lagrange form of this expression is:
[0073] (4),
[0074] in , For Lagrange multipliers, This is the penalty factor. The above formula uses the alternating direction multiplier method, iteratively solving for each variable in an alternating manner, and can be decomposed into three sub-problems:
[0075] Step 3.2: Fix other variables and update the abundance coefficients. :
[0076] (5),
[0077] Step 3.3: Fix other variables and update auxiliary variables. :
[0078] (6),
[0079] In the above formula, It is a soft threshold operator, and ,in For non-negative parameters, Represents a symbolic function;
[0080] Step 3.4: Fix other variables and update auxiliary variables. :
[0081] (7),
[0082] Step 3.5: Fix other variables and update the Lagrange multipliers. :
[0083] (8),
[0084] Step 4: Construct a deep unfolded network, incorporating the alternating direction multiplier method. The next iteration process unfolds into having The deep neural network is constructed in several stages, and the hyperparameters and several approximate operators in the iteration are parameterized into learnable parameters. The steps are as follows:
[0085] Step 4.1, will Matrix inversion operation in the update subproblem Fixed as based on endmember spectral library The pre-calculated terms, and the penalty factor and Replace with learnable parameters and ,Will Updated to:
[0086] (9),
[0087] Step 4.2, set the threshold of the soft threshold operator. , Replace with learnable parameters , Subtraction is used to approximate the soft threshold operator, and a linear rectified function is employed. limit and The nonnegativity of will Updated to:
[0088] (10)
[0089] Step 4.3, using the same method as in step 4.2, will proceed to the next stage. Updated to:
[0090] (11),
[0091] Step 4.4, Introduce learnable parameters The next stage Updated to:
[0092] (12)
[0093] Step 5: Train the deep unfolded network, based on the hyperspectral data and endmember spectral library, and give the following loss function:
[0094] (13)
[0095] in, It is a set of learnable parameters ; , These are preset weighting coefficients; It is the spectral angular distance loss. It is the mean square error loss, It is the spectral information divergence loss, based on the first... Abundance estimate obtained in the second iteration Hyperspectral data for network reconstruction With input hyperspectral data get:
[0096] (14)
[0097] In the above formula, and These represent the input hyperspectral data. and reconstructed hyperspectral data The The spectral vector of each pixel; Indicates the total number of pixels; express Norm.
[0098] The parameters in the deep unfolded network are continuously optimized using backpropagation and gradient descent algorithms. And obtain the reconstructed spectrum until the loss function Stablize;
[0099] Step 6: After training, new inputs only need to be forward-propagated to obtain fast abundance coefficient estimation. .
[0100] The effects of this invention can be further illustrated by the following simulation experiments:
[0101] Simulation conditions:
[0102] The simulation experiment used simulated hyperspectral data. The spectral library was obtained from the Ground Feature Spectral Database published by the U.S. Geological Survey (USGS), containing 224 bands and 240 pure material endmembers. For the simulated hyperspectral data, the image size was 64. 64 pixels, containing 224 bands, with Gaussian noise added at a signal-to-noise ratio of 40 dB. That is, the input hyperspectral data is... The endmember spectral library is .
[0103] The evaluation metrics used in this simulation experiment are the mean square error of abundance (AMSE) and the signal-to-reconstruction error (SRE). The smaller the mean square error of abundance and the larger the signal-to-reconstruction error, the higher the unmixing accuracy.
[0104] Simulation content:
[0105] This invention employs simulated hyperspectral data to test the unmixing performance of the algorithm. To evaluate the performance of the algorithm, a proposed total variation regularized deep unfolded network method (TVDU-Net) for hyperspectral image unmixing is compared with currently popular unmixing algorithms internationally. The comparison methods include: a sparse unmixing algorithm based on variable splitting and augmented Lagrange (SUnSAL), and a algorithm based on... The robust sparse unmixing algorithm based on norm (RSU) and the unmixing algorithm based on deep neural network autoencoder (DAE).
[0106] Analysis of simulation experiment results:
[0107] Table 1 shows the comparison results of mean square error of abundance and signal-reconstruction error of simulated hyperspectral data under different unmixing algorithms:
[0108] Table 1 - AMSE and SRE of different algorithms on simulated hyperspectral data:
[0109]
[0110] As shown in Table 1, under simulated hyperspectral data conditions, TVDU-Net has the lowest mean square error of abundance and the highest signal-to-reconstruction error compared to other algorithms because it incorporates the total variation regularization prior into the sparse unmixing model and unfolds it into a deep network.
[0111] Figure 2The abundance maps obtained from simulated hyperspectral data using different unmixing algorithms are shown. It can be seen that the abundance maps obtained by SUnSAL, RSU and DAE all contain a certain degree of noise. In contrast, TVDU-Net shows higher unmixing accuracy and can obtain results that are closer to the real abundance maps.
[0112] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.
Claims
1. A total variation regularized depth unrolling network method for hyperspectral image unmixing, characterized in that, Includes the following steps: Step 1: Input hyperspectral image data and endmember spectral library; Step 2: Construct a hyperspectral data unmixing model that integrates sparse constraint terms and total variation regularization terms, and optimize its objective function; Step 3: Transform the objective function into an equivalent augmented Lagrangian form and use the alternating direction multiplier method to iteratively solve for each variable; Step 4: Expand the iterative process into a deep learnable network structure with several layers, and parameterize the hyperparameters and several approximate operators in the iteration into learnable network parameters; Step 5: Construct a comprehensive loss function that combines spectral reconstruction error and spectral similarity measurement. Based on the hyperspectral data and endmember spectral library, continuously optimize the network parameters using backpropagation and gradient descent methods. Step 6: Input the hyperspectral data into the trained deep network and perform forward propagation to obtain abundance coefficient estimates.
2. The method according to claim 1, characterized in that, The input hyperspectral data and endmember spectral library mentioned in step 1 are as follows: Input the hyperspectral image data to be unmixed and endmember spectral library , It is the number of bands. It is the number of pixels in the image. It is the end-member number.
3. The method according to claim 2, characterized in that, Step 2, which involves constructing a hyperspectral data unmixing model optimization solution function that includes sparse priors and spatial total variation priors, is detailed below: In hyperspectral unmixing, this model assumes that the spectral response of each pixel can be represented by a linear combination of the spectra of its constituent endmembers, and its mathematical expression can be written as: （1）， In formula (1) , where is the abundance coefficient, and the elements in each column represent the proportion of the corresponding endmember spectrum in a single mixed pixel; It is Gaussian noise; By introducing sparse constraints and total variation space constraints, the optimization solution function of the model described in equation (1) is obtained: （2）， In equation (2), and For regularization parameters, ; Denotes the Frobenius norm of a matrix; Representing a matrix Norm; , Represents the abundance coefficient matrix No. Line 1 The elements of the column, when When it is a non-negative value, It equals 0, otherwise it equals positive infinity; This is the total variation regularization term.
4. The method according to claim 3, characterized in that, The alternating iterative solution described in step 3 is as follows: Introducing auxiliary variables and ,in express Spatial gradient, The spatial gradient operator has discrete gradients in the horizontal and vertical directions, respectively. and , express No. Line 1 In vectorized form, the spatial gradient of the matrix is represented as a block matrix, i.e. ; Equation (2) can be transformed into: （3）， In equation (3), Let the norm of total variation with different species be defined as follows: , and They represent in No. Line 1 The gradient components in the horizontal and vertical directions at column elements. express No. Line 1 Column elements; Define function Equation (3) can be transformed into the method of augmented Lagrange multipliers: （4）， In equation (4), For Lagrange multipliers, The penalty factor decomposes equation (4) into three subproblems: Keep other variables fixed and update the abundance coefficients. : （5）， In equation (5), Represents the identity matrix. This is the transpose of the spatial gradient operator. The discrete Laplace operator is defined as follows: , express No. Line 1 Column elements; Fix other variables and update auxiliary variables. : （6）， In equation (6), It is a soft threshold operator, and ,in For non-negative parameters, Represents a symbolic function; Fix other variables and update auxiliary variables. : （7）， With other variables fixed, update the Lagrange multipliers. : （8）。 5. The method according to claim 4, characterized in that, The construction of the deep unfolded network described in step 4 is as follows: The alternating direction multiplier method The iteration process unfolds into a process with A deep neural network with several stages, where equations (5), (6), (7), and (8) are calculated sequentially in each stage. Invert the matrix in equation (5) Fixed as based on endmember spectral library The pre-calculated terms, and the penalty factor and Replace with learnable parameters and Equation (5) can be expressed as: （9）， The threshold of the soft threshold operator in equations (6) and (7) , Replace with learnable parameters , Subtraction is used to approximate the soft threshold operator, and a linear rectified function is employed. limit and The nonnegativity of can be expressed as: （10）， Equation (7) can be expressed as: （11）， Introduce learnable parameters into equation (8) Equation (8) can be expressed as: （12）， The set of learnable parameters is as follows: .
6. The method according to claim 1, characterized in that, The given loss function mentioned in step 5 is the comprehensive loss function. , is the spectral angular distance loss Mean square error loss and spectral information divergence loss The weighted sum is defined as follows: （13）， In equation (13), , These are preset weighting coefficients; , , Based on the first Abundance estimate obtained in the second iteration Hyperspectral data for network reconstruction With input hyperspectral data The calculation formula is as follows: （14）， In equation (14), and These represent the input hyperspectral data. and reconstructed hyperspectral data The The spectral vector of each pixel; Indicates the total number of pixels; express Norm.