A dual-domain constraint hyperspectral image reconstruction method based on deep learning

Abstract

The application discloses a kind of dual-domain constraint hyperspectral image reconstruction methods based on deep learning, it is related to the technical field of artificial intelligence and computational imaging.The application simultaneously imposes constraint in spectral reconstruction domain and sparse coefficient domain, while ensuring that its inherent sparse structure conforms to physical prior, the complement and verification of dual-domain information significantly improve the fidelity of reconstruction result, reduce artifact and noise;Using deep neural network to realize reconstruction, only once forward propagation is needed for new compression measurement value to output reconstruction result, which greatly reduces the computational complexity, improves the reconstruction speed, and has the potential for real-time processing;The sparsity physical prior is explicitly integrated into the network learning goal, which can effectively resist noise interference and reduce the influence of noise on the reconstruction result;Through back propagation algorithm, the dual-head deep neural network is optimized and trained end-to-end, and the best mapping relationship is automatically learned, which greatly reduces the operation difficulty and application threshold of the method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computational imaging and artificial intelligence, in particular to a dual-domain constrained hyperspectral image reconstruction method based on deep learning. BACKGROUND

[0002] Hyperspectral imaging technology can simultaneously obtain two-dimensional spatial information and one-dimensional spectral information of a target to form a three-dimensional data cube. The essence of this process is to solve an underdetermined linear inverse problem, and the original hyperspectral data can be reconstructed from compressed measurements. Existing reconstruction algorithms mainly fall into two categories:

[0003] 1. Traditional algorithms based on iterative optimization: use prior knowledge such as sparsity of hyperspectral images to transform the reconstruction problem into a convex optimization problem for solving; for example, by minimizing the L1 norm to promote sparsity in a certain transform domain; these algorithms have a solid theoretical foundation, but are usually computationally complex, slow to reconstruct, and require manual adjustment of regularization parameters, making it difficult to meet the needs of real-time applications;

[0004] 2. Algorithms based on deep learning: through training on a large amount of data, achieve extremely fast reconstruction speed. However, most existing deep learning models directly learn the mapping relationship between measurements and spectra, ignoring the crucial sparsity prior in compressive sensing theory; this approach may result in insufficient model generalization, sensitivity to noise, and weak physical interpretability, and the reconstruction results may sometimes violate the intrinsic structural properties of the signal.

[0005] Therefore, it is of great theoretical significance and application value to develop a new method that not only has the efficiency of deep learning, but also deeply integrates the core physical prior of signal sparsity into network design and training process, thereby improving reconstruction fidelity and robustness. SUMMARY

[0006] The purpose of the present application is to provide a dual-domain constrained hyperspectral image reconstruction method based on deep learning, which uses a deep neural network with dual output heads to apply constraints in the spectral reconstruction domain and the sparse transform domain simultaneously for end-to-end training, thereby achieving fast and high-fidelity recovery of three-dimensional hyperspectral data cubes from single-frame two-dimensional compressed measurements, and realizing fast and high-precision reconstruction of hyperspectral images to solve the problems raised in the background technology.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] A dual-domain constrained hyperspectral image reconstruction method based on deep learning, comprising:

[0009] obtaining a compressed measurement value, using a snapshot compressive spectral imaging system to optically encode and sample a target scene, obtaining a single two-dimensional compressed measurement image, and arranging an encoded compressed signal corresponding to each spatial position in the single two-dimensional compressed measurement image into a measurement value vector;

[0010] constructing a double-head deep neural network model, taking the measurement value vector as input data of the double-head deep neural network, wherein the double-head deep neural network comprises a shared feature extraction backbone network and two parallel output heads;

[0011] constructing a double-domain joint loss function, which is composed of a reconstruction domain loss and a sparse domain loss weighted together, wherein the sparse domain loss comprises a sparsity promotion loss and a coefficient reconstruction loss;

[0012] end-to-end model training, obtaining a data pair of the measurement value vector and the real spectrum, based on the data pair of the measurement value vector and the real spectrum, optimizing and training the double-head deep neural network through a back propagation algorithm, minimizing the double-domain joint loss function, until the double-head deep neural network model converges;

[0013] fast reconstruction and deployment, deploying the trained and converged double-head deep neural network model, based on the double-head deep neural network model, performing a forward propagation on a newly obtained compressed measurement value, and taking an output of the reconstruction domain head as a final hyperspectral reconstruction result.

[0014] Further, in the obtaining of the compressed measurement value, the imaging process of the snapshot compressive spectral imaging system satisfies a linear model, and the linear model is as follows:

[0015]

[0016] wherein, is the measurement value vector, represents a sensing matrix of the optical encoding and compression process of the snapshot compressive spectral imaging system, is a vectorized original spectrum signal, is a system noise; and the dimension of the measurement value vector is less than the number of wavebands of the original spectrum signal;

[0017] Further, the spatial position is specifically a 4x4 pixel block, and the encoded compressed signal corresponding to each 4x4 pixel block is arranged into a measurement value vector, and the dimension of the measurement value vector is consistent with the total number of pixels of the 4x4 pixel block.

[0018] Further, the double-head deep neural network model specifically comprises:

[0019] Local spatial feature extraction: Based on the spatial location corresponding to the measurement value vector, a convolutional neural network front-end is used to reshape and extract features from the measurement value vector, extracting local spatial features that are reshaped into a single-channel two-dimensional feature map;

[0020] Global feature acquisition: The extracted local spatial features are flattened to obtain local spatial feature vectors, and the local spatial feature vectors are input into the multilayer perceptron backbone network for deep nonlinear feature extraction to obtain global feature vectors;

[0021] The dual-head deep neural network model outputs: the global feature vector obtained by the multilayer perceptron backbone network is fed into two independent fully connected layers to generate the reconstructed spectral curve and sparse coefficient values, respectively.

[0022] Furthermore, the two independent fully connected layers include a spectral head and a coefficient head, forming a reconstruction domain head and a sparse domain head. The spectral head is used to map the global feature vector to the reconstructed spectral curve at the corresponding spatial location, and the coefficient head is used to map the global feature vector to the sparse coefficient values ​​of the reconstructed spectral curve under a preset sparse basis. The preset sparse basis is a discrete wavelet transform basis, and the sparse coefficient values ​​output by the sparse domain head are the estimated wavelet decomposition coefficients of the reconstructed spectral curve under the discrete wavelet transform basis.

[0023] Furthermore, in the dual-domain joint loss function, the reconstruction domain loss is used to calculate the difference between the spectral curve output by the reconstruction domain head and the true spectral curve. The sparse domain loss includes a sparsity promotion loss to promote the sparsity of the sparse coefficients, and a coefficient reconstruction loss to calculate the error between the indirect reconstructed spectrum obtained after the sparse coefficients output by the sparse domain head are subjected to sparse inverse transformation and the true spectrum.

[0024] Furthermore, the dual-domain joint loss function is shown in the following formula:

[0025]

[0026] In the formula, For reconstructing domain loss; Weighting coefficients that contribute to the loss due to sparsity; Sparsity promotes loss; These are the weighting coefficients for the coefficient reconstruction loss; For coefficient reconstruction loss;

[0027] The reconstruction domain loss is a composite term, as shown in the following formula:

[0028]

[0029] In the formula, for The weight of the loss; is a weight of the SAM loss; is a weight of the spectral smoothness loss; and is a weight of the SAM loss; is a weight of the spectral smoothness loss; and ; is a real spectral curve, is a reconstructed spectral curve output by the reconstruction domain head; is a mean absolute error between and , is a mean square error between and ; is a spectral angle matching error between and ; is a spectral smoothness error between and ;

[0030] The sparsity-promoting loss is shown in the following formula:

[0031]

[0032] In the formula, is a sparse coefficient value of the sparse domain head output, denotes an L2 norm; ;

[0033] The coefficient reconstruction loss is shown in the following formula:

[0034]

[0035] In the formula, denotes an inverse discrete wavelet transform, is a sparse coefficient value of the sparse domain head output, is a real spectral curve, denotes an L2 norm; ;

[0036] Further, the generation process of the data pair of the measurement value vector and the real spectrum is as follows:

[0037] A plurality of groups of real spectral signals are extracted from a hyperspectral data set, wherein the number of wavebands of the real spectral signals is consistent with the number of wavebands of the original spectral signals;

[0038] The real spectral signals are linearly transformed based on a sensing matrix of an optical encoding compression process of a snapshot compressive spectral imaging system to obtain a noise-free measurement vector;

[0039] System noise is superimposed in the noise-free measurement vector according to a preset signal-to-noise ratio to obtain a measurement value vector;

[0040] Pair each of the measurement value vectors with the corresponding true spectral signal to form a data pair of the measurement value vector and the true spectrum, and divide the training set and the verification set according to a preset ratio.

[0041] Further, the fast reconstruction process further comprises spectral inversion:

[0042] Extract a normalized reconstruction spectrum curve corresponding to the current compressed measurement value from the reconstruction domain header output, and save the global maximum value during normalization of the training data;

[0043] Perform point-by-point multiplication operation on each waveband pixel intensity value of the normalized reconstruction spectrum curve and the global maximum value to obtain a physical intensity reconstruction spectrum curve;

[0044] Reasonably check the physical intensity reconstruction spectrum curve, and output a hyperspectral reconstruction result based on the checking result.

[0045] Further, the optical encoding module of the snapshot compression spectral imaging system is any one of a microlens array, a digital micromirror device or a coded aperture, and the encoding mode of the optical encoding module corresponds to the column vector of the sensing matrix one by one.

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] 1. By simultaneously applying constraints in the spectral reconstruction domain and the sparse coefficient domain, the accuracy of the reconstructed spectrum in numerical value and shape is ensured, and the internal sparse structure is ensured to conform to the physical priori. The complementarity and verification of the dual-domain information significantly improve the fidelity of the reconstruction result, reduce artifacts and noise; the reconstruction is realized by using a deep neural network, and for new compressed measurement values, only one forward propagation is needed to output the reconstruction result, without manual intervention in the iteration process, which greatly reduces the computational complexity, improves the reconstruction speed, and has the potential for real-time processing;

[0048] 2. The sparse physical priori is explicitly integrated into the network learning goal, so that the model learns the signal essential structural representation, enhances the resistance to noise and the generalization ability to unseen data, can effectively resist noise interference, reduces the influence of noise on the reconstruction result, and improves the adaptability of the method in complex practical applications; the dual-head deep neural network is optimized and trained end-to-end by using the back propagation algorithm, and the best mapping relationship is automatically learned, which greatly reduces the operation difficulty and application threshold of the method, avoids the problem that the traditional iterative algorithm causes the reconstruction effect to fluctuate due to improper parameter adjustment, so that non-professional technical personnel can also conveniently and stably use the method for hyperspectral image reconstruction, and promote the popularization and application of hyperspectral imaging technology in more fields. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 It is a whole flow chart of the dual-domain constraint hyperspectral image reconstruction method of the present application.

[0050] Figure 2 This is a schematic diagram of the dual-head deep neural network structure of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the construction of the dual-domain joint loss function of the present invention;

[0052] Figure 4 This is a flowchart of the model training and reconstruction process of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] To address the technical challenges of slow imaging with traditional hyperspectral imaging equipment, low efficiency and manual parameter tuning required by traditional iterative reconstruction algorithms, and poor generalization and insufficient accuracy caused by deep learning reconstruction models ignoring sparse priors, please refer to [link to relevant documentation]. Figures 1-4 This embodiment provides the following technical solution:

[0055] A deep learning-based dual-domain constrained hyperspectral image reconstruction method includes:

[0056] To obtain compressed measurement values, a snapshot-type compressed spectral imaging system is used to optically encode and sample the target scene, obtain a single-frame two-dimensional compressed measurement image, and organize the encoded compressed signal corresponding to each spatial location in the single-frame two-dimensional compressed measurement image into a measurement value vector.

[0057] A dual-head deep neural network model is constructed, and the measured value vector is used as the input data of the dual-head deep neural network. The dual-head deep neural network includes a shared feature extraction backbone network and two parallel output heads, namely a reconstruction domain head for predicting and outputting the reconstructed spectral curve at the corresponding spatial location, and a sparse domain head for predicting and outputting the sparse coefficient value of the spectral curve under the dual-head sparse basis.

[0058] Construct a dual-domain joint loss function, which is a dual-domain joint loss function containing multiple components. The dual-domain joint loss function is composed of a weighted sum of reconstruction domain loss and sparse domain loss. The sparse domain loss includes a sparsity promotion loss for promoting the sparsity of sparse coefficient values ​​and a coefficient reconstruction loss for verifying the effectiveness of sparse coefficient values.

[0059] End-to-end model training involves acquiring data pairs of measured value vectors and true spectra. Based on these data pairs, the dual-head deep neural network is optimized end-to-end using the backpropagation algorithm to minimize the dual-domain joint loss function until the dual-head deep neural network model converges.

[0060] Rapid reconstruction and deployment: Deploy a trained and converged dual-head deep neural network model, perform a forward propagation on the newly acquired compressed measurement values ​​based on the dual-head deep neural network model, and extract the output of the reconstruction domain head as the final hyperspectral reconstruction result.

[0061] In this embodiment, a dual-domain constraint design ensures both the numerical and morphological accuracy of the reconstructed spectrum and its compliance with sparse physical priors, significantly improving reconstruction fidelity and reducing artifact noise. Reconstruction can be completed with a single forward propagation using a deep neural network, greatly increasing speed and providing real-time processing potential. Integrating sparse priors into network learning enhances the model's noise resistance and generalization ability. End-to-end training automatically learns the optimal mapping relationship without manual parameter tuning, lowering the application threshold and making it suitable for hyperspectral image acquisition scenarios across multiple fields.

[0062] In this embodiment, the imaging process of the snapshot-type compressed spectral imaging system in acquiring compressed measurement values ​​follows a linear model, the formula of which is shown below:

[0063]

[0064] In the formula, For the measurement value vector, The sensor matrix represents the optical encoding compression process of a snapshot-type compressed spectral imaging system. The original spectral signal is vectorized. This represents system noise; and the dimension of the measured value vector is less than the number of bands in the original spectral signal.

[0065] In this embodiment, we assume the sensing matrix It is known that, determined by the hardware structure and imaging parameters, it will generate a spectral vector containing L bands. Compressed into a vector of measured values ,in , To measure the dimension of the value vector, This represents the number of bands in the original spectral signal;

[0066] In this embodiment, the optical encoding module of the snapshot compressed spectral imaging system can be any one of a microlens array, a digital micromirror device, or an encoding aperture. It can be flexibly selected according to the hardware requirements and cost budget of different application scenarios such as remote sensing, medical imaging, and industrial inspection. The encoding mode of the optical encoding module corresponds one-to-one with the column vectors of the sensing matrix, ensuring that the optical encoding and mathematical model are accurately matched, and reducing measurement errors caused by adaptation deviation.

[0067] In this embodiment, the spatial location is specifically a 4×4 pixel block. The encoded compression signal corresponding to each 4×4 pixel block is organized into a measurement value vector, and the dimension of the measurement value vector is consistent with the total number of pixels in the 4×4 pixel block.

[0068] In this embodiment, it is clearly stated that the snapshot-type compressed spectral imaging system satisfies a linear model, and the sensing matrix is ​​determined by the hardware structure and imaging parameters. This makes the physical logic of the imaging process clear and traceable, avoiding the uncontrollable problems caused by black-box imaging. It provides reliable physical model support for the subsequent accurate reconstruction of the original spectral signal from the compressed measurement values. By using a linear model, the original spectral vector containing L bands is compressed into a measurement value vector of a certain dimension. While ensuring the preservation of key spectral and spatial information, the amount of data is greatly reduced. This reduces the pressure on data storage and transmission, and also meets the high efficiency characteristics of snapshot imaging in obtaining compressed data in a single snapshot, thus adapting to the imaging needs of dynamic scenes.

[0069] In this embodiment, the dual-head deep neural network model specifically includes:

[0070] Local spatial feature extraction: Based on the spatial location corresponding to the measurement value vector, a convolutional neural network front-end is used to reshape and extract features from the measurement value vector, extracting local spatial features that are reshaped into a single-channel two-dimensional feature map;

[0071] Global feature acquisition: The extracted local spatial features are flattened to obtain local spatial feature vectors, and the local spatial feature vectors are input into the multilayer perceptron backbone network for deep nonlinear feature extraction to obtain global feature vectors;

[0072] The dual-head deep neural network model outputs: the global feature vector obtained by the multilayer perceptron backbone network is fed into two independent fully connected layers to generate the reconstructed spectral curve and sparse coefficient values, respectively.

[0073] In this embodiment, the two independent fully connected layers include a spectral head and a coefficient head, forming a reconstruction domain head and a sparse domain head. The spectral head is used to map the global feature vector to the reconstructed spectral curve at the corresponding spatial location, and the coefficient head is used to map the global feature vector to the sparse coefficient values ​​of the reconstructed spectral curve under a preset sparse basis. The preset sparse basis is a discrete wavelet transform basis, and the sparse coefficient values ​​output by the sparse domain head are the estimated wavelet decomposition coefficients of the reconstructed spectral curve under the discrete wavelet transform basis.

[0074] In this embodiment, the dual-head deep neural network model is specifically a CNN-MLP-DualHead model. The multilayer perceptron backbone network consists of multiple residual blocks. Each residual block contains a convolutional layer, a batch normalization layer, and a ReLU activation layer. Gradient transfer is achieved between adjacent residual blocks through shortcut connections.

[0075] In this embodiment, the measurement value vector is reshaped into a single-channel two-dimensional feature map and local spatial features are extracted through the front end of a convolutional neural network, accurately capturing local correlation information in the measurement data. After flattening, deep nonlinear feature extraction is performed by a multilayer perceptron backbone network containing multiple residual blocks, which can effectively enhance the feature expression capability, avoid the gradient vanishing problem in deep network training, and ensure the efficient extraction and quality of global feature vectors. Two independent fully connected layers form a reconstruction domain head and a sparse domain head. The spectral head directly maps the global feature vector to obtain the reconstructed spectral curve at the corresponding spatial location, while the coefficient head maps to obtain the estimated wavelet decomposition coefficients of the spectral curve under the discrete wavelet transform basis, realizing the parallel output of the spectral reconstruction domain and the sparse coefficient domain. This not only ensures the direct generation of the reconstructed spectrum, but also verifies the physical rationality of the spectrum through sparse coefficients, improving the reliability of the reconstruction results.

[0076] In this embodiment, in the dual-domain joint loss function, the reconstruction domain loss is used to calculate the difference between the spectral curve output by the reconstruction domain head and the true spectral curve. The sparse domain loss includes the sparsity promotion loss for promoting the sparsity of the sparse coefficients, and the coefficient reconstruction loss for calculating the error between the indirect reconstructed spectrum and the true spectrum obtained after the sparse coefficients output by the sparse domain head are subjected to the inverse sparse transformation.

[0077] In this embodiment, the dual-domain joint loss function is shown in the following formula:

[0078]

[0079] In the formula, For reconstructing domain loss; Weighting coefficients that contribute to the loss due to sparsity; Sparsity promotes loss; These are the weighting coefficients for the coefficient reconstruction loss; For coefficient reconstruction loss;

[0080] The reconstruction domain loss is a composite term, as shown in the following formula:

[0081]

[0082] In the formula, for The weight of the loss; for The weight of the loss; The weights for the SAM loss; The weights for spectral smoothness loss; and ; For the true spectral curve, To reconstruct the reconstructed spectral curve output by the domain header; for and The mean absolute error, for and The mean square error; for and spectral angle matching error; for Spectral smoothing error;

[0083] The sparsity-enhancing loss is shown in the following formula:

[0084]

[0085] In the formula, These are the sparse coefficient values ​​output by the sparse domain header. express Norm;

[0086] The coefficient reconstruction loss is shown in the following formula:

[0087]

[0088] In the formula, Represents the inverse discrete wavelet transform. These are the sparse coefficient values ​​output by the sparse domain header. For the true spectral curve, express The square of the norm.

[0089] In this embodiment, the difference between the reconstructed spectrum and the true spectrum is measured from multiple dimensions, including mean absolute error, mean square error, spectral angle consistency, and spectral curve smoothness. This comprehensively constrains the output of the reconstruction domain head, ensuring that the reconstructed spectrum closely matches the true spectrum in terms of numerical values, morphology, and physical properties, effectively reducing the biased errors that may result from a single loss function. The sparsity-enhancing loss, through norm constraints, forces the coefficients of the sparse domain head output to be sparse, matching the sparse characteristics of hyperspectral signals under the discrete wavelet transform basis, enabling the model to learn the essential structured representation of the signal. The coefficient reconstruction loss calculates the norm squared error between the indirectly reconstructed spectrum and the true spectrum after the sparse coefficients undergo inverse discrete wavelet transform, establishing a closed-loop verification between the sparse domain and the reconstruction domain, ensuring the effectiveness of the sparse coefficients, and allowing the sparse prior to truly serve to improve reconstruction accuracy. By setting weight coefficients for the reconstruction domain loss, sparsity-enhancing loss, and coefficient reconstruction loss, the influence of different losses in model training can be flexibly balanced, allowing the model to simultaneously consider reconstruction accuracy, sparsity characteristics, and coefficient effectiveness during the learning process. This avoids performance deviations caused by a single loss dominating training, improving the overall optimization effect and generalization ability of the model.

[0090] In this embodiment, the process of generating the data pair of the measured value vector and the true spectrum is as follows:

[0091] Multiple sets of real spectral signals are extracted from the hyperspectral dataset, wherein the number of bands in the real spectral signals is consistent with the number of bands in the original spectral signals;

[0092] Based on the sensing matrix of the optical coding compression process of the snapshot compressed spectral imaging system, a linear transformation is performed on the real spectral signal to obtain a noise-free measurement vector.

[0093] The system noise is superimposed on the noise-free measurement vector according to the preset signal-to-noise ratio to obtain the measurement value vector;

[0094] Each set of measured value vectors is paired with the corresponding real spectral signal to form a data pair of measured value vector and real spectrum, and the training set and validation set are divided according to a preset ratio.

[0095] In this embodiment, the rapid reconstruction process further includes spectral inversion:

[0096] Extract the normalized reconstructed spectral curve corresponding to the current compressed measurement value from the output of the reconstructed domain head, and save the global maximum value when the training data is normalized;

[0097] The pixel intensity value of each band of the normalized reconstructed spectral curve is multiplied point by point with the global maximum value to obtain the physical intensity reconstructed spectral curve;

[0098] The rationality of the reconstructed spectral curve based on physical intensity is verified, and the final output of the hyperspectral reconstruction result with physical intensity significance is based on the verification result.

[0099] In this embodiment, the backpropagation algorithm uses the AdamW optimizer and dynamically adjusts the network weights during training;

[0100] In this embodiment, during the training process, the measured value vector and the real spectral signal are subjected to maximum and minimum value normalization processing, and the normalization range is [0,1].

[0101] In this embodiment, the method is applied to hyperspectral image acquisition scenarios in the fields of remote sensing, medical imaging, and industrial inspection.

[0102] In this embodiment, real spectra are extracted from the hyperspectral dataset. After linear transformation of the sensing matrix and superposition of system noise with signal-to-noise ratio, a measurement value vector is obtained. This vector is then paired with the real spectra and divided into training and validation sets to simulate a real imaging noise environment. This makes the training data more realistic and improves the model's generalization and robustness in noisy scenarios. Normalized reconstructed spectral curves are extracted and combined with point-by-point multiplication of the global maximum value during training to obtain physical intensity reconstructed spectral curves. After rationality verification, the reconstructed spectra are accurate and reliable in both numerical and physical intensity, meeting the stringent requirements of remote sensing, medicine, and other fields for the physical meaning of spectra. The AdamW optimizer is used to dynamically adjust network weights, optimize the training process, and normalize the data to [0, 1] maximum and minimum values ​​to eliminate dimensional differences, accelerate model convergence, and effectively improve training efficiency and stability. This allows the model to learn effective mapping relationships more quickly, adapting to the needs of different fields for hyperspectral data and providing high-quality hyperspectral data support for tasks such as material identification and component analysis in various fields, demonstrating strong practicality.

[0103] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based dual-domain constrained hyperspectral image reconstruction method, characterized in that, include: To obtain compressed measurement values, a snapshot-type compressed spectral imaging system is used to optically encode and sample the target scene, obtain a single-frame two-dimensional compressed measurement image, and organize the encoded compressed signal corresponding to each spatial location in the single-frame two-dimensional compressed measurement image into a measurement value vector. A dual-head deep neural network model is constructed, and the measured value vector is used as the input data of the dual-head deep neural network. The dual-head deep neural network includes a shared feature extraction backbone network and two parallel output heads. A dual-domain joint loss function is constructed, which is composed of a weighted sum of reconstruction domain loss and sparse domain loss, wherein the sparse domain loss includes sparsity promotion loss and coefficient reconstruction loss; End-to-end model training involves acquiring data pairs of measured value vectors and true spectra. Based on these data pairs, the dual-head deep neural network is optimized and trained using the backpropagation algorithm to minimize the dual-domain joint loss function until the dual-head deep neural network model converges. Rapid reconstruction and deployment: Deploy a trained and converged dual-head deep neural network model, perform a forward propagation on the newly acquired compressed measurement values ​​based on the dual-head deep neural network model, and extract the output of the reconstruction domain head as the final hyperspectral reconstruction result; The dual-head deep neural network model specifically includes: Local spatial feature extraction: Based on the spatial location corresponding to the measurement value vector, a convolutional neural network front-end is used to reshape and extract features from the measurement value vector, extracting local spatial features that are reshaped into a single-channel two-dimensional feature map; Global feature acquisition: The extracted local spatial features are flattened to obtain local spatial feature vectors, and the local spatial feature vectors are input into the multilayer perceptron backbone network for deep nonlinear feature extraction to obtain global feature vectors; The dual-head deep neural network model outputs: the global feature vector obtained by the multilayer perceptron backbone network is fed into two independent fully connected layers to generate the reconstructed spectral curve and sparse coefficient values, respectively. The two independent fully connected layers include a spectral head and a coefficient head, forming a reconstruction domain head and a sparse domain head. The spectral head is used to map the global feature vector to the reconstructed spectral curve at the corresponding spatial location, and the coefficient head is used to map the global feature vector to the sparse coefficient values ​​of the reconstructed spectral curve under a preset sparse basis. The preset sparse basis is a discrete wavelet transform basis, and the sparse coefficient values ​​output by the sparse domain head are the estimated wavelet decomposition coefficients of the reconstructed spectral curve under the discrete wavelet transform basis.

2. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 1, characterized in that, In acquiring compressed measurement values, the imaging process of the snapshot-type compressed spectral imaging system follows a linear model, the formula of which is shown below: ； In the formula, For the measurement value vector, The sensor matrix represents the optical encoding compression process of a snapshot-type compressed spectral imaging system. The original spectral signal is vectorized. This represents system noise; and the dimension of the measured value vector is less than the number of bands in the original spectral signal.

3. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 2, characterized in that, The spatial location is specifically a 4×4 pixel block. The encoded compressed signal corresponding to each 4×4 pixel block is organized into a measurement value vector. The dimension of the measurement value vector is the same as the total number of pixels in the 4×4 pixel block.

4. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 3, characterized in that, In the dual-domain joint loss function, the reconstruction domain loss is used to calculate the difference between the spectral curve output by the reconstruction domain head and the true spectral curve. The sparse domain loss includes the sparsity promotion loss, which is used to promote the sparsity of the sparse coefficients, and the coefficient reconstruction loss, which is used to calculate the error between the indirect reconstructed spectrum and the true spectrum obtained after the sparse coefficients output by the sparse domain head are subjected to sparse inverse transformation.

5. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 4, characterized in that, The dual-domain joint loss function is shown in the following formula: ； In the formula, For reconstructing domain loss; Weighting coefficients that contribute to the loss due to sparsity; Sparsity promotes loss; These are the weighting coefficients for the coefficient reconstruction loss; For coefficient reconstruction loss; The reconstruction domain loss is a composite term, as shown in the following formula: ； In the formula, for The weight of the loss; for The weight of the loss; The weights for the SAM loss; The weights for spectral smoothness loss; and ; For the true spectral curve, To reconstruct the reconstructed spectral curve output by the domain header; for and The mean absolute error, for and The mean square error; for and spectral angle matching error; for Spectral smoothing error; The sparsity-enhancing loss is shown in the following formula: ； In the formula, These are the sparse coefficient values ​​output by the sparse domain header. express Norm; The coefficient reconstruction loss is shown in the following formula: ； In the formula, Represents the inverse discrete wavelet transform. These are the sparse coefficient values ​​output by the sparse domain header. For the true spectral curve, express The square of the norm.

6. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 5, characterized in that, The process of generating the data pair between the measured value vector and the true spectrum is as follows: Multiple sets of real spectral signals are extracted from the hyperspectral dataset, wherein the number of bands in the real spectral signals is consistent with the number of bands in the original spectral signals; Based on the sensing matrix of the optical coding compression process of the snapshot compressed spectral imaging system, a linear transformation is performed on the real spectral signal to obtain a noise-free measurement vector. The system noise is superimposed on the noise-free measurement vector according to the preset signal-to-noise ratio to obtain the measurement value vector; Each set of measured value vectors is paired with the corresponding real spectral signal to form a data pair of measured value vector and real spectrum, and the training set and validation set are divided according to a preset ratio.

7. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 6, characterized in that, The rapid reconstruction process also includes spectral inversion: Extract the normalized reconstructed spectral curve corresponding to the current compressed measurement value from the output of the reconstructed domain head, and save the global maximum value when the training data is normalized; The pixel intensity value of each band of the normalized reconstructed spectral curve is multiplied point by point with the global maximum value to obtain the physical intensity reconstructed spectral curve; The rationality of the reconstructed spectral curve based on physical intensity is verified, and the hyperspectral reconstruction result is output based on the verification result.

8. The deep learning-based dual-domain constrained hyperspectral image reconstruction method as described in claim 7, characterized in that, The optical encoding module of the snapshot compressed spectral imaging system can be any one of a microlens array, a digital micromirror device, or an encoding aperture, and the encoding mode of the optical encoding module corresponds one-to-one with the column vectors of the sensing matrix.

Images