Hyperspectral compressive sensing reconstruction method based on deep dual-prior progressive fusion
By employing a hyperspectral compressed sensing reconstruction method with deep dual-prior progressive fusion, combined with a self-supervised learning model and an inner loop network training, the blurring and artifact problems in hyperspectral imaging are solved, achieving high-quality image reconstruction.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HUNAN UNIV
- Filing Date
- 2025-09-29
- Publication Date
- 2026-06-04
Smart Images
Figure CN2025125187_04062026_PF_FP_ABST
Abstract
Description
A hyperspectral compressed sensing reconstruction method based on deep dual-prior progressive fusion
[0001] This application claims priority to Chinese Patent Application No. CN202411699618.1, filed on November 26, 2024, entitled “A Hyperspectral Compressed Sensing Reconstruction Method Based on Deep Dual Prior Progressive Fusion”, the entire contents of which are incorporated herein by reference. Technical Field
[0002] This invention belongs to the field of computer technology, and in particular relates to a hyperspectral compressed sensing reconstruction method based on deep dual-prior progressive fusion. Background Technology
[0003] Traditional hyperspectral imaging mainly includes point scanning or line scanning, which involves multiple exposures in the spatial or spectral dimensions during the shooting process to obtain images of the scene. This imaging method is relatively slow and makes it difficult to capture high-speed dynamic scenes.
[0004] Hyperspectral compressed sensing technology can compress three-dimensional spectral data into two-dimensional images by using an encoding matrix, and then reconstruct the compressed two-dimensional measurement images using a reconstruction algorithm, thereby achieving the acquisition of complete spectral data of the entire scene in a single imaging.
[0005] Hyperspectral compressed sensing reconstruction algorithms are a key technology for converting measurement images into three-dimensional spectral images. Self-supervised hyperspectral reconstruction algorithms can reconstruct a single measurement image without using additional training data and have good generalization performance.
[0006] However, current self-supervised learning algorithms based on hyperspectral compressed sensing reconstruction still have significant shortcomings in imaging quality, which manifest as reconstruction blur and artifacts in visualization. Summary of the Invention
[0007] To address the above technical problems, this invention provides a hyperspectral compressed sensing reconstruction method based on deep dual-prior progressive fusion.
[0008] The technical solution adopted by this invention to solve its technical problem is:
[0009] This invention provides a hyperspectral compressed sensing reconstruction method based on deep dual-prior progressive fusion, comprising the following steps:
[0010] S1. Capture the original compressed measurement image, and then normalize the original compressed measurement image to obtain the normalized compressed measurement image.
[0011] S2. Construct an objective function based on the compressed sensing reconstruction problem, and optimize the objective function to obtain the final optimized objective function;
[0012] S3. Establish a self-supervised learning model for hyperspectral compressed sensing reconstruction based on the normalized compressed measurement images; the self-supervised learning model includes an outer loop iterative optimization and two inner loop network training.
[0013] S4. Solve the optimized final objective function using a self-supervised learning model, and obtain the final reconstructed image by fusing the deep prior terms obtained from the training processes of the two inner loop networks.
[0014] Furthermore, step S2 specifically includes the following steps:
[0015] S21. By introducing two deep prior terms, and constructing an objective function based on the compressed sensing reconstruction problem;
[0016] S22. By introducing the first auxiliary variable z1 and the second auxiliary variable z2, the objective function is optimized to obtain the objective function and constraints after the first optimization.
[0017] S23. In order to increase the correlation between the first auxiliary variable z1, the second auxiliary variable z2 and the compressed measurement image y, the objective function after the first optimization is optimized again to obtain the final optimized objective function and constraints.
[0018] Furthermore, the objective function in S21 is:
[0019] Where y represents the normalized compressed measurement image, x represents the hyperspectral image to be determined, and Φ represents the perception matrix; DP1(x) and DP2(x) represent the depth priors introduced from two different inner loop networks; argmin represents the argmin function, which indicates that when... When the value is at its minimum, the value of x is taken as the value of the argmin function, and λ is the weight coefficient of the two depth priors, which are taken to be the same size here;
[0020] The objective function and constraints after the first optimization in S22 are as follows:
[0021] Constraints: x = z1, x = z2;
[0022] The final optimized objective function in S23 is:
[0023] Constraints: x = z1, x = z2;
[0024] Where ρ is the penalty parameter.
[0025] Furthermore, the outer loop iterative optimization in the self-supervised learning model is based on the semi-quadratic splitting method for iterative optimization.
[0026] Furthermore, step S4 specifically includes the following steps:
[0027] S41. First, the semi-quadratic splitting method is used to convert the final optimized objective function into three sub-problems, namely the x-term sub-problem, the z1-term sub-problem, and the z2-term sub-problem. The solution process of the x-term sub-problem is the outer loop iterative optimization process in the self-supervised learning model, and the solution process of the z1-term sub-problem and the z2-term sub-problem is the training process of the two inner loop networks in the self-supervised learning model.
[0028] The subproblem x is expressed by a formula, as follows:
[0029] Where c represents the defined value; The definition operation is indicated by μ, which represents the penalty parameter for forcing x, z1, and z2 to converge to a common fixed point, and Diag(·) represents extracting the diagonal elements of the subsequent matrix. represents element-wise partitioning; T represents the transpose of the matrix;
[0030] S42, By introducing one of the inner loop networks in the self-supervised learning model Output The first auxiliary variable z1 is replaced and the prior term λD1(z1) is absorbed to obtain the optimized z1 term subproblem; another inner loop network in the self-supervised learning model is introduced. Output The second auxiliary variable z2 is used to replace the prior term λD2(z2) and absorb the prior term λD2(z2) to obtain the optimized z2 term subproblem;
[0031] S43. Construct the first loss function used for training the two inner loop networks. Second loss function
[0032] S44. Enter the outer loop iterative optimization to solve the x-term subproblems, obtain the outer loop output feature map, and input the outer loop output feature map into the training of the two inner loop networks respectively, and calculate the first loss function for each. Second loss function The two loss values are used to optimize the inner loop network. and internal circulation network After the training satisfies the inner loop convergence condition, the inner loop training strategy is exited, and the feature maps generated by the two inner loop networks are used as inputs to the outer loop for iterative optimization. After the outer loop iterative optimization is completed, the first loss function is calculated based on the latest outer loop output feature map. Second loss function The loss values are calculated, and the inner loop network is further optimized using the two newly obtained loss values. and internal circulation network The model iterates through the outputs of its two inner loop networks until the self-supervised learning model converges, ultimately outputting x as the final reconstructed image.
[0033] Furthermore, the subproblem z1 in S41 is expressed by a formula, as follows:
[0034] The optimized z1 subproblem in S42 is as follows:
[0035] Where ε1 represents the input to the inner loop network Gaussian noise.
[0036] Furthermore, the z2 subproblem in S41 is expressed by a formula, as follows:
[0037] The optimized z1 subproblem in S42 is as follows:
[0038] Where ε2 represents the input to the inner loop network Gaussian noise.
[0039] Furthermore, the first loss function Specifically:
[0040] Furthermore, the second loss function Specifically:
[0041] Furthermore, S44 specifically includes the following steps:
[0042] S441. First, input the compressed measurement image y, the perception matrix Φ, and the penalty parameter μ; and initialize the output x, the first auxiliary variable z1, the second auxiliary variable z2, and the iteration number k.
[0043] S442, Update iteration count k = k + 1;
[0044] S443. Update the outer loop output feature map x by solving the x-term subproblems.k+1 ;
[0045] S444, Output feature map x from the outer loop k+1 The inputs are fed into the two inner loop networks for training.
[0046] S445. Calculate the first loss function respectively. Second loss function The loss value; and through the first loss function The loss value and the second loss function The loss values are used to update the inner loop network respectively. and internal circulation network
[0047] S446. Determine whether the training of the two inner loop networks has reached the adaptive convergence criterion. If so, pass the training of the inner loop networks. Output and internal circulation network Output Update the final feature maps generated by subproblems z1 and z2 respectively. and Otherwise, repeat steps S445 to S446 until the adaptive convergence criterion is met;
[0048] S447. Determine if the number of iterations k has reached the set number of iterations. If yes, output x as the final reconstructed image. Otherwise, loop from S442 to S447 until the number of iterations k reaches the set number of iterations.
[0049] The beneficial effects of this invention are:
[0050] 1. This invention provides a hyperspectral compressed sensing reconstruction method based on deep dual prior progressive fusion. Compared with existing self-supervised spectral reconstruction methods, it achieves higher imaging quality and avoids abnormalities such as blurring and artifacts because it fully integrates the deep priors obtained from the training processes of two inner loop networks.
[0051] 2. This invention also proposes a self-supervised learning model that integrates deep prior terms from different inner recurrent networks during the reconstruction of compressed images using hyperspectral data, thus avoiding overfitting issues caused by using a single prior. In this model, no additional training images are required; the model solves the problem solely through a model-driven approach on the input compressed measurement image. Attached Figure Description
[0052] Figure 1 is a structural block diagram of the outer loop iterative optimization in this invention;
[0053] Figure 2 is a structural block diagram of the training of the two inner loop networks in this invention;
[0054] Figure 3 is a flowchart of the deep dual-prior progressive fusion method for hyperspectral compressed sensing reconstruction in this invention. Detailed Implementation
[0055] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.
[0056] Referring to Figures 1 and 2, this application provides a hyperspectral compressed sensing reconstruction method based on deep dual prior progressive fusion, comprising the following steps:
[0057] S1. Capture the original compressed measurement image, and then normalize the original compressed measurement image to obtain the normalized compressed measurement image.
[0058] S2. Construct an objective function based on the compressed sensing reconstruction problem, and optimize the objective function to obtain the final optimized objective function;
[0059] S3. Establish a self-supervised learning model for hyperspectral compressed sensing reconstruction based on the normalized compressed measurement images; the self-supervised learning model includes an outer loop iterative optimization and two inner loop network training.
[0060] S4. Solve the optimized final objective function using a self-supervised learning model, and obtain the final reconstructed image by fusing the deep prior terms obtained from the training processes of the two inner loop networks.
[0061] In some embodiments, S2 specifically includes the following steps:
[0062] S21. By introducing two deep prior terms, and constructing an objective function based on the compressed sensing reconstruction problem;
[0063] S22. By introducing the first auxiliary variable z1 and the second auxiliary variable z2, the objective function is optimized to obtain the objective function and constraints after the first optimization.
[0064] S23. Since the optimization of the first auxiliary variable z1 and the second auxiliary variable z2 is not directly related to the compressed measurement image y, this will increase the difficulty for the self-supervised learning model to effectively learn spatial information. Therefore, in order to increase the correlation between the first auxiliary variable z1, the second auxiliary variable z2 and the compressed measurement image y, the objective function after the first optimization is optimized again to obtain the final optimized objective function and constraints.
[0065] In some embodiments, the objective function in S21 is:
[0066] Where y represents the normalized compressed measurement image, x represents the hyperspectral image to be determined, and Φ represents the perception matrix; DP1(x) and DP2(x) represent the depth priors introduced from two different inner loop networks; argmin represents the argmin function, which indicates that when... When the value is at its minimum, the value of x is taken as the value of the argmin function, and λ is the weight coefficient of the two depth priors, which are taken to be the same size here;
[0067] The objective function and constraints after the first optimization in S22 are as follows:
[0068] Constraints: x = z1, x = z2;
[0069] The final optimized objective function in S23 is:
[0070] Constraints: x = z1, x = z2;
[0071] Where ρ is the penalty parameter.
[0072] In some embodiments, the outer loop iterative optimization in the self-supervised learning model is performed iteratively based on a semi-quadratic splitting method.
[0073] In some embodiments, S4 specifically includes the following steps:
[0074] S41. First, the semi-quadratic splitting method is used to convert the final optimized objective function into three sub-problems, namely the x-term sub-problem, the z1-term sub-problem, and the z2-term sub-problem. The solution process of the x-term sub-problem is the outer loop iterative optimization process in the self-supervised learning model, and the solution process of the z1-term sub-problem and the z2-term sub-problem is the training process of the two inner loop networks in the self-supervised learning model.
[0075] Due to the special properties of Φ in the hyperspectral compression system, the x-term subproblem in the outer loop has a closed-form solution. The x-term subproblem is expressed by the following formula:
[0076] Where c represents the defined value; The definition operation is indicated by μ, which represents the penalty parameter for forcing x, z1, and z2 to converge to a common fixed point, and Diag(·) represents extracting the diagonal elements of the subsequent matrix. represents element-wise partitioning; T represents the transpose of the matrix;
[0077] S42, By introducing one of the inner loop networks in the self-supervised learning model Output The first auxiliary variable z1 is replaced and the prior term λD1(z1) is absorbed to obtain the optimized z1 term subproblem; another inner loop network in the self-supervised learning model is introduced. Output The second auxiliary variable z2 is replaced and the prior term λD2(z2) is absorbed to obtain the optimized z2 term subproblem; output and output These are all output feature maps;
[0078] S43. Construct the first loss function used for training the two inner loop networks. Second loss function
[0079] Referring to Figure 2, specifically, the first loss function Second loss function The construction process is similar; the following uses the first loss function. The following explanation uses the construction process as an example:
[0080] Firstly, the output is utilized through an internal loop network. and outer loop output feature map x k+1 Constructing the loss function
[0081] On the other hand, the inner loop network is output through the perception matrix Φ. Make a prediction to obtain the predicted measurement image. Then, the predicted measurement images are used. The loss function is constructed using the normalized compressed measurement image y.
[0082] Finally, the loss function and loss function The sum is the first loss function.
[0083] S44. Enter the outer loop iterative optimization to solve the x-term subproblems, obtain the outer loop output feature map, and input the outer loop output feature map into the training of the two inner loop networks respectively, and calculate the first loss function for each. Second loss function The two loss values are used to optimize the inner loop network. and internal circulation network After the training satisfies the inner loop convergence condition, the inner loop training strategy is exited, and the feature maps generated by the two inner loop networks are used as inputs to the outer loop for iterative optimization. After the outer loop iterative optimization is completed, the first loss function is calculated based on the latest outer loop output feature map. Second loss function The loss values are calculated, and the inner loop network is further optimized using the two newly obtained loss values. and internal circulation network The outputs of the model and its two inner loop networks are used to iterate repeatedly until the self-supervised learning model converges, and finally outputs x as the final reconstructed image.
[0084] In some embodiments, the z1 subproblem in S41 is expressed by a formula, as follows:
[0085] The optimized z1 subproblem in S42 is as follows:
[0086] Where ε1 represents the input to the inner loop network Gaussian noise.
[0087] In some embodiments, the z2 subproblem in S41 is expressed by a formula, as follows:
[0088] The optimized z1 subproblem in S42 is as follows:
[0089] Where ε2 represents the input to the inner loop network G Θ2 (·) Gaussian noise.
[0090] In some embodiments, the first loss function Specifically:
[0091] In some embodiments, the second loss function Specifically:
[0092] In some embodiments, referring to FIG3, step S44 specifically includes the following steps:
[0093] S441. First, input the compressed measurement image y, the sensing matrix Φ (i.e., the sensing matrix in Figure 1), and the penalty parameter μ; and initialize the output x, the first auxiliary variable z1, the second auxiliary variable z2, and the iteration number k.
[0094] S442, Update iteration count k = k + 1;
[0095] S443. Update the outer loop output feature map x by solving the x-term subproblems. k+1 ;
[0096] S444, Output feature map x from the outer loop k+1The inputs are fed into the two inner loop networks for training.
[0097] S445. Calculate the first loss function respectively. Second loss function The loss value; and through the first loss function The loss value and the second loss function The loss values are used to update the inner loop network respectively. and internal circulation network
[0098] S446. Determine whether the training of the two inner loop networks has reached the adaptive convergence criterion. If so, pass the training of the inner loop networks. and internal circulation network Update the final feature maps generated by subproblems z1 and z2 respectively. and Otherwise, repeat steps S445 to S446 until the adaptive convergence criterion is reached; in this embodiment, the adaptive convergence criterion is satisfied when the loss of the current inner loop is less than the loss of the previous entry into the outer loop.
[0099] or
[0100] in, Represents the first loss function in the previous iteration. Second loss function The loss value;
[0101] Once the above conditions are met, immediately exit the inner loop and save the output inner loop network. Output feature map and internal circulation network Output feature map And update the final feature map respectively. Then the outer loop is updated; this design ensures that the depth priors obtained from the training processes of the two inner loop networks are fully integrated during the solution process.
[0102] S447. Determine whether the iteration count k has reached the set iteration count. If yes, output x as the final reconstructed image; otherwise, repeat steps S442 to S447 until the iteration count k reaches the set iteration count. In this embodiment, the value of k is 300.
[0103] This invention provides a hyperspectral compressed sensing reconstruction method based on deep dual prior progressive fusion. Compared with existing self-supervised spectral reconstruction methods, it achieves higher imaging quality and avoids abnormalities such as blurring and artifacts because it fully integrates the deep priors obtained from the training processes of two inner loop networks.
[0104] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
Claims
1. A hyperspectral compressed sensing reconstruction method based on deep dual-prior progressive fusion, characterized in that, Includes the following steps: S1. Capture the original compressed measurement image, and then normalize the original compressed measurement image to obtain the normalized compressed measurement image. S2. Construct an objective function based on the compressed sensing reconstruction problem, and optimize the objective function to obtain the final optimized objective function; S3. Establish a self-supervised learning model for hyperspectral compressed sensing reconstruction based on the normalized compressed measurement images; the self-supervised learning model includes an outer loop iterative optimization and two inner loop network training. S4. Solve the optimized final objective function using a self-supervised learning model, and obtain the final reconstructed image by fusing the deep prior terms obtained from the training processes of the two inner loop networks. S4 specifically includes the following steps: S41. First, the semi-quadratic splitting method is used to convert the final optimized objective function into three sub-problems, namely the x-term sub-problem, the z1-term sub-problem, and the z2-term sub-problem. The solution process of the x-term sub-problem is the outer loop iterative optimization process in the self-supervised learning model, and the solution process of the z1-term sub-problem and the z2-term sub-problem is the training process of the two inner loop networks in the self-supervised learning model. The subproblem x is expressed by a formula, as follows: Where c represents the defined value; The definition operation is indicated by μ, which represents the penalty parameter for forcing x, z1, and z2 to converge to a common fixed point, and Diag(·) represents extracting the diagonal elements of the subsequent matrix. Indicates element-wise partitioning; T represents the transpose of the matrix; x k+1 This represents the feature map output by the outer loop. S42, By introducing one of the inner loop networks in the self-supervised learning model Output The first auxiliary variable z1 is replaced and the prior term λD1(z1) is absorbed to obtain the optimized z1 term subproblem; another inner loop network in the self-supervised learning model is introduced. Output The second auxiliary variable z2 is used to replace the prior term λD2(z2) and absorb the prior term λD2(z2) to obtain the optimized z2 term subproblem; S43. Construct the first loss function used for training the two inner loop networks. Second loss function S44. Enter the outer loop iterative optimization to solve the x-term subproblems, obtain the outer loop output feature map, and input the outer loop output feature map into the training of the two inner loop networks respectively, and calculate the first loss function for each. Second loss function The two loss values are used to optimize the inner loop network. and internal circulation network After the training satisfies the inner loop convergence condition, the inner loop training strategy is exited, and the feature maps generated by the two inner loop networks are used as inputs to the outer loop for iterative optimization. After the outer loop iterative optimization is completed, the first loss function is calculated based on the latest outer loop output feature map. Second loss function The loss values are calculated, and the inner loop network is further optimized using the two newly obtained loss values. and internal circulation network The outputs of the model and its two inner loop networks are used to iterate repeatedly until the self-supervised learning model converges, and finally outputs x as the final reconstructed image.
2. The hyperspectral compressed sensing reconstruction method according to claim 1, characterized in that, S2 specifically includes the following steps: S21. By introducing two deep prior terms, and constructing an objective function based on the compressed sensing reconstruction problem; S22. By introducing the first auxiliary variable z1 and the second auxiliary variable z2, the objective function is optimized to obtain the objective function and constraints after the first optimization. S23. In order to increase the correlation between the first auxiliary variable z1, the second auxiliary variable z2 and the compressed measurement image y, the objective function after the first optimization is optimized again to obtain the final optimized objective function and constraints.
3. The hyperspectral compressed sensing reconstruction method according to claim 2, characterized in that, The objective function in S21 is: Where y represents the normalized compressed measurement image, x represents the hyperspectral image to be determined, and Φ represents the perception matrix; DP1(x) and DP2(x) represent the depth priors introduced from two different inner loop networks; argmin represents the argmin function, which indicates that when... When the value is at its minimum, the value of x is taken as the value of the argmin function, and λ is the weight coefficient of the two depth priors, which are taken to be the same size here; The objective function and constraints after the first optimization in S22 are as follows: Constraints: x = z1, x = z2; The final optimized objective function in S23 is: Constraints: x = z1, x = z2; Where ρ is the penalty parameter.
4. The hyperspectral compressed sensing reconstruction method according to claim 3, characterized in that, The outer loop iterative optimization in the self-supervised learning model is based on the semi-quadratic splitting method.
5. The hyperspectral compressed sensing reconstruction method according to claim 4, characterized in that, The subproblem z1 in S41 is expressed by a formula, as follows: The optimized z1 subproblem in S42 is as follows: Where ε1 represents the input to the inner loop network Gaussian noise.
6. The hyperspectral compressed sensing reconstruction method according to claim 5, characterized in that, The z2 subproblem in S41 is expressed by the following formula: The optimized z1 subproblem in S42 is as follows: Where ε2 represents the input to the inner loop network Gaussian noise.
7. The hyperspectral compressed sensing reconstruction method according to claim 6, characterized in that, The first loss function Specifically:
8. The hyperspectral compressed sensing reconstruction method according to claim 6, characterized in that, The second loss function Specifically:
9. The hyperspectral compressed sensing reconstruction method according to claim 6, characterized in that, S44 specifically includes the following steps: S441. First, input the compressed measurement image y, the perception matrix Φ, and the penalty parameter μ; and initialize the output x, the first auxiliary variable z1, the second auxiliary variable z2, and the iteration number k. S442, Update iteration count k = k + 1; S443. Update the outer loop output feature map x by solving the x-term subproblems. k+1 ; S444, Output feature map x from the outer loop k+1 The inputs are fed into the two inner loop networks for training. S445. Calculate the first loss function respectively. Second loss function The loss value; and through the first loss function The loss value and the second loss function The loss values are used to update the inner loop network respectively. and internal circulation network S446. Determine whether the training of the two inner loop networks has reached the adaptive convergence criterion. If so, pass the training of the inner loop networks. Output and internal circulation network Output Update the final feature maps generated by subproblems z1 and z2 respectively. and Otherwise, repeat steps S445 to S446 until the adaptive convergence criterion is met; S447. Determine if the number of iterations k has reached the set number of iterations. If yes, output x as the final reconstructed image. Otherwise, loop from S442 to S447 until the number of iterations k reaches the set number of iterations.