A convolution sparse coding compressive sensing spectral reconstruction method and system

Abstract

The application belongs to the technical field of spectral image, and discloses a convolution sparse coding compressive sensing spectral reconstruction method and system, which comprises the following steps: step S1, constructing a reconstruction equation based on convolution sparse representation by using an observation image output by a spectral acquisition system and an observation matrix of the spectral acquisition system; step S2, performing low-frequency solving on the image based on the reconstruction equation to obtain preliminary three-dimensional spectral data; step S3, performing constraint solving on a high-frequency part of the image based on the preliminary three-dimensional spectral data to obtain sparse coefficients of a convolution kernel at an Lth waveband; step S4, performing convolution summation on the sparse coefficients and the convolution kernel to obtain three-dimensional spectral data; and step S5, outputting the three-dimensional spectral data meeting a condition to obtain reconstructed spectra. The generalization, accuracy and timeliness of the reconstructed image are all good.

Description

Technical Field

[0001] This invention belongs to the field of spectral image technology, specifically relating to a method and system for spectral reconstruction based on convolutional sparse coding compressed sensing. Background Technology

[0002] Spectral image analysis plays a crucial role in various fields such as environmental monitoring, geological exploration, art conservation, and life science research. Spectral image data is a 3D data cube, comprising 2D spatial information and 1D spectral information, resulting in a massive data volume. Ordinary focal plane detectors are 2D; to obtain a 3D data cube, spectral imaging instruments often need to perform spatial or spectral scanning of the target in the temporal dimension, almost sacrificing temporal resolution. Compressed sensing spectral imaging is a spectral imaging system based on compressed sensing technology. It can sample data at a rate lower than the Nyquist sampling rate and, under certain conditions, reconstruct the original data, significantly reducing the amount of data the detector needs to sample. This makes it possible for the detector to obtain a 3D data cube within a single integration time.

[0003] A spectral imaging system based on compressed sensing technology essentially uses a 2D sensor to sample 3D spectral data, thus requiring compression of the third dimension, the spectral dimension. Dividing the 3D spectral data according to the spectral dimension can be viewed as a combination of multiple 2D images. Referring to the observation method of 2D compressed sensing, i.e., a single-pixel camera, each spectral band of the image is multiplied by a 2D encoding template with a random pattern. Finally, these images are superimposed along the spectral dimension to form a compressed and coded observation image.

[0004] The original 3D spectral data is used to recover high-dimensional spectral data from a small amount of low-dimensional observation data, which is an underdetermined problem. Therefore, additional prior constraints are usually required for the solution. For example, spectral images contain a lot of redundancy and can be sparsely represented in specific transform domains, such as the DCT domain and wavelet domain. Therefore, L1 norm sparsity constraints can be applied to the transformed coefficients. Figueiredo et al. proposed the GPSR method for solving the L1 norm, which has good generalization and is usually less computationally expensive. However, general dictionaries have certain limitations in representing spectral data. Therefore, J. Yang et al. proposed using a Gaussian Mixture Model (GMM) to divide the image into multiple small patches. They assumed that each small patch is composed of a weighted sum of multiple Gaussian components. The specific parameters of the multiple Gaussian components are obtained by pre-training with a small amount of 3D spectral data, and then solved by Bayesian posterior probability estimation. However, dividing the image into multiple small patches for computation ignores the image's structural information. Furthermore, when performing L1 constraint solving, the GPSR method typically converts the image into a one-dimensional vector for computation, which also fails to utilize the global structural information of the image. As a result, the improvement in reconstruction accuracy of such methods is limited.

[0005] As mentioned earlier, spectral image data is a 3D data cube, including 2D spatial information and 1D spectral information. Therefore, spectral images are similar to ordinary 2D images in a specific spectral band, exhibiting smoothness characteristics in local regions. Bioucas et al. proposed the Twist algorithm, which uses total variation as a constraint to smooth the spatial dimension and solve underdetermined equations. Wagadarikar et al. introduced the Twist method into the classic compressed sensing spectral imaging system CASSI for spectral reconstruction. Because it processes the global image, it can better recover the structural information of the scene, and experiments have shown that it can achieve good results. Also based on total variation constraints, X. Yuan et al. proposed the GAP-TV method, which transforms the general ADMM solution framework into one requiring only a few iterative equations, thus shortening the reconstruction time while maintaining reconstruction quality. However, total variation constraints apply smoothness constraints to the image, leading to the loss of details in the reconstructed image. Furthermore, it only constrains the spatial dimension and ignores the inherent correlations between data in the spectral dimension, thus significantly limiting its ability to improve spectral reconstruction accuracy. In recent years, some scholars have introduced the nonlocal self-similarity of images as a constraint term into the image denoising problem and demonstrated excellent results. For example, the WNNM method proposed by Gu et al. divides the image into multiple small patches and performs weighted nuclear norm constraint solving between patches. Based on WNNM denoising of images, Liu et al. proposed the DeSCI algorithm, which fully utilizes the low-rank characteristics of the spatial and spectral dimensions for constraint reconstruction by dividing the 3D spectral image into multiple 3D small patches, and experimental results show excellent reconstruction effects. However, the DeSCI method requires a large amount of time-consuming 3D block matching calculations, and block noise in the reconstructed image affects the accuracy of image details. Therefore, for compressed sensing spectral imaging systems, the DeSCI method still fails to simultaneously improve the accuracy of reconstructed spectral consistency and image quality.

[0006] With the widespread application of deep learning, some scholars have transformed the compressed sensing constrained reconstruction problem into an end-to-end mapping problem by training deep CNN networks. This involves inputting a 2D observation image and an encoding template, and outputting a 3D spectral data cube, thus reducing reconstruction time while improving reconstruction accuracy. However, the end-to-end CNN network structure is highly dependent on the structure of the observation system. When the system structure changes, such as by changing the encoding template or sampling rate, the network needs to be retrained, resulting in poor generalization of this type of method. Summary of the Invention

[0007] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method and system for spectral reconstruction based on convolutional sparse coding compressed sensing, which solves the problems of poor generalization of spectral reconstruction and poor accuracy of spectral consistency and image quality in the prior art.

[0008] To achieve the above objectives, the present invention employs the following technical solution: a method for compressed sensing spectral reconstruction based on convolutional sparse coding, comprising the following steps:

[0009] Step S1: Construct a reconstruction equation based on convolutional sparse representation using the observation image output by the spectral acquisition system and the observation matrix of the spectral acquisition system;

[0010] Step S2: Solve the low-frequency image equation based on convolutional sparse representation in step S1 to obtain preliminary three-dimensional spectral data;

[0011] Step S3: Constrain the preliminary three-dimensional spectral data from step S2 to solve the high-frequency part of the image and obtain the sparse coefficients of the convolution kernel at the Lth band.

[0012] Step S4: Convolve the sparse coefficients and the convolution kernel to obtain the three-dimensional spectral data;

[0013] After the three-dimensional spectral data in steps S5 and S4 are projected through the system observation matrix, a reconstructed observation image is obtained. If the error between the reconstructed observation image and the observation image in step S1 is less than a threshold, the three-dimensional spectral data is output to obtain the reconstructed spectrum. If the error is greater than or equal to the threshold, the process returns to step S2 until the error between the reconstructed observation image and the observation image in step S1 is less than the threshold, at which point the loop ends.

[0014] Preferably, in step S1, the observed image y output by the spectral acquisition system and the observation matrix H of the spectral acquisition system are combined with the TV operator and convolutional sparseness to construct a reconstruction equation based on convolutional sparse representation.

[0015] Preferably, the reconstruction equation based on the convolutional sparse representation in step S1 is as shown in equation (1):

[0016]

[0017] Where I represents the initial three-dimensional spectral data, TV represents the TV operator, β represents the weights of the TV operator constraints, and L represents the Lth band in the spectral data; L This is a two-dimensional image of the spectral data.

[0018] Preferably, step S2 uses GAP and introduces an auxiliary variable v to solve the low-frequency image problem of the reconstruction equation based on convolutional sparse representation in step S1, as shown in equation (2):

[0019]

[0020] Among them, I t Let I be the three-dimensional spectral data obtained at time t. t-1 The three-dimensional spectral data at time t-1 is given by v. t v is an auxiliary variable used to solve the problem at time t. t-1 This is an auxiliary variable used when solving the problem at time t-1.

[0021] Preferably, the v t The solution is shown in equation (3).

[0022] v t =v t-1 +H T (HH T ) -1 (y-HI t-1 (3)

[0023] Among them, H T This is the conjugate transpose of the observation matrix H.

[0024] Preferably, the preliminary three-dimensional spectral data I t-1 According to I in equation (2) t I received t The solution is shown in equation (4).

[0025] I t =TV(v t (4)

[0026] Preferably, step S3 includes the following steps:

[0027] Step S31, Preliminary three-dimensional spectral data I t-1 By combining convolutional sparseness with L 2,1 The norm is obtained from equation (5). Equation (3) is used to solve the high-frequency part of the image and iteratively obtains the sparse coefficients at L bands after j+1 iterations.

[0028]

[0029] Where, x L,m For convolution kernel d m The sparsity coefficient at the Lth band, I L t-1 Preliminary three-dimensional spectral data I t-1 Two-dimensional image, d represents the sparse coefficients at the Lth band after j+1 cycles. m The kernel is ρ, and the weight coefficients are ρ. These are all auxiliary variables in the jth iteration of the loop;

[0030] Step S32, for the auxiliary variable of the (j+1)th iteration of the loop The solution is shown in equation (6), for the auxiliary variable of the (j+1)th iteration. The solution is shown in equation (7);

[0031]

[0032]

[0033] Where, δ L,m It is an auxiliary variable.

[0034] Preferably, step S4 obtains the three-dimensional spectral data I using equation (8). t ;

[0035]

[0036] The present invention also discloses a convolutional sparse coding-based compressed sensing spectral reconstruction system, including an equation construction module and an output module;

[0037] The equation construction module is used to construct reconstruction equations based on convolutional sparse coding representation using the observation images output by the spectral acquisition system and the observation matrix of the spectral acquisition system;

[0038] The output module is used to solve the reconstruction equation based on convolutional sparse coding representation and output three-dimensional spectral data to obtain the reconstructed spectrum.

[0039] Compared with existing technologies, this invention has the following advantages: This invention discloses a spectral reconstruction method based on convolutional sparse coding. Utilizing the characteristic that convolutional kernels operate on global image information, the reconstruction result can better preserve the structural information of the image. A reconstruction equation based on convolutional sparse representation is constructed using the observed image y output by the spectral acquisition system and the observation matrix H of the spectral acquisition system. Since convolutional sparse coding is generally insensitive to low-frequency information in images, the reconstruction process requires first solving the low-frequency part of the image to obtain preliminary three-dimensional spectral data. Then, constraints are applied to the preliminary three-dimensional spectral data to solve the high-frequency part of the image, resulting in the three-dimensional spectral data I. t After projection onto the system observation matrix H, if the error between the projection and the observed image y is less than a preset threshold, the resulting three-dimensional spectral data I is output. t It exhibits good generalization ability and demonstrates good accuracy in reconstructed spectral consistency and image quality.

[0040] Furthermore, since the observed image y is encoded and aliased, it is difficult to effectively extract low-frequency information from it. Therefore, a global total variation (TV) operator is added to constrain the low-frequency components, ensuring that the low-frequency parts of the image can be solved. Simultaneously, the convolutional sparse coding is transformed into a denoising process, reducing computational complexity and simplifying the solution process. Simulation experiments show that the reconstruction time of this invention is significantly reduced, the reconstruction quality is high, and the reconstruction details are greatly improved.

[0041] Furthermore, L-type spectral data were analyzed from the preliminary three-dimensional spectral data. 2,1 Norm constraints utilize the correlation between spectra to improve reconstruction accuracy.

[0042] Furthermore, by treating spectral data as a weighted result of convolution of multiple convolution kernels and corresponding sparse coefficients, and by using convolution kernels to operate on global image information, the structural information of the image can be well preserved in the reconstruction result.

[0043] The present invention also discloses a spectral reconstruction system based on convolutional sparse coding compressed sensing, which not only ensures the accuracy and timeliness of reconstruction, but also has strong adaptability, with the reconstruction accuracy not changing with the structure of the observation system. Attached Figure Description

[0044] Figure 1 This is a flowchart of a spectral reconstruction method based on convolutional sparse coding according to the present invention;

[0045] Figure 2 A computational spectral acquisition system adapted to the reconstruction method of the present invention;

[0046] Figure 3 This serves as proof that the same convolution kernel proposed in this invention exhibits stationary characteristics in its response to two-dimensional spectral images of different spectral bands;

[0047] Figure 4 This is a comparison chart of the spatial dimension accuracy of the present invention and mainstream reconstruction algorithms;

[0048] Figure 5 This is a comparison diagram between the reconstruction results of this invention and the actual spectral distribution curve. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0051] The present invention will now be described in further detail with reference to the accompanying drawings:

[0052] A method for compressed sensing spectral reconstruction based on convolutional sparse coding, see [link to relevant documentation]. Figure 1 This includes the following steps:

[0053] Step S1: Construct a reconstruction equation based on convolutional sparse representation using the observed image y output by the spectral acquisition system and the observation matrix H of the spectral acquisition system. The spectral acquisition system is as follows: Figure 2 As shown;

[0054] Step S2: Solve the low-frequency image equation based on convolutional sparse representation in step S1 to obtain preliminary three-dimensional spectral data I. t-1 ;

[0055] Step S3: Process the preliminary three-dimensional spectral data I from step S2. t-1 Constraints are applied to solve for the high-frequency components of the image to obtain the convolution kernel d. m The sparsity coefficient x at the Lth band L,m ;

[0056] Step S4: Adjust the sparsity coefficient x L,m With convolution kernel d m Convolution and summation are performed to obtain the three-dimensional spectral data I;

[0057] After the three-dimensional spectral data in steps S5 and S4 are projected through the system observation matrix, a reconstructed observation image is obtained. If the error between the reconstructed observation image and the observation image in step S1 is less than a threshold, the three-dimensional spectral data is output to obtain the reconstructed spectrum. If the error is greater than or equal to the threshold, the process returns to step S2 until the error between the reconstructed observation image and the observation image in step S1 is less than the threshold, at which point the loop ends.

[0058] The size of the convolution kernel in this invention is fixed, and multiple convolution kernels only need to be obtained in advance. In practice, a general convolution kernel can be used, or a convolution kernel can be relearned for specific content.

[0059] To fully utilize the spectral characteristics of spectral images, experimental analysis revealed that after convolution with the same kernel, spectral images of different spectral bands exhibit spatially identical sparse coefficient maps, differing only in numerical value. Figure 3 As shown, therefore, this invention performs L-type convolution on the preliminary three-dimensional spectral data after convolution and summation of the sparse coefficient maps corresponding to the convolution kernel. 2,1 Norm constraints utilize the correlation between spectra to improve reconstruction accuracy.

[0060] This invention verifies a compressed sensing spectral reconstruction method based on convolutional sparse coding through simulation experiments. The simulated spectral system is the CASSI system, and the experimental data uses the CAVE spectral dataset, which is 512×512×31 in size and covers a spectral range of 400-700 nm. The convolutional kernel size used in the experiment is 12×12×144. The experiment compares the results with currently recognized advanced compressed sensing spectral reconstruction methods such as GAP-TV and DeSCI. In the spatial dimension, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are compared. Regarding spectral accuracy, specific image patches are selected, and the consistency between the reconstruction results and the true spectral distribution curves for each band is plotted. The results are as follows: Figure 5 As shown in the figure. Experiments compared the reconstruction quality of the spectral reconstruction method based on convolutional sparse coding of this invention with that of Gap-TV and DeSCI methods in 10 scenarios. The spatial dimension metrics were peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and the corresponding results are shown in Tables 1 and 2:

[0061] Table 1 Comparison of Reconstructed Data Quality

[0062]

[0063] Table 2 Comparison of Reconstructed Data Quality

[0064]

[0065]

[0066] As can be seen from the results in Tables 1 and 2, the reconstruction method proposed in this invention exhibits good accuracy in spectral reconstruction results across multiple scenarios. It can improve reconstruction quality PSNR by up to 7 dB and SSIM by up to 10%, and also significantly enhances the detail of the reconstructed image. These results demonstrate that… Figure 4The proposed method in this paper demonstrates significant improvements in reconstruction detail compared to other mainstream methods. By comparing the current mainstream GAP-TV and DeSCI algorithms, the proposed method significantly reduces reconstruction time compared to DeSCI, achieves reconstruction quality (PSNR, SSIM, and spectral accuracy) comparable to the best DeSCI methods, and exhibits substantial improvements in image detail compared to both GAP-TV and DeSCI.

[0067] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for compressed sensing spectral reconstruction based on convolutional sparse coding, characterized in that, Includes the following steps: Step S1: Construct a reconstruction equation based on convolutional sparse representation using the observation image output by the spectral acquisition system and the observation matrix of the spectral acquisition system. y and the observation matrix of the spectral acquisition system H We construct a reconstruction equation based on convolutional sparse representation by combining the TV operator and convolutional sparsity. Step S2: Solve the low-frequency image data of the reconstruction equation based on convolutional sparse representation in step S1 to obtain preliminary three-dimensional spectral data, and use GAP and introduce auxiliary variables. The low-frequency solution of the image is performed on the reconstruction equation based on convolutional sparse representation in step S1, as shown below: （2） in, The three-dimensional spectral data at time t are obtained. The three-dimensional spectral data are obtained at time t-1. Let be the auxiliary variable for solving at time t. These are auxiliary variables used when solving the problem at time t-1; Step S3: Constrain the preliminary three-dimensional spectral data from step S2 and solve for the high-frequency part of the image to obtain the convolution kernel at the 1st... L The sparsity coefficients at each band are determined using the following steps: Step S31: Preliminary three-dimensional spectral data I t-1 By combining convolution and sparsity The norm is obtained from equation (5). Equation (3) is used to solve the high-frequency part of the image and iterate to obtain the sparse coefficients at L bands after j+1 iterations. ; （5） in, convolution kernel In the L The sparsity coefficients at each band For preliminary three-dimensional spectral data I t-1 Two-dimensional image, Let be the sparse coefficients at the Lth band after j+1 cycles. For convolution kernel, These are the weighting coefficients. , These are all auxiliary variables in the jth iteration of the loop; Step S32, for the auxiliary variable of the (j+1)th iteration of the loop The solution is shown in equation (6), for the auxiliary variable of the (j+1)th iteration. The solution is shown in equation (7); （6） （7） in, As an auxiliary variable; Step S4: Convolve the sparse coefficients and the convolution kernel to obtain the three-dimensional spectral data; After the three-dimensional spectral data in steps S5 and S4 are projected through the system observation matrix, a reconstructed observation image is obtained. If the error between the reconstructed observation image and the observation image in step S1 is less than a threshold, the three-dimensional spectral data is output to obtain the reconstructed spectrum. If the error is greater than or equal to the threshold, the process returns to step S2 until the error between the reconstructed observation image and the observation image in step S1 is less than the threshold, at which point the loop ends.

2. The method for compressed sensing spectral reconstruction based on convolutional sparse coding according to claim 1, characterized in that, The reconstruction equation based on the convolutional sparse representation in step S1 is shown in equation (1): （1）； in, For the initial three-dimensional spectral data, TV for TV Operator, for TV Weights of operator constraints L Indicating the first spectral data L One band; This is a two-dimensional image of the spectral data.

3. The method for compressed sensing spectral reconstruction based on convolutional sparse coding according to claim 1, characterized in that, The The solution is shown in equation (3). （3） in, Observation matrix H The conjugate transpose of .

4. The method for compressed sensing spectral reconstruction based on convolutional sparse coding according to claim 3, characterized in that, The preliminary three-dimensional spectral data I t-1 According to formula (2) get, The solution is shown in equation (4). （4）。 5. The method for compressed sensing spectral reconstruction based on convolutional sparse coding according to claim 1, characterized in that, Step S4 obtains the three-dimensional spectral data through equation (8). ; （8）。 6. A spectral reconstruction system based on convolutional sparse coding compressed sensing, characterized in that, Includes a module for constructing equations and an output module; The equation construction module is used to construct a reconstruction equation based on convolutional sparse coding representation using the observation images output by the spectral acquisition system and the observation matrix of the spectral acquisition system. y and the observation matrix of the spectral acquisition system H We construct a reconstruction equation based on convolutional sparse representation by combining the TV operator and convolutional sparsity. The output module is used to solve the reconstruction equation based on convolutional sparse coding representation and output three-dimensional spectral data to obtain the reconstructed spectrum; Specifically, the image low-frequency solution of the reconstruction equation based on convolutional sparse representation in the equation construction module is used to obtain preliminary three-dimensional spectral data. GAP is adopted and auxiliary variables are introduced. The low-frequency solution of the image is performed on the reconstruction equation based on convolutional sparse representation of the equation construction module, as shown below: （2） in, The three-dimensional spectral data at time t are obtained. The three-dimensional spectral data are obtained at time t-1. Let be the auxiliary variable for solving at time t. These are auxiliary variables used when solving the problem at time t-1; Constraining the preliminary three-dimensional spectral data and solving for the high-frequency components of the image yields the convolution kernel at the 1st... L The sparsity coefficients at each band are determined using the following steps: Preliminary three-dimensional spectral data I t-1 By combining convolution and sparseness The norm is obtained from equation (5). Equation (3) is used to solve the high-frequency part of the image and iterate to obtain the sparse coefficients at L bands after j+1 iterations. ; （5） in, convolution kernel In the L The sparsity coefficients at each band For preliminary three-dimensional spectral data I t-1 Two-dimensional image, Let be the sparse coefficients at the Lth band after j+1 cycles. For convolution kernel, These are the weighting coefficients. , These are all auxiliary variables in the jth iteration of the loop; Auxiliary variable for loop (j+1) The solution is shown in equation (6), for the auxiliary variable of the (j+1)th iteration. The solution is shown in equation (7); （6） （7） in, As an auxiliary variable; The sparse coefficients are convolved with the convolution kernel to obtain the three-dimensional spectral data. The three-dimensional spectral data is then projected onto the system observation matrix to obtain the reconstructed observation image. If the error between the reconstructed observation image and the observation image in the equation construction module is less than a threshold, the three-dimensional spectral data is output to obtain the reconstructed spectrum. If the error is greater than or equal to the threshold, the loop returns until the error between the reconstructed observation image and the observation image in the equation construction module is less than the threshold, at which point the loop ends.

Images