Wavelet guided deep unfolding network design method for compressive sensing reconstruction

Abstract

The application faces wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction and belongs to the field of image quality enhancement, and solves the compressed sensing image reconstruction problem. First, a compressed sensing reconstruction model based on discrete wavelet transform is designed, and the model is decomposed into two sub-problems: a wavelet domain reconstruction problem and an image domain reconstruction problem, and the solutions of the two domains are mutually guided and iteratively updated; second, the iterative optimization process is unfolded into a wavelet-guided deep learning unfolding network for compressed sensing reconstruction; finally, in order to effectively remove feature noise in the image reconstruction process, two continuous denoising modules, namely a dual-domain guided filtering module and a self-guided filtering enhancement module, are designed, the two modules realize image feature denoising by introducing guided filtering, and reduce information loss between stages through an improved long short-term memory mechanism, and the reconstruction performance of the application is better than that of existing multiple image compressed sensing reconstruction methods.

Description

Technical Field

[0001] This invention belongs to the field of image quality enhancement, specifically involving a wavelet-guided deep learning network design method for compressed sensing reconstruction. Background Technology

[0002] Compressed sensing (CS) is a technique for finding sparse solutions to underdetermined linear systems, which can compress data during the sampling process. CS is a hot research topic in image processing. In the CS framework, the image is first sampled and compressed simultaneously, then the original image is reconstructed using measurements at a rate far lower than the Nyquist sampling rate. Because the CS method can recover a near-perfect original image from very few measurements, it significantly reduces compression, storage, and transmission costs. Currently, CS has been successfully applied in various fields, such as image compression, single-pixel cameras, medical imaging, snapshot compressed imaging, remote sensing, radar image acquisition, and wireless remote monitoring.

[0003] By acquiring and reconstructing signals using extremely low sampling rates, signal scalar computation (CS) has sparked a research boom in academia in recent years. According to mathematical theory, given the original signal x∈R... N and sampling matrix Φ∈R M×N The CS measurement value can be solved by y = Φx, where the measurement value y ∈ R. M The goal of CS is to infer the original signal x from the obtained random measurement y. Since M << N in most cases, the above problem is an ill-posed inverse problem, often solved by... We use this to solve the CS reconstruction problem. Here, F(x) is the regularization function used for sparse representation of natural images, and λ is a hyperparameter used to balance the data fidelity term and the regularization term. This model is usually solved using the proximal gradient descent algorithm.

[0004] Traditional CS image reconstruction methods are primarily model-based. These methods iteratively reconstruct the original signal by solving an optimization problem. Since most traditional CS image reconstruction methods rely on sparse prior knowledge, and the most common prior knowledge assumes that the original signal is sparse in a certain transform domain, methods for solving such sparse models mainly include convex relaxation methods, greedy matching pursuit methods, and Bayesian methods. These methods are based on interpretable prior knowledge, thus possessing advantages such as strong convergence and theoretical guarantees. However, traditional CS image reconstruction methods typically have high computational complexity, poor adaptability, and limited reconstruction accuracy.

[0005] In recent years, deep learning (DL) technology has experienced rapid development in both industry and academia. It has achieved superior performance in many traditional signal processing and recognition tasks, demonstrating its ability to handle various complex tasks. Therefore, many researchers have begun to explore the use of DL to address problems in related fields. Problems with traditional signal processing (CS) image reconstruction methods can also be solved using DL. Since models in practical applications do not satisfy the sparsity assumptions of traditional algorithms, DL can address this issue using a data-driven approach. DL technology relaxes the conditions for signal sparsity, allowing the network to adaptively adjust weights. For example, convolutional neural networks, residual networks, and denoising autoencoders can learn the structural features of real signals by training on a large number of samples, thereby improving signal reconstruction accuracy. Furthermore, DL technology can significantly reduce network computation time and model complexity. However, the architectures of these networks are black-box designs, lacking theoretical analysis and interpretability. Considering the interpretability of traditional algorithms and the excellent reconstruction performance of deep learning technology, researchers have begun to combine traditional algorithms with DL technology, proposing a series of deep unfolded networks. These networks are deconstructed from traditional CS algorithms and optimized iteratively, with convolutional neural networks introduced at each iteration step. This allows these methods to gradually improve the reconstruction performance of the model while gaining interpretability. Although these deep unfolded networks have significantly improved reconstruction performance, most of them are limited to transmitting information in the image domain, which greatly restricts further improvements in network performance.

[0006] Recent studies have found that frequency information can further improve network performance. For example, Zhuang et al. enhanced low-brightness images by optimizing the Fourier coefficients of the image. Zhou et al. proposed adaptively learning low-frequency information integration in both spatial and frequency domains simultaneously. Zhang et al. and Zou et al. used discrete wavelet transform to transform the network from the image domain to the frequency domain to complete image super-resolution tasks. Since frequency information can reflect the semantic information of images in different bands, it plays a crucial role in reconstructing high-frequency details of images. Inspired by this observation, this invention introduces discrete wavelet transform into the image CS optimization model, using frequency information to guide compressed sensing reconstruction of images. Summary of the Invention

[0007] This invention proposes a wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction. By introducing discrete wavelet transform into the compressed sensing reconstruction model and unfolding its optimization algorithm into a wavelet-guided deep learning unfolding network for compressed sensing reconstruction, this network consists of three parts: a sampling network, a preliminary reconstruction network, and a deep dual-domain hybrid reconstruction network. The deep dual-domain hybrid reconstruction network includes two basic units: a Frequency Information Enhancement Unit (FIEU) and a Wavelet-Guided Image Denoising Unit (WIDU). Within the WIDU, two denoising modules are defined: a Dual-Domain Guided Filtering Module (DGFM) and a Self-Guided Filter Enhancement Module (SFEM). These modules utilize guided filtering to remove noise from features. Furthermore, this network effectively addresses the problems of uninterpretability in existing compressed sensing reconstruction models and the presence of noise interference in the reconstructed image.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions.

[0009] A wavelet-guided deep learning network design method for compressed sensing reconstruction includes the following steps:

[0010] Step 1) Construct training data for a wavelet-guided deep learning unfolding network for compressed sensing reconstruction.

[0011] The T91 dataset was used as the training dataset for this network.

[0012] Step 2) Construct and optimize a compressed sensing reconstruction model based on discrete wavelet transform.

[0013] First, the Discrete Wavelet Transform (DWT) is incorporated into the existing compressed sensing reconstruction model, along with the consistency constraint of the wavelet coefficients, to construct a compressed sensing reconstruction model based on DWT. Then, this model is decomposed into two sub-problems, which are iteratively solved using the proximal gradient descent algorithm. Finally, the iterative solution algorithm is expanded into a wavelet-guided deep learning unfolding network for compressed sensing reconstruction.

[0014] Step 3) Constructing the sampling network and preliminary reconstruction network

[0015] First, input the image x, then decompose it into non-overlapping image patches. use Obtain the observation values ​​for each image patch. Here, a large stride convolution operation is used to implement compressed sensing observations. This convolution operation can be simply referred to as a sampling network, which is then... Obtain the initial image block Here, image initialization is achieved through a sampling convolution operation, and then a pixel-shuffle operation is used to reassemble the initialized image blocks to obtain the preliminary reconstructed image x. (0) The convolution and pixel-shuffle operations here can be simply referred to as the initial reconstruction of the network.

[0016] Step 4) Construct a deep dual-domain hybrid reconstruction network

[0017] The iterative solution algorithm obtained in step 2) is expanded into a deep dual-domain hybrid reconstruction network, which contains two basic units: a frequency information enhancement unit and a wavelet-guided image denoising unit. The frequency information enhancement unit is used to optimize and enhance the frequency information obtained through discrete wavelet transform, and the wavelet-guided image denoising unit is used to remove reconstructed feature noise.

[0018] Step 5) Construct a dual-domain guided filtering module and a self-guided filtering enhancement module.

[0019] Two modules are constructed using guided filtering theory: In the wavelet-guided image denoising unit in step 4), the dual-domain guided filtering module DGFM and the self-guided filtering enhancement module SFEM are both used to remove feature noise.

[0020] Step 6) Loss Function

[0021] The overall loss function consists of three parts: image reconstruction loss, wavelet coefficient consistency loss, and sampling matrix orthogonality loss. The L1 norm is used to constrain the error loss term.

[0022] Further, in step 1), the T91 dataset is first used as the training dataset for the network, and five networks are trained on a set of CS ratios {10%, 25%, 30%, 40%, 50%}. During training, image patches of size 32×32 with a batch size of 64 are used for training, for a total of 100 epochs, and the Adam optimizer is used to optimize the network parameters. The initial learning rate is set to 5e-4, and a cosine annealing strategy is used to update the learning rate and reduce it to 5e-5. The network is trained for a total of 9 stages. Except for the input and output layers, the number of output channels of all other convolutional layers is 32 by default. Three publicly available datasets are used as test datasets, including Set11, CBSD68, and Urban100. RGB color images are transformed into YCbCr color images, and compressed sensing observation and reconstruction are performed only on the Y channel. Finally, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) on the Y channel are used to evaluate the CS reconstruction results.

[0023] Furthermore, in step 2), x is first used to... w=DWT(x) yields the wavelet coefficients, where x is the input image and DWT(·) represents the discrete wavelet transform. Since this process is reversible, it can be obtained by x = IDWT(x w The original signal is reconstructed, where IDWT(·) represents the inverse discrete wavelet transform. Introducing DWT into the optimization model of compressed sensing, the proposed optimization model for compressed sensing reconstruction can be expressed as:

[0024]

[0025] In formula (1), the first three terms are data fidelity terms, and the last two terms are regularization terms. F(·) and Ω(·) are regularization functions. α, β, λ, and μ are four learnable step size parameters, and x... w The wavelet coefficients, x′, are obtained by processing the input image x using DWT. w These are the reconstructed wavelet coefficients, and IDWT(x′) w ) is at x′ w This was obtained under known conditions. Clearly, since it's impossible to solve for both variables simultaneously, formula (1) is decomposed into the following two optimization problems:

[0026]

[0027]

[0028] Next, the proximal gradient descent (PGD) algorithm will be used to solve the above optimization problem.

[0029] Update x′ w Applying the PGD algorithm to formula (2), the iterative optimization process can be described as the following two update steps:

[0030]

[0031] in, Let represent the gradient operator, δ1 be the learnable step size parameter, k be the iteration index, and prox be the step size parameter. μ,Ω (·) is about the implicit prior μ||Ω(x′) w The proximal operator of )||1.

[0032] Update x: Applying the PGD algorithm to formula (3), the iterative optimization process can be described as the following two update steps:

[0033]

[0034] in, Let denote the gradient operator, δ² be the learnable step size parameter, and prox λ,F(·) is the proximal operator corresponding to the implicit prior λ||F(x)||1. Equations (4) and (5) are the alternating iterative solution expressions of the proposed optimization model. Based on these two equations, the iterative solution process is expanded into a wavelet-guided deep learning unfolding network for compressed sensing reconstruction.

[0035] Furthermore, in step 3), the sampling network first learns the sampling matrix Φ∈R from the training network. M×N Next, all elements of the sampling matrix Φ are constrained to a binary pattern of {0, 1}, thus obtaining a data-driven binary sampling matrix. Then, the input image x is divided into... Non-overlapping image blocks Finally by Obtain the linear measurement vector of the image patch. The initial network reconstruction involves initializing the obtained measurement vectors using a convolution operation and a pixel-shuffle operation. Specifically, the sampling matrix is ​​first transposed to obtain Φ. T Then, based on the inverse problem An initial image patch is obtained through a convolutional layer, and finally, a pixel-shuffle operation is used to reassemble the image patch into a preliminary reconstructed image x. (0) .

[0036] Furthermore, the deep dual-domain hybrid reconstruction network in step 4) consists of K stages, each stage corresponding to one iteration in formulas (4) and (5), and each stage consists of two units, namely the frequency information enhancement unit FIEU and the wavelet-guided image denoising unit WIDU.

[0037] Frequency Information Enhancement Unit (FIEU): In the frequency information enhancement unit, the frequency features of the image are first obtained through the gradient descent algorithm shown in formula (4). Due to the prox operator μ,Ω The presence of a regularization term Ω(·) in the equation makes explicit solving impossible; therefore, a convolutional neural network (CNN) is used to approximate the proximal operator operation. Next, the obtained frequency information is processed through residual blocks to extract shallow features and enhance the frequency information representation. Each residual block consists of three sets of 3×3 convolutions and ReLU activation functions. Finally, the updated wavelet coefficients are... The reconstructed image of the DWT domain is recovered using IDWT.

[0038] Wavelet-guided image denoising unit (WIDU): In the wavelet-guided image denoising unit, image features are first obtained through the gradient descent algorithm shown in formula (5). Due to the prox operator λ,FThe presence of a regularization function F(·) in (·) makes it impossible to solve explicitly; therefore, a CNN is used to replace the proximal operator operation. Then, a 3×3 convolution is used to extract r. (k) The shallow features are extracted and their channel count is increased to 32. Then, a dual-domain guided filtering module (DGFM) and a self-guided filtering enhancement module (SFEM) are used to remove noise from the features. Finally, the filtered features are converted into a 1-channel image through a 3×3 convolution and r is added. (k) The final residual output x is obtained. (k) .

[0039] Furthermore, the design of the dual-domain guided filtering module DGFM and the self-guided filtering enhancement module SFEM in step 5) is as follows:

[0040] In the Dual-Domain Guided Filtering (DGFM) module, the input features are first subjected to 3×3 convolution and ReLU activation to extract shallow features. Then, the wavelet map and reconstructed map output from the previous stage are used as guide maps to perform feature-guided filtering on the extracted features. Next, the filtered features are enhanced by 3×3 convolution and ReLU activation, and the two enhanced feature sets are concatenated along the channel dimension. Residual connections are used to supplement the enhanced features. Subsequently, 3×3 convolution and ReLU activation are used again to extract deep features. Finally, considering the information loss problem during feature transmission, an improved Long Short-Term Memory (I-LSTM) mechanism is used to effectively control the information flow and capture long-term dependencies during the iteration process. In addition, the cell state of I-LSTM can retain important information and can be modified and updated through gating mechanisms, effectively reducing information loss during transmission. In the Self-Guided Filtering Enhancement Module (SFEM), the features output by DGFM are first processed by a 1×1 convolution to obtain a feature map, which is then used as the guide map. Next, the features processed by 3×3 convolution and ReLU activation are subjected to guided filtering. The filtered features are then processed by another 3×3 convolution and ReLU activation to extract depth information. Residual connections are then used to supplement the information. The supplemented information is then processed by 3×3 convolution and ReLU activation to extract features. Finally, the I-LSTM mechanism is used to select the extracted features and retain the effective information.

[0041] Furthermore, the loss function in step 6) consists of three components: the first component is the data fidelity loss L. data It can guarantee the reconstructed image Approximately equal to the true Ground-Truth image x q The second component is the consistency constraint loss L of the wavelet coefficients. wave It can guarantee the wavelet coefficients of the reconstructed image Wavelet coefficients x of the real Ground-Truth image wq Consistency; the last component is the orthogonal loss term L applied to the sampling matrix. orth The loss functions for the three components are shown below:

[0042]

[0043]

[0044]

[0045] Where, N a N is the number of pixels in each image. b Where I is the batch size of the training images, I represents the identity matrix, K is the total number of stages in the network, and the total loss function can be expressed as:

[0046] L(Θ)=L data +γL wave +ηL orth (9)

[0047] in, γ is a learnable parameter in the network, and η is two weight parameters.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] 1. This invention first proposes to incorporate discrete wavelet transform into the compressed sensing reconstruction model, and simultaneously incorporates the consistency constraint of wavelet coefficients into the model, thus constructing a new compressed sensing reconstruction model based on discrete wavelet transform. Then, the proximal gradient descent algorithm is used to solve the model. Finally, the optimization solution algorithm is expanded into a wavelet-guided deep learning unfolding network for compressed sensing reconstruction. Compared with existing compressed sensing reconstruction methods, this invention has better interpretability and higher reconstruction performance.

[0050] 2. The deep dual-domain hybrid reconstruction network of the present invention has two basic units: a frequency information enhancement unit (FIEU) and a wavelet-guided image denoising unit (WIDU). These two units guide each other and update iteratively, aiming to explore the nonlinear mapping potential of different domains. FIEU can effectively utilize the frequency information of different bands to reconstruct high-frequency texture details to supplement the image reconstruction of WIDU. Since the present invention introduces the frequency information of the image and uses the frequency information to guide the image reconstruction, the reconstructed image has clearer texture and details compared with other compressed sensing methods.

[0051] 3. Considering the profound impact of varying degrees of noise on the CS reconstruction process, this invention designs two consecutive denoising modules to eliminate feature noise: a Dual-Domain Guided Filtering (DGFM) module and a Self-Guided Filter Enhancement (SFEM) module. These two modules achieve feature denoising through guided filtering. Furthermore, to address the information loss problem between modules and stages, an improved Long Short-Term Memory (I-LSTM) mechanism is introduced into these two modules. This mechanism can effectively filter the filtered features and fully exploit long-term dependencies. Therefore, compared with other modules, the DGFM and SFEM designed in this invention have better denoising capabilities. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall network architecture of the present invention.

[0053] Figure 2 This is a schematic diagram of the k-th stage of the deep dual-domain hybrid reconstruction network of the present invention.

[0054] Figure 3 This is a schematic diagram of the DGFM and SFEM of the present invention.

[0055] Figure 4 This is a comparison of the subjective quality of the method of this invention and six recent interpretable compressed sensing reconstruction methods in recovering the "Barbara" image on the Set11 dataset.

[0056] Figure 5 This is a comparison of the subjective quality of the method of this invention and six recent interpretable compressed sensing reconstruction methods in recovering the image "24077" on the CBSD68 dataset.

[0057] Figure 6 This is a comparison of the subjective quality of the method of this invention and six recent interpretable compressed sensing reconstruction methods in recovering the "img_060" image on the Urban100 dataset. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. To better understand the image enhancement method of the present invention, the network structure of the present invention will be described in detail first.

[0059] I. Specific Implementation of Wavelet-Guided Deep Learning Deployment Network for Compressed Sensing Reconstruction

[0060] like Figure 1 As shown, the proposed wavelet-guided deep learning unfolding network for compressed sensing reconstruction consists of three basic networks: a sampling network, a preliminary reconstruction network, and a deep dual-domain hybrid reconstruction network. The implementation steps of the proposed method are as follows:

[0061] Step 1) Construction of training and test datasets

[0062] Before training the network, the T91 dataset was used as the training dataset, and the Set11, CBSD68 and Urban100 datasets were used as the test datasets.

[0063] Step 2) Sampling Network

[0064] The sampling network first learns the sampling matrix Φ∈R from the training network. M×N Next, all elements of the sampling matrix Φ are constrained to a binary pattern of {0, 1}, thus obtaining a data-driven binary sampling matrix. Then, the input image x is divided into... Non-overlapping image blocks Finally by Obtain the linear measurement vector of the image patch.

[0065] Step 3) Initial network reconstruction

[0066] The initial network reconstruction involves initializing the obtained measurement vectors using a convolution operation and a pixel-shuffle operation. Specifically, the sampling matrix is ​​first transposed to obtain Φ. T Then, based on the inverse problem An initial image patch is obtained through a convolutional layer, and finally, a pixel-shuffle operation is used to reassemble the image patch into a preliminary reconstructed image x. (0) .

[0067] Step 4) Deep dual-domain hybrid reconstruction network

[0068] like Figure 2 As shown, the deep dual-domain hybrid reconstruction network consists of K stages, each stage corresponding to one iteration in formulas (4) and (5), and each stage consists of two units, namely the frequency information enhancement unit FIEU and the wavelet-guided image denoising unit WIDU.

[0069] Frequency Information Enhancement Unit (FIEU): In the frequency information enhancement unit, the frequency features of the image are first obtained through the gradient descent algorithm shown in formula (4). Due to the prox operator μ,Ω The presence of a regularization term Ω(·) in the wavelet coefficients (·) makes explicit solution impossible. Therefore, a convolutional neural network (CNN) is used to approximate the proximal operator operation. Next, the obtained frequency information is processed through residual blocks to extract shallow features and enhance the frequency information representation. Each residual block consists of three sets of 3×3 convolutions and ReLU activation functions. Finally, the updated wavelet coefficients are... The reconstructed image of the DWT domain is recovered using IDWT.

[0070] Wavelet-guided image denoising unit (WIDU): In the wavelet-guided image denoising unit, image features are first obtained through the gradient descent algorithm shown in formula (5). Due to the prox operator λ,F The regularization function F(·) exists in (·), which cannot be explicitly solved. Therefore, a CNN is used to replace the proximal operator operation, and then a 3×3 convolution is used to extract r. (k) The shallow features were extracted and their channel count was increased to 32. Then, a dual-domain guided filtering module (DGFM) and a self-guided filtering enhancement module (SFEM) were used to remove noise from the features. Figure 3 As shown, in the Dual-Domain Guided Filtering (DGFM) module, the input features are first subjected to 3×3 convolution and ReLU activation to extract shallow features. Then, the wavelet map and reconstructed map output from the previous stage are used as guide maps to perform guided filtering on the extracted features. Next, the filtered features are enhanced by 3×3 convolution and ReLU activation, and the two enhanced feature sets are concatenated along the channel dimension. Residual connections are used to supplement the enhanced features. Subsequently, 3×3 convolution and ReLU activation are used again to extract deep features. Finally, considering the information loss problem during feature transmission, an improved Long Short-Term Memory (I-LSTM) mechanism is used to effectively control the information flow and capture long-term dependencies during the iteration process. In addition, the cell state of I-LSTM can retain important information and can be modified and updated through a gating mechanism, effectively reducing information loss during transmission. In the Self-Guided Filtering Enhancement (SFEM) module, the features output by DGFM are first processed through a 1×1 convolution to obtain a feature map, which is then used as the guide map. Next, the features processed by 3×3 convolution and ReLU activation are subjected to guided filtering. The filtered features are then processed again through 3×3 convolution and ReLU activation to extract depth information. Residual connections are then used to supplement this information, followed by 3×3 convolution and ReLU activation to extract features. Finally, an I-LSTM mechanism is used to select and retain valid information from the extracted features. Finally, in the Wavelet-Guided Image Denoising Unit (WIDU), the filtered features are converted into a 1-channel image through a 3×3 convolution and then r is added. (k) The final residual output x is obtained. (k) .

[0071] Step 5) Loss Function

[0072] The overall loss function consists of three parts: image reconstruction loss, wavelet coefficient consistency loss, and sampling matrix orthogonality loss. The L1 norm is used to constrain the error loss term.

[0073] II. Wavelet-guided deep learning for compressed sensing reconstruction network training

[0074] Network training parameter configuration

[0075] The T91 dataset was used as the training dataset for the network, and five networks were trained on a set of CS ratios {10%, 25%, 30%, 40%, 50%}. During training, image patches of size 32×32 with a batch size of 64 were used for training, for a total of 100 epochs. The Adam optimizer was used to optimize the network parameters, with the initial learning rate set to 5e-4. A cosine annealing strategy was used to update the learning rate and reduce it to 5e-5. The network was trained for a total of 9 stages. Except for the input and output layers, the number of output channels of all other convolutional layers was set to 32 by default. Three publicly available datasets were used as test datasets, including Set11, CBSD68, and Urban100. RGB color images were transformed into YCbCr color images, and compressed sensing observation and reconstruction were performed only on the Y channel. Finally, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) on the Y channel were used to evaluate the CS reconstruction results.

[0076] Network training

[0077] The network's loss function consists of three components: the first component is the data fidelity loss L. data It can guarantee the reconstructed image Approximately equal to the true Ground-Truth image x q The second component is the consistency constraint loss L of the wavelet coefficients. wave It can guarantee the wavelet coefficients of the reconstructed image Wavelet coefficients x of the real Ground-Truth image wq Consistency; the last component is the orthogonal loss term L applied to the sampling matrix. orth The loss functions for the three components are shown below:

[0078]

[0079]

[0080]

[0081] Where, N a N is the number of pixels in each image. b Where I is the batch size of the training images, I represents the identity matrix, and K is the total number of stages in the network. The overall loss function can be expressed as:

[0082] L(Θ)=Ldata +γL wave +ηL orth (9)

[0083] in, γ is a learnable parameter in the network, and η is two weight parameters.

[0084] III. Testing of Wavelet-Guided Deep Learning Deployment Networks for Compressed Sensing Reconstruction

[0085] 1. Comparison of objective indicators of network test results

[0086] To verify the feasibility of this patented method, it was compared with six representative image compressed sensing reconstruction methods based on black-box networks: IRCNN, ReconNet, CSNet, and others. + ReconNet, DPA-Net, and MAC-Net are shown in Table 1. Table 1 compares the average PSNR / SSIM of several methods on the Set11 and CBSD68 datasets, with the best results indicated in bold and the second-best results underlined. It is worth noting that ReconNet, DPA-Net, and MAC-Net use the same T91 dataset as our method as the training dataset, while IRCNN and CSNet... + Both SCSNet and CSNet use the BSDS400 dataset as their training dataset. As shown in Table 1, the objective metrics of the reconstructed images using the proposed method significantly outperform these six uninterpretable CS methods on the Set11 dataset, demonstrating the advantage of the proposed method. However, on the CBSD68 dataset, the PSNR of the proposed method is lower than that of CSNet at certain sampling rates. + And SCSNet, because CSNet + SCSNet uses the BSDS400 dataset with 400 images as its training dataset, while the proposed method only uses 91 images. It is well known that larger datasets increase the generalization ability of a network, and its reconstruction performance is naturally better than that of smaller datasets. Although the training datasets differ, the SSIM value of the proposed method still surpasses that of these six competing methods. In conclusion, the reconstruction capability of this patented method is superior to these six uninterpretable CS methods, sufficiently demonstrating that this patented method improves the network's reconstruction performance while possessing interpretability.

[0087] To further verify the feasibility of this patented method, it was compared with six latest interpretable compressed sensing reconstruction methods, including ISTA-Net. + OPINE-Net +MADUN, COAST, FSOINet, and DGUNet + As is well known, the total number of images in the training dataset has a significant impact on CS reconstruction performance. To ensure the fairness of the comparative experiments, all comparative methods used the T91 dataset as the training dataset and the Set11, CBSD68, and Urban100 datasets as the test datasets. The six comparative methods were retrained and retested using the source code provided by the authors. Table 2 shows the comparison of the objective quality (average PSNR and SSIM) of this patented method with the latest six competing methods at five sampling rates. As can be observed from Table 2, the reconstruction performance of this patented method reached the optimal level at all five sampling rates, OPINE-Net. + FSOINet is next, followed by ISTA-Net. + The reconstruction performance is the worst. This is due to ISTA-Net. + The network is relatively simple and does not impose constraints on the sampling matrix, resulting in poor reconstruction performance. OPINE-Net + In ISTA-Net + Based on this, binary constraints and orthogonal constraints are added to the sampling matrix, thereby improving the reconstruction performance of the network. FSOINet makes full use of feature information to reconstruct images, thus achieving high reconstruction performance. In contrast, this patented method uses frequency domain information to supplement the image domain reconstruction. Since frequency domain information can effectively preserve the high-frequency texture details of the image, its reconstruction performance is superior to competing methods.

[0088] 2. Comparison of subjective indicators of network test results

[0089] To verify the effectiveness of the method proposed in this patent, the wavelet-guided deep learning unfolding network for compressed sensing reconstruction proposed in this patent was combined with ISTA-Net. + OPINE-Net + MADUN, COAST, FSOINet, and DGUNet + Compare them. Figure 4 , Figure 5 and Figure 6 The figures show a visual comparison of several methods on the Set11, CBSD68, and Urban100 datasets, respectively. From these three figures, it can be observed that, at an observation rate of 30%, ISTA-Net... + MADUN and DGUNet + The reconstructed image suffers from severe clogging artifacts and loses some detail. OPINE-Net + COAST and FSOINet outperform ISTA-Net in reconstruction. + MADUN and DGUNet+ However, their reconstructed images are relatively blurry at the boundaries. Compared with these compressed sensing reconstruction methods, the image reconstructed by the method of this patent has more high-frequency texture details, thus achieving the best reconstruction performance, which fully verifies the superiority of the method of this patent.

[0090] Table 1

[0091]

[0092] Table 2

[0093]

[0094] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A wavelet-guided deep learning network design method for compressed sensing reconstruction, characterized in that, Includes the following steps: Step 1) Construct training data for a wavelet-guided deep learning unfolding network for compressed sensing reconstruction. Step 2) Construct and optimize a compressed sensing reconstruction model based on discrete wavelet transform. First, the Discrete Wavelet Transform (DWT) is added to the original compressed sensing reconstruction model, and the consistency constraint of the wavelet coefficients is also added to the model to construct a compressed sensing reconstruction model based on the Discrete Wavelet Transform. Then, the model is decomposed into two sub-problems, and the two sub-problems are solved iteratively using the proximal gradient descent algorithm. Step 3) Constructing the sampling network and preliminary reconstruction network First, input the image. Decompose it into non-overlapping image patches ,use Obtain the observation values ​​for each image patch. ,in For the sampling matrix, a large stride convolution operation is used here to implement compressed sensing observations. This convolution operation is abbreviated as the sampling network, and then... Obtain the initial image block Here, image initialization is achieved through a sampling convolution operation, and then a pixel-shuffle operation is used to reassemble the initialized image blocks to obtain a preliminary reconstructed image. The convolution and pixel-shuffle operations here can be simply referred to as the initial network reconstruction. Step 4) Construct a deep dual-domain hybrid reconstruction network The iterative solution algorithm obtained in step 2) is expanded into a deep dual-domain hybrid reconstruction network. This network contains two basic units: a frequency information enhancement unit and a wavelet-guided image denoising unit. The frequency information enhancement unit is used to optimize and enhance the frequency information obtained by discrete wavelet transform, and the wavelet-guided image denoising unit is used to remove reconstruction feature noise. Frequency Information Enhancement Unit (FIEU): In the Frequency Information Enhancement Unit, the frequency features of the image are first obtained through a gradient descent algorithm. ,in, Represents the gradient operator. It is a learnable step size parameter. These are the reconstructed wavelet coefficients. The index value represents the iteration, due to the proximal operator. There is a regularization term in it. Since explicit solutions are not possible, a convolutional neural network (CNN) is used to approximate the proximal operator operation. Next, the obtained frequency information is processed through residual blocks to extract shallow features and enhance the frequency information representation. Each residual block consists of three sets of 3×3 convolutions and ReLU activation functions. Finally, the updated wavelet coefficients are... The reconstructed image of the DWT domain is recovered using IDWT; Wavelet-guided image denoising unit (WIDU): In the wavelet-guided image denoising unit, image features are first obtained through a gradient descent algorithm. ,in, Represents the gradient operator. It is a learnable step size parameter. For the input image, due to the proximal operator There exists a regularization function. Since explicit solutions are not possible, a CNN is used to replace the proximal operator operation; then a 3×3 convolution is used to extract... The shallow features are extracted and their channel count is increased to 32. Then, a dual-domain guided filtering module (DGFM) and a self-guided filtering enhancement module (SFEM) are used to remove noise from the features. Finally, the filtered features are converted into a 1-channel image through a 3×3 convolution and then... Obtain the final residual output. ; Step 5) Construct a dual-domain guided filtering module and a self-guided filtering enhancement module. Two modules are constructed using guided filtering theory: In the wavelet-guided image denoising unit in step 4), both the dual-domain guided filtering module DGFM and the self-guided filtering enhancement module SFEM are used to remove feature noise. Step 6) Loss Function The overall loss function consists of three parts: image reconstruction loss, wavelet coefficient consistency loss, and sampling matrix orthogonality loss. The L1 norm is used to constrain the error loss term.

2. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 1, characterized in that: The specific method for constructing the training data for the wavelet-guided deep learning unfolding network for compressed sensing reconstruction in step 1) is as follows: The T91 dataset was used as the training dataset for the network, and five networks were trained on a set of CS ratios {10%, 25%, 30%, 40%, 50%}. During training, image patches of size 32×32 with a batch size of 64 were used for training, for a total of 100 epochs. The Adam optimizer was used to optimize the network parameters, with the initial learning rate set to 5e−4. A cosine annealing strategy was used to update the learning rate and reduce it to 5e−5. The network was trained in 9 stages. Except for the input and output layers, the number of output channels of all other convolutional layers was set to 32 by default. Three publicly available datasets were used as test datasets, including Set11, CBSD68, and Urban100. RGB color images were transformed into YCbCr color images, and compressed sensing observation and reconstruction were performed only on the Y channel. Finally, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) on the Y channel were used to evaluate the CS reconstruction results.

3. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 1, characterized in that: The specific steps of step 2) are as follows: pass The wavelet coefficients are obtained, where For the input image, This represents the discrete wavelet transform, and since the process is reversible, it can be achieved through... The original signal was restored, in which The inverse discrete wavelet transform (DWT) is introduced into the compressed sensing reconstruction model, which is expressed as follows: （1） In formula (1), the first three terms are data fidelity terms, and the last two terms are regularization terms. and It is a regularization function. and There are four learnable step size parameters. It is the input image Wavelet coefficients obtained by DWT These are the reconstructed wavelet coefficients, and Is Given the known conditions, since it is impossible to solve for both variables simultaneously, formula (1) is decomposed into the following two optimization problems: （2） （3） Next, the proximal gradient descent (PGD) algorithm will be used to solve the above optimization problem. renew Applying the near-end gradient descent (PGD) algorithm to formula (2), the iterative optimization process can be described as the following two update steps: （4） in, Represents the gradient operator. It is a learnable step size parameter. Indicates the index value of the iteration. It's about implicit priors. Proximal operators; renew Applying the PGD algorithm to formula (3), the iterative optimization process can be described as the following two update steps: （5） in, Represents the gradient operator. It is a learnable step size parameter. It corresponds to implicit priors The proximal operator, equations (4) and (5) are the alternating iterative solution expressions for the compressed sensing reconstruction model.

4. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 1, characterized in that: In step 3), the sampling network is first trained to obtain the sampling matrix. Next, all elements in the sampling matrix Φ are constrained to a binary pattern of {0, 1}, thus obtaining a data-driven binary sampling matrix, and then the input image is... Divided into Non-overlapping image blocks Finally by Obtain the linear measurement vector of the image patch. The initial reconstruction of the network involves initializing the obtained measurement vectors using a convolution operation and a pixel-shuffle operation. Specifically, the sampling matrix is ​​first transposed to obtain Φ. T Then, based on the inverse problem An initial image patch is obtained through a convolutional layer, and finally, a pixel-shuffle operation is used to reassemble the image patch into a preliminary reconstructed image. .

5. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 3, characterized in that: The deep dual-domain hybrid reconstruction network in step 4) consists of K stages, each stage corresponding to one iteration in formulas (4) and (5), and each stage consists of two units, namely the frequency information enhancement unit FIEU and the wavelet-guided image denoising unit WIDU.

6. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 1, characterized in that: The design of the dual-domain guided filtering module DGFM and the self-guided filtering enhancement module SFEM in step 5) is as follows: In the Dual-Domain Guided Filtering (DGFM) module, the input features are first subjected to 3×3 convolution and ReLU activation to extract shallow features. Then, the wavelet map and reconstructed map output from the previous stage are used as guide maps to perform guided filtering on the extracted features. Next, the filtered features are enhanced by 3×3 convolution and ReLU activation, and the two enhanced features are concatenated along the channel dimension. Residual connections are used to supplement the enhanced features. Subsequently, 3×3 convolution and ReLU activation are used again to extract deep features. Finally, considering the information loss problem during feature transmission, an improved Long Short-Term Memory (I-LSTM) mechanism is used to effectively control the information flow and capture long-term dependencies during the iteration process. In addition, the cell state of I-LSTM can retain important information and can be modified and updated through a gating mechanism, thereby effectively reducing information loss during transmission. In the Self-Guided Filtering Enhancement Module (SFEM), the features output by DGFM are first processed by a 1×1 convolution to obtain a feature map, which is then used as the guide map. Next, the features processed by 3×3 convolution and ReLU activation are subjected to guided filtering. The filtered features are then processed by another 3×3 convolution and ReLU activation to extract depth information. Residual connections are then used to supplement the information. The supplemented information is then processed by 3×3 convolution and ReLU activation to extract features. Finally, the I-LSTM mechanism is used to select the extracted features and retain the effective information.

7. The wavelet-guided deep learning unfolding network design method for compressed sensing reconstruction according to claim 4, characterized in that: The loss function in step 6) consists of three components: the first component is the data fidelity loss. It can guarantee the reconstructed image Approximately equal to the true Ground-Truth image The second component is the consistency constraint loss of the wavelet coefficients. It can guarantee the wavelet coefficients of the reconstructed image Wavelet coefficients of the real Ground-Truth image Consistency; the last component is the orthogonal loss term applied to the sampling matrix. The loss functions for the three components are shown below: （6） （7） （8） in, It is the number of pixels in each image. Where I is the batch size of the training images, I represents the identity matrix, K is the total number of stages in the network, and the total loss function can be expressed as: （9） in, These are learnable parameters in the network. and There are two weight parameters.

Images