Compressed sensing reconstruction method and system based on introduction of attention supervision deep expansion network

Abstract

The invention discloses a compressed sensing reconstruction method and system based on introduction of an attention supervision deep expansion network. Firstly, a two-dimensional image is acquired and preprocessed; and then inputting the preprocessed image into SuperTA-Net for compressed sensing reconstruction to obtain a compressed sensing measurement value of the image, and finally outputting a reconstructed high-precision image. Through a multi-channel transmission enhancement strategy, the SuperTA-Net not only can effectively reduce key information loss in an information transmission process, but also remarkably improves the performance and stability of the network by taking the pixel mean value of the image as auxiliary information. Besides, in order to further improve the reconstruction precision, a residual information supplement module (RR-Net) is designed, so that detail and texture information of the image is reconstructed better, and the problem of detail loss in image reconstruction is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and image compressed sensing, and relates to a compressed sensing reconstruction method and system based on introducing an attention-supervised deep expansion network, specifically utilizing the introduction of an attention-supervised deep expansion network SuperTA-Net to accelerate traditional compressed sensing image reconstruction and improve reconstruction accuracy and efficiency. Background Art

[0002] Compressed sensing is one of the emerging tasks in the field of computer vision. It aims to achieve data compression and transmission more efficiently by effectively capturing and utilizing the structural features and redundant information of signals such as images and videos. Compared with traditional compression methods based on signal sampling, compressed sensing methods can significantly reduce the sampling rate and storage space while maintaining high-quality reconstruction, thereby saving computing and storage costs and improving resource utilization efficiency. It has a wide range of application value and can be applied to industrial design, medical imaging, sensor machinery, public safety and other fields. However, the problems that traditional compressed sensing image reconstruction still faces include at least the following:

[0003] Low computational efficiency: Traditional algorithms are time-consuming and labor-intensive in implementing compressed sensing image reconstruction. For example, for large-scale data, the processing time is too long.

[0004] The model lacks explainability: Although the deep learning model has good reconstruction effect, its working principle is not intuitive, lacks transparency, and its internal mechanism cannot be clearly explained.

[0005] Difficulty in parameter optimization: It is difficult to select appropriate hyperparameters and perform conversions, which affects the promotion and application of the model.

[0006] Information loss problem: Deeply expanded networks are prone to losing key information in multi-layer structures, which has a negative impact on the final reconstruction quality.

[0007] The deep unfolding network is a deep learning model for compressed sensing. By combining traditional optimization iterative methods with deep neural networks, the deep unfolding network not only provides clear interpretability, but also can give full play to the advantages of neural networks to achieve fast and accurate signal or image reconstruction. However, although the deep unfolding network has improved the effect of compressed sensing reconstruction to a certain extent, it still faces problems such as information loss mentioned above, and needs further optimization and improvement to better meet the needs of practical applications. It is based on this background that the present invention proposes an innovative deep unfolding network structure SuperTA-Net, which aims to solve the problems existing in the prior art and improve the effect of compressed reconstruction. Summary of the invention

[0008] The purpose of the present invention is to address the deficiencies of the prior art and propose a compressed sensing reconstruction method and system based on introducing an attention-supervised deep expansion network.

[0009] In a first aspect, the present invention provides a compressed sensing reconstruction method based on introducing an attention-supervised deep expansion network, the method comprising the following steps:

[0010] Step 1: Obtain a two-dimensional image;

[0011] Step 2: preprocessing the two-dimensional image;

[0012] Step 3: Input the preprocessed image into the SuperTA-Net deep unrolling network with attention supervision, obtain the compressed sensing measurement value of the image and output the reconstructed image;

[0013] Wherein: the SuperTA-Net introduced with attention supervision deep unfolding network includes sampling stage, reconstruction stage and output layer;

[0014] The sampling stage is to rearrange each input image into a plurality of vectorized image block data, and obtain a de-averaged measurement value and an image block pixel mean after compression sampling;

[0015] The reconstruction stage is responsible for restoring the original image according to the de-meaning measurement value output by the sampling stage; wherein the reconstruction stage includes an initial reconstruction module IRM and a deep reconstruction layer DRMs; the initial reconstruction module IRM performs initial reconstruction on the input de-meaning measurement value through a convolution layer to generate a preliminary reconstructed image, which is then transmitted to the deep reconstruction layer DRMs; the deep reconstruction layer DRMs performs further reconstruction according to the output of the initial reconstruction module IRM;

[0016] The output layer adds the output result of the reconstruction stage to the pixel mean of the image block to obtain an image vector; the image vector is matrixed and converted into image blocks, which are finally spliced ​​into the entire image.

[0017] Preferably, the depth reconstruction layer DRMs includes K series-connected depth reconstruction modules DRM; the depth reconstruction module DRM includes an auxiliary variable update module AVDM, a multiplier update module LGDM, an image block update module IPUM, an attention supervision transmission enhancement module ASTM, and a high-frequency compensation module HFC, which are sequentially connected in series.

[0018] Preferably, the specific implementation process of the auxiliary variable update module AVDM is:

[0019]

[0020] in Represents the auxiliary variable output by the i-th iteration of the current depth reconstruction module; when k≠1, x i (k-1) represents the output of the previous depth reconstruction module; when k = 1, x i (0) represents the output of the initial reconstruction module IRM; i-1 (k) Represents the multiplier vector updated by the i-1th iteration of the current depth reconstruction module; Net z represents the subnetwork that updates the auxiliary variables; K represents the total number of deep reconstruction modules.

[0021] Preferably, the subnetwork Net for updating the auxiliary variables z It includes feature extraction module, feature enhancement module and feature aggregation module; the feature extraction module is responsible for extracting the features of input data; the feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features enhanced by the feature enhancement module through the convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 through jump connections.

[0022] Preferably, the multiplier updating module LGDM updates the Lagrange multiplier according to the auxiliary variables updated by the auxiliary variable updating module AVDM, and the specific implementation process is:

[0023]

[0024] in represents the updated auxiliary variable output by the auxiliary variable update module AVDM, Represents the image block output by the previous deep reconstruction module, is the Lagrange multiplier of the i-th iteration in the current depth reconstruction module.

[0025] Preferably, the image block updating module IPUM updates the auxiliary variables obtained by the auxiliary variable denoising module AVDM and the multiplier updating module LGDM. Multiplier vector λ i (k) , update the basic image signal to obtain the updated image block The specific implementation process is:

[0026]

[0027] Where A is the sampling matrix, y i is the observed data, is the learnable penalty parameter of the current image update module LPUN, is the auxiliary variable updated from the auxiliary variable update module AVDM, λ i (k) is the multiplier vector updated from the multiplier update module LGDM, I n represents the n×n identity matrix, where n represents the image block size.

[0028] Preferably, the specific implementation process of the attention supervision transmission enhancement module ASTM is:

[0029]

[0030] in, Represents the output of the current image block update module IPUM, Represents the transmission information of the previous depth reconstruction module, Represents the channel feature aggregation image output by the current attention supervision transfer enhancement module ASTM; ⊕

[0031] represents the channel connection operation, E(·) is the channel expansion operation, S(·) is the attention supervision operation, and A(·) is the channel aggregation operation.

[0032] Preferably, the high-frequency information supplement block HFC includes a feature extraction module, a feature enhancement module and a feature aggregation module; the output of the attention supervision transmission enhancement module The pixel mean of the image block The result after addition is used as the input of the high-frequency information supplement block HFC; the feature extraction module is responsible for extracting the features of the input data; the feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features enhanced by the feature enhancement module through the convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 through the jump connection. The final fusion result is combined with the pixel mean of the image block. The result after subtraction is output as x i (k) .

[0033] Preferably, the reconstruction stage further includes a residual recovery network RR-Net;

[0034] The residual recovery network RR-Net includes an initial residual reconstruction module and a nonlinear depth reconstruction module;

[0035] The input of the residual recovery network RR-Net is the sampling value of the residual information obtained by sampling the output of the deep reconstruction layer DRMs with the sampling matrix A and then subtracting it from the sampling value;

[0036] The initial residual reconstruction module is initialized with a 1×1 convolutional layer, each initialized residual vector is reshaped into a residual block, and all residual blocks are concatenated into a complete residual image to enter nonlinear depth reconstruction;

[0037] The nonlinear deep reconstruction module includes a residual feature extraction block, a residual feature enhancement block, and a residual feature fusion block;

[0038] The residual feature extraction block is responsible for processing the output of the initial residual reconstruction module into a high-dimensional feature map;

[0039] The residual feature enhancement block gradually extracts deeper feature representations from the high-dimensional feature map output by the residual feature extraction block;

[0040] The residual feature fusion block generates a residual image for the output of the residual feature enhancement block, and fuses the original input and the residual image through a jump connection, and then divides them into residual image blocks. After vectorization, the residual

[0041] In a second aspect, the present invention provides an image compression sensing reconstruction system based on the above method, characterized by comprising:

[0042] A data acquisition module, used for acquiring a two-dimensional image;

[0043] A data preprocessing module, used for preprocessing the two-dimensional image;

[0044] The image compressed sensing reconstruction module is used to input the preprocessed two-dimensional image into the SuperTA-Net deep unfolding network with attention supervision, gradually reconstruct the image details through the initial reconstruction and deep reconstruction modules, compensate for the lost high-frequency information, and finally output a high-precision reconstructed image.

[0045] The beneficial effects of the present invention are as follows:

[0046] 1. The deep reconstruction layer DRMs in the attention-supervised deep unfolding network SuperTA-Net adopted in the present invention includes K series-connected deep reconstruction modules DRM. The modules implement a multi-channel transmission strategy through a carefully designed information transmission method, which effectively reduces the loss of key information between deep reconstruction modules and significantly improves the reconstruction accuracy of the image.

[0047] 2. The attention supervision transmission enhancement module ASTM adopted by the present invention utilizes the attention supervision operation S to dynamically adjust the weights of each channel according to the image features. In the image block update module IPUM, the basic image signal is updated based on the update results of the auxiliary variable update module AVDM and the multiplier update module LGDM. At the same time, the attention supervision mechanism plays a role in the whole process, reducing information redundancy and avoiding excessive computational overhead. At the same time, while ensuring the reconstruction quality, a higher computational efficiency is achieved.

[0048] 3. The present invention adds the residual recovery network RR-Net to the compressed sensing reconstruction network, which can effectively reconstruct the residual information between the deep reconstruction layer DRMs output and the original image, further improving the reconstruction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is one of the structural diagrams of the deep expansion network structure SuperTA-Net provided by the present invention.

[0050] Figure 2 This is the second structural diagram of the deep expansion network structure SuperTA-Net provided by the present invention.

[0051] Figure 3 It is a detailed structural diagram of DRM in the present invention.

[0052] Figure 4 It is a detailed structural diagram of the subnetwork for updating auxiliary variables in the AVDM of the present invention.

[0053] Figure 5 It is a detailed structural diagram of ASTM in the present invention.

[0054] Figure 6 It is a detailed structural diagram of the residual recovery network of the present invention.

[0055] Figure 7 When the CS compression ratio is 30%, architectural images reconstructed by 9 different methods, among which (a) is the target image Ground truth, (b) is iPiano-Net, (c) is BCSnet, (d) is AMP-Net, (e) is COAST, (f) is TransCS, (g) is SODAS-Net, (h) is PIPO-Net, (i) is SuperTA-Net without RR-Net, and (j) is SuperTA with RR-Net.

[0056] Figure 8When the CS compression ratio is 25%, the images of the ship are reconstructed by 9 different methods, among which (a) is the target image Ground truth, (b) is iPiano-Net, (c) is BCSnet, (d) is AMP-Net, (e) is COAST, (f) is TransCS, (g) is SODAS-Net, (h) is PIPO-Net, (i) is SuperTA-Net without RR-Net, and (j) is SuperTA with RR-Net. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0058] As can be seen from the background technology, the problems still faced by traditional compressed sensing image reconstruction include at least low computational efficiency, lack of model interpretability, difficulty in parameter optimization, information loss, etc.

[0059] Therefore, the present invention proposes a novel deep unfolding network structure SuperTA-Net, which is used to improve the effect of compression reconstruction. The network enhances signal performance through a multi-channel transmission strategy, combines the attention mechanism to supervise the dynamic adjustment of channel weights, and introduces a lightweight RR-Net for residual recovery, thereby improving the reconstruction accuracy. SuperTA-Net adopts a physics-guided unfolding model, which enables the network to handle signal recovery tasks and maintains good interpretability and theoretical guarantee. SuperTA-Net works in collaboration through the sampling stage, reconstruction stage and output layer. The sampling stage uses a de-meaning sampling strategy to process the image. The reconstruction stage gradually reconstructs the image details through the initial reconstruction module IRM and the deep reconstruction layer DRMs. The multiple modules in the deep reconstruction layer DRMs and the residual recovery network RR-Net work together, and finally the output image is integrated through the output layer. During the training process, a fusion loss function is used, including a spatial domain loss function (using the average normalized root mean square error MSE) and a wavelet loss function, and the network is optimized by calculating the loss values ​​of the reconstructed image and the target image in different aspects.

[0060] The present invention will be further analyzed below in conjunction with the accompanying drawings.

[0061] This embodiment provides a system for image compression sensing reconstruction, including:

[0062] A data acquisition module is used to acquire two-dimensional images and construct a training set;

[0063] A data preprocessing module, used for preprocessing the two-dimensional image;

[0064] The image compressed sensing reconstruction module is used to input the preprocessed two-dimensional image into the SuperTA-Net deep unfolding network with attention supervision, gradually reconstruct the image details through the initial reconstruction and deep reconstruction modules, compensate for the lost high-frequency information, and finally output a high-precision reconstructed image.

[0065] The SuperTA-Net introduced with attention supervision includes sampling stage, reconstruction stage and output layer. The sampling stage is used to perform compressed sensing sampling on the original image, and adopts the mean sampling strategy to realize the compression processing of the image block. The output of this stage includes the mean measurement value and the pixel mean of the image block. The reconstruction stage is responsible for restoring the mean measurement value to the original image, which is mainly composed of the initial reconstruction module and multiple deep reconstruction modules. In the initial reconstruction module, the mean measurement value is used to generate a preliminary reconstructed image. Subsequently, the image is passed through multiple deep reconstruction modules in sequence, and finally a high-precision reconstruction is achieved through gradual feature enhancement and information compensation. A high-frequency compensation module (HFC) is added at the end of each deep reconstruction module to eliminate artifacts and restore the lost high-frequency information, combining the image signal and the pixel mean to reconstruct an image with more complete details. This architecture combines the advantages of traditional optimization iterative algorithms and deep neural networks to achieve efficient and accurate compressed sensing reconstruction.

[0066] Based on the above system, a compressed sensing reconstruction method based on introducing an attention-supervised deep expansion network is also provided, comprising the following steps:

[0067] Step 1: Get the required dataset, which comes from the BSD500 public dataset. This dataset contains rich and diverse image data, providing sufficient and representative sample sources for subsequent model training, helping to improve the model's ability to process different types of images, thereby enhancing the generalization of the model.

[0068] Step 2: Preprocess the two-dimensional images in the data set. The preprocessing methods include two-dimensional image scaling and cropping.

[0069] First, the 2D image is scaled to adjust the image to a suitable size for subsequent processing. For example, in some cases, the image is uniformly scaled to a specific resolution to ensure the consistency of the size of all input images and avoid adverse effects on model training and reconstruction due to image size differences. Then, the cropping operation is performed to remove irrelevant edges or interference parts that may exist in the image, so that the image focuses on the key information area, further optimize the data quality, and improve the efficiency of model processing.

[0070] Step 3: Input the preprocessed image into the SuperTA-Net deep unrolling network with attention supervision, obtain the compressed sensing measurement value of the image and output the reconstructed image.

[0071] in:

[0072] Build a complete attention-supervised deep unrolling network SuperTA-Net, which includes the sampling stage, reconstruction stage, and output layer, such as Figure 1 shown.

[0073] In the sampling stage, each input image is first re-divided into vectorized image block data of size n×1, and a de-meaning sampling strategy is adopted to generate de-meaning measurement values ​​and pixel means of the image block, thereby realizing compressed sampling processing of the image block.

[0074] In this embodiment, the specific process of the de-averaging sampling strategy is as follows:

[0075] Let i be the number of iterations, and let the quantized image block data be The pixel mean of the image block is

[0076] Expand the pixel mean of the image block into a vector with the same dimension as the original vectorized image block

[0077]

[0078] Perform de-meaning on the vectorized image block to obtain the de-meaning vector x i :

[0079]

[0080] The de-meaning sampling process to be achieved by compressed sensing can be formulated as:

[0081] y i =Ax i (3)

[0082] in represents the original sampling matrix;

[0083] Since x before sampling * i Unknown, so it is impossible to obtain the image data after removing the mean; to solve this problem, construct a new augmented sampling matrix:

[0084]

[0085] That is, a row of n all-1 sampling vectors is expanded after the original sampling matrix A; based on this, the actual sampling process is recorded as:

[0086]

[0087] in y i,m+1represents the total number of pixels in the image block, then The pixel mean of the image block can be obtained

[0088] The actual sample value y * i Calculate the mean measurement value y i :

[0089]

[0090] In summary, by introducing the augmented sampling matrix, the mean-removing compressed sampling of the image is achieved.

[0091] The reconstruction stage restores the original image according to the de-averaged measurement value output by the sampling stage, and includes an initial reconstruction module IRM and a deep reconstruction layer DRMs. In the initial reconstruction module, the de-averaged measurement value is used for preliminary reconstruction to generate a preliminary reconstructed image. Then, the preliminary reconstructed image is sequentially passed through the deep reconstruction layer DRMs for further reconstruction, gradually improving the reconstruction accuracy. In order to eliminate blocking artifacts in the image and restore the lost high-frequency details, a high-frequency information compensation module (HFC) is added to the end of each deep reconstruction module, which uses the basic image signal and the pixel mean of the image block to compensate for the lost high-frequency information. After high-frequency compensation, the image block is reassembled into a complete image and enters the next high-frequency information compensation module for further processing. The output image of the high-frequency information compensation module is then divided into blocks again and passed to the next deep reconstruction module for further processing.

[0092] The depth reconstruction layer DRMs includes K series-connected depth reconstruction modules DRM;

[0093] In this embodiment, if Figure 3 As shown, the deep reconstruction module (DRM) includes an auxiliary variable update module AVDM, a multiplier update module LGDM, an image block update module IPUM, an attention supervision transmission enhancement module ASTM, and a high-frequency compensation module HFC. The auxiliary variable update module AVDM, the multiplier update module LGDM, the image block update module IPUM, the attention supervision transmission enhancement module ASTM, and the high-frequency compensation module HFC are responsible for gradually optimizing and improving the reconstruction quality of the image, and the high-frequency compensation module HFC is responsible for compensating for the lost high-frequency information and eliminating blocking artifacts.

[0094] The specific implementation process of the auxiliary variable update module AVDM is:

[0095]

[0096] When k≠1, x i (k-1)represents the output of the previous depth reconstruction module; when k = 1, x i (0) represents the output of the initial reconstruction module IRM; i-1 (k) Represents the multiplier vector updated by the i-1th iteration of the current depth reconstruction module; Net z represents the subnetwork that updates the auxiliary variable z; K represents the total number of deep reconstruction modules; Indicates the auxiliary variable output by the current auxiliary variable update module AVDM

[0097] like Figure 4 As shown, the subnetwork for updating the auxiliary variable z includes a feature extraction module, a feature enhancement module and a feature aggregation module; the feature extraction module includes a convolution filter CONV1, a batch normalization (BN) and a rectified linear unit (ReLU) connected in series in sequence, aiming to extract the features of the input data; the feature enhancement module includes a first residual block RES1, a rectified linear unit (ReLU) and a second residual block RES2 connected in series in sequence, and further extracts and enhances the features extracted by the feature extraction module; the feature aggregation module includes a batch normalization (BN), a rectified linear unit (ReLU), a convolution layer CONV2 and an aggregation layer connected in series in sequence, and integrates the features enhanced by the feature enhancement module to extract deeper features, and fuses the subnetwork input for updating the auxiliary variable z and the output of the convolution layer CONV2 through a jump connection, which can promote the convergence of the subnetwork.

[0098] The multiplier updating module LGDM updates the Lagrange multiplier according to the auxiliary variables updated by the auxiliary variable updating module AVDM, which is expressed as:

[0099]

[0100] in represents the updated auxiliary variable output by the auxiliary variable update module AVDM, Represents the image block output by the previous deep reconstruction module, is the Lagrange multiplier of the i-th iteration in the current depth reconstruction module.

[0101] The main function of LGDM is to update the multiplier vector λ based on the Lagrange multiplier method through the gradient descent strategy to ensure that the constraint z = x is satisfied. The update operation of this module is essentially a gradient descent step, which is used to approximate the optimal solution and dynamically adjust the Lagrange multiplier to make the network gradually converge in each layer iteration. This module can effectively ensure that the Lagrange multiplier λ is updated in each round of optimization, thereby constraining the consistency of the reconstructed signal x and the auxiliary variable z, and improving the convergence and reconstruction accuracy of the network.

[0102] The image block update module IPUM updates the auxiliary variables obtained by the auxiliary variable denoising module AVDM and the multiplier update module LGDM. Multiplier vector λ i (k) , update the basic image signal to obtain the updated image block It is expressed as:

[0103]

[0104] Where A is the original sampling matrix, y i is the observed data, ρ is the learnable penalty parameter of the current image update module LPUN, is the auxiliary variable updated from the auxiliary variable update module (AVDM), λ i (k) is the multiplier vector updated from the multiplier update module (LGDM), I n represents the n×n identity matrix, and n represents the image block size.

[0105] The main function of IPUM is to update the current image block by solving the above formula Among them, A T A represents the autocorrelation of the sampling matrix, and ρI represents the regularization part controlled by the penalty term, which ensures that the updated image block can balance data fidelity and prior information. This module uses forward and backward projection operations to achieve efficient updates of image blocks, ensuring that the optimal solution of the image block is gradually approached in each deep reconstruction module, thereby improving the quality and accuracy of image reconstruction.

[0106] like Figure 5 As shown, the main function of the attention supervision transmission enhancement module ASTM is to enhance the feature transmission capability between modules through a multi-channel transmission strategy to reduce the loss of key information when it is transmitted between different modules. The specific update mode is:

[0107]

[0108]

[0109] in, Represents the output of the current image block update module IPUM, Represents the transmission information of the previous depth reconstruction module, Represents the output of the attention supervision transmission enhancement module ASTM. ⊕ represents the channel connection operation, E(·) is the channel expansion operation, which is implemented by the convolutional layer, S(·) is the attention supervision operation, and A(·) is the channel aggregation module, which is implemented by the convolutional layer. Through this module, the transmission of features between different depth reconstruction modules becomes more reliable and efficient, which not only improves the data transmission quality, but also reduces the impact of redundant information on the computational complexity through the attention mechanism, thereby effectively improving the accuracy of image reconstruction.

[0110] Through this module, the transmission of features between reconstruction modules at different depths becomes more reliable and efficient, which not only improves the data transmission quality, but also reduces the impact of redundant information on computational complexity through the attention mechanism, thereby effectively improving the accuracy of image reconstruction.

[0111] The high-frequency information supplement block HFC is a CNN network, and its structure is the same as the sub-network Net that updates the auxiliary variable z. z Consistent, including feature extraction, feature enhancement and feature fusion; the output of the attention supervision transmission enhancement module Add the pixel mean of the image block Get the image vector; process the image vector into a matrix, convert it into an image block, and finally splice it into the entire image as input, and send it to the high-frequency information supplement block HFC, use deep supervision to learn the features, and get the output result of the high-frequency information supplement block HFC;

[0112] The output result of the high-frequency information supplement block HFC and the pixel mean of the image block Subtract the two to get the image vector; process the image vector into a matrix and convert it into image blocks, which are finally spliced ​​into the entire image, recorded as the output x of the depth reconstruction module i (k) .

[0113] The output layer adds the output result of the reconstruction stage to the pixel mean of the image block to obtain an image vector; the image vector is matrixed and converted into image blocks, which are finally spliced ​​into the entire image.

[0114] Another embodiment, such as Figure 2 As shown, the reconstruction stage also includes a residual recovery network RR-Net; the main function of the residual recovery network is to reconstruct the residual information between the output of the deep reconstruction layer DRMs and the original image.

[0115] The residual recovery network RR-Net includes an initial residual reconstruction module and a nonlinear depth reconstruction module, such as Figure 6 shown.

[0116] The input of the residual recovery network RR-Net is the sampling value of the residual information obtained by sampling the output of the deep reconstruction layer DRMs with the sampling matrix A and subtracting it from the sampling value.

[0117] The initial residual reconstruction module is initialized with a 1×1 convolutional layer, each initialized residual vector is reshaped into a residual block, and all residual blocks are concatenated into a complete residual image to enter nonlinear depth reconstruction.

[0118] The nonlinear deep reconstruction module includes a residual feature extraction block, a residual feature enhancement block, and a residual feature fusion block;

[0119] The residual feature extraction block consists of 3×3CONV1 and ReLU activation function to obtain a high-dimensional feature map.

[0120] The residual feature enhancement block sets 5 residual blocks and relu to increase the nonlinearity of the network, which helps to gradually extract deeper feature representations for subsequent parts.

[0121] The residual feature fusion block integrates the obtained features using a 3×3 convolutional layer to generate a residual image, and fuses the input of the nonlinear deep reconstruction module and the output of the residual feature aggregation block through a jump connection to achieve the purpose of reconstructing the residual information between the output of SuperTA-Net and the original image.

[0122] The residual image output by the nonlinear depth reconstruction module is divided into residual image blocks and vectorized to obtain the residual

[0123] At this time, the output of the reconstruction phase includes the output of the residual recovery network RR-Net and the output of the last deep reconstruction layer.

[0124] During the training process of the attention-supervised deep unfolding network SuperTA-Net, the reconstructed image data is input into SuperTA-Net for training. The training will not only calculate the loss value of the reconstructed image output by the network and the target image through the spatial domain loss function, but also use the wavelet loss function to calculate the loss value of the output image of the high-frequency information supplement block of each deep reconstruction module and the target image. Through the back propagation of the fused loss value, the network iteration parameters continuously reduce the loss value to complete the training. The present invention uses the fused loss function to make the loss value calculation more comprehensive.

[0125] The total loss function is expressed as:

[0126] L total =L MSE +γL WT (12)

[0127] Where γ represents the weight;

[0128] Spatial domain loss function L MSE Use the average normalized root mean square error (MSE):

[0129]

[0130] Where numel(·) represents the image pixel counting function, ||·||F is the Frobenius norm, N represents the number of complete two-dimensional images selected for the i-th training, and X j Represents the target image Groundtruth, Represents the reconstructed image output by the SuperTA-Net deep expansion network with attention supervision;

[0131] Wavelet loss function L WT express:

[0132]

[0133] Among them, WT(·) is the Haar wavelet decomposition operation, H(·) is the high frequency compensation module (HFC) operation, is the reconstructed image after high frequency compensation processing in the kth depth reconstruction module, and k is the number of depth reconstruction modules.

[0134] During the test of SuperTA-Net, an image of 264×264 is input and a set compression ratio is selected. In the sampling stage, the image is sampled by the mean sampling strategy (MSS) method to obtain the mean measurement value and the pixel mean of the image block, and then these data are sent to the reconstruction stage; in the reconstruction stage, the initial reconstruction module (IRM) is first entered, and the mean measurement value is processed by a 1×1 convolution layer and converted into an initial reconstruction map, thereby realizing preliminary image reconstruction. Then, the initial reconstruction map will be transmitted to K deep reconstruction modules (DRM) in sequence for further processing. Each DRM module consists of multiple key submodules, including the auxiliary variable update module AVDM, the multiplier update module LGDM, the image block update module IPUM, the attention supervision transmission enhancement module ASTM, and the high-frequency compensation module HFC, which are used to gradually optimize and reconstruct the details of the image. After completing an image block update, each DRM module will be transmitted to the next module for accuracy improvement. Through these K rounds of deep reconstruction, high-precision image reconstruction can be gradually achieved. Finally, it is transmitted to the residual recovery network to supplement the residual information. Finally, after processing by all modules, the reconstructed image of size 264×264 is output, completing the compressed sensing reconstruction task.

[0135] In order to verify the effectiveness of SuperTA-Net, SuperTA-Net was compared with a variety of existing reconstruction methods, including iPiano-Net, BCSnet, AMP-Net, COAST, TransCS, SODAS-Net and PIPO-Net. Among these methods, iPiano-Net and BCSnet are compressed sensing reconstruction methods based on deep expansion, while AMP-Net and COAST are also popular compressed sensing methods in recent years. The experiment is based on the Set11 dataset, and five different compression rates (1%, 5%, 10%, 25% and 50%) are selected for testing, with PSNR value and SSIM value as evaluation indicators. Through the experimental results, the performance of SuperTA-Net at different compression rates can be comprehensively evaluated and compared with other advanced methods, thereby verifying the effectiveness and superior performance of SuperTA-Net in image reconstruction tasks.

[0136] The formula for PSNR is as follows:

[0137]

[0138] Among them, MAX is the maximum possible value of the original image or video pixel, and MSE is the mean square error. The unit of PSNR is decibel (dB). Generally speaking, the higher the PSNR value, the better the quality of the image or video.

[0139] The calculation of SSIM is based on brightness, contrast and structural similarity measures within a local window, as well as a constant C for stability. The calculation formula of SSIM is as follows:

[0140]

[0141] Among them, x and y represent the original image and the processed image respectively, μ x and μ y Represent their local brightness means, σ x and σ y Represent their local variances, σ xy represents their local covariance. C1 and C2 are constants used to avoid the denominator being zero. The result of SSIM is between -1 and 1. The closer the value is to 1, the higher the structural similarity between images, that is, the better the image quality. Compared with PSNR, SSIM can better reflect the human eye's perception of image quality. Therefore, in image processing tasks, SSIM is generally considered to be more effective than PSNR.

[0142] Compared with other methods, SuperTA-Net shows overall superior performance in image reconstruction. Under different compression ratios, SuperTA-Net achieves high reconstruction accuracy, especially under low compression ratio. By adopting multi-channel transmission strategy (MSS) and high-frequency compensation module (HFC), SuperTA-Net significantly outperforms other algorithms. On the Set11 dataset, compared with existing algorithms such as AMP-Net, SuperTA-Net has achieved significant improvements at various compression rates. For example, at compression rates of 1% and 10%, SuperTA-Net is 1.44dB and 0.40dB higher than AMP-Net, respectively. At the same time, at a compression rate of 50%, SuperTA-Net achieves a reconstruction accuracy of 40.52dB. In addition, by comparing with AMP-Net and several other advanced methods, SuperTA-Net can achieve high image quality under various compression ratios and reconstruct clearer image details. In terms of running speed, SuperTA-Net not only ensures reconstruction accuracy, but also shows a faster reconstruction speed. When the CS compression ratio is 30%, the architectural images reconstructed by 9 different methods are shown in Figure 2. Figure 7 Middle (b) to Figure 7 Middle (j), Figure 7 The target image is shown in (a). When the CS compression ratio is 25%, the images of the ship reconstructed using 9 different methods are shown in Figure 8 Middle (b) to Figure 8 Middle (j), Figure 8 The middle (a) image is the target image.

[0143] Table 1 Comparison on the Set11 dataset

[0144]

[0145] In order to verify the robustness and generalization of the proposed method on different datasets, experiments were also conducted on the BSDS500 dataset. The experimental results on the BSDS500 dataset are shown in Table 2.

[0146] Table 2 Comparison on the BSD68 dataset

[0147]

[0148] According to the experimental results, the best results are marked in bold. Compared with other methods, SuperTA-Net performs well at different compression rates. Compared with the results of AMP-Net on the Set11 dataset, except for the 50% compression rate, the PSNR and SSIM indicators of SuperTA-Net perform better in most cases. This is mainly due to the independent parameter optimization model in SuperTA-Net, which enables the network to flexibly find the optimal solution. In addition, SuperTA-Net enhances the performance and generalization ability of image reconstruction by adopting multi-channel transmission enhancement and attention supervision mechanism.

[0149] SuperTA-Net performs very well at various compression rates, especially at low compression rates (1% and 10%). For example, at a compression rate of 1%, the PSNR of SuperTA-Net reaches 22.11dB, which is a significant improvement over AMP-Net's 21.82dB, and at a compression rate of 10%, SuperTA-Net is also 1.33dB higher than AMP-Net. This can be explained by the fact that at low compression rates, important information of the original image may be severely lost due to the sampling process. For most networks, the reconstruction performance will drop significantly without additional auxiliary information, while SuperTA-Net uses pixel mean information as additional assistance through the MSS strategy to significantly improve the reconstruction performance.

[0150] In addition, SuperTA+RR-Net further improves the reconstruction accuracy by introducing the residual recovery module. At each compression rate, the PSNR and SSIM of SuperTA+RR-Net are better than other methods, especially at a compression rate of 50%, reaching 38.32dB, exceeding AMP-Net's 37.51dB.

[0151] The experimental results in the table clearly show that SuperTA-Net can not only produce better reconstruction results faster, but also maintain low computational complexity. This makes the method more feasible in practical applications. The ship image reconstruction results at a compression ratio of 25% shown in the figure also show that SuperTA-Net restores more texture details, especially in high-frequency information compensation. The HFC module effectively eliminates the block effect and further improves the visual quality of the image. These results highlight the superior performance of SuperTA-Net in image compressed sensing reconstruction.

[0152] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A compressed sensing reconstruction method based on introducing an attention-supervised deep unfolding network, the method comprising the following steps: Step 1: Obtain a two-dimensional image; Step 2: preprocessing the two-dimensional image; Step 3: Input the preprocessed image into the SuperTA-Net deep unrolling network with attention supervision, obtain the compressed sensing measurement value of the image and output the reconstructed image; It is characterized in that The SuperTA-Net introduced with attention supervision deep expansion network includes sampling stage, reconstruction stage and output layer; The sampling stage is to rearrange each input image into a plurality of vectorized image block data, and obtain a de-averaged measurement value and an image block pixel mean after compression sampling; The reconstruction stage is responsible for restoring the original image according to the de-meaning measurement value output by the sampling stage; wherein the reconstruction stage includes an initial reconstruction module IRM and a deep reconstruction layer DRMs; the initial reconstruction module IRM performs initial reconstruction on the input de-meaning measurement value through a convolution layer to generate a preliminary reconstructed image, which is then transmitted to the deep reconstruction layer DRMs; the deep reconstruction layer DRMs performs further reconstruction according to the output of the initial reconstruction module IRM; The output layer adds the output result of the reconstruction stage to the pixel mean of the image block to obtain an image vector; the image vector is matrixed and converted into image blocks, which are finally spliced ​​into the entire image.

2. The method according to claim 1, characterized in that: The deep reconstruction layer DRMs includes K series-connected deep reconstruction modules DRM; the deep reconstruction module DRM includes an auxiliary variable update module AVDM, a multiplier update module LGDM, an image block update module IPUM, an attention supervision transmission enhancement module ASTM, and a high-frequency compensation module HFC, which are connected in series in sequence.

3. The method according to claim 2, characterized in that: The specific implementation process of the auxiliary variable update module AVDM is: in Represents the auxiliary variable output by the i-th iteration of the current depth reconstruction module; when k≠1, x i (k-1) represents the output of the previous depth reconstruction module; when k = 1, x i (0) represents the output of the initial reconstruction module IRM; i-1 (k) Represents the multiplier vector updated by the i-1th iteration of the current depth reconstruction module; Net z represents the subnetwork that updates the auxiliary variables; K represents the total number of deep reconstruction modules.

4. The method according to claim 3, characterized in that: The subnetwork Net that updates the auxiliary variables z It includes feature extraction module, feature enhancement module and feature aggregation module; the feature extraction module is responsible for extracting the features of input data; The feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features enhanced by the feature enhancement module through the convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 through jump connections.

5. The method according to claim 2, characterized in that: The multiplier updating module LGDM updates the Lagrange multiplier according to the auxiliary variables updated by the auxiliary variable updating module AVDM. The specific implementation process is: in represents the updated auxiliary variable output by the auxiliary variable update module AVDM, Represents the image block output by the previous deep reconstruction module, is the Lagrange multiplier of the i-th iteration in the current depth reconstruction module.

6. The method according to claim 2, characterized in that: The image block update module IPUM updates the auxiliary variables obtained by the auxiliary variable denoising module AVDM and the multiplier update module LGDM. Multiplier vector λ i (k) , update the basic image signal to obtain the updated image block The specific implementation process is: Where A is the sampling matrix, y i is the observed data, is the learnable penalty parameter of the current image update module LPUN, is the auxiliary variable updated from the auxiliary variable update module AVDM, λ i (k) is the multiplier vector updated from the multiplier update module LGDM, I n represents the n×n identity matrix, where n represents the image block size.

7. The method according to claim 2, characterized in that: The specific implementation process of the attention supervision transmission enhancement module ASTM is: in, Represents the output of the current image block update module IPUM, Represents the transmission information of the previous depth reconstruction module, represents the channel feature aggregation image output by the current attention supervision transfer enhancement module ASTM; ⊕ represents the channel connection operation, E(·) is the channel expansion operation, S(·) is the attention supervision operation, and A(·) is the channel aggregation operation.

8. The method according to claim 2, characterized in that: The high-frequency information supplement block HFC includes a feature extraction module, a feature enhancement module and a feature aggregation module; the channel features output by the attention supervision transmission enhancement module ASTM are aggregated into an image The pixel mean of the image block The result after addition is used as the input of the high-frequency information supplement block HFC; the feature extraction module is responsible for extracting the features of the input data; The feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features enhanced by the feature enhancement module through the convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 through the jump connection. The final fusion result is combined with the pixel mean of the image block. The result after subtraction is output as x i (k) .

9. The method according to claim 1, characterized in that: The reconstruction stage also includes a residual recovery network RR-Net; The residual recovery network RR-Net includes an initial residual reconstruction module and a nonlinear depth reconstruction module; The input of the residual recovery network RR-Net is the sampling value of the residual information obtained by sampling the output of the deep reconstruction layer DRMs with the sampling matrix A and then subtracting it from the sampling value; The initial residual reconstruction module is initialized with a 1×1 convolutional layer, each initialized residual vector is reshaped into a residual block, and all residual blocks are concatenated into a complete residual image to enter nonlinear depth reconstruction; The nonlinear deep reconstruction module includes a residual feature extraction block, a residual feature enhancement block, and a residual feature fusion block; The residual feature extraction block is responsible for processing the output of the initial residual reconstruction module into a high-dimensional feature map; The residual feature enhancement block gradually extracts deeper feature representations from the high-dimensional feature map output by the residual feature extraction block; The residual feature fusion block generates a residual image for the output of the residual feature enhancement block, and fuses the original input and the residual image through a jump connection, and then divides them into residual image blocks. After vectorization, the residual 10. An image compression sensing reconstruction system implementing the method according to any one of claims 1 to 9, characterized in that include: A data acquisition module, used for acquiring a two-dimensional image; A data preprocessing module, used for preprocessing the two-dimensional image; The image compressed sensing reconstruction module is used to input the preprocessed two-dimensional image into the SuperTA-Net deep unfolding network with attention supervision, gradually reconstruct the image details through the initial reconstruction and deep reconstruction modules, compensate for the lost high-frequency information, and finally output a high-precision reconstructed image.

Images