A compressed sensing reconstruction method based on a deep unfolding network with attention supervision and a system thereof

Abstract

The application discloses a compressed sensing reconstruction method based on a deep unfolding network with attention supervision and a system thereof. Firstly, a two-dimensional image is acquired and preprocessed; then the preprocessed image is input into a SuperTA-Net for compressed sensing reconstruction to obtain a compressed sensing measurement value of the image, and finally a high-precision reconstructed image is output. The SuperTA-Net adopts a multi-channel transmission enhancement strategy, which can not only effectively reduce the loss of key information in the information transmission process, but also significantly improve the performance and stability of the network by taking the pixel mean value of the image as auxiliary information. In addition, in order to further improve the reconstruction accuracy, the application designs a residual information supplement module (RR-Net), so that the details and texture information of the image can be better reconstructed, and the problem of detail loss in image reconstruction can be effectively solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and image compressive sensing, and relates to a compressive sensing reconstruction method based on a deep unfolding network with attention supervision and a system thereof, in particular to a deep unfolding network SuperTA-Net with attention supervision, which is used to accelerate the traditional compressive sensing image reconstruction and improve the reconstruction accuracy and efficiency. BACKGROUND

[0002] Compressive sensing is one of the emerging tasks in the field of computer vision, aiming to effectively capture and utilize the structural features and redundant information of images, videos and other signals to achieve more efficient data compression and transmission. Compared with traditional compression methods based on signal sampling, the compressive sensing method can greatly reduce the sampling rate and storage space while maintaining high-quality reconstruction, thereby saving computing and storage costs, improving resource utilization efficiency, and having wide application value, which can be applied to industrial design, medical images, sensors, public safety and other fields. However, the traditional compressive sensing image reconstruction still faces at least the following problems:

[0003] Low computational efficiency: Traditional algorithms consume a lot of time and effort in realizing compressive sensing image reconstruction, such as for large-scale data, the processing time is too long.

[0004] Lack of model interpretability: Although deep learning models have good reconstruction effect, the working principle is not intuitive, lack of transparency, and the internal mechanism cannot be clearly explained.

[0005] Parameter optimization difficulty: It is difficult to choose appropriate hyperparameters and conversion, which affects the promotion and application of the model.

[0006] Information loss problem: Deep unfolding network is easy to lose key information in multi-layer structure, which has adverse effects on the final reconstruction quality.

[0007] Deep unfolding network is a deep learning model for compressive sensing, which combines traditional optimization iterative methods with deep neural networks. Deep unfolding network not only provides explicit interpretability, but also fully utilizes the advantages of neural networks to realize fast and accurate signal or image reconstruction. However, although deep unfolding network improves the effect of compressive sensing reconstruction to some extent, it still faces problems such as information loss mentioned above, which needs to be further optimized and improved to better meet the actual application requirements. Based on this background, the present application proposes an innovative deep unfolding network structure SuperTA-Net, which aims to solve the problems existing in the prior art and improve the effect of compression reconstruction. SUMMARY

[0008] The invention aims at the deficiencies of the prior art and provides a compressed sensing reconstruction method based on a deep unfolding network with attention supervision and a system thereof.

[0009] In a first aspect, the invention provides a compressed sensing reconstruction method based on a deep unfolding network with attention supervision, which comprises the following steps:

[0010] Step one, obtaining a two-dimensional image;

[0011] Step two, preprocessing the two-dimensional image;

[0012] Step three, inputting the preprocessed image into the deep unfolding network with attention supervision SuperTA-Net, obtaining the compressed sensing measurement value of the image and outputting the reconstructed image;

[0013] The deep unfolding network with attention supervision SuperTA-Net comprises a sampling stage, a reconstruction stage, and an output layer.

[0014] The sampling stage rearranges each input image into multiple vectorized image block data, and obtains the mean-removed measurement value and the image block pixel mean value after compressed sampling.

[0015] The reconstruction stage is responsible for restoring the original image from the mean-removed measurement value output by the sampling stage; wherein the reconstruction stage comprises an initial reconstruction module IRM and a deep reconstruction layer DRMs; the initial reconstruction module IRM performs initial reconstruction on the input mean-removed measurement value through a convolution layer to generate a preliminary reconstructed image, which is then transmitted to the deep reconstruction layer DRMs; the deep reconstruction layer DRMs further reconstructs according to the output of the initial reconstruction module IRM.

[0016] The output layer adds the output result of the reconstruction stage to the image block pixel mean value to obtain an image vector; the image vector is matrixed and converted into an image block, and finally spliced into an entire image.

[0017] Preferably, the deep reconstruction layer DRMs comprises K deep reconstruction modules DRM connected in series; the deep reconstruction module DRM comprises an auxiliary variable update module AVDM, a multiplier update module LGDM, an image block update module IPUM, an attention supervision transmission enhancement module ASTM, and a high-frequency compensation module HFC connected in series.

[0018] Preferably, the specific implementation process of the auxiliary variable update module AVDM is as follows:

[0019] (1)

[0020] wherein denotes the auxiliary variable representing the output of the i-th iteration of the current depth reconstruction module; when k≠1, denotes the output of the previous depth reconstruction module; when k=1, denotes the output of the initial reconstruction module IRM; denotes the multiplier vector updated by the i-1-th iteration of the current depth reconstruction module; denotes the sub-network for updating the auxiliary variable; K denotes the total number of the depth reconstruction modules.

[0021] Preferably, the sub-network for updating the auxiliary variable includes a feature extraction module, a feature enhancement module and a feature aggregation module; the feature extraction module is responsible for extracting the features of the input data; the feature enhancement module is responsible for further extracting and strengthening the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features strengthened by the feature enhancement module through a convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 together through a skip connection.

[0022] Preferably, the multiplier updating module LGDM updates the Lagrange multiplier according to the auxiliary variable updated by the auxiliary variable updating module AVDM, and the specific implementation process is:

[0023] (2)

[0024] wherein denotes the updated auxiliary variable output by the auxiliary variable updating module AVDM, denotes the image block output by the previous depth reconstruction module, is the Lagrange multiplier of the i-th iteration in the current depth reconstruction module.

[0025] Preferably, the image block updating module IPUM updates the basic image signal according to the auxiliary variable updated by the auxiliary variable denoising module AVDM and the multiplier vector updated by the multiplier updating module LGDM, to obtain an updated image block , and the specific implementation process is:

[0026] (3)

[0027] wherein A is a sampling matrix, is an observation data, is a learnable penalty parameter of the current image updating module LPUN, is the auxiliary variable updated by the auxiliary variable updating module AVDM, is the multiplier vector updated by the multiplier updating module LGDM, denotes an n x n identity matrix, n denotes the image block size.

[0028] Preferably, the specific implementation process of the attention supervision transmission enhancement module ASTM is:

[0029] (4)

[0030] (5)

[0031] wherein, denotes the output of the current image block update module IPUM, denotes the transmission information of the previous depth reconstruction module, denotes the channel feature aggregated image output by the current attention supervision transmission enhancement module ASTM; denotes the channel connection operation, E(·) is the channel expansion operation, S(·) is the attention supervision operation, A(·)

[0032] is the channel aggregation operation.

[0033] Preferably, the high-frequency information supplement block HFC includes a feature extraction module, a feature enhancement module and a feature aggregation module; the output of the attention supervision transmission enhancement module is added to the image block pixel mean value , and the result is taken as the input of the high-frequency information supplement block HFC; the feature extraction module is responsible for extracting the features of the input data; the feature enhancement module is responsible for further extracting and strengthening the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the features strengthened by the feature enhancement module through a convolution layer CONV2, extracting deeper features, and then fusing the original input and the output of the convolution layer CONV2 through a skip connection. Finally, the fusion result is subtracted from the image block pixel mean value , and the result is taken as the output .

[0034] Preferably, the reconstruction stage further includes a residual recovery network RR-Net.

[0035] The residual recovery network RR-Net includes an initial residual reconstruction module and a nonlinear depth reconstruction module.

[0036] The input of the residual recovery network RR-Net is the residual information sampling value obtained by subtracting the sampling value from the output of the depth reconstruction layer DRMs after sampling with the sampling matrix A;

[0037] The initial residual reconstruction module initializes using a 1x1 convolution layer, reshapes each initialized residual vector into a residual block, and splices all residual blocks into a whole residual image, which enters the nonlinear depth reconstruction.

[0038] The nonlinear depth reconstruction module comprises a residual feature extraction block, a residual feature enhancement block and a residual feature fusion block.

[0039] The residual feature extraction block is responsible for processing the output of the initial residual reconstruction module into a high-dimensional feature map.

[0040] The residual feature enhancement block gradually extracts deeper feature representations from the high-dimensional feature map output by the residual feature extraction block.

[0041] The residual feature fusion block generates a residual image from the output of the residual feature enhancement block, and fuses the original input and the residual image through a skip connection, and then divides the residual image into residual image blocks, and obtains the residual .

[0042] In a second aspect, the present application provides an image compressive sensing reconstruction system based on the above method, characterized in that it comprises:

[0043] A data acquisition module is configured to acquire a two-dimensional image.

[0044] A data preprocessing module is configured to preprocess the two-dimensional image.

[0045] An image compressive sensing reconstruction module is configured to input the preprocessed two-dimensional image into the attention supervision deep unfolding network SuperTA-Net, gradually reconstruct the image details through the initial reconstruction and depth reconstruction modules, compensate for the lost high-frequency information, and finally output a high-precision reconstructed image.

[0046] The present application has the following advantages:

[0047] 1. The depth reconstruction layer DRMs in the attention supervision deep unfolding network SuperTA-Net used in the present application comprises K depth reconstruction modules DRM connected in series, and the multi-channel transmission strategy is realized through the carefully designed information transmission mode between the modules, effectively reducing the loss of key information between the depth reconstruction modules, and significantly improving the reconstruction accuracy of the image.

[0048] 2. The attention supervision transmission enhancement module ASTM used in the present application utilizes the attention supervision operation S, which can dynamically adjust the weights of each channel according to the image features. In the image block update module IPUM, based on the update results of the auxiliary variable update module AVDM and the multiplier update module LGDM, the basic image signal is updated, and the attention supervision mechanism plays a role in the whole process, reducing information redundancy and avoiding excessive computational overhead. At the same time, while ensuring the reconstruction quality, a high computational efficiency is achieved.

[0049] 3、The residual recovery network RR-Net is added to the compressed sensing reconstruction network, residual information between the output of the depth reconstruction layer DRM and the original image can be effectively reconstructed, and the reconstruction accuracy is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is one of the structure diagrams of the deep unfolding network structure SuperTA-Net provided by the application.

[0051] Figure 2 is the second structure diagram of the deep unfolding network structure SuperTA-Net provided by the application.

[0052] Figure 3 is a detailed structure diagram of the DRM in the application.

[0053] Figure 4 is a detailed structure diagram of the sub-network for updating auxiliary variables in the AVDM in the application.

[0054] Figure 5 is a detailed structure diagram of the ASTM in the application.

[0055] Figure 6 is a detailed structure diagram of the residual recovery network in the application.

[0056] Figure 7 When the CS compression ratio is 30%, the building images reconstructed by 9 different methods are shown, wherein (a) is a target image Ground truth, (b) is iPiano-Net, (c) is BCSnet, (d) is AMP-Net, (e) is COAST, (f) is TransCS, (g) is SODAS-Net, (h) is PIPO-Net, (i) is SuperTA-Net without RR-Net, and (j) is SuperTA with RR-Net.

[0057] Figure 8 When the CS compression ratio is 25%, the ship images reconstructed by 9 different methods are shown, wherein (a) is a target image Ground truth, (b) is iPiano-Net, (c) is BCSnet, (d) is AMP-Net, (e) is COAST, (f) is TransCS, (g) is SODAS-Net, (h) is PIPO-Net, (i) is SuperTA-Net without RR-Net, and (j) is SuperTA with RR-Net. DETAILED DESCRIPTION

[0058] From the background art, the traditional compressed sensing image reconstruction still faces problems at least including low computational efficiency, lack of model interpretability, difficulty in parameter optimization, information loss, etc.

[0059] Therefore, the present application proposes a new deep unfolding network structure SuperTA-Net for improving the effect of compressed reconstruction. The network enhances signal performance through a multi-channel transmission strategy, combines attention mechanism to supervise dynamic adjustment of channel weights, and introduces a lightweight RR-Net for residual recovery, thereby improving reconstruction accuracy. SuperTA-Net adopts a physical guidance-based unfolding model, enabling the network to perform signal recovery tasks and maintaining good interpretability and theoretical guarantee. SuperTA-Net works cooperatively through a sampling stage, a reconstruction stage, and an output layer. The sampling stage uses a mean-removed sampling strategy to process the image, the reconstruction stage gradually reconstructs image details through an initial reconstruction module IRM and deep reconstruction layers DRMs, multiple modules in the deep reconstruction layers DRMs and a residual recovery network RR-Net jointly act, and finally the output image is integrated through the output layer. In the training process, a fusion loss function is used, including a spatial domain loss function (using mean normalized mean square error MSE) and a wavelet loss function, and the network is optimized by calculating the loss value of the reconstructed image and the target image in different aspects.

[0060] The present application will be further analyzed in combination with the accompanying drawings.

[0061] The present embodiment provides an image compressed sensing reconstruction system, comprising:

[0062] A data acquisition module is configured to acquire a two-dimensional image and construct a training set.

[0063] A data preprocessing module is configured to preprocess the two-dimensional image.

[0064] An image compressed sensing reconstruction module is configured to input the preprocessed two-dimensional image into a deep unfolding network SuperTA-Net with attention supervision, gradually reconstruct image details through an initial reconstruction module and deep reconstruction modules, compensate for lost high-frequency information, and finally output a high-precision reconstructed image.

[0065] The introduction of attention supervision deep unfolding network SuperTA-Net includes a sampling stage, a reconstruction stage, and an output layer. The sampling stage is used for compressive sensing sampling of the original image, and a mean removal sampling strategy is adopted to realize the compression of the image block. The output of this stage includes the mean removal measurement value and the pixel mean value of the image block. The reconstruction stage is responsible for restoring the mean removal measurement value to the original image, mainly composed of an initial reconstruction module and multiple deep reconstruction modules. In the initial reconstruction module, the mean removal measurement value is used to generate a preliminary reconstructed image. Subsequently, the image is sequentially passed through multiple deep reconstruction modules, and through step-by-step feature enhancement and information compensation, a high-precision reconstruction is finally achieved. A high-frequency compensation module (HFC) is added at the end of each deep reconstruction module to eliminate artifacts and restore lost high-frequency information, combining image signals and pixel means to reconstruct an image with more complete details. This architecture combines the advantages of traditional optimization iterative algorithms and deep neural networks, achieving efficient and accurate compressive sensing reconstruction.

[0066] Based on the above system, a compressive sensing reconstruction method based on the introduction of attention supervision deep unfolding network is also provided, including the following steps:

[0067] Step one, obtain the required data set, and the data source is the BSD500 public data set. This data set contains a wide variety of image data, providing sufficient and representative sample sources for subsequent model training, which helps to improve the model's processing ability for different types of images, thereby enhancing the model's generalization.

[0068] Step two, pre-process the two-dimensional images in the data set. The pre-processing method includes two-dimensional image scaling and cropping.

[0069] First, scale the two-dimensional image to adjust the image to the appropriate size for subsequent processing. For example, in some cases, the image is uniformly scaled to a specific resolution to ensure the consistency of all input images in size, avoiding adverse effects on model training and reconstruction results due to differences in image size. Then, the cropping operation is performed to remove irrelevant edges or interference parts that may exist in the image, focusing the image on the key information area, further optimizing data quality and improving model processing efficiency.

[0070] Step three, input the pre-processed image into the introduction of attention supervision deep unfolding network SuperTA-Net to obtain the compressive sensing measurement value of the image and output the reconstructed image.

[0071] Wherein:

[0072] A complete introduction of attention supervision deep unfolding network SuperTA-Net is built, which includes a sampling stage, a reconstruction stage, and an output layer, as shown in Figure 1 .

[0073] During the sampling phase, each input image is first re-divided into vectorized image block data of size n×1, and a mean-removing sampling strategy is adopted to generate mean-removing measurements and pixel mean values ​​of the image blocks, thereby achieving compressed sampling processing of the image blocks.

[0074] In this embodiment, the specific process of the mean-removal sampling strategy is as follows:

[0075] Let i be the iteration number, and let the vectorized image patch data be... The average pixel value of the image block is ;

[0076] The average pixel value of the image patch is augmented to a vector with the same dimension as the original vectorized image patch. :

[0077] (1)

[0078] The vectorized image patch is then subjected to mean-removal processing to obtain the mean-removed vector. :

[0079] (2)

[0080] The mean-free sampling process to be implemented in compressed sensing can be formulated as follows:

[0081] (3)

[0082] in Represents the original sampling matrix;

[0083] Due to sampling Since the image data is unknown, the mean-reduced image data cannot be obtained. To solve this problem, a new augmented sampling matrix is ​​constructed:

[0084] (4)

[0085] That is, after the original sampling matrix A, add a row of n sampling vectors all 1s; based on this, the actual sampling process is denoted as:

[0086] (5)

[0087] in ; If the sum of pixels in an image block is represented, then... This allows us to obtain the average pixel value of an image block. ;

[0088] From actual sampled values Calculate the mean measurement value :

[0089] (6)

[0090] In summary, by introducing the augmented sampling matrix, the image is de-meaning compressed sampling.

[0091] The reconstruction stage recovers the original image from the de-meaning measurements output by the sampling stage, which includes an initial reconstruction module (IRM) and depth reconstruction layers (DRMs). In the initial reconstruction module, the de-meaning measurements are used for preliminary reconstruction to generate a preliminary reconstructed image. Then, the preliminary reconstructed image is sequentially passed through the depth reconstruction layers (DRMs) for further reconstruction, gradually improving the accuracy of the reconstruction. In order to eliminate blocking artifacts in the image and recover the lost high-frequency details, a high-frequency information compensation module (HFC) is added at the end of each depth reconstruction module. This module uses the basic image signal and the pixel mean of the image block to compensate for the lost high-frequency information. After high-frequency compensation, the image block is reassembled into a complete image and enters the next high-frequency information compensation module for further processing. The output image of the high-frequency information compensation module is then re-blocked and passed to the next depth reconstruction module for further processing.

[0092] The depth reconstruction layers (DRMs) include K depth reconstruction modules (DRMs) connected in series;

[0093] In this embodiment, as shown in Figure 3 , the depth reconstruction module (DRM) includes an auxiliary variable update module (AVDM), a multiplier update module (LGDM), an image block update module (IPUM), an attention supervision transmission enhancement module (ASTM), and a high-frequency compensation module (HFC). The auxiliary variable update module (AVDM), the multiplier update module (LGDM), the image block update module (IPUM), the attention supervision transmission enhancement module (ASTM), and the high-frequency compensation module (HFC) are responsible for gradually optimizing and improving the reconstruction quality of the image, and the high-frequency compensation module (HFC) is responsible for compensating for the lost high-frequency information and eliminating blocking artifacts.

[0094] The specific implementation process of the auxiliary variable update module (AVDM) is as follows:

[0095] (7)

[0096] where k≠1, represents the output of the previous depth reconstruction module; when k=1, represents the output of the initial reconstruction module (IRM); represents the multiplier vector obtained by the current depth reconstruction module after the i-1th iteration update; represents the subnetwork for updating the auxiliary variable z; K represents the total number of depth reconstruction modules; auxiliary variable outputted by the current auxiliary variable update module AVDM.

[0097] As shown in Figure 4 The sub-network for updating the auxiliary variable z includes a feature extraction module, a feature enhancement module, and a feature aggregation module. The feature extraction module includes a convolution filter CONV1, a batch normalization (BN), and a rectified linear unit (ReLU) connected in series, aiming to extract the features of the input data. The feature enhancement module includes a first residual block RES1, a rectified linear unit (ReLU), and a second residual block RES2 connected in series, further extracting and strengthening the features extracted by the feature extraction module. The feature aggregation module includes a batch normalization (BN), a rectified linear unit (ReLU), a convolution layer CONV2, and an aggregation layer connected in series, integrating the features strengthened by the feature enhancement module, extracting deeper features, and fusing the sub-network input for updating the auxiliary variable z and the output of the convolution layer CONV2 through a skip connection, which can promote the convergence of the sub-network.

[0098] The Lagrange multiplier updating module LGDM updates the Lagrange multiplier based on the auxiliary variable outputted by the auxiliary variable update module AVDM, which is represented as:

[0099] (8)

[0100] wherein represents the updated auxiliary variable outputted by the auxiliary variable update module AVDM, represents the image block outputted by the previous deep reconstruction module, is the Lagrange multiplier of the i-th iteration in the current deep reconstruction module.

[0101] The main function of the LGDM is to update the multiplier vector λ based on the Lagrange multiplier method through a gradient descent strategy, thereby ensuring the satisfaction of the constraint condition z=x. The updating operation of this module is essentially a gradient descent step, which is used to approximate the optimal solution and make the network gradually converge in each iteration by dynamically adjusting the Lagrange multiplier. This module can effectively ensure the updating of the Lagrange multiplier λ in each optimization round, thereby constraining the consistency of the reconstructed signal x and the auxiliary variable z, and improving the convergence and reconstruction accuracy of the network.

[0102] The image block updating module IPUM updates the basic image signal based on the auxiliary variable and the multiplier vector outputted by the auxiliary variable denoising module AVDM and the Lagrange multiplier updating module LGDM, respectively, to obtain the updated image block , which is represented as:

[0103] (9)

[0104] where A is the original sampling matrix, is the observed data, and p is the learnable penalty parameter of the current image update module LPUN, is the auxiliary variable updated from the auxiliary variable update module (AVDM), is the multiplier vector updated from the multiplier update module (LGDM), denotes the n x n identity matrix, and n denotes the image patch size.

[0105] The main function of the IPUM is to update the current image patch by solving the above formula . Where A T A represents the autocorrelation of the sampling matrix, and pI represents the regularization part controlled by the penalty term, which ensures that the updated image patch can balance data fidelity and prior information. This module uses forward and backward projection operations to achieve efficient updating of the image patch, ensuring that the optimal solution of the image patch is gradually approached in each depth reconstruction module, thereby improving the quality and accuracy of image reconstruction.

[0106] As shown in Figure 5 , the main function of the attention supervision transmission enhancement module ASTM is to enhance the feature transmission capability between modules through a multi-channel transmission strategy, to reduce the loss of key information during transmission between different modules. The specific update mode is:

[0107] (10) (11)

[0108] where denotes the output of the current image patch update module IPUM, denotes the transmission information of the previous depth reconstruction module, denotes the output of the attention supervision transmission enhancement module ASTM. ⊕ denotes the channel connection operation, E(·) is the channel expansion operation, which is realized by a convolution layer, S(·) is the attention supervision operation, and A(·) is the channel aggregation module, which is realized by a convolution layer. Through this module, the transmission of features between different depth reconstruction modules becomes more reliable and efficient, not only improving the transmission quality of data, but also reducing the impact of redundant information on computational complexity through the attention mechanism, thereby effectively improving the accuracy of image reconstruction.

[0109] Through this module, the transmission of features between different depth reconstruction modules becomes more reliable and efficient, not only improving the transmission quality of data, but also reducing the impact of redundant information on computational complexity through the attention mechanism, thereby effectively improving the accuracy of image reconstruction.

[0110] The high-frequency information supplement block HFC is a CNN network, and a subnetwork for updating an auxiliary variable z Consistent, including feature extraction, feature enhancement and feature fusion three parts; attention supervision transmission enhancement module output Join the image block pixel mean Get the image vector; the image vector is matrixed and converted into an image block, and finally spliced into an entire image as an input, which is sent to the high-frequency information supplement block HFC to learn the features using deep supervision, and the output result of the high-frequency information supplement block HFC is obtained.

[0111] The output result of the high-frequency information supplement block HFC and the image block pixel mean Subtract to get the image vector; the image vector is matrixed and converted into an image block, and finally spliced into an entire image, denoted as the output of the deep reconstruction module .

[0112] The output layer adds the output result of the reconstruction stage to the image block pixel mean to obtain an image vector; the image vector is matrixed and converted into an image block, and finally spliced into an entire image.

[0113] Another embodiment, as Figure 2 The reconstruction stage also includes a residual recovery network RR-Net; the main function of the residual recovery network is to reconstruct the residual information between the output of the deep reconstruction layer DRMs and the original image.

[0114] The residual recovery network RR-Net includes an initial residual reconstruction module and a nonlinear deep reconstruction module, as Figure 6 shown.

[0115] The input of the residual recovery network RR-Net is the residual information obtained by subtracting the sampling value from the output of the deep reconstruction layer DRMs sampled by the sampling matrix A.

[0116] The initial residual reconstruction module uses a 1x1 convolution layer to initialize, reshapes each initialized residual vector into a residual block, splices all residual blocks into a whole residual image, and enters the nonlinear deep reconstruction.

[0117] The nonlinear deep reconstruction module includes a residual feature extraction block, a residual feature enhancement block, and a residual feature fusion block.

[0118] The residual feature extraction block is composed of 3x3 CONV1 and ReLU activation function, and a high-dimensional feature map is obtained.

[0119] The residual feature enhancement block sets five residual blocks and relu to increase the nonlinearity of the network, which helps to gradually extract deeper feature representations for subsequent parts.

[0120] The residual feature fusion block integrates the obtained features by using a 3x3 convolution layer to generate a residual image, and fuses the input of the nonlinear depth reconstruction module and the output of the residual feature aggregation block through a skip connection to achieve the purpose of reconstructing the residual information between the output of the SuperTA-Net and the original image.

[0121] The residual image output by the nonlinear depth reconstruction module is divided into residual image blocks, and vectorized to obtain residual ;

[0122] At this time, the output of the reconstruction stage includes the output of the residual recovery network RR-Net and the output of the last depth reconstruction layer.

[0123] In the training process of the attention supervised depth unfolding network SuperTA-Net, the reconstructed image data is input into the SuperTA-Net for training. The training not only calculates the loss value of the reconstructed image output by the network and the target image through the spatial domain loss function, but also calculates the loss value of the output image of each high-frequency information supplement block of the depth reconstruction module and the target image using the wavelet loss function. Through the back propagation of the fusion loss value, the network iteration parameters are constantly reduced to complete the training. The present application uses the fusion loss function, so that the loss value calculation is more comprehensive.

[0124] The total loss function is represented as:

[0125] (12)

[0126] wherein represents the weight;

[0127] The spatial domain loss function The mean normalized mean square error (MSE) is adopted:

[0128] (13)

[0129] wherein numel(·) represents an image pixel number counting function, ||·||F is the Frobenius norm, N represents the number of complete two-dimensional images selected in the i-th training, represents the target image Ground truth, represents the reconstructed image output by the attention supervised depth unfolding network SuperTA-Net;

[0130] The wavelet loss function is represented as:

[0131] (14)​

[0132] where WT(·) is the Haar wavelet decomposition operation, H(·) is the high frequency compensation module (HFC) operation, is the reconstructed image after high frequency compensation processing in the kth depth reconstruction module, k is the number of depth reconstruction modules.

[0133] In the test process of introducing the attention supervision depth unfolding network SuperTA-Net, an image with a size of 264x264 is input, and a set compression ratio is selected. The image is sampled by the mean shift sampling strategy (MSS) method in the sampling stage to obtain the mean shift measurement value and the image block pixel mean, and then these data are sent to the reconstruction stage; in the reconstruction stage, first, an initial reconstruction module (IRM) is entered, and a 1x1 convolution layer is used to process the mean shift measurement value to convert it into an initial reconstructed image, thereby realizing preliminary image reconstruction. Then, the initial reconstructed image is transmitted to K depth reconstruction modules (DRM) in turn for further processing. Each DRM module is composed of multiple key sub-modules, including an auxiliary variable update module AVDM, a multiplier update module LGDM, an image block update module IPUM, an attention supervision transmission enhancement module ASTM, and a high frequency compensation module HFC, which are used to gradually optimize and reconstruct the details of the image. After each DRM module completes an image block update, it is transmitted to the next module for accuracy improvement. Through K rounds of depth reconstruction, high-precision image reconstruction can be gradually achieved. Finally, it is transmitted to the residual recovery network for residual information supplement. Finally, after processing by all modules, a reconstructed 264x264 image is output, and the compressed sensing reconstruction task is completed.

[0134] In order to verify the effectiveness of SuperTA-Net, SuperTA-Net is compared with a variety of existing reconstruction methods, including iPiano-Net, BCSnet, AMP-Net, COAST, TransCS, SODAS-Net, and PIPO-Net. Among these methods, iPiano-Net and BCSnet are depth-unfolding-based compressed sensing reconstruction methods, while AMP-Net and COAST are also popular compressed sensing methods in recent years. The experiment is based on the Set11 dataset, and five different compression ratios (1%, 5%, 10%, 25%, and 50%) are selected for testing, with PSNR and SSIM values as evaluation indicators. Through the experimental results, the performance of SuperTA-Net under different compression ratios can be comprehensively evaluated, and compared with other advanced methods, thereby verifying the effectiveness and superior performance of SuperTA-Net in image reconstruction tasks.

[0135] The formula of PSNR is as follows:

[0136] (10)

[0137] where MAX is the maximum possible value of the original image or video pixel, and MSE is the mean squared error. PSNR is measured in decibels (dB), and generally, the higher the PSNR value, the better the quality of the image or video.

[0138] The calculation of SSIM is based on the luminance, contrast, and structural similarity measures within a local window, as well as a constant C for stability. The calculation formula of SSIM is as follows:

[0139] (11)

[0140] where x and y represent the original image and the processed image, respectively, and represent their local luminance mean, and represent their local variance, represent their local covariance. C1 and C2 are constants to avoid zero denominator. The result of SSIM is between -1 and 1, and the closer the value is to 1, the higher the structural similarity between images, i.e., the better the image quality. Compared with PSNR, SSIM can better reflect the perception of human eyes to image quality, so in image processing tasks, SSIM is generally considered more effective than PSNR.

[0141] Compared with other methods, SuperTA-Net shows overall superior performance in image reconstruction. Under different compression ratios, SuperTA-Net achieves higher reconstruction accuracy, especially under low compression ratio. By adopting the multi-channel transmission strategy (MSS) and the high-frequency compensation module (HFC), SuperTA-Net is significantly superior to other algorithms. On the Set11 dataset, compared with existing algorithms such as AMP-Net, SuperTA-Net has made great progress under each compression ratio. For example, under the compression ratios of 1% and 10%, SuperTA-Net is 1.44dB and 0.40dB higher than AMP-Net, respectively. At the same time, under the compression ratio of 50%, SuperTA-Net achieves a reconstruction accuracy of 40.52dB. In addition, by comparing with AMP-Net and other advanced methods, SuperTA-Net can achieve higher image quality and reconstruct clearer image details under various compression ratios. In terms of running speed, SuperTA-Net also shows faster reconstruction speed while ensuring reconstruction accuracy. When the CS compression ratio is 30%, the building image reconstructed by 9 different methods is shown in FIG. (b) to Figure 7 (b) toFigure 7 Fig. (j), Figure 7 Fig. (a) is the target image. The images of the ship reconstructed by 9 different methods when the CS compression ratio is 25%, see Figure 8 Fig. (b) to Figure 8 Fig. (j), Figure 8 Fig. (a) is the target image.

[0142] Table 1 Comparison on Set11 dataset

[0143]

[0144] In order to verify the robustness and generalization of the proposed method on different datasets, experiments were also conducted on the BSDS500 dataset, and the experimental results on the BSDS500 dataset are shown in Table 2.

[0145] Table 2 Comparison on BSD68 dataset

[0146]

[0147] According to the experimental results, the best results are marked in bold, and compared with other methods, SuperTA-Net performs excellently under different compression ratios. Compared with the results of AMP-Net on the Set11 dataset, except for the 50% compression ratio, SuperTA-Net performs better in most cases in terms of both PSNR and SSIM. This is mainly due to the independent parameter optimization model in SuperTA-Net, which enables the network to flexibly find the optimal solution. In addition, SuperTA-Net enhances the performance and generalization ability of image reconstruction by adopting multi-channel transmission enhancement and attention supervision mechanism.

[0148] SuperTA-Net performs very well under each compression ratio, especially in the case of low compression ratio (1% and 10%). For example, under 1% compression ratio, the PSNR of SuperTA-Net reaches 22.11 dB, which is significantly higher than that of AMP-Net (21.82 dB), and under 10% compression ratio, SuperTA-Net is also 1.33 dB higher than AMP-Net. This can be explained as, under low compression ratio, important information of the original image may be severely lost due to the sampling process. For most networks, the reconstruction performance will decrease significantly without additional auxiliary information, while SuperTA-Net uses the pixel mean information as an additional auxiliary through the MSS strategy, which significantly improves the reconstruction performance.

[0149] In addition, SuperTA+RR-Net further improves the reconstruction accuracy by introducing a residual recovery module. Under each compression ratio, the PSNR and SSIM of SuperTA+RR-Net are better than those of other methods, especially at a compression ratio of 50%, which reaches 38.32 dB, exceeding the 37.51 dB of AMP-Net.

[0150] The experimental results in the table clearly show that SuperTA-Net not only produces better reconstruction results more quickly, but also maintains a low computational complexity. This makes the method more feasible in practical applications. The ship image reconstruction results shown in the figure at a compression ratio of 25% also show that SuperTA-Net recovers more texture details, especially in high-frequency information compensation. The HFC module effectively eliminates blocking effects, further improving the visual quality of the image. These results highlight the superior performance of SuperTA-Net in image compressive sensing reconstruction.

[0151] The above is a preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, which are also considered within the scope of protection of the present application.

Claims

1. A compressed perception reconstruction method based on an attention-supervised deep unfolded network, the method comprising the following steps: Step 1: Obtain a two-dimensional image; Step 2: Preprocess the two-dimensional image; Step 3: Input the preprocessed image into the attention-supervised deep unrolling network SuperTA-Net to obtain the compressed sensing measurement values ​​of the image and output the reconstructed image; Its features are, The SuperTA-Net, which incorporates attention-supervised deep unfolding network, includes a sampling phase, a reconstruction phase, and an output layer. The sampling stage involves rearranging each input image into multiple vectorized image block data, and then compressing the sampling to obtain the mean-free measurement value and the mean pixel value of the image block. The reconstruction stage is responsible for restoring the original image based on the mean-reduced measurement values ​​output by the sampling stage. The reconstruction stage includes an initial reconstruction module (IRM), depth reconstruction layers (DRMs), and a residual recovery network (RR-Net). The initial reconstruction module (IRM) performs initial reconstruction on the input mean-reduced measurement values ​​through convolutional layers to generate a preliminary reconstructed image, which is then transmitted to the depth reconstruction layers (DRMs). The depth reconstruction layers (DRMs) perform further reconstruction based on the output of the initial reconstruction module (IRM). The residual recovery network (RR-Net) includes an initial residual reconstruction module and a nonlinear depth reconstruction module. The input to the residual recovery network RR-Net is the sampled value of the residual information obtained by sampling the output of the depth reconstruction layer DRMs with the sampling matrix A and subtracting the sampled value from the sampled value. The initial residual reconstruction module uses a 1×1 convolutional layer to initialize, reshape each initialized residual vector into a residual block, and stitch all residual blocks together to form a complete residual image, which then enters nonlinear depth reconstruction. The nonlinear deep reconstruction module includes a residual feature extraction block, a residual feature enhancement block, and a residual feature fusion block; The residual feature extraction block is responsible for processing the output of the initial residual reconstruction module into a high-dimensional feature map; The residual feature enhancement block progressively extracts deeper feature representations from the high-dimensional feature map output by the residual feature extraction block; The residual feature fusion block generates a residual image from the output of the residual feature enhancement block, and then fuses the original input and the residual image through a skip connection, and then divides it into residual image blocks, which are then vectorized to obtain the residual. The output layer adds the output result of the reconstruction stage to the average pixel value of the image block to obtain an image vector; the image vector is then matrixed and converted into image blocks, which are finally stitched together to form the entire image.

2. The method according to claim 1, characterized in that, The depth reconstruction layers (DRMs) include K cascaded depth reconstruction modules (DRMs); each depth reconstruction module (DRM) includes, in sequence, an auxiliary variable update module (AVDM), a multiplier update module (LGDM), an image patch update module (IPUM), an attention-supervised transfer enhancement module (ASTM), and a high-frequency compensation module (HFC).

3. The method according to claim 2, characterized in that, The specific implementation process of the auxiliary variable update module AVDM is as follows: （1） in This represents the auxiliary variable output by the i-th iteration of the current depth reconstruction module; when k≠1, This represents the output of the previous depth reconstruction module; when k=1, This represents the output of the initial reconstruction module (IRM). This represents the multiplier vector obtained from the (i-1)th iteration update of the current depth reconstruction module; This represents the subnetwork that updates auxiliary variables; K represents the total number of deep reconstruction modules.

4. The method according to claim 3, characterized in that, The subnetwork that updates auxiliary variables It includes a feature extraction module, a feature enhancement module, and a feature aggregation module; the feature extraction module is responsible for extracting features from the input data. The feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the enhanced features of the feature enhancement module through the convolutional layer CONV2 to extract deeper features, and then fusing the original input and the output of the convolutional layer CONV2 together through skip connections.

5. The method according to claim 2, characterized in that, The multiplier update module LGDM updates the Lagrange multipliers based on the auxiliary variables obtained by the auxiliary variable update module AVDM. The specific implementation process is as follows: （2） in This represents the updated auxiliary variable output by the auxiliary variable update module AVDM. This represents the image patch output by the previous depth reconstruction module. It is the Lagrange multiplier of the i-th iteration in the current deep reconstruction module.

6. The method according to claim 2, characterized in that, The image block update module IPUM updates the auxiliary variables based on the auxiliary variable denoising module AVDM and the multiplier update module LGDM. Multiplier vector The basic image signal is updated to obtain the updated image patch. The specific implementation process is as follows: （3） Where A is the sampling matrix. For observation data, The learnable penalty parameters for the current image update module LPUN. The auxiliary variables are obtained from the auxiliary variable update module AVDM. It is the multiplier vector obtained from the multiplier update module LGDM. Let n represent an n×n identity matrix, where n represents the size of the image patch.

7. The method according to claim 2, characterized in that, The specific implementation process of the attention supervision transmission enhancement module ASTM is as follows: （4） （5） in, This represents the output of the current image patch update module IPUM. This indicates the transmission information from the previous depth reconstruction module. This represents the aggregated image of the channel features output by the current attention-supervised transmission enhancement module (ASTM); ⊕ indicates a channel connection operation. For channel expansion operations, For attention-monitored operations, This is a channel aggregation operation.

8. The method according to claim 2, characterized in that, The high-frequency compensation module HFC includes a feature extraction module, a feature enhancement module, and a feature aggregation module; Transmit the output of the attention supervision enhancement module Image patch pixel mean The summation result serves as the input to the High Frequency Compensation (HFC) module; the feature extraction module is responsible for extracting features from the input data. The feature enhancement module is responsible for further extracting and enhancing the features extracted by the feature extraction module; the feature aggregation module is responsible for integrating the enhanced features from the feature enhancement module through the convolutional layer CONV2 to extract deeper features, and then fusing the original input and the output of the convolutional layer CONV2 together through skip connections. The final fusion result is then compared with the average pixel value of the image patch. The result of the subtraction is used as the output. .

9. An image compressed sensing reconstruction system implementing the method of any one of claims 1-8, characterized in that... include: The data acquisition module is used to acquire two-dimensional images; The data preprocessing module is used to preprocess two-dimensional images; The image compression-sensory reconstruction module is used to input the preprocessed 2D image into the attention-supervised depth unfolding network SuperTA-Net. The image details are gradually reconstructed through the initial reconstruction and depth reconstruction modules, the lost high-frequency information is compensated, and finally a high-precision reconstructed image is output.

Images