Image reconstruction method coupling multi-branch feature domain gradient update and attention mechanism

Abstract

The application discloses an image reconstruction method coupled with multi-branch feature domain gradient updating and attention mechanism, and comprises the following steps: constructing and training a deep unfolding compressed sensing model coupled with a multi-branch feature domain gradient updating module and a cross-attention feature reconstruction module, using the trained deep unfolding compressed sensing model to recover an original image to obtain a reconstructed image x^; the specific process of the deep unfolding compressed sensing model recovery is as follows: firstly, for a target compressed sensing rate, a block sparse sampling is performed on the input original image by using a convolutional form sampling matrix Phi; then, an initialized image is mapped to a feature space as input features of the first reconstruction stage. The application introduces a multi-branch feature domain gradient updating module and a cross-iteration / cross-channel attention mechanism, thereby improving the accuracy and stability of compressed sensing image reconstruction.

Description

Technical Field

[0001] This invention relates to the technical fields of image processing and deep learning, and in particular to an image reconstruction method that couples multi-branch feature domain gradient update with attention mechanism. Background Technology

[0002] Compressive sensing (CS) is a signal processing theory that breaks through the limitations of the traditional Nyquist sampling theorem. It can reconstruct the original signal using a small amount of randomly sampled data at a rate far lower than the Nyquist sampling rate, thus significantly reducing data acquisition and storage costs. In the image domain, CS reconstructs a complete image by sparse sampling and nonlinear reconstruction, acquiring only partial frequency domain coefficients (such as random measurements or undersampled K-space data). Since Candes et al. proposed the CS theory in 2004, this technology has received widespread attention and application in information theory, medical imaging, remote sensing, hyperspectral imaging, and other fields.

[0003] Existing compressed sensing image reconstruction methods are mainly divided into two categories: traditional optimization algorithms and deep learning-based methods. Traditional methods (such as the ISTA algorithm based on convex optimization, greedy algorithms, and iterative thresholding algorithms) solve the reconstruction problem through prior sparsity constraints. These algorithms have good mathematical interpretability and convergence, but also suffer from high computational complexity, limited ability to model complex signals, and long reconstruction time. With the rapid development of deep learning, deep learning-based image CS reconstruction methods have greatly improved reconstruction speed and accuracy. These methods are generally divided into two categories: Deep Non-Unfolded Networks (DNUN) and Deep Unfolded Networks (DUN). The former directly learns the mapping from the measurement domain to the original image, which has high performance but is like a black box and lacks interpretability; the latter borrows from the traditional optimization iterative process, unfolding each iteration into network layers, combining computational efficiency and a certain degree of interpretability, and has become the current mainstream, such as ISTA-Net+ and COAST.

[0004] However, even based on the aforementioned DNUN and traditional DUN, existing work still has significant shortcomings in modeling measurement consistency and prior constraints. On the one hand, most deep unfolded networks still use a single-path convolutional residual structure in the gradient update stage. The update rules are highly uniform in spatial and channel dimensions, making it difficult to adaptively allocate different update intensities for smooth regions, texture edges, and complex details. This easily leads to a dilemma of "either over-smoothing or noise amplification" in low sampling rate and strong texture scenes. On the other hand, the attention mechanisms incorporated in existing methods are mostly limited to spatial or channel self-attention within a single iteration. They lack explicit modeling of the cross-channel correlation between "1-channel image structural information" and "multi-channel deep features," and rarely perform selective cross-iteration interaction on the feature representations of adjacent iterations. This makes it difficult to transfer and reuse the useful information accumulated in intermediate iterations in a long-term and stable manner. More importantly, the gradient update process in existing methods is often equivalent to "fixed-form step size adjustment," lacking the ability to adaptively adjust the update path and amplitude based on content, starting from the joint measurement residuals and multi-scale features. The aforementioned issues collectively limit the reconstruction performance of existing DUNs at low sampling rates, complex textures, and long iteration depths. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] In view of the problems existing in the above-mentioned image reconstruction methods that couple multi-branch feature domain gradient update and attention mechanism, this invention is proposed.

[0007] Therefore, the purpose of this invention is to provide an image reconstruction method that couples multi-branch feature domain gradient update with an attention mechanism. By introducing a multi-branch feature domain gradient update module and a cross-iteration / cross-channel attention mechanism, it improves the accuracy and stability of compressed sensing image reconstruction.

[0008] To address the aforementioned technical problems, this invention provides the following technical solution: an image reconstruction method coupling multi-branch feature domain gradient update and attention mechanism, comprising the following steps: A deep unfolded compressed sensing model coupled with a multi-branch feature domain gradient update module and a cross-attention feature reconstruction module is constructed and trained. The trained deep unfolded compressed sensing model is used to reconstruct the original image to obtain the reconstructed image x^. The specific process of reconstructing the deep-unfolded compressed sensing model is as follows: First, to achieve the target compression sensing rate, a block-based sparse sampling method is used on the input original image using a convolutional sampling matrix Φ; then, the transpose of the sampling matrix Φ is used for convolution. The measurement data is back-projected to obtain the initial image. ; Next, the image will be initialized. Mapped to the feature space as input features for the first reconstruction stage. The network structure is then deployed in K reconstruction phases. Each reconstruction phase includes: a multi-branch gradient update module MBGU based on measurement consistency, and cross-iteration channel attention CICA and cross-channel attention module CCAM. The multi-branch gradient update module MBGU, cross-iteration channel attention CICA, and cross-channel attention module CCAM are connected in series to form the gradient-attention iterative module FAI. Let x be the original image after full sampling, and y be the measurement data after sampling by the sampling matrix. Let z(k-2) be the input features of the k-th stage FAI module, and z(k-2) be the cross-iteration features of the k-th stage feature reconstruction module. After K reconstruction stages, a convolution operation is performed to reduce the number of channels to 1, and the final output is the reconstructed image. .

[0009] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, the compressed sensing image reconstruction method of branch feature domain gradient update and cross-attention mechanism is characterized in that the calculation process of the initial image is as follows: The input original image x is divided into blocks of size 32×32, and the j-th block is denoted as . ; Use a convolutional sampling matrix corresponding to the CS ratio of compressed sensing Sparse sampling is performed on each image patch. Since each row of the sampling matrix can be regarded as a 32×32 filter kernel, convolution operation is used to perform the sampling process. The sampling formula is as follows: ; For all measurements The measured image y is obtained by stitching the images together; then the sampling matrix is ​​used. transposed convolution By back-projecting the measured value y, we obtain the initial image: ; in, For the first image in the original image A matrix of pixel values ​​for each sampling block. The first sample obtained after sampling The measured values ​​of each sampling block.

[0010] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, wherein: for the first... The reconstruction phase includes data consistency updates in the feature domain and interactive refinement of hidden state features, specifically comprising the following steps: First, the output features from the previous stage Input to the Multi-Branch Gradient Update (MBGU) module, based on measurements Perform feature domain gradient correction under data consistency constraints to obtain corrected features. ; Secondly, the correction features Features of hidden states and A common input feature refinement module; wherein, the feature refinement module first uses cross-iteration channel attention (CICA) to... and Cross-stage information interaction and enhancement are performed to obtain the fused latent state representation; subsequently, the feature refinement module uses cross-channel attention CCAM to integrate the fused latent state representation with image features. Perform attention-guided feature fusion and refinement, and output refined features. ; Finally, the output of the MBGU is added to the output of the feature refinement module via residual connection to obtain the first... Phase Reconstruction Output

[0011] And update the hidden state so that the hidden state of the next phase satisfies: the hidden state of the current phase... As the next stage and will from The new hidden state features extracted are used as the next stage. This enables the recursive updating of hidden states across stages and the transfer of feature memory.

[0012] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, the multi-branch gradient update module (MBGU) performs gradient descent update with data consistency constraints in the feature domain. Its update process includes "feature-pixel mapping, measurement residual calculation, residual backprojection and upsampling, feature domain lifting, and multi-branch residual correction," specifically: firstly, the input features are... A single-channel image is obtained by projecting it onto the pixel domain through convolution. Then, the sampling operator was used. right Perform forward sampling to obtain the predicted measurement and compare it with the actual measurement value. Calculate the measurement residual Then, the transpose sampling operator is used. For residuals Perform back projection and use the sub-pixel upsampling operator PixelShuffle to restore the block domain result to the original spatial resolution to obtain the pixel domain residual. Then The feature is corrected by mapping back to the feature domain through convolution and inputting it into a multi-branch residual block; finally, the output feature is obtained by adding the input features through residual concatenation. Its mathematical expression is as follows:

[0013] in, Represents the convolution operator; For sampling operators, The back projection operator corresponding to its transpose; This represents the convolution projection operator that maps features to the pixel domain. This represents a convolutional mapping operator that lifts pixel-domain residuals back to the feature domain. It contains three parallel convolutional branches, whose outputs are programmed with learnable weights. Weighted fusion is performed to adaptively adjust the contribution of different branches to the feature correction amount.

[0014] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, the feature refinement module FRM includes a cross-iteration channel attention module CICA and a cross-channel attention module CCAM. The cross-iteration channel attention module CICA is used to model the correlation between latent state features in adjacent iteration stages to achieve cross-stage information interaction and memory transfer. The CICA module uses the latent state features from the previous stage... Features of the hidden state at the current stage As input, a query vector is generated through linear mapping / convolutional mapping. Key vector AND value vector And based on channel-dimensional attention, the attention weights are calculated, and the attention weights are obtained. Perform weighted aggregation to update the hidden state features, and output the fused hidden state features. The cross-channel attention module (CCAM) is used to utilize the fused latent state features. Attention-guided reconstruction and refinement of image features; the CCAM module uses image features Features of hidden states As input, calculate the mutual attention between the two to obtain the enhanced image features. In addition, the feature refinement module also includes a feedforward network (FFN), which... A nonlinear transformation is performed and added to its residual to further enhance the feature representation ability, thus obtaining the predicted features. Its mathematical expression is as follows:

[0015] in, This represents a feedforward neural network, which includes forward and backward convolutional paths. It is used to perform channel transformation and nonlinear mapping on input features, thereby enhancing the feature refinement effect.

[0016] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, the cross-iteration channel attention module CICA normalizes the query features and key features and calculates the channel attention weights to realize the correlation modeling and information interaction of latent state features in adjacent iteration stages. Its specific calculation formula is as follows:

[0017] in, Generated by convolutional mapping of input latent state features; This is a normalization operation; Used to generate the attention weight matrix; For convolution mapping; This represents the hidden state characteristics of the current stage; The updated hidden state features output by the CICA module; the above Residual connections are used to preserve the hidden state information of the current stage and enhance training stability; The cross-channel attention module (CCAM) is used to guide the reconstruction and refinement of image features using latent state features. It achieves cross-channel information fusion by calculating the cross-similarity between latent state features and image features and generating an attention map. The specific calculation formula is as follows:

[0018] in, For the input image features, Input hidden state features (can be output from the CICA module); and For the convolutional mapping operator used to generate mutual attention, A convolution operator that performs feature transformation on hidden state features. A convolution operator for channel alignment of attention aggregation results. Output the merged convolutional operator; This is a normalization operation; This is a location encoding used to inject spatial location information; Indicates feature concatenation operation; This is a similarity matrix. For attention mapping, This refers to the refined features output by the CCAM module.

[0019] As a preferred embodiment of the image reconstruction method coupled with multi-branch feature domain gradient update and attention mechanism described in this invention, the following steps are taken: When training the depth unfolded compressed sensing model, the input training image is cropped into 89,600 sub-images and data augmentation is performed to expand the training samples and improve the model's generalization ability; in terms of network parameter settings, the reconstruction block size is set to 32 and the batch size is set to 16; the Adam optimizer is used to iteratively update the network parameters, and the training is performed using a cosine annealing learning rate strategy, with a total training epoch of 80 epochs and an initial learning rate of... The warm-up rounds consist of 2 epochs; during training, mean squared error is used as the loss function to constrain the difference between the reconstructed image and the fully sampled real image. The loss function is defined as:

[0020] in, Indicates network parameters; Indicates the first A fully sampled real image; This represents the corresponding compressed sensing reconstructed image; This indicates the number of training samples.

[0021] The beneficial effects of this invention are: (1) This invention introduces a multi-branch gradient update module (MBGU) in each iteration stage. This module first uses a learnable convolutional sampling matrix Φ to map the current features to the pixel domain and then performs forward measurement, calculates the residual between the current feature and the actual measurement value, and then completes block-level back projection by combining the transposed convolution of Φ with PixelShuffle. Finally, it updates the feature domain through a multi-branch residual convolutional network. Compared with reconstruction networks that rely solely on stacked black-box convolutions, this design explicitly embeds the compressed sensing measurement equation into the network, ensuring strict consistency between the reconstruction results and the actual measurement data, improving the physical interpretability of the algorithm, accelerating the iteration convergence speed, and significantly enhancing the reconstruction stability under high compression ratios.

[0022] (2) This invention constructs a cross-iteration channel attention (CICA) mechanism to adaptively fuse feature information between adjacent iteration stages. This module uses the feature map of the previous stage as the key and value, and the feature map of the current stage as the query. It calculates attention weights along the channel dimension, selectively enhances or suppresses historical features, and then feeds them back to the current stage for updating. Compared to traditional deep unfolded networks that only use simple residual accumulation or independent stage processing, this cross-iteration attention mechanism can explicitly establish "iteration memory," fully explore the correlation between multiple gradient updates, adaptively retain historical information beneficial to the current reconstruction, and suppress the accumulation of useless or even harmful information, thereby achieving higher reconstruction accuracy and better robustness with the same number of iteration layers.

[0023] (3) This invention designs a cross-channel attention module (CCAM) between the image domain and the feature domain. It jointly normalizes the reconstructed image of the current 1-channel image with the 31-channel high-dimensional features, generates query, projection, and value features through three-branch convolution, establishes cross-attention relationships in the channel space, and combines position embedding of depthwise separable convolution with 1×1 convolution to complete the fusion and reprojection of 32 channels (1-channel image + 31-channel features). Compared with schemes that only perform convolution processing within a single feature domain, this module can adaptively adjust the weights of each channel feature according to the image content, achieving bidirectional information interaction between the image domain and the feature domain, highlighting key information such as edges and textures, and effectively suppressing noise and block artifacts. Combined with the aforementioned measurement consistency gradient update and end-to-end learnable convolutional sampling matrix, this invention can achieve a higher peak signal-to-noise ratio and better subjective visual reconstruction results at the same compressed sensing sampling rate. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is an overall framework diagram of the present invention; Figure 2 This is a schematic diagram of the structure of the multi-branch gradient update module (MBGU) in the embodiment; Figure 3 This is a schematic diagram of the overall structure of the cross-attention module in the embodiment; Figure 4 This is a schematic diagram of the internal structure of the Cross-Iteration Channel Attention (CICA) module in the embodiment; Figure 5 This is a schematic diagram of the internal structure of the cross-channel attention module (CCAM) in the embodiment. Detailed Implementation

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0027] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0028] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0029] Reference Figures 1-5 This paper presents a compressed sensing image reconstruction method that couples multi-branch feature domain gradient update with a cross-attention mechanism. A deep unfolded compressed sensing model coupled with this mechanism is constructed and trained. The trained model is then used to reconstruct the original image, yielding the reconstructed image. ; The specific process of reconstructing the depth-unfolded compressed sensing model is as follows: First, a sampling matrix corresponding to the compressed sensing sampling rate is used. The input original image is compressed and sampled. The sampling process can be represented as performing a convolution operation on the image to obtain measurement values; then, the sampling matrix is ​​used... transposed convolution Obtain the initial reconstructed image; Then, the initial reconstructed image is input into a depthwise unfolded reconstruction network, and K iterations of reconstruction are performed. Each iteration includes a feature domain gradient update module and a feature attention iteration module to progressively optimize the features of the reconstructed image. The feature domain gradient update module uses a multi-branch residual structure to update the gradient descent direction of the current reconstructed features; the feature attention iteration module consists of a cross-iteration channel attention module and a cross-channel attention module, combining the feature information from the previous iteration and the features of the current iteration for attention enhancement. Through K iterations, the feature representation of the image reconstruction result is continuously corrected and refined.

[0030] Assumption y represents the original image after full sampling, and y represents the image to be reconstructed after sampling by the sampling matrix. The initial image is obtained by performing a convolution with weights equal to the transpose of the sampling matrix. To represent the image reconstructed from the initialization The initial cross-iteration channel features (memory tensor) obtained through the feature extraction network. This is the input to the Kth stage FAI module. After K reconstruction stages, a final convolution operation is performed to reduce the number of channels to 1, resulting in the final output. This invention combines multi-branch feature domain gradient update with a cross-attention mechanism, improving the accuracy and stability of compressed sensing image reconstruction.

[0031] Furthermore, the calculation process for the initial image is as follows: First, block-based compressed perceptual sampling is performed on the input grayscale image. The image is then divided into non-overlapping segments. Sub-blocks, each sub-block vectorized as A learnable convolution kernel matrix is ​​used. As the sampling matrix, its convolution stride is set to 32 to complete the compressed sampling of each image patch:

[0032] in For the first The measurement vector of each block. From the perspective of convolution, Each row can be understood as a trainable sampling filter.

[0033] After obtaining the measurements of all blocks, use The transposed convolution, combined with the pixel shuffle operation, is used to perform initial reconstruction of the entire image, resulting in an initial image estimate:

[0034] This process is essentially equivalent to using a pair of learnable matrices. and Compressed sampling and linear backprojection reconstruction are implemented in the image block domain.

[0035] Furthermore, for the Kth reconstruction stage, the feature representation output from the previous stage is denoted as... Then, it enters the multi-branch gradient update module. The core idea of ​​the multi-branch gradient update module is to perform a learnable "gradient descent + nonlinear enhancement" in the feature domain while ensuring measurement consistency.

[0036] First, a single-channel convolutional kernel is used. Mapping features back to a pixel-domain approximation:

[0037] Then, using the currently learnable sampling matrix right Perform a "forward measurement":

[0038] And calculate and compare with actual measurements residuals:

[0039] The measurement residual is passed through The transposed convolution and PixelShuffle operation are back-projected back into the image space to obtain the residual image:

[0040] Then use convolution kernel The residual is boosted from the single-channel pixel domain back to the multi-channel feature domain and then fed into a multi-branch residual block for nonlinear feature enhancement to obtain the residual features. .

[0041] Finally, the first The feature update form for the next iteration is:

[0042] From an optimization algorithm perspective, this process is equivalent to performing gradient descent updates with a learnable step size and a nonlinear mapping in the feature domain. This can be abstracted as: in A learnable "step size" factor. Implemented by multi-branch residual blocks, it is used to enhance feature representation and suppress noise.

[0043] To further leverage the correlation between different iteration stages, a cross-iteration channel attention module is introduced to adaptively fuse features from the previous and current stages. Let the features from the previous iteration be... The current iterative feature is Both are the same size .

[0044] After performing LayerNorm normalization on both, convolution is used to generate the query (Q), key (K), and value (V):

[0045] After flattening the spatial dimensions The dimensions are uniform .

[0046] Next, the attention weights are calculated along the channel dimension:

[0047] Then apply it to the value vector: in for Convolution is used for channel fusion. Residual connections ensure that cross-iteration information is introduced without excessively distorting the features of the current iteration. By explicitly establishing channel correlations between adjacent iterations through cross-iteration self-attention, it plays the role of "inertial memory".

[0048] To further enhance the interaction between single-channel image information and multi-channel features, a cross-channel attention module is designed to achieve bidirectional information flow between the image domain and the feature domain.

[0049] Let the input image features (1 channel) be... Multi-channel characteristics First, LayerNorm normalization is performed on both: Then, three branches of convolution are used to generate: : "Query" representation as a multi-channel feature; : "Projection-guided" representation generated from single-channel image features; : As a weighted "value" feature.

[0050] Calculate the cross-channel attention matrix:

[0051] And make it act on Attention-weighted features are obtained. .go through convolution After being combined with the location embedding module (depthiably separable convolution + GELU), residual fusion is performed with the original multi-channel features:

[0052] Finally, the original single-channel image features are combined with After splicing in the channel dimension, and then going through one Convolutional processes are used to fuse the data, resulting in the output features for this stage. By guiding the information interaction between the image domain and the feature domain through cross-channel self-attention, it is beneficial to recover texture details and structural information.

[0053] Furthermore, the depth-unfolded compressed sensing model was trained on a 3090 GPU. 400 natural images (including the original training and test sets) were selected from the BSD500 database, uniformly converted to grayscale, and then subjected to data augmentation operations such as random cropping, rotation, and horizontal / vertical flipping to generate a large number of images. Image patches are used as training samples. For network parameters, the patch size is 32, and the default batch size is 16. In the experiment, the number of feature channels per layer was fixed at 16, and the number of iteration phases was set to 16, forming a network with an overall unfolded depth of 16 layers. The batch size during training was set to 32. Compressed sensing sampling rates of 10%, 25%, 30%, 40%, and 50% were selected, corresponding to the reconstruction difficulty under different measurement resources.

[0054] End-to-end training is performed using the Adam optimizer, with an initial learning rate set to... The training consists of 100 epochs. The first 3 epochs use a learning rate warm-up strategy, followed by cosine annealing to gradually reduce the learning rate in order to improve convergence stability and final performance.

[0055] The loss function uses pixel-level mean squared error (MSE), which squares the difference between the reconstructed and true values ​​for all pixels in each image and then averages the results.

[0056] Among them, such as Figure 1The diagram illustrates the overall model framework of this invention. This invention establishes a deep network based on iterative unfolding to recover the original image from compressed measurements. The network comprises K iteration stages, where K is set to 10 in this embodiment. Let x be the original fully sampled image, and y be the measurement value after sampling by the sampling matrix Φ (i.e., y = Φx). In the network initialization stage, the measurement value y is first mapped back to the initial image domain using a transposed convolution operation (utilizing the transpose of Φ) to obtain an initial estimated image. Subsequently, it is boosted to the feature space through a convolutional layer to obtain the initial feature input. Simultaneously, a feature extractor extracts the initial hidden state features z_pre and z_cur from the initial image. Each iteration stage k (k = 1...K) mainly includes several core sub-modules: Multi-Branch Gradient Update Module (MBGU), Cross-Iteration Channel Attention (CICA), and Cross-Channel Attention Module (CCAM). The final model output maps the feature space back to the grayscale image space through a convolutional layer.

[0057] like Figure 2 As shown, the Multi-Branch Gradient Update Module (MBGU) performs a gradient update with "data consistency constraints" within the feature domain. Unlike traditional direct iteration in the pixel domain, this module first maps the features to the pixel domain to calculate the measurement residuals, then backprojects the residuals back to the original space and boosts them back to the feature domain. Finally, it uses a multi-branch residual block for adaptive correction, achieving a unification of "feature domain update + measurement domain consistency". Specifically: Feature → Pixel Mapping: Mapping the input features... Compressed into a single-channel image via convolution. Used in sample consistency calculations. Forward projection and residuals: using sampling operators. right Perform block convolution sampling (stride=32) to obtain the predicted measurement. and with actual measurements Find the residual Back projection and upsampling: This involves adjusting the residual... through Back projection yields the block domain residual tensor (number of channels is...) Then, the single-channel residual map is restored through subpixel rearrangement (PixelShuffle(32)). Pixel → Feature Enhancement: [This section appears to be incomplete and requires further context.] Convolution to boost back to high-dimensional feature residuals Multi-branch feature domain correction and residual summation: ... Input a multi-branch residual block, output three branches with learnable coefficients. Weighted fusion, and then with Adding them together yields the updated features. The mathematical formula is as follows:

[0058] in, Represents the convolution operator. Indicates step size is The convolution sampling operator (in this embodiment) ), This indicates the subpixel upsampling operator PixelShuffle (upsampling factor is...). ), This represents the sampling operator (a block sampling matrix consisting of a group of convolutional kernels). This represents the corresponding transpose back projection operator; For input features (feature domain representation). This is a single-channel image obtained by mapping features to the pixel domain. The predicted measurement value is obtained from the current estimate. This is the actual measured value. To measure the residual, The output is the block domain of the residual after back projection. The pixel domain residual after being restored to the original spatial resolution via PixelShuffle. The feature residuals are the pixel-domain residuals after convolution enhancement. The output features are those after a gradient update; The feature-to-pixel mapping operator has the following weights: , This represents the pixel-to-feature lifting operator, with weights of... ; This represents a multi-branch residual block (corresponding to MultiBranchResBlock), where These correspond to the three convolution branches conv1 / conv2 / conv3 in the code, The learnable scalar fusion coefficients for the three branches (initialized to) This module is used to adaptively adjust the contribution of different branches to the correction amount. It completes a gradient update with data consistency constraints in the feature domain through the chain of "feature → pixel → measurement residual → backprojection → pixel → feature → multi-branch correction → residual summation".

[0059] like Figure 3The diagram shows the overall structure of the cross-attention module in this embodiment. This module is mainly responsible for refining the features updated by gradients using an attention mechanism. It receives three inputs: the current image feature x_img, and two state feature variables—the feature state z_pre from the previous stage and the feature state z_cur from the current stage. The processing flow is as follows: First, z_pre and z_cur enter the cross-channel attention module (CICA), where the fused feature z is updated by calculating the correlation between historical and current information. This step realizes the transfer and filtering of memory information. Subsequently, the updated feature z and the current image feature x_img enter the cross-channel attention module (CCAM). This module uses the rich semantic information in feature z to guide the reconstruction of image feature x_img, focusing on restoring lost texture details. Finally, the feature is transformed and residual connected through a feedforward neural network (a convolutional layer containing a normalization layer and an activation function), outputting the final feature of this stage.

[0060] like Figure 4 The diagram shown illustrates the internal structure of the Cross-Iteration Channel Attention (CICA) module in this embodiment. The core of this module lies in capturing the temporal dependencies during the iteration process. In compressed sensing iterative solutions, features from adjacent iterations are highly correlated. The specific calculation process is as follows: the current input feature z_cur is normalized and a query vector Q is generated through convolution; the previous stage feature z_pre is normalized and a key vector K and a value vector V are generated through convolution. The formula for calculating the attention map is: Attention = Softmax(Q * K^T). The final output is: z_out = (Attention * V) + z_cur. In this way, the model can adaptively retrieve information useful for the current reconstruction from the memory of the previous stage, suppressing invalid historical interference. The mathematical formula is as follows:

[0061] like Figure 5The diagram shows the internal structure of the cross-channel attention module (CCAM) in this embodiment. This module is designed to address the lack of single-channel image feature information by introducing multi-channel deep features z to assist in reconstruction. The module contains two input branches: (1) Image branch: The input is a single-channel image feature x, which is normalized and convolutionally transformed to obtain p(x). (2) Feature branch: The input is a multi-channel feature z, which is convolutionally transformed to obtain t(z) and g(z). The calculation of cross-attention aims to mine the non-local similarity between the spatial distribution of the image and the deep channel features: Sim=t(z)*p(x)^TMap=Softmax(Sim) The generated attention map Map is used to weight and aggregate the feature g(z), and combined with position encoding to preserve spatial structure information. Finally, the aggregated features are concatenated and fused with the original image features x to output the enhanced image features. This mechanism enables the network to focus on long-distance dependencies in the image and adaptively enhance the edge and texture regions of the image based on deep semantic features. The mathematical formula is as follows:

[0062] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An image reconstruction method coupling multi-branch feature domain gradient update and attention mechanism, characterized in that, Includes the following steps: A deep unfolded compressed sensing model coupled with a multi-branch feature domain gradient update module and a cross-attention feature reconstruction module is constructed and trained. The trained deep unfolded compressed sensing model is used to reconstruct the original image to obtain the reconstructed image x^. The specific process of reconstructing the deep-unfolded compressed sensing model is as follows: First, to achieve the target compression sensing rate, a block-based sparse sampling method is used on the input original image using a convolutional sampling matrix Φ; then, the transpose of the sampling matrix Φ is used for convolution. The measurement data is back-projected to obtain the initial image. ; Next, the image will be initialized. Mapped to the feature space as input features for the first reconstruction stage. The network structure is then deployed in K reconstruction phases. Each reconstruction phase includes: a multi-branch gradient update module MBGU based on measurement consistency, and cross-iteration channel attention CICA and cross-channel attention module CCAM. The multi-branch gradient update module MBGU, cross-iteration channel attention CICA, and cross-channel attention module CCAM are connected in series to form the gradient-attention iterative module FAI. Let x be the original image after full sampling, and y be the measurement data after sampling by the sampling matrix. Let z(k-2) be the input features of the k-th stage FAI module, and z(k-2) be the cross-iteration features of the k-th stage feature reconstruction module. After K reconstruction stages, a convolution operation is performed to reduce the number of channels to 1, and the final output is the reconstructed image. .

2. The image reconstruction method according to claim 1, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: A compressed sensing image reconstruction method based on branch feature domain gradient update and cross-attention mechanism, characterized in that the calculation process of the initial image is as follows: The input original image x is divided into blocks of size 32×32, and the j-th block is denoted as . ; Use a convolutional sampling matrix corresponding to the CS ratio of compressed sensing Sparse sampling is performed on each image patch. Since each row of the sampling matrix can be regarded as a 32×32 filter kernel, convolution operation is used to perform the sampling process. The sampling formula is as follows: ； For all measurements The measured image y is obtained by stitching the images together; then the sampling matrix is ​​used. transposed convolution By back-projecting the measured value y, we obtain the initial image: ； in, For the first image in the original image A matrix of pixel values ​​for each sampling block. The first sample obtained after sampling The measured values ​​of each sampling block.

3. The image reconstruction method according to claim 2, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: For the The reconstruction phase includes data consistency updates in the feature domain and interactive refinement of hidden state features, specifically comprising the following steps: First, the output features from the previous stage Input to the Multi-Branch Gradient Update (MBGU) module, based on measurements Perform feature domain gradient correction under data consistency constraints to obtain corrected features. ; Secondly, the correction features Features of hidden states and Common input feature refinement module; Among them, the feature refinement module first uses cross-iteration channel attention CICA to... and Cross-stage information interaction and enhancement are performed to obtain the fused latent state representation; subsequently, the feature refinement module uses cross-channel attention CCAM to integrate the fused latent state representation with image features. Perform attention-guided feature fusion and refinement, and output refined features. ; Finally, the output of the MBGU is added to the output of the feature refinement module via residual connection to obtain the first... Phase Reconstruction Output ； And update the hidden state so that the hidden state of the next phase satisfies: the hidden state of the current phase... As the next stage and will from The new hidden state features extracted are used as the next stage. This enables the recursive updating of hidden states across stages and the transfer of feature memory.

4. The image reconstruction method according to claim 3, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: The multi-branch gradient update module (MBGU) performs gradient descent updates with data consistency constraints in the feature domain. The update process includes "feature-pixel mapping, measurement residual calculation, residual back projection and upsampling, feature domain lifting, and multi-branch residual correction," specifically: first, the input features are... A single-channel image is obtained by projecting it onto the pixel domain through convolution. Then, the sampling operator was used. right Perform forward sampling to obtain the predicted measurement and compare it with the actual measurement value. Calculate the measurement residual Then, the transpose sampling operator is used. For residuals Perform back projection and use the sub-pixel upsampling operator PixelShuffle to restore the block domain result to the original spatial resolution to obtain the pixel domain residual. Then The feature is corrected by mapping back to the feature domain through convolution and inputting it into a multi-branch residual block; finally, the output feature is obtained by adding the input features through residual concatenation. Its mathematical expression is as follows: ； in, Represents the convolution operator; For sampling operators, The back projection operator corresponding to its transpose; This represents the convolution projection operator that maps features to the pixel domain. This represents a convolutional mapping operator that lifts pixel-domain residuals back to the feature domain. It contains three parallel convolutional branches, whose outputs are programmed with learnable weights. Weighted fusion is performed to adaptively adjust the contribution of different branches to the feature correction amount.

5. The image reconstruction method according to claim 1, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: The Feature Refinement Module (FRM) includes a Cross-Iteration Channel Attention Module (CICA) and a Cross-Channel Attention Module (CCAM). The CICA module models the correlation between latent state features in adjacent iteration stages to achieve cross-stage information interaction and memory transfer. The CICA module uses the latent state features from the previous stage... Features of the hidden state at the current stage As input, a query vector is generated through linear mapping / convolutional mapping. Key vector AND value vector And based on channel-dimensional attention, the attention weights are calculated, and the attention weights are applied to... Perform weighted aggregation to update the hidden state features, and output the fused hidden state features. The cross-channel attention module (CCAM) is used to utilize the fused latent state features. Attention-guided reconstruction and refinement of image features; the CCAM module uses image features Features of hidden states As input, calculate the mutual attention between the two to obtain the enhanced image features. In addition, the feature refinement module also includes a feedforward network (FFN), which... A nonlinear transformation is performed and added to its residual to further enhance the feature representation ability, thus obtaining the predicted features. Its mathematical expression is as follows: ； in, This represents a feedforward neural network, which includes forward and backward convolutional paths. It is used to perform channel transformation and nonlinear mapping on input features, thereby enhancing the feature refinement effect.

6. The image reconstruction method according to claim 5, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: The cross-iteration channel attention module (CICA) normalizes the query features and key features and calculates channel attention weights to achieve correlation modeling and information interaction of latent state features in adjacent iteration stages. Its specific calculation formula is as follows: ； in, Generated by convolutional mapping of input latent state features; This is a normalization operation; Used to generate the attention weight matrix; For convolution mapping; Features of the hidden state at the current stage; The updated hidden state features output by the CICA module; the above Residual connections are used to preserve the hidden state information of the current stage and enhance training stability; The cross-channel attention module (CCAM) is used to guide the reconstruction and refinement of image features using latent state features. It achieves cross-channel information fusion by calculating the cross-similarity between latent state features and image features and generating an attention map. The specific calculation formula is as follows: ； in, For the input image features, Input hidden state features (can be output from the CICA module); and For the convolutional mapping operator used to generate mutual attention, A convolution operator that performs feature transformation on hidden state features. A convolution operator for channel alignment of attention aggregation results. Output the merged convolutional operator; This is a normalization operation; This is a location encoding used to inject spatial location information; Indicates feature concatenation operation; This is a similarity matrix. For attention mapping, This refers to the refined features output by the CCAM module.

7. The image reconstruction method according to claim 5, which couples multi-branch feature domain gradient update and attention mechanism, is characterized in that: When training the deep unfolded compressed sensing model, the input training images were cropped into 89,600 sub-images and data augmentation was performed to expand the training samples and improve the model's generalization ability. Regarding network parameter settings, the reconstruction block size was set to 32, and the batch size was set to 16. The Adam optimizer was used to iteratively update the network parameters, and training was performed using a cosine annealing learning rate strategy for a total of 80 epochs. The initial learning rate was... The warm-up rounds consist of 2 epochs; during training, mean squared error is used as the loss function to constrain the difference between the reconstructed image and the fully sampled real image. The loss function is defined as: ； in, Indicates network parameters; Indicates the first A fully sampled real image; This represents the corresponding compressed sensing reconstructed image; This indicates the number of training samples.

Images