Residual non-local scarcity attention based undersampled MRI reconstruction method

Abstract

The application discloses a residual non-local sparse attention-based undersampling MRI reconstruction method, which is applied to a U-net, original features are input into the U-net to predict a residual image, and then the original image and the residual image are added to obtain a reconstructed image; the U-net comprises an encoder and a decoder, B1-B3 of the encoder are residual non-local sparse attention blocks, and B4-B7 of the encoder are all convolution blocks; J7-J4 of the decoder are combinations of convolution blocks and inverse convolution blocks, and J3-J1 of the decoder are combinations of residual non-local sparse attention blocks and inverse convolution blocks. The residual connection and the sparse attention mechanism are combined to improve the quality and efficiency of undersampling MRI reconstruction, the residual connection can add the input and output of the network to make the model converge faster and reduce overfitting, and the sparse attention mechanism can weight and fuse different parts of the input signal by calculating the similarity between a small part and different positions of the current position.

Description

Technical Field

[0001] This invention relates to the field of magnetic resonance imaging (MRI) technology, and in particular to an undersampling MRI reconstruction method based on residual nonlocal scarce attention. Background Technology

[0002] MRI is an imaging technique that uses strong magnetic fields and specific radiofrequency pulse sequences to obtain detailed images of human tissues or organs. It has the advantages of being non-invasive, having no ionizing radiation, and providing rich contrast information, and can provide high-resolution structural and functional information for clinical diagnosis.

[0003] MRI requires acquiring 2D or 3D data matrices (k-domain data) in a Cartesian coordinate system. After inverse Fourier transform, the corresponding image domain information can be obtained. Each row of the k-domain data corresponds to a phase encoding step. Acquiring different phase-encoded data requires waiting for the recovery of longitudinal magnetization, i.e., a waiting time (TR). Depending on the MRI imaging method, the TR varies from hundreds of milliseconds to several seconds. Therefore, MRI imaging is time-consuming. Clear MRI imaging requires the imaging site to remain stationary; otherwise, motion artifacts are easily introduced into the image. Patients need to maintain a single position during the examination, remaining still for one to several minutes. This places high demands on patient compliance and cooperation. Therefore, since the birth of MRI, rapid MRI technology has been a hot research topic.

[0004] Undersampling MRI reconstruction is a technique that enables rapid MRI scanning by deleting some sampling points from the original MRI sampling data. It has the advantages of reducing imaging time and improving imaging efficiency. However, because the amount of data acquired by undersampling is reduced, the signal-to-noise ratio is lowered, which makes the reconstructed image more prone to blurring and distortion. Therefore, some algorithms are needed to recover the lost information and reconstruct a clear and accurate image. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an undersampling MRI reconstruction method based on residual nonlocal scarce attention. When applied to undersampling MRI reconstruction, it has good reconstruction quality and high computational efficiency.

[0006] The technical solution of this invention is: an undersampling MRI reconstruction method based on residual nonlocal sparse attention, applied to U-net; U-net includes an encoder and a decoder, each consisting of 8 layers, each layer of the encoder and decoder following a set of normalized and corrected linear units; the 8 layers of the encoder are named B1, B2...B7, BJ in order from front to back, and the 8 layers of the decoder are named BJ, J7, J6...J1 in order from front to back, wherein BJ of the encoder and BJ of the decoder are shared layers used for matrix transformation; B1 to B3 of the encoder are all residual nonlocal sparse attention blocks, and B4 to B7 of the encoder are all convolutional blocks; J7 to J4 of the decoder are combinations of convolutional and deconvolutional blocks, and J3 to J1 of the decoder are combinations of residual nonlocal sparse attention blocks and deconvolutional blocks; the transfer of features from the encoder to the decoder is achieved through skip connections between the encoder and the decoder; The method is as follows: S01, downsampled MRI is transformed into matrix data through convolution processing. The matrix data is the feature. The feature is input into U-net. In U-net, the feature is first upsampled by the encoder and then downsampled by the decoder. S02, the features output by the decoder are restored to the size of the original features through matrix transformation, and then the number of channels of the restored features is reduced to 1 through matrix transformation to obtain the predicted features. Finally, the residual between the original features and the predicted features is obtained to obtain the undersampled MRI reconstructed image. In this step, the original features are the features obtained by convolution processing in downsampled MRI.

[0007] A further technical solution of the present invention is: the skip connection is used to connect any convolutional layer B of the encoder. i The features are passed to the corresponding convolutional layer J of the decoder. i Where i ranges from 1 to 7, representing the number of convolutional layers; based on skip connections, the decoder achieves feature upsampling between the corresponding encoder layer and the lower-level decoder layer at any given layer; the current decoder layer is Ji, J i+1 ~BJ is J i All front-end layers, J i-1 ~J1 is J i The back-end layers are all layers of the decoder, and the lower-level layers of the decoder are all layers of the front-end of the current layer of the decoder.

[0008] A further technical solution of the present invention is: the kernel size of the convolution block is 4, the stride is 2, the convolution block uses convolution for downsampling, and there is no separate max pooling layer.

[0009] A further technical solution of the present invention is: the residual nonlocal sparse attention block consists of a 3×3 convolutional layer, five residual convolutional blocks and a nonlocal sparse attention block; The implementation process of the residual nonlocal sparse attention block is as follows: After the feature enters the residual nonlocal sparse attention block, it sequentially enters a 3×3 convolutional layer, a residual convolutional block, and another residual convolutional block to extract feature x1. Feature x1 then passes through a nonlocal sparse attention block, a residual convolutional block, and another residual convolutional block to extract feature x2. Feature x1 and feature x2 are then fused to obtain feature x3. Feature x3 then passes through a residual convolutional block to complete all feature extraction.

[0010] A further technical solution of the present invention is that the residual convolutional block consists of two 3×3 convolutional layers, one calibration linear unit, and one residual scaling layer; The implementation process of the residual convolutional block is as follows: After feature a1 enters the residual convolutional block, it passes through a 3×3 convolutional layer, a calibration linear unit, a 3×3 convolutional layer, and a residual scaling layer in sequence, and then extracts feature a2. Then, feature a1 and feature a2 are fused to obtain feature a3, which is the feature output by the residual convolutional block.

[0011] A further technical solution of the present invention is: the nonlocal sparse attention block uses local sensitive hashing to calculate attention, and divides the embedding space into attention buckets with similar features according to the angular distance of the embedding space; the index set indicates the pixel position group that should be focused on for a given query, and the similar position group to be focused on is defined in an attention bucket; The implementation process of the nonlocal sparse attention block is as follows: A. The input feature is a 3D tensor with dimensions h, w, and c, where h and w are the width and height of the block, and c is the number of channels of the input feature. B. Calculate the bucket values ​​of the input features using Locality Sensitive Hashing (LSH) and perform block partitioning; the calculation and partitioning process is as follows: B1. Project the 3D tensor of the input features onto a hypersphere, and then use a matrix... Rotate it randomly, the matrix This is a random selection matrix obtained by sampling the iid Gaussian terms, and the result after rotation is: In the matrix, m represents the desired number of hash buckets, R is the real number field, c is the number of channels for the input features, and in the Gaussian term, x is the value of the tensor at the corresponding specific location; B2, based on the calculated Gaussian term, the hash bucket is defined as: B3, after hashing all pixels in the 3D tensor, divides each element into buckets of its related elements, and the attention bucket of any element xi can be obtained through the index set. To identify; the specific identification method is: to identify the bucket value of xi. Compare the bucket values ​​with all other elements to find the bucket value that is closest. B4, thus dividing xi and xj into one attention bucket; B5, sorting the features after dividing the attention buckets, defining the sorting as... After knowing the new positions of the features after dividing the attention buckets, the features after dividing the attention buckets are divided into blocks of size k. Therefore, the attention buckets of xi are updated to the corresponding blocks. C. Calculate the attention for each query within each bucket or between adjacent buckets, and then sum the features of all buckets based on the attention to obtain the final features.

[0012] Compared with the prior art, the present invention has the following advantages: 1. It combines residual connections and non-local sparse attention mechanisms to improve the quality and efficiency of undersampled MRI reconstruction. Residual connections can add the network's input and output, allowing the model to converge faster and reducing overfitting. Non-local sparse attention mechanisms weight and fuse different parts of the input signal by calculating the similarity between different locations in a small subset of the input signal related to the current location. This reduces computational complexity and avoids unnecessary noise and interference, allowing the network to better understand the overall structure of the input signal and improve the network's performance and accuracy.

[0013] 2. It employs a sparse non-local attention mechanism and a deep convolutional neural network (U-net). On the one hand, it can better utilize the non-local information in the image to improve the image reconstruction effect and make the reconstructed image clearer and more accurate. On the other hand, it can adaptively select image patches with high correlation for reconstruction to further improve the reconstruction quality.

[0014] 3. It employs a sparse non-local attention mechanism and a deep convolutional neural network (U-net), which can improve the robustness of the algorithm, reduce the impact of noise, and make the algorithm more adaptable and stable to different datasets and different noise levels.

[0015] 4. It adopts sparse coding technology and multi-scale feature extraction method. On the one hand, it can adaptively process features of different scales, thereby reconstructing multi-scale data more effectively. On the other hand, it can reduce the amount of computation and improve the reconstruction speed. In particular, for the processing of large image data, it can significantly shorten the reconstruction time.

[0016] 5. In U-Net, skip connections are introduced into the encoder-decoder architecture to fuse features from convolutional and deconvolutional layers. This compensates for information lost during downsampling, helping to recover fine-grained details and improve reconstruction quality. The motivation for skip connections is that they provide continuous gradient transfer within the network, addressing the vanishing gradient problem. In residual networks (all residual operations involved in U-Net can be called residual networks), short-duration skip connections allow features to be reused, stabilizing gradient updates in U-Net and making training more stable.

[0017] 6. In U-net, the residual nonlocal sparse attention block is a plug-and-play module that can be easily incorporated into other existing deep neural networks without introducing any additional inference cost.

[0018] The present invention will be further described below with reference to the figures and embodiments. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of the deep convolutional neural network U-net; Figure 2 This is a schematic diagram of the structure of a residual nonlocal sparse attention block. Detailed Implementation

[0020] Example 1: An undersampling MRI reconstruction method based on residual nonlocal scarce attention is applied to a deep convolutional neural network U-net. The original features are input into U-net to predict the residual image, and then the original image and the residual image are added together to obtain the reconstructed image.

[0021] U-net comprises an encoder and a decoder, each consisting of eight layers. Each layer in both the encoder and decoder follows a set of normalized and corrected linear units. The eight layers of the encoder are named B1, B2...B7, BJ from front to back, and the eight layers of the decoder are named BJ, J7, J6...J1 from front to back. The BJ layer in both the encoder and decoder is a shared layer used for matrix transformations. Encoder layers B1-B3 are residual nonlocal sparse attention blocks, and layers B4-B7 are convolutional blocks. Decoder layers J7-J4 are combinations of convolutional and deconvolutional blocks, and layers J3-J1 are combinations of residual nonlocal sparse attention blocks and deconvolutional blocks. Feature transfer between the encoder and decoder is achieved through skip connections. The U-net network architecture effectively extracts and reconstructs feature information from MRI, thereby improving reconstruction quality and accuracy.

[0022] The method is as follows: S01, downsampled MRI is converted into matrix data through convolution processing. The matrix data is the feature. The feature is input into U-net. In U-net, the feature is first upsampled by the encoder and then downsampled by the decoder.

[0023] S02, the features output by the decoder are restored to the size of the original features through matrix transformation, and then the number of channels of the restored features is reduced to 1 through matrix transformation, thus obtaining the predicted features. Finally, the residual between the original features (the original features are the features obtained by convolution processing in downsampled MRI) and the predicted features is calculated to obtain the undersampled MRI reconstructed image.

[0024] Regarding jump links: The skip connection is used to connect any layer B of the encoder. i The features are passed to the corresponding layer J of the decoder. i Where i takes values ​​from 1 to 7, and is used to represent the number of layers, such as Figure 1 As shown, the top layers of the encoder and decoder are B1 and J1, respectively, with the layer numbers increasing sequentially downwards. Based on skip connections, the decoder enables feature upsampling between the corresponding layer of the encoder and the lower-level layers of the decoder at any given layer. The current layer of the decoder is Ji, J... i+1 ~BJ is J i All front-end layers, J i-1 ~J1 is J i The backend layers of the decoder, and the lower-level layers of the decoder are all the frontend layers of the current decoder layer. See [link / reference] Figure 1 Assuming the current layer of the decoder is J4, then J5, J6, J7, and BJ are all the front-end layers of J4, and J3, J2, and J1 are all the back-end layers of J4. Therefore, the lower-level layers of the decoder corresponding to J4 are J5, J6, J7, and BJ.

[0025] Regarding convolutional blocks: The convolutional blocks have a kernel size of 4 and a stride of 2. These blocks use convolution for downsampling and do not have a separate max-pooling layer. The different convolutional sizes in each block facilitate the extraction of features at different resolutions.

[0026] Note the following regarding residual nonlocal sparsity: See Figure 2 The residual nonlocal sparse attention block consists of a 3×3 convolutional layer (Conv), five residual convolutional blocks, and a nonlocal sparse attention block (NLSA). This block construction method can learn information from the spatial domain, capture short-term interactions by modeling local details, and capture long-term interactions by connecting them to the global context. Although stacking more residual convolutional blocks (ResBlocks) can expand the effective receptive field, it also increases computational complexity, and the receptive field of the initial layer is still limited. Therefore, integrating five residual convolutional blocks can effectively balance the effective receptive field and computational complexity.

[0027] The implementation process of the residual nonlocal sparse attention block is as follows: After the feature enters the residual nonlocal sparse attention block, it sequentially enters a 3×3 convolutional layer, a residual convolutional block, and another residual convolutional block to extract feature x1. Feature x1 then sequentially passes through a nonlocal sparse attention block, a residual convolutional block, and another residual convolutional block to extract feature x2. Then, feature x1 and feature x2 are fused to obtain feature x3. Feature x3 then passes through a residual convolutional block to complete all feature extraction.

[0028] Regarding residual convolution blocks: The residual convolutional block consists of two 3×3 convolutional layers, a correction linear unit, and a residual scaling layer. It excludes max pooling and batch normalization layers, as max pooling can miss image details and thus affect the quality of image restoration.

[0029] The implementation process of the residual convolutional block is as follows: After feature a1 enters the residual convolutional block, it passes through a 3×3 convolutional layer, a calibration linear unit, a 3×3 convolutional layer, and a residual scaling layer in sequence, and then extracts feature a2. Then, feature a1 and feature a2 are fused to obtain feature a3, which is the feature output by the residual convolutional block.

[0030] Regarding nonlocal sparse attention blocks: The nonlocal sparse attention block uses locality-sensitive hashing to compute attention. It divides the embedding space into attention buckets with similar characteristics based on angular distance within the embedding space. Therefore, even if the attention only spans one bucket and remains sparse, it can still capture most relevant elements. The set of indexes indicates the group of pixel locations that should be focused on for a given query, and the groups of similar locations to be focused on are defined in an attention bucket.

[0031] The implementation process of the nonlocal sparse attention block is as follows: A. The input feature is a 3D tensor with dimensions h, w, and c, where h and w are the width and height of the block, and c is the number of channels of the input feature.

[0032] B. Calculate the bucket values ​​of the input features using Locality Sensitive Hashing (LSH) and perform block partitioning; the calculation and partitioning process is as follows: B1. Project the 3D tensor of the input features onto a hypersphere, and then use a matrix... Rotate it randomly, the matrix This is a random selection matrix obtained by sampling the iid Gaussian terms, and the result after rotation is: In the matrix, m represents the desired number of hash buckets, R is the real number field, c is the number of channels for the input features, and in the Gaussian term, x is the value of the tensor at the corresponding specific location; B2, based on the calculated Gaussian term, the hash bucket is defined as: B3, after hashing all pixels in the 3D tensor, divides each element into buckets of its related elements, and the attention bucket of any element xi can be obtained through the index set. To identify; the specific identification method is: to identify the bucket value of xi. Compare the bucket values ​​with all other elements to find the bucket value that is closest. B4, thus dividing xi and xj into one attention bucket; B5, sorting the features after dividing the attention buckets, defining the sorting as... After knowing the new positions of the features after dividing the attention buckets, the features after dividing the attention buckets are divided into blocks of size k. Therefore, the attention buckets of xi are updated to the corresponding blocks.

[0033] C. Calculate the attention for each query within each bucket or between adjacent buckets, and then sum the features of all buckets based on the attention to obtain the final features.

Claims

1. An undersampling MRI reconstruction method based on residual nonlocal scarce attention, characterized by: This method is applied to U-net. U-net consists of an encoder and a decoder, each composed of 8 layers. Each layer of the encoder and decoder follows a set of normalized and corrected linear units. The 8 layers of the encoder are named B1, B2...B7, BJ in order from front to back, and the 8 layers of the decoder are named BJ, J7, J6...J1 in order from front to back. Among them, BJ of the encoder and BJ of the decoder are shared layers used for matrix transformation. B1 to B3 of the encoder are residual nonlocal sparse attention blocks, and B4 to B7 of the encoder are convolutional blocks. J7 to J4 of the decoder are combinations of convolutional and deconvolutional blocks, and J3 to J1 of the decoder are combinations of residual nonlocal sparse attention blocks and deconvolutional blocks. Features are transferred from the encoder to the decoder through skip connections. The skip connection is used to connect any convolutional layer B of the encoder. i The features are passed to the corresponding convolutional layer J of the decoder. i Where i ranges from 1 to 7, representing the number of convolutional layers; based on skip connections, the decoder achieves feature upsampling between the corresponding encoder layer and the lower-level decoder layer at any given layer; the current decoder layer is Ji, J i+1 ~BJ is J i All front-end layers, J i-1 ~J1 is J i The back-end layers of the decoder are all layers at the bottom level, and the decoder's lower-level layers are all layers at the front level of the current decoder layer. The residual nonlocal sparse attention block consists of a 3×3 convolutional layer, five residual convolutional blocks, and one nonlocal sparse attention block. The implementation process of the residual nonlocal sparse attention block is as follows: After a feature enters the residual nonlocal sparse attention block, it sequentially enters a 3×3 convolutional layer, a residual convolutional block, and another residual convolutional block to extract feature x1. Feature x1 then sequentially passes through a nonlocal sparse attention block, a residual convolutional block, and another residual convolutional block to extract feature x2. Then, feature x1 and feature x2 are fused to obtain feature x3. Feature x3 then passes through a residual convolutional block to complete all feature extraction. The method is as follows: S01, downsampled MRI is transformed into matrix data through convolution processing. The matrix data is the feature. The feature is input into U-net. In U-net, the feature is first upsampled by the encoder and then downsampled by the decoder. S02, the features output by the decoder are restored to the size of the original features through matrix transformation, and then the number of channels of the restored features is reduced to 1 through matrix transformation to obtain the predicted features. Finally, the residual between the original features and the predicted features is obtained to obtain the undersampled MRI reconstructed image. In this step, the original features are the features obtained by convolution processing in downsampled MRI.

2. The undersampling MRI reconstruction method based on residual nonlocal scarce attention as described in claim 1, characterized in that: The convolutional block has a kernel size of 4 and a stride of 2. The convolutional block uses convolution for downsampling and does not have a separate max pooling layer.

3. The undersampling MRI reconstruction method based on residual nonlocal scarce attention as described in claim 2, characterized in that: The residual convolutional block consists of two 3×3 convolutional layers, one corrected linear unit, and one residual scaling layer. The implementation process of the residual convolutional block is as follows: After feature a1 enters the residual convolutional block, it passes through a 3×3 convolutional layer, a calibration linear unit, a 3×3 convolutional layer, and a residual scaling layer in sequence, and then extracts feature a2. Then, feature a1 and feature a2 are fused to obtain feature a3, which is the feature output by the residual convolutional block.

4. The undersampling MRI reconstruction method based on residual nonlocal scarce attention as described in claim 3, characterized in that: The nonlocal sparse attention block uses local sensitive hashing to calculate attention and divides the embedding space into attention buckets with similar characteristics based on the angular distance of the embedding space. The index set indicates the group of pixel locations that should be focused on for a given query, and the similar locations that should be focused on are defined in an attention bucket; The implementation process of the nonlocal sparse attention block is as follows: A. The input feature is a 3D tensor with dimensions h, w, and c, where h and w are the width and height of the block, and c is the number of channels of the input feature. B. Calculate the bucket values ​​of the input features using Locality Sensitive Hashing (LSH) and perform block partitioning; the calculation and partitioning process is as follows: B1. Project the 3D tensor of the input features onto a hypersphere, and then use a matrix... Rotate it randomly, the matrix This is a random selection matrix obtained by sampling the iid Gaussian terms, and the result after rotation is: In the matrix, m represents the desired number of hash buckets, R is the real number field, c is the number of channels for the input features, and in the Gaussian term, x is the value of the tensor at the corresponding specific location; B2, based on the calculated Gaussian term, the hash bucket is defined as: B3, after hashing all pixels in the 3D tensor, divides each element into buckets of its related elements, and the attention bucket of any element xi can be obtained through the index set. To identify; the specific identification method is: to identify the bucket value of xi. Compare the bucket values ​​with all other elements to find the bucket value that is closest. This allows xi and xj to be grouped into a single attention bucket; B4, sort the features after dividing them into attention buckets, and define the sorting as... After knowing the new positions of the features after dividing the attention buckets, the features after dividing the attention buckets are divided into blocks of size k. Therefore, the attention buckets of xi are updated to the corresponding blocks. C. Calculate the attention for each query within each bucket or between adjacent buckets, and then sum the features of all buckets based on the attention to obtain the final features.

Images