Reconstruction Method of High-fold Undersampled Magnetic Resonance Images Based on ICU-Net

Abstract

A magnetic resonance image reconstruction method based on high-fold undersampling of ICU-Net, comprising the following steps: Step 1, input the u-fold undersampled magnetic resonance image #imgabs0# Step 2, use ICU-Net f ICU (·; Θ) to obtain the reconstructed magnetic resonance image #imgabs1# Figure 1 shows an example of ICU-Net. The ICU-Net proposed in the present invention combines dense connections, residual connections, feature map splicing, data consistency principles, etc. with the U-Net network structure, which can greatly save the parameter scale of the U-Net, making it applicable to magnetic resonance image training samples with a small scale, and more effectively reconstruct the missing detail information in the high-fold undersampled magnetic resonance image, improving the reconstruction quality of the magnetic resonance image.

Description

Technical Field

[0001] The present invention relates to the technical field of magnetic resonance imaging, and is mainly used to effectively reconstruct highly undersampled magnetic resonance images. Background Art

[0002] Magnetic Resonance Imaging (MRI) is currently one of the most important non-invasive examination methods in clinical medical imaging. It is another major advancement in the imaging industry after CT. The imaging quality is superior to B-ultrasound and CT, and it does not produce ionizing radiation like CT and other imaging methods. It has the advantages of temperature sensitivity, multi-directional, multi-parameter, multi-modal, and multi-plane imaging capabilities. It can not only display the anatomical information of human tissues but also display functional information. Currently, MRI has been widely applied in various clinical applications such as angiography, neurology, and cardiology examinations, and has been widely recognized in interventional therapy, and is known as the pearl on the crown of modern medical imaging technology.

[0003] However, the two major problems that have persisted in the MRI field for many years are still the overly slow imaging speed and relatively high dielectric artifacts. The long scanning time and slow imaging speed of MRI will cause the following problems: (1) It causes additional pain to patients, especially for patients with "claustrophobia"; (2) Involuntary movements of organs (such as involuntary movements like breathing, blinking, and swallowing) are more likely to cause image blurring and increase artifacts; (3) It cannot meet the needs of dynamic real-time imaging and surgical navigation.

[0004] How to improve the imaging speed of MRI has always been the goal pursued by those in the magnetic resonance field. Currently, there are mainly three methods for accelerating MRI: Parallel MRI (PMRI), Compressed Sensing MRI (CS-MRI), and Deep Neural Network MRI (DNN-MRI). PMRI techniques (such as SENSE, GRAPPA, etc.) use multiple coils to image simultaneously. However, due to the use of undersampling, the signal-to-noise ratio of the reconstructed image will decrease as the acceleration factor increases; CS-MRI techniques (such as BM3D-MRI, CS-SENSE, etc.) are restricted by factors such as data acquisition, spatial transformation, reconstruction algorithms, and processing systems, and have not been widely applied in the magnetic resonance field for more than a decade.

[0005] In recent years, with the rapid development of deep learning theory and high-performance computing, DNN-MRI technologies (such as DeepCascadeNet, ConvRNN, PC-RNN, etc.) have achieved breakthrough improvements in both reconstruction quality and speed. U-Net is a deep convolutional network structure widely used in existing DNN-MRI technologies. However, U-Net has the following problems: (1) The parameter scale is often relatively large (hundreds of megabytes at every turn) and is usually deployed on high-performance servers. If it is to be deployed on local MRI devices, a lightweight network structure and a matching sequence are required; (2) A large amount of training data is needed. However, due to the limitation of patient privacy protection, it is often not easy to obtain a large amount of training data; (3) For high-fold undersampled MRI (such as when the acceleration factor ≥ 8), the DNN-MRI technology based on UNet usually produces relatively significant aliasing artifacts, which limits its widespread use in practice. Summary of the Invention

[0006] To solve the three problems existing in U-Net, the present invention proposes a new network structure based on the idea of dense connection: Intensively Connected U-Net (ICU-Net), which improves U-Net. ICU-Net can greatly save the parameter scale of U-Net, more effectively learn the lost detail information in undersampled MRI images, and improve the reconstruction quality of MRI images.

[0007] The present invention provides the following technical solutions:

[0008] A method for reconstructing high-fold undersampled magnetic resonance images based on ICU-Net, comprising the following steps:

[0009] Step 1: Input the u-fold undersampled MRI image represents the pixel space of the MRI image, N is the dimension of x u of, x u is the MRI image obtained by zero-padding and inverse Fourier transform of the u-fold undersampled MRI image in the k-space after, represents the k-space of the MRI image;

[0010] Step 2: Obtain the reconstructed MRI image from ICU-Net f ICU (·; Θ)

[0011] Further, in the above step 1, xu Obtained by the following method:

[0012]

[0013] Wherein, represents the fully sampled image in k-space, represents the fully sampled image in pixel space, is the Fourier transform matrix, and H represents the Hermitian transpose operation, which is used to transform the MRI image from k-space to pixel space; M u ∈{0,1} M×N represents the mask matrix for u-fold undersampling of y.

[0014] Furthermore, in the step 2, ICU-Net f ICU (·; Θ) consists of an encoding module E(·; Θ E ) and a decoding module D(·; Θ D ), and the specific definitions are as follows:

[0015] f ICU (x u ; Θ) = F H f DC (D(E(x u ; Θ E ); Θ D ), x u ) (2)

[0016] Wherein, Θ = {Θ E , Θ D}, f DC (·, x u ) represents data consistency calculation, E(·; Θ E ) is the encoding module of f ICU , D(·; Θ D ) is the decoding module of f ICU , and Θ E and Θ D are the parameter sets of the encoding module and the decoding module respectively.

[0017] Specifically, the encoding module E(·; Θ E ) in formula (2) consists of K encoding subnets, and is defined as:

[0018]

[0019] Wherein, K is the depth of the encoding module, is defined as follows:

[0020]

[0021] Here, is a convolution operation, and its parameters; is the residual dense block RDB, and its parameters are defined as follows:

[0022]

[0023] where z 0 is the input feature map of f RDB , z T is the feature map output by the last convolutional layer of f RDB , z T is calculated from z 0 and the feature maps z RDB output by the first T - 1 convolutional layers of f 1 , …, z T-1 , and T is the depth of the residual dense block; The feature map z t can be recursively calculated as follows:

[0024]

[0025] where [·, ·, …, ·] represents the concatenation operation, that is: concatenating the participating feature maps in the channel direction; is a convolution operation, and its parameters;

[0026] The decoding module D(·; Θ D ) in formula (2) consists of K decoding subnets and is defined as:

[0027]

[0028] where, is defined as follows:

[0029]

[0030] Here, is a transposed convolution operation, and its parameters;

[0031] The data consistency function f DC (·, x u ) in formula (2) is defined as follows: Let The elements in are calculated as follows:

[0032]

[0033] Among them, Ω represents the index subset of the elements in constructing y u = M u y = Fx(y u The relationship between y and x and y is shown in formula (1)).

[0034] Furthermore, in step 2, the parameter set Θ in ICU-Netf ICU (·; Θ) is obtained by training through existing deep learning optimization algorithms.

[0035] The technical concept of the present invention is as follows: U-Net is a network structure widely used in existing deep learning-based fast magnetic resonance imaging methods. However, the U-Net model usually requires a large storage space and is difficult to effectively reconstruct highly undersampled MRI images. For highly undersampled MRI images, the shallow features generated by the network are particularly important. The idea of dense connection has been successfully applied in fields such as natural image recognition and super-resolution reconstruction. It can not only effectively compress the scale of network parameters but also make full use of the underlying features of the network to enhance the non-linear reconstruction ability of high-level features. Therefore, based on the idea of dense connection, the present invention proposes a new network structure: Intensively Connected U-Net (ICU-Net). ICU-Net makes three improvements to U-Net: (1) Based on dense connection and residual connection, a Residual Dense Block (RDB) is constructed, and the convolutional modules inside the encoding module and decoding module in U-Net are replaced with RDB to enhance the feature learning ability in the encoding and decoding stages; (2) The combination of residual connection and feature map concatenation is used to replace the original feature map concatenation in U-Net to strengthen the connection from the encoding module to the decoding module; (3) The combination of residual connection and data consistency connection is used to connect the input end and output end of U-Net to strengthen the end-to-end connection of U-Net. ICU-Net can greatly save the parameter scale of U-Net, more effectively learn the lost detail information in undersampled MRI images, and improve the reconstruction quality of MRI images.

[0036] Compared with existing fast magnetic resonance imaging algorithms based on deep learning, the beneficial effects of the present invention are mainly reflected in: (1) Since ICU-Net uses enhanced dense connections, it can effectively reuse features, greatly reducing the scale of network parameters, and thus being better applicable to the fast reconstruction of MRI images with a small-scale training sample; (2) ICU-Net can effectively utilize the features generated by the low-level modules of the network, effectively improving the reconstruction effect of highly undersampled magnetic resonance images. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is the network structure diagram of ICU-Net proposed by the present invention. Among them, Conv represents a convolution operation with a convolution kernel size greater than 1×1, 1×1Conv represents a convolution operation with a convolution kernel size of 1×1, Sum represents a summation operation, Cat represents a feature map concatenation operation (see details in Figure 6 ), ConvDn represents a convolution operation with a stride of 2, and Deconv is a deconvolution operation.

[0038] Figure 2 is the structure diagram of the module. Among them, (a) is the structure diagram of the Convolutional Block (CB), Figure 2 (b) is the structure diagram of one of the core modules of ICU-Net, the Residual Dense Block (RDB).

[0039] Figure 3 is the network structure diagram. Among them, (a) is the network structure diagram of U-Net, Figure 3 (b) is the network structure diagram of RDU-Net.

[0040] Figure 4 is an example of an MRI image and a fully sampled MRI image. Among them, (a) and (b) are examples of an 8-fold undersampled MRI image and a fully sampled MRI image in the k-space, respectively, and (c) and (d) are examples of an MRI image obtained by the zero-padding method and a fully sampled MRI image in the pixel space, respectively.

[0041] Figure 5 is an example of a mask matrix. Among them, (a) is an example of a central undersampling mask matrix, and (b) is an example of a Gaussian undersampling mask matrix.

[0042] Figure 6 is a schematic diagram of the feature map concatenation operation, that is, the participating feature maps are concatenated in the channel direction.

[0043] Figure 7 is for the data consistency operator f DCExplanation: Subfigure (a) is the input undersampled MRI image, subfigure (b) is the reconstructed MRI image, and subfigure (c) is the MRI image after performing data consistency calculation on subfigure (b) using the information in subfigure (a). Essentially, the data consistency operator replaces the information in the corresponding frequency band of the reconstructed MRI image with the frequency information (mainly low-frequency information) of the input undersampled MRI image.

[0044] Figure 8 gives the MR training image set example: The first row is an example of an undersampled MRI image example, and the second row is an example of the fully sampled MRI image x g example.

[0045] Figure 9 are the reconstruction effect diagrams of 8-fold undersampled MRI images by various methods: (a) fully sampled MRI image; (b) MRI image reconstructed by the zero-filling method; (c) MRI image reconstructed by U-Net; (d) MRI image reconstructed by RDU-Net; (e) MRI image reconstructed by ICU-Net.

[0046] Figure 10 are the reconstruction effect diagrams of 16-fold undersampled MRI images by various methods: (a) fully sampled MRI image; (b) MRI image reconstructed by the zero-filling method; (c) MRI image reconstructed by U-Net; (d) MRI image reconstructed by RDU-Net; (e) MRI image reconstructed by ICU-Net. Detailed implementation manner

[0047] The implementation steps of the present invention are further described below with reference to the accompanying drawings.

[0048] Refer to Figures 1 to 10 , a fast magnetic resonance imaging method based on ICU-Net, includes the following steps:

[0049] Step 1: Input a u-fold undersampled MRI image

[0050] Step 2: Obtain the reconstructed MRI image from ICU-Netf ICU (·; Θ) Figure 1 gives the specific network structure diagram of ICU-Netf ICU (·; Θ). For comparison, Figure 3(a) and (b) respectively show the network structure diagrams of U-Net and RDU-Net. Here, RDU-Net refers to Residual Dense U-Net. Compared with U-Net, RDU-Net only replaces the Convolutional Block (CB) in U-Net with a Residual Dense Block (RDB). The structures of CB and RDB are shown in detail in Figure 2 (a) and (b).

[0051] In the said step 1, represents the pixel space, N is the dimension of x u x u is the MRI image that is u-fold undersampled in the k-space (represented by ), and after zero-padding and inverse Fourier transform, the obtained MRI image is obtained through the following method:

[0052]

[0053] where, represents the fully sampled image in the k-space, represents the fully sampled image in the pixel space, is the Fourier transform matrix, H represents the Hermitian transpose operation, which is used to transform the MRI image from the k-space to the pixel space; M u ∈{0,1} M×N represents the mask matrix for u-fold undersampling of y. Figure 4 (a) to (d) respectively show the image examples of y u , y, x u , x; the mask matrix M u can use a central undersampling matrix or a Gaussian undersampling matrix, as shown in Figure 5 shown, Figure 4 (a) is the MRI image in the k-space obtained by central undersampling of Figure 4 (b).

[0054] In the said step 2, ICU-Netf ICU (·; Θ) consists of an encoding module E(·; Θ E ) and a decoding module D(·; Θ D ), and is defined as follows:

[0055] f ICU (x u ; Θ) = F H f DC (D(E(x​u ; Θ E ); Θ D ), x u ) (4)

[0056] Among them, Θ = {Θ E , Θ D}}, f DC (·, x u ) represents data consistency calculation, E(·; Θ E ) is the encoding module of f ICU , D(·; Θ D ) is the decoding module of f ICU , and Θ E and Θ D are the parameter sets of the encoding module and the decoding module respectively.

[0057] Specifically, the encoding module E(·; Θ E ) in formula (4) is composed of K encoding subnets, which is defined as:

[0058]

[0059] Among them, K is the depth of the encoding module, is defined as follows:

[0060]

[0061] Here, is the convolution operation, is its parameter; is the residual dense block (RDB), is its parameter, which is defined as follows:

[0062]

[0063] Among them, z 0 is the input feature map of f RDB , z T is the feature map output by the last convolution layer of f RDB , z T is calculated from z 0 and the feature maps z RDB output by the first T - 1 convolution layers of f 1 , …, z T-1 , T is the depth of the residual dense block. In specific implementation, the value of T can be set to 4 or 5. In order to compress the parameter scale of the network model, the last convolution layer of f RDB uses a 1×1 convolution. For the specific design of f RDB , please refer to the RDB module in Figure 2 (b); Feature map z t can be recursively calculated as follows:

[0064]

[0065] where [·, ·, …, ·] represents the concatenation operation, that is: the participating feature maps are concatenated in the channel direction, Figure 6 and a schematic diagram of the concatenation operation is given; is the convolution operation, and Θ is its parameter;

[0066] The decoding module D(·; Θ D ) in formula (4) consists of K decoding subnets, defined as:

[0067]

[0068] where, is defined as follows:

[0069]

[0070] Here, is the transposed convolution operation, and Φ is its parameter;

[0071] The data consistency function f DC (·, x u ) in formula (4) is specifically defined as follows: Let The element in is calculated as follows:

[0072]

[0073] where, Ω represents the index subset of the elements used to construct y u = M u y in y = Fx(y u (see formula (3) for the relationship between y and x). The data consistency operator essentially replaces the information in the corresponding frequency band of the reconstructed MRI image with the frequency information (mainly low-frequency information) of the input MRI image, Figure 7 and a graphical explanation of the data consistency operator is given.

[0074] In step 2, ICU-Net f ICUThe parameter set Θ in (·; Θ) can be obtained by training with existing optimization algorithms for deep learning. Here, we use the Adam method, where the momentum parameter is set to 0.9, the number of training epochs is set to 100, the batch size is set to 6, and the initial learning rate is set to 10 -4 , and after 50 epochs, the learning rate is reset to 10 -5 .

[0075] Furthermore, experiments are conducted to verify the beneficial effects of the proposed ICU-Net of the present invention. Five subjects are scanned using a Philips Ingenia 3T scanner to obtain MRI images in the T2WI modality, which are preprocessed to remove slices without any brain tissue, and each MR slice is cropped to a size of 336×336×261 Figure 8 An example of the training set used is given . Here Γ = {1, 2, …, G} is the index set of training samples, G is the number of training samples, and u is the undersampling multiple of the MRI image is the g-th fully sampled MRI image

[0076]

[0077] Table 1

[0078] Table 1 compares the quantitative evaluation results of four methods, namely zero-padding, U-Net, RDU-Net, and ICU-Net, on 3 kinds of undersampled MRI images with undersampling multiples of 4 times, 8 times, and 16 times. To quantitatively evaluate the reconstruction performance of the network, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used to evaluate the quality of the reconstructed MRI images. It can be seen that ICU-Net exhibits better reconstruction accuracy than other methods

[0079] Figure 9 and Figure 10 respectively conduct a visual comparison of the reconstruction effects of zero-padding, U-Net, RDU-Net, and ICU-Net for 8-fold and 16-fold undersampled MRI images. It can be seen that ICU-Net also exhibits better reconstruction effects visually than other methods

Claims

1. A reconstruction method for high-fold undersampled magnetic resonance images based on ICU-Net, Characterized in that, The method includes the following steps: Step 1: Input the u-fold undersampled MRI image denotes the pixel space, and N is the dimension of x u where x u is the u-fold undersampled MRI image in the k-space and x is the MRI image obtained after zero-padding and inverse Fourier transform denotes the k-space of the MRI image Step 2 is performed by ICU-Net f ICU (·; Θ), to obtain the reconstructed MRI image In step 2, ICU-Net f ICU (·; Θ) consists of an encoding module E(·; Θ E ) and a decoding module D(·; Θ D ), and the specific definitions are as follows: f ICU (x u ; Θ) = F H f DC (D(E(x u ; Θ E )); Θ D ), x u ) (1) where, Θ = {Θ E , Θ D}, f DC (·, x u ) represents data consistency calculation, E(·; Θ E ) is the encoding module of f ICU , D(·; Θ D ) is the decoding module of f ICU , Θ E and Θ D are the parameter sets of the encoding module and the decoding module respectively; The encoding module E(·; Θ E ) in formula (1) consists of K encoding subnets and is defined as: where K is the depth of the encoding module, is defined as follows: Here, is a convolution operation, and its parameters; is the residual dense block RDB, and its parameters are defined as follows: Among them, z 0 is the input feature map of f RDB , z T is the feature map output by the last convolutional layer of f RDB , and z T is calculated from z 0 and the feature maps z RDB output by the first T - 1 convolutional layers of f 1 , …, z T-1 , where T is the depth of the residual dense module; The feature map z t can be recursively calculated in the following way: Among them, [·, ·, …, ·] represents the concatenation operation, that is: the feature maps participating in the operation are concatenated in the channel direction; is the convolution operation, is its parameter; The decoding module D(·; Θ D ) in formula (1) consists of K decoding subnets, which are defined as: Among them, are defined as follows: Here, is the deconvolution operation, is its parameter; The data consistency function f in formula (1) DC (·, x u ) is defined as follows: Let The elements in are calculated in the following way: Among them, Ω represents the index subset of the elements in y u = M u in y = Fx, where the relationship between y u , y and x is shown in formula (2); In step 1, x u is obtained by the following method: Among them, represents the fully sampled image in k-space, represents the fully sampled image in pixel space, is the Fourier transform matrix, and H represents the Hermitian transpose operation, which is used to transform the MRI image from k-space to pixel space; M u ∈{0, 1} M×N represents the mask matrix for u-fold undersampling of y.

Images