Unsupervised Deep Learning Magnetic Resonance Reconstruction Method and Device with Iterative Optimization Expansion

Abstract

This application is applicable to the field of magnetic resonance imaging technology, and provides an unsupervised deep learning magnetic resonance reconstruction method and device with iterative optimization unfolding. The method includes: obtaining an undersampled MRI image to be reconstructed; inputting the undersampled MRI image into a trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstruction image; and during the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network. This application can improve the reconstruction efficiency of magnetic resonance images.

Description

Technical Field

[0001] This application belongs to the technical field of magnetic resonance imaging, and particularly relates to an unsupervised deep learning magnetic resonance reconstruction method and device with iterative optimization expansion. Background Art

[0002] Magnetic Resonance Imaging (MRI) is an imaging technology widely used in clinical diagnosis and treatment. However, MRI needs to sequentially acquire data samples in the Fourier transform domain (k-space), the acquisition process is relatively slow, and the data acquisition speed is often restricted by the physiological constraints of the imaging object or the hardware constraints of the imaging device. At the same time, the long scanning process will increase the discomfort of the patient and generate motion artifacts due to the position change of the scanning object. These artifacts often confuse with pathological features and affect the medical diagnosis results.

[0003] A common method to achieve fast MRI is to first undersample the magnetic resonance data at a relatively high undersampling rate, and then reconstruct it through image processing and other technologies to restore high-quality images. Currently, the reconstruction of magnetic resonance images is mainly completed by a reconstruction method based on iterative solution of the traditional Compressed Sensing (CS) algorithm. However, the iterative process of this reconstruction method is long and the computational complexity is high, resulting in low reconstruction efficiency of magnetic resonance images. Summary of the Invention

[0004] The embodiments of this application provide an unsupervised deep learning magnetic resonance reconstruction method and device with iterative optimization expansion, which can solve the problem of low reconstruction efficiency of magnetic resonance images.

[0005] In a first aspect, the embodiments of this application provide an unsupervised deep learning magnetic resonance reconstruction method with iterative optimization expansion, including:

[0006] Obtain an undersampled MRI image to be reconstructed;

[0007] Input the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstruction image;

[0008] During the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network.

[0009] Optionally, the training process of the GAP algorithm model includes:

[0010] Construct a self-supervised network model; the self-supervised network model includes a first GAP algorithm model and a second GAP algorithm model set in parallel;

[0011] Obtain a plurality of training samples, where the training samples include a first image obtained by performing an inverse Fourier transform on a first subset of undersampled k-space data, and a second image obtained by performing an inverse Fourier transform on a second subset of undersampled k-space data;

[0012] Use the plurality of training samples to train a self-supervised network model, obtain a loss value of the self-supervised network model, and adjust the network parameters of the self-supervised network model according to the loss value. When the loss value is lower than a preset threshold, use either the first GAP algorithm model or the second GAP algorithm model as the trained GAP algorithm model.

[0013] Optionally, using the plurality of training samples to train a self-supervised network model to obtain a loss value of the self-supervised network model includes:

[0014] Input the plurality of training samples into the self-supervised network model;

[0015] Based on the images iteratively output during the image reconstruction process of the first image of each training sample by the first GAP algorithm model, and the images iteratively output during the image reconstruction process of the second image of each training sample by the second GAP algorithm model, obtain the image domain self-supervised loss value of the self-supervised network model;

[0016] Based on the reconstructed images output by the first GAP algorithm model for the first image of each training sample, and the reconstructed images output by the second GAP algorithm model for the second image of each training sample, obtain the k-space self-supervised loss value and the difference loss value of the self-supervised network model;

[0017] Take the sum of the image domain self-supervised loss value, the k-space self-supervised loss value, and the difference loss value as the loss value of the self-supervised network model.

[0018] Optionally, obtaining a plurality of training samples includes:

[0019] Obtain a plurality of undersampled k-space data;

[0020] For each undersampled k-space data, perform the following steps:

[0021] Randomly generate a first selection matrix and a second selection matrix, and based on the undersampled k-space data, the first selection matrix, and the second selection matrix, obtain a first subset and a second subset of the undersampled k-space data;

[0022] Perform an inverse Fourier transform on the first subset to obtain a first image corresponding to the undersampled k-space data;

[0023] Perform an inverse Fourier transform on the second subset to obtain a second image corresponding to the undersampled k-space data;

[0024] Use the first image and the second image corresponding to the undersampled k-space data as a training sample.

[0025] Optionally, denoise the iteratively output image through an image denoising neural network, including:

[0026] Through the formula θ (t) = D w (x (t) ) to denoise the iteratively output image;

[0027] Wherein, x (t) represents the image output at the t-th iteration in the image reconstruction process by the GAP algorithm model, and θ (t) represents the image after denoising x (t) , D w represents the image denoising neural network, t is an integer, and 0 < t ≤ T, where T is the preset number of iterations.

[0028] Optionally, the image denoising neural network is a U-Net neural network.

[0029] In a second aspect, an embodiment of the present application provides an iterative optimization-expanded unsupervised deep learning magnetic resonance reconstruction device, including:

[0030] A first acquisition module, configured to acquire an undersampled MRI image to be reconstructed;

[0031] A reconstruction module, configured to input the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstruction image;

[0032] During the image reconstruction process, the GAP algorithm model denoises the iteratively output image through an image denoising neural network.

[0033] Optionally, the device further includes:

[0034] A construction module, configured to construct a self-supervised network model; the self-supervised network model includes a first GAP algorithm model and a second GAP algorithm model arranged in parallel;

[0035] A second acquisition module, configured to acquire a plurality of training samples, where the training samples include a first image obtained by performing an inverse Fourier transform on a first subset of the undersampled k-space data, and a second image obtained by performing an inverse Fourier transform on a second subset of the undersampled k-space data;

[0036] A training module for training a self-supervised network model using multiple training samples, obtaining a loss value of the self-supervised network model, and adjusting the network parameters of the self-supervised network model according to the loss value until the loss value is lower than a preset threshold, and then taking either the first GAP algorithm model or the second GAP algorithm model as the trained GAP algorithm model.

[0037] Optionally, the training module includes:

[0038] An input unit for inputting multiple training samples into the self-supervised network model;

[0039] A first calculation unit for obtaining an image domain self-supervised loss value of the self-supervised network model based on the images iteratively output during the image reconstruction process of the first image of each training sample by the first GAP algorithm model and the images iteratively output during the image reconstruction process of the second image of each training sample by the second GAP algorithm model;

[0040] A second calculation unit for obtaining a k-space self-supervised loss value and a difference loss value of the self-supervised network model based on the reconstructed images output by the first GAP algorithm model for the first image of each training sample and the reconstructed images output by the second GAP algorithm model for the second image of each training sample;

[0041] A third calculation unit for taking the sum of the image domain self-supervised loss value, the k-space self-supervised loss value, and the difference loss value as the loss value of the self-supervised network model.

[0042] Optionally, the second acquisition module includes:

[0043] An acquisition unit for acquiring multiple undersampled k-space data;

[0044] An execution unit for respectively performing the following steps for each undersampled k-space data:

[0045] Randomly generating a first selection matrix and a second selection matrix, and obtaining a first subset and a second subset of the undersampled k-space data based on the undersampled k-space data, the first selection matrix, and the second selection matrix;

[0046] Performing an inverse Fourier transform on the first subset to obtain a first image corresponding to the undersampled k-space data;

[0047] Performing an inverse Fourier transform on the second subset to obtain a second image corresponding to the undersampled k-space data;

[0048] Taking the first image and the second image corresponding to the undersampled k-space data as a training sample.

[0049] Optionally, the reconstruction module is specifically used by the formula θ(t) = D w (x (t) ) denoises the iteratively output image;

[0050] Among them, x (t) represents the image output in the t-th iteration during the image reconstruction by the GAP algorithm model, and θ (t) represents the image after denoising x (t) , D w represents the image denoising neural network, t is an integer, and 0 < t ≤ T, where T is the preset number of iterations.

[0051] Optionally, the image denoising neural network is a U-Net neural network.

[0052] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above method is implemented.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0054] In a fifth aspect, an embodiment of the present application provides a computer program product, which when running on a terminal device, causes the terminal device to execute the method according to any one of the above first aspects.

[0055] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0056] In the embodiments of the present application, the undersampled MRI image to be reconstructed is input into the GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstruction image. Among them, since the denoising process in the GAP algorithm model is implemented by a deep learning image denoising neural network, the GAP algorithm model can utilize the characteristics of deep learning to accelerate the reconstruction of the undersampled MRI image and improve the reconstruction efficiency of the magnetic resonance image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 is a flowchart of an unsupervised deep learning magnetic resonance reconstruction method provided by an embodiment of the present application;

[0059] Figure 2 is a flowchart of the training process of the GAP algorithm model provided by an embodiment of the present application;

[0060] Figure 3 is a schematic structural diagram of the self-supervised network model provided by an embodiment of the present application;

[0061] Figure 4 is a flowchart of calculating the loss value of the self-supervised network model provided by an embodiment of the present application;

[0062] Figure 5 is a schematic structural diagram of the unsupervised deep learning magnetic resonance reconstruction device provided by an embodiment of the present application;

[0063] Figure 6 is a schematic structural diagram of the terminal device provided by an embodiment of the present application. Detailed implementation manners

[0064] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0065] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0066] It should also be understood that the term "and / or" as used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0067] As used in the specification of the present application and the appended claims, the term "if" can be interpreted as "when...", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0068] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0069] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0070] Currently, the reconstruction of magnetic resonance images is mainly completed by a reconstruction method based on iterative solution of traditional compressed sensing algorithms. However, the iterative process of this reconstruction method is relatively long and the computational complexity is high, resulting in low reconstruction efficiency of magnetic resonance images.

[0071] In view of the above problems, the embodiments of the present application perform image reconstruction on the undersampled MRI image to be reconstructed through the GAP algorithm model to obtain a magnetic resonance reconstructed image. Since the denoising process in the GAP algorithm model is implemented through a deep learning image denoising neural network, the GAP algorithm model can utilize the characteristics of deep learning to accelerate the reconstruction of the undersampled MRI image and improve the reconstruction efficiency of magnetic resonance images.

[0072] The unsupervised deep learning magnetic resonance reconstruction method provided by the present application will be exemplarily described below in conjunction with specific embodiments.

[0073] The unsupervised deep learning magnetic resonance reconstruction method provided by the embodiments of the present application can be executed by a terminal device or by a device (such as a chip) applied to the terminal device. The following embodiments take the method being executed by the terminal device as an example. As an example, the terminal device can be a tablet, a server, a laptop computer, etc., and the embodiments of the present application do not make any limitations thereto.

[0074] As Figure 1 shown, the embodiments of the present application provide an iterative optimization-expanded unsupervised deep learning magnetic resonance reconstruction method, and this method includes the following steps:

[0075] Step 11, obtain the undersampled MRI image to be reconstructed.

[0076] In some embodiments of the present application, the undersampled MRI image is an image obtained by performing an inverse Fourier transform on the undersampled k-space data. Specifically, the terminal device can obtain the undersampled k-space data from the magnetic resonance imaging device, and then perform an inverse Fourier transform on the undersampled k-space data to obtain the undersampled MRI image.

[0077] Step 12: Input the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstructed image. During the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network.

[0078] It is worth mentioning that, in some embodiments of the present application, the undersampled MRI image to be reconstructed is input into the GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstructed image. Since the denoising processing in the GAP algorithm model is implemented through a deep learning image denoising neural network, the GAP algorithm model can utilize the characteristics of deep learning to accelerate the reconstruction of the undersampled MRI image and improve the reconstruction efficiency of the magnetic resonance image.

[0079] The above Generalized Alternating Projection (GAP) algorithm is a type of CS algorithm. In the related art, the compressed sensing magnetic resonance (CS-MRI) image reconstruction problem can be formulated as the solution of the following non-linear optimization problem:

[0080]

[0081] where A is the observation matrix, A = PF, P is the undersampling matrix, F is the Fourier transform matrix, y is the undersampled Fourier frequency domain data (i.e., the undersampled k-space data), x is the fully sampled original image, R(x) is the regularization term regarding the original image, and λ is the regularization parameter.

[0082] Specify the regularization term R(x) as the sparsity constraint on the original image x, i.e., R(x) = ‖Dx‖1, where D is the image sparse transform domain. By introducing an auxiliary variable θ, the GAP algorithm can equivalently formulate the above optimization problem as:

[0083]

[0084] where t is the number of iterations. Under the GAP algorithm, the solution of this optimization problem can be carried out under the alternating optimization of the original variable x and the auxiliary variable θ. Given θ, the update of x can be regarded as an Euclidean projection in the linear manifold space, expressed as:

[0085] x (t) = θ (t-1) + A H (AAH ) -1 (y - Aθ (t-1) )

[0086] Given the original variable x, the update of the auxiliary variable θ can be regarded as an image denoising problem. In some embodiments of the present application, an image denoising neural network is used for image denoising. That is, through the formula θ (t) = D w (x (t) ), the iteratively output image (i.e., x (t) ) is denoised. Wherein, x (t) is the original variable, representing the image output by the GAP algorithm model at the t-th iteration during image reconstruction; θ (t) is the auxiliary variable, representing the image after denoising x (t) ; D w represents the image denoising neural network. As a preferred example, the image denoising neural network is a U-Net neural network (the U-Net neural network is a variant of the convolutional neural network. The entire neural network mainly includes a contracting path and an expanding path, resembling the letter "U", so it is named the U-Net neural network); t is an integer, and 0 < t ≤ T, where T is the preset number of iterations. It should be noted that T can be determined according to the effects of multiple experiments and set in advance.

[0087] Thus, it can be seen that in some embodiments of the present application, the process of the GAP algorithm model for image reconstruction is as follows: (1) According to θ (t) , using the formula x (t) = θ (t-1) + A H (AA H ) -1 (y - Aθ (t-1) ) to obtain x (t+1) ; (2) According to the formula θ (t) = D w (x (t) ) to obtain θ (t+1) ; (3) Perform multiple iterative updates according to steps (1) and (2) to obtain the final reconstructed image θ T . It should be noted that the initial value in the GAP algorithm model is set as θ 0 = A H y, that is, θ 0 is initially the undersampled image (i.e., the undersampled MRI image).

[0088] The following is an exemplary description of the training process of the GAP algorithm model in combination with specific embodiments.

[0089] As Figure 2As shown, the training process of the GAP algorithm model includes the following steps:

[0090] Step 21, construct a self-supervised network model.

[0091] As Figure 3 shown, the above self-supervised network model 300 includes a first GAP algorithm model 301 and a second GAP algorithm model 302 arranged in parallel. Among them, the first GAP algorithm model 301 and the second GAP algorithm model 302 can be GAP algorithm models with the same structure. It should be noted that the process of image reconstruction of the undersampled images input to the first GAP algorithm model 301 and the second GAP algorithm model 302 is the same as the process of image reconstruction of the GAP algorithm model described above.

[0092] Step 22, obtain multiple training samples.

[0093] The above training samples include a first image obtained by performing inverse Fourier transform on a first subset of undersampled k-space data, and a second image obtained by performing inverse Fourier transform on a second subset of undersampled k-space data. It should be noted that the undersampled k-space data corresponding to multiple training samples are different from each other.

[0094] In some embodiments of the present application, the specific implementation manner of the above step 22 of obtaining multiple training samples includes the following steps:

[0095] Step 1, obtain multiple undersampled k-space data.

[0096] In some embodiments of the present application, multiple undersampled k-space data can be specifically collected by a magnetic resonance imaging device.

[0097] Step 2, for each undersampled k-space data, perform the following steps:

[0098] Step 1, randomly generate a first selection matrix and a second selection matrix, and obtain a first subset and a second subset of the undersampled k-space data according to the undersampled k-space data, the first selection matrix, and the second selection matrix.

[0099] In some embodiments of the present application, the above first selection matrix and second selection matrix can be generated by a random matrix generation algorithm. After obtaining the first selection matrix and the second selection matrix, the first subset and the second subset can be calculated by the following formula: y1 = S1y, y2 = S2y. Where y1 represents the first subset, y2 represents the first subset, S1 represents the first selection matrix, S2 represents the second selection matrix, and y represents the undersampled k-space data.

[0100] In the second step, perform an inverse Fourier transform on the first subset to obtain a first image corresponding to the undersampled k-space data, and perform an inverse Fourier transform on the second subset to obtain a second image corresponding to the undersampled k-space data.

[0101] It should be noted that the above-mentioned first image is the image input into the first GAP algorithm model when training the self-supervised network model, and the second image is the image input into the second GAP algorithm model when training the self-supervised network model.

[0102] In the third step, use the first image and the second image corresponding to the undersampled k-space data as a training sample.

[0103] It can be seen that in some embodiments of the present application, by obtaining a plurality of undersampled k-space data and obtaining a first image and a second image corresponding to each undersampled k-space data, a plurality of training samples for training the self-supervised network model can be obtained.

[0104] Step 23: Use a plurality of training samples to train the self-supervised network model to obtain a loss value of the self-supervised network model, and adjust the network parameters of the self-supervised network model according to the loss value. When the loss value is lower than a preset threshold, use any one of the first GAP algorithm model and the second GAP algorithm model as the trained GAP algorithm model.

[0105] In some embodiments of the present application, as Figure 4 shown, the specific implementation manner of using a plurality of training samples to train the self-supervised network model to obtain a loss value of the self-supervised network model includes the following steps:

[0106] Step 41: Input a plurality of training samples into the self-supervised network model.

[0107] Step 42: Obtain an image domain self-supervised loss value of the self-supervised network model according to the images iteratively output during the image reconstruction process of the first image of each training sample by the first GAP algorithm model and the images iteratively output during the image reconstruction process of the second image of each training sample by the second GAP algorithm model.

[0108] In some embodiments of the present application, the image domain self-supervised loss value of the self-supervised network model can be calculated by the formula

[0109]

[0110] wherein, is the image domain self-supervised loss value of the self-supervised network model, N is the total number of training samples, i represents the i-th training sample, k is the total number of iterations, is the random selection matrix used for randomly masking image pixel points during the t-th iteration of the first image of the i-th training sample by the first GAP algorithm model. is the output result of the t-th image denoising neural network during the image reconstruction process of the first image of the i-th training sample by the first GAP algorithm model. is the input of the t-th image denoising neural network during the image reconstruction process of the first image of the i-th training sample by the first GAP algorithm model. is the random selection matrix used for randomly masking image pixel points during the t-th iteration of the second image of the i-th training sample by the second GAP algorithm model. is the output result of the t-th image denoising neural network during the image reconstruction process of the second image of the i-th training sample by the second GAP algorithm model. is the input of the t-th image denoising neural network during the image reconstruction process of the second image of the i-th training sample by the second GAP algorithm model.

[0111] Step 43: Obtain the k-space self-supervised loss value and the difference loss value of the self-supervised network model according to the reconstructed images output by the first GAP algorithm model for the first images of each training sample and the reconstructed images output by the second GAP algorithm model for the second images of each training sample.

[0112] In some embodiments of the present application, the formula

[0113]

[0114] is used to calculate the k-space self-supervised loss value of the self-supervised network model. Wherein, is the k-space self-supervised loss value of the self-supervised network model, N is the total number of training samples, i represents the i-th training sample, is the reconstructed image output by the first GAP algorithm model for the first image of the i-th training sample, y i is the undersampled k-space data corresponding to the i-th training sample, P i is the undersampling matrix when collecting y i is the Fourier inverse transform matrix, F -1 is the Fourier inverse transform matrix, is the reconstructed image output by the second GAP algorithm model for the second image of the i-th training sample.

[0115] The formula

[0116]

[0117] is used to calculate the difference loss value of the self-supervised network model. Wherein, is the difference loss value of the self-supervised network model, N is the total number of training samples, i represents the i-th training sample, I is the identity matrix, and P i is the undersampling matrix when acquiring the undersampled k-space data corresponding to the i-th training sample, and I - P i is the un-scanned frequency points when acquiring the undersampled k-space data corresponding to the i-th training sample, and F -1 is the inverse Fourier transform matrix, is the reconstructed image output by the first GAP algorithm model for image reconstruction of the first image of the i-th training sample, is the reconstructed image output by the second GAP algorithm model for image reconstruction of the second image of the i-th training sample.

[0118] Step 44, take the sum of the image domain self-supervised loss value, the k-space self-supervised loss value, and the difference loss value as the loss value of the self-supervised network model.

[0119] That is, in some embodiments of the present application, the loss value of the self-supervised network model can be calculated by the formula where, is the loss value of the self-supervised network model, is the image domain self-supervised loss value of the self-supervised network model, is the k-space self-supervised loss value of the self-supervised network model, is the difference loss value of the self-supervised network model.

[0120] In some embodiments of the present application, after obtaining the loss value of the self-supervised network model, the network parameters of the self-supervised network model can be adjusted backward according to this loss value. After updating the network parameters, the self-supervised network model is retrained with multiple training samples until the loss value of the self-supervised network model is lower than the preset threshold, and any GAP algorithm model in the self-supervised network model is used as the trained GAP algorithm model.

[0121] It is worth mentioning that, in some embodiments of the present application, during the denoising process, the self-supervised denoising effect in the image domain is achieved by randomly masking some pixel points of the input image and restoring the information of these pixel points through the network to obtain the image domain self-supervised loss value; at the same time, the self-supervised effect in the k-space is achieved by calculating the correctness of the information recovery at the undersampled frequency points of the output image to obtain the k-space self-supervised loss value. By imposing a consistency constraint on the output images of the first GAP algorithm model and the second GAP algorithm model in parallel in the self-supervised network model, the constraint on the information of the un-scanned frequency points is achieved to obtain the difference loss value. Finally, the sum of these three loss values is used as the loss value of the self-supervised network model, and the network parameters of the self-supervised network model are adjusted, so that the image reconstruction quality of the finally trained GAP algorithm model is greatly improved.

[0122] In summary, the unsupervised deep learning magnetic resonance reconstruction method provided by the embodiments of the present application has the following effects:

[0123] First, since the denoising process in the GAP algorithm model is implemented through a deep learning image denoising neural network, the GAP algorithm model can utilize the characteristics of deep learning to accelerate the reconstruction of undersampled MRI images and improve the reconstruction efficiency of magnetic resonance images;

[0124] Second, since the GAP algorithm model is a magnetic resonance image reconstruction network model that combines the GAP algorithm with deep learning, the parameters in the GAP algorithm model do not need to be empirically set, and at the same time, by utilizing the characteristics of the CS algorithm, the mathematical interpretability of the GAP algorithm model is increased, and the amount of training samples required in the network model training stage is reduced;

[0125] Third, by constructing a self-supervised network model to complete the training of the GAP algorithm model, the optimization of network parameters can be carried out without any fully sampled images, effectively solving the problem of dependence of traditional deep learning methods on fully sampled images;

[0126] Fourth, during the training process of the GAP algorithm model, a consistency constraint is imposed on the output images of the two parallel GAP algorithm models in the self-supervised network model, thereby indirectly constraining the information recovery process at the un-scanned frequency points and improving the image reconstruction quality;

[0127] Fifth, by comprehensively utilizing the information in the k-space and the image domain to complete the self-supervised training process of the GAP algorithm model, the reconstruction quality of the GAP algorithm model is further improved.

[0128] Next, the unsupervised deep learning magnetic resonance reconstruction device provided by the present application will be exemplarily described in conjunction with specific embodiments.

[0129] As Figure 5 shown, the embodiments of the present application provide an unsupervised deep learning magnetic resonance reconstruction device with iterative optimization expansion. The unsupervised deep learning magnetic resonance reconstruction device 500 includes:

[0130] A first acquisition module 501, configured to acquire an undersampled MRI image to be reconstructed;

[0131] A reconstruction module 502, configured to input the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstruction image;

[0132] During the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network.

[0133] Optionally, the unsupervised deep learning magnetic resonance reconstruction apparatus 500 further includes:

[0134] A construction module for constructing a self-supervised network model; the self-supervised network model includes a first GAP algorithm model and a second GAP algorithm model arranged in parallel;

[0135] A second acquisition module for acquiring a plurality of training samples, where the training samples include a first image obtained by performing an inverse Fourier transform on a first subset of undersampled k-space data, and a second image obtained by performing an inverse Fourier transform on a second subset of undersampled k-space data;

[0136] A training module for training the self-supervised network model using a plurality of training samples to obtain a loss value of the self-supervised network model, and adjusting network parameters of the self-supervised network model according to the loss value until the loss value is lower than a preset threshold, and using any one of the first GAP algorithm model and the second GAP algorithm model as the trained GAP algorithm model.

[0137] Optionally, the training module includes:

[0138] An input unit for inputting a plurality of training samples into the self-supervised network model;

[0139] A first calculation unit for obtaining an image domain self-supervised loss value of the self-supervised network model according to the images iteratively output during the image reconstruction process of the first image of each training sample by the first GAP algorithm model, and the images iteratively output during the image reconstruction process of the second image of each training sample by the second GAP algorithm model;

[0140] A second calculation unit for obtaining a k-space self-supervised loss value and a difference loss value of the self-supervised network model according to the reconstructed images output by the first GAP algorithm model for the first image of each training sample, and the reconstructed images output by the second GAP algorithm model for the second image of each training sample;

[0141] A third calculation unit for using the sum value of the image domain self-supervised loss value, the k-space self-supervised loss value, and the difference loss value as the loss value of the self-supervised network model.

[0142] Optionally, the second acquisition module includes:

[0143] An acquisition unit for acquiring a plurality of undersampled k-space data;

[0144] An execution unit for respectively performing the following steps for each undersampled k-space data:

[0145] Randomly generate a first selection matrix and a second selection matrix, and obtain a first subset and a second subset of the undersampled k-space data according to the undersampled k-space data, the first selection matrix, and the second selection matrix;

[0146] Perform an inverse Fourier transform on the first subset to obtain a first image corresponding to the undersampled k-space data;

[0147] Perform an inverse Fourier transform on the second subset to obtain a second image corresponding to the undersampled k-space data;

[0148] Use the first image and the second image corresponding to the undersampled k-space data as a training sample.

[0149] Optionally, the reconstruction module 502 is specifically configured to denoise the iteratively output image through the formula θ (t) = D w (x (t) );

[0150] where x (t) represents the image output by the GAP algorithm model in the t-th iteration during image reconstruction, θ (t) represents the image after denoising x (t) , D w represents an image denoising neural network, t is an integer, and 0 < t ≤ T, where T is a preset number of iterations.

[0151] Optionally, the image denoising neural network is a U-Net neural network.

[0152] It should be noted that for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiment of the present application, their specific functions and the technical effects brought can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0153] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0154] As Figure 6 shown, an embodiment of the present application provides a terminal device. As Figure 6 shown, the terminal device D10 of this embodiment includes: at least one processor D100( Figure 6 only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the steps in any of the above method embodiments are implemented.

[0155] The so-called processor D100 may be a central processing unit (CPU, Central Processing Unit). This processor D100 may also be other general-purpose processors, digital signal processors (DSP, Digital Signal Processor), application specific integrated circuits (ASIC, Application Specific Integrated Circuit), field-programmable gate arrays (FPGA, Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0156] The memory D101 may be an internal storage unit of the terminal device D10 in some embodiments, such as the hard disk or memory of the terminal device D10. The memory D101 may also be an external storage device of the terminal device D10 in other embodiments, such as a plug-in hard disk, a smart media card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both the internal storage unit and the external storage device of the terminal device D10. The memory D101 is used to store an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or will be output.

[0157] It should be noted that for the information interaction, execution process, etc. between the above devices / units, since they are based on the same concept as the method embodiments of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part for details, and will not be elaborated here.

[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0159] The embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0160] The embodiments of this application provide a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the foregoing method embodiments when executed.

[0161] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the unsupervised deep learning magnetic resonance reconstruction device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0162] In the above embodiments, the descriptions of the various embodiments each have their own focuses. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0163] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0164] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the device or unit can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An unsupervised deep learning magnetic resonance reconstruction method with iterative optimization expansion, characterized in that, Including: Obtain an undersampled MRI image to be reconstructed; Input the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstructed image; During the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network; The Generalized Alternating Projection (GAP) algorithm is a type of CS algorithm. In the related art, the problem of compressed sensing magnetic resonance (CS-MRI) image reconstruction can be expressed as the solution of the following non-linear optimization problem: where A is the observation matrix, A = PF, P is the undersampling matrix, F is the Fourier transform matrix, y is the undersampled Fourier frequency domain data, x is the fully sampled original image, R(x) is the regularization term regarding the original image, and λ is the regularization parameter; Specify the regularization term R(x) as the sparsity constraint on the original image x, that is, R(x) = ||Dx||1, where D is the image sparse transform domain. By introducing an auxiliary variable θ, the GAP algorithm equivalently expresses the above optimization problem as: where t is the number of iterations. Under the GAP algorithm, the solution of this optimization problem can be carried out through the alternating optimization of the original variable x and the auxiliary variable θ. Given θ, the update of x can be regarded as the Euclidean projection in the linear manifold space, expressed as: x (t) = θ (t-1) + A H (AA H ) -1 (y - Aθ (t-1) ); And given the original variable x, the update of the auxiliary variable θ can be regarded as an image denoising problem.

2. The method according to claim 1, wherein The training process of the GAP algorithm model includes: Construct a self-supervised network model; the self-supervised network model includes a first GAP algorithm model and a second GAP algorithm model set in parallel; Obtain a plurality of training samples, where the training samples include a first image obtained by performing inverse Fourier transform on a first subset of the undersampled k-space data, and a second image obtained by performing inverse Fourier transform on a second subset of the undersampled k-space data; Use the plurality of training samples to train the self-supervised network model to obtain the loss value of the self-supervised network model, and adjust the network parameters of the self-supervised network model according to the loss value until the loss value is lower than a preset threshold, and then use either the first GAP algorithm model or the second GAP algorithm model as the trained GAP algorithm model.

3. The method according to claim 2, characterized in that, The step of using the plurality of training samples to train the self-supervised network model to obtain the loss value of the self-supervised network model includes: Input the plurality of training samples into the self-supervised network model; Based on the images iteratively output during the image reconstruction process of the first image of each training sample by the first GAP algorithm model, and the images iteratively output during the image reconstruction process of the second image of each training sample by the second GAP algorithm model, obtain the image domain self-supervised loss value of the self-supervised network model; Based on the reconstructed images output by the first GAP algorithm model for the first image of each training sample and the reconstructed images output by the second GAP algorithm model for the second image of each training sample, obtain the k-space self-supervised loss value and the difference loss value of the self-supervised network model; Use the sum of the image domain self-supervised loss value, the k-space self-supervised loss value, and the difference loss value as the loss value of the self-supervised network model.

4. The method according to claim 2, wherein The obtaining of multiple training samples includes: Obtain multiple undersampled k-space data; For each undersampled k-space data, perform the following steps: Randomly generate a first selection matrix and a second selection matrix, and obtain a first subset and a second subset of the undersampled k-space data according to the undersampled k-space data, the first selection matrix, and the second selection matrix; Perform an inverse Fourier transform on the first subset to obtain a first image corresponding to the undersampled k-space data; Perform an inverse Fourier transform on the second subset to obtain a second image corresponding to the undersampled k-space data; Use the first image and the second image corresponding to the undersampled k-space data as a training sample.

5. The method according to claim 1, characterized in that The denoising process of the iteratively output image by the image denoising neural network includes: Denoise the iteratively output image through the formula θ (t) = D w (x (t) ) Among them, x (t) represents the image output by the GAP algorithm model at the t-th iteration during the image reconstruction process, and θ (t) represents the image after denoising processing of x (t) , D w represents the image denoising neural network, t is an integer, and 0 < t ≤ T, where T is the preset number of iterations.

6. The method according to claim 1, characterized in that, The image denoising neural network is a U-Net neural network.

7. An unsupervised deep learning magnetic resonance reconstruction device with iterative optimization expansion, characterized in that, It includes: A first acquisition module for acquiring an undersampled MRI image to be reconstructed; A reconstruction module for inputting the undersampled MRI image into the trained GAP algorithm model for image reconstruction to obtain a magnetic resonance reconstructed image; During the image reconstruction process, the GAP algorithm model performs denoising processing on the iteratively output image through an image denoising neural network; The Generalized Alternating Projection (GAP) algorithm is a type of CS algorithm. In the related art, the compressed sensing magnetic resonance (CS-MRI) image reconstruction problem can be expressed as the solution of the following non-linear optimization problem: Where, A is the observation matrix, A = PF, P is the undersampling matrix, F is the Fourier transform matrix, y is the undersampled Fourier frequency domain data, x is the fully sampled original image, R(x) is the regularization term regarding the original image, and λ is the regularization parameter; Specify the regularization term R(x) as the sparsity constraint on the original image x, that is, R(x) = ||Dx||1, where D is the image sparse transform domain. By introducing an auxiliary variable θ, the GAP algorithm equivalently expresses the above optimization problem as: Where, t is the number of iterations. Under the GAP algorithm, the solution of this optimization problem can be carried out under the alternating optimization of the original variable x and the auxiliary variable θ. Given θ, the update of x can be regarded as an Euclidean projection in the linear manifold space, expressed as: x (t) = θ (t-1) + A H (AA H ) -1 (y - Aθ (t-1) ); And given the original variable x, the update of the auxiliary variable θ can be regarded as an image denoising problem.

8. The device according to claim 7, characterized in that The device further includes: A construction module for constructing a self-supervised network model; the self-supervised network model includes a first GAP algorithm model and a second GAP algorithm model arranged in parallel; A second acquisition module for acquiring multiple training samples, the training samples including a first image obtained by performing an inverse Fourier transform on a first subset of the undersampled k-space data, and a second image obtained by performing an inverse Fourier transform on a second subset of the undersampled k-space data; A training module, configured to train the self-supervised network model by using the multiple training samples, obtain a loss value of the self-supervised network model, and adjust network parameters of the self-supervised network model according to the loss value. When the loss value is lower than a preset threshold, either the first GAP algorithm model or the second GAP algorithm model is used as the trained GAP algorithm model.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images