Magnetic resonance imaging method, neural network, neural network generation method and readable medium

Abstract

The invention relates to a magnetic resonance imaging method, a neural network and a generation method thereof, and a readable medium. In one embodiment, a method for performing magnetic resonance imaging on a subject includes obtaining one or more time domain basis functions and one or more preliminary spatial weighting functions from undersampled imaging data from a region of interest of the subject, the undersampled imaging data corresponding to an image sequence having a plurality of frames, each of the time domain basis functions corresponds to at least one time-varying dimension of the object, and each of the preliminary spatial weighting functions corresponds to a space-varying dimension of the object; and inputting the one or more preliminary spatial weighting functions into a neural network to produce one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a corresponding one of the one or more preliminary spatial weighting functions, the one or more final spatial weighting functions are an artifact-free version of the one or more preliminary spatial weighting functions and can be multiplied by the one or more time domain basis functions to generate a sequence of images.

Description

[0001] This application is a divisional application of a patent application with application number 202080063343.6, whose application date is September 10, 2020 and invention name is “Deep learning system and method for large-scale dynamic magnetic resonance image reconstruction”. Technical Field

[0002] The present disclosure relates to biomedical imaging and analysis. More specifically, the present disclosure relates to MRI image reconstruction using deep learning techniques. Background Art

[0003] Dynamic imaging plays an important role in many clinical magnetic resonance imaging (MRI) examinations. Dynamic imaging enables healthcare professionals to assess tissue health by visualizing and / or measuring dynamic processes occurring in the body (e.g., heart motion, breathing, etc.). However, dynamic MRI can be a relatively slow process, and acceleration methods are often required to reconstruct images from incomplete imaging data. Therefore, new systems and methods are needed that can more efficiently and quickly decode image data and reconstruct image sequences. Summary of the invention

[0004] According to aspects of the present disclosure, a method for performing magnetic resonance (MR) imaging on an object includes: acquiring undersampled imaging data from a region of interest of the object, the undersampled imaging data corresponding to an image sequence having a plurality of image frames; extracting one or more time-domain basis functions from the undersampled imaging data, each of the one or more time-domain basis functions corresponding to at least one time-varying dimension of the object; extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a space-varying dimension of the object; inputting the one or more preliminary spatial weighting functions into a neural network to generate one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a corresponding one of the one or more preliminary spatial weighting functions; and multiplying the one or more final spatial weighting functions with the one or more time-domain basis functions to generate the image sequence.

[0005] According to aspects of the present disclosure, a method for training a neural network having an architecture including one or more operations and one or more adjustable weights for analyzing magnetic resonance (MR) data from a subject comprises: receiving one or more training input data sets to the neural network, the one or more training input data sets comprising one or more sets of training preliminary spatial weighting functions and one or more sets of training final spatial weighting functions, each of the one or more sets of training final spatial weighting functions being an artifact-free version of a corresponding one of the one or more sets of training preliminary spatial weighting functions previously obtained, each of the one or more sets of training preliminary spatial weighting functions and the one or more sets of training final spatial weighting functions corresponding to a time-varying dimension of the subject; and adjusting the one or more sets of training preliminary spatial weighting functions according to the architecture of the neural network. The training preliminary spatial weighting functions are processed to generate one or more sets of estimated final spatial weighting functions; each of the one or more sets of estimated final spatial weighting functions is compared with a corresponding one of the one or more sets of training final spatial weighting functions; it is determined whether the one or more sets of estimated final spatial weighting functions satisfy a cost function of the neural network, the cost function being configured to minimize the difference between the one or more sets of training final spatial weighting functions and the one or more sets of estimated final spatial weighting functions; in response to a determination that the cost function of the neural network is not satisfied, the one or more network weights are adjusted; and the one or more training preliminary spatial weighting functions are repeatedly processed using the adjusted one or more network weights until the cost function is satisfied.

[0006] According to aspects of the present disclosure, a system for performing magnetic resonance (MR) imaging on an object includes: a magnet for providing a magnetic field; a transmitter for transmitting to a region within the magnetic field; a receiver for receiving magnetic resonance signals from the region using the magnetic field; and one or more processors for controlling the transmitter and the receiver, the one or more processors being configured to cause the following method to be performed: acquiring undersampled imaging data from a region of interest of the object, the undersampled imaging data corresponding to an image sequence having a plurality of image frames; extracting one or more time-domain basis functions from the undersampled imaging data, each of the one or more time-domain basis functions corresponding to at least one time-varying dimension of the object; extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a space-varying dimension of the object; inputting the one or more preliminary spatial weighting functions into a neural network to generate one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a corresponding one of the one or more preliminary spatial weighting functions; and multiplying the one or more final spatial weighting functions with the one or more time-domain basis functions to generate the image sequence.

[0007] According to various aspects of the present disclosure, a non-transitory machine-readable medium stores thereon instructions for performing magnetic resonance (MR) imaging on an object, and when the instructions are executed by at least one processor, the instructions cause the following method to be executed: acquiring undersampled imaging data from a region of interest of the object, the undersampled imaging data corresponding to an image sequence having a plurality of image frames; extracting one or more time-domain basis functions from the undersampled imaging data, each of the one or more time-domain basis functions corresponding to at least one time-varying dimension of the object; extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a space-varying dimension of the object; inputting the one or more preliminary spatial weighting functions into a neural network to generate one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a corresponding one of the one or more preliminary spatial weighting functions; and multiplying the one or more final spatial weighting functions with the one or more time-domain basis functions to generate the image sequence.

[0008] According to aspects of the present disclosure, a system for training a neural network having an architecture including one or more operations and one or more adjustable weights for analyzing magnetic resonance (MR) data from a subject comprises: a magnet for providing a magnetic field; a transmitter for transmitting to a region within the magnetic field; a receiver for receiving magnetic resonance signals from the region using the magnetic field; and one or more processors for controlling the transmitter and the receiver, the one or more processors being configured to cause the following method to be performed: receiving one or more training input data sets to the neural network, the one or more training input data sets comprising one or more sets of training preliminary spatial weighting functions and one or more sets of training final spatial weighting functions, each of the one or more sets of training final spatial weighting functions being an artifact-free version of a corresponding one of the one or more sets of training preliminary spatial weighting functions previously obtained, the one or more sets of training preliminary spatial weighting functions being artifact-free versions of corresponding ones ... Each of the preliminary spatial weighting functions and the trained final spatial weighting functions corresponds to the time-varying dimension of the object; the one or more sets of trained preliminary spatial weighting functions are processed according to the architecture of the neural network to generate one or more sets of estimated final spatial weighting functions; each of the one or more sets of estimated final spatial weighting functions is compared with a corresponding one of the one or more sets of trained final spatial weighting functions; it is determined whether the one or more sets of estimated final spatial weighting functions satisfy the cost function of the neural network, and the cost function is configured to minimize the difference between the one or more sets of trained final spatial weighting functions and the one or more sets of estimated final spatial weighting functions; in response to a determination that the cost function of the neural network is not satisfied, the one or more network weights are adjusted; and the one or more trained preliminary spatial weighting functions are repeatedly processed using the adjusted one or more network weights until the cost function is satisfied.

[0009] According to aspects of the present disclosure, a non-transitory machine-readable medium storing instructions for training a neural network having an architecture including one or more operations and one or more adjustable weights for analyzing magnetic resonance (MR) data from a subject causes the following method to be performed: receiving one or more training input data sets to the neural network, the one or more training input data sets including one or more sets of training preliminary spatial weighting functions and one or more sets of training final spatial weighting functions, each of the one or more sets of training final spatial weighting functions being an artifact-free version of a corresponding one of the one or more sets of training preliminary spatial weighting functions previously obtained, each of the one or more sets of training preliminary spatial weighting functions and the training final spatial weighting functions corresponding to a time-varying dimension of the subject; training a neural network having an architecture including one or more operations and one or more adjustable weights according to the architecture of the neural network; The method comprises the steps of processing the one or more sets of training preliminary spatial weighting functions to generate one or more sets of estimated final spatial weighting functions; comparing each of the one or more sets of estimated final spatial weighting functions with a corresponding one of the one or more sets of training final spatial weighting functions; determining whether the one or more sets of estimated final spatial weighting functions satisfy a cost function of the neural network, the cost function being configured to minimize the difference between the one or more sets of training final spatial weighting functions and the one or more sets of estimated final spatial weighting functions; adjusting the one or more network weights in response to a determination that the cost function of the neural network is not satisfied; and repeatedly processing the one or more training preliminary spatial weighting functions using the adjusted one or more network weights until the cost function is satisfied.

[0010] The foregoing and other various aspects and embodiments of the present disclosure will be readily apparent to those of ordinary skill in the art in view of the detailed description of various embodiments and / or implementations given with reference to the accompanying drawings, a brief description of which is provided below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other advantages of the present disclosure will become apparent by reading the following detailed description and by referring to the accompanying drawings.

[0012] Figure 1 A method for performing magnetic resonance imaging of a subject according to aspects of the present disclosure is shown;

[0013] Figure 2 A method for training a neural network according to aspects of the present disclosure is shown;

[0014] Figure 3A is a first example of a neural network for generating a magnetic resonance image sequence according to aspects of the present disclosure;

[0015] Figure 3Bis a second example of a neural network for generating a magnetic resonance image sequence according to aspects of the present disclosure;

[0016] Figure 3C According to various aspects of the present disclosure, Figure 3A and Figure 3B An example of a densely connected block of the first or second example neural network;

[0017] Figure 4A Showing the use of various aspects of the present disclosure Figure 3A and Figure 3B A magnetic resonance image sequence of the systolic cardiac phase constructed by a neural network;

[0018] Figure 4B Showing the use of various aspects of the present disclosure Figure 3A and Figure 3B A diastolic cardiac phase magnetic resonance image sequence constructed using a neural network;

[0019] Figure 5 Showing the use of various aspects of the present disclosure Figure 3A and Figure 3B T1 maps and Bland-Altman curves generated by neural networks and traditional techniques; and

[0020] Figure 6 A system for performing magnetic resonance imaging according to aspects of the present disclosure is shown;

[0021] Although the present disclosure is susceptible to various modifications and alternative forms, specific embodiments and examples have been shown by way of example in the accompanying drawings and will be described in detail herein. However, it should be understood that the present disclosure is not intended to be limited to the specific forms disclosed. On the contrary, the present disclosure is intended to cover all modifications, equivalents and alternatives that fall within the spirit and scope of the present disclosure defined by the appended claims. DETAILED DESCRIPTION

[0022] Although the present disclosure has been described with reference to one or more specific embodiments or implementations, those skilled in the art will recognize that many changes may be made to the present disclosure without departing from the spirit and scope of the present disclosure. Each of these embodiments or implementations and obvious variations thereof are considered to fall within the spirit and scope of the present disclosure. It is also contemplated that additional embodiments according to various aspects of the present disclosure may be combined with any number of features of any embodiment or implementation described herein.

[0023] Magnetic resonance-based imaging (MR imaging) is the most commonly used technique for imaging the human body, and it takes into account the principle of nuclear magnetic resonance. For example, doctors and other medical professionals often use MR imaging to observe tissues in the human body. Nuclear magnetic resonance is a phenomenon in which atomic nuclei (e.g., protons in human tissue) located in a magnetic field emit energy that can be detected. This detected energy can be used to create an image. MR imaging generally involves two main steps. First, the magnetic moments of the nuclei (vector properties of the nuclei caused by the intrinsic spin properties of the elementary particles) are aligned (or polarized) due to the presence of an external magnetic field. In the presence of such an external magnetic field, the magnetic moment of each nucleus will generally precess around an axis parallel to the magnetic field. The rate of this precession, ω, is generally proportional to γB0, where B0 is the magnitude of the external magnetic field, and γ is the gyromagnetic ratio of the nucleus, which is the ratio of the magnetic moment of the nucleus to the angular momentum of the nucleus. The rate of precession, ω, is considered to be the resonant frequency of the nucleus.

[0024] The second major step of MR imaging is to apply a sequence of electromagnetic pulses (usually radio frequency pulses or RF pulses) to the nuclei. When the frequency of the RF pulse sequence is roughly equal to the resonant frequency of the nuclei, the nuclei absorb the energy of the RF pulses and the magnetic moment rotates out of alignment with the magnetic field. The magnetic moments of the excited nuclei eventually realign in the presence of an external magnetic field in a process called relaxation, which has two components, T1 and T2. T1 relaxation describes how the component of the magnetic moment parallel to the external magnetic field returns to its initial value. T2 relaxation describes how the component of the magnetic moment perpendicular to the external magnetic field returns to its initial value. Since the magnetic moment of the nuclei in an external magnetic field is usually parallel to the external magnetic field without applying an RF pulse sequence, T1 relaxation usually describes how the parallel component of the magnetic moment returns to its maximum value, while T2 relaxation usually describes how the perpendicular component of the magnetic moment decays. Nuclei of different materials relax at different rates and therefore emit different signals, which can be detected and can be used to form images for identifying different materials.

[0025] Dynamic MR is able to produce a sequence of spatiotemporal images l(x,t) that is a function of (i) the spatial position within the object and (ii) one or more time-varying parameters associated with the dynamic process. The spatial position is represented by the vector x = [x1, x2, x3] T represents x=[x1,x2,x3] T , which contains up to three space-varying parameters x i The time-varying parameters consist of R time-varying independent variables t i The vector t=[t1,t2,…,t R ] T Representation. The image sequence l(x,t) can be represented as element Aij = l(x i ,t j ) of the matrix A∈C MxN l(x,t), where the matrix A has M spatial locations (voxels) and N time points (frames). The matrix A is spatially encoded by the MR scanner, which produces a vector of encoded data d=E(A)d=E(A), where E(·) typically includes at least partial spatial Fourier encoding and additional spatial encoding based on the receiver coil sensitivity pattern. In order to recover the image sequence A from the measured data d, the operation f must be determined such that A=f(d). In general, it is not possible to sample the data d at or above the spatiotemporal Nyquist rate, so the data d is undersampled. This leads to an ill-posed inverse problem, resulting in the general formula f(·)=E -1 (·) does not exist. Therefore, it is difficult to directly operate on the data d to obtain the matrix A of the image from these incomplete data.

[0026] Reference now Figure 1 , a method 100 for performing magnetic resonance imaging on an object and generating a desired image sequence is shown. In step 102, imaging data of an object (which may be a human or non-human such as a patient in a medical institution) is collected using an MRI device. Multiple different spatial encodings are typically used to obtain the imaging data. Typically, the imaging data comes from a specific region of interest of the object. In an example, the region of interest may be the abdomen or chest of the object. In other examples, the region of interest of the object is more specific. For example, the region of interest may be an organ such as the liver, lungs, heart, pancreas, brain, prostate, breast, or any other organ of the object. The imaging data depends on or is related to the spatially varying parameters and time-varying parameters of the region of interest of the above-mentioned object. These parameters can include parameters such as T1, T 1ρ The parameters may also include relaxation parameters such as T1-rho, T2 or T2* (also known as T2-star). The parameters may also include contrast agent kinetic parameters, diffusion parameters (including change in intensity, change in direction or both), elastic imaging wave propagation, dynamic contrast enhancement, magnetization transfer, chemical exchange saturation transfer, free induction decay, flow, cardiac motion, respiratory motion or general time lapse.

[0027] Parameters can vary with space or with time, and some parameters can vary with both time and space. For example, cardiac motion is generally a time-varying parameter, while relaxation parameters, contrast agent kinetic parameters, and diffusion parameters are generally time-varying. Typically, the imaging data indicates the value or magnitude of a space-varying parameter and / or a time-varying parameter.

[0028] In an example, the region of interest is the abdomen of the subject including the liver, and the space-variant parameter being measured is the T1 relaxation parameter. The T1 relaxation parameter can vary with space, which means that the value of the T1 relaxation parameter at a first physical position within the subject's liver can be different from the value of the T1 relaxation parameter at a second physical position within the subject's liver. In the resulting image showing the values ​​of the T1 relaxation parameter measurements, different positions in the image (corresponding to different physical positions within the subject's liver) will display different values. In some embodiments, the space-variant parameter can also vary with time. In other embodiments, the space-variant parameter can additionally or alternatively be related to the physical movement of the region of interest of the subject. In general, the technology disclosed herein can be used to perform dynamic imaging that resolves parameters that can vary with space and time.

[0029] In order to collect imaging data, an RF pulse sequence is applied to the region of interest of the object. The generated signal is then measured. Therefore, the RF pulse sequence can be designed by using a saturation recovery pulse for measuring T1 relaxation, a T2 pre-pulse for measuring T2 relaxation, or any other specific pulse sequence for measuring a specific space-varying parameter or a combination of space-varying parameters. There is usually a pause after each RF pulse in the RF pulse sequence to allow the generated signal to be measured. In some embodiments, imaging data is collected in a semi-continuous mode. In this semi-continuous mode, the RF pulse sequence includes an additional pause to allow the T1 signal to recover (because the measurement can reduce / saturate the T1 signal). In other embodiments, imaging data is collected in a continuous mode, in which the applied RF pulse sequence is not paused to allow the T1 signal to recover, and only includes a pause for allowing the generated signal to be measured. In any embodiment, the applied RF pulse sequence is not paused to wait for any specific breathing position, but allows the object to breathe freely.

[0030] In step 104, auxiliary data is obtained from the object. The auxiliary data is related to one or more time-varying parameters of the region of interest of the object and generally indicates the value or magnitude of the time-varying parameters. In some embodiments, the auxiliary data and the imaging data are collected simultaneously. In other embodiments, the auxiliary data is collected before or after the imaging data. Generally, the auxiliary data (sometimes referred to as subspace training data or navigator data) is a subset of d. For example, the auxiliary data is obtained using only some different spatial encodings. The time-varying parameters measured by the auxiliary data can be related to the physical locations of different structures within the object. For example, one time-varying parameter is the position of the chest or abdomen of the object during the breathing cycle, e.g., when the patient is breathing. In some embodiments, the auxiliary data can take the form of an external respiratory gating signal. In this embodiment, the time-varying parameter measured by the auxiliary data is the chest wall position. In other embodiments, the auxiliary data can take the form of a respiratory navigator signal, which can measure the position of the dome of the liver of the object. In additional embodiments, the auxiliary data can take the form of training data interleaved with the imaging data. In these embodiments, the training data is generally a subset of the imaging data (which measures the spatial-varying parameters), and the difference is only in its acquisition time. This subset of the imaging data is generally measured at a rate sufficient to resolve any motion of the region of interest.

[0031] As described herein, it is difficult to directly operate on the data d to obtain the matrix A of the image. However, due to the strong correlation between different image frames, the following linear subspace modeling can be used to represent the image sequence: When the rank of the matrix A is low, i.e., L < min(M, N), this model can generally be used. Using this model, the matrix A can be decomposed as A = UΦ, where U ∈ C MxL with elements U il = u l (x i ), and Φ ∈ C LxN with elements

[0032] In this formula, U is called the spatial factor. In some embodiments, this formula is used for dynamic imaging with only one time dimension. In this case, U is a spatial factor matrix. In other embodiments, this formula is used for dynamic imaging with multiple time dimensions. In this case, U is a spatial factor tensor. The spatial factor contains one or more spatial weighting functions that describe the properties of the spatial dimension x i . Similarly, Φ is called the temporal factor, which contains one or more temporal basis functions that describe the properties of the various time dimensions t i . The set of spatial weighting functions in the spatial factor U is called the spatial feature map. Therefore, the columns of the spatial factor U are the ones that contain the The feature map of the coordinates of l(x,t) in the covered temporal feature space.

[0033] Referring to step 106, the time domain basis functions can be extracted directly from the auxiliary data using techniques such as principal component analysis. As noted, each time domain basis function typically corresponds to at least one time-varying dimension of the object. Since the time domain basis functions can be extracted directly from the data d, the time factor Φ is predetermined and the problem formulation is updated to d = E(UΦ) = E Φ (U). Therefore, once the time domain basis functions are extracted, generating the desired image sequence requires identifying a function f Φ , so that f Φ (d)≈U, for example, solve for U.

[0034] In some cases, sparse recovery methods such as compressed sensing can be used to find U which itself has a sparse representation Ψ(U), for example by solving a nonlinear reconstruction problem It can be achieved by back-projecting d onto the feature space (e.g., one or more spatial weighting functions) as (where * indicates concomitant) or as a precondition This equation is then solved for U by performing a nonlinear iterative reconstruction of the result entirely in the temporal feature space, such as the alternating direction method of multipliers. However, this process can be quite slow, especially for non-Cartesian sampling patterns where E Φ (·)E Φ (·) Including one or more irreversible, non-separable multi-dimensional non-uniform fast Fourier transforms instead of reversible, separable fast Fourier transforms.

[0035] Now referring to steps 108 and 110, a neural network can be used to obtain a spatial factor U containing a spatial weighting function. In step 108, a preliminary feature map U0 is extracted from the imaging data d, for example, a preliminary spatial weighting function is extracted. In some embodiments, Among them, Ω * (·)Φ H Ω * (·)Φ H Convert imaging data into temporal feature space, By applying a density compensation function (diagonal matrix W) and the accompanying non-uniform fast Fourier transform (Similar to the process of filtered back projection) Regridding of non-Cartesian data and pseudo-inverse matrix Complex coil combinations are performed. Typically, these preliminary spatial weighting functions contain some type of artifacts or corruption that makes them unsuitable for generating the desired image sequence.

[0036] In step 110, the preliminary spatial weighting function (e.g., preliminary feature map) is input into a neural network including a specific architecture with various parameters. In some embodiments, the neural network is a multi-channel neural network, such as an expanded multi-level densely connected network, an autoencoder (AE) network, or other neural networks such as DenseUnet, Unet, AlexNet, ResidualNetwork (ResiNet) and NIN (Network in Network). Generally, any suitable neural network can be used. In some embodiments, the neural network includes one or more densely connected blocks, which include multiple convolutional (Conv) layers, multiple activation functions and a certain growth rate. The convolution layer can be a 3x3 convolution layer. The activation function can be an exponential linear unit. The growth rate of the densely connected block can be 128. Generally, a variety of different neural networks can be used.

[0037] The neural network processes the preliminary spatial weighting functions according to the parameters of its architecture and the weights of these parameters. Typically, the various parameters refer to the different types of operations being used, and the weights refer to the scale of these operations, the thresholds used, or any other adjustable properties of the parameters. The output of the neural network is one or more final spatial weighting functions. Typically, each final spatial weighting function is an artifact-free (e.g., complete) version of the corresponding preliminary spatial weighting function. As a result, the neural network is able to learn efficient image representations and quickly decode image data, even for image sequences of more than 40,000 frames.

[0038] Finally, in step 112, the final spatial weighting functions are multiplied with the temporal basis functions in any desired sequence or manner to generate the desired image sequence. In some embodiments, the neural network itself multiplies the final spatial weighting functions with the temporal basis functions, in which case the temporal basis functions must also be input to the neural network. In other embodiments, this multiplication is performed by some other entity, such as a separate processing device, or a separate software or hardware module.

[0039] In order to properly utilize a neural network, it must first be trained using reference data. Figure 2A method 200 for training a neural network used in MR image reconstruction is shown in FIG. In method 200, the neural network is trained using training data, which typically includes a set of training preliminary spatial weighting functions and a set of training final spatial weighting functions. The training final spatial weighting functions are typically artifact-free versions of the training preliminary spatial weighting functions that have been obtained using methods other than neural networks (e.g., traditional iterative reconstruction). The goal of training the neural network is to have the neural network learn how to process the training preliminary spatial weighting functions to produce the training final spatial weighting functions.

[0040] In step 202, the neural network receives training preliminary spatial weighting functions that are related to various time-varying dimensions of the object being imaged. In step 204, the neural network processes the training preliminary spatial weighting functions to produce estimated final spatial weighting functions. In step 206, the neural network compares the estimated final spatial weighting functions to the training final spatial weighting functions, which are typically the "targets" of the neural network.

[0041] In step 208, the neural network determines whether its cost function is satisfied. The cost function (also called the loss function) of the neural network describes how closely the current output of the neural network matches the desired output that the neural network is trying to reproduce. Therefore, in method 200, the cost function describes how closely the estimated final spatial weighting function matches the training final spatial weighting function. As shown in step 208, after generating the estimated final spatial weighting function and comparing them with the training final spatial weighting function, the neural network determines whether the cost function is satisfied, for example, whether the estimated final spatial weighting function is similar enough to the training final spatial weighting function. The difference between the estimated final spatial weighting function and the training final spatial weighting function is called the loss of the network.

[0042] If the cost function has not been met, the method 200 proceeds to step 210, where the various weights of the neural network are adjusted according to the cost function. Typically, the architecture of a neural network includes a variety of different filters, operations, activations, etc. Each of these different processing steps can have different properties, such as which filter is applied, how the various inputs are multiplied, etc. The values ​​associated with the processing steps are called the "weights" of the neural network architecture. These weights can be adjusted to modify the output of the neural network. The neural network then repeats steps 204, 206, and 208 to (i) generate a new set of estimated final spatial weighting functions; (ii) compare the new set of estimated final spatial weighting functions to the training final spatial weighting functions; and (iii) determine whether the adjusted network weights meet the cost function. This process is repeated until the cost function is met, at which point the neural network has been fully trained.

[0043] Many different optimization techniques or algorithms can be used to train a neural network. In one embodiment, a gradient descent method is used. The gradient is defined as the direction in which the network loss increases. Therefore, to update the weights of the parameters, we use Apply negative gradients to the network parameters, where (x i ,x i )(x i ,x i ) is the data point of the input data set, and θ is the specific parameter whose weight is being updated. In other embodiments, other optimization techniques or algorithms can be used, such as mini-batch stochastic gradient descent, momentum optimizer (which uses the previous gradient as momentum), Adagrad and Adadelta (adaptive gradient descent, in which the learning rate varies for each weight according to the gradient history), RMSProp (root mean square propagation, which uses a moving average of the gradient to improve stability), and ADAM (adaptive moment estimation, which applies the first and second moments separately for each parameter, and which is a combination of Adagrad and RMSProp).

[0044] In some embodiments, the cost function of the neural network is configured to minimize the difference between the one or more trained final spatial weighting functions and the one or more estimated final spatial weighting functions. In other embodiments, the cost function is configured to minimize the difference between a training image sequence based on the one or more trained final spatial weighting functions and an estimated image sequence based on the one or more estimated final spatial weighting functions.

[0045] Neural networks can be implemented using a variety of different systems or platforms, such as Tensorflow, Keras, Pytorch, or CNTK. The input data sets for training can be instance normalized by subtracting their mean and dividing by the standard deviation before being input to the neural network. In some embodiments, the cost function is an Adam optimizer that minimizes the L1, L2, or mixed L1-L2 loss between the input data set and the network output. In some embodiments, different L1 and L2 regularizations can be applied to the network weights to avoid overfitting.

[0046] Figure 3A and Figure 3B Two different neural networks are shown that can be used in conjunction with aspects of the present disclosure. Figure 3A shows an expanded multi-stage densely connected network 302, and Figure 3B DenseUnet 304 is shown. Both neural networks 302 and 304 can have one or more densely connected blocks (or "DenseBlocks") 306, examples of which are shown in FIG. Figure 3C shown.

[0047] In one example, neural networks 302 and 304 are trained using an image matrix with a rank of L=3232 and a matrix size of 160×160. In this example, neural networks 302 and 304 are tasked with obtaining a spatial factor matrix U consisting of 32 complex-valued 160×160 feature maps. In some examples of the training process, the real and imaginary parts of U can be concatenated into a set of 64 real-valued 160×160 feature maps.

[0048] In some embodiments, a domain discriminator is also used to train the neural network. The domain discriminator trains the neural network to be robust to data from domains other than the original MRI data. In these embodiments, the training input data set includes the original MRI data and non-MRI data. In one example, the non-MRI data is a general video data set including a large number of video clips, such as ImageNet-VID. During the training process (e.g., method 200), the neural network is trained to extract the final spatial weighting function (e.g., feature map) from the MRI data and the non-MRI data until the neural network produces a final spatial weighting function of sufficient quality from both data sets.

[0049] The training is performed in an alternating manner. First, the neural network is trained to distinguish between the final spatial weighting function generated by the MRI data and the final spatial weighting function generated by the non-MRI data. Then, the neural network is trained to produce a final spatial weighting function that is as close as possible to the training final spatial weighting function obtained using conventional methods. Then, the neural network is trained again to distinguish between the MRI final spatial weighting function and the non-MRI final spatial weighting function. These two training schemes are performed alternately until the neural network generates a high-quality final spatial weighting function for MRI data and non-MRI data, so that the neural network cannot distinguish between the generated final spatial weighting functions originating from different sources. This ensures that the neural network can show the same performance on different image domains, so the neural network is more robust to unseen data.

[0050] Table 1 below shows various different measures of the performance of the mDCN neural network 302 and the DenseUnet 304 compared to conventional iterative reconstruction methods (e.g., non-neural network methods) for determining the final spatial weighting function. The different measures include: (i) the normalized root mean square error (NRSMSE) of the spatial weighting function; (ii) 3 different image similarity metrics (SSIM, PSNR, and NRMSE) for the reconstructed image sequence (920 frames) for the entire cardiac cycle within the inversion time corresponding to the bright blood and dark blood contrast weights in the end-expiratory (EE) breathing phase; and (iii) the accuracy and precision of the T1 map generated from the final spatial weighting function. Each of these measurements is performed at three different L1 and L2 regularization scales. The running time of each neural network is also shown.

[0051]

[0052] Table 1

[0053] As can be seen from Table 1, the mDCN network 302 is generally better than the DenseUnet 304 and the traditional methods.

[0054] Table 2 shows a comparison between different dilation rates of the mDCN network 302 and the DenseUnet 304 .

[0055]

[0056] Table 2

[0057] Figure 4A and Figure 4B An example of a reconstructed MR image showing multiple contrast and time dimensions reconstructed using the mDCN network 302 and conventional techniques is shown. The MR image varies with inversion time (vertical) representing T1 recovery and cardiac phase (horizontal) representing heart motion. Figure 4A shows an MR image during the systolic phase of the heart, and Figure 4B An MR image during the diastolic phase of the heart is shown. Figure 4A Images 402A, 402B, 402C, and 402D in FIG. 4 are obtained using conventional iterative reconstruction techniques. Figure 4A The images 404A, 404B, 404C, and 404D in FIG. 1 are obtained using the mDCN network 302 . Figure 4B Images 406A, 406B, 406C, and 406D in FIG. 4 are obtained using conventional iterative reconstruction techniques. Figure 4B The images 408A, 408B, 408C, and 408D in FIG. 4 are obtained using the mDCN network 302 .

[0058] Figure 5Various example T1 maps constructed using conventional iterative reconstruction techniques (graph 502A), mDCN network 302 (graph 502B), and DenseUnet network 304 (graph 502C) are shown. Figure 5 Bland-Altman curves of the T1 fitting results are also shown. Curve 504A shows the difference between the mDCN network 302 and the traditional iterative reconstruction technique. Curve 504B shows the difference between the DenseUnet network 304 and the traditional iterative reconstruction technique. In general, the mDCN T1 map is more accurate (e.g., shows smaller deviations) and more precise (e.g., has stricter consistency limits) than the DenseUnet T1 map and the traditional T1 map. The T1 maps from the mDCN network 302 and the DenseUnet network 304 do not show statistically significant deviations. The T1 map from the mDCN network 302 is slightly correlated with the T1 map from the traditional technique.

[0059] Aspects of the present disclosure can be implemented using a variety of hardware. Figure 6 Such an embodiment is shown. A system 600 for performing magnetic resonance imaging on an object includes an imaging device 602, a processing device 612, and a reconstruction workstation 622. The imaging device 602 can be used for standard magnetic resonance imaging and can include a main magnet 604, a gradient coil 606, an RF transmission system 608, and an RF receiving system 610. The main magnet 604 can be a permanent magnet, an electromagnet (e.g., a coil), or any other suitable magnet. The main magnet 604 is used to generate an external magnetic field applied to the sample during imaging. The gradient coil 606 generates a secondary magnetic field that distorts the external magnetic field and can cause the resonant frequency of the protons in the sample to vary with position. Therefore, the gradient coil 606 can be used to spatially encode the position of the protons in the entire sample, for example, it can be used to select which plane intersecting the sample is used for imaging. The RF transmission system 608 is used to apply an RF pulse sequence that provides energy to the protons in the sample to rotate their magnetic moments, thereby misaligning the magnetic moments with the external magnetic field and saturating the solute material with protons. The RF transmission system 608 typically includes a frequency generator (e.g., an RF synthesizer), a power amplifier, and a transmission coil. The RF receiving system 610 receives signals emitted by the protons in the sample when they relax back to their standard alignment. The RF receiving system 610 can include a receiving coil that receives the emitted signals and a preamplifier for enhancing the received signals and ensuring that the signals are suitable for processing. In some embodiments, the RF receiving system 610 can include a signal processing component that processes the received signals to provide data that can be used by a processing device 612. Each component of the imaging device can be disposed within one or more housings.

[0060] The processing device 612 can be communicatively connected to the imaging device 602 and can include a processor 614, a memory 616 executable by the processor, a display 618, and a user input device 620. The processing device 612 is used to manage the operation of the imaging device 602, and thus can be configured to cause the imaging device 602 to perform dynamic imaging according to the principles disclosed herein. The memory 616 can contain instructions that, when executed by the processor 614, cause the imaging device 602 to operate as desired. The memory 616 can also store data obtained from the MRI sequence.

[0061] The reconstruction workstation 622 is typically a separate processing device or system that receives the imaging data from the processing device 612. The reconstruction workstation can be configured as desired to perform any analysis of the data, including any or all of the steps of method 100 and method 200. In some embodiments, the neural network is implemented on the reconstruction workstation 622. In other embodiments, the neural network is implemented on separate hardware that is capable of communicating with the reconstruction workstation 622.

[0062] In some embodiments, a non-transitory machine-readable medium has instructions stored on the medium for implementing any method or any process discussed herein. The machine processor is configured to execute the instructions in order to perform these methods or processes.

[0063] Aspects of the present disclosure can be implemented on various types of processing devices, such as: a general-purpose computer system; a microprocessor; a digital signal processor; a microcontroller; an application-specific integrated circuit (ASIC); a programmable logic device (PLD); a field programmable logic device (FPLD); a programmable gate array (PGA); a field programmable gate array (FPGA); a mobile device such as a mobile phone, a personal digital assistant (PDA), or a tablet computer; a local server; a remote server; or a wearable computer, etc.

[0064] The memory storage device of one or more processing devices can include a machine-readable medium on which is stored one or more sets of instructions (e.g., software) embodying any one or more of the methods or functions described herein. These instructions can also be sent or received over a network via a network transceiver. Although the machine-readable medium can be a single medium, the term "machine-readable medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database; and / or associated caches and servers) storing one or more sets of instructions. The term "machine-readable medium" can also be considered to include any medium that can store, encode, or carry a set of instructions executed by a machine and cause the machine to perform any one or more methods of various embodiments, or it can store, encode, or carry a data structure used by or associated with such a set of instructions. Therefore, the term "machine-readable medium" can be considered to include, but is not limited to, solid-state memory, optical media, and magnetic media. A memory or memories can use various different types of memory storage devices such as random access memory (RAM) or read-only memory (ROM) in a system; or a floppy disk, hard disk, CD ROM, DVD ROM, flash memory; or other computer readable media that is read and / or written by a magnetic, optical or other read and / or write system connected to a processing device.

[0065] Although various aspects of the present disclosure have been described with reference to one or more specific embodiments, those skilled in the art will recognize that many changes may be made to the present disclosure without departing from the spirit and scope of the present disclosure. These embodiments and obvious variations thereof are considered to fall within the spirit and scope of the present disclosure. It is also contemplated that additional embodiments according to various aspects of the present disclosure may be combined with any number of features of any embodiment described herein.

[0066] CROSS-REFERENCE TO RELATED APPLICATIONS

[0067] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 900,279, filed on September 13, 2019, the entire contents of which are incorporated herein by reference.

[0068] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0069] This invention was made with government support under Grant No. R01 EB028146 awarded by the National Institutes of Health. The government has certain rights in this invention.

Claims

1. A method for performing magnetic resonance (MR) imaging on a subject, the method comprising: deriving one or more temporal basis functions and one or more preliminary spatial weighting functions from undersampled imaging data from a region of interest of the object, wherein the undersampled imaging data corresponds to an image sequence having a plurality of frames, wherein each of the one or more temporal basis functions corresponds to at least one time-varying dimension of the object, and wherein each of the one or more preliminary spatial weighting functions corresponds to a space-varying dimension of the object; and inputting the one or more preliminary spatial weighting functions into a neural network to generate one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a corresponding one of the one or more preliminary spatial weighting functions, wherein the one or more final spatial weighting functions are artifact-free versions of the one or more preliminary spatial weighting functions, and The one or more final spatial weighting functions can be multiplied with the one or more time domain basis functions to generate the image sequence.

2. The method according to claim 1, wherein: The neural network is a multi-channel neural network.

3. The method according to claim 1, wherein: The neural network is an expanded multi-level densely connected network.

4. The method according to claim 1, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks includes one or more convolutional layers.

5. The method according to claim 1, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks including one or more activation functions.

6. The method according to claim 5, wherein: At least one of the one or more activation functions is an exponential linear unit.

7. The method according to claim 1, wherein: The neural network includes one or more dense blocks.

8. The method according to claim 1, wherein: The undersampled imaging data is acquired using a plurality of spatial encodings, and wherein the undersampled imaging data comprises training data acquired with a subset of the plurality of spatial encodings, and wherein the one or more time domain basis functions are extracted from the training data.

9. The method according to claim 1, wherein: The subject is a human being.

10. A method of generating a neural network for reconstructing magnetic resonance (MR) data of an object, the method comprising: receiving a plurality of training data sets to the neural network, the plurality of training data sets comprising a plurality of sets of MR training data and a plurality of sets of non-MR training data, each of the plurality of sets of MR training data comprising a set of one or more sets of MR training preliminary spatial weighting functions and a set of one or more sets of MR training final spatial weighting functions, each of the plurality of sets of non-MR training data comprising a set of one or more sets of non-MR training preliminary spatial weighting functions and a set of one or more sets of non-MR training final spatial weighting functions; as well as The neural network is trained by alternating between a first training process and a second training process, wherein: The first training process includes training the neural network to distinguish between (i) the one or more sets of MR trained final spatial weighting functions and (ii) the one or more sets of non-MR trained final spatial weighting functions, and The second training process includes training the neural network to (i) generate estimated versions of the one or more sets of MR training final spatial weighting functions based on the one or more sets of MR training preliminary spatial weighting functions and (ii) generate estimated versions of the one or more sets of non-MR training final spatial weighting functions based on the one or more sets of non-MR training preliminary spatial weighting functions.

11. The method according to claim 10, wherein: The non-MR training data includes video data.

12. The method according to claim 10, wherein: One or more methods other than the use of a neural network are used to obtain a final spatial weighting function of the one or more sets of MR trained final spatial weighting functions and the one or more sets of MR trained final spatial weighting functions.

13. The method according to claim 12, wherein: The one or more methods include iterative reconstruction.

14. The method according to claim 10, wherein: Each of the estimated versions of the one or more sets of MR trained final spatial weighting functions corresponds to a space-varying dimension of the object, and wherein an image sequence can be generated by multiplying the estimated versions of the one or more sets of MR trained final spatial weighting functions with one or more time-domain basis functions each corresponding to at least one time-varying dimension of the object.

15. The method according to claim 10, wherein: The neural network is a multi-channel neural network.

16. The method according to claim 10, wherein: The neural network is an expanded multi-level densely connected network.

17. The method according to claim 10, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks includes one or more convolutional layers.

18. The method according to claim 10, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks including one or more activation functions.

19. The method according to claim 14, wherein: At least one of the one or more activation functions is an exponential linear unit.

20. The method according to claim 10, wherein: The neural network includes one or more dense blocks.

21. A non-transitory machine-readable medium storing instructions for performing magnetic resonance (MR) imaging on a subject, the instructions causing the method of any one of claims 1-20 to be performed when the instructions are executed by at least one processor.

22. A neural network for reconstructing magnetic resonance (MR) data of a subject, the neural network being trained to receive one or more preliminary spatial weighting functions from undersampled imaging data and to generate one or more final spatial weighting functions from the preliminary spatial weighting functions, wherein: Each of the one or more preliminary spatial weighting functions corresponds to a space-varying dimension of the object, and wherein each of the one or more final spatial weighting functions is an artifact-free version of a corresponding one of the one or more preliminary spatial weighting functions.

23. The neural network of claim 22, wherein: An image sequence can be generated by multiplying the one or more final spatial weighting functions with one or more temporal basis functions each corresponding to at least one time-varying dimension of the object.

24. The neural network of claim 23, wherein: The neural network is trained to multiply the one or more final spatial weighting functions with the one or more temporal basis functions to generate the image sequence.

25. The neural network of claim 22, wherein: The neural network is a multi-channel neural network.

26. The neural network of claim 22, wherein: The neural network is an expanded multi-level densely connected network.

27. The neural network of claim 22, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks includes one or more convolutional layers.

28. The neural network of claim 22, wherein: The neural network includes one or more dense blocks, at least one of the one or more dense blocks including one or more activation functions.

29. The neural network of claim 28, wherein: At least one of the one or more activation functions is an exponential linear unit.

30. The neural network of claim 22, wherein: The neural network includes one or more dense blocks.

Images