Deep learning techniques for magnetic resonance image reconstruction

Abstract

A magnetic resonance imaging (MRI) system comprises: a magnetic system, including: a B0 magnet, which is configured to provide a B0 field for the MRI system; a gradient coil, which is configured to provide a gradient field for the MRI system; and at least one RF coil, which is configured to detect magnetic resonance (MR) signals; and a controller, which is configured to: control the magnetic system to acquire MR spatial frequency data using non-Cartesian sampling; and generate an MR image based on the acquired MR spatial frequency data using a neural network model, the neural network model including one or more neural network blocks, the one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to use non-uniform Fourier transform for data consistency processing.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] 119(e) of Title 35, United States Code, this application claims the benefit of U.S. Provisional Application Serial No. 62 / 711,895, filed on July 30, 2018, with Attorney Docket No. O0354.70028US00, and entitled “DEEP LEARNING TECHNIQUES FOR MAGNETIC RESONANCE IMAGE RECONSTRUCTION”; U.S. Provisional Application Serial No. 62 / 737,524, filed on September 27, 2018, with Attorney Docket No. O0354.70028US02, filed on October 11, 2018, and entitled “DEEP LEARNING TECHNIQUES FOR MAGNETIC RESONANCE IMAGE RECONSTRUCTION”; and U.S. Provisional Application Serial No. 62 / 737,524, filed on September 27, 2018, with Attorney Docket No. O0354.70028US03, filed on October 11, 2018, and entitled “DEEP LEARNING TECHNIQUES FOR MAGNETIC RESONANCE IMAGE No. 62 / 744,529, entitled “END-TO-END LEARNABLE MR IMAGE RECONSTRUCTION”; and priority to U.S. Provisional Application Serial No. 62 / 820,119, filed on March 18, 2019, with Attorney Docket No. O0354.70039US00, and entitled “END-TO-END LEARNABLE MR IMAGE RECONSTRUCTION,” each of which is incorporated herein by reference in its entirety. Background Art

[0003] Magnetic resonance imaging (MRI) provides an important imaging modality for many applications and is widely used in clinical and research settings to produce images of the interior of the human body. MR is based on detecting magnetic resonance (MR) signals, which are electromagnetic waves emitted by atoms in response to a change of state caused by an applied electromagnetic field. For example, nuclear magnetic resonance (NMR) technology involves detecting MR signals emitted from the nuclei of excited atoms when the nuclear spins of atoms in the imaged object (e.g., atoms in human tissue) realign or relax. The detected MR signals can be processed to produce images, which enables the investigation of internal structures and / or biological processes in the body for diagnostic, therapeutic and / or research purposes in the context of medical applications.

[0004] MRI provides an attractive imaging modality for biological imaging due to the ability to produce non-invasive images with relatively high resolution and contrast without the safety issues of other modalities (e.g., without exposing the subject to ionizing radiation (such as x-rays) or introducing radioactive materials into the body). In addition, MRI is particularly well suited to provide soft tissue contrast, which can be used to image subjects that cannot be satisfactorily imaged with other imaging modalities. In addition, MR technology is able to capture information related to structures and / or biological processes that cannot be obtained by other modalities. However, conventional MRI technology has many disadvantages, which for a given imaging application may include the relatively high cost of the equipment, limited availability (e.g., the difficulty and expense of obtaining access to a clinical MRI scanner), and the length of the image acquisition process.

[0005] In order to improve the imaging quality, the trend of clinical and research MRI has been to increase the field strength of the MRI scanner to improve one or more specifications of the scan time, image resolution and image contrast, which in turn drives the cost of MRI imaging up. The vast majority of the installed MRI scanners use at least 1.5 or 3 Tesla (T) to work, where 1.5 or 3 Tesla refers to the field strength of the main magnetic field B0 of the scanner. The rough cost of a clinical MRI scanner is estimated to be about one million US dollars per Tesla, which does not even include the large amount of operating, service and maintenance costs involved in operating such an MRI scanner. In addition, traditional high-field MRI systems generally require large superconducting magnets and related electronic devices to generate a strong uniform static magnetic field (B0) for imaging the subject (e.g., patient). Superconducting magnets further require cryogenic equipment to keep the conductor in a superconducting state. The size of this system is quite large, and a typical MRI device includes multiple rooms for magnetic components, electronic components, thermal management systems and console areas (including special shielded rooms to isolate the magnetic components of the MRI system). The size and cost of MRI systems generally limit their use to facilities such as hospitals and academic research centers, which have sufficient space and resources to purchase and maintain them. The high cost and substantial space requirements of high field MRI systems result in limited availability of MRI scanners. As such, there are often clinical situations where an MRI scan would be beneficial, but is impractical or impossible due to the limitations noted above and as described in further detail below. Summary of the invention

[0006] Some embodiments relate to a method comprising: using a neural network model to generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, the neural network model comprising: a first neural network sub-model configured to process spatial frequency domain data; and a second neural network sub-model configured to process image domain data.

[0007] Some embodiments relate to a system comprising: at least one computer hardware processor; and at least one non-transitory computer-readable storage medium storing processor-executable instructions which, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to: generate an MR image based on magnetic resonance spatial frequency data, i.e., MR spatial frequency data, using a neural network model, the neural network model comprising: a first neural network portion configured to process data in a spatial frequency domain; and a second neural network portion configured to process data in an image domain.

[0008] Some embodiments relate to at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the following operations: generate an MR image based on magnetic resonance spatial frequency data, i.e., MR spatial frequency data, using a neural network model, the neural network model comprising: a first neural network portion configured to process data in a spatial frequency domain; and a second neural network portion configured to process data in an image domain.

[0009] Some embodiments relate to a method comprising the following steps: using a neural network model to generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, wherein the neural network model includes a neural network sub-model, which is configured to process spatial frequency domain data and has a locally connected neural network layer.

[0010] Some embodiments relate to a system comprising: at least one processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions, which, when executed, cause the at least one processor to perform the following operations: using a neural network model to generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, the neural network model comprising a neural network sub-model, which is configured to process spatial frequency domain data and has a locally connected neural network layer.

[0011] Some embodiments relate to at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed, cause at least one processor to perform the following operations: generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, using a neural network model, wherein the neural network model includes a neural network sub-model, which is configured to process spatial frequency domain data and has a locally connected neural network layer.

[0012] Some embodiments provide at least one non-transitory computer-readable storage medium storing processor-executable instructions which, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform a method comprising: generating an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, using a neural network model, wherein the neural network model comprises one or more neural network blocks, wherein the one or more neural network blocks comprise a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transform for transforming image domain data into spatial frequency domain data.

[0013] Some embodiments provide a magnetic resonance imaging system, i.e., an MRI system, comprising: a magnetic system, including: a B0 magnet, configured to provide a B0 field for the MRI system; a gradient coil, configured to provide a gradient field for the MRI system; and at least one radio frequency coil, i.e., an RF coil, configured to detect magnetic resonance signals, i.e., MR signals; a controller, configured to: control the magnetic system to acquire MR spatial frequency data; generate an MR image based on the MR spatial frequency data using a neural network model, the neural network model comprising: a first neural network part, configured to process data in a spatial frequency domain; and a second neural network part, configured to process data in an image domain.

[0014] Some embodiments provide a magnetic resonance imaging system, i.e., an MRI system, comprising: a magnetic system, including: a B0 magnet, configured to provide a B0 field for the MRI system; a gradient coil, configured to provide a gradient field for the MRI system; and at least one radio frequency coil, i.e., an RF coil, configured to detect magnetic resonance signals, i.e., MR signals; a controller, configured to: control the magnetic system to acquire MR spatial frequency data; generate an MR image based on the input MR spatial frequency data using a neural network model, the neural network model including a neural network sub-model, the neural network sub-model being configured to process spatial frequency domain data and having a locally connected neural network layer.

[0015] Some embodiments provide a method comprising the following steps: using a neural network model to generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, the neural network model comprising one or more neural network blocks, the one or more neural network blocks comprising a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transform for transforming image domain data into spatial frequency domain data.

[0016] Some embodiments provide a system comprising: at least one computer hardware processor; and at least one non-transitory computer-readable storage medium storing processor-executable instructions, which, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform a method comprising the following operations: using a neural network model to generate an MR image based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, the neural network model comprising one or more neural network blocks, the one or more neural network blocks comprising a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transform for transforming image domain data into spatial frequency domain data.

[0017] Some embodiments provide a magnetic resonance imaging system, i.e., an MRI system, comprising: a magnetic system, including: a B0 magnet, configured to provide a B0 field for the MRI system; a gradient coil, configured to provide a gradient field for the MRI system; and at least one radio frequency coil, i.e., an RF coil, configured to detect magnetic resonance signals, i.e., MR signals; a controller, configured to: control the magnetic system to acquire MR spatial frequency data using a non-Cartesian sampling trajectory; and generate an MR image based on the acquired MR spatial frequency data using a neural network model, the neural network model including one or more neural network blocks, the one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to use a non-uniform Fourier transform for data consistency processing.

[0018] The above is a non-limiting summary of the invention as defined by the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various aspects and embodiments of the disclosed technology will be described with reference to the following drawings. It should be understood that the drawings are not necessarily drawn to scale.

[0020] Figure 1A The architecture of an exemplary neural network model for generating a magnetic resonance (MR) image from input MR spatial frequency data, according to some embodiments of the techniques described herein, is shown.

[0021] Figure 1B The architecture of another exemplary neural network model for generating MR images from input MR spatial frequency data in accordance with some embodiments of the techniques described herein is shown.

[0022] Figure 1C The architecture of yet another exemplary neural network model for generating an MR image from input MR spatial frequency data, according to some embodiments of the techniques described herein.

[0023] Figure 2A is a flow chart of an illustrative process 200 for generating an MR image from input MR spatial frequency data using a neural network model, according to some embodiments of the techniques described herein.

[0024] Figure 2B is a flow chart of an illustrative process for processing MR spatial frequency data in the spatial frequency domain (which may be part of the illustrative process 200 ) to obtain output spatial frequency data in accordance with some embodiments of the techniques described herein.

[0025] Figure 2C is a flow chart of an exemplary process for processing spatial frequency domain data (which may be part of the exemplary process 200 ) to generate an MR image in accordance with some embodiments of the techniques described herein.

[0026] Figure 2D is a flow chart of another exemplary process for processing image domain data (which may be part of the exemplary process 200 ) to generate an MR image in accordance with some embodiments of the techniques described herein.

[0027] Figure 3 Performance of techniques described herein for generating MR images from input MR spatial frequency data using a neural network model having locally connected layers for operating on data in the spatial frequency domain is shown, according to some embodiments of the techniques described herein.

[0028] Figure 4 The performance of the techniques described herein for generating MR images from input MR spatial frequency data using different embodiments of the neural network models described herein is shown.

[0029] Figure 5A The architecture of another exemplary neural network model for generating a magnetic resonance (MR) image from input MR spatial frequency data in accordance with some embodiments of the techniques described herein is shown.

[0030] Figure 5B The architecture of another exemplary neural network model for generating a magnetic resonance (MR) image from input MR spatial frequency data in accordance with some embodiments of the techniques described herein is shown.

[0031] Figure 5C The architecture of another exemplary neural network model for generating a magnetic resonance (MR) image from input MR spatial frequency data in accordance with some embodiments of the techniques described herein is shown.

[0032] Figures 6A to 6CIllustration of the distribution of weights of a fully connected network layer in a neural network sub-model configured to process spatial-frequency domain data according to some embodiments of the techniques described herein.

[0033] Figure 7 Results of generating an MR image from undersampled spatial frequency domain data sampled using a non-Cartesian sampling trajectory using the techniques described herein and a zero-filled inverse Fourier transform according to some embodiments of the techniques described herein are shown.

[0034] Figure 8 Aspects of training a neural network model for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein are shown.

[0035] Fig. 9A

[0013] Aspects of generating synthetic complex-valued images for training a neural network model for generating MR images from undersampled spatial-frequency domain data are shown according to some embodiments of the techniques described herein.

[0036] Fig. 9B A loss function having a spatial frequency domain component and an image domain component that can be used to train a neural network model for generating MR images from undersampled spatial frequency domain data is shown according to some embodiments of the techniques described herein.

[0037] Figures 10A to 10H MR images reconstructed using a zero-filled inverse discrete Fourier transform (DFT) and using a neural network model are shown, wherein the neural network model is trained with and without transfer learning according to some embodiments of the techniques described herein.

[0038] Fig.11 The performance of some neural network models for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein is shown.

[0039] Fig.12 Further illustrated is the performance of some neural network models for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein.

[0040] Fig.13A is a diagram of an illustrative architecture of an exemplary neural network model for generating MR images from input MR spatial frequency data according to some embodiments of the techniques described herein.

[0041] Fig. 13B According to some embodiments of the technology described herein, Fig.13A A diagram of one type of architecture of the blocks of a neural network model.

[0042] Fig. 13C is a data consistency block (which may be Fig. 13B A diagram of an illustrative architecture of a portion of the blocks shown.

[0043] Fig.13D is a convolutional neural network block (which may be Fig. 13B A diagram of an illustrative architecture of a portion of the blocks shown.

[0044] Fig.13E According to some embodiments of the technology described herein, Fig.13A A diagram of another type of architecture of the blocks of a neural network model.

[0045] Fig.14 is a flow chart of an illustrative process 1400 for generating an MR image from input MR spatial frequency data using a neural network model, wherein the MR spatial frequency data is obtained using non-Cartesian sampling, according to some embodiments of the techniques described herein.

[0046] Fig.15A A T1-weighted MR image reconstructed using a conventional neural network model is shown, along with a neural network model according to some embodiments of the techniques described herein.

[0047] Fig. 15B A T2-weighted MR image reconstructed using a conventional neural network model, and a neural network model according to some embodiments of the techniques described herein are shown.

[0048] Fig. 15C MR images reconstructed by a neural network model at different processing stages are shown according to some embodiments of the techniques described herein.

[0049] Fig.16 is a schematic diagram of a low-field MRI system according to some embodiments of the technology described herein.

[0050] Fig.17A and 17B A dual-planar permanent magnet structure of a B0 magnet is shown, according to some embodiments of the technology described herein.

[0051] Fig.18A and 18B A diagram showing a portable MRI system according to some embodiments of the technology described herein.

[0052] Fig. 18C A portable MRI system performing a head scan is shown in accordance with some embodiments of the techniques described herein.

[0053] Fig.18D A portable MRI system performing knee scans is shown in accordance with some embodiments of the technology described herein.

[0054] Fig.19 is a diagram of an illustrative computer system in which the embodiments described herein may be implemented. DETAILED DESCRIPTION

[0055] Conventional magnetic resonance imaging techniques require time-consuming MRI scans of patients in a closed room to obtain high-resolution cross-sectional images of the patient's anatomical structures. Long scan durations limit the number of patients that can be scanned using an MR scanner, cause patient discomfort, and increase the cost of the scan. The inventors have developed techniques for generating medically relevant, clinically accepted MRI images from MRI scans of shorter duration, thereby improving conventional MRI techniques.

[0056] The duration of an MRI scan is proportional to the number of data points acquired in the spatial frequency domain (sometimes referred to as "k-space"). Therefore, one way to reduce the duration of a scan is to acquire fewer data points. For example, fewer samples may be acquired in the frequency encoding direction, the phase encoding direction, or both the frequency encoding direction and the phase encoding direction. However, when fewer data points are acquired than required by the spatial Nyquist criterion (this is often referred to as "undersampled" k-space), the MR image generated from the collected data points by inverse Fourier transform contains artifacts due to aliasing. As a result, although the scan time is shortened by undersampling in the spatial frequency domain, the resulting MRI image is of low quality and may be unusable because the introduced artifacts may severely degrade the image quality, fidelity, and interpretability.

[0057] Conventional techniques for reconstructing MR images from undersampled k-space data also have drawbacks. For example, compressed sensing techniques have been applied to the problem of generating MR images from undersampled spatial frequency data by using random k-space undersampled trajectories that produce incoherent aliasing that is removed using an iterative image reconstruction process. However, iterative reconstruction techniques require significant computational resources, do not work well without extensive empirical parameter tuning, and typically result in lower resolution MR images with lost detail.

[0058] Deep learning techniques have also been used to reconstruct MR images from undersampled k-space data. The neural network parameters underlying such techniques can be estimated using fully sampled data (data collected by sampling the spatial frequency space so that the Nyquist criterion is not violated), and although training such models can be time consuming, the trained models can be applied in real time during acquisition because neural network-based image reconstruction methods are significantly more computationally efficient than iterative reconstruction techniques utilized in the context of compressed sensing.

[0059] The present inventors have recognized that conventional deep learning MR image reconstruction techniques can be improved. For example, conventional deep learning MR image reconstruction techniques operate only in the image domain or only in the spatial frequency domain, and as such fail to take into account the cross-correlation structure in both the spatial frequency domain and the image domain. As another example, conventional deep learning MR image reconstruction techniques (or the above-mentioned compressed sensing techniques) do not work with non-Cartesian (e.g., radial, spiral, rosette, variable density, Lissajou, etc.) sampling trajectories, which are commonly used to accelerate MRI acquisition and are also robust to the motion of the subject. In contrast, the present inventors have developed novel deep learning techniques for generating high-quality MR images based on undersampled spatial frequency data, which deep learning techniques: (1) operate in both the spatial frequency domain and the image domain; and (2) enable reconstruction of MR images based on non-Cartesian sampling trajectories. As described herein, the deep learning techniques developed by the present inventors improve conventional MR image reconstruction techniques (including both compressed sensing and deep learning techniques) and improve MR scanning techniques by reducing scan duration while generating high-quality MR images.

[0060] Some embodiments described herein address all of the above-mentioned problems recognized by the inventors for conventional techniques for generating MR images from undersampled spatial frequency domain data. However, not every embodiment described below addresses every one of these problems, and some embodiments may not address any of these problems. As such, it should be understood that embodiments of the technology provided herein are not limited to addressing all or any of the above-mentioned problems of conventional techniques for generating MR images from undersampled spatial frequency domain data.

[0061] Thus, some embodiments provide a method for generating an MR image from undersampled spatial-frequency domain data, the method comprising generating an MR image from input magnetic resonance (MR) spatial-frequency data using a neural network model, wherein the neural network model comprises: (1) a first neural network sub-model configured to process spatial-frequency domain data; and (2) a second neural network sub-model configured to process image domain data. In this manner, the techniques described herein operate in both the spatial-frequency domain and the image domain.

[0062] In some embodiments, the first neural network sub-model is applied before the second neural network sub-model. In this manner, the neural network is applied to the spatial-frequency domain data before transforming the spatial-frequency domain data to the image domain to exploit the cross-correlation structure in the spatial-frequency domain data. Thus, in some embodiments, generating an MR image may include: (1) processing input MR spatial frequency data using the first neural network sub-model to obtain output MR spatial frequency data; (2) transforming the output MR spatial frequency data to the image domain to obtain input image domain data; and (3) processing the input image domain data using the second neural network sub-model to obtain the MR image.

[0063] In some embodiments, the first neural network submodel may include one or more convolutional layers. In some embodiments, one or more (e.g., all) of these convolutional layers may have a stride greater than 1, which may provide for downsampling of the spatial frequency data. In some embodiments, the first neural network submodel may include one or more transposed convolutional layers, which may provide for upsampling of the spatial frequency data. Additionally or alternatively, the first neural network submodel may include at least one locally connected layer, at least one data consistency layer, and / or at least one complex conjugate symmetry layer. In some embodiments, the locally connected layer may include a corresponding set of parameter values ​​for each data point in the MR spatial frequency data.

[0064] In some embodiments, the first neural network sub-model includes at least one convolutional layer, a locally connected layer, and at least one transposed convolutional layer, and processing the input MR spatial frequency data using the first neural network sub-model may include: (1) applying at least one convolutional layer to the input MR spatial frequency data; (2) applying the locally connected layer to data obtained using the output of the at least one convolutional layer; and (3) applying at least one transposed convolutional layer to data obtained using the output of the locally connected layer. In such embodiments, the first neural network sub-model may be considered to have a "U" shaped structure consisting of a downsampling path (the left arm of the "U" - implemented using a series of convolutional layers, one or more of which have a stride greater than 1), a locally connected layer (the bottom of the "U"), and an upsampling path (the right arm of the "U" - implemented using a series of transposed convolutional layers).

[0065] In some embodiments, the use of transposed convolutional layers (sometimes referred to as fractional sliding convolutional layers or deconvolutional layers) may result in checkerboard artifacts in the upsampled output. To address this issue, in some embodiments, upsampling may be performed by a convolutional layer in which the kernel size is divisible by the stride, such a convolutional layer may be considered a "sub-pixel" convolutional layer. Alternatively, in other embodiments, upsampling to a higher resolution may be performed without relying entirely on a convolutional layer to do so. For example, upsampling may be performed by resizing the input image (e.g., using interpolation such as bilinear interpolation or nearest neighbor interpolation) and following this operation with a convolutional layer. It should be understood that this approach may be used in place of and / or in conjunction with a transposed convolutional layer in any of the embodiments described herein.

[0066] In some embodiments, the first neural network sub-model further takes into account the complex conjugate symmetry of the spatial frequency data by including a complex conjugate symmetry layer. In some such embodiments, the complex conjugate symmetry layer can be applied at the output of the transposed convolutional layer, such that processing the input MR spatial frequency data using the first neural network sub-model includes applying the complex conjugate symmetry layer to data obtained using the output of at least one transposed convolutional layer.

[0067] In some embodiments, the first neural network sub-model further includes a data consistency layer to ensure that application of the first neural network sub-model to the spatial frequency data does not change the values ​​of the spatial frequency data obtained by the MR scanner. In this way, the data consistency layer forces the first neural network sub-model to insert missing data based on the undersampled spatial frequency data without disturbing the undersampled spatial frequency data itself. In some embodiments, the data consistency layer can be applied to the output of the complex conjugate symmetric layer.

[0068] In some embodiments, the first neural network submodel includes residual connections. In some embodiments, the first neural network submodel includes one or more nonlinear activation layers. In some embodiments, the first neural network submodel includes a rectified linear unit activation layer. In some embodiments, the first neural network submodel includes a leaky rectified linear unit activation layer.

[0069] The inventors have also recognized that improved MR image reconstruction can be achieved by directly generating MR images from spatial frequency data samples without gridding the spatial frequency data as is typically done in conventional MR image reconstruction techniques. In gridding, the acquired spatial frequency data points are mapped to a two-dimensional (2D) Cartesian grid (e.g., the values ​​at each grid point are interpolated from data points within a threshold distance), and an image is reconstructed from the grid values ​​using a 2D discrete Fourier transform (DFT). However, this local interpolation introduces reconstruction errors.

[0070] The inventors have developed multiple deep learning techniques for reconstructing MR images from data obtained using non-Cartesian sampling trajectories. Some of these techniques involve using a non-uniform Fourier transform (e.g., a non-uniform fast Fourier transform - NuFFT) at each of a plurality of blocks that are part of a neural network model to facilitate data consistency with (non-gridded) spatial frequency data obtained by an MRI system. This data consistency processing can be performed in a variety of different ways, although each approach can utilize a non-uniform Fourier transform (e.g., as represented by the forward operator A described herein) and input MR spatial frequency data y. For example, in some embodiments, a non-uniform Fourier transform can be used in a neural network model block to transform image domain data representing an MR reconstruction in a block into spatial frequency data so that the MR reconstruction in the block can be compared with the spatial frequency data obtained by the MRI system. A neural network model that implements the method may be referred to as a non-uniform variational network (NVN) and is described herein (including references to NVN). Figures 13A to 13D for description).

[0071] As another example, in some embodiments, a non-uniform Fourier transform can be applied to the spatial frequency data, and the result can be provided as an input to each of the one or more neural network blocks to reconstruct an MR image from the spatial frequency data. These innovations provide a prior art deep learning technique for reconstructing MR images from spatial frequency data obtained using a non-Cartesian sampling trajectory. The neural network model implementing the method may be referred to as a generalized non-uniform variational network (GNVN) and is described herein (including references to Fig.13A , 13D and 13E).

[0072] Thus, some embodiments provide a method for generating an MR image based on input magnetic resonance (MR) spatial frequency data using a neural network model, the neural network model including one or more neural network blocks, the one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to use a non-uniform Fourier transform (e.g., a non-uniform fast Fourier transform - NuFFT) for transforming image domain data into spatial frequency domain data for data consistency processing. The MR spatial frequency data can be obtained using a non-Cartesian sampling trajectory, an example of which is provided herein. In some embodiments, the neural network model can include multiple blocks, each of which is configured to use a non-uniform Fourier transform for data consistency processing.

[0073] In some embodiments, a method for generating an MR image based on input MR spatial frequency data includes: obtaining input MR spatial frequency data; generating an initial image based on the input MR spatial frequency data using a non-uniform Fourier transform; and applying a neural network model to the initial image at least in part by using a first neural network block to perform data consistency processing using a non-uniform Fourier transform.

[0074] In some embodiments, the data consistency processing may involve applying a data consistency block to the data, which may apply a non-uniform Fourier transform to the data to transform the data from the image domain to the spatial frequency domain, where the data may be compared to the input MR spatial frequency data. In other embodiments, the data consistency processing may involve applying an adjoint non-uniform Fourier transform to the input MR spatial frequency data and providing the result as input to each of the one or more neural network blocks (e.g., as input to each of the one or more convolutional neural network blocks that are part of the overall neural network model).

[0075] In some embodiments, the first neural network block is configured to use a non-uniform Fourier transform to perform data consistency processing at least in part by applying a gridded interpolation transform, a fast Fourier transform, and a de-apodization transform to the data. In this way, the non-uniform Fourier transform A is represented as three transforms—a gridded interpolation transform G, a fast Fourier transform F, and a de-apodization transform. s and a de-apodization transform D such that A = GF s D, and applying A to the data can be done by transforming D, F s and G in that order (e.g., Fig. 13C The gridded interpolation transform may be determined based on a non-Cartesian sampling trajectory used to obtain the initial MR input data. In some embodiments, applying the gridded interpolation transform to the data may be performed using a sparse graphics processing unit (GPU) matrix multiplication. Exemplary implementations of these constituent transforms are described herein.

[0076] In some embodiments, a neural network model for reconstructing an MR image based on spatial frequency data may include a plurality of neural network blocks, each of which includes: (1) a data consistency block configured to perform data consistency processing; and (2) a convolutional neural network block including one or more convolutional layers (e.g., having one or more convolutional layers and / or transposed convolutional layers, having a U-net structure, etc.). Such a neural network model may be referred to herein as a non-uniform variational network (NVN).

[0077] In some embodiments, the data consistency block is configured to: apply a non-uniform Fourier transform to a first image provided as an input to the data consistency block to obtain first MR spatial frequency data; and apply a concomitant non-uniform Fourier transform to a difference between the first MR spatial frequency data and the input MR spatial frequency data. In some embodiments, applying the non-uniform Fourier transform to the first image domain data comprises: applying a de-apodization transform, followed by a Fourier transform, followed by a gridded interpolation transform to the first image domain data.

[0078] In some embodiments, a first neural network block is applied to image domain data, the applying comprising: applying a data consistency block to the image domain data to obtain a first output; applying a plurality of convolutional layers to the image domain data to obtain a second output; and determining a linear combination of the first output and the second output.

[0079] In some embodiments, a neural network model for reconstructing an MR image from spatial frequency data may include a plurality of neural network blocks, each of which includes a plurality of convolutional layers configured to receive as input: (1) image domain data (e.g., a network current reconstruction representing the MR data); and (2) an output obtained by applying a companion non-uniform Fourier transform to the input MR spatial frequency data. Such a neural network model may be referred to herein as a Graded Non-Uniform Variational Network (GNVN). In some embodiments, the plurality of convolutional layers are further configured to receive as input: an output obtained by applying a non-uniform Fourier transform and a companion non-uniform Fourier transform to the image domain data.

[0080] Another approach developed by the inventors for reconstructing MR images from input MR spatial frequency data without gridding is to use at least one fully connected layer in the spatial frequency domain. Thus, in some embodiments, the first neural network submodel may include at least one fully connected layer that is directly applied to the spatial frequency data points obtained by the scanner. Prior to applying the at least one fully connected layer, the data points are not mapped to a grid (by gridding and / or any other type of local interpolation). In some embodiments, prior to applying the at least one fully connected layer, the data points may be irregularly spaced.

[0081] In some embodiments where the first neural network sub-model includes a fully connected layer, the fully connected layer is applied to the real part of the spatial frequency domain data, and the same fully connected layer is applied to the imaginary part of the spatial frequency domain data. In other words, the data is channelized, and the same fully connected layer is applied to both the real data channels and the imaginary data channels.

[0082] Optionally, in some embodiments where the first neural network sub-model includes a fully connected layer, the first neural network sub-model includes a first fully connected layer for applying to the real part of the spatial frequency domain data and a second fully connected layer for applying to the imaginary part of the spatial frequency domain data. In some embodiments, the first fully connected layer and the second fully connected layer share at least some parameter values ​​(e.g., weights). In some embodiments, the outputs of the first fully connected layer and the second fully connected layer are transformed using a Fourier transform (e.g., a two-dimensional inverse discrete Fourier transform) to obtain image domain data. In turn, the image domain data can be provided as input to the second neural network sub-model.

[0083] The reference to 2D Fourier transforms in the preceding paragraphs should not be taken to mean that the techniques described herein are limited to operating on two-dimensional data (e.g., spatial frequency domain data and / or image domain data corresponding to a 2D MR image of a brain "slice"). In some embodiments, the techniques described herein can be applied to 3D data (e.g., spatial frequency domain data and / or image domain data corresponding to a stack of 2D MR images of different corresponding brain slices).

[0084] In some embodiments, batch normalization is applied to the output of the fully connected layer before using a Fourier transform to obtain image domain data.

[0085] In some embodiments, the second neural network submodel includes at least one convolutional layer and at least one transposed convolutional layer. In some embodiments, the second neural network submodel includes a series of blocks including corresponding sets of neural network layers, each block in the plurality of blocks including at least one convolutional layer and at least one transposed convolutional layer. In some embodiments, each block in the plurality of blocks further includes: a Fourier transform layer, a data consistency layer, and an inverse Fourier transform layer.

[0086] In some embodiments, a loss function including a spatial frequency domain loss function and an image domain loss function can be used to train a neural network model for generating MR images based on undersampled spatial frequency data. In some embodiments, the loss function is a weighted sum of the spatial frequency domain loss function and the image domain loss function. In some embodiments, the spatial frequency domain loss function includes a mean square error.

[0087] In some embodiments, the techniques described herein can be used to generate MR images based on undersampled spatial frequency data, which can be suitable for application to spatial frequency data collected using a low-field MRI system, which low-field MRI system includes, for example, but is not limited to any low-field MR system described herein and in U.S. patent application publication No. 2018 / 0164390, entitled “ELECTROMAGNETIC SHIELDING FOR MAGNETIC RESONANCE IMAGING METHODS AND APPARATUS,” which is incorporated herein by reference in its entirety.

[0088] As used herein, "high field" generally refers to MRI systems currently used in clinical settings, and more particularly refers to MRI systems that operate with a main magnetic field (i.e., B0 field) of 1.5T or more, but clinical systems that operate between .5T and 1.5T are also generally characterized as "high field". Field strengths between about .2T and 0.5T are characterized as "mid-field", and as the field strength in the high field region continues to increase, the field strength in the range between .5T and 1T is also characterized as mid-field. In contrast, "low field" generally refers to MRI systems that operate with a B0 field of less than or equal to about 0.2T, but due to the increased field strength at the high end of the high field region, systems with a B0 field between .2T and about 0.3T are sometimes also characterized as low field. In the low field region, low-field MRI systems that operate with a B0 field of less than .1T are referred to as "extremely low field" here, and low-field MRI systems that operate with a B0 field of less than 10mT are referred to as "ultra-low field" here.

[0089] In order to train the neural network model described herein to generate MR images from (e.g., undersampled) spatial frequency data obtained by a low-field MRI system, training data obtained using a low-field MRI system is required. However, there are few low-field MRI scanners on the market, and there is little low-field MRI data available for training such a neural network model. To address this limitation, the inventors have developed a novel two-stage training technique for training a neural network model for generating MR images from spatial frequency data obtained by a low-field MRI system. In the first stage, a neural network model (e.g., any neural network model described herein having a first neural network sub-model and a second neural network sub-model) is trained using a set of images obtained using a "high-field" or "mid-field" MR system, and is subsequently adapted by using a set of images obtained using a low-field MRI system.

[0090] Various concepts and embodiments thereof related to methods and apparatus for generating MR images based on spatial frequency domain data are described in more detail below. It should be understood that the various aspects described herein can be implemented in any of a variety of ways. Examples of specific implementations are provided here only for illustrative purposes. In addition, the various aspects described in the following embodiments can be used alone or in any combination, and are not limited to the combinations explicitly described herein.

[0091] Figure 1A 1 shows the architecture of an exemplary neural network model for generating an MR image from input magnetic resonance (MR) spatial frequency data according to some embodiments of the techniques described herein. Figure 1A As shown, the neural network model 100 includes a first neural network sub-model 102 configured to process spatial frequency domain data, an inverse fast Fourier transform (IFFT) layer 112 configured to transform the spatial frequency domain data into image domain data, and a second neural network sub-model 120 configured to process image domain data. After obtaining initial spatial frequency MR data using an MR scanner (e.g., using any low-field MR scanner described herein or any other suitable type of MR scanner), the initial spatial frequency MR data can be processed using the first neural network sub-model 102 to obtain output MR spatial frequency data 111. Then, the MR spatial frequency data 111 is transformed by the IFFT layer 112 to obtain input image domain data 113, which is processed by the second neural network sub-model 120 to obtain an MR image 127.

[0092] like Figure 1A As shown, the first neural network sub-model 102 includes one or more convolutional layers 104, a local connection layer 106, one or more transposed convolutional layers 108, a residual connection 109, a complex conjugate symmetric layer 105 and a data consistency layer 110.

[0093] When the first neural network sub-model 102 is applied to the initial MR spatial frequency data, the initial MR spatial frequency data is first processed by one or more convolutional layers 104, then by a locally connected layer 106, and then by a transposed convolutional layer 108. In some embodiments, the convolutional layer 104 may be used to downsample the data, and the transposed convolutional layer may be used to upsample the data. In such embodiments, these three processing steps may be viewed as providing a "U" shaped neural network architecture, where the convolutional layer 104 provides a downsampling path (the left arm of the "U"), the locally connected layer 106 is at the bottom of the "U", and the transposed convolutional layer 108 provides an upsampling path (the right arm of the "U").

[0094] exist Figure 1AIn an exemplary embodiment of the invention, the convolutional layers 104 include m0 convolutional layers. In some embodiments, m0 may be 1, 2, 3, 4, 5, or any number of layers between 1 and 20. In some embodiments, one or more of the m0 convolutional layers may have a stride greater than or equal to 1. In some embodiments, one or more of the m0 convolutional layers have a stride greater than 1, thereby providing downsampling or pooling of data through processing of these layers.

[0095] exist Figure 1A In the exemplary embodiment of , the transposed convolution layer 108 includes m0 transposed convolution layers. Figure 1A In the exemplary embodiment of FIG. 1 , the number of convolutional layers 104 and the number of transposed convolutional layers 108 are the same, but in other embodiments, the number of convolutional layers and the number of transposed convolutional layers may be different.

[0096] In some embodiments, a locally connected layer 106 is provided to exploit local cross-correlations with K-space. In some embodiments, the locally connected layer 106 is not a convolutional layer (in which the same set of weights is applied across different portions of the data), but rather has a corresponding set of weights for each data point in the spatial frequency domain data. Figure 1A In the exemplary embodiment of , the locally connected layer is placed between the downsampling and upsampling paths at the bottom of the "U" structure so that it will have fewer parameters (because the resolution of the data is lowest at this point), which reduces the number of parameters that must be learned during training.

[0097] In some embodiments, the local connection layer can take into account energy density variations in the spatial frequency domain (e.g., a central region in the spatial frequency domain has a higher energy density than a peripheral region). Figure 1A In the exemplary embodiment of , the local connection layer 106 operates in the spatial frequency domain and focuses on inserting missing data (due to undersampling) directly in the spatial frequency domain. In fact, the local connection layer, which has much fewer parameters than the fully connected layer but more parameters than the convolutional layer, provides a good balance between training time and the ability to use the local context correlation of the spatial frequency domain data to insert missing data points.

[0098] It should be understood that the use of a local connection layer to take into account energy density variations in the spatial frequency domain is a novel approach developed by the inventors. The previous method divides the spatial frequency domain into three square regions, and the data in each of the three regions is input into a separate model consisting of a stack of convolutional layers (thus for three different square regions, input into three separate models). In contrast, the use of a local connection layer does not involve partitioning the k-space into three square regions, but involves assigning independent weights to each symbol pixel, which takes into account various energy densities in a more general and flexible manner than the previous method, resulting in improved performance.

[0099] Figure 3 The performance improvement obtained by using a neural network model with locally connected layers to generate MR images from input MR spatial frequency data is shown. Figure 3 As can be seen from the middle column of , the MR images generated by the convolutional layer model without local connection layers generate artifacts (artifact streaks) that degrade the image quality. Figure 3 As shown in the right column of , using a neural network model including a sub-model having a local connection layer (e.g., the local connection layer 106) eliminates this artifact and produces an image closer to the original image in terms of the mean square error ( Figure 3 ).

[0100] Back to Figure 1A After the data is processed by layers 104, 106, and 108, the data is provided to a complex conjugate symmetry layer 105 (also referred to as a k-space symmetry layer), and the output of the complex conjugate symmetry layer 105 is provided as input to a data consistency layer 110. The output of the data consistency layer 110 (which is also the output of the first neural network sub-model) is then provided as input to an IFFT layer 112.

[0101] In some embodiments, the complex conjugate symmetry layer 105 performs interpolation based on complex conjugate symmetry in the spatial frequency domain (whereby S(x,y)=S'(-x,-y), where (x,y) are the coordinates of the data point and S' represents the complex conjugate of S). In some embodiments, applying the complex conjugate symmetry layer 105 to the spatial frequency domain data involves symmetrically mapping any missing points from existing samples. For example, if a value is obtained for point (x,y) but a corresponding value is not obtained for point (-x,-y), a complex conjugate symmetry layer can be used to provide the value of point (-x,-y) as the complex conjugate of the value obtained for point (x,y). Using the complex conjugate symmetry layer 105 speeds up the convergence of training a neural network model and improves the quality of images produced by the neural network model, such as Figure 4 In fact, if Figure 4As shown in the right panel of , the use of complex conjugate symmetric layers allows for the use of fewer training epochs when training the neural network model while obtaining improved model performance, which in this illustrative example is measured by the relative pixel intensity change in the central region of the image between the model reconstructed image and the fully sampled image.

[0102] In some embodiments, the data consistency layer 110 can be used to ensure that applying the first neural network sub-model to the spatial frequency data does not change the values ​​of the spatial frequency data obtained by the MR scanner. To the extent that any such values ​​are modified by other layers in the first neural network sub-model (e.g., the convolutional layer 104, the locally connected layer 106, and the transposed convolutional layer 108), the modified values ​​are replaced by the original values. In this way, the data consistency layer forces the first neural network sub-model to interpolate the missing data from the undersampled spatial frequency data without disturbing the undersampled spatial frequency data itself.

[0103] In some embodiments, any neural network layer may include an activation function, which may be non-linear. In some embodiments, the activation function may be a rectified linear unit (ReLU) activation function, a leaky ReLU activation function, a hyperbolic tangent, a sigmoid, or any other suitable activation function, as aspects of the techniques described herein are not limited thereto. For example, one or more convolutional layers 104 may include an activation function.

[0104] After the spatial frequency data is processed by the data consistency layer 110, the data is provided as input to the IFFT layer 112, which transforms the spatial frequency data to the image domain - the output is the initial image domain data 113. In some embodiments, the transformation can be performed using a discrete Fourier transform, which can be implemented using a fast Fourier transform. The initial image domain data 113 output by the IFFT layer 112 is provided as input to the second neural sub-model 120.

[0105] like Figure 1A As shown, the second neural network submodel 120 includes a plurality of convolution blocks 122, 124, and 126. Convolution block 122 may include one or more convolution layers 128, FFT layers 130, complex conjugate symmetric layers 105, data consistency layers, IFFT layers 134, and residual connections. Blocks 122, 124, and 126 may each have the same neural network architecture (e.g., the blocks may have the same types of layers arranged in the same sequence), although various parameter values ​​of the layers may vary (e.g., the weights of the convolution layers in block 122 may be different from the weights of the convolution layers in block 124). Although in Figure 1AIn the illustrative embodiment of the present invention, second neural network sub-model 120 includes three convolution blocks, but this is by way of example, as in other embodiments, second neural network sub-model 120 may include any suitable number of convolution blocks (e.g., 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, or 15), as aspects of the techniques described herein are not limited in this regard.

[0106] When the second neural network sub-model 120 is applied to the initial image domain data 113 obtained at the output of the IFFT block 112, the convolution blocks 122, 124 and 126 are applied in that order to the initial image domain data 113. Next, the application of the convolution block 122 is described, and it should be understood that the convolution blocks 124 and 126 can be applied in a similar manner to the image domain data provided as input to them.

[0107] like Figure 1A As shown, the convolution block 122 includes at least one convolution layer 128 , followed by an FFT layer 130 , a complex conjugate symmetry layer 105 , a data consistency layer 132 , and an IFFT layer 134 .

[0108] In some embodiments, the convolution block 128 includes one or more convolution layers with a stride greater than 1 (e.g., 2 or greater) to downsample the image, followed by one or more transposed convolution layers with a stride greater than 1 (e.g., 2 or greater) that upsample the image to its original size. This downsampling, then upsampling structure allows operation at different resolutions, which helps the neural network model capture both local and global features. In turn, this helps eliminate image artifacts that may be caused by undersampling in the spatial frequency domain. In this exemplary embodiment, the convolution layer does not include jump connections that may consume a lot of memory. For example, in some embodiments, the convolution block 128 has five layers, where the number of filters is 16, 32, 64, 32, and 2, respectively. In some embodiments, each filter can be a 3×3 filter with a leaky ReLU activation, although filters of different sizes and / or different activation functions may be used in other embodiments.

[0109] exist Figure 4 The left panel of shows the effect of the variable resolution layer. Figure 4 As shown in the left panel of , using variable resolution layers allows using fewer training epochs when training the neural network model while obtaining improved model performance, which in this illustrative example is measured by the relative pixel intensity change in the central region of the image between the model reconstructed image and the fully sampled image.

[0110] like Figure 1AAs shown in the exemplary embodiment of , after applying the convolution layer of convolution block 122, the data can be transformed into the spatial frequency domain so that complex conjugate symmetry and data consistency blocks can be applied, after which the data can be transformed back into the image domain and one or more other convolution blocks can be applied.

[0111] exist Figure 1A In the exemplary embodiment of , convolution blocks 122, 124, and 126 each include a complex conjugate symmetry and data consistency block. However, in other embodiments, one or more (or all) of the convolution blocks that are part of second neural network sub-model 120 may not have one or both of these blocks, as aspects of the techniques described herein are not limited in this regard.

[0112] Figure 1B The architecture of another example neural network model 140 for generating MR images from input MR spatial frequency data according to some embodiments of the techniques described herein is shown. The neural network model 140 includes a first neural network sub-model 142 having a convolutional layer 146 instead of a locally connected layer (e.g., in contrast to the first neural network sub-model 102 of model 100 having a locally connected layer 106). Such an embodiment may be advantageous because the convolutional layer 142 has fewer parameters to learn during training than the locally connected layer 106. In other respects, the neural network models 140 and 100 are identical.

[0113] Figure 1C The architecture of yet another exemplary neural network model 150 for generating an MR image from input MR spatial frequency data according to some embodiments of the techniques described herein is shown. The neural network model 150 includes a first neural network sub-model 152 having a convolution block 154 and a transposed convolution block 158. However, unlike the corresponding convolution block 104 and transposed convolution block 108 of the neural network model 100, the convolution blocks 154 and 158 contain convolution (and transposed convolution) layers using stride 1. As a result, the first neural network sub-model 152 does not perform upsampling or downsampling. This architecture can be advantageous when there is a large amount of training data available.

[0114] Figure 2A is a flow chart of an exemplary process 200 for generating an MR image from input MR spatial frequency data using a neural network model according to some embodiments of the techniques described herein. The process 200 may use any suitable neural network architecture described herein (including reference Figures 1A to 1C5A-5C). Process 200 can be performed using any suitable computing device, as aspects of the technology described herein are not limited thereto. For example, in some embodiments, process 200 can be performed by a computing device that is communicatively coupled to or is part of an MR imaging system.

[0115] Process 200 begins at act 202, where spatial frequency domain data is obtained. In some embodiments, the spatial frequency domain data may be obtained by using an MR scanner, including any MR scanner described herein. In other embodiments, the spatial frequency domain data may have been obtained by an MR scanner prior to performing process 200, stored, and the stored data may be accessed during act 202.

[0116] In some embodiments, the spatial frequency domain data may be undersampled relative to the Nyquist sampling criterion. For example, in some embodiments, the spatial frequency domain data may include less than 90% (or less than 80%, or less than 75%, or less than 70%, or less than 65%, or less than 60%, or less than 55%, or less than 50%, or less than 40%, or less than 35%, or any percentage between 25 and 100) of the number of data samples required by the Nyquist criterion.

[0117] The spatial frequency domain data obtained at act 202 may (or may have been) obtained by an MR scanner using any suitable pulse sequence and sampling scheme. For example, in some embodiments, the spatial frequency domain data may be collected using a Cartesian sampling scheme. In other embodiments, the spatial frequency domain data may be collected using a non-Cartesian sampling scheme (e.g., radial, spiral, rosette, Lissajou, etc.).

[0118] Next, process 200 proceeds to act 204, where a first neural network sub-model (e.g., reference Figure 1A The sub-model 102, reference Figure 1B The sub-model 142, reference Figure 1C The sub-model 152, reference Figure 5A The sub-model 502, reference Figure 5B The submodel 522 and reference Figure 5C The submodel 532 described above is used to process the MR spatial frequency data obtained at action 202. Figure 2B and 2C 2. An illustrative example of how act 204 may be implemented is described below.

[0119] Next, process 200 proceeds to act 206 , where the spatial frequency domain data obtained upon completion of act 204 is transformed (eg, using a Fourier transform) to obtain initial image domain data.

[0120] Next, process 200 proceeds to act 208, where a second neural network sub-model (e.g., reference Figure 1A The sub-model 120, reference Figure 5A The sub-model 510) is used to process the initial image domain data obtained at the completion of act 206 to generate an MR image. Figure 2D 2. An illustrative example of how act 208 may be implemented is described below.

[0121] Figure 2B is a flow chart of an exemplary process for processing MR spatial frequency data in the spatial frequency domain (which may be part of the exemplary process 200) to obtain output spatial frequency data according to some embodiments of the techniques described herein. In particular, Figure 2B An illustrative embodiment for implementing act 204 of process 200 is shown.

[0122] like Figure 2B As shown, action 204 may be implemented using actions 212 to 218. At action 212, one or more convolutional layers may be applied to the spatial frequency domain data obtained at action 202. In some embodiments, the convolutional layers applied at action 212 may be reference Figure 1A A portion of block 104 or reference Figure 1C The convolutional layers may be implemented using a stride greater than 1 (e.g., 2) to downsample the data. In other embodiments, the convolutional layers may be implemented using a stride of 1.

[0123] Next, at act 214, a local connection layer is applied to the spatial frequency domain data obtained at the completion of act 212. In some embodiments, the local convolution layer may be a reference Figure 1A The local convolution layer 106. In some embodiments, the local connection layer has a corresponding set of weights for each data point in the spatial frequency domain data.

[0124] Next, at act 216, one or more transposed convolutional layers are applied to the spatial frequency domain data obtained upon completion of act 214. In some embodiments, the transposed convolutional layers may be used as reference. Figure 1A The block 108 or reference Figure 1C The transposed convolutional layer is a portion of block 158. In some embodiments, the transposed convolutional layer may upsample the data.

[0125] Next, at act 218, a complex conjugate symmetry layer is applied to the spatial frequency domain data output upon completion of act 216. In some embodiments, the complex conjugate symmetry layer may be a reference Figure 1A The complex conjugate symmetry layer 105. As described herein, applying the complex conjugate symmetry layer 105 to the spatial frequency domain data can involve symmetrically mapping any missing points from the existing samples. For example, if a value is obtained for point (x, y), but a corresponding value is not obtained for point (-x, -y), a complex conjugate symmetry layer can be used to provide the value of point (-x, -y) as the complex conjugate of the value obtained for point (x, y).

[0126] Next, at act 220, a data consistency layer is applied to the spatial frequency domain data output upon completion of act 218. In some embodiments, the data consistency layer may be a reference Figure 1A The data consistency layer 110. As described herein, the data consistency layer can be used to ensure that applying the first neural network sub-model to the spatial frequency data does not change the values ​​of the spatial frequency data obtained by the MR scanner.

[0127] Figure 2C is a flow chart of an exemplary process for processing spatial frequency domain data (which may be part of exemplary process 200) to generate an MR image according to some embodiments of the techniques described herein. In particular, Figure 2C Another illustrative embodiment for implementing act 204 of process 200 is shown.

[0128] like Figure 2C As shown, action 204 may be implemented using actions 222 and 224. At action 222, one or more fully connected layers are applied to the spatial frequency data obtained at action 202. In some embodiments, the fully connected layers applied at action 222 may be reference Figure 5A The fully connected layer 502. As described herein, the fully connected layer represents a learned mapping from non-Cartesian coordinates of the data to Cartesian coordinates, which allows reconstruction of MR images from non-Cartesian samples without relying on traditional gridding or other interpolation schemes that are independent of the data.

[0129] In some embodiments, at action 222, the spatial frequency data obtained at action 202 is divided into a real part and an imaginary part, and the same fully connected layer is applied to each of the two parts. Equivalently, it can be considered that these data are provided to a fully connected layer with shared weights of real and imaginary channels. This weight sharing scheme ensures that the same interpolation operation is applied to both real and imaginary channels, thereby maintaining the symmetry of the underlying spatial frequency domain in the whole process. In addition, sharing weights between the real and imaginary parts reduces the number of trainable parameters in the model by 2 times. However, in other embodiments, the spatial frequency data can be fed to the fully connected layer with partial weight sharing or without weight sharing between the real and imaginary channels.

[0130] Next, at act 224, batch normalization is applied so that subsequent layers receive inputs with substantially zero mean and substantially unit (or any other suitable constant) variance.

[0131] It should be understood that Figure 2C The processing of is illustrative and variations exist. For example, in some embodiments, batch normalization can be omitted.

[0132] Figure 2D is a flow chart of another exemplary process for processing image domain data (which may be part of exemplary process 200) to generate an MR image according to some embodiments of the techniques described herein. In particular, Figure 2D An illustrative embodiment for implementing act 208 of process 200 is shown.

[0133] like Figure 2D As shown, action 208 may be implemented using actions 230-236 and decision block 238. In particular, at action 230, one or more convolutional layers are applied to the image domain data obtained at action 206 by transforming the spatial frequency domain data into the image domain. In some embodiments, the convolutional layers applied at action 230 may be Figure 1A Block 128 or Figure 5A 5. In some embodiments, the convolutional layers may include any suitable number of layers, including any number of layers in the range of 1 to 20 layers. In some embodiments, the convolutional layers may be implemented using a stride greater than 1 (e.g., 2) to downsample the data. In other embodiments, the convolutional layers may be implemented using a stride of 1.

[0134] Next, at act 232, one or more transposed convolutional layers may be applied to the image domain data output upon completion of act 230. In some embodiments, the transposed convolutional layers applied at act 232 may be Figure 1A The transpose block 128 shown or Figure 5AA portion of block 512 is shown. In some embodiments, the convolutional layers can include any suitable number of layers, including any number of layers in the range of 1 to 20 layers. In some embodiments, a transposed convolutional layer can be implemented to upsample the data (e.g., using a fractional stride).

[0135] Next, at act 234, a complex conjugate symmetry layer may be applied to the data. As the complex conjugate symmetry layer is applied in the spatial frequency domain, the image domain data output at the completion of act 232 is transformed to the spatial frequency domain before the complex conjugate symmetry layer is applied. In some embodiments, the complex conjugate symmetry layer may be a reference Figure 1A The complex conjugated symmetric layer 105.

[0136] Next, at act 236, a data consistency layer may be applied to the data. In some embodiments, the data consistency layer may be applied to the spatial frequency domain data output upon completion of act 234. In other embodiments, if act 234 is omitted, the image domain data output upon completion of act 232 may be transformed to the spatial frequency domain and the data consistency layer may be applied to the data. In some embodiments, the data consistency layer may be a reference to the spatial frequency domain. Figure 1A The data consistency layer 110.

[0137] Next, at decision block 238, a determination is made as to whether one or more additional image domain processing blocks are to be applied. If it is determined that no additional blocks are to be applied, processing is complete. Otherwise, processing returns to action 230 via the "yes" branch and actions 230-236 and decision block 238 are repeated. For example, if Figure 1A As shown, after block 122 is applied to the image domain data, it may be determined that block 124 will be applied to the data.

[0138] It should be understood that Figure 2D The processing of is illustrative and variations exist. For example, in some embodiments, image domain data may be processed only in the image domain without applying a complex conjugate symmetry layer and a data consistency layer.

[0139] Figure 5A The architecture of another exemplary neural network model 500 for generating magnetic resonance (MR) images from input MR spatial frequency data in accordance with some embodiments of the techniques described herein is shown.

[0140] like Figure 5AAs shown, the neural network model 500 includes a first neural network sub-model 502 configured to process spatial frequency domain data, an inverse fast Fourier transform (IFFT) layer 508 configured to convert the spatial frequency domain data into image domain data, and a second neural network sub-model 510 configured to process image domain data. After obtaining initial spatial frequency MR data using an MR scanner (e.g., using any low-field MR scanner described herein or any other suitable type of MR scanner), the initial spatial frequency MR data can be processed using the first neural network sub-model 502 to obtain output MR spatial frequency data 511. Then, the MR spatial frequency data 511 is transformed by the IFFT layer 508 to obtain initial image domain data 513, which is processed by the second neural network sub-model 510 to obtain an MR image 518.

[0141] like Figure 5A As shown, the initial spatial frequency domain MR data is divided into a real part 504 (e.g., the amplitude of the complex valued data) and an imaginary part 506 (e.g., the phase of the complex valued data). The first neural network submodel 502 includes a fully connected layer that operates on the real part 504 and the imaginary part 506. Figure 5A In the illustrated embodiment, the fully connected layer shares weights between real and imaginary channels. As such, the fully connected layer applies the same operations to both real and imaginary channels, thereby maintaining the underlying spatial frequency domain symmetry throughout the process. Additionally, sharing weights between real and imaginary parts reduces the number of trainable parameters in the model by a factor of 2. However, in other embodiments (e.g., Figure 5C In an embodiment of the present invention, the spatial frequency data can be fed to the fully connected layer with partial weight sharing or no weight sharing between real channels and imaginary channels.

[0142] In some embodiments, when a neural network model including fully connected layers is trained using input MR images generated using the same sample trajectory, the fully connected layers learn a data-dependent mapping from non-Cartesian coordinates to Cartesian coordinates that can be used to perform data-dependent gridding of non-Cartesian spatial frequency data that may be generated by an MR scanner operating according to a non-Cartesian sequence. Figures 6A to 6C Further examples are given in .

[0143] Fig. 6A An illustrative embodiment is shown in which each data point in the spatial frequency domain has a corresponding 128×128 weight matrix having a weight for each location in the 128×128 output k-space, thereby creating a non-local interpolation. Figure 6B The weight distribution for three spatial frequency domain data points (#300, #2800 and #5000) is shown in FIG. Figure 6CThe 2D distribution of these same three data points is shown in , where the zoomed-in view shows the details of the weight distribution.

[0144] like Figures 6B to 6C As shown in the 1D and 2D weight distributions of , when plotting the 2D weight map of a specific spatial frequency domain data point, it is mainly the weights with non-negligible values ​​in the local neighborhood of the data point, while the values ​​of other weights are close to zero. The weight distribution indicates that the mapping performed by the fully connected layer is locally interpolated. It should be noted that the first neural network sub-model 502 does not include a data consistency layer, which allows the first neural network sub-model 502 to process non-Cartesian samples.

[0145] Back to Figure 5A After the spatial frequency data is processed by the first neural network model 502, the data is provided as input to the IFFT layer 508, which transforms the spatial frequency data to the image domain - the output is the initial image domain data 513. In some embodiments, the transformation can be performed using a discrete Fourier transform, which can be implemented using a fast Fourier transform. The initial image domain data 513 output by the IFFT layer 508 is provided as input to the second neural sub-model 510.

[0146] like Figure 5A As shown, the second neural network submodel 510 includes a plurality of convolution blocks 512, 514, and 516. Convolution block 512 may include one or more convolution layers and residual connections. Convolution blocks 512, 514, and 516 may each have the same neural network architecture (e.g., the blocks may have the same type of layers arranged in the same sequence), although the various parameter values ​​of the layers may vary (e.g., the weights of the convolution layers in block 512 may be different from the weights of the convolution layers in block 514). Although Figure 5A In the illustrative embodiment of the present invention, second neural network sub-model 510 includes three convolution blocks, but this is by way of example, as in other embodiments, second neural network sub-model 510 may include any suitable number of convolution blocks (e.g., 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, or 15), as aspects of the technology described herein are not limited in this regard.

[0147] When the second neural network sub-model 510 is applied to the initial image domain data 513 obtained at the output of the IFFT block 508, the convolution blocks 512, 514 and 516 are applied in that order to the initial image domain data 513. The application of the convolution block 512 is described next, and it should be understood that the convolution blocks 514 and 516 can be applied to the image domain data in a similar manner (after the image domain data provided to them as input is output from the previous blocks).

[0148] In some embodiments, the convolution block 512 includes one or more convolutional layers with a stride greater than 1 (e.g., 2 or greater) to downsample the image, followed by one or more transposed convolutional layers with a stride greater than 1 (e.g., 2 or greater) that upsample the image to its original size. This downsampling, then upsampling structure allows operation at different resolutions, which helps the neural network model capture both local and global features. In turn, this helps eliminate image artifacts that may be caused by undersampling in the spatial frequency domain.

[0149] For example, in some embodiments, the convolution block 512 may include two sequential convolutional layers (with 32 3×3 filters and 64 3×3 filters in two corresponding layers with a stride of 2), followed by two transposed convolutional layers (with 128 3×3 filters and 64 3×3 filters in two corresponding layers with a stride of 2), followed by a final convolutional layer (2 3×3 filters with a stride of 1). Except for the final convolutional layer, non-linear activations (e.g., ReLU or leaky ReLU activations) may be applied in each of the first four layers. Nevertheless, it should be understood that in other embodiments, filters of different sizes and / or different activation functions may be used, as aspects of the techniques described herein are not limited in this regard.

[0150] Figure 5B The architecture of another exemplary neural network model 520 for generating an MR image from input magnetic resonance (MR) spatial frequency data according to some embodiments of the technology described herein is shown. The neural network 520 has a first neural network submodel 522, which has a batch normalization layer 507 after applying a fully connected layer and before the data is output from the first neural network submodel 522 to the IFFT layer 508. The introduction of the batch normalization layer at this time improves the performance of the neural network and can reduce the time required for training. In other respects, the neural network models 520 and 500 are the same.

[0151] Figure 5C The architecture of another exemplary neural network model 530 for generating an MR image from input magnetic resonance (MR) spatial frequency data according to some embodiments of the techniques described herein is shown. The neural network 530 has a first neural network sub-model 532 that includes a fully connected layer that does not use weight sharing between the real and imaginary parts of the acquired MR data. In other respects, the neural network models 530 and 500 are the same.

[0152] The inventors have developed a novel non-Cartesian sampling trajectory to accelerate the acquisition of spatial domain data while maintaining as much information as possible. The sampling trajectory consists of a full sampling grid of k-space centers generated by an unstructured triangular and tetrahedral mesh used to uniformly undersample the entire spatial frequency domain and a Gaussian kernel, because complete coverage of the k-space center is important for the reconstruction of low signal-to-noise ratio (SNR) images. The sampling trajectory samples 33% of the spatial frequency domain samples required to meet the Nyquist criterion (although as mentioned above, the sampling trajectory can be used with any other percentage described herein (for example, including any percentage in the range of 25 to 100, such as 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, etc.). K space. Figure 7 The novel non-Cartesian sampling trajectory (panel 702), the image reconstructed from the samples obtained using the trajectory of panel 702 and the zero-filled inverse fast Fourier transform (panel 704), the image reconstructed from the trajectory of panel 702 and the reference image (panel 705), and the image reconstructed from the samples obtained using the trajectory of panel 702 and the reference image (panel 706). Figure 5B The sample reconstructed image (panel 706) obtained by the neural network model, and the original MR image. As can be seen from panels 704 and 706, the MR image obtained using zero-filled IFFT is blurry and has artifacts, while the MR image obtained using Figure 5B The MR images obtained by the neural network model do not have these defects.

[0153] The inventors have developed specialized techniques for training the neural network models described herein. The training process involves generating complex image data, undersampling the complex image data, and using pairs of undersampled and fully sampled complex image data to train the neural network model using any suitable training technique (e.g., stochastic gradient descent and back propagation). As described below, to generate the complex image data, the amplitude image is used to synthesize the phase information.

[0154] Figure 8 Aspects of training a neural network model for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein are shown. Figure 8 As shown, the training process involves synthesizing phase information using an input magnitude image. The magnitude and phase information constitute complex image data that can be retrospectively undersampled in the spatial frequency domain using a Cartesian or non-Cartesian (e.g., radial, etc.) sampling trajectory. The undersampled data will be used as the input to the neural network model being trained, and the fully sampled image will be the output of the model.

[0155] Although there are many publicly available MR image data sets available, they typically only include magnitude images. In order to simulate the complex data acquired by an MR scanner, the inventors have developed techniques for generating phase information to append to the magnitude images. Therefore, in some embodiments, a weighted sum of spherical harmonic basis functions is used to generate the phase information. The combination of these functions can characterize magnetic field variations derived from inhomogeneities in B0, magnetic field drift with temperature, gradient eddy currents, spatially varying RF coil sensitivity fields, inaccuracies in the gradient fields in the sequence, and / or other effects that may contribute to phase variations. Fig. 9A The process of generating phase information using spherical harmonics is shown in FIG.

[0156] In some embodiments, to simulate non-Cartesian undersampling, a non-uniform FFT (NuFFT) is used to transform the MR image to the spatial frequency domain with a non-Cartesian undersampling mask applied. In turn, the undersampled spatial frequency data can be converted to the image domain using an inverse (also referred to as reverse) NuFFT, which can be provided as input to the image domain sub-model. In this way, the use of NuFFT enables non-uniform K-space sampling, which is actually very similar to non-Cartesian sampling.

[0157] In some embodiments, the available training data is enhanced by applying affine transformations to individual slices to create images with different orientations and sizes, adding noise to create images with different SNRs, introducing motion artifacts, incorporating phase and / or signal modulation for more complex sequences (such as echo trains), and / or modeling dephasing of the signal to adapt the model to sequences such as diffusion weighted imaging.

[0158] Since the neural network model described herein operates in both the spatial frequency domain and the image domain, the inventors have developed a new loss function to facilitate training such a mixed-domain neural network model. The new loss function speeds up the training process of the neural network model described herein (e.g., by reducing the number of training epochs required to achieve a given performance level).

[0159] In some embodiments, the loss function includes a first loss function for capturing errors in the spatial frequency domain and a second loss function for capturing errors in the image domain. Fig. 9BAs shown, the output of the first neural network submodel (labeled as "Subnet 1k spatial") can be compared with the ground truth in the spatial frequency domain to obtain a first error measure in the spatial frequency domain (e.g., a mean squared error labeled as "MSE loss 1"), and the output of the second neural network submodel (labeled as "Subnet 2 image domain") can be compared with the ground truth to obtain a second error measure in the image domain (e.g., a mean squared error labeled as "MSE loss 2"). The first error measure and the second error measure can be combined (e.g., via a weighted combination) to produce an overall error measure that will be minimized during the training process. For example, in the illustrative example of FIG. 9, the two loss functions are combined using a weight of λ<1 so that the overall loss function is given by Loss1+λ*Loss2.

[0160] As described herein, in order to train the neural network model developed by the inventors to generate MR images based on undersampled spatial frequency data obtained by a low-field MRI system, training data obtained using the low-field MRI system is required. However, the amount of such data may not be sufficient to learn all the parameters of the model described herein.

[0161] Thus, in some embodiments, a neural network model is first trained using images acquired using one or more "high-field" and / or "mid-field" MR systems, and then transfer learning is used to adapt the trained neural network model to a "low-field" context by using one or more MR images acquired using a low-field MRI system.

[0162] Figures 10A to 10H MR images generated using a zero-filled inverse DFT and using a neural network model trained with and without transfer learning are shown according to some embodiments of the techniques described herein. The results show that using transfer learning (100 epochs in this illustrative example) improves the performance of the model on low-field MR images. In particular, Figures 10A to 10D Shows training with zero-padded inverse FFT and without transfer learning Figure 5B Neural network models trained with transfer learning Figure 5B The neural network and the reconstructed MR images obtained with full sampling data. Figures 10E to 10G shows the absolute difference between the reconstructed MR image and the fully sampled MR image, while Fig. 10H The undersampling mask is shown.

[0163] Fig.11 The performance of some neural network models for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein is shown. In particular, Fig.11 The second line shows the reference Figure 1Athe performance of the neural network model 100, and Fig.11 The third line shows the reference Figure 5B The performance of the neural network model 520. For these two models, Fig.11 The performance of the corresponding first and second sub-models (sub-models 102 and 120 and sub-models 522 and 510) are shown. Fig.11 The first row shows the undersampled image and the fully sampled image (both amplitude and phase). Fig.11 As can be seen in the , the output of the first sub-model of neural network model 100 (the first two columns of the middle row) has improved quality and fewer artifacts, which is also indicated by the increased peak SNR (PSNR). The output of the second sub-model of neural network model 100 (the last two columns of the middle row) shows that the second sub-model further improves the reconstruction by increasing the contrast of the magnitude image and generating a smoother phase map that is closer to the fully sampled image. For neural network model 520, the second sub-model contributes less to the improvement reflected by the PSNR than the first sub-model. The opposite is true for the first neural network sub-model.

[0164] Fig.12 Further illustrating the performance of some neural network models for generating MR images from undersampled spatial frequency domain data according to some embodiments of the techniques described herein. In particular, Fig.12 The performance of some neural networks developed here relative to other techniques for undersampling an image to 33% of the number of samples required to sample at the Nyquist sampling rate is shown. Fig.12 The performance of the neural network models 100 and 520 (shown in the fourth and fifth columns of FIG. 1 ) is comparable to that of the compressed sensing (implemented using the ADMM regularizer (where the regularization parameter = 5e-3) and Fig.12 ) and the neural network model 100 without the first sub-model (in Fig.12 The performance of the output images is compared with that of the output images. The normalized mean square error and peak SNR are measured to quantify the difference in the output images. Fig.12 As shown, undersampling introduces blurring and non-uniform artifacts. The compressed sensing method removes the artifacts, but over-smoothes the image and changes the phase image. Model 100 without the first sub-model cannot recover the image. In contrast, neural network models 100 and 520 output MR images that are closer to the fully sampled image in both amplitude and phase, as reflected by higher PSNR and lower normalized MSE compared to competing methods.

[0165] As discussed herein, the inventors have developed a neural network model for reconstructing MR images from spatial frequency data acquired using a non-Cartesian sampling trajectory.

[0166] Fig.13Ais a diagram of an illustrative architecture of an exemplary neural network model 1310 for generating MR images from input MR spatial frequency data according to some embodiments of the techniques described herein. Fig.13A As shown, the neural network model 1310 reconstructs an output MR image 1315 based on the input MR spatial frequency data 1305 by hierarchically processing the input MR spatial frequency data. First, the input MR spatial frequency data 1305 is processed using an initial processing block 1312 to produce an initial image 1314, which is then processed by a series of neural network blocks 1316-1, 1316-2, ..., 1316-n.

[0167] In some embodiments, one or more of blocks 1316-1, 1316-2, ..., 1316-n may be operators in the image domain. In some embodiments, one or more of blocks 1316-1, 1316-2, ..., 1316-n may transform input data to a different domain (including but not limited to a spatial frequency domain), perform processing in the different domain (e.g., reconstruction processing), and then transform back to the image domain.

[0168] In some embodiments, the initializer block transforms the input MR spatial frequency data into the image domain to generate an initial image for subsequent processing by the neural network model 1310. The initializer block can be implemented in any suitable manner. For example, in some embodiments, the initializer block can apply an adjoint non-uniform Fourier transform to the input MR spatial frequency data to obtain the initial image. As another example, in some embodiments, the initializer block can apply a gridded reconstruction to the input MR spatial frequency data to obtain the initial image.

[0169] exist Fig. 13B (corresponding to non-uniform variational networks) and Fig.13E (corresponding to a generalized non-uniform variational network) is shown in FIG. Thus, in some embodiments, at least one, at least some, or all of the blocks 1316-1, 1316-2, ..., 1316-n may have a structure as described for Fig. 13B The architecture shown in the exemplary block 1316-i in FIG. Fig. 13B As shown, the neural network block 1316-i includes a data consistency block 1320 and a convolutional neural network block 1350, both of which are applied to the input terminal x labeled 1321. i . Enter x i can represent the MR image reconstruction generated by the neural network 1310 at the completion of the (i-1)th neural network block. In this example, by applying the data consistency block 1320 to the input x i To obtain a first result, apply the convolutional neural network block 1350 to x iTo obtain the second result, and from x i The output 1335 of block 1316-i is obtained by subtracting the linear combination of the first result and the second result, using the block-specific weight λ i Compute linear combinations.

[0170] The data consistency block 1320 may be implemented in any of a variety of ways. In some embodiments, the data consistency block 1320 may be implemented by using a non-uniform Fourier transform to transform x i Data consistency is processed by transforming the represented input image to the spatial frequency domain, comparing the result to the original MR spatial frequency data 1305, and transforming the difference between the two back to the image domain using a concomitant transform of the non-uniform Fourier transform.

[0171] exist Fig. 13C An exemplary implementation of data consistency block 1320 is shown in FIG. Fig. 13C In an exemplary implementation of , the image domain input 1322 (which may be an intermediate reconstruction x) is transformed through a series of three transformations 1324, 1326 and 1328. i 1321) is transformed into the spatial frequency domain. The combination of the three transforms in this series is used to implement a non-uniform fast Fourier transform from the image domain to the spatial frequency domain. In particular, transform 1324 is a de-apodized and zero-filled transform D, and transform 1326 is an oversampled FFT transform F s , and transform 1328 is the gridded interpolation transform G. As described herein, the non-uniform fast Fourier transform A is composed of these transforms according to A=DF s G represents. Example implementations of these composition transformations are described here.

[0172] After the image domain input 1322 is transformed to the spatial frequency domain, the image domain input 1322 is compared to the original MR spatial frequency data 1305, and the difference between the two is transformed back to the image domain using transforms 1330, 1332, and 1334, in that order. Transform 1330 is the companion transform to the gridded interpolation transform 1328. Transform 1332 is the companion transform to the oversampled FFT transform 1326. Transform 1334 is the companion transform to the de-apodized transform 1324. In this manner, G can be written H F H s D H =A H The combination of transformations 1330, 1332, 1334 represents the adjoint transformation A of the non-uniform Fourier transform A. H .

[0173] Convolutional neural network block 1350 can be implemented in any of a variety of ways. In some embodiments, block 1350 can have multiple convolutional layers, including one or more convolutional layers and one or more transposed convolutional layers. In some embodiments, block 1350 can have a U-net structure, whereby multiple convolutional layers downsample the data and subsequent transposed convolutional layers upsample the data, for example, as described for block 1350. Fig.13D An exemplary U-net architecture is shown in FIG.

[0174] like Fig.13D As shown, the input of the convolutional network block 1350 is processed by a downsampling path and then an upsampling path. In the downsampling path, the input is processed by repeatedly applying two convolutions with a 3×3 kernel, and a nonlinearity (e.g., a rectified linear unit or ReLU) is applied after each convolution, i.e., an average 2×2 pooling operation with a stride of 2 for downsampling. At each downsampling step, the number of feature channels is doubled from 64 to 128 to 256. In the upsampling path, the data is processed by repeatedly upsampling the feature map using an average unpooling step that halves the number of feature channels, a concatenation with the corresponding feature map from the downsampling path, and two 3×3 convolutions, and a nonlinearity (e.g., ReLU) is applied after each convolution.

[0175] Fig.13E According to some embodiments of the technology described herein Fig.13A Another type of architecture diagram of the blocks of a neural network model. Fig.13E The neural network model of the blocks of the architecture shown may be referred to as a "generalized non-uniform variational network" or "GNVN". Although the data consistency blocks are not used directly, features similar to the image features generated by these blocks may be useful for incorporation into a neural network model, in the sense that a neural network model having features such as Fig.13E The neural network model of the blocks of the architecture shown is "generalized".

[0176] like Fig.13E As shown, the i-th GNVN block 1360-i takes the following as input: (1) image domain data x denoted as 1362 i ; and (2) initial MR spatial frequency data 1364. Input x i The MR image reconstruction generated by the neural network 1310 at the completion of the (i-1)th GNVN block (1360-(i-1)) can be represented. These inputs of block 1360-i are then used to generate inputs to the convolutional neural network block 1372 as part of block 1360-i. In turn, based on these inputs, the CNN block 1372 generates x i+1 The next MR image is reconstructed.

[0177] exist Fig.13E In the embodiment of the present invention, inputs 1362 and 1364 are used to generate three inputs of CNN block 1372: (1) reconstructing x i itself is provided as input to the CNN block; (2) a non-uniform Fourier transform 1366, followed by a spatial frequency domain convolutional neural network 1368, followed by an adjoint non-uniform Fourier transform 1370 is applied to reconstruct x i and (3) the result of applying a spatial frequency domain convolutional neural network 1368 followed by an accompanying non-uniform Fourier transform 1370 to the initial MR spatial frequency data 1364.

[0178] In some embodiments, the non-uniform Fourier transform 1366 may be a transform A represented as a combination of three transforms (a de-apodized transform D, an oversampled Fourier transform F, and a locally gridded interpolated transform G), such that A=DF s G. Example implementations of these composition transformations are described here.

[0179] The spatial frequency domain CNN 1368 can be any suitable type of convolutional neural network. For example, the CNN 1368 can be a five-layer convolutional neural network with residual connections. However, in other embodiments, the spatial frequency domain network 1368 can be any other type of neural network (e.g., a fully convolutional neural network, a recurrent neural network, and / or any other suitable type of neural network), as aspects of the techniques described herein are not limited in this regard.

[0180] Next, we discuss neural network models for MR image reconstruction from non-Cartesian data (such as Figures 13A to 13E The neural network model shown in Figure 1 is used to describe other aspects and details of the neural network model. First, some notations are introduced. represents the complex-valued MR image to be reconstructed, which is represented as a vector and N=N x N y , where N x and N y are the width and height of the image. represents the undersampled k-space measurements used to reconstruct the complex-valued MR image x. Reconstructing x from y can be formulated as an unconstrained optimization problem:

[0181]

[0182] Among them, operator A is a non-uniform Fourier sampling operator, denotes the regularization term of x, and λ is a hyperparameter associated with the noise level. In the case of obtaining k-space measurements y using a Cartesian sampling trajectory, the operator A can be expressed according to A=MF, where M is the sampling mask and F is the discrete Fourier transform. In the case of a non-Cartesian sampling trajectory, the measurements no longer fall on a uniform k-space grid, and the sampling operator A is now given by a non-uniform discrete Fourier transform of type I:

[0183]

[0184] in (rather than ). An efficient implementation of the forward model described above can be achieved using the so-called non-uniform fast Fourier transform (NUFFT). The idea is to approximate Equation 2 by the following decomposition: A = GF s D, where G is the gridded interpolation kernel, F s is the Fast Fourier Transform (FFT) with an oversampling factor s, and D is the de-apodization weight. This decomposition will be described in further detail below.

[0185] In contrast, the inversion of A is more involved. For the (approximately) full sampling case, one can consider direct inversion ( ) or a more computationally efficient gridded reconstruction in the form of x gridding =A H Wy, where W is a diagonal matrix for density compensation of non-uniformly spaced measurements. For the under-sampling case, the inversion is ill-posed and Eq. 1 should be solved by an iterative algorithm.

[0186] The inventors have developed a new deep learning algorithm to approximate the solution to the optimization problem of Equation 1. The method first considers a gradient descent algorithm, which provides a local optimal solution for Equation 1 specified by the following equation for initialization and subsequent iterations:

[0187] x0=f init (A, y); (Formula 3)

[0188]

[0189] where f init is the initializer, α is the step size, is the gradient of the objective function, which is given by:

[0190]

[0191] In some embodiments, the initializer may be selected to accompany f init (A, y) = A H y reconstruction or gridding reconstruction f init (A, y) = AH Wy. The deep learning method used to solve Equation 1 involves expanding the sequence of updates in Equation 4 into a feed-forward model and approximating the gradient term through a series of trainable convolutional (or other types of neural network) layers and nonlinearities This method obtains N it The end-to-end trainable network of blocks is given by:

[0192] x0=f init-cnn (A, y|θ0) (Formula 6)

[0193]

[0194] The learnable parameters are Note that the step size α i is absorbed in the learnable parameters. In this way, a general non-convex regularization functional is used (e.g., instead of an expert domain model), and the regularization functional can be approximated by a convolutional neural network. In fact, Figures 13A to 13D The neural network model shown is implemented according to equations 6 and 7. For example, in Fig. 13C The implementation of the data consistency item DC-i is shown in Fig.13D The CNN-i term is shown in FIG.

[0195] The inventors have realized that the computational complexity of this method is how to achieve the forward calculation function, since A is a large complex-valued matrix that can take up a lot of memory to store. As described here, in contrast to the Cartesian case, A is represented as GF s D. For the 2D case, this can be a large matrix that consumes a large portion of the GPU memory (e.g., for N=192 2 and M = 10,000 (ie, ≈3×acceleration), storing the complex-valued matrix alone already takes up 3GB of memory). To overcome this challenge, the inventors have implemented the gridded interpolation transform Gi as a sparse GPU matrix multiplication. s is the FTT, which can be implemented efficiently on GPUs. Finally, D is a diagonal matrix, which can be implemented as a Hadamard product of matrices. The adjoint transform can be similarly implemented as A H =D H F s H G H ,in. H is the complex conjugate transpose.

[0196] Next, we describe the forward operator A = GF s More details on the decomposition of D. First, some preliminary knowledge. We can use two-dimensional or three-dimensional coordinates (for example (k x , ky ) or (k x , k y , k z )) indexes the spatial frequency domain (sometimes referred to as k-space). In this way, each entry of the vector y representing the input MR spatial frequency data represents a value associated with a particular coordinate in k-space. A regular grid in k-space refers to a regularly spaced grid of points in k-space such that there is a fixed distance Δ between each k-space coordinate that can be indexed. In general, the input MR spatial frequency data y may include k-space samples that are spaced apart or irregularly spaced on a regular grid. Regularly spaced points are sometimes referred to as Cartesian data points. Irregularly spaced points are sometimes referred to as non-Cartesian (data) points.

[0197] The interpolation transform G is used to interpolate the non-Cartesian sensor data y onto a regular k-space grid. When the transform is represented as a matrix G, each row in the matrix corresponds to a specific regular grid point in k-space, and the entry j in row i (i.e., entry G ij ) represents the weight of the association between the i-th regular grid and the j-th k-th sample.

[0198] In some embodiments, the interpolation matrix entries may be calculated using any of the following four functions:

[0199] Two cosines:

[0200] Three cosines:

[0201] Gauss:

[0202] Bessel:

[0203] Where u is the distance between the i-th regular grid point and the j-th non-Cartesian data coordinate. Parameters α, β, W, σ are free design parameters to be specified by the user, and I0 is a first-kind zero-order modified Bessel function. However, it should be understood that any other function can be used to calculate the interpolation matrix entries instead of or in the vicinity of the exemplary four functions listed above.

[0204] In some embodiments, the entries of the interpolation weight matrix can be calculated using an optimization method. For example, these entries can be calculated by solving a min-max optimization problem, as shown in Equations 16 and 21 of Fessler, JA, Sutton BP, "Non-uniform fast Fourier transforms using min-max interpolation", IEEE Transactions on Signal Processing, 51(2), pp. 560-574, 2003, which is incorporated herein by reference in its entirety. In some embodiments, the Fourier transform F can be obtained by using an oversampled Fourier matrix F s Indicates that the oversampled Fourier matrix F s is a dense matrix where each entry is of the form e depending on the index γ iγ The function of this matrix is ​​to perform Fourier transform. In some embodiments, F s This can be achieved using a Fast Fourier Transform with an oversampling factor s. For example, if the image x to be reconstructed is N×N pixels, then an oversampled FFT is performed on the image size sN×sN.

[0205] In some embodiments, the de-apodization transform may be represented by a matrix D that weights each pixel in the image by a corresponding weight to reduce the interpolation error using a given decomposition approximation A. In some embodiments, this may be achieved via pixel-wise weighting of intermediate reconstructions in the image domain. For example, a spatially varying low-order smoothing polynomial may be used to achieve pixel-wise weighting. In some embodiments, the matrix D may be set as discussed in Section IV-C of Fessler, JA, Sutton BP, "Non-uniform fast Fourier transforms using min-max interpolation," IEEE Transactions on Signal Processing, 51(2), 560-574, 2003.

[0206] The inventors also realized that Figures 13A to 13D The network enforces a bottleneck constraint at the end of each iteration. However, another view is that the network only benefits from the image features given by the data consistency (DC-i) blocks. This observation motivates a generalized approach where, instead of using data consistency blocks, Figures 13A to 13D Each CNN-i block in the model is provided with a cascade of the following inputs: the intermediate reconstruction x i , self-adjoint transformation A H Ax i And input A H The adjoint transformation of y. In addition, you can also consider using f sensor-cnn(.|φ) applies a 1D convolution in the original sensory domain to exploit the information along the sampled trajectories and remove unnecessary information (e.g., isolable artifacts or noise). Fig.13A , 13D The resulting network shown in and 13E is given by:

[0207] x0=f init-cnn (A,f sensor-cnn (y|φ0)|θ0,)x i+1

[0208] f cnn (x i , A H f sensor-cnn (Ax i |φ i ), x0|θ i , ),

[0209] The learnable parameters are As described here, this type of neural network model is called a generalized non-uniform variational network (GNVN).

[0210] The inventors have recognized that some embodiments of the neural network architectures described herein may be considered as embodiments of a neural network model that may be represented according to the following formula:

[0211] x rec =f rec (A, y|θ) (Formula 8).

[0212] This general type of neural network model can accept as input any input that is a combination of a forward operator A and the original spatial frequency domain data y and an additional learnable parameter θ (which can be of any dimension). The parameter θ can be adjusted during the training process.

[0213] The input of the neural network of Equation 8 may be data obtained by one or more RF coils of the MRI system, since aspects of the technology described herein are not limited to reconstructing an image based on data collected by a single RF coil. In addition, the input data y may be obtained using multiple contrasts and / or different acquisition parameter sets (e.g., by changing the repetition time (TR), echo time (TE), deflection angle θ, etc.). In some embodiments, the input of the network may be, but is not limited to, the raw data y. Additionally or alternatively, the input of the network may be data associated with the reconstruction A H y, where (.) H is the conjugate transpose of the matrix.

[0214] In some embodiments where the data y includes data collected by multiple RF coils, the data y may be divided into (i) (where i=1, ..., Ncoil· ) coil N separate datasets. coil It can be any number (e.g., any number in the range of 2-20, such as 8 or 9 or 10). In some such embodiments, the neural network input can be the concomitant reconstruction of each coil image. and (where i=1, ..., N coil ) can be stacked together and form the input of a network (e.g., a convolutional layer as part of a network).

[0215] In some embodiments, the raw data y may include multiple measurements obtained by each of the one or more RF coils. For example, if the data is measured multiple times (e.g., N avg times), then these data or an accompanying reconstruction of these data or any other function of these data measurements and the forward operator A may form the input to the neural network. For example, multiple measurements may be obtained for signal averaging and / or as part of acquiring images with different contrasts.

[0216] In some embodiments, as described above, the input to the neural network of Equation 8 may also be any function based on A and / or y. For example, in some embodiments, a gridded reconstruction may be an input to the network. The gridded reconstruction may have x0=A H Wy is in the form of, where W is called the sample density compensation weight, which is a matrix that scales each element in the vector y.

[0217] The sample density compensation weight W may be calculated using any of a variety of techniques. For example, in some embodiments, the sample density compensation weight W may be calculated based on W=A H A1 is used to calculate the weights W, where 1 is a vector of 1s. As another example, the weights W can be any suitable user-defined function. As yet another example, the weights W can be learned and adjusted during neural network training, in which case the weights can be referred to as learning sample density compensation weights. In some embodiments, the input to the network can be a combination of y and weights W (whether learned or fixed learnable) without using a forward operator A.

[0218] It should also be understood that the neural network need not operate on the raw data y, and in some embodiments, the data may be pre-processed. For example, in some embodiments, the data may be pre-processed to perform operations such as interference removal, denoising, filtering, smoothing, image pre-whitening, etc. More generally, the network has the form f(y, A, θ).

[0219] Regarding the neural network weights θ, these weights can be initialized in any suitable manner as part of the training process. For example, the weights can be randomly initialized (e.g., using the initialization according to Equation 12 in "Deep residual learning for image recognition" published by He, K. et al. in 2015 in the IEEE Conference Paper on Computer Vision and Pattern Recognition (CVPR), pages 1026-1034). As another example, the network weights can be initialized according to settings provided by the user. As another example, the network weights can include learned sampling density weights (e.g., the learned sampling density weights can be a subset of the network weights, the network weights can be initialized using the learned sampling density weights, and all weights can be subsequently adjusted during training).

[0220] Regarding the output of the neural network in Equation 8, the output may include one or more images for each corresponding RF coil. For example, if the input data includes images from N coil If data for each of the plurality of RF coils is provided, the output may include one MR image for each such RF coil or multiple MR images for each such coil (eg, when each coil performs multiple acquisitions, for example, using different contrast).

[0221] In some embodiments, multiple neural networks of the type specified in Equation 8 may be employed, and the outputs of these networks may be combined so that multiple neural networks are used as an ensemble. Any suitable type of aggregation rule may be used for the combination, including but not limited to averaging, weighted averaging, averaging with outlier rejection, and selection of the "best" reconstruction according to user-defined criteria (e.g., manual inspection, automatic selection based on quantitative metrics such as signal-to-noise ratio, perceptual metrics, and / or any other suitable metrics). Optionally, in some embodiments, x from a single neural network is rec Multiple instances of can be stacked together and are considered the output of the network.

[0222] As described above, there are many possible embodiments of the neural network formula of Equation 8, including but not limited to the embodiments described herein, such as: (1) as described herein (including reference Figures 13A to 13D ) as described in the non-uniform variational network (NVN); (2) as referred to herein Fig.13A , 13D and the generalized non-uniform variational network (GNVN) described in 13E; (3) as herein (including references Figures 1A to 1C ) as described in the Deep K-space Interpolation Reconstruction (DKIR) network; and (4) as described herein (including reference Figures 5A to 5C ) described in the Deep Non-local Reconstruction (DNR) network.

[0223] It should be noted that while some of the above network architectures include convolutional neural network blocks, other types of blocks may be used in addition to or instead of convolutional neural network blocks, including, for example, residual networks, densely connected networks, or squeeze and excitation networks.

[0224] In some embodiments, mean square error can be used to train any of the above networks. For example, in some embodiments, each reconstruction block in the NVN architecture (e.g., block 1316-i) or each reconstruction block in the GNVN architecture (e.g., block 1360-i) can be trained using the mean square error criterion according to the following formula:

[0225]

[0226] In some embodiments, the reconstruction block may reconstruct each coil weighted image x individually or jointly. c It is also possible to attempt to reconstruct the signals n jointly or individually avg =1,...,N avg .

[0227] Fig.14 is a flow chart of an exemplary process 1400 for generating an MR image from input MR spatial frequency data using a neural network model, wherein the MR spatial frequency data is obtained using non-Cartesian sampling, according to some embodiments of the techniques described herein. In some embodiments, the process 1400 may use a non-uniform variational network (e.g., reference Figures 13A to 13D The neural network described above), the generalized non-uniform variational network (for example, reference Fig.13A , 13D and 13E), or any other suitable type of neural network model.

[0228] In some embodiments, any suitable computing device may be used to perform the exemplary process 1400. For example, in some embodiments, the process 1400 may be performed by a computing device located in the same location (e.g., in the same room) as the MRI system that obtains the input MR spatial frequency data by imaging the subject. As another example, in some embodiments, the process 1400 may be performed by one or more processors located at a location remote from the MRI system that obtains the input spatial frequency data by imaging the subject (e.g., as part of a cloud computing environment).

[0229] Process 1400 begins at act 1402, where input MR spatial frequency data is obtained. In some embodiments, the input MR spatial frequency data has been previously obtained by the MRI system and stored for subsequent analysis, such that it is accessed at act 1402. In other embodiments, the input MR spatial frequency data may be obtained by an MRI system (including any MRI system described herein) as part of process 1400. Whenever the MRI system performs imaging to obtain input MR spatial frequency data, the data may be obtained using a non-Cartesian sampling trajectory, an example of which is provided herein.

[0230] Next, process 1400 proceeds to act 1404, where the input MR spatial frequency data may be preprocessed to obtain an initial image reconstruction. For example, in some embodiments, the input MR spatial frequency data may be transformed into the image domain using a non-uniform Fourier transform. For example, the adjoint operator A described herein may be used. H (For example, by calculating A H y) transforms the input MR spatial frequency data y into the image domain. As another example, a method such as A H The gridded reconstruction of Wy et al. transforms the input MR spatial frequency data into the image domain, where the matrix W is the sampling density compensation matrix, which can be: H A1 (where 1 is a vector of 1s), a user-specified matrix, a matrix learned during training, and / or any suitable combination thereof. Fig.13A In the illustrative example of , preprocessing can be performed by initial processing block 1312.

[0231] In some embodiments, the initializer block transforms the input MR spatial frequency data into the image domain to generate an initial image for subsequent processing by the neural network model 1310. The initializer block can be implemented in any suitable manner. For example, in some embodiments, the initializer block can apply an adjoint non-uniform Fourier transform to the input MR spatial frequency data to obtain the initial image. As another example, in some embodiments, the initializer block can apply a gridded reconstruction to the input MR spatial frequency data to obtain the initial image.

[0232] Next, process 1400 proceeds to act 1406, where the blocks of the neural network model are applied to the initial image obtained at act 1404 (or current image data when act 1406 is being performed on a loop from decision block 1408 after one or more neural network blocks have been applied to the initial image). In some embodiments, the blocks of the neural network model may be configured to perform data consistency processing by using a non-uniform Fourier transform to account for the initial MR spatial frequency data obtained at act 1402. This may be done in any suitable manner. For example, in some embodiments, the blocks may be processed by, for example, a reference to Fig. 13BThe data consistency processing may be performed by a data consistency block such as block 1316-i described above. In such a block, the data consistency processing involves transforming the intermediate reconstruction transformed to the spatial frequency domain using a non-uniform Fourier transform and comparing the result with the input MR spatial frequency data. As another example, in some embodiments, the data consistency processing may be performed by transforming the input MR spatial frequency data to the image domain using a non-uniform Fourier transform and providing the result as input to one or more convolution blocks, such as in reference to FIG. Fig.13E As in the neural network block 1360-i.

[0233] Next, the process 1400 proceeds to decision block 1408, where it is determined whether another neural network block is to be applied. If it is determined that another block is to be applied, the process 1400 returns to action 1406, where another neural network block is applied to the image data generated when the last iteration of block 1406 is completed. Otherwise, the image data is output at action 1410 as the final reconstructed MR image.

[0234] The inventors have evaluated the methods described herein (including references) for real-world MR images. Figures 13A to 13E and 14) to test the performance of the neural network architectures described. The details of these experiments are then described.

[0235] As part of the experiment, 640 randomly selected T1-weighted and T2-weighted brain images were obtained from the Human Connectome Project (https: / / www.humanconnectome.org / study / hcp-young-adult / document / 1200-subj ects-data-release). 600 of these images were used to train the neural network, while 40 of these images were used to evaluate the performance of the trained neural network. In order to perform actual simulations, multiple preprocessing steps were performed. First, since only amplitude images were provided from the Human Connectome Project, complex-valued images were created by adding phase information to the amplitude data using a two-dimensional Fourier basis with randomly sampled low-order coefficients. Secondly, the images were multiplied by a spatially localized complex coil sensitivity distribution derived from an analytical model of the MRI RF coil. Finally, an actual amount of noise that can be observed for parallel image acquisition was added to the images. For the experiments, the images were resampled to an isotropic resolution of 3.4×3.4×3.4mm 3 , 1.7×1.7×1.7mm 3 and 1.15×1.15×1.15mm 3 180×180×180mm 3 The field of view (FOV) of the matrix is ​​64. 3 , 128 3and 192 3 .

[0236] In these experiments, single coil reconstruction is evaluated to study the behavior of reconstruction of inhomogeneous MR data. The MR data are undersampled using 2D inhomogeneous variable density, where the sampling density decays quadratically from the center of k-space. For each matrix size, sampling trajectories with a target acceleration factor R∈{2,4} are generated. For evaluation, we measure the mean square error (MSE), structural similarity index measure (SSIM), and peak signal-to-noise ratio (PSNR).

[0237] The techniques developed here are developed using many conventional techniques that have been applied to inhomogeneous MR data, including: (1) AUTOMAP (Zhu B. et al., “Image reconstruction by domain-transform manifold learning”, Nature 555(7697), 487, 2018); (2) image-domain U-net (Han, Y. et al., “Deep learning with domain adaptation for acceleration projection-reconstruction MR”, Magnetic resonance in medicine 80(3), 118–1205, 2018); and (3) k-space-domain U-net. Similarly, All deep learning methods are trained using MSE. AUTOMAP is only trained for matrix sizes of 64×64 due to high GPU memory requirements. Figures 13A to 13D The NVN method of the architecture shown uses a U-net with 3-level downsampling for each convolutional sub-block (see e.g. Fig.13D ). it = 5 blocks for the number of blocks, and with A H y for f init For the GNVN method, a 5-layer convolutional neural network f is used. sensor-cnn . Use α = 10 -4 Each network was trained for 8000 epochs using the Adam optimizer with β1=0.9, β2=0.999. All methods were implemented in TensorFlow.

[0238] The evaluation results are summarized in Table 1 below. The NVN and GNVN methods developed by the inventors outperform the baseline methods for both acceleration factors. AUTOMAP and k-space U-net both perform poorly compared to the other methods.

[0239]

[0240] Table 1. Quantitative results for acceleration factors (R) of 2 and 4. For each metric, the mean and standard deviation were calculated. For the mean square error (MSE), the values ​​were scaled by 10 3 Scaling.

[0241] Between the NVN and GNVN methods, although the NVN method shows higher data fidelity (lower mean square error), the GNVN method provides better PSNR and SSIM values. Fig.15A and Fig. 15B Sample reconstructions of a T1-weighted image with R=2 and a T2-weighted image with R=4 are shown in . Although the overall differences between U-net, NVN, and GNVN are small, the reconstructions of NVN and GNVN yield lower errors due to the data consistency processing. GNVN yields the lowest overall error and preserves more fine details. However, a certain degree of blurring can be observed in all images due to the added noise. Again, the single-coil U-net (k-space) yields qualitatively suboptimal reconstructions. Fig. 15C In Figure 5, we visualize the outputs of NVN and GNVN at individual blocks. Interestingly, unlike compressed sensing methods, the intermediate images can be separated from the final image. This is not surprising since there are no constraints enforcing such behavior. For NVN, most of the outputs of individual blocks appear to be closer to the ground truth, likely because the outputs of the DC-i and CNN-i blocks are explicitly combined. In contrast, GNVN exhibits more interesting features at all intermediate levels, mainly highlighting high-frequency information.

[0242] In these experiments, the number of parameters is 128.1M, 22.0M, 6.6M, and 7.3M for AUTOMAP (64×64), U-net, NVN, and GNVN, respectively. 3 ), U-net, U-net(k-space), NVN and GNVN, with reconstruction speeds of 5.928±0.020ms, 19.145±0.072ms, 19.459±0.077ms, 44.934±0.088ms and 65.520±0.100ms, respectively.

[0243] Fig.16 is a block diagram of typical components of an MRI system 1600. Fig.16 In the illustrative example of MRI system 1600, a workstation 1604, a controller 1606, a pulse sequence storage unit 1608, a power management system 1610, and a magnetic assembly 1620. It should be understood that system 1600 is illustrative and that in addition to Fig.16 In addition to or in place of the components shown Fig.16 In addition to the components shown, the MRI system may have one or more other components of any suitable type.

[0244] like Fig.16 As shown, the magnetic assembly 1620 includes a B0 magnet 1622, a shim coil 1624, an RF transmit and receive coil 1626, and a gradient coil 1628. The B0 magnet 1622 can be used to at least partially generate a main magnetic field B0. The B0 magnet 1622 can be any suitable type of magnet capable of generating a main magnetic field (e.g., a low field strength of about 0.2 T or less), and can include one or more B0 coils, correction coils, etc. The shim coil 1624 can be used to contribute a magnetic field to improve the uniformity of the B0 field generated by the magnet 1622. The gradient coil 1628 can be arranged to provide a gradient field, and for example, can be arranged to generate gradients in the magnetic field along three substantially orthogonal directions (X, Y, Z) to locate the position where the MR signal is induced.

[0245] The RF transmit and receive coils 1626 may include one or more transmit coils that can be used to generate RF pulses to induce a magnetic field B1. The transmit / receive coils may be configured to generate any suitable type of RF pulses that are configured to excite the MR response of the subject and detect the resulting MR signals emitted. The RF transmit and receive coils 1626 may include one or more transmit coils and one or more receive coils. The structure of the transmit / receive coil varies with implementation and may include a single coil for both transmitting and receiving, a separate coil for transmitting and receiving, a plurality of coils for transmitting and / or receiving, or any combination to implement a single channel or parallel MRI system. Therefore, the transmit / receive magnetic assembly is often referred to as Tx / Rx or Tx / Rx coil to generally refer to the various structures of the transmit and receive components of an MRI system.

[0246] Each magnetic assembly 1620 can be of any suitable type and can be constructed in any suitable manner. For example, in some embodiments, the B0 magnet 1622 can be an electromagnet or a permanent magnet (e.g., as described below with reference to Figures 17A to 17B As another example, in some embodiments, one or more magnetic components 1620 (eg, shim coils 1624 and / or gradient coils 1628) may be manufactured using lamination techniques.

[0247] The power management system 1610 includes electronics for providing operating power to one or more components of the low-field MRI system 1600. For example, the power management system 1610 may include one or more power supplies, gradient power amplifiers, transmit coil amplifiers, and / or any other suitable power electronics required to provide suitable operating power to energize and operate the components of the low-field MRI system 1600.

[0248] like Fig.16 As shown, the power management system 1610 includes a power supply 1612, an amplifier 1614, a transmit / receive switch 1616, and a thermal management component 1618. The power supply 1612 includes an electronic device for providing operating power to a magnetic component 1620 of the low-field MRI system 1600. For example, in some embodiments, the power supply 1612 may include an electronic device for providing operating power to one or more B0 coils (e.g., B0 magnet 1622), one or more shim coils 1624, and / or one or more gradient coils 1628 for generating the main magnetic field of the low-field MRI system. In some embodiments, the power supply 1612 may be a unipolar continuous wave power supply, however, any suitable power supply may be used. The transmit / receive switch 1616 may be used to select whether the RF transmit coil is being operated or the RF receive coil is being operated.

[0249] In some embodiments, the amplifier 1614 may include one or more RF receive (Rx) pre-amplifiers for amplifying MR signals detected by one or more RF receive coils (e.g., coil 1624), one or more RF transmit (Tx) amplifiers configured to provide power to one or more RF transmit coils (e.g., coil 1626), one or more gradient power amplifiers configured to provide power to one or more gradient coils (e.g., gradient coil 1628), and / or one or more shim amplifiers configured to provide power to one or more shim coils (e.g., shim coil 1624).

[0250] In some embodiments, the thermal management component 1618 provides cooling for components of the low-field MRI system 1600 and may be configured to provide cooling by facilitating the transfer of thermal energy generated by one or more components of the low-field MRI system 1600 away from these components. The thermal management component 1618 may include, but is not limited to, a component for water-based or air-based cooling that may be integrated with or disposed in the vicinity of MRI components that generate heat, including but not limited to B0 coils, gradient coils, shim coils, and / or transmit / receive coils. The thermal management component 1618 may include any suitable heat transfer medium for transferring heat away from components of the low-field MRI system 1600, including but not limited to air and water.

[0251] like Fig.16 As shown, the low-field MRI system 1600 includes a controller 106 (also referred to as a console) having control electronics for sending instructions to the power management system 110 and receiving information from the power management system 1610. The controller 1606 may be configured to implement one or more pulse sequences that are used to determine the instructions sent to the power management system 1610 to operate the magnetic assembly 1620 in a desired sequence. For example, the controller 1606 may be configured to control the power management system 1610 to operate the magnetic assembly 1620 according to a balanced steady-state free precession (bSSFP) pulse sequence, a low-field gradient echo pulse sequence, a low-field spin echo pulse sequence, a low-field inversion recovery pulse sequence, arterial spin labeling, diffusion weighted imaging (DWI), and / or any other suitable pulse sequence. The controller 1606 may be implemented as hardware, software, or any suitable combination of hardware and software, as aspects of the invention provided herein are not limited thereto.

[0252] In some embodiments, the controller 1606 may be configured to implement a pulse sequence by obtaining information related to the pulse sequence from a pulse sequence repository 1608, which stores information of one or more pulse sequences. The information stored in the pulse sequence repository 1608 for a particular pulse sequence may be any suitable information that allows the controller 1606 to implement the particular pulse sequence. For example, the information stored in the pulse sequence repository 1608 for a pulse sequence may include one or more parameters for operating the magnetic assembly 1620 according to the pulse sequence (e.g., parameters for operating the RF transmit and receive coils 1626, parameters for operating the gradient coils 1628, etc.), one or more parameters for operating the power management system 1610 according to the pulse sequence, one or more programs containing instructions that, when executed by the controller 1606, cause the controller 1606 to control the system 1600 to operate according to the pulse sequence, and / or any other suitable information. The information stored in the pulse sequence repository 1608 may be stored on one or more non-transitory storage media.

[0253] like Fig.16 As shown, in some embodiments, the controller 1606 can interact with a computing device 1604 that is programmed to process the received MR data (which, in some embodiments, can be spatial frequency domain MR data). For example, the computing device 1604 can process the received MR data to generate one or more MR images using any suitable image reconstruction process (including using any of the techniques described herein that utilize a neural network model to generate MR images from spatial frequency MR data). For example, the computing device 1604 can perform the processing described herein. Figure 2A , 2B, 2C, 2D, and 14. The controller 1606 may provide information related to one or more pulse sequences to the computing device 1604 for processing the data by the computing device. For example, the controller 1606 may provide information related to one or more pulse sequences to the computing device 1604, and the computing device may perform image reconstruction processing based at least in part on the provided information.

[0254] In some embodiments, the computing device 1604 may be any electronic device configured to process the acquired MR data and generate one or more images of the subject being imaged. In some embodiments, the computing device 1604 may include a fixed electronic device, such as a desktop computer, a server, a rack computer, or any other suitable fixed electronic device that may be configured to process the MR data and generate one or more images of the subject being imaged, etc. Alternatively, the computing device 1604 may be a portable device, such as a smartphone, a personal digital assistant, a laptop computer, a tablet computer, or any other portable device that may be configured to process the MR data and generate one or more images of the subject being imaged, etc. In some embodiments, the computing device 1304 may include a plurality of computing devices of any suitable type, as aspects of the technology described herein are not limited thereto.

[0255] In some embodiments, the user 1602 can interact with the computing device 1604 to control aspects of the low-field MR system 1600 (e.g., programming the system 1600 to operate according to a particular pulse sequence, adjusting one or more parameters of the system 1600, etc.) and / or view images obtained by the low-field MR system 1600. According to some embodiments, the computing device 1604 and the controller 1606 form a single controller, while in other embodiments, the computing device 1604 and the controller 1606 each include one or more controllers. It should be understood that the functions performed by the computing device 1604 and the controller 1606 can be distributed in any manner over any combination of one or more controllers, as aspects of the technology described herein are not limited to use with any particular implementation or architecture.

[0256] Fig.17A and 17B A dual-plane permanent magnet structure of a B0 magnet is shown according to some embodiments of the technology described herein. Fig.17AA permanent B0 magnet 2100 according to some embodiments is shown. In the illustrated embodiment, the B0 magnet 2100 is formed by permanent magnets 2110a and 2110b arranged in a bi-planar geometry and a yoke 2120 for capturing the electromagnetic flux generated by the permanent magnets and transferring the flux to the opposing permanent magnets to increase the flux density between the permanent magnets 2110a and 2110b. The permanent magnets 2110a and 2110b are each formed by a plurality of concentric permanent magnet rings. In particular, as shown in FIG. Fig.17A As can be seen in the figure, permanent magnet 2110b includes an outer permanent magnet ring 2114a, an intermediate permanent magnet ring 2114b, an inner permanent magnet ring 2114c, and a permanent magnetic disk 2114d located at the center. Although shown as having four concentric permanent magnet rings, permanent magnet 2110b (and permanent magnet 2110a) can have any suitable number of permanent magnet rings, as aspects of the technology described herein are not limited thereto. Permanent magnet 2110a can be formed substantially the same as permanent magnet 2110b, and for example include a set of permanent magnet rings identical to permanent magnet 2110b.

[0257] The permanent magnet material used can be selected according to the design requirements of the system. For example, according to some embodiments, the permanent magnet (or a portion thereof) can be made of NdFeB, which, once magnetized, produces a magnetic field in a manner that has a relatively high magnetic field per unit volume of material. In some embodiments, SmCo material is used to form the permanent magnet or a portion thereof. Although NdFeB produces higher field strengths (and is generally cheaper than SmCo), SmCo exhibits less thermal drift and therefore provides a more stable magnetic field in the face of temperature fluctuations. Other types of permanent magnet materials may also be used, as aspects of the technology described herein are not limited thereto. In general, the type of permanent magnet material utilized will depend at least in part on the field strength, temperature stability, weight, cost, and / or ease of use requirements achieved by a given B0 magnet.

[0258] In some embodiments, the size and arrangement of the permanent magnet rings are designed to produce a uniform field of desired strength in the imaging region (field of view) between permanent magnets 2110a and 2110b. Fig.17AIn the exemplary embodiment shown, each permanent magnet ring includes a plurality of segments, each segment is formed using a plurality of permanent magnet blocks stacked in a radial direction and positioned to be positioned adjacent to each other around the periphery to form a corresponding ring. The inventors have realized that by changing the width of each permanent magnet (in a direction tangential to the ring), less waste of useful space can be achieved while using less material. For example, by changing the width of the block (e.g., as a function of the radial position of the block), the space between stacks that do not produce a useful magnetic field can be reduced, thereby allowing a tighter fit to reduce wasted space and maximize the amount of magnetic field that can be produced in a given space. The size of the block can also be changed in any desired manner to facilitate the generation of a magnetic field with a desired strength and uniformity. For example, in some embodiments, the heights of the blocks of different rings can be different from each other, and / or the heights of one or more blocks within a particular ring can be different from each other to achieve a magnetic field with a desired strength and uniformity.

[0259] like Fig.17A As shown, the B0 magnet 2100 also includes a yoke 2120, which is configured and arranged to capture the magnetic flux generated by the permanent magnets 2110a and 2110b and guide it to the opposite side of the B0 magnet to increase the flux density between the permanent magnets 2110a and 2110b, thereby increasing the field strength within the field of view of the B0 magnet. By capturing the magnetic flux and guiding it to the area between the permanent magnets 2110a and 2110b, less permanent magnet material can be used to achieve the desired field strength, thereby reducing the size, weight and cost of the B0 magnet 2100. Optionally, for a given permanent magnet, the field strength can be increased, thereby improving the SNR of the system without using an increased amount of permanent magnet material. For a typical B0 magnet 2100, the yoke 2120 includes a frame 2122 and plates 2124a and 2124b. The plates 2124a and 2124b can capture the magnetic flux generated by the permanent magnets 2110a and 2110b and direct it to the frame 2122 to circulate through the magnetic loop of the yoke, thereby increasing the flux density in the field of view of the B0 magnet. The yoke 2120 can be constructed of any desired ferromagnetic material (e.g., mild steel, CoFe and / or silicon steel, etc.) to provide the yoke with desired magnetic properties. In some embodiments, the plates 2124a and 2124b (and / or the frame 2122 or portions thereof) can be constructed of silicon steel, etc., in the area where the gradient coils can most commonly induce eddy currents.

[0260] Typical frame 2122 comprises arms 2123a and 2123b attached to plates 2124a and 2124b, respectively, and supports 2125a and 2125b providing magnetic circuits for the flux generated by the permanent magnets. The arms are generally designed to reduce the amount of material required for supporting the permanent magnets, while providing sufficient loop cross-sections for the magnetic flux generated by the permanent magnets. Frame 2122 has two supports in the magnetic circuit for the B0 field generated by the B0 magnets. A gap 2127 is formed between supports 2125a and 2125b, thereby providing a measure of stability for the frame and / or for the lightness of the structure while providing sufficient cross-sections for the magnetic flux generated by the permanent magnets. For example, the cross-section required for the loop of the magnetic flux can be divided between two support structures, thereby providing sufficient loops while increasing the structural integrity of the frame.

[0261] Fig. 17B FIG. 2 shows a B0 magnet 2200 according to some embodiments. The B0 magnet 2200 may be used with Fig.17A The B0 magnet 2100 shown shares design components. In particular, the B0 magnet 2200 is formed by permanent magnets 2210a and 2210b arranged in a dual-plane geometry and a yoke 2220 coupled to the permanent magnets 2210a and 2210b to capture the electromagnetic flux generated by the permanent magnets and transfer the flux to the relative permanent magnet to increase the flux density between the permanent magnets 2210a and 2210b. The permanent magnets 2210a and 2210b are each formed by a plurality of concentric permanent magnets, as shown in the permanent magnet 2210b, which includes an outer permanent magnet ring 2214a, an intermediate permanent magnet ring 2214b, an inner permanent magnet ring 2214c, and a permanent magnetic disk 2214d located at the center. The permanent magnet 2210a can include a set of permanent magnet elements identical to the permanent magnet 2210b. The permanent magnet material used can be selected according to the design requirements of the system (eg, NdFeB, SmCo, etc., depending on the desired properties).

[0262] The size and arrangement of the permanent magnet rings are designed to produce a uniform field of the desired strength in the central region (field of view) between permanent magnets 2210a and 2210b. Fig. 17B In an exemplary embodiment of the present invention, each permanent magnet ring includes a plurality of arc segments, the size and position of which are designed to generate a desired B0 magnetic field. Fig.17AIn a similar manner to the yoke 2120 shown, the yoke 2220 is configured and arranged to capture the magnetic flux generated by the permanent magnets 2210a and 2210b and guide it to the opposite side of the B0 magnet to increase the flux density between the permanent magnets 2210a and 2210b. The yoke 2220 thereby increases the field strength within the field of view of the B0 magnet with less permanent magnet material, reducing the size, weight and cost of the B0 magnet. The yoke 2220 also includes a frame 2222 and plates 2224a and 2224b, which capture the magnetic flux generated by the permanent magnet 2210a in a manner similar to that described above in conjunction with the yoke 2220 and circulate it via the magnetic circuit of the yoke to increase the flux density in the field of view of the B0 magnet. The structure of the yoke 2220 can be similar to the above structure to provide sufficient material to accommodate the magnetic flux generated by the permanent magnets and provide sufficient stability, while minimizing the amount of material used to, for example, reduce the cost and weight of the B0 magnet.

[0263] Since the permanent B0 magnet will generate its own persistent magnetic field once magnetized, no electricity is required to operate the permanent B0 magnet to generate its magnetic field. As a result, the use of permanent magnets (as opposed to, for example, electromagnets that require electricity) eliminates a significant (usually dominant) contribution to the overall power consumption of the MRI system, thereby promoting the development of MRI systems that can be powered by mains electricity (e.g., via a standard wall socket or an ordinary large household appliance socket). As described above, the inventors have developed a low-power, portable, low-field MRI system that can be deployed in almost any environment and can be brought to a patient who will undergo an imaging procedure. In this way, patients in emergency rooms, intensive care units, operating rooms, and many other places can benefit from MRI when MRI is not normally available.

[0264] Fig.18A and 18BA view of a portable MRI system 3800 according to some embodiments of the technology described herein is shown. The portable MRI system 3800 includes a B0 magnet 3810 formed in part by an upper magnet 3810A and a lower magnet 3810B, the B0 magnet 3810 having a yoke 3820 coupled to the B0 magnet 3810 to increase the flux density within the imaging region. The B0 magnet 3810 can be housed in a magnet housing 3812 together with a gradient coil 3815 (e.g., any gradient coil described in U.S. Application No. 14 / 845,652, entitled “Low Field Magnetic Resonance Imaging Methods and Apparatus” and filed on September 4, 2015, which is incorporated herein by reference in its entirety). In some embodiments, the B0 magnet 3810 includes an electromagnet. In some embodiments, the B0 magnet 3810 comprises a permanent magnet (e.g., any of the permanent magnets described in U.S. Application No. 15 / 640,369, filed on June 30, 2017, entitled “LOW-FIELD MAGNETIC RESONANCE IMAGING METHODS AND APPARATUS,” which is incorporated herein by reference in its entirety). For example, in some embodiments, the B0 magnet 3810 may be a permanent magnet such as a permanent magnet such as a permanent magnet such as a permanent magnet such as a permanent magnet such as a permanent magnet Fig.17A The permanent magnet 2100 or reference Fig. 17B The permanent magnet 2200.

[0265] The exemplary portable MRI system 3800 also includes a base 3850 for accommodating the electronic components for operating the MRI system. For example, the base 3850 can accommodate electronic components including, but not limited to, one or more gradient power amplifiers, a system computer, a power distribution unit, one or more power supplies, and / or any other power components configured to operate the MRI system using mains power (e.g., via a connection to a standard wall outlet and / or a large electrical outlet). For example, the base 3870 can accommodate low-power components such as those described herein, at least in part, enabling the portable MRI system to be powered from an easily accessible wall outlet. Therefore, the portable MRI system 3800 can be brought to the patient and plugged into a wall outlet near the patient.

[0266] The portable MRI system 3800 also includes a movable slide 3860, which can be opened and closed and positioned in various structures. The slide 3860 includes an electromagnetic shield 3865, which can be made of any suitable conductive material or magnetic material to form a movable shield, so as to attenuate the electromagnetic noise in the operating environment of the portable MRI system so that the imaging area is shielded from at least some electromagnetic noise. As used herein, the term electromagnetic shield refers to a conductive or magnetic material that is configured to attenuate the electromagnetic field in the spectrum of interest and is positioned or arranged to shield the space, object and / or component of interest. In the context of an MRI system, electromagnetic shielding can be used to shield the electronic components (e.g., power components, cables, etc.) of the MRI system, to shield the imaging area (e.g., field of view) of the MRI system, or both.

[0267] The degree of attenuation achieved by an electromagnetic shield depends on many factors, including the type of material used, the thickness of the material, the frequency spectrum for which electromagnetic shielding is desired or required, the size and shape of the holes in the electromagnetic shield (e.g., the size of the spaces in the conductive mesh, the size of the unshielded portions or gaps in the shield, etc.), and / or the orientation of the holes relative to the incident electromagnetic field. Thus, electromagnetic shielding generally refers to any conductive or magnetic barrier that is used to attenuate at least some electromagnetic radiation and is positioned to at least partially shield a given space, object, or component by attenuating at least some electromagnetic radiation.

[0268] It should be understood that the spectrum of the desired shielding (attenuation of the electromagnetic field) can be different depending on what is shielded. For example, the electromagnetic shielding of certain electronic components can be configured to attenuate different frequencies compared to the electromagnetic shielding of the imaging area of ​​the MRI system. With respect to the imaging area, the spectrum of interest includes frequencies that change, affect and / or reduce the ability of the MRI system to excite and detect MR responses. In general, the spectrum of interest in the imaging area of ​​the MRI system corresponds to a frequency related to the rated operating frequency (e.g., Larmor frequency) at a given B0 magnetic field strength that the receiving system is configured to or can detect. This spectrum is referred to here as the operating spectrum of the MRI system. Therefore, the electromagnetic shielding that provides shielding for the operating spectrum refers to a conductive or magnetic material that is arranged or positioned to attenuate the frequency within the operating spectrum of at least a portion of the imaging area of ​​the MRI system.

[0269] exist Fig.18A and 18BIn the illustrated portable MRI system 3800, the movable shields 18A and 18B can thus be configured to provide shielding in different arrangements that can be adjusted as needed to accommodate the patient, provide access to the patient, and / or conform to a given imaging protocol. For example, for an imaging procedure such as a brain scan, once the patient has been positioned, the handle 3862 can be used, for example, to close the slide 3960 to provide an electromagnetic shield 3965 around the imaging region in addition to an opening to accommodate the patient's upper torso. As another example, for an imaging procedure such as a knee scan, the slide 3960 can be arranged to have openings on both sides to accommodate the patient's legs. Thus, the movable shield enables the shield to be configured in an arrangement suitable for the imaging procedure and facilitates proper positioning of the patient within the imaging region.

[0270] In some embodiments, a noise reduction system including one or more noise reduction and / or compensation techniques may be implemented to suppress at least some electromagnetic noise that is not blocked or sufficiently attenuated by the shield 3865. In particular, the inventors have developed a noise reduction system that is configured to suppress, avoid and / or reject electromagnetic noise in the operating environment in which the MRI system is located. According to some embodiments, these noise suppression techniques work in conjunction with movable shields to facilitate operation in various shielding structures in which the slide can be arranged. For example, when the slide 3960 is opened, the level of increased electromagnetic noise will likely enter the imaging area via the opening. As a result, the noise suppression component will detect the increased electromagnetic noise level and adapt the noise suppression and / or avoidance response accordingly. Due to the dynamic nature of the noise suppression and / or avoidance techniques described herein, the noise reduction system is configured to respond to changing noise conditions (including noise conditions generated by different arrangements of movable shields). Therefore, the noise reduction system according to some embodiments may be configured to work in conjunction with movable shields to suppress electromagnetic noise in the operating environment of the MRI system in any shielding structure available (including a structure that is substantially shieldless (e.g., a structure without movable shields)).

[0271] To ensure that the movable shield provides shielding regardless of the arrangement in which the slide is placed, the electrical gasket may be arranged to provide continuous shielding along the periphery of the movable shield. Fig.18B As shown, electrical gaskets 3867a and 3867b may be disposed at the interface between the slider 3860 and the magnet housing to maintain continuous shielding along the interface. According to some embodiments, the electrical gaskets are beryllium detents or beryllium copper detents, etc. (e.g., aluminum shims) used to maintain electrical connection between the shield 3865 and ground during and after the slider 3860 is moved to a desired position around the imaging area.

[0272] To facilitate transport, a motorized assembly 3880 is provided to allow the portable MRI system to be driven from one location to another using, for example, controls such as a joystick or other control mechanism disposed on or remote from the MRI system. In this manner, the portable MRI system 3800 can be transported to the patient and maneuvered to the bedside for imaging.

[0273] The portable MRI system described herein can be operated from a portable electronic device such as a laptop, tablet computer, smart phone, etc. For example, a tablet computer 3875 can be used to operate a portable MRI system to run a desired imaging protocol and view the resulting images. The tablet computer 3875 can be connected to a secure cloud to transmit images for data sharing, telemedicine, and / or deep learning on a data set. Any technology using a network connection described in U.S. Application No. 14 / 846,158, entitled "Automatic Configuration of a Low Field Magnetic Resonance Imaging System," filed on September 4, 2015, can be utilized in conjunction with the portable MRI system described herein, and the above-mentioned document is incorporated herein by reference in its entirety.

[0274] As discussed above, Fig. 18C A portable MRI system 3900 is shown transported to a patient's bedside for performing a brain scan. Fig.18D Portable MRI system 3900 is shown transported to a patient's bedside to perform a scan of the patient's knee. Fig.18D As shown, shield 3960 has electrical gasket 3867c.

[0275] It should be understood that Figures 18A to 18D The electromagnetic shielding shown is typical, and providing shielding for the MRI system is not limited to the exemplary electromagnetic shielding described herein. Electromagnetic shielding can be implemented in any suitable manner using any suitable material. For example, a conductive mesh, fabric, etc. that can provide a removable "curtain" to shield the imaging area can be used to form an electromagnetic shield. One or more conductive tapes (e.g., one or more conductive material tapes) coupled to the MRI system can be used as fixed, removable, or configurable components to form an electromagnetic shield to shield the imaging area from electromagnetic interference, some examples of which will be described in further detail below. Electromagnetic shielding can be provided by embedding the material in any removable or fixed part of a door, a slide, or a housing. The electromagnetic shielding can be deployed as a fixed or removable component, because the aspect is not limited thereto.

[0276] Fig.19 is a diagram of an illustrative computer system in which the embodiments described herein may be implemented. Fig.19An exemplary implementation of a computer system 1900 that can be used in conjunction with any of the embodiments of the present invention provided herein is shown in FIG. Figures 2A to 2D The processing described in and 14 can be implemented on and / or using computer system 1900. As another example, computer system 1900 can be used to train and / or use any neural network statistical model described herein. Computer system 1900 may include one or more processors 1910 and one or more articles of manufacture including non-transitory computer-readable storage media (e.g., memory 1920 and one or more non-volatile storage media 1930). Processor 1910 can control data writing and data reading relative to memory 1920 and non-volatile storage device 1930 in any suitable manner, as the aspects of the present invention provided herein are not limited thereto. In order to perform any of the functions described herein, processor 1910 can execute one or more processor-executable instructions stored in one or more non-transitory computer-readable storage media (e.g., memory 1920), which can be used as a non-transitory computer-readable storage medium for storing processor-executable instructions for execution by processor 1910.

[0277] Therefore, after describing several aspects and embodiments of the technology set forth in the present invention, it should be understood that those skilled in the art will easily make various changes, modifications and improvements. Such changes, modifications and improvements are intended to be within the spirit and scope of the technology described here. For example, a person of ordinary skill in the art will easily conceive of various other components and / or structures for performing functions and / or obtaining results and / or one or more advantages described herein, and these changes and / or modifications are each considered to be within the scope of the embodiments described here. Those skilled in the art will recognize or be able to determine that only routine experiments, that is, many equivalent embodiments of the specific embodiments described here, have been used. Therefore, it should be understood that the above embodiments are presented only by way of example, and within the scope of the attached claims and their equivalents, in addition to specific descriptions, embodiments of the present invention may be implemented in other ways. In addition, if the features, systems, articles, materials, kits and / or methods described herein are not inconsistent with each other, any combination of two or more such features, systems, articles, materials, kits and / or methods is included within the scope of the present invention.

[0278] The above embodiments can be implemented in any number of ways. One or more aspects and embodiments of the present invention related to the performance of processing or method can utilize program instructions executable by a device (e.g., a computer, a processor or other device) to process or method or control the performance of processing or method. In this regard, various inventive concepts can be embodied as a computer-readable storage medium (or multiple computer-readable storage media) encoded with one or more programs (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, tapes, flash memory, circuit structures in field programmable gate arrays or other semiconductor devices, or other tangible computer storage media), which are used to implement one or more of the above-mentioned various embodiments when executed on one or more computers or other processors. The computer-readable medium can be transportable so that the program stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the above-mentioned aspects. In some embodiments, the computer-readable medium can be a non-temporary medium.

[0279] As used herein, the term "program" or "software" refers in a general sense to any type of computer code or computer executable instruction set that can be used to program a computer or other processor to implement the above-mentioned aspects. In addition, it should be understood that, according to one aspect, one or more computer programs that perform the methods of the present invention when executed do not need to reside on a single computer or processor, but can be distributed in a modular manner between multiple different computers or processors to implement various aspects of the present invention.

[0280] Computer executable instructions may take a variety of forms, such as program modules executed by one or more computers or other devices. In general, program modules include routines, programs, objects, components, data structures, etc. for performing specific tasks or implementing specific abstract data types. In general, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0281] In addition, the data structure may be stored in a computer-readable medium in any suitable form. To simplify the description, the data structure may be shown as having fields that are related by position in the data structure. Likewise, such a relationship can be achieved by assigning the fields with storage having a position in a computer-readable medium for transmitting the relationship between the fields. However, any suitable mechanism may be used to establish the relationship between the information in the fields of the data structure, including by using pointers, tags or other mechanisms that establish the relationship between the data elements.

[0282] When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0283] Furthermore, it should be appreciated that a computer may be embodied in any of a variety of forms such as, by way of non-limiting example, a rack-mount computer, a desktop computer, a laptop computer, or a tablet computer, etc. Additionally, a computer may be embedded in a device not generally regarded as a computer but with suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device.

[0284] In addition, the computer may have one or more input and output devices. These devices may be used to present a user interface, etc. Examples of output devices that can be used to provide a user interface include a printer or display screen for visual presentation of the output and a speaker or other sound generating device for audible presentation of the output. Examples of input devices that can be used for a user interface include a keyboard and a pointing device, such as a mouse, a touch pad, and a digitized tablet computer. As another example, the computer may receive input information by voice recognition or in other audible formats.

[0285] Such computers may be interconnected by one or more networks of any suitable form, including local or wide area networks, such as enterprise networks and intelligent networks (IN) or the Internet, etc. Such networks may be based on any suitable technology and may operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0286] In addition, as described, some aspects may be embodied as one or more methods. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which the actions are performed in a different order than shown, which may include performing some actions simultaneously, even though in an illustrative embodiment, they are shown as sequential actions.

[0287] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0288] As used herein, in the specification and claims, the indefinite articles "a" and "an" should be understood to mean "at least one" unless explicitly indicated to the contrary.

[0289] As used herein, the phrase "and / or" should be understood to refer to "either or both" of the elements so combined (i.e., elements that are presented combined in some cases and separately in other cases). Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so combined. In addition to the elements specifically identified by the "and / or" clause, other elements may optionally be present, whether related or unrelated to those specifically identified elements. Thus, as a non-limiting example, in one embodiment, a reference to "A and / or B", when used in conjunction with open language such as "including", may refer to A only (optionally including elements other than B); in another embodiment, may refer to B only (optionally including elements other than A); in yet another embodiment, may refer to both A and B (optionally including other elements); and so on.

[0290] In the specification and claims, the phrase "at least one" as used herein, when referring to a list of one or more elements, should be understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one element of each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows that elements other than the elements specifically identified in the list of elements referenced by the phrase "at least one" may optionally be present, whether related to or unrelated to the specifically identified elements. Thus, as a non-limiting example, "at least one of A and B" (or equivalently "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one that optionally includes more than one A without B (and optionally includes elements other than B) in one embodiment; may refer to at least one that optionally includes more than one B without A (and optionally includes elements other than A) in another embodiment; may refer to at least one that optionally includes more than one A and at least one that optionally includes more than one B (and optionally includes elements other than A) in yet another embodiment; and so on.

[0291] In the claims, as well as in the above description, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," and "covering" shall be construed as open-ended, i.e., meaning including, but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively.

[0292] The terms "about" and "approximately" may be used to mean within ±20% of a target value in some embodiments, within ±10% of a target value in some embodiments, within ±5% of a target value in some embodiments, within ±2% of a target value in some embodiments. The terms "about" and "approximately" may include a target value.

Claims

1. A method for generating a magnetic resonance image, comprising: An MR image is generated based on input magnetic resonance spatial frequency data, i.e., input MR spatial frequency data, using a neural network model, the neural network model comprising: A first neural network sub-model configured to process spatial frequency domain data; and a second neural network sub-model configured to process image domain data; wherein the first neural network sub-model comprises one or more convolutional layers for down-sampling the spatial frequency domain data, followed by one or more transposed convolutional layers for up-sampling the spatial frequency domain data, and / or The second neural network sub-model includes one or more convolutional layers for down-sampling the image domain data, followed by one or more transposed convolutional layers for up-sampling the image domain data.

2. The method according to claim 1, wherein: The generating comprises: processing the input MR spatial frequency data using the first neural network sub-model to obtain output MR spatial frequency data; transforming the output MR spatial frequency data into an image domain to obtain input image domain data; and The input image domain data is processed using the second neural network sub-model to obtain the MR image.

3. The method according to claim 1, wherein: The first neural network sub-model comprises at least one convolutional layer having a stride greater than 1, and / or wherein, The second neural network sub-model includes at least one convolutional layer with a stride greater than 1.

4. The method according to claim 1, wherein: The first neural network sub-model includes at least one locally connected layer.

5. The method according to claim 1, wherein: The first neural network sub-model includes at least one data consistency layer.

6. The method according to claim 1, wherein: The first neural network sub-model includes at least one complex conjugate symmetric layer.

7. The method according to claim 1, wherein: The first neural network sub-model includes residual connections.

8. The method according to claim 1, wherein: The first neural network sub-model includes a rectified linear unit activation layer.

9. The method according to claim 1, wherein: The first neural network sub-model includes a layer with leaky rectified linear unit activation.

10. The method according to claim 1, wherein: The first neural network sub-model includes at least one convolutional layer, a local connection layer and at least one transposed convolutional layer, and Processing the input MR spatial frequency data using the first neural network sub-model includes: applying the at least one convolutional layer to the input MR spatial frequency data; applying the locally connected layer to data obtained using an output of the at least one convolutional layer; and The at least one transposed convolutional layer is applied to data obtained using the output of the locally connected layer.

11. The method according to claim 10, wherein: The first neural network sub-model includes a complex conjugate symmetric layer, and using the first neural network sub-model to process the input MR spatial frequency data includes: The complex conjugate symmetric layer is applied to data obtained using the output of the at least one transposed convolutional layer.

12. The method according to claim 10, wherein: The first neural network sub-model includes a data consistency layer, and using the first neural network sub-model to process the input MR spatial frequency data includes: The data consistency layer is applied to data obtained using the output of the complex conjugate symmetric layer.

13. The method according to claim 10, wherein: Each convolutional layer in the at least one convolutional layer has a stride greater than 1.

14. The method according to claim 10, wherein: The locally connected layer includes a corresponding set of parameter values ​​for each data point in the MR spatial frequency data.

15. The method according to claim 1, wherein: The first neural network sub-model includes at least one fully connected layer.

16. The method according to claim 1, wherein: The first neural network sub-model includes a fully connected layer, and the method further includes: Applying the fully connected layer to the real part of the spatial frequency domain data; The fully connected layer is applied to the imaginary part of the spatial frequency domain data.

17. The method according to claim 1, wherein: The first neural network sub-model includes a first fully connected layer and a second fully connected layer, and the method further includes: Applying the first fully connected layer to the real part of the spatial frequency domain data; The second fully connected layer is applied to the imaginary part of the spatial frequency domain data.

18. The method according to claim 17, wherein: The first fully connected layer and the second fully connected layer share at least some weights.

19. The method according to claim 17, further comprising: Using Fourier transform to transform the outputs of the first fully connected layer and the second fully connected layer to obtain image domain data; as well as The image domain data is provided as input to the second neural network sub-model.

20. The method according to claim 19, further comprising: Batch normalization is applied to outputs of the first fully connected layer and the second fully connected layer before using Fourier transform to obtain image domain data.

21. The method according to claim 1, wherein: The points in the spatial frequency domain data are irregularly spaced.

22. The method according to claim 1, wherein: The points in the spatial frequency domain data are not on a Cartesian grid.

23. The method according to claim 1, wherein: The MR image is generated without gridding the spatial frequency domain data.

24. The method according to claim 1, wherein: The second neural network sub-model includes a series of blocks including corresponding sets of neural network layers, each block in the plurality of blocks including at least one convolutional layer and at least one transposed convolutional layer.

25. The method according to claim 24, wherein: Each of the plurality of blocks further includes a Fourier transform layer, a data consistency layer, and an inverse Fourier transform layer.

26. The method of claim 1, further comprising: The neural network model is trained using a loss function, wherein the loss function includes a spatial frequency domain loss function and an image domain loss function.

27. The method according to claim 26, wherein: The loss function is a weighted sum of the spatial frequency domain loss function and the image domain loss function.

28. The method according to claim 27, wherein: The spatial frequency domain loss function includes a mean square error.

29. The method of claim 1, further comprising: Using the high field image set to train the neural network model to obtain a first trained neural network model; as well as The first trained neural network model is adapted by using a low-field image set.

30. The method of claim 1, wherein: The spatial frequency domain data is undersampled relative to the Nyquist criterion.

31. A system for generating a magnetic resonance image, comprising: at least one computer hardware processor; as well as At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to: The MR image is generated based on the magnetic resonance spatial frequency data, i.e., the MR spatial frequency data, using a neural network model, the neural network model comprising: a first neural network portion configured to process data in the spatial frequency domain; and a second neural network portion configured to process data in an image domain; wherein the first neural network portion comprises one or more convolutional layers for downsampling data in the spatial frequency domain, followed by one or more transposed convolutional layers for upsampling data in the spatial frequency domain, and / or Wherein, the second neural network portion includes one or more convolutional layers for down-sampling data in the image domain, followed by one or more transposed convolutional layers for up-sampling data in the image domain.

32. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to: The MR image is generated based on the magnetic resonance spatial frequency data, i.e., the MR spatial frequency data, using a neural network model, the neural network model comprising: a first neural network portion configured to process data in a spatial frequency domain; as well as a second neural network portion configured to process data in an image domain; wherein the first neural network portion comprises one or more convolutional layers for downsampling data in the spatial frequency domain, followed by one or more transposed convolutional layers for upsampling data in the spatial frequency domain, and / or Wherein, the second neural network portion includes one or more convolutional layers for down-sampling data in the image domain, followed by one or more transposed convolutional layers for up-sampling data in the image domain.

33. A magnetic resonance imaging system, i.e., an MRI system, comprising: Magnetic system, including: a B0 magnet configured to provide a B0 field for the MRI system; a gradient coil configured to provide a gradient field for the MRI system; and at least one radio frequency coil, or RF coil, configured to detect magnetic resonance signals, or MR signals; A controller configured to: controlling the magnetic system to acquire MR spatial frequency data; and An MR image is generated from the MR spatial frequency data using a neural network model, the neural network model comprising: a first neural network portion configured to process data in the spatial frequency domain; and a second neural network portion configured to process data in an image domain; wherein the first neural network portion comprises one or more convolutional layers for downsampling data in the spatial frequency domain, followed by one or more transposed convolutional layers for upsampling data in the spatial frequency domain, and / or Wherein, the second neural network portion includes one or more convolutional layers for down-sampling data in the image domain, followed by one or more transposed convolutional layers for up-sampling data in the image domain.

34. The MRI system of claim 33, wherein: The B0 magnet is a permanent magnet.

35. A computer program product comprising processor-executable instructions which, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the method of any one of claims 1 to 30.

Images