Highly-scalable image reconstruction using deep convolutional neural networks with bandpass filtering

Abstract

A method for magnetic resonance imaging (MRI) scans a field of view and acquires sub-sampled multi-channel k-space data U. An imaging model A is estimated. Sub-sampled multi-channel k-space data U is divided into sub-sampled k-space patches, each of which is processed using a deep convolutional neural network (ConvNet) to produce corresponding fully-sampled k-space patches, which are assembled to form fully-sampled k-space data V, which is transformed to image space using the imaging model adjoint Aadj to produce an image domain MRI image. The processing of each k-space patch ui preferably includes applying the k-space patch ui as input to the ConvNet to infer an image space bandpass-filtered image yi, where the ConvNet comprises repeated de-noising blocks and data-consistency blocks; and estimating the fully-sampled k-space patch vi from the image space bandpass-filtered image yi using the imaging model A and a mask matrix.

Description

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT[0001]This invention was made with Government support under contracts R01 EB019241 and R01-EB009690 awarded by the National Institutes of Health. The Government has certain rights in the invention.FIELD OF THE INVENTION[0002]The present invention relates generally to techniques for magnetic resonance imaging. More specifically, it relates to improved methods for magnetic resonance image reconstruction and artifact reduction.BACKGROUND OF THE INVENTION[0003]The ability to reconstruct magnetic resonance (MR) images from vastly undersampled acquisitions has significant clinical value. It allows the duration of the MR scan to be reduced and enables the visualization of rapid hemodynamics.[0004]Using advanced image reconstruction algorithms, images can be reconstructed with negligible loss in image quality despite high undersampling factors (R>6). To achieve this performance, algorithms exploit the data acquisition model w...

AI Technical Summary

Benefits of technology

[0010]In contrast to prior techniques, which divide imaging data into image-domain patches for training and inference, the present invention divides the imaging data into frequency-domain patches. At the same time, the techniques of the present invention leverage the use of the imaging model to ensure that the reconstructed images do not deviate from the undersampled measurement data. The invention also can naturally account for images with differing resolutions and sizes by reconstructing different frequency bands independently. The technique is able to train and apply a model for images of varying resolutions which increases the flexibility of the network and minimize the need to re-train the network for each specific case.

[0011]The techniques of the present invention train and apply ConvNets on patches of k-space domain data. In other words, a bandpass filter is used to select and isolate the reconstruction to small localized patches in the k-space domain. With contiguous patches of k-space, the ability to exploit the data acquisition model is maintained which enables a ConvNet architecture to enforce consistency with the measured data. Also, by selecting small patches of k-space domain, the input data sizes into the ConvNets are reduced which decreases the memory footprint and increases the computational speed. This smaller memory requirement enables the processing of extremely large datasets in terms of size of each dimension and / or the number of dimensions. Thus, the possible resolutions are not limited by the computation hardware or the acceptable computation duration for high-speed applications. Each k-space patch can be reconstructed independently which enables simple parallelization of the algorithm that further reduces the reconstruction times. All these features allow for this type of ConvNet to be applied and trained on high-dimensional (≥256) and multi-dimensional (two, three, and higher dimensional) images.

[0013]The processing of the sub-sampled k-space patches to produce corresponding fully-sampled k-space patches preferably involves processing each k-space patch ui of the sub-sampled k-space patches separately and independently from other patches to produce a corresponding fully-sampled k-space patch vi, thereby allowing for parallel processing.

[0019]The techniques of the present invention perform rapid and robust image reconstruction for magnetic resonance imaging scans that are prospectively subsampled. Subsampling reduces the acquisition time for each scan, reducing the total MRI exam duration. The techniques of the invention are especially useful for situations where the reconstruction is memory limited, as in the case of multi-dimensional imaging (three or more dimensions) that may include volumetric spatial dimensions, cardiac motion, respiratory motion, contrast-enhancement, velocity, diffusion, and echo dimensions. This invention can be applied for the scaling and enlargement of images for display in high-resolution displays and for prints. This invention enables the flexibility to use a single trained network for the enlargement of images to different sizes and spatiotemporal resolutions. Further, these techniques can be applied to other imaging applications where the measurement is performed in the image frequency domain.

Problems solved by technology

These traditional iterative algorithms, however, have considerable computational complexity for undersampled data.

There are various challenges in applying ConvNets to MRI reconstruction, however.

Consequently, many of the known techniques for image processing with ConvNets do not directly translate to MRI image reconstruction.

Without a data consistency step, the ConvNets may “hallucinate” new structures in the image or remove existing ones, leading to erroneous diagnosis.

On the other hand, if an attempt is made to use a data consistency step, the training and application can not be image-patch based, because if only small image patches are used, known information in the measurement domain (k-space domain) is lost.

This limitation increases the memory footprint of the ConvNet and decreases the speed of training and inference.

In addition, existing ConvNet techniques are not easily extendable to high-dimensional MR images and multi-dimensional MR images, because the training and inference of the ConvNet can never be fully parallelized: specific steps within the ConvNet (such as transforming from k-space domain to image domain) requires the gathering of all data before proceeding to the next step of the network.

Images