A multiple contextual information-based adaptive deep learning for controllable magnetic resonance image reconstruction

Abstract

An MRI image reconstruction system (100) enhances MRI image quality and accuracy by integrating an MRI scanner (102) that maps under-sampled k-space measurements to fully sampled MRI image (106) A deep learning model (108), controlled by a processor (110), drives the reconstruction. This model includes a dynamic weight prediction auxiliary network (112) and a base reconstruction convolutional neural network (118). The auxiliary network (112) processes acquisition context (114) to create context-specific weights (116), which the convolutional network (118) uses to process the under-sampled data (106). This process generates a context-specific fully-sampled MRI image. The final reconstructed MRI image (124) is obtained by utilizing the scanner settings or the acquisition context information together with the acquired under-sampled k-space (104)

Description

[0001] A MULTIPLE CONTEXTUAL INFORMATION-BASED ADAPTIVE DEEP LEARNING FOR CONTROLLABLE MAGNETIC RESONANCE IMAGE RECONSTRUCTION

[0002] TECHNICAL FIELD

[0003] The present invention relates to an MR image reconstruction method that is adaptable to varying acquisition / scan settings, to improve the scan quality of the MR image reconstruction as well as provide a scalable approach that supports numerous acquisition settings.

[0004] BACKGROUND

[0005] Background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently disclosed invention, or that any publication specifically or implicitly referenced is prior art.

[0006] Magnetic Resonance Imaging (MRI) is a non-invasive modality that covers a wide range of protocols, each designed for optimally assessing anatomical regions or pathological processes in neuro diagnosis, spine, cardiac, musculoskeletal, and soft tissue, extending to chronic, degenerative diseases, cancer, infection, injuries, and abnormalities. The fundamental challenge in MRI is the long acquisition time and complex workflows with multiple parameters, ways of imaging, and different conditions that require different imaging protocols. To achieve cost-effective imaging methods and improved applicability of MRI, the key functional enablers include reduced complexity, better throughput, and quality. We address these challenges, from the perspective of deep learning that has recently emerged in medical imaging enabling new trends in healthcare applications.

[0007] Conventional MRI workflows use Compressed sensing MRI (CS-MRI) methods for reconstruction. CS-MRI methods have been used to accelerate the acquisition and are deployed in clinical environments. However, parameter tweaking, and iterative computation of non-linear optimization solvers lead to relatively long reconstruction times. Secondly, inappropriate choice of hyper-parameters in current CS-MRI methods can cause over-regularization that yields overly smooth and unnatural-looking reconstructions or images with residual under- sampling artifacts.

[0008] Capitalizing on the diverse imaging system of MRI, recent research has seen a surge in using deep learning (DL) models for various imaging tasks to develop efficient MR imaging workflows at every stage of imaging, right from acquisition to patient- specific longitudinal analysis, and has generally returned dividends in performance. To this end, data-driven learning-based methods can offer a faster and more effective solution for image reconstruction with promising image quality.

[0009] Contemporary deep learning models can effectively operate only for a specific configuration of settings used at train time. This limitation poses deployment challenges in translating these models to an MRI workstation in real clinical environments due to the data diversity of MRI that leads to numerous acquisition settings. We refer to the models that operate for a specific configuration of acquisition settings (or acquisition contexts) as context- specific models (CSMs). CSMs suffer from the limitation of overfitting and necessitate a combinatorial number of models to be trained and stored for different configurations of acquisition contexts. Examples of acquisition contexts in MRI reconstruction are the different combinations of acceleration factors, under- sampling mask patterns, and the anatomy under study. For instance, combinations of five acceleration factors - 2x, 3.3x, 4x, 5x, and 8x, two under- sampling mask types - Cartesian and Gaussian and two anatomies under study - knee, and brain would generate 20 ACs, that is, 20 CSMs. Storing CSMs for each context makes the maintainability and configuration management of workflows challenging.

[0010] Another way to train the model is by combining the data from all acquisition contexts. This approach is also referred to as joint training on different acquisition contexts limits the model to learn a context- agnostic or shared representation with a single weight set. This limitation might result in inadequate scalability to multiple contexts, under-fitting specific contexts, and the deterioration of the reconstruction quality. To summarize, a conventional DL models trained for a specific context or jointly trained on multiple contexts have the following limitations. 1. Models have to be trained for every configuration of acquisition contexts to operate on different configurations of the scanner. 2. Models lack discriminative context- specific knowledge of multiple acquisition contexts together with a shared representation across contexts in a single model for scalable image reconstruction.

[0011] Therefore, there is a need for a device or a system in the MRI workflow that addresses these limitations to realize a single scalable model capable of reconstructing images acquired with multiple acquisition settings combining different acceleration factors, mask patterns, and anatomy under study. The model also exhibits generalization to continuously varying settings, namely the acceleration factor, including unknown acceleration factors.

[0012] SUMMARY OF THE INVENTION

[0013] The following presents a simplified summary of the subject matter in order to provide a basic understanding of some aspects of subject matter embodiments. This summary is not an extensive overview of the subject matter. It is not intended to identify key / critical elements of the embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description that is presented later.

[0014] The MRI image reconstruction system disclosed here involves a new process that can potentially be integrated into the reconstruction stage of the MRI workflow. The MRI image reconstruction system is used for improving the scan quality of under-sampled MRI scans. The MRI image reconstruction system provides a new application and improves existing methods. The MRI image reconstruction system explicitly utilizes important scan-specific information required for under-sampled reconstruction, namely acceleration factor, under- sampling mask type, and the anatomy under study, each combination representing an acquisition context. The MRI image reconstruction system is a model-based meta-leaming method that gains common knowledge across different acquisition contexts and the knowledge specific to each context.

[0015] The MRI image reconstruction system involves a method that comprises mapping the under-sampled measurements (or under-sampled k-space) from an MRI scanner to a fully-sampled MRI image. Acquisition context information is then fed from the MRI scanner to a dynamic weight prediction auxiliary network. The dynamic weight prediction network outputs context- specific weights which are used to process the hidden image features of a convolutional neural network (CNN). The CNN maps the under-sampled k- space measurements to fully sampled reconstructed image specific to the context.

[0016] In an embodiment, the method includes applying a Fourier transformation to the scanner measurements to create the under- sampled MRI data. An inverse Fourier transformation is then applied to this under-sampled data before it is processed by the base reconstruction convolutional neural network. In an embodiment, the training data is prepared from the under-sampled MRI data and input into the base reconstruction convolutional neural network. This training data, along with the context- specific weights, is used to process the under- sampled MRI data.

[0017] In an embodiment, training data and validation data are utilized in a correction module to correct the processed under-sampled MRI data, improving the accuracy of the reconstruction. The acquisition context information includes details such as the anatomy of the subject, the under- sampling pattern of the k-space, and the degree of undersampling or the acceleration factor of under-sampling, all derived from the MRI scanner’s settings. The dynamic weight prediction auxiliary network receives scanner setting information, which includes a numerical array representing anatomy, under- sampling pattern, and degree of under-sampling. The network predicts context- specific weights based on this information. The MRI image reconstruction system has the ability to dynamically adjust the base reconstruction CNN weights using the output of the dynamic weight prediction auxiliary network in response to changes in scanner settings. In an embodiment, the context- specific weights modulate the convolutional features of a main image reconstruction Al system, allowing for the reconstruction of images from under-sampled measurements with varying MRI scanner settings without the need for retraining. The base reconstruction convolutional neural network is pre-trained to reconstruct images from under- sampled MRI data. The dynamic weight prediction auxiliary network provides additional context- specific adjustments to enhance the accuracy of the reconstruction based on specific MRI scanner settings. The MRI scanner adjusts the under- sampling pattern and amount of under- sampling according to predefined protocols, which influences the scanner setting information provided to the dynamic weight prediction auxiliary network.

[0018] BRIEF DESCRIPTION OF FIGURES

[0019] The foregoing and further objects, features and advantages of the present subject matter will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings, wherein like numerals are used to represent like elements.

[0020] It is to be noted, however, that the appended drawings along with the reference numerals illustrate only typical embodiments of the present subject matter, and are therefore, not to be considered for limiting its scope, for the subject matter may admit to other equally effective embodiments.

[0021] Figures 1A and IB exemplarily illustrate a schematic diagram of the MRI image reconstruction system and a flow chart that show the real-world process flow associated with the MRI image reconstruction system, as an example embodiment of the present disclosure. MCI-Net in Figure IB refers to the Multiple Contextual Information-based network and the dynamic weight prediction auxiliary network of the MRI image reconstruction system.

[0022] Figure 2 exemplarily illustrates a concept diagram of multiple acquisition contextual information based neural network for MRI reconstruction. Figure 3 exemplarily illustrates a concept diagram that shoes infusing context-specific weights from the meta-leamer to the base network.

[0023] DETAILED DESCRIPTION

[0024] Illustrative examples of the subject matter will now be disclosed. In the interest of clarity, not all features of an actual implementation are described in this specification. It will be appreciated that in the development of any such actual implementation, numerous implementation- specific decisions may be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort, even if complex and time-consuming, would be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.

[0025] Exemplary embodiments now will be described with reference to the accompanying drawings. The disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey its scope to those skilled in the art. The terminology used in the detailed description of the particular exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting. In the drawings, like numbers refer to like elements.

[0026] It is to be noted, however, that the reference numerals used herein illustrate only typical embodiments of the present subject matter, and are therefore, not to be considered for limiting its scope, for the subject matter may admit to other equally effective embodiments.

[0027] The specification may refer to “an”, “one” or “some” embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

[0028] As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms “includes”, “comprises”, “including” and / or “comprising” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Furthermore, “connected” or “coupled” as used herein may include operatively connected or coupled. As used herein, the term “and / or” includes any and all combinations and arrangements of one or more of the associated listed items.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0030] The figures depict a simplified structure only showing some elements and functional entities, all being logical units whose implementation may differ from what is shown. The connections shown are logical connections; the actual physical connections may be different. It is apparent to a person skilled in the art that the structure may also comprise other functions and structures.

[0031] Also, all logical units described and depicted in the figures include the software and / or hardware components required for the unit to function. Further, each unit may comprise within itself one or more components which are implicitly understood. These components may be operatively coupled to each other and be configured to communicate with each other to perform the function of the said unit.

[0032] DEFINITIONS:

[0033] The term “MRI scanner” as used herein, refers to a device that uses magnetic fields and radio waves to create detailed images of the internal structures of the body.

[0034] The term “Scanner measurements” as used herein, refers to raw data collected by the MRI scanner during imaging, which can include signal intensities and spatial information.

[0035] The term “Under-sampled MRI data” as used herein, refers to MRI data that has been intentionally sampled less frequently than normally required, often to speed up the imaging process or reduce data acquisition time.

[0036] The term “Deep learning model” as used herein, refers to a computational model that uses neural networks with multiple layers to process data and make predictions or decisions based on learned patterns.

[0037] The term “Processor” as used herein, refers to a hardware or software component responsible for executing instructions, managing tasks, and controlling operations within the deep learning model.

[0038] The term “Dynamic weight prediction auxiliary network” as used herein, refers to a part of the deep learning model that generates weights based on context- specific information, used to adapt the model’s performance to varying conditions.

[0039] The term “Acquisition context information” as used herein, refers to information related to the MRI scanning procedure, such as anatomical details, scanning patterns, and settings, which helps tailor the reconstruction process.

[0040] The term “Context-specific weights” as used herein, refers to adjustments or parameters predicted by the dynamic weight prediction deep learning model to improve the accuracy of MRI image reconstruction based on specific scanning conditions.

[0041] The term “Base reconstruction convolutional neural network” as used herein, refers to a type of neural network used for processing and reconstructing images from under-sampled MRI data by learning and applying spatial patterns. The term “Context- specific K-space” as used herein, refers to a frequency domain representation of MRI data that is adapted based on specific context information to improve the reconstruction of MRI images.

[0042] The term “Reconstructed MRI image” as used herein, refers to the final high- quality image produced from the processed and corrected MRI data, representing internal body structures.

[0043] The term “Training data” as used herein, refers to data used to train the deep learning model, consisting of example MRI scans with known corrections, to help the model learn how to accurately reconstruct images.

[0044] The term “Correction module” as used herein, refers to a component that uses training and validation data to refine and correct the output of the image reconstruction process for improved accuracy.

[0045] The term “Validation data” as used herein, refers to Data used to assess the performance of the deep learning model during training, ensuring that the model generalizes well to new, unseen data.

[0046] Figures 1A and IB exemplarily illustrate a schematic diagram of the MRI image reconstruction system and a flow chart that show the real-world process flow associated with the MRI image reconstruction system, as an example embodiment of the present disclosure. In short, these measurements are converted to under-sampled images and fed to the reconstruction network. The acquisition context information is fed to the metalearner. The predicted measurements and the original under-sampled measurements are combined (measurement correction step) to obtain the final context- specific reconstructed image output.

[0047] In detail, referring to Figures A and IB, the MRI image reconstruction system (100) is designed to enhance the quality and accuracy of MRI images. At its core, the system integrates an MRI scanner (102) that generates scanner measurements (104), which then form under-sampled MRI data (106). The reconstruction process is driven by a deep learning model (108), controlled by at least one processor (110). The deep learning model (108) includes a dynamic weight prediction auxiliary network (112) and a base reconstruction convolutional neural network (118). The dynamic weight prediction auxiliary network (112) processes acquisition context information (114) from the MRI scanner (102) to produce context- specific weights (116). These context- specific weights (116) are utilized by the base reconstruction convolutional neural network (118) to process the under-sampled MRI data (106), ultimately generating a context- specific K- space (122). The reconstructed MRI image (124) is produced by combining the scanner measurements (104) and the context- specific K-space (122).

[0048] Application of Fourier Transformations: In the MRI image reconstruction system (100), Fourier transformations play a crucial role. A Fourier transformation is applied to the scanner measurements (104) to create the under-sampled MRI data (106). This undersampled MRI data (106) is then fed into the base reconstruction convolutional neural network (118) after an inverse Fourier transformation. This dual transformation process helps in accurately reconstructing the MRI images by converting the spatial data into frequency data and vice versa, ensuring that the base reconstruction convolutional neural network (118) can effectively process and enhance the image quality.

[0049] Training Data Preparation: Here, the use of under-sampled MRI data (106) to prepare training data (126) is described. This training data (126) is critical for input into the base reconstruction convolutional neural network (118). By utilizing the training data (126) along with context- specific weights (116), the MRI image reconstruction system (100) effectively processes the under-sampled MRI data (106). This approach ensures that the base reconstruction convolutional neural network (118) is well-trained to handle various scenarios and produce high-quality reconstructed images, leveraging the dynamic adjustments provided by the context- specific weights (116).

[0050] Correction Module Functionality: the MRI image reconstruction system (100) includes a correction module (120). This correction module (120) is responsible for utilizing both training data (126) and validation data to correct the processed under-sampled MRI data (106). The correction process ensures that any discrepancies or errors in the initial processing are addressed, resulting in more accurate and reliable MRI images. This correction module (120) adds an additional layer of refinement to the image reconstruction process. Acquisition Context Information: The acquisition context information (114) is a critical component of the MRI image reconstruction system (100). This acquisition context information (114) includes specifics about the subject's anatomy, the under- sampling pattern, and the acceleration factor, all derived from the settings of the MRI scanner (102). By incorporating these details, the MRI image reconstruction system (100) tailors the reconstruction process to the specific conditions under which the MRI data was acquired, thereby improving the accuracy and relevance of the reconstructed images.

[0051] Dynamic Weight Prediction: The MRI image reconstruction system (100) also includes the role of the dynamic weight prediction auxiliary network (112), which receives scanner setting information from the acquisition context information (114). This scanner setting information is represented as a numerical array, encompassing details such as anatomy, under- sampling pattern, and the extent of under-sampling. The dynamic weight prediction auxiliary network (112) uses this scanner setting information to predict context-specific weights (116), dynamically adjusting them in response to changes in the scanner settings. This adaptability allows the MRI image reconstruction system (100) to maintain high reconstruction accuracy across varying conditions.

[0052] Modulation of Convolutional Features: The context- specific weights (116) produced by the dynamic weight prediction auxiliary network (112) play a pivotal role in modulating the convolutional features of the main image reconstruction Al system within the MRI image reconstruction system (100). This capability enables the reconstruction of images from under-sampled measurements even when the settings of the MRI scanner (102) vary. This flexibility means that the MRI image reconstruction system (100) does not require re-training for different scanner configurations, significantly enhancing its utility and efficiency.

[0053] Pre-training and Context- Specific Adjustments: The base reconstruction convolutional neural network (118) is pre-trained to handle the reconstruction of images from undersampled MRI data (106). The dynamic weight prediction auxiliary network (112) then provides additional context- specific adjustments to this pre-trained network. These adjustments are based on the specific settings of the MRI scanner (102), further enhancing the accuracy and quality of the reconstructed images. This combination of pre-training and dynamic adjustment ensures robust performance of the MRI image reconstruction system (100) across different imaging scenarios.

[0054] Influence of Predefined Protocols: The MRI image reconstruction system (100) also highlights the role of predefined protocols in influencing the under- sampling pattern and amount of under- sampling applied by the MRI scanner (102). These predefined protocols dictate the scanner setting information provided to the dynamic weight prediction auxiliary network (112). By adhering to these predefined protocols, the MRI image reconstruction system (100) can maintain consistency and reliability in the acquisition and reconstruction of MRI images, ensuring that the context- specific weights (116) are always based on accurate and relevant information.

[0055] Referring to Figure 2, the process begins with the collection (202) of diverse datasets from various acquisition context information (114) (1, 2, ..., N). These datasets (204) undergo data pre-processing (206), which includes normalization and cropping. The pre- processed data is then fed into a dynamic weight prediction auxiliary network (112) within the Base network (118) of the MCI-Net - DL model (108), or the deep learning model (108). This deep learning model (108) generates a context- specific model (122) that produces a predicted reconstruction (208). The quality of this reconstruction is evaluated (210), and if the reconstruction error is not minimized (212), the process iterates back to the MCI-Net - DL model (108) for further training. If the reconstruction error is minimized (212) but the epochs are not yet completed (214), the process continues through the MCI-Net - DL model (108) for additional training. Once the epochs are completed (214) and the reconstruction error is minimized, the training is considered complete (216).

[0056] In view of the above, the prior art devices / methods primarily comprise MRI reconstruction methods that perform well for specific acquisition settings. These methods involve training a neural network once, resulting in a fixed set of weights tailored to a particular scanner scenario. In contrast, the MRI image reconstruction system (100) disclosed here offers flexibility by allowing the neural network to adjust its weights based on the acquisition setting after training.

[0057] Prior art methods are unable to function properly when there are deviations in the scanner acquisition settings. However, the MRI image reconstruction system (100) is robust against such deviations, enabling it to generalize to new under- sampling amounts and patterns. While the prior art relies solely on a convolutional neural network for image reconstruction, the MRI image reconstruction system (100) incorporates a dynamic weight prediction neural network that modulates the convolution features of the reconstruction network, providing flexibility based on scanner settings.

[0058] This MRI image reconstruction system (100) predicts different neural network weights for varying acquisition settings, simplifying the design and configuration of Al-driven workflows for MRI radiologists with minimal neural network training efforts during reconstruction. Consequently, these Al-driven workflows are faster than traditional workflows that rely on iterative optimization-based methods for image reconstruction.

[0059] The MRI image reconstruction system (100) can be extensively used in clinical settings, especially in the MRI workflow. The invention can reduce the scan time and training time of models significantly and can provide better reconstruction quality. The concept diagram of the invention is outlined in Figure 2.

[0060] Figure 2 exemplarily illustrates a concept diagram of multiple acquisition contextual information based neural network for MRI reconstruction. As illustrated in Figures 1A and IB:

[0061] 1. On-the-fly MRI image reconstruction of under-sampled images from multiple different contexts combining anatomies, mask patterns, and acceleration factors, using a single model.

[0062] 2. Generalize to unseen acceleration factors that change continuously in the degradation amounts. The inventive method can generate numerous context- specific weights and potentially eliminate the need for training task-specific models.

[0063] 3. Faster adaptation and learning without forgetting multiple new contexts. 4. Task-based learning as the model considers the meta-data information about the data by means of a context vector.

[0064] 5. The invention is easily extensible to multi-coil MRI reconstruction to adaptively learning multiple coil configurations involving varying numbers of coils. At test time, the model can generalize to an unseen coil configuration. The acquisition context in this case is a numerical vector that indicates the presence or absence of a particular coil.

[0065] Figure 3 exemplarily illustrates a concept diagram that shoes infusing context-specific weights from the meta- learner to the base network. The meta- learner is a fully connected network which takes the contextual information in the form of an acquisition context vector and gives out context- specific weights which are reshaped to the corresponding base network convolution layer and convolved with the layer features.

[0066] Referring to Figures 2 and 3, the neural network for MRI reconstruction incorporates training datasets, pre-processing steps, acquisition context input, and image input to build a scalable MRI reconstruction model based on the acquisition context information, at the same time, improves the quality of the reconstructed images from under-sampled MRI scans. These aspects are described as follows:

[0067] 1) Acquisition context- specific dataset(s):

[0068] Each combination of three acquisition settings, namely anatomy type, mask type and acceleration factor form an acquisition context. These settings are explained as follows. Various acquisition settings as given below are considered for the process: i) Anatomy type: This setting signifies the type of anatomy and the MRI contrast associated with it. Examples of anatomy types include cardiac, brain, and knee, and MRI contrast types include Tl, and FLAIR (Fluid Attenuated Inversion Recovery). Anatomy type is a categorical acquisition setting. ii) Mask type: Mask type signifies the trajectory pattern of the MRI scanner for under- sampling. Examples of mask types include Cartesian, radial, and spiral. The mask type is a categorical acquisition setting. iii) Acceleration factor: The acceleration factor decides the amount of undersampling. An acceleration factor of 2 indicates that the acquired measurements from the MRI scanner are prospectively under-sampled by 50%. 2) Data pre-processing:

[0069] Standardizing the various artifact dataset using the following steps: i) Cropping: Region of interest (ROI) defined by the anatomy is extracted. ii) Normalization: The intensity profile of the heterogeneous dataset is modified such that each image has contrast information fitted within a defined range.

[0070] 3) Inputs to the model:

[0071] The two inputs to the model are the under- sampled MR images and the acquisition context information represented as a vector. The context vector represents the enumerated information about the anatomy under study, mask type and the acceleration factor for under- sampling. The anatomy under study and the mask type are discrete settings but the acceleration factor is a continuously varying setting.

[0072] 4) DL model:

[0073] The DL model is a parameterized neural network that has two modules - a base reconstruction module and a meta-learner.

[0074] • The base reconstruction module is a neural network that takes the under- sampled MRI image as input and predicts the fully sampled image as output. The base reconstruction network is a convolutional neural network. The base neural network consists of a convolutional layer, instance norm layer and nonlinear activation function.

[0075] • The meta-leamer is a dynamic weight prediction neural network that takes the acquisition context vector as the input and outputs acquisition context- specific weights, which are infused into the base reconstruction module. The network architecture of the meta-learner is a multi-layer perceptron network.

[0076] The two modules synchronize together to provide context- specific MR reconstructed images as output. A parameterized neural network architecture consisting of the two networks facilitates a functional mapping from the input degraded image to the corresponding clean image at its output. The model is adaptive to multiple acquisition contexts combining anatomy type, mask type, and acceleration factor. In addition, the model can interpolate and extrapolate to unseen contexts along the dimension of the acceleration factor, which is a continuously varying setting. The parameters of these modules are appropriately adjusted to adaptive reconstruct the output image based on the acquisition context.

[0077] 5) Context-specific model:

[0078] From a single DL model, based on the acquisition context, the meta learner generates context- specific weights which are convolved with the layers of the base reconstruction network to obtain context- specific features. Hence, a separate expert model for every context is obtained, using a correction mechanism.

[0079] 6) Meta-leaming approach:

[0080] Meta-leaming is a data-efficient task-based learning process that enables the model to learn a strong prior from the dataset of several imaging acquisition contexts and effectively uses the experience to quickly adapt to novel imaging contexts. Specifically, model-based meta-leaming is used to predict the weights that are used by another neural network to obtain task-specific knowledge, that can be used to interpolate and extrapolate in task contexts without the need for additional training of models. We exploit this paradigm by considering multi-modal MRI data with acquisition contexts of a diverse set of MRI contrasts. Our focus lies on building meta-leaming models for regression tasks like image reconstruction with real-world multi-modal MRI data with a view to promoting these methods to be deployed in ubiquitous clinical scenarios.

[0081] 7) Correction mechanism and evaluation mechanism:

[0082] The DL model consists of a neural network with weights. The DL model involves two steps for operation - training and validation. These two steps together constitute the correction mechanism of the DL model iteratively during training. The correction mechanism during training is achieved using backpropagation that updates the parameters of the DL model by optimizing an error function. The evaluation mechanism involves computing the error function, which is the mean-squared error loss function between the desired outputs to achieved system outputs using validation data samples. The evaluation mechanism uses the loss function to find the best set of weights of the convolutional neural network and the dynamic weight prediction network that minimizes the error function. The inventive step consists of incorporating a model-based meta-leamer (dynamic weight prediction network) to the base convolutional neural network architecture that performs multiple acquisition context- specific MR image reconstruction. The meta-leamer takes acquisition context- specific information as an input vector and gives context- specific weights as output. These weights are used to modulate the convolution layer of the base network, which performs the image reconstruction task.

[0083] 8) Image quality assessment measure: a metric that is used to quantify the image quality of the output provided by the model. These and other objects, embodiments and advantages of the present invention will become readily apparent to those skilled in the art from the following detailed description of the embodiments having reference to the attached figures, the invention not being limited to any particular embodiments disclosed.

[0084] The disclosed neural network for MRI reconstruction has been tested on the following datasets. To consider various scenarios for achieving a good balance between high reconstruction quality and enabling capabilities for multiple contextual settings, we have considered diverse datasets with multiple contrasts, mask patterns, and widely spanning acceleration factors. Our datasets are as follows, (i) The Cardiac ACDC dataset contains 1841 training and 1076 validation slices of 150 and 50 subjects of the cardiac anatomy. The images are cropped to 150x150, (ii) The knee dataset contains 200 training and validation single channel slices of 10 subjects, each obtained from a 15-element coil knee data cropped to 320x320, (iii) The MRBrainS dataset contains 60 training and 144 validation slices of 5 and 2 subjects of brain anatomy with Tl, T2-FLAIR, and Tl-IR volumes, acquired on a 3.0T Philips Achieva scanner (TR = 7.9 ms \& TE = 4.5 ms) of size 240x240 (iv) The SRI24 atlas dataset contains 63 training and 32 validation T1W brain slices of brain anatomy, acquired on a 3.0T GE scanner (TR = 6.5 ms, TE = 1.54 ms) of size 240x240 (v) The fastMRI dataset contains 7040 training, 3317 validation and 3903 testing slices of 200, 99 and 108 volumes respectively of coronal proton-density without (PD) and with fat suppression (PDFS).

[0085] In general, the disclosed neural network for MRI reconstruction contributes to these various aspects: a. Contemporary deep learning models have to be trained and stored for every configuration of acquisition contexts. Given the wide data diversity in MRI, this would lead to a combinatorial number of models being trained and stored. The proposed method employs a model-based meta-leamer to address this limitation and can scale to multiple settings using a single model. b. Contemporary deep learning models do not learn a strong prior by jointly training on multiple datasets and learn a shared representation that might not work well for certain contexts. The task-specific models cannot interpolate and extrapolate in contexts involving continuously varying acquisition settings like acceleration factor. As a result, generalization to unseen settings is not possible. The proposed meta-leamer can interpolate unseen acceleration factors at the same time, matching the accuracy of context- specific models. c. Given datasets from new contexts, task-specific models cannot be adopted quickly due to poor prior experience. The proposed approach leverages the learning experience from multiple acquisition contexts to effectively learn multiple new contexts rapidly.

[0086] It will be understood that each block of the block diagrams can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0087] In the drawings and specification, there have been disclosed exemplary embodiments of the invention. Although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation of the scope of the invention.

[0088] The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the disclosure. However, it will be apparent to one skilled in the art that the specific details are not required in order to practice the systems and methods described herein. The foregoing descriptions of specific examples are presented for purposes of illustration and description. They are not intended to be exhaustive of or to limit this disclosure to the precise forms described. Obviously, many modifications and variations are possible in view of the above teachings. The examples are shown and described in order to best explain the principles of this disclosure and practical applications, to thereby enable others skilled in the art to best utilize this disclosure and various examples with various modifications as are suited to the particular use contemplated.

Claims

We Claim:

1. An MRI image reconstruction system (100), comprising: an MRI scanner (102) to generate scanner measurements (104) and forming under-sampled MRI data (106); a deep learning model (108) controlled by at least one processor (110), the deep learning model (108) comprising:• a dynamic weight prediction auxiliary network (112) to receive acquisition context information (114) from the MRI scanner (102) and output context- specific weights (116);• a base reconstruction convolutional neural network (118) that processes the under-sampled MRI data (106) using the contextspecific weights (116); wherein the deep learning module (108) generates a context- specific K-space (122) from the corrected under- sampled MRI data (106), and wherein the scanner measurements (104) and the context- specific K-space (122) are combined to generate a reconstructed MRI image (124).

2. The MRI image reconstruction system (100) as claimed in claim 1, wherein a Fourier transformation is applied to scanner measurements (104) to form the undersampled MRI data (106), and an inverse Fourier transformation is applied to the undersampled MRI data (106) that is fed to the base reconstruction convolutional neural network (118).

3. The MRI image reconstruction system (100) as claimed in claim 2, wherein the under-sampled MRI data (106) is used to prepare a training data (126) that is input to the base reconstruction convolutional neural network (118), wherein the training data (126) and the context specific weights are used to process the under-sampled MRI data (106).

4. The MRI image reconstruction system (100) as claimed in claim 1, comprising a correction module (120) that utilizes the training data (108) and validation data to correct the processed under-sampled MRI data (106).

5. The MRI image reconstruction system (100) as claimed in claim 1, wherein the acquisition context information (114) includes an anatomy of the subject, an undersampling pattern, and an acceleration factor from settings of the MRI scanner (102).

6. The MRI image reconstruction system (100) as claimed in claim 1, wherein the dynamic weight prediction auxiliary network (112) receives scanner setting information from the acquisition context information (114), wherein the scanner setting information comprises a numerical array of numbers representing the anatomy, under- sampling pattern, and amount of under- sampling, which are used to predict the context- specific weights (116) based on the scanner setting information including dynamically adjusting the predicted context- specific weights (116) in response to changes in the scanner setting information.

7. The MRI image reconstruction system (100) as claimed in claim 6, wherein the context- specific weights (116) modulate convolutional features of a main image reconstruction Al system, enabling the reconstruction of images for under-sampled measurements with varying MRI scanner (102) settings without re-training.

8. The MRI image reconstruction system (100) as claimed in claim 1, wherein base reconstruction convolutional neural network (118) is pre-trained to reconstruct images from the under-sampled MRI data (106), wherein the dynamic weight prediction auxiliary network (112) provides additional context- specific adjustments to enhance reconstruction accuracy based on the specific MRI scanner (102) settings.

9. The MRI image reconstruction system (100) as claimed in claim 1, wherein the MRI scanner (102) varies the under- sampling pattern and amount of under- sampling according to predefined protocols, influencing the scanner setting information provided to the dynamic weight prediction auxiliary network (112).

10. A method for reconstructing MRI images, comprising: generating scanner measurements (104) using an MRI scanner (102); forming under-sampled MRI data (106) from the scanner measurements (104); receiving acquisition context information (114) from the MRI scanner (102) by a dynamic weight prediction auxiliary network (112); outputting context- specific weights (116) from the dynamic weight prediction auxiliary network (112); processing the under-sampled MRI data (106) using a base reconstruction convolutional neural network (118) and the context- specific weights (116) to generate a context- specific K-space (122); and combining the scanner measurements (104) and the context- specific K-space (122) to generate a reconstructed MRI image (124).

11. The method of as claimed in 10, comprising: applying a Fourier transformation to the scanner measurements (104) to form the under-sampled MRI data (106); and applying an inverse Fourier transformation to the under- sampled MRI data (106) before feeding it to the base reconstruction convolutional neural network (118).

12. The method as claimed in claim 11, comprising: preparing training data (126) from the under- sampled MRI data (106); and inputting the training data (126) to the base reconstruction convolutional neural network (118), wherein the training data (126) and the context- specific weights (116) are used to process the under-sampled MRI data (106).

13. The method as claimed in claim 10, comprising utilizing training data (126) and validation data in a correction module (120) to correct the processed under- sampled MRI data (106).

14. The method as claimed in claim 10, wherein the acquisition context information (114) includes an anatomy of the subject, an under-sampling pattern, and an acceleration factor from the settings of the MRI scanner (102).

15. The method as claimed in claim 10, wherein the dynamic weight prediction auxiliary network (112) receives scanner setting information from the acquisition context information (114), wherein the scanner setting information comprises a numerical array representing the anatomy, under- sampling pattern, and amount of under- sampling, and predicting the context- specific weights (116) based on the scanner setting information including dynamically adjusting the predicted context- specific weights (116) in response to changes in the scanner setting information.

16. The method as claimed in claim 15, wherein the context-specific weights (116) modulate convolutional features of a main image reconstruction Al system, enabling the reconstruction of images for under-sampled measurements with varying MRI scanner (102) settings without re-training.

17. The method as claimed in claim 10, wherein the base reconstruction convolutional neural network (118) is pre-trained to reconstruct images from the undersampled MRI data (106), and the dynamic weight prediction auxiliary network (112) provides additional context- specific adjustments to enhance reconstruction accuracy based on the specific MRI scanner (102) settings.

18. The method as claimed in claim 10, wherein the MRI scanner (102) varies the under- sampling pattern and amount of under- sampling according to predefined protocols, influencing the scanner setting information provided to the dynamic weight prediction auxiliary network (112).

Images