Systems and methods for improving low dose volume contrast enhanced MRI

Abstract

Methods and systems for improving robustness and generalizability of models are provided. The method can include: acquiring a medical image of a subject using a medical imaging device; reformatting the medical image of the subject in a plurality of scan orientations; applying a deep network model to the medical image to improve a quality of the medical image; and outputting the improved quality image of the subject for analysis by a physician.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 905,689, filed on September 25, 2019, the contents of which are incorporated herein by reference in their entirety.

[0003] Statement on Federally Funded Research

[0004] This invention was made with government support under grant number R44 EB027560 granted by the National Institutes of Health. The government owns certain rights to this invention. Background Technology

[0005] Contrast agents (e.g., gallium-based contrast agents (GBCA)) are used in approximately one-third of magnetic resonance imaging (MRI) examinations worldwide to create essential image contrast for a wide range of clinical applications. However, they pose health risks to patients with kidney failure and can deposit in the brain and body of patients with known normal kidney function. Recently, deep learning techniques have been used to reduce GBCA doses in volumetric contrast-enhanced MRI, but challenges remain regarding universality due to variability in scanning hardware and intra-site and cross-site clinical protocols. Summary of the Invention

[0006] This disclosure provides improved imaging systems and methods that address various drawbacks of conventional systems, including those recognized above. The methods and systems described herein improve image quality by reducing the dose level of contrast agents (e.g., gallium-based contrast agents (GBCA)). Specifically, a generalized deep learning (DL) model is used to predict contrast-enhanced images, where contrast agent reduction occurs across different sites and scanners.

[0007] Traditionally, contrast agents (e.g., gallium-based contrast agents (GBCAs)) have been widely used in contrast-enhanced medical imaging, such as magnetic resonance imaging (MRI) or nuclear magnetic resonance imaging, to examine pathology, predict the prognosis of gliomas, multiple sclerosis (MS), Alzheimer's disease (AD), and assess treatment response. GBCAs are also commonly used in other clinical applications, such as assessing coronary artery disease (CAD), characterizing lung masses, diagnosing hepatocellular carcinoma (HCC), and imaging spinal metastases. In 2006, an association was found between GBCA administration and the development of renal systemic fibrosis (NSF) in patients with impaired renal function. Other acute side effects of GBCAs in subjects with normal renal function included hypersensitivity reactions, nausea, and chest pain. Subsequently, in 2017, the US FDA issued warnings and safety measures related to gallium retention, and regulatory agencies in Canada, Australia, and other countries issued similar warnings. In addition to safety recommendations, the European Medicines Agency suspended the use of linear GBCAs. Gallium retention has been reported not only in high-intensity form in CNS tissues on non-contrast T1W MRI, but also in other parts of the body. As gallium is an emerging water pollutant, concerns about environmental sustainability are increasing. Other disadvantages of contrast-enhanced scanning include patient inconvenience during intravenous injection, prolonged scan time, and an overall increase in imaging costs. Although GBCA has a good pharmacovigilance safety profile, dose reduction remains a significant need due to the aforementioned safety concerns. Specifically, there is a desire to provide a safe imaging technique in which the contrast dose can be reduced regardless of the nature or type of contrast material, without compromising image quality or introducing additional safety issues.

[0008] Recent advancements in deep learning (DL) or machine learning (ML) techniques have made it a potential alternative to contrast-enhanced methods. DL / ML has already found numerous applications in medical imaging, including noise reduction, super-resolution, and modality conversion such as MRI to CT and T1 to T2. DL models have the potential to generate contrast-enhanced images using a fraction of the standard dose and the pre-contrast image. While such approaches may be able to reduce dose levels while maintaining non-inferior image quality, DL-enhanced images are often susceptible to artifacts, such as stripes on reformatted images (e.g., reformatted volumetric images or reconstructed 3D images viewed from different planes, orientations, or angles).

[0009] There is a need for a robust deep learning model that can be used in various clinical settings (and sometimes regardless of them), such as different scanner vendors, scanning protocols, patient demographics, and clinical indications. There is also a need for such a model to produce artifact-free images and support a variety of clinical use cases, such as multi-planar reformatting (MPR) for tilt visualization of 3D images, enabling the model to be deployed and integrated into standard clinical workflows.

[0010] The systems and methods described in this paper address the aforementioned shortcomings of conventional solutions. Specifically, the provided systems and methods may relate to deep learning models that include a unique set of algorithms and methods to improve the robustness and generality of the model. These algorithms and methods may include, for example, multi-plane reconstruction, 2.5D deep learning models, augmented weighted L1, perceptual loss and adversarial loss algorithms and methods, as well as preprocessing algorithms for preprocessing the input pre-contrast and low-dose images before the model predicts the corresponding contrast-enhanced images.

[0011] In one aspect, a computer-implemented method is provided for improving image quality by reducing the dosage of a contrast agent. The method includes: acquiring medical images of a subject with a reduced dosage of a contrast agent using a medical imaging device; reformatting the medical images of the subject in multiple orientations to generate a plurality of reformatted medical images; and applying a deep network model to the plurality of reformatted medical images to generate a predicted medical image with improved quality.

[0012] In a related but separate aspect, a non-transitory computer-readable storage medium includes instructions that, when executed by one or more processors, cause the one or more processors to perform operations. The operations include: acquiring medical images of a subject with a reduced dose of contrast agent using a medical imaging device; reformatting the medical images of the subject in multiple orientations to generate a plurality of reformatted medical images; and applying a deep network model to the plurality of reformatted medical images to generate predictive medical images with improved quality.

[0013] In some embodiments, the medical imaging device is a transformation magnetic resonance (MR) device. In some embodiments, the medical image is a 2.5D volumetric image.

[0014] In some embodiments, the plurality of orientations includes at least one orientation not in the direction of the scanning plane. In some embodiments, the method or operation further includes rotating each of the plurality of reformulated medical images to various angles to produce a plurality of rotated reformulated medical images. In some cases, the deep network model is applied to the plurality of rotated reformulated medical images to output a plurality of predicted images. The plurality of predicted images, as the output of the deep network model, are rotated to align with the scanning plane. In some cases, the method or operation further includes averaging the plurality of predicted images after rotation to align with the scanning plane to produce the predicted medical image with improved quality. In some embodiments, the predicted medical image with improved quality is obtained by averaging the plurality of predicted medical images corresponding to the plurality of reformulated medical images.

[0015] Furthermore, the methods and systems disclosed herein can be applied to existing systems without altering the underlying infrastructure. Specifically, the provided methods and systems can reduce contrast agent dosage levels without additional hardware component costs and can be deployed regardless of the configuration or specifications of the underlying infrastructure.

[0016] Other aspects and advantages of this disclosure will become readily apparent to those skilled in the art from the following detailed description, in which only illustrative embodiments of this disclosure are shown and described. It will be appreciated that this disclosure can have other and different embodiments, and that certain details thereof can be modified in various readily understood ways, all without departing from this disclosure. Therefore, the drawings and detailed descriptions should be considered illustrative in nature and not restrictive.

[0017] Incorporation

[0018] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference to the extent that each individual publication, patent, or patent application is specifically and individually cited and incorporated herein by reference. Where a publication or patent or patent application incorporated by reference conflicts with the disclosure contained in this specification, this specification is intended to supersede and / or take precedence over any such contradictory material. Attached Figure Description

[0019] The novel features of the invention are specifically set forth in the appended claims. A better understanding of the features and advantages of the invention will be obtained by referring to the following detailed description of illustrative embodiments in which the principles of the invention are utilized, along with the accompanying drawings (also referred to herein as “Figures”):

[0020] Figure 1An example of a workflow for processing and reconstructing magnetic resonance imaging (MRI) volumetric image data is shown.

[0021] Figure 2 An example of data collected from two different sites is shown.

[0022] Figure 3 The analysis results of the study are shown.

[0023] Figure 4 A magnetic resonance imaging (MRI) system in which the imaging intensifier of this disclosure can be implemented is schematically illustrated.

[0024] Figure 5 An example of a scanning process or scanning protocol used to collect experimental data in a study is shown.

[0025] Figure 6 An example of a reformulated MPR reconstructed image with higher quality than a reformulated MRI image produced using conventional methods is shown.

[0026] Figure 7 Examples of preprocessing methods according to some embodiments of this document are shown.

[0027] Figure 8 Examples of U-Net-style encoder-decoder network architectures according to some implementations of this paper are shown.

[0028] Figure 9 Examples of discriminators according to some embodiments of this document are shown.

[0029] Figure 10 An experiment is shown that includes the data distribution and heterogeneity of research datasets from three institutions, three different manufacturers, and eight different scanner models.

[0030] Figure 11 The system and method for monotonically improving image quality are illustrated schematically.

[0031] Figure 12 Examples of quantitative indicators from cases from different sites and scanners are shown, including pre-contrast, low-dose, and full-dose ground truth image data and synthetic images.

[0032] Figure 13 An example illustrating the effect of the number of rotation angles in an MPR on the quality of the output image and processing time is shown. Detailed Implementation

[0033] Although various embodiments of the invention have been shown and described herein, it will be readily understood by those skilled in the art that these embodiments are provided by way of example only. Many variations, modifications, and substitutions will occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.

[0034] Gallium-based contrast agents (GBCAs) are widely used in magnetic resonance imaging (MRI) and are essential for monitoring treatment and studying pathology in a variety of applications, including angiography, multiple sclerosis, and tumor detection. Recently, the prolonged identification of gallium deposition in the brain and body has raised concerns about the safety of GBCA use. Reducing the GBCA dose can decrease the degree of deposition, but it also reduces contrast enhancement and tumor visibility. Therefore, reduced-dose examinations that preserve contrast enhancement are highly relevant for patients requiring repeated contrast agent administration (e.g., patients with multiple sclerosis) and those at high risk of gallium deposition (e.g., children).

[0035] While this article primarily provides examples based on MRI, gallium-based contrast agents, and MRI data, it should be understood that this method can be used in other imaging modalities and / or other contrast-enhanced imaging environments. For example, the method described herein can be used with data acquired by other types of computed tomography scanners, including but not limited to computed tomography (CT), single-photon emission computed tomography (SPECT), positron emission tomography (PET), functional magnetic resonance imaging (fMRI), or various other types of imaging scanners or techniques in which contrast agents can be used to enhance contrast.

[0036] Deep learning (DL) frameworks have been used to reduce GBCA dose levels while maintaining image quality and contrast enhancement in volumetric MRI. As an example, DL models can use a U-net encoder-decoder architecture to enhance image contrast from low-dose contrast images. However, conventional DL models may only be applicable to scans from a single clinical site, without considering the versatility across different sites with varying clinical workflows. Furthermore, conventional DL models may assess the image quality of a single 2D slice within a 3D volume, even though clinicians often require volumetric images to visualize complex 3D enhanced structures (e.g., blood vessels and tumors) from various angles or orientations.

[0037] This disclosure provides systems and methods that address various shortcomings of conventional systems, including those recognized above. The methods and systems of this disclosure improve the robustness and deployment of models in real-world clinical settings. For example, the provided methods and systems can be adapted to different clinical sites, each with different MRI scanner hardware and imaging protocols. Furthermore, the provided methods and systems can provide improved performance while retaining multiplanar reformatting (MPR) capabilities to maintain clinician workflows and enable oblique visualization of complex, enhanced microstructures.

[0038] The methods and systems described in this paper provide enhancements to deep learning (DL) models to address real-world variability in clinical settings. The DL models are trained and tested on patient scans from different hospitals, across different MRI platforms, with varying scan planes, scan times, and resolutions, and different GBCA dosing mechanisms. In these settings, the robustness of the DL models can be improved by enhancing generalizability across data heterogeneity.

[0039] Multiplane reformatting (MPR)

[0040] In a standard deep learning (DL) pipeline, standard 2D data augmentations (e.g., rotation and flipping) can be used to process and train 2D slices from 3D volumes separately. The choice of 2D model is often motivated by memory constraints during training and performance requirements during inference. In some cases, DL frameworks may process data in a “2.5D” manner, where multiple adjacent slices are fed into the network and the central slice is predicted. However, both 2D and 2.5D processing can ignore the true volumetric properties of the acquisition. Since 3D volumes are often reformatted to arbitrary planes in clinical workflows (e.g., oblique views, views from orientations / angles tilted relative to the scan plane / orientation), and sites may use different scan orientations as part of their MRI protocols, 2D processing can result in images with stripe artifacts in the reformatted volumetric images (e.g., reformatted to a plane orthogonal to the scan plane).

[0041] The methods and systems described herein can beneficially eliminate artifacts (e.g., streak artifacts) in reformatted images, thereby enhancing image quality by reducing contrast dose. As mentioned above, it is common in standard clinical workflows to reformat 3D volumetric images to view them in multiple planes (e.g., orthogonal or oblique planes). In some cases, although training a model to enhance 2.5D images can reduce streak artifacts in the acquisition plane, reformatting to other orientations may still result in streak artifacts. The methods and systems described herein can enable artifact-free visualization in any selected plane or viewing orientation (e.g., oblique view). Furthermore, models can be trained to learn complex or sophisticated 3D enhancement structures, such as blood vessels or tumors.

[0042] Figure 1 An example workflow for processing and reconstructing MRI volumetric image data is illustrated. As shown in the example, input image 110 can be an image slice acquired without contrast agent (e.g., pre-contrast image slice 101) and / or at a reduced contrast dose (e.g., low-dose image slice 103). In some cases, the original input image can be a 2D image slice. For example, a deep learning (DL) model of the U-net encoder-decoder 111 model can be used to predict the inference result 112. Although the DL model 111 can be a 2D model trained to produce an enhanced image within each slice, it may produce inconsistent image enhancements across slices, such as stripe artifacts in image reformatting. For example, when the inference result is reformatted 113 to produce a reformatted image in the orthogonal direction 114, the reformatted image 114 may contain reformatting artifacts, such as stripe artifacts in the orthogonal direction, because the input 2D image 110 matches the scan plane.

[0043] This reformatting artifact can be mitigated by employing a multiplanar reformatting (MPR) method 120 and training the model in 2.5D 131. The MPR method can beneficially increase the volumetric size of the input data across multiple orientations. For example... Figure 1As shown, a selected number of input slices from the pre-contrast or low-dose image 110 can be stacked in the channel direction to create a 2.5D volumetric input image. The number of input slices used to form the 2.5D volumetric input image can be any number, for example, at least two, three, four, five, six, seven, eight, nine, or ten slices can be stacked. In some cases, the number of input slices can be determined based on physiologically or biochemically important structures in the region of interest, such as microstructures where an artifact-free volumetric image is highly desirable. For example, the number of input slices can be selected such that microstructures (e.g., blood vessels or tumors) can be predominantly contained in the input 2.5D volumetric image. Alternatively or additionally, the number of slices can be determined based on empirical data or selected by the user. In some cases, the number of slices can be optimized based on the computing power and / or memory storage of the computing system.

[0044] Next, the input 2.5D volumetric image can be reformatted along multiple axes, such as principal axes (e.g., sagittal, coronal, and axial), to produce multiple reformatted volumetric images 121. The multiple orientations used to reformat the 2.5D volumetric images can be in any suitable direction that does not require alignment with the principal axes. Furthermore, the number of orientations used to reformat the volumetric images can be any number greater than 1, 2, 3, 4, 5, etc., as long as at least one of the multiple reformatted volumetric images is along an orientation that is either inclined or perpendicular to the scan plane.

[0045] During the inference phase, each of the multiple reformulated volumetric images can be rotated by a series of angles to produce multiple rotated reformulated volumetric images 122, thereby further enhancing the input data. For example, each of three reformulated volumetric images 121 (e.g., sagittal, coronal, and axial) can be rotated by five equidistant angles between 0 and 90°, resulting in 15 volumetric images 122. It should be noted that the angle step size and angle range can be within any suitable range. For example, the angle step size may not be constant, and the number of rotation angles may vary depending on different applications, situations, or deployment schemes. In another example, the volumetric images can be rotated within any angle range greater than, less than, or partially overlapping 0-90°. The impact of the number of rotation angles on the predicted MPR images will be described later.

[0046] Multiple rotated 2.5D volumetric images 122 can then be fed into a 2.5D training model 131 for inference. The output of the 2.5D training model includes multiple contrast-enhanced 2.5D volumetric images. In some cases, the final inference result 132, referred to as "MPR reconstruction," can be the average of multiple contrast-enhanced 2.5D volumetric images rotated back to the original acquisition / scan plane. For example, 15 enhanced 2.5D volumetric images can be rotated back to align with the scan plane, and the average of these volumetric images is the MPR reconstruction or final inference result 132. Multiple predicted 2.5D volumetric images can be rotated to align with the original scan plane or the same orientation, allowing the average of multiple 2.5D volumetric images to be calculated. Multiple enhanced 2.5D stereo images can be rotated to align with the same orientation, which may or may not be in the original scan plane. The MPR reconstruction method advantageously allows the addition of 3D context to the network while benefiting from the performance improvements of 2D processing.

[0047] like Figure 1 As shown, when the MPR reconstructed image 132 is reformatted 133 to a plane orthogonal to the original acquisition plane, the reformatted image 135 does not exhibit stripe artifacts. The quality of the predicted MPR reconstructed image can be quantified using quantitative image quality metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). Image quality metrics are calculated for both the conventional model 111 and the proposed model 131, and... Figure 3 The illustration shows examples of the results demonstrating the quality of reformatted images 114, 135, and ground truth 140.

[0048] Data collection

[0049] In the example, with IRB approval and patient consent, the scanning protocol was implemented at both sites. Figure 2 Examples of data collected from these two sites are shown. 24 patients were recruited from site 1 (16 for training, 8 for testing), and 28 patients were recruited from site 2 (23 for training, 5 for testing). Differences between scanner hardware and protocols are highlighted in Table 1. In particular, different scanner hardware was used at the two sites, and the scanning protocols were highly variable. Notably, site 1 used powered injection to administer GBCA, while site 2 used manual injection, resulting in differences in enhancement time and intensity.

[0050] As an example of collecting data for training the model, multiple scans with reduced dose levels and full-dose scans can be performed. Multiple scans with reduced dose levels can include, for example, low-dose (e.g., 10%) contrast-enhanced MRI and can be performed pre-contrast (e.g., zero contrast). For example, for each participant, two 3D T1-weighted images were obtained: pre-contrast and post-contrast at 10% dose (0.01 mmol / kg). For training and clinical validation, the remaining 90% of the standard contrast dose (full-dose equivalent, 100% dose) was administered, and a third 3D T1-weighted image (100% dose) was obtained. Signal normalization was performed to eliminate systemic differences (e.g., transmit and receive gain) that could cause variations in signal intensity between different acquisitions across different scanner platforms and hospital sites. Nonlinear affine co-matching was then performed between the pre-dose, 10% dose, and 100% dose images. The DL model uses a U-Net encoder-decoder architecture, the basic assumption of which is that the contrast-related signal between the pre-contrast image and the low-dose contrast-enhanced image is nonlinearly scaled to the full-dose contrast image. Additionally, images from other contrast sources (such as T2 and T2-FLAIR) can be included as part of the input to improve model predictions.

[0051] Figure 5 It shows the use for Figure 2 , Figure 3 and Figures 10 to 12 The illustrated example shows a scanning procedure or protocol 500 for collecting data in a study or experiment. In the illustrated scanning protocol, each patient underwent three scans during a single imaging session. Scan 1 was a pre-contrast 3D T1-weighted MRI, followed by Scan 2 with 10% of the standard dose of 0.1 mmol / kg. Images from Scan 1 and Scan 2 were used as input to the DL network. After administration of the remaining 90% of the contrast dose (i.e., the full dose), a true image was obtained from Scan 3.

[0052] During inference, after deploying the provided system, only one scan without contrast agent can be performed (e.g., similar to scan 1), or a scan with reduced contrast agent dose can be performed (e.g., similar to scan 2). This input image data can then be processed by a trained model to output a predicted MPR reconstructed image with enhanced contrast. In some cases, after deploying the model to a clinical site, the user (e.g., a physician) can select the reduced dose level, which can be any level ranging from 0% to 30% to obtain medical image data. It should be noted that, depending on the actual implementation and the user's desired dose reduction level, the reduced dose level can be any amount within the range greater than 30%.

[0053] Inter-site compatibility

[0054] A standard model can be limited by using the same scanning protocol to evaluate patients from a single site. In a real clinical setting, each site can customize its protocol based on the capabilities of the scanner hardware and standard procedures. For example, a model trained on site 2 might perform poorly in the case of site 1. Figure 2 ,middle).

[0055] The provided DL model can have improved generality. The DL model can be trained using a proprietary training pipeline. For example, the training pipeline may include first scaling each image to 1mm. 3 The nominal resolution and in-plane matrix size of 256×256 are then used, followed by MPR processing. Since the DL model is fully convolutional, inference can be run at the original acquired resolution without resampling.

[0056] Based on qualitative and quantitative results, the addition of MPR processing, resolution resampling, and inter-site training significantly improves the model's robustness and versatility. In an alternative implementation, the model can be a fully 3D model. For example, the model can be a 3D patch-based model, which can alleviate MPR processing and memory usage. The provided training methods and model framework can be applied to different sites with different scanner platforms and / or across different MRI providers.

[0057] Network architecture and processes

[0058] Figure 6 Another example of an MPR reconstructed image 624 with improved quality compared to an MRI image predicted using conventional method 611 is illustrated schematically. The workflow 600 for processing and reconstructing MRI volumetric image data 623 and the reformulated MPR reconstructed image 624 can be compared with... Figure 1 The same applies. For example, input image 610 may include multiple 2D image slices acquired without contrast agent (e.g., pre-contrast image slices) and / or with reduced contrast dose (e.g., low-dose image slices). The input image may be acquired in the scan plane (e.g., axially) or along the scan orientation. A selected number of image slices are stacked to form a 2.5D volumetric input image, which is further processed using a multi-plane reconstruction (MPR) method 620, as described above.

[0059] For example, an input 2.5D volumetric image can be reformatted to multiple axes (e.g., principal axes (e.g., sagittal, coronal, and axial)) to produce multiple reformatted volumetric images (e.g., SAG, AX, COR). It should be noted that a 2.5D stereo image can be reformatted to any orientation that may or may not be aligned with the principal axes.

[0060] Each of multiple reformulated volumetric images can be rotated a series of angles to generate multiple rotated reformulated images. For example, each of three reformulated volumetric images (e.g., sagittal, coronal, and axial) can be rotated five angles between 0 and 90°, resulting in 15 rotated reformulated volumetric images. Multiple reformulated volumetric images (e.g., sagittal, coronal, and axial) can be rotated at the same angle or not at the same angle, or rotated to the same number of orientations, or not rotated to the same number of orientations.

[0061] The trained model 621 can then process multiple rotated volumetric images 122 to generate multiple enhanced volumetric images. In some cases, the MPR reconstructed image 623 or the inferred image is the average of multiple inferred volumes after being rotated back to the original acquisition plane. When the MPR reconstructed image is reformatted for viewing in a selected orientation (e.g., perpendicular to / tilted to the scan plane), the reformatted image 624 may not contain stripe artifacts compared to a reformatted image obtained using a single inference method 611 and / or a single inference model.

[0062] Network architecture and data processing

[0063] Using multiplanar reconstruction (MPR) techniques, a deep learning model can be trained using volumetric images (e.g., enhanced 2.5D images) from multiple orientations (e.g., three principal axes). This model can be a trained deep learning model used to enhance the quality of volumetric MRI images acquired using reduced contrast doses. In some implementations, the model can include an artificial neural network that can employ any type of neural network model, such as a feedforward neural network, radial basis function network, recurrent neural network, convolutional neural network, deep residual learning network, etc. In some implementations, the machine learning algorithm can include a deep learning algorithm, such as a convolutional neural network (CNN). Examples of machine learning algorithms can include support vector machines (SVM), Naive Bayes classification, random forests, deep learning models (e.g., neural networks), or other supervised or unsupervised learning algorithms. The model network can be a deep learning network that can include multiple layers, such as a CNN. For example, a CNN model can include at least an input layer, multiple hidden layers, and an output layer. A CNN model can include any total number of layers and any number of hidden layers. The simplest architecture of a neural network begins with an input layer, followed by a series of intermediate or hidden layers, and finally an output layer. Hidden or intermediate layers can act as learnable feature extractors, while in this example, the output layer provides a 2.5D stereo image with enhanced quality (e.g., enhanced contrast). Each layer of a neural network can include multiple neurons (or nodes). Neurons receive input directly from input data (e.g., low-quality image data, image data acquired with reduced contrast dose, etc.) or the output of other neurons and perform specific operations, such as summation. In some cases, the connections from the input to the neuron are associated with weights (or weighting factors). In some cases, the neuron can summarize the product of all input pairs and their associated weights. In some cases, the weighted sum is biased. In some cases, a threshold or activation function can be used to control the neuron's output. Activation functions can be linear or non-linear. The activation function can be, for example, the rectified linear unit (ReLU) activation function or other functions, such as saturated hyperbolic tangent, identity, binary step, logic, arcTan, softsign, parametric rectified linear unit, exponential linear unit, softPlus, bending identity, softExponential, Sinusoid, Sinc, Gaussian, Sigmoid function or any combination thereof.

[0064] In some implementations, the network can be an encoder-decoder network or a U-net encoder-decoder network. U-net is an autoencoder where the output of one half of the encoder from the network is concatenated with a mirror copy in one half of the decoder. U-net can improve the output resolution by upsampling the operators instead of using pooling operations.

[0065] In some implementations, supervised learning can be used to train a model for enhancing volumetric image quality. For example, to train a deep learning network, paired pre-contrast and low-dose images from multiple subjects, scanners, clinical sites, or databases can be provided as input, with full-dose images as the ground truth.

[0066] In some cases, the input dataset can be preprocessed before training or inference. Figure 7 An example of a preprocessing method 700 according to some embodiments of this document is shown. As shown in the example, input data including raw pre-contrast, low-dose, and full-dose images (i.e., real) can be preprocessed sequentially to produce preprocessed image data 710. Raw image data can be received from standard clinical workflows, such as DICOM-based software applications or other imaging software applications. As an example, methods such as... Figure 5 The scanning protocol described herein is used to acquire input data 701. For example, three scans may be performed, including a first scan with zero contrast agent, a second scan with a reduced dose level, and a third scan with a full dose. However, the reduced-dose image data used to train the model may include images acquired at various reduced dose levels, such as any number not exceeding 1%, 5%, 10%, 15%, 20%, greater than 20%, or less than 1%, or any number in between. For example, input data may include image data acquired from two scans, which include a full-dose scan as real data and paired scans at reduced levels (e.g., zero dose or any level as described above). Alternatively, more than three scans may be used to acquire input data, with multiple scans performed at different contrast doses. Additionally, input data may include augmented datasets obtained from simulations. For example, image data from a clinical database may be used to generate low-quality image data that mimics image data acquired at reduced contrast doses. In the example, artifacts may be added to the original image data to mimic image data reconstructed from images acquired at reduced contrast doses.

[0067] In the example shown, preprocessing algorithms (e.g., skull stripping 703) can be performed to separate brain images from skull or non-brain tissues by using a DL-based library to remove signals from extracranial and non-brain tissues. Other suitable preprocessing algorithms can be employed to improve processing speed and diagnostic accuracy, depending on the tissue, organ, and application. In some cases, to account for patient movement between three scans, low-dose and full-dose images can be co-registered to the pre-contrast image 705. In some cases, assuming that transmit and receive gains may differ for different acquisitions, signal normalization can be performed via histogram equalization 707. Relative intensity scaling can be performed between pre-contrast, low-dose, and full-dose images for intra-scan image normalization. Since multi-institution datasets contain images with different voxel and matrix sizes, 3D volume interpolation can be performed to 0.5 mm. 3 The image data has an isotropic resolution, and where applicable, the zero-padding image at each slice is interpolated to a size of 512×512. The image data may have a sufficiently high resolution to enable the DL network to learn small enhancing structures, such as lesions and metastases. In some cases, scaling and registration parameters can be estimated on the skull-stripped image and then applied to the original image 709. Preprocessing parameters estimated from the skull-stripped brain can be applied to the original image to obtain a preprocessed image volume 710.

[0068] Next, the preprocessed image data 710 is used to train the encoder-decoder network to reconstruct the contrast-enhanced image. It can be assumed that the contrast signal at full dose is a non-linearly scaled version of the noise contrast absorption between the low-dose and pre-contrast images, thus training the network. The model may not explicitly require the difference image between the low-dose and pre-contrast images.

[0069] Figure 8 An example of a U-Net-style encoder-decoder network architecture 800 according to some embodiments of this paper is shown. In the example shown, each encoder block has three 2D convolutional layers (3×3) with ReLU followed by maxpool (2×2) to downsample the feature space by a factor of two. The decoder module has a similar structure, where maxpool is replaced by an upsampling layer. To recover the spatial information lost during downsampling and to prevent resolution loss, skip connections are used to link the functions of the decoder layers with those of the corresponding encoder layers. The network can be trained using a combination of L1 (mean absolute error) and structural similarity index (SSIM) loss. This U-Net-style encoder / decoder network architecture may be able to generate a linearly proportional contrast absorption between low and zero doses without picking up noise along with the augmented signal.

[0070] like Figure 8As shown, the input data to the network can be multiple enhanced volumetric images generated using the MPR method described above. In this example, seven slices from each pre-contrast and low-dose image are stacked in the channel direction to create 14-channel input volumetric data for training the model to predict the central full-dose slice 803.

[0071] Enhanced and weighted L1 loss

[0072] In some cases, even after signal normalization and scaling, the difference between the low-dose image and the pre-contrast image may have similarly enhanced noise perturbations, which can mislead network training. To make the network focus more on the actual enhanced regions, an enhancement mask can be used to weight the L1 loss. The mask is inherently continuous and is calculated based on the difference in skull dissection between the low-dose image and the pre-contrast image (normalized between 0 and 1). The enhancement mask can be viewed as a normalized, smoothed form of contrast absorption.

[0073] Perceived loss and adversarial loss

[0074] The goal is to train the network to focus on enhancing structural information, including high-frequency and texture details, which is crucial for making reliable diagnostic decisions. A simple combination of L1 and Structural Similarity Index (SSIM) losses may tend to suppress high-frequency signal information, resulting in a smoother appearance, which is considered a loss of image resolution. To address this, a perceptual loss derived from convolutional networks (e.g., the 19-layer VGG-19 network, consisting of 6 convolutional layers, 3 fully connected layers, 5 MaxPool layers, and 1 SoftMax layer pre-trained on the ImageNet dataset) is employed. Perceptual loss is effective in style transfer and super-resolution tasks. For example, the perceptual loss can be computed based on the third convolutional layer of the third block (e.g., block3 conv3) of the VGG-19 network by obtaining the mean squared error (MSE) of layer activations on both the ground truth and prediction.

[0075] In some cases, to further improve overall perceptual quality, adversarial loss is introduced by a discriminator trained in parallel with the encoder-decoder network to predict whether the generated image is real or fake. Figure 9An example of a discriminator 900 according to some embodiments of this paper is shown. The discriminator 900 has a series of spectrally normalized convolutional layers with leaked ReLU activations and predicts 32×32 color patches. Unlike conventional discriminators that predict binary values ​​(e.g., 0 for fakes, 1 for real), the "color patch discriminator" 900 predicts a probability matrix, which contributes to the stability of the training process and faster convergence. The spectrally normalized convolutional layers employ weight normalization techniques to further stabilize discriminator training. Figure 9 As shown, the color block discriminator can be trained using MSE loss, and Gaussian noise can be added to the input to achieve smooth convergence.

[0076] The function used to configure the network model can be described as follows:

[0077] G*=argmin G [λ GAN L GAN (G)+λ L1 L L1 (M enh .G)+λ SSIM L SSIM (G)+λ VGG L VGG (G)]

[0078] Among them, M enh It is an enhanced mask, and it is also effective against loss L. GAN It can be written as L GAN =max D L GAN (G, D), where G is the U-Net generator and D is the color block discriminator. Loss weight λ L , λ SSIM , λ VGG and λ GAN This can be determined empirically. Using the above process and methods, a single model was trained to make accurate predictions for images from various institutions and scanners.

[0079] Example

[0080] Figure 3 An example of research analysis results evaluating the generality and accuracy of the provided model is shown. In the example shown, the results compare the inference results of the test cases from site 1 (left), the original model (middle), and the proposed model (right) (red arrows indicate obvious deficiencies). A regular model was trained only on data from site 2. This example is compared with... Figure 2The MRI scan data shown are consistent. The provided model was trained on data from both sites, using MPR processing and resolution resampling. In this study, the results qualitatively demonstrate the effect of MPR processing on an example of the test set. By averaging the results of many MPR reconstructions, streaking artifacts that appear as pseudo-enhancement can be suppressed. Figure 3 As shown, an image with enhanced true contrast (left) is compared with inference results from a model trained on site 2 (middle) and a model trained simultaneously on sites 1 and 2 (right). By taking into account resolution differences and other protocol biases, the proposed model demonstrates quantitative improvements in generality. Quantitative image quality metrics, such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), were calculated for all conventional and proposed models. The mean PSNR and SSIM on the test set for the conventional and proposed models were 32.81 dB (38.12 dB) and 0.872 (0.951), respectively. Better image quality can be achieved using the methods and systems of this disclosure.

[0081] In such Figure 3 In the study shown, a deep learning (DL) framework, as described elsewhere in this paper, was applied to low-dose (e.g., 10%) contrast-enhanced MRI. For each participant, two 3D T1-weighted images were obtained: pre-contrast and post-contrast at 10% dose (0.01 mmol / kg). For training and clinical validation, the remaining 90% of the standard contrast dose (full dose equivalent, 100% dose) was administered, and a third 3D T1-weighted image (100% dose) was obtained. Signal normalization was performed to eliminate systemic differences (e.g., transmit and receive gain) that could cause variations in signal intensity between different acquisitions across different scanner platforms and hospital sites. Nonlinear affine co-registration was then performed between the pre-dose, 10% dose, and 100% dose images. The DL model used a U-Net encoder-decoder architecture, the basic assumption of which is to nonlinearly scale the contrast-related signal between the pre-contrast image and the low-dose contrast-enhanced image to the full-dose contrast image. Images from other contrasts (e.g., T2 and T2-FLAIR) could be included as part of the input to improve model predictions.

[0082] As a combination Figures 10 to 13 Another example of the experiment, in Figure 10The data distribution and heterogeneity of the study dataset from three institutions, three different manufacturers, and eight different scanner models are shown in the figure. This retrospective study identified 640 patients (323 women; 52 ± 16 years) who underwent clinical brain MRI examinations from three institutions, three scanner manufacturers, and eight scanner models, using different institutional scanning protocols, which included different imaging planes, field strengths, voxel sizes, matrix sizes, use of fat suppression, contrast agents, and injection methods. Clinical indications included suspected tumors, postoperative tumor follow-up, routine brain scans, and other indications requiring MRI examination with GBCA. Each subject underwent 3D precontrast T1w imaging followed by a low-dose contrast-enhanced T1w scan at 10% (0.01 mmol / kg) of the standard dose (0.1 mmol / kg). For training and evaluation, a third 3D T1w image was obtained using the remaining 90% (0.09 mmol / kg) of the total dose, which was considered true. All three acquisitions were completed in a single imaging session, and patients did not receive any additional gallium doses compared to the standard protocol.

[0083] Of the 640 cases, 56 were used for training. Figure 11 The model shown was fine-tuned with hyperparameters using 13 validation cases, and the optimal combination of loss weights was found empirically. To ensure the model generalizes well across sites and vendors, the training and validation sets consisted of approximately equal numbers of studies from all institutions and scanner manufacturers (see [link to relevant documentation]). Figure 10 The remaining 571 cases were reserved for testing and model evaluation. The model was implemented in Python 3.5 using Keras with a Tensorflow backend and trained for 100 epochs on an Nvidia Tesla V100 (SXM2 32GB) GPU with a batch size of 8. Model optimization was performed using the Adam optimizer with a learning rate of 0.001.

[0084] The model was quantitatively evaluated using multiple metrics. Peak signal-to-noise ratio (PSNR) is a scaled form of pixel differences, while the structural similarity index (SSIM) is sensitive to changes in local structure, thus capturing the structural aspects of the predicted image relative to the true image. The model was quantitatively evaluated using PSNR and SSIM metrics across 571 test cases, calculated between real full-dose and synthetic images. These values ​​were compared to PSNR and SSIM values ​​between low-dose and full-dose images. Metrics were also calculated and compared for each site and each scanner to demonstrate the model's generalizability.

[0085] From the test set, a subset of images from 26 patients (13 males; 58 ± 15 years old) with different types and grades of tumor enhancement (pre- or post-operative) was identified and used for in-depth evaluation of model performance. In terms of heterogeneity, these enhanced tumor cases were similar to the training dataset and used methods such as... Figure 5 The same scanning protocol was used for acquisition. Binary evaluation was performed to determine if the enhancement patterns were consistent without any false positives or false negatives (referencing a true full-dose image). When present, image artifacts in the synthetic image were recorded, and the reduction of image artifacts was demonstrated using the provided model.

[0086] To further validate the similarity between the model predictions and the actual full-dose images, automated tumor segmentation was performed on 26 enhanced tumor cases. A variant of the model was applied, using only the contrast-enhanced images to segment the tumor core. Following the segmentation model's requirements, skull dissection was performed on both the actual and predicted full-dose images, interpolated to 1 mm. 3 The resolution is determined and co-registered to the anatomical template. Evaluation is performed by calculating the Dice score of the predicted tumor core between the true segmented mask and the mask created using the synthetic image.

[0087] Figure 11 Systems and methods for monotonic improvement of image quality are schematically illustrated. An example of a sagittal-acquired MR image with enhanced frontal lobe tumor is shown. As shown in panel a, vertical stripes can be seen in the axial format of the 2.5D model results, fixed by MPR training and inference, as shown in panel b. As shown in panel c, increasing the perceptual and adversarial losses further improves the texture within the tumor and restores overall perceptual quality. Furthermore, as shown in panel d, weighting the L1 loss using a smooth enhancement mask allows the enhancement pattern to match reality. Monotonic increases in metrics relative to reality (as shown in panel e) also illustrate the model improvement. The model improvement for each proposed technical solution for 26 tumor enhancement cases is shown below.

[0088] PSNR (dB) 31.84±4.88 32.38±4.67 33.56±5.19 34.28±4.88 35.22±4.79 SSIM 0.88±0.06 0.89±0.06 0.90±0.06 0.92±0.05 0.93±0.04

[0089] Figure 12 Real and synthetic images before comparison, low-dose, and full-dose, along with quantitative metrics from cases from different sites and scanners, are presented. The metrics show that the model using the proposed technical improvements outperforms the original model (metrics 31.84 ± 4.88 dB, 0.88 ± 0.06). The best-performing model combines SSIM, perceptual, adversarial, and enhancement-weighted L1 loss with five rotations using MPR. For a volume of 512 × 512 × 300, preprocessing and inference of the best model on a GeForce RTX 2080 (16GB) GPU takes approximately 135 seconds.

[0090] Figure 13 Examples of different rotation speeds and their corresponding effects on image quality and performance are shown. For example... Figure 13 As shown, the number of rotations in MPR has an impact on image quality. Larger angles can reduce horizontal stripes within the tumor (better quality) while also increasing inference time. When deploying the trained model to a physical site, the number of rotations and different angles can be determined based on the desired image quality and deployment environment (e.g., computing power, memory storage, etc.).

[0091] System Overview

[0092] The provided DL framework for low-dose contrast-enhanced MRI reduces the GBCA dose used in contrast-enhanced MRI while preserving image quality and avoiding degradation of contrast enhancement. The robustness and versatility of the DL model have been improved, allowing for better adaptation to a variety of applications across heterogeneous patients and field populations. Figure 4 A magnetic resonance imaging (MRI) system 400 in which the imaging intensifier 440 of this disclosure may be implemented is schematically illustrated. The MRI system 400 may include a magnet system 403, a patient transport table 405 connected to the magnet system, and a controller 401 operatively coupled to the magnet system. In one example, a patient may lie on the patient transport table 405, with the magnet system 403 bypassing the patient. The controller 401 may control the magnetic field and radio frequency (RF) signals provided by the magnet system 403 and may receive signals from detectors in the magnet system 403.

[0093] The MRI system 400 may further include a computer system 410 and one or more databases operatively coupled to the controller 401 via a network 430. The computer system 410 may be used to implement a volumetric MR imaging intensifier 440. The volumetric MR imaging intensifier 440 may implement the DL framework and methods described herein. For example, the volumetric MR imaging intensifier may employ the MPR reconstruction method described herein and various other training algorithms and data processing methods. The computer system 410 may be used to generate the imaging intensifier using a training dataset. Although the figures shown depict the controller and computer system as separate components, the controller and computer system can be integrated into a single component.

[0094] Computer system 410 may include laptop computers, desktop computers, central servers, distributed computing systems, etc. The processor may be a hardware processor, such as a central processing unit (CPU), graphics processing unit (GPU), general-purpose processing unit (which may be a single-core or multi-core processor), or multiple processors for parallel processing. The processor may be any suitable integrated circuit, such as a computing platform or microprocessor, logic device, etc. Although the reference processor describes this disclosure, other types of integrated circuits and logic devices are also applicable. The processor or machine may not be limited by data manipulation capabilities. The processor or machine can perform 512-bit, 256-bit, 128-bit, 64-bit, 32-bit, or 16-bit data operations.

[0095] The MRI system 400 may include one or more databases 420, which may utilize any suitable database technology. For example, a Structured Query Language (SQL) or “NoSQL” database may be used to store reconstructed / reformatted image data, raw collection data, reconstructed image data, training datasets, trained models (e.g., hyperparameters), weight coefficients, rotation angles, rotation numbers, reformatted orientations, etc. Some databases may be implemented using various standard data structures, such as arrays, hashes, (linked) lists, structures, structured text files (e.g., XML), tables, JSON, NoSQL, etc. Such data structures may be stored in memory and / or (structured) files. In another alternative, a subject-oriented database may be used. The subject database may contain many subject sets grouped and / or linked together by common attributes; they may be related to other subject sets by some common attributes. The implementation of a subject-oriented database is similar to that of a relational database, except that subjects are not only data fragments but may also have other types of functionality encapsulated within a given subject. If the database of this disclosure is implemented as a data structure, the use of the database of this disclosure can be integrated into another component, such as the component of this invention. Furthermore, databases can be implemented as a hybrid of data structures, subject-specific structures, and relational structures. Databases can be integrated and / or distributed using standard data processing techniques. Parts of the database, such as tables, can be exported and / or imported, thus enabling distribution and / or integration.

[0096] Network 430 can establish connections between components within the MRI platform and between the MRI system and external systems. Network 430 may include any combination of local area networks (LANs) and / or wide area networks (WANs) using wireless and / or wired communication systems. For example, network 430 may include the Internet and mobile phone networks. In one implementation, network 430 uses standard communication technologies and / or protocols. Therefore, network 430 may include links using technologies such as Ethernet, 802.11, WiMAX, 2G / 3G / 4G mobile communication protocols, Asynchronous Transfer Mode (ATM), wireless broadband, PCI Express advanced switching, etc. Other network protocols used on network 430 may include Multiprotocol Label Switching (MPLS), Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), File Transfer Protocol (FTP), etc. Data exchanged over the network may be represented using technologies and / or formats including binary image data (e.g., Portable Web Graphics (PNG)), Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc. Furthermore, all or part of the link can be encrypted using conventional encryption techniques, such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), and Internet Protocol Security (IPsec). In another implementation, entities on the network can use custom and / or dedicated data communication technologies to replace or supplement the aforementioned technologies.

[0097] Whenever the terms "at least," "greater than," or "greater than or equal to" precede the first value in a series of two or more values, the terms "at least," "greater than," or "greater than or equal to" apply to each value in the series. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0098] Whenever the terms "not exceeding," "less than," or "less than or equal to" precede the first value in a series of two or more values, the terms "not exceeding," "less than," or "less than or equal to" apply to each value in that series. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0099] As used herein, A and / or B includes one or more of A or B, and combinations thereof, such as A and B. It should be understood that although the terms “first,” “second,” “third,” etc., are used herein to describe various elements, components, regions, and / or portions, these elements, components, regions, and / or portions should not be limited by these terms. These terms are used only to distinguish one element, component, region, or portion from another. Therefore, without departing from the teachings of the invention, a first element, component, region, or portion discussed herein may be referred to as a second element, component, region, or portion.

[0100] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” include a plural referent. It will be further understood that, when used in this specification, the terms “comprising” and / or “including” or “comprising” and / or “including” specify the presence of stated features, areas, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, areas, integrals, steps, operations, elements, components, and / or combinations thereof.

[0101] Throughout this specification, references to "some embodiments" or "one embodiment" indicate that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "in some embodiments" or "in one embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, in one or more embodiments, particular features, structures, or characteristics may be combined in any suitable manner.

[0102] While preferred embodiments of the invention have been shown and described herein, it will be readily understood by those skilled in the art that these embodiments are provided by way of example only. This is not to imply that the invention is limited to the specific examples provided herein. Although the invention has been described with reference to the foregoing description, the description and exposition of embodiments herein are not intended to be construed as limiting. Various variations, modifications, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it should be understood that all aspects of the invention are not limited to the specific descriptions, configurations, or relative proportions presented herein under various conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein can be used to implement the invention. Therefore, it is contemplated that the invention should also cover any such alternatives, modifications, alterations, or equivalents. The following claims are intended to define the scope of the invention and thereby cover the methods and structures within the scope of these claims and their equivalents.

Claims

1. A computer-implemented method for improving image quality by reducing the dosage of a contrast agent, the method comprising: (a) Using a medical imaging device to acquire a volumetric medical image of a subject with a reduced dose of contrast agent, wherein the volumetric medical image is acquired in a first direction of the scanning plane; (b) Generating one or more reformulated volumetric medical images of the subject by reformulating the volumetric medical images in one or more orientations, wherein the one or more orientations include a second orientation different from the first orientation of the scanning plane; as well as (c) During the inference phase, an input including the one or more reformatted volumetric medical images is fed into a deep network model to produce a predicted medical image with improved quality.

2. The computer-implemented method according to claim 1, wherein the medical imaging device is a transformation magnetic resonance (MR) device.

3. The computer-implemented method according to claim 1, wherein the volumetric medical image is a 2.5D volumetric image.

4. The computer-implemented method of claim 1, further comprising rotating each of the one or more reformulated volumetric medical images to various angles to produce a plurality of rotated reformulated medical images.

5. The computer-implemented method of claim 4, further comprising applying the deep network model to the plurality of rotated, reformatted medical images to output a plurality of predicted images.

6. The computer-implemented method of claim 5, wherein the plurality of predicted images are rotated to align with a scanning plane.

7. The computer-implemented method of claim 6, further comprising averaging the plurality of predicted images after rotation to align with the scanning plane to produce the predicted medical image with improved quality.

8. The computer-implemented method of claim 1, wherein the predicted medical image with improved quality is obtained by averaging one or more predicted medical images corresponding to the one or more reformatted volumetric medical images.

9. The computer-implemented method of claim 1, wherein the parameters of the deep learning model are adjusted at least in part based on perceptual loss or adversarial loss.

10. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform an operation, comprising: (a) Using a medical imaging device to acquire a volumetric medical image of a subject with a reduced dose of contrast agent, wherein the volumetric medical image is acquired in a first direction of the scanning plane; (b) Generating one or more reformulated volumetric medical images by reformulating the volumetric medical images of the subject in one or more orientations, wherein the one or more orientations include a second orientation different from the first orientation of the scanning plane; as well as (c) During the inference phase, the input, including the one or more reformatted volumetric medical images, is fed into a deep network model to produce a predicted medical image with improved quality.

11. The non-transitory computer-readable storage medium of claim 10, wherein the medical imaging device is a transformation magnetic resonance (MR) device.

12. The non-transitory computer-readable storage medium of claim 10, wherein the volumetric medical image is a 2.5D volumetric image.

13. The non-transitory computer-readable storage medium of claim 10, wherein the operation further comprises rotating each of the one or more reformatted volumetric medical images to various angles to produce a plurality of rotated reformatted medical images.

14. The non-transitory computer-readable storage medium of claim 13, wherein the operation further comprises applying the deep network model to the plurality of rotated, reformatted medical images to output a plurality of predicted images.

15. The non-transitory computer-readable storage medium of claim 14, wherein the plurality of predicted images are rotated to align with the scanning plane.

16. The non-transitory computer-readable storage medium of claim 15, wherein the operation further comprises averaging the plurality of predicted images after rotation to align with the scanning plane to produce the predicted medical image with improved quality.

17. The non-transitory computer-readable storage medium of claim 10, wherein the predicted medical image with improved quality is obtained by averaging one or more predicted medical images corresponding to the one or more reformatted volumetric medical images.

18. The non-transitory computer-readable storage medium of claim 10, wherein the parameters of the deep learning model are adjusted at least in part based on perceptual loss or adversarial loss.

Images