Diffusion-weighted magnetic resonance imaging

Abstract

A method for performing diffusion-weighted magnetic resonance imaging is provided, comprising the steps of: filtering a diffusion-weighted image of an object acquired by a magnetic resonance imaging scanner using a neural network, wherein the neural network is programmed to generate an output image from the acquired image. The neural network improves the signal-to-noise ratio of the output image relative to the acquired image. When applied to a synthetic knife-edge image with added Rician noise that provides a signal-to-noise ratio of 13 or greater, the neural network: forming a curve of normalized values ​​of the modulation transfer function versus frequency having a higher area thereunder compared to the area under a corresponding curve of normalized values ​​of the modulation transfer function versus frequency for the Gaussian smoothing filter referenced in JPEG2022548924000035.jpg54; JPEG2022548924000036.jpg14126, where f(x,y) is the synthetic knife-edge image of noise, g(x,y) is the filtered image, JPEG2022548924000037.jpg56 is the convolution parameter, and σ 2 is σ 2 =4.

Description

[Technical field]

[0001] The present disclosure relates to the performance of diffusion-weighted magnetic resonance imaging. [Background technology]

[0002] Whole-body diffusion-weighted magnetic resonance imaging (WBDWI) is an attractive, non-invasive tool for the assessment of disease stage and response in: malignant lymphoma, metastatic bone disease from the prostate [1], [2], and breast cancer [3]; WBDWI was recently introduced into the NICE guidelines for assessing the extent of disease in the evaluation of myeloma-associated bone disease [4]. This technique sensitizes magnetic resonance imaging contrast to the diffusion rate of water in tissues through the application of a magnetic gradient, which can be manipulated to adjust the magnitude of diffusion weighting in the image. The contrast generated between diseased and healthy tissue using this technique can provide radiologists with a sensitive tool to review the extent of bone disease in the skeleton. The degree of diffusion weighting in the image can be manipulated at will through the modification of the so-called "b-value," and the timing and strength of the diffusion weighting gradient can be encapsulated in one variable (typically between 0 and 5000 s / mm in clinical systems). 2 By acquiring images for the same anatomical location with a b-value of ≥2, WBDWI provides voxel-wise quantification of the water "apparent diffusion coefficient" (ADC), a potential surrogate marker of tumor response to antitumor therapy. Previous anatomic pathology studies have shown that an increase in ADC after treatment can indicate the destruction of the tumor microenvironment and, consequently, a decrease in tumor cellularity, which can indicate tumor necrosis [5]-[7]. Summary of the Invention [Problem to be solved by the invention]

[0003] Current WBDWI studies are typically acquired in the axial plane through a series of consecutive imaging stations from head to mid-thigh, where each station consists of 30-50 imaging slices. In addition, (i) b-values ​​are applied in three or more directions, resulting in directionally independent measurements of ADC, and (ii) each diffusion-weighted image is acquired multiple times (averaging results in images with higher SNR) because this technique has a lower signal-to-noise ratio (SNR) compared to anatomical magnetic resonance imaging. This can result in 30-50 image acquisitions per slice (when using three b-values), requiring a total acquisition time of 20-30 minutes. Adding in the anatomical imaging sequence and patient setup and positioning, this can add up to a total study time of approximately one hour. This creates a disastrous bottleneck in the adoption of WBDWI into general clinical workflow; the ever-growing patient population increases capacity pressure on imaging centers, which discourages them from performing such long imaging studies. Moreover, the patients for whom WBDWI is clinically useful are typically frail and symptomatic with painful bones, which reduces patient acceptance of the technique. Schemes to accelerate the acquisition of WBDWI are then greatly needed. [Means for solving the problem]

[0004] Summary of the Invention

[0005] In general terms, the present disclosure provides for the use of a neural network to filter diffusion weighted images of an object acquired by a magnetic resonance imaging scanner, the neural network being programmed to generate an output image from the acquired images. Surprisingly, the neural network used in this manner can provide "clinical grade" images from subsampled data, potentially reducing a significant amount of time to acquire.

[0006] More specifically, in a first aspect, the present disclosure provides a method of performing diffusion-weighted magnetic resonance imaging, the method comprising the steps of: using a neural network to filter diffusion weighted images of an object acquired by a nuclear magnetic resonance imaging scanner, said neural network being programmed to generate an output image from said acquired images; wherein the neural network improves the signal-to-noise ratio of the output image relative to the derived image; and Here, when applied to a synthetic knife-edge image with added Rician noise that gives a signal-to-noise ratio of 13 or more (preferably 5 or more, more preferably 3 or more), the neural network

number

number

number

[0007] Advantageously, the neural network can be used to improve the image quality of a single acquired DWI image, which may thus be acquired by a magnetic resonance imaging scanner at a given b-value and at a given orientation with respect to that b-value, without signal averaging.

[0008] The neural network may be a spatially-varying filter. The degree of smoothing performed by the network in a particular region of the acquired image may depend on the location of that region within the overall imaging field. Particularly in the context of objects being human or animal specimens, this allows the network to learn anatomical locations in order to tune the degree of smoothing it performs at those particular anatomical locations.

[0009] The neural network may be a convolutional neural network.

[0010] The method may include the preliminary step of: obtaining the diffusion weighted image using the magnetic resonance imaging scanner.

[0011] The obtained image may be obtained at a first b-value and at a given location within the object, in which case the method may further comprise the step of combining the output image with one or more further diffusion weighted images, wherein the further diffusion weighted images are obtained at the given location within the object (e.g. by repeating the obtaining and using steps for each further image), but each obtained at a different b-value to produce an image of the apparent diffusion coefficient of water at the given location within the object.

[0012] The object may be a human or animal specimen, in which case the method may further comprise the step of: analysing the output image or an image derived therefrom to assess the degree of disease (degree of cancer) in the human or animal specimen.

[0013] The method is typically implemented on a computer. Accordingly, further aspects of the present disclosure provide: a computer program comprising a code, which, when executed on a computer, causes the computer to perform the method of the first aspect; a computer-readable medium having a computer program stored thereon, which comprises a code, which, when executed on a computer, causes the computer to perform the method of the first aspect; and a computer system programmed to perform the method of the first aspect. For example, further aspects of the present disclosure provide: an imaging system for performing diffusion-weighted magnetic resonance imaging, the system including: a nuclear magnetic resonance imaging scanner for obtaining a diffusion weighted image of the object; and a computer system receiving the acquired image and programmed with a neural network, the neural network filtering the acquired image and generating an output image from the acquired image; where the neural network improves the signal to noise ratio of the output image relative to the derived image by a factor X scaled by Y; and Here, when applied to a synthetic knife-edge image with added Rician noise that gives a signal-to-noise ratio of 13 or more (preferably 5 or more, more preferably 3 or more), the neural network

number

number

number

[0014] The network may be a spatially-varying filter.The network may be a convolutional neural network.

[0015] In another aspect, the present disclosure provides a method of training a neural network, the network being programmed to generate an output image by filtering a diffusion weighted image of an object obtained by a magnetic resonance imaging scanner, the method comprising: providing a training set of diffusion weighted images of one or more objects acquired by one or more magnetic resonance imaging scanners, each image of the training set acquired by a respective magnetic resonance imaging scanner at a respective location of the body, at a respective b-value, in each orientation for the b-value, and without signal averaging, where for each location, images are provided at different b-values ​​and in different orientations for each b-value, and where a plurality of images are provided for each combination of a given location, a given b-value, and an orientation for the given b-value; combining the images from the training set for each combination of a given location, a given b-value, and an orientation of the given b-value to form a respective ground truth image for the given location and the given b-value; and The neural network is trained to minimize a cost function that measures the similarity between each image in the training set and the corresponding ground truth image for the same b-value at the same location.

[0016] Thus, the method of this other aspect may be used to train the network of the first aspect above.

[0017] Conveniently, the cost function may use mean absolute error to measure similarity. The neural network may be a convolutional neural network. The training set of diffusion weighted images may be images of human or animal subjects.

[0018] The term "computer-readable medium" may refer to one or more devices for storing data and may include: read-only memory (ROM), random-access memory (RAM), magnetic RAM, core memory, magnetic disk storage media, optical storage media, flash memory devices, and / or other machine-readable media for storing information. The term "computer-readable medium" includes, but is not limited to: portable or built-in storage devices, optical storage devices, wireless channels, and various other media that can store, contain, or carry instruction(s) and / or data.

[0019] Further, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments to perform the necessary tasks may be stored on a computer readable medium. One or more processors may perform the necessary tasks. A code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, sent, or transmitted through any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0020] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0021] [Figure 1] The generation of a "clinical standard" image z from a single acquired image xij was achieved by: (i) computing the geometric mean over the different directions j, and the resulting arithmetic mean over the trace-weighted image yi. These operations mimic the processing performed by most clinical scanners when acquiring WBDWI. In clinical scanners, only the averaged image (z) is kept, while all other data is removed to reduce storage requirements. [Diagram 2] We show our U-NET-like architecture for processing an input noisy image from a single acquisition at random orientation xj to predict an image zpred. The mean absolute error (L) is used as a cost function, which evaluates the perceptual closeness of the predicted image to the acquired clinical standard / ground-truth image z. [Diagram 3] An exemplary test patient dataset from the training WBDWI cohort. After applying the De-Noising Image Filter (DNIF) filter (middle row), the image quality at b=900s / mm2 is improved over the originally noisy single acquisition image (left row), where the image quality is comparable to the clinical standard (right row). At the same b-value, both the axial image (middle row) and the resulting coronal maximum intensity projection (top row) are very clear. Furthermore, the image quality in the ADC map is also greatly improved (bottom row), where the image quality is comparable to the clinical standard ADC map. [Figure 4] Violin plots of ADC variance within segmented bone lesions for the three test patients in the training WBDWI cohort. Only positive ADC values ​​were included. There is a decrease in the range of ADC values ​​for deep learning DNIF networks trained with MAE and MAE / SSIM cost functions, respectively. We attribute this to an improvement in the SNR of the resulting ADC maps after DNIF smoothing. [Figure 5-1] Example images and ADC maps with axis b=900 s / mm2, clinical standard, unfiltered single acquisition (NSA=1) and DNIF filtered images of patients in the pilot WBDWI cohort. With the DNIF filter, a clear improvement in image and map quality is seen in all cases compared to the unfiltered image with NSA=1. [Figure 5-2] Example images and ADC maps with axis b=900 s / mm2, clinical standard, unfiltered single acquisition (NSA=1) and DNIF filtered images of patients in the pilot WBDWI cohort. With the DNIF filter, a clear improvement in image and map quality is seen in all cases compared to the unfiltered image with NSA=1. [Figure 6-1] Example dataset of three patients from the test arm of the lung cohort. Although there was a clear improvement in image quality for the filter with DNIF pre-trained with WBDWI images, further improvements were observed when the filter was retrained with data acquired specifically in patients with MPM. Specifically, improvements were observed in crisper disease contrast in the following areas: higher b-value images (arrow A), sharper tissue boundaries (arrow B), and more accurate ADC measurements (arrow C). [Figure 6-2] Example dataset of three patients from the test arm of the lung cohort. Although a clear improvement in image quality was seen with the filter using DNIF pre-trained with WBDWI images, further improvements were seen when the filter was retrained with data acquired specifically in patients with MPM. Specifically, such improvements were observed in the following areas of crisper disease contrast: higher b-value images (arrow A), sharper tissue boundaries (arrow B), and more accurate ADC measurements (arrow C). [Figure 7] Our AI filter operates on a low-quality single average image f(x;y)

number

number

[0022] Detailed Description

[0023] Below, we describe a study exploring the innovative use of artificial intelligence (AI) to dramatically reduce scan times in WBDWI, thus making it a feasible clinical approach for diagnosing, staging, and assessing treatment response in diseases such as malignant lymphoma, metastatic bone disease from the prostate, and breast cancer. In essence, such an approach effectively recovers fully sampled WBDWI images from significantly undersampled diffusion-weighted images (e.g., in a single b-value direction and without multiple averaging), cutting acquisition times by an order of magnitude of 5–10 times. To train the AI ​​model, we use a promising WBDWI imaging protocol that may be acquired on most clinical scanners. We validate our technique through a second prospective cohort of patients with metastatic prostate cancer and myeloma-related bone disease. Furthermore, we demonstrate the feasibility of the technique with diffusion-weighted imaging acquired over a smaller field of view through a retrospective analysis of a cohort of patients with malignant pleural mesothelioma (MPM) [8]. Our approach provides an image processing pipeline that can be easily implemented in most WBDWI post-processing systems.

[0024] Materials and Methods

[0025] Patient population

[0026] All studies were performed in accordance with the Declaration of Helsinki and ethical approval was obtained for these patient cohorts.

[0027] Training WBDWI Cohort

[0028] WBDWI was obtained in 17 patients with suspected metastatic prostate cancer in the skeleton using a 1.5T MR system (Siemens Aera, Erlangen, Germany). Axial, spin-echo, and eco-planar imaging (SE-EPI) were acquired from the base of the skull to the mid-thigh using the following conditions: slice thickness 5 mm, inversion time (TI) 180 ms (to suppress fat signal), eco-time (TE) 79 ms, repetition time (TR) 12.7 s, field of view (FoV) 128 × 104 pixels (interpolated to 256 × 208), resolution 1.68 × 1.68 mm. 2 (after interpolation), GRAPPA parallel imaging with a pixel readout bandwidth (BW) of 1955 Hz / pixel and an acceleration factor R = 2. Images at each axial position were acquired using the following b-values: 50, 600, and 900 s / mm 2 ;For each b value, we consider the orthogonal orientation (b x ,b y ,b x Images were acquired using θ = (-1,0,0), (0,1,0), and (0,0,1), respectively. A single image was acquired for each b-value / orientation pair (i.e., no signal averaging was used), and a "trace-weighted" image was computed as the geometric mean of these images in post-processing (see Fig. 1 for an exemplary illustration of this process). These acquisitions were repeated three times, and a conventional clinical WBDWI image was provided as the "gold standard" by the arithmetic mean of three consecutive trace-weighted images (shown in Fig. 1) (number of signal averages, NSA = 3). All images (individually acquired and the final average image obtained) were recorded for further processing. These data were split 14 / 3 into training / validation cohorts, respectively.

[0029] Study WBDWI cohort

[0030] WBDWI was prospectively acquired in a cohort of 24 patients with metastatic prostate cancer (N=19), metastatic breast cancer (N=2), and multiple myeloma (N=3) on the same scanner as the training cohort. For each patient, we acquired data using two WBDWI protocols within the same study (patients were kept on the couch during the study). First, the same protocol was used as the training cohort, but with a single average (NSA=1) and a single b-value orientation (b x ,b y ,b x ) ∝ (0,0,1). Second, we performed a conventional clinical WBDWI scan with the following parameters: slice thickness = 5 mm, TI = 180 ms, TE = 64 ms, TR = 6.2 s, FoV = 134 × 108 pixels (interpolated to 268 × 216), resolution = 1.6 × 1.6 mm. 2 (after interpolation), BW=2330Hz / pixel, R=2 (GRAPPA), where NSA=4; data are at b-values ​​of 50, 600, and 900s / mm 2 For each protocol, trace-weighted images were acquired across three orthogonal diffusion encoding directions. We attempted to match the field of view of both protocols, from either the base of the skull (prostate cancer) or the vertex (myeloma) to the mid-thigh. In post-processing, the clinical protocol was resampled to match the field of view and resolution of the data acquired using the shorter single acquisition protocol. The approximate acquisition times for these protocols were approximately 5 min for the single acquisition protocol and 22-25 min for the clinical protocol.

[0031] Lung cohort

[0032] To demonstrate the success of our approach in other diffusion-weighted imaging (DWI) studies, we retrospectively evaluated data from a cohort of patients evaluated for the presence of malignant pleural mesothelioma (MPM). This cohort consisted of 28 patients scanned using DWI to evaluate the response of MPM to targeted agents; in this study, the baseline scan was used for evaluation only. Imaging consisted of a similar protocol to the whole-body training cohort, using a 1.5T scanner (Siemens Avanto, Erlangen, Germany): images were acquired axially with SE-EPI, where b-values ​​= 100, 500, and 800 s / mm 2 and in three orthogonal directions (b x ,b y ,b x ) ∝ (-1,0,0), (0,1,0), and (0,0,1). For each b-value / orientation pair, the acquisition was repeated four times, and for each, a clinical-grade scan from the geometric and arithmetic mean of the different orientations and repetitions was obtained (according to Figure 1). Images were acquired over two imaging stations (30 slices / station) to cover the entire chest with the following conditions: SPAIR fat suppression, TE = 92 ms, TR = 6 s, FoV = 128 × 92, Resolution = 3 × 3 mm. 2 , BW=1860Hz / pixel and R=2 (GRAPPA). These data were split into training / validation cohorts of 20 / 8 patients, respectively.

[0033] AI Network Architecture

[0034] Our deep-learned denoising image filter (DNIF) contains a convolutional neural network based on the U-Net architecture (as shown in Figure 2) [9]. Images of size 256 × 208 pixels are provided as input to the network (image size after interpolation of the training WBDWI cohort data) and normalized from the range 0-4095 to the range 0-1. For the last layer, we utilize linear activation, while ReLU activation functions are used in all preceding layers. We impose weights incident on each hidden unit to ensure that its norm value is less than or equal to 3; we randomly initialize the layer weights using He normal initialization

[10] , and train the network using Adam

[11] with a learning rate of 10 -3 It was optimized.

[0035] To train the network, we minimize a cost function that measures the similarity between DNIF filter images resulting from a noisy input image (NSA=1) and a clinical image that serves as the training ground truth after averaging (NSA=9). The default choice is to minimize the mean squared error (MSE) between the two images. However, due to the known limitations of the metric in assessing image quality as perceived by human observers

[12] , we used two different measures of image similarity that are more correlated with human perception of image quality: (i) the mean absolute error (MAE), and (ii) a combination of the mean absolute error and the structural similarity (SSIM) index

[13] (Eq. 1). L mae / SSIM =a L mae +(1-a)·L SSIM (Formula 1)

[0036] where a is the weight of the contribution of each error measure to the function. After testing different values, we empirically set a to 0.7 (we observed that small changes in a did not significantly affect the results).

[0037] The network was trained on 59400 WBDWI slices from the first 14 patients in the training WBDWI cohort (approximately 14 patients x 3 orientations x 3 acquisitions x 3 b-values ​​x 150 slices) using a batch size of 36 images for a number of epochs of 15. The performance of the network was validated on an independent validation set consisting of 15120 images from the three remaining patients. The trained network was subsequently applied to single acquisition data acquired in the test WBDWI cohort.

[0038] For the lung cohort, we retrained the network on 43,200 WBDWI slices from the first 20 patients (network weights were randomly initialized), and then tested the retrained network on an independent test set of 15,120 MRI slices from the remaining 8 patients. We resized the images using linear interpolation so that the input image size was 256 × 208 pixels, and then normalized the images from the 0-939 range to the 0-1 range before inputting them into the model. For comparison, we also applied the network trained from the training WBDWI cohort to the same set of test lung patients.

[0039] Data analysis

[0040] Training WBDWI Cohort

[0041] An expert radiologist (>10 years of experience) blindly scored images from three study patients in the cohort for: (i) SNR, (ii) contrast-to-noise ratio (CNR), (iii) tumor detection, (iv) presence of image artifacts, and (v) overall image quality, using the following Likert scale: 1=poor to 5=excellent. NSA=1, NSA=9, and b=900s / mm resulting from the DNIF method, respectively. 2 Diffusion-weighted images and ADC maps from the CT scan were used for this visual assessment. Additionally, for the study patients, radiologists roughly identified regions of bone disease on the averaged data using an in-house semi-automated segmentation tool for metastatic disease identified in the WBDWI study. These regions of interest (ROIs) were copied onto ADC maps calculated from (i) single-acquired images (NSA=1), (ii) averaged "clinical standard" images (NSA=9), and (iii) DNIF-filtered images obtained by applying the network to the NSA=1 data. The mean ADC of bone disease within these regions was compared.

[0042] Study WBDWI and Lung Cohort

[0043] b=900s / mm 2 The quality of DNIF filtered images at b = 50, 600, and 900 s / mm was visually compared to the clinical standard for all patients in the study WBDWI cohort. 2 A similar visual comparison was performed for the eight study patients from the lung patient cohort, where b = 800 s / mm2. 2 DNIF filtered at b = 100, 500, and 800 s / mm 2 The images were calculated using the ADC maps and compared.

[0044] result

[0045] AI Network Architecture

[0046] Within 15 epochs, the network achieved a mean absolute error of 0.87×10 -3 to 0.53 x 10 -3 and the MAE / SSIM error is minimized to 0.39×10 -2 to 0.11 x 10 -2 Both cost functions yielded identical MAE solutions (0.53 × 10 -3 ). Interestingly, L mae Or L mae / SSIM By using either of the cost functions, the network arrived at a better solution in terms of mean squared error than by trying to minimize the mean squared error directly (L mae MSE from: 1.89×10 -6 vs L mae / SSIM MSE from: 1.88×10 -6 vs L MSE : 2.7×10 -6 ). Visual inspection by expert radiation practitioners concluded that L mse Networks trained with the cost function applied too much smoothing to the images without preserving edges well, and this measure was not used in further analysis.

[0047] The network took 8 hours to train on the WBDWI data using a Tesla P100-PCIE-16GB GPU card. In terms of computational efficiency, the trained network requires only about 1 second to process a single low-SNR image on a MacBook Pro (3.5 GHz, Intel Core i7 CPU).

[0048] Training WBDWI Cohort

[0049] An example of the DNIF filter applied to the entire field of view in one of the study patients is shown in FIG. 3. Clearly, the DNIF filter can successfully reduce the effects of imaging noise in the input NSA=1 images. Such noise reduction can lead to superior image quality in the resulting and calculated ADC maps. The DNIF filter images were radiologically superior to the NSA=1 and NSA=9 images in all three study patients (mean Likert scale across all five criteria: 3.73 [3.6-4.0] vs. 1.6 [1.2-1.8] and 2.53 [2.4-2.6], respectively), thus improving image quality.

[0050] The mean ADC values ​​were measured within the bone lesions from the AI-generated images, and the AI-generated images were generated from the mean lesion ADC, calculated from NSA=9 images, with an average of 2.4% [range: 0.6-4.5] (within the previously reported repeatability limits for the mean ADC measurements). A violin plot of the ADC values ​​within the segmented regions is shown in Figure 4, showing the ability of DNIF to reduce the range of the calculated ADC measurements as a result of the reduction in SNR in the calculated ADC map. Since little difference was observed in the performance of the MAE and MAE / SSIM cost functions, we chose to use the MAE-derived network for further evaluation, due to the simplicity of the cost function and the fact that no weighting of the parameter "a" is required in its definition (Eq. 1).

[0051] Study WBDWI and Lung Cohort

[0052] Application of the DNIF filter was successful in all 22 study patients in the study WBDWI cohort. Improvements in image quality in terms of contrast-to-noise ratio for high b-value images and in the resulting ADC maps were seen for all patients, and results for six randomly selected patients are shown in Figure 5.

[0053] Figure 6 compares the results for three of the test patient datasets from the lung cohort, showing NSA=1 images filtered using a network trained with the original training WBDWI cohort and another network retrained with new lung data. The results show that while the WBDWI-trained network can still be useful in smoothing DW images acquired in MPM, the architecture can indeed be improved by retraining with disease-specific data.

[0054] discussion

[0055] Our WBDWI deep-learned denoising image filter (DNIF) architecture significantly improves image quality in heavily subsampled WBDWI acquisitions. We tested our training algorithm on a dataset of pre-acquired and collected WBDWI images in patients with either metastatic prostate cancer or myeloma-related bone disease, and compared the DNIF-filtered images to “clinical standard” WBDWI images acquired during the same imaging study. Our results suggest that DNIF can provide “clinical-grade” WBDWI images from subsampled data, potentially significantly reducing the amount of time to acquire (from about 25 minutes in our prior work to about 5 minutes). Such time savings could reduce the cost of scanning and make WBI suitable for screening studies, in terms of patient time commitment and / or discomfort. We provide evidence that ADC measurements generated using DNIF filtered images provide equivalent information to measurements made using fully sampled WBDWI data, and estimates of average ADC in bone lesions calculated using DNIF may be robust enough to monitor treatment response. Our approach to obtain the training data required to generate a DNIF network can be adopted and applied in any imaging center with existing MR hardware. We show that our proposed method can be adopted for other disease types investigated using DWI (e.g., malignant pleural mesothelioma), which typically use smaller fields of view than those used for whole-body measurements. Furthermore, we show that WBDWI-trained DNIF filters can be used to improve image quality for single-acquired DWI images acquired in MPM, but such techniques improve if training data is acquired for the disease in question.More generally, the network will benefit from adding training data from other institutions, MR vendors, and different protocols, with the aim of developing filters that are robust enough to evaluate data from any WBDWI study.

[0056] Using the protocol we described for acquiring individual WBDWI acquisitions and then, through post-processing, obtaining clinical standard WBDWI scans, we expect that the network can be trained for clinical protocols on different scanners. Such an approach can exploit the concept of "transfer learning": by using weights from our DNIF network as initialization, individual sites may not need to acquire more data to train a robust network specific to that site.

[0057] The training of DNIF neural networks is based on trying to make the input image as similar as possible to the "ground truth". However, the correct assessment of image similarity by algorithms is an ongoing problem in the field of computer vision. As a default choice, mean squared error is mostly used due to its simplicity and being well understood, but it suffers from many known limitations, including strong assumptions; such a metric assumes that noise is white (Gaussian distributed) and does not depend on local image characteristics

[14] . Moreover, although such a metric may be valid in other applications, it produces images that do not correlate with the image quality perceived by humans (two images with very low mean squared error can look quite different to a human observer)

[12] . In the healthcare domain, this can pose a major problem, since radiology personnel must make clinical decisions based on the images obtained. In this work, we investigate another local metric, the mean absolute error, and we also combine this with SSIM, a metric that is closer to human perception.

[0058] Similarity measures such as MSE, MAE, and SSIM show that the input image is numerically close to the “ground truth” image, despite the poor visual quality and low SNR of the input image. This does not leave much room for the network to further minimize the error measures. However, we achieve low MSE by minimizing MAE, even though these measures have different convergence properties. Specifically, MAE does not over-penalize large errors. This is also in line with the findings of Zhao et al., where they try to combine denoising and de-mosaicing in an image restoration problem

[14] .

[0059] Understanding the inner workings of any deep learning algorithm can be a challenging task. However, if such techniques are to be adopted within the healthcare sector and support applications for medical formal approval, evidence of how deep learning networks arrive at specific results is useful. In the appendix, we provide evidence of how our DNIF filter can work. We also provide preliminary evidence that the DNIF filter is at least (i) nonlinear, (ii) spatially variable, (iii) nonlocal, and (iv) edge-preserving. This demonstrates the advantages of a deep learning approach, since it is generally impractical to generate a “handcrafted” filter that can simultaneously achieve these properties. From these results, we hypothesize that the DNIF filter learns the complex relationships between pixels in the field of view based on their relative positions and relative intensities within the field of view. Furthermore, we believe (especially due to spatial variation) that the DNIF filter learns anatomical locations in order to tune the degree of smoothing it exerts at specific anatomical locations.

[0060] It is to be understood that the present invention is not limited to the above-described embodiments and various modifications, and may be modified without departing from the concepts described herein. Any feature may be employed separately or in combination with any other feature, except where mutually exclusive, and the present disclosure extends to and includes all combinations or subcombinations of one or more features described herein.

[0061] References All references mentioned herein are hereby incorporated by reference. [1] D. M. Koh et al., “Whole-body diffusion-weighted mri: Tips, tricks, and pitfalls,” American Journal of Roentgenology, vol. 199, no. 2. pp. 252-262, 2012. [2] M. Eiber et al., “Whole-body MRI including diffusion-weighted imaging (DWI) for patients with recurring prostate cancer: Technical feasibility and assessment of lesion conspicuity in DWI,” J. Magn. Reson. Imaging, vol. 33, no. 5, pp. 1160-1170, 2011. [3] A. R. Padhani, D.-M. Koh, and D. J. Collins, “Whole-Body Diffusion-weighted MR Imaging in Cancer: Current Status and Research Directions,” Radiology, vol. 261, no. 3, pp. 700-718, 2011. [4] S. L. Giles et al., “Whole-Body Diffusion-weighted MR Imaging for Assessment of Treatment Response in Myeloma,” Radiology, vol. 271, no. 3, pp. 785-794, 2014. [5] D. A. Hamstra, K. C. Lee, B. A. Moffat, T. L. Chenevert, A. Rehemtulla, and B. D. Ross, “Diffusion Magnetic Resonance Imaging: An Imaging Treatment Response Biomarker to Chemoradiotherapy in a Mouse Model of Squamous Cell Cancer of the Head and Neck,” Transl. Oncol., vol. 1, no. 4, pp. 187-194, 2008. [6] H. C. Thoeny and B. D. Ross, “Predicting and monitoring cancer treatment response with diffusion-weighted MRI,” Journal of Magnetic Resonance Imaging, vol. 32, no. 1. pp. 2-16, 2010. [7] D. K. Hill et al., “Non-invasive prostate cancer characterization with diffusion-weighted MRI: Insight from in silico studies of a transgenic mouse model,” Front. Oncol., vol. 7, no. DEC, 2017. [8] L. Cheng et al., “Response evaluation in mesothelioma: Beyond RECIST,” Lung Cancer, vol. 90, no. 3, 2015. [9] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” Miccai, pp. 234-241, 2015.

[10] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, vol. 2015 International Conference on Computer Vision, ICCV 2015, pp. 1026-1034.

[11] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic gradient descent,” ICLR Int. Conf. Learn. Represent., pp. 1-15, 2015.

[12] L. Zhang, L. Zhang, X. Mou, and D. Zhang, “A comprehensive evaluation of full reference image quality assessment algorithms,” in Proceedings - International Conference on Image Processing, ICIP, 2012, pp. 1477-1480.

[13] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, 2004.

[14] H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss Functions for Image Restoration With Neural Networks,” IEEE Trans. Comput. Imaging, vol. 3, no. 1, pp. 47-57, 2016.

[15] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018, pp. 586-595.

[16] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen, “Image reconstruction by domain-transform manifold learning,” Nature, vol. 555, no. 7697, pp. 487-492, 2018.

[17] B. Zhu, B. Bilgic, C. Liao, B. Rosen, and M. Rosen, “Deep learning MR reconstruction with automated transform by manifold approximation (AUTOMAP) in real-world acquisitions with imperfect training,” in Proc. Intl. Soc. Mag. Res. Med., 2018, p. 572.

[0062] Appendix

[0063] Understanding how deep learning can refine spatial smoothing in DWI

[0064] 1. Background

[0065] In this appendix, we explore how our deep-learned de-noising image filter (DNIF) can improve the perceived image quality in single-averaged DWI acquisitions. In this study, we take a top-down approach to gain insight into how the DNIF filter works: we apply the

number

number

number

number

number

[0066] 2. Hypothesis

[0067] In the following, we explain our hypotheses about how our DNIF filters can provide optimal smoothing of noisy DWI images. In each case, we state our hypothesis, explain the theory behind what our hypothesis represents in terms of image processing, and we describe our experimental setup of how we test our hypothesis. We present our results and draw our conclusions accordingly.

[0068] 2.1 DNIF is a nonlinear filter

[0069] We define the nonlinearity of the DNIF filter with respect to the input image intensity as follows:

number

number

number

number

[0070] In Figure 9, we also show the effect of varying the magnitude of the bias term k, for k∈{1,2…9}. Clearly, as k increases, deep learning results less and less from linear filters.

[0071] 2.2 DNIF is a spatially variable filter Spatially invariant imaging filters (e.g., Gaussian smoothing filters)

number

number

number

number

number

number

[0072] To show the impact of the shift magnitude on the DNIF filter image, we also performed the same experiment for x0 = y0 ∈ {1, 2, 3, 5, 8, 13, 21, 34, 55} pixels. The results are shown in Figure 11, where it is clear that as the shift magnitude increases, the degree of deviation of deep learning also increases.

[0073] 2.3 DNIF is a non-local filter

[0074] In a general sense, any imaging filter

number

number

number

number

[0075] 2.4 DNIF is an edge-preserving filter

[0076] We attribute some of the success of the DNIF filter to its ability to preserve edges in the output filter image. To demonstrate the quality of these edge preservations, we synthetically generated a noisy image containing a synthetic "knife edge" where the left half of the image has uniform pixel intensity μ and the right half is zero-filled. Rise-distributed noise (where scale parameter σ) was added to this image. The signal-to-noise ratio (SNR) can be varied in these synthetic images by adjusting μ such that SNR=μ / σ, while we used a constant σ=6:99 in all experiments (this value was estimated from the background noise of a single average DWI data set as an example). The synthetic knife edge image was passed through the DNIF filter and a Gaussian filter in order to obtain estimates of the edge spread function (ESF), line spread function (LSF), and modulation transfer function (MTF) for each filter; a full description of this experiment is given in FIG. 14.

[0077] MTF is a well-known method for characterizing the performance of an imaging system and provides quantitative information on how well the imaging system can resolve structures over a range of spatial length scales and intensity contrasts. Here, we compare the MTF curves generated by DNIF and Gaussian filters at different SNR values ​​(range SNR ∈ {2, 3, 5, 8, 13, 21, 34, 55, 89}). For each SNR, we simulated a knife-edge image and measured the resulting MTF 100 times, taking the average of these measurements to provide the final MTF at that SNR. Higher edge-preserving performance was identified as the filter that exhibited a higher area under the normalized MTF curve. The results of these experiments are shown in Figure 15, where we observe that the DNIF filter significantly outperforms the conventional smoothing filter when we observe SNR ≥ 3. Thus, we conclude that DNIF exhibits improved edge-preserving properties over the Gaussian smoothing filter.

[0078] 3. Conclusion

[0079] Throughout this appendix, we have detailed our main hypothesis of how our DNIF filter can produce optimal results for smoothing diffusion weighted images with low SNR. We note that currently it is impractical to fully understand the complexities of how deep learning methods derive inputs from specific predictions due to the thousands of parameters that are tuned during the training process. However, we have provided preliminary evidence that at least the DNIF filter is (i) nonlinear, (ii) spatially variable, (iii) nonlocal, and (iv) edge-preserving. This shows the true power of such methods, since it is impractical to generate a “handcrafted” filter that can simultaneously achieve these properties, and thus deep learning methods are important. Indeed, from these results, we hypothesize that the DNIF filter learns the complex relationships between pixels in the field of view based on their position and relative intensity within the field of view. Moreover, we believe that (especially due to spatial variability) the DNIF filter can learn anatomical locations in order to tune the degree of smoothing it exerts at specific locations. These properties suggest that DNIF filters should be used with caution, and potentially only to process images acquired with the particular diffusion-weighted sequence for which the initial cohort of training data was obtained. Transfer learning of our network may also prove valuable, to accelerate the training process and reduce the amount of new training data required for other diffusion-weighted sequences. In one aspect, the present disclosure includes the following invention. (Invention 1) 1. A method for performing diffusion weighted magnetic resonance imaging, comprising the steps of: filtering diffusion weighted images of the object acquired by a magnetic resonance imaging scanner using a neural network, the neural network being programmed to generate an output image from the acquired images; wherein the neural network improves the signal-to-noise ratio of the output image relative to the derived image; and Now, when applied to a synthetic knife-edge image with added Rician noise giving a signal-to-noise ratio of 13 or greater, the neural network

number

number

number

number

number

number

Claims

1. 1. A method for performing diffusion weighted magnetic resonance imaging, comprising the steps of: providing a convolutional neural network, the convolutional neural network pre-trained to minimize a cost function that measures similarity between (A) each image of a training set of diffusion weighted images of one or more objects acquired by one or more magnetic resonance imaging scanners, and (B) a corresponding ground truth image, such that the neural network is programmed to generate an output image from the acquired images; using said convolutional neural network to filter a diffusion weighted image of an object obtained by a magnetic resonance imaging scanner; wherein each image of the training set was acquired by a respective magnetic resonance imaging scanner at each location of the body, at each b-value, in each orientation of the b-value, and without signal averaging, where for each location, images are provided at different b-values ​​and in different orientations for each b-value, and where multiple images are provided for each combination of a given location, a given b-value, and an orientation of the given b-value; wherein each corresponding ground truth image is formed by combining the images from the training set for each combination of a given location, a given b-value, and an orientation of the given b-value to form a respective ground truth image for the given location and the given b-value; wherein the convolutional neural network improves the signal-to-noise ratio of the output image relative to the derived image; and Now, when applied to a synthetic knife-edge image with added Rician noise giving a signal-to-noise ratio of 13 or greater, the convolutional neural network gives the following: [0010] forming a curve of normalized values ​​of the modulation transfer function versus frequency having a higher area thereunder as compared to an area under a corresponding curve of normalized values ​​of the modulation transfer function versus frequency for a reference Gaussian smoothing filter; [0025] where f(x,y) is the composite knife-edge image of the noise and g(x,y) is the filter image. [0030] is the convolution parameter, and σ 2 is σ 2 = 4.

2. 2. The method of claim 1, wherein the acquired image is obtained by the magnetic resonance imaging scanner at a given b-value and at a given orientation with respect to that b-value without signal averaging.

3. 3. The method of claim 1 or 2, wherein the neural network is a spatially-varying filter.

4. The method according to any one of claims 1 to 3, comprising the preliminary steps of: Obtaining the diffusion weighted image using the magnetic resonance imaging scanner.

5. 5. The method of claim 1, wherein the acquired image is acquired at a first b-value and a given location within the object, and the method further comprises the steps of: Combining the output image with one or more further diffusion weighted images obtained at the given location within the object, each obtained at a different b-value to produce an image of the apparent diffusion coefficient of water at the given location within the object.

6. The method according to any one of claims 1 to 5, wherein the object is a human or animal specimen.

7. 7. The method of claim 6, further comprising the steps of: Analysing the output image or an image derived therefrom to assess the degree of disease in the human or animal subject.

8. 1. An imaging system for performing diffusion weighted magnetic resonance imaging, the system comprising: a nuclear magnetic resonance imaging scanner for obtaining a diffusion weighted image of the object; and a computer system receiving the acquired image and programmed with a convolutional neural network, the convolutional neural network filtering the acquired image and generating an output image from the acquired image; wherein the convolutional neural network is pre-trained to minimize a cost function that measures the similarity between (A) each image in a training set of diffusion-weighted images of one or more objects acquired by one or more magnetic resonance imaging scanners, and (B) a corresponding ground truth image. wherein each image of the training set was acquired by a respective magnetic resonance imaging scanner at each location of the body, at each b-value, in each orientation of the b-value, and without signal averaging, where for each location, images are provided at different b-values ​​and in different orientations for each b-value, and where multiple images are provided for each combination of a given location, a given b-value, and an orientation of the given b-value; wherein each corresponding ground truth image is formed by combining the images from the training set for each combination of a given location, a given b-value, and an orientation of the given b-value to form a respective ground truth image for the given location and the given b-value; wherein the convolutional neural network improves the signal-to-noise ratio of the output image relative to the derived image; and Now, when applied to a synthetic knife-edge image with added Rician noise giving a signal-to-noise ratio of 13 or greater, the convolutional neural network gives the following: [0045] forming a curve of normalized values ​​of the modulation transfer function versus frequency having a higher area thereunder as compared to an area under a corresponding curve of normalized values ​​of the modulation transfer function versus frequency for a reference Gaussian smoothing filter; [0050] where f(x,y) is the composite knife-edge image of the noise and g(x,y) is the filter image. [006] is the convolution parameter, and σ 2 is σ 2 = 4.

9. 9. The computer system of the imaging system of claim 8.

10. A computer program comprising code, which, when executed on a computer, causes the computer to carry out the method according to any one of claims 1 to 3.

11. A computer readable medium having stored thereon the computer program of claim 10.

12. 1. A method for training a convolutional neural network programmed to generate an output image by filtering a diffusion weighted image of an object obtained by a nuclear magnetic resonance imaging scanner, the method comprising: providing a training set of diffusion weighted images of one or more objects acquired by one or more magnetic resonance imaging scanners, each image of the training set acquired by a respective magnetic resonance imaging scanner at a respective location of the body, at a respective b-value, in each orientation for the b-value, and without signal averaging, where for each location, images are provided at different b-values ​​and in different orientations for each b-value, and where a plurality of images are provided for each combination of a given location, a given b-value, and an orientation for the given b-value; combining the images from the training set for each combination of a given location, a given b-value, and a given b-value orientation to form a respective ground truth image for the given location and the given b-value; and Training the convolutional neural network to minimize a cost function that measures the similarity between each image in the training set and the corresponding ground truth image at the same location and b-value as that image.

13. 13. The method of claim 12, wherein the cost function uses mean absolute error to measure similarity.

14. 14. The method of claim 12 or 13, wherein the training set of diffusion weighted images are images of human or animal subjects.

Images