Systems and methods for reducing colored noise in medical images using deep neural networks

Abstract

The present invention provides a method and system for denoising a medical image using a deep neural network. In one embodiment, a method includes: receiving a medical image acquired by an imaging system, wherein the medical image includes colored noise; mapping the medical image to a denoised medical image using a trained convolutional neural network (CNN); and displaying the denoised medical image via a display device. The deep neural network can therefore reduce the colored noise in the acquired noisy medical image, thereby improving the clarity and diagnostic quality of the image.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to processing medical images, such as magnetic resonance images (MRI), CT images, and the like, and more particularly, to reducing noise in medical images using deep neural networks. Background Art

[0002] Medical imaging systems are typically used to obtain anatomical and / or internal physiological information of a subject, such as a patient. For example, a medical imaging system can be used to obtain medical images of a patient's skeletal structure, brain, heart, lungs, and various other features. A medical image can be an image generated by a medical imaging system. Medical imaging systems can include magnetic resonance imaging (MRI) systems, computed tomography (CT) systems, x-ray systems, ultrasound systems, and various other imaging modalities.

[0003] Medical images obtained by certain imaging modalities (such as MRI) may contain one or more types of noise, which may reduce image clarity and resolution. The presence of noise in medical images may affect diagnostic quality. Specifically, k-space sampling patterns, image reconstruction, and post-processing may produce medical images with colored noise (e.g., noise that is unevenly distributed in the spatial frequency domain) in magnetic resonance (MR) images, which may be difficult to reduce by existing image processing methods. Deep learning methods have been proposed for removing colored noise from medical images, however current deep learning methods perform inconsistently in removing colored noise and generally fail to produce a sufficient degree of noise reduction. Therefore, it is generally desirable to explore deep learning techniques to identify new methods for consistently reducing colored noise in medical images. Summary of the invention

[0004] The present disclosure at least partially solves the above-mentioned problems. In one embodiment, the present disclosure provides a method, comprising: acquiring a medical image via an imaging system, wherein the medical image includes colored noise; mapping the medical image to a denoised medical image using a trained convolutional neural network (CNN); and displaying the denoised medical image via a display device. By mapping the medical image including colored noise to the denoised medical image using a trained CNN, the colored noise in the image can be significantly reduced, thereby improving the clarity and diagnostic quality of the image.

[0005] The above advantages and other advantages and features of the present specification will be apparent from the following detailed description when taken alone or in conjunction with the accompanying drawings. It should be understood that the above summary is provided to introduce a selection of concepts further described in the detailed description in a simplified form. This is not meant to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. In addition, the claimed subject matter is not limited to embodiments that address any disadvantages mentioned above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present disclosure may be better understood by reading the following detailed description and referring to the accompanying drawings, in which:

[0007] Figure 1 is a schematic diagram illustrating an image processing system for removing colored noise from a medical image using a deep neural network according to an exemplary embodiment;

[0008] Figure 2 is a diagram showing a method that can be used according to an exemplary embodiment. Figure 1 A schematic diagram of a layout of an embodiment of a deep neural network used in an image processing system;

[0009] Figure 3 is a diagram showing an exemplary embodiment of the Figure 1 A schematic diagram of the architecture of the deep neural network used in the system;

[0010] Figure 4 is a flow chart illustrating a method for removing colored noise from a medical image using a deep neural network according to an exemplary embodiment;

[0011] Figure 5 is a flow chart illustrating a method for synthesizing colored noise based on noise power of a k-space sampling pattern according to an exemplary embodiment;

[0012] Figure 6 An example of sampling density in the spatial frequency domain, an example of noise power in the spatial frequency domain, an example of synthetic colored noise in the spatial frequency domain, and an example of synthetic colored noise in the spatial domain are shown according to an exemplary embodiment;

[0013] Figure 7 Additional exemplary k-space sampling patterns are shown according to exemplary embodiments;

[0014] Figure 8 is a flow chart illustrating a method for generating training data for a deep neural network according to an exemplary embodiment;

[0015] Fig. 9shows a comparison between a medical image with noise, a medical image after white noise reduction, and a medical image after colored noise reduction according to an exemplary embodiment; and

[0016] Fig.10 A comparison between a medical image with noise and a medical image after colored noise reduction according to an exemplary embodiment is shown.

[0017] The accompanying drawings illustrate specific aspects of the described systems and methods for reducing colored noise in medical images using deep neural networks. Together with the following description, the accompanying drawings illustrate and explain the structures, methods, and principles described herein. In the accompanying drawings, the sizes of components may be exaggerated or otherwise modified for clarity. Well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the described components, systems, and methods. DETAILED DESCRIPTION

[0018] In magnetic resonance imaging (MRI), a subject is placed in a magnet. The subject is a person (alive or dead), an animal (alive or dead), or a part of a person or an animal. When the subject is in a magnetic field generated by a magnet, the magnetic moments of nuclei such as protons attempt to align with the magnetic field, but will precess around the magnetic field in a random order at the Larmor frequency of the nucleus. The magnetic field of the magnet is called B0 and extends in the longitudinal or z direction. During the acquisition of MRI images, a magnetic field in the xy plane and close to the Larmor frequency (called the excitation magnetic field B1) is generated by a radio frequency (RF) coil and can be used to rotate or "tilt" the net magnetic moment Mz of the nucleus from the z direction to the transverse or xy plane. After the excitation signal B1 terminates, the nucleus emits a signal called an MR signal. In order to use the MR signal to generate an image of the subject, magnetic field gradient pulses (Gx, Gy, and Gz) are used. Gradient pulses are used to scan through k space, the space or reciprocal of the spatial frequency. There is a Fourier relationship between the acquired MR signal and the image of the subject, so the image of the subject can be derived by reconstructing the MR signal. As used herein, frequency specifically refers to spatial frequency, rather than the temporal frequency domain.

[0019] Medical images may contain noise, which limits image resolution and reduces diagnostic quality. In one example, a magnetic resonance (MR) image may contain white noise caused by thermal energy provided by a patient during the imaging process. White noise is uniformly distributed in the spatial frequency domain (e.g., k-space) of the image. The acquired radio frequency MR data typically contains uniform noise, and if the data is used to fill the k-space in a uniform manner, the noise in the k-space is uniform (e.g., white noise). However, certain image acquisition and image processing procedures may produce colored noise in medical images. Colored noise is unevenly distributed in the k-space of the image and may be more difficult to reduce using existing methods. For example, colored noise in a magnetic resonance (MR) image may be caused by uneven oversampling, in which some of the sampling overlaps and / or exceeds the Nyquist sampling criterion in at least one region of the frequency domain. As another example, colored noise in an MR image may be caused by uneven undersampling caused by noise amplification during image reconstruction (such as during kernel-based parallel imaging reconstruction). As another example, colored noise in an MR image may be caused by variable receive bandwidth or variable readout speed. As another example, colored noise in MR images may be caused by image post-processing techniques such as image smoothing, image sharpening, etc. As another example, noise coloration may be caused by combining multiple images (or multiple k-space image representations). For example, multiple images may be averaged during image post-processing techniques, which may change the noise coloration. In another example, colored noise in MR images may be caused by uneven spatial weighting, where different parts of the medical image may have different noise distributions, resulting in noise coloration.

[0020] The process of removing colored noise from medical images can be complicated when the type / source of the noise is unknown, or when there are multiple sources of colored noise. For example, MR images may contain a first type of colored noise due to non-uniform sampling patterns (such as the Periodic Rotation Overlapping Parallel Lines with Enhanced Reconstruction (PROPELLER) sampling pattern), and a second type of colored noise due to image sharpening during post-processing. Noise is inherently random and can be difficult to accurately reduce via traditional computational methods, especially without full knowledge of the k-space sampling pattern and any other image processing performed.

[0021] The following description relates to various embodiments for removing colored noise from medical images using a deep neural network, which at least partially solves the above-mentioned problems. Specifically, a medical image is acquired by an imaging system and may be subjected to reconstruction and post-processing. The acquired medical image, which may contain colored noise, is then processed using a trained deep neural network, which may optionally be assigned one or more noise parameters of the image. A medical image containing colored noise is defined as a noisy image in this article. The noise parameters may be related to the type and distribution of colored noise in the medical image, and may include various imaging system settings used to acquire the medical image, such as a k-space sampling pattern. The noise parameters may also include, for example, noise power in the spatial frequency domain, and other information about the noisy image. In other words, the noise parameters may correspond to the source of colored noise in the medical image. The deep neural network may reduce the colored noise in the image and output an image with significantly less colored noise (e.g., a denoised image). In this way, a denoised medical image may be generated from a corresponding noisy medical image. In addition, the deep neural network may be trained before being put into use. In one embodiment, training data for a neural network may be generated by synthesizing colored noise based on noise parameters, such as based on a k-space sampling pattern.

[0022] As used herein, denoising is the process of reducing noise, such as colored noise, in a medical image. Noise may be synthesized in the frequency domain according to noise power, and may be distributed differently in the frequency domain based on the characteristics of the noise. Thus, denoising includes a process of at least partially reducing the presence of at least one type of colored noise in an image to improve image resolution and diagnostic quality. A denoised image may be generated by denoising a noisy image (e.g., a medical image with colored noise), and conversely, a noisy image may be generated by introducing one or more sources of colored noise into an original image (e.g., a medical image without colored noise). Specifically, a noisy image may be generated from a corresponding original image based on a k-space sampling pattern and / or expected noise characteristics, and the resulting image pair (e.g., a noisy image and a corresponding original image) may be used as training data for a deep neural network. It should be understood that in some cases, the original medical image may be acquired directly using an imaging system.

[0023] refer to Figure 1, a medical image processing system 100 according to an exemplary embodiment is shown. In some embodiments, the medical image processing system 100 is incorporated into a medical imaging system such as an MR system. In some embodiments, the medical image processing system 100 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively connected to the medical imaging system via a wired connection and / or a wireless connection. In some embodiments, the medical image processing system 100 is disposed at a separate device (e.g., a workstation) that can receive images from the medical imaging system or from a storage device that stores images generated by the medical imaging system. The medical image processing system 100 may include an image processing system 31, a user input device 32, and a display device 33.

[0024] The image processing system 31 includes a processor 104 configured to execute machine-readable instructions stored in a non-transitory memory 106. The processor 104 can be single-core or multi-core, and the program executed thereon can be configured to perform parallel or distributed processing. In some embodiments, the processor 104 can optionally include separate components distributed in two or more devices, which can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 104 can be virtualized and executed by a remotely accessible networked computing device configured with a cloud computing configuration.

[0025] The non-transitory memory 106 may store a deep neural network module 108, a colored noise module 116, a training module 112, and medical image data 114. The deep neural network module 108 may include one or more deep neural networks including a plurality of parameters (including weights, biases, activation functions), and instructions for implementing the one or more deep neural networks to receive a noisy medical image and map the noisy medical image to an output, wherein a denoised medical image corresponding to the noisy medical image may be generated from the output. For example, the deep neural network module 108 may store instructions for implementing a neural network such as Figure 3 300 of the convolutional neural network (CNN) architecture 300 shown. However, other architectures may also be used, such as a combination of a fully connected network and a CNN or a generative adversarial network and variations thereof.

[0026] The deep neural network module 108 may include trained and / or untrained neural networks, and may also include various data or labels related to one or more neural networks stored therein. In some embodiments, the deep neural network labels may include an indication of the training data used to train the deep neural network, the training method used to train the deep neural network, the accuracy / validation score of the deep neural network, and the type of anatomical structures / imaging protocols to which the deep neural network can be applied.

[0027] The non-transitory memory 106 also stores a colored noise module 116, which includes instructions for synthesizing colored noise based on an expected k-space sampling pattern. The colored noise module 116 may include instructions that, when executed by the processor 104, cause the image processing system 31 to perform one or more of the steps of the method 500 discussed in detail below. In some embodiments, the colored noise module 116 includes instructions for generating a spatial representation of colored noise based on the k-space sampling pattern by weighting a k-space (e.g., spatial frequency domain) representation of white noise with noise power derived from a sampling density map, and then transforming the resulting k-space representation into a spatial domain. The colored noise reduction module 116 may be used by the training module 112 to generate training images for training a deep neural network.

[0028] The non-transitory memory 106 may also store a training module 112, which includes instructions for training one or more deep neural networks in the deep neural network stored in the deep neural network module 108. The training module 112 may include instructions that, when executed by the processor 104, cause the image processing system 31 to perform one or more of the steps of the method 700 discussed in detail below. In some embodiments, the training module 112 includes instructions for implementing one or more gradient descent algorithms, applying one or more loss functions and / or training routines for adjusting one or more deep neural networks of the deep neural network module 108. In some embodiments, the training module 112 includes instructions for generating training data pairs from the medical image data 114. In some embodiments, the training data pairs include corresponding noise and original medical image pairs of the same anatomical region. In some embodiments, the training module 112 includes instructions for generating noisy medical images by applying / adding noise synthesized by the colored noise module 116 to the original medical image. In some embodiments, the training module 112 is not provided at the image processing system 31. The deep neural network module 108 includes a network for training and validation.

[0029] The non-transitory memory 106 also stores medical image data 114. The medical image data 114 includes, for example, MR images acquired using an MRI system, ultrasound images acquired by an ultrasound system, etc. For example, the medical image data 114 may store noisy and / or original medical images. In some embodiments, the medical image data 114 may include a plurality of training data pairs, the plurality of training data pairs including noisy and original medical image pairs.

[0030] In some embodiments, non-transitory memory 106 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 106 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0031] The image processing system 100 may also include a user input device 32. The user input device 32 may include one or more of a touch screen, a keyboard, a mouse, a trackpad, a motion sensing camera, or other devices configured to enable a user to interact with and manipulate data within the image processing system 31. For example, the user input device 32 may enable a user to select a medical image on which to perform colored noise reduction.

[0032] The display device 33 may include one or more display devices utilizing virtually any type of technology. In some embodiments, the display device 33 may include a computer monitor and may display unprocessed and processed MR images and / or parametric maps. The display device 33 may be combined with the processor 204, the non-transitory memory 106, and / or the user input device 32 in a shared housing, or may be a peripheral display device and may include a monitor, a touch screen, a projector, or other display device known in the art that enables a user to view medical images and / or interact with various data stored in the non-transitory memory 106.

[0033] It should be understood that Figure 1 The image processing system 100 shown is for illustration and not limitation. Another suitable image processing system may include more, fewer, or different components.

[0034] For example, a system includes: a magnetic resonance imaging (MRI) system, a memory storing a trained deep neural network, a display device, and a processor communicatively coupled to the memory and configured to remove colored noise from a medical image.

[0035] Go to Figure 2, a schematic diagram of a first embodiment of a colored noise reduction process 200 for reducing noise in a medical image is shown. The colored noise reduction process 200 may be implemented by the image processing system 100 to at least partially reduce colored noise in a noisy medical image. The colored noise reduction process 200 is used to illustrate an embodiment of a noise reduction system in which a deep neural network 224 is used to reduce colored noise in a noisy medical image 206. In some embodiments, noise parameters 204 may be incorporated into the deep neural network 224. The noise parameters 204 may characterize the colored noise in the noisy medical image 206. For example, the noise parameters 204 may include a k-space sampling pattern, a k-space sampling density, a noise density, image processing parameters, etc. By incorporating the noise parameters 204, the deep neural network 224 may receive at least partial information about the degree, type, and distribution of colored noise in the noisy medical image 206 so as to increase the amount and accuracy of the colored noise reduction. In some embodiments, the noise parameters 204 may not be incorporated into the deep neural network 224, and the deep neural network may reduce colored noise in the noisy medical image 206 without information about the noise characteristics of the image.

[0036] The colored noise reduction system 200 includes a deep neural network 224 that receives inputs including noise parameters 204 and a noisy medical image 206, and produces a denoised image (e.g., an image with reduced colored noise) as an output. The input is received by the deep neural network 224 and mapped to a denoised image 220. The deep neural network 224 includes a learned convolution filter 214 (learned during a training process) and a learned deconvolution filter 218 (learned during a training process). By propagating the input through the convolution layer and the deconvolution layer of the deep neural network 224, a denoised image 220 is produced.

[0037] The noise parameters 204 may include one or more parameters that at least partially characterize the noise in the noisy medical image 206. For example, the noise parameters 204 may include k-space noise power. As another example, the noise parameters 204 may include a spatial map of the noise power in k-space. In some examples, the noise parameters 204 may be determined based on an acquisition method, while in other examples, the noise parameters 204 may be determined independent of the acquisition method. For example, the noise parameters 204 may include whether a certain type of colored noise is expected based on an image processing technique applied to the noisy medical image 206 prior to denoising.

[0038] In one embodiment, the noise parameters 204 include acquisition data of the noisy medical image 206. The acquisition data of the medical image includes one or more settings used by the imaging system during the acquisition of the noisy medical image 206 and / or one or more physiological properties and / or environmental conditions of the imaged tissue / patient. For example, if the noisy medical image 206 is an MR image, the acquisition data may include one or more of the echo train length, repetition time, echo time, echo spacing, target flip angle, k-space sampling pattern, k-space sampling density, acquisition order, physiological signals, or other parameters / settings used by the MRI system during the acquisition of the noisy medical image 206 or associated with the imaged patient. The acquisition data may also include whether post-processing (e.g., such as image sharpening) has been applied to the noisy medical image 206. In particular, the k-space sampling pattern and the k-space sampling density may affect the type and distribution of colored noise in the medical image, especially when the k-space sampling pattern is characterized by non-uniform oversampling or non-uniform undersampling. For example, a k-space sampling pattern with non-uniform oversampling may produce noise coloration due to non-uniform noise reduction.

[0039] The noisy medical image 206 is a medical image of an anatomical region, including a plurality of values, each of which may be referred to as a pixel (for a 2D image) or a voxel (for a 3D image). The noisy medical image 206 includes noise, which may generally be described as unwanted visual data that degrades the quality of the medical image. Furthermore, noise may be mathematically described as a degree of randomness in the frequency domain, where the color of the noise is determined by its distribution in the frequency domain. For example, the noisy medical image 206 includes an MR image of a human abdomen, the MR image including one or more colored noises, where the degree and / or type of the noise varies within a k-space representation of the noisy medical image 206. That is, the k-space representation of the noisy medical image 206 may include a first region having a noise distribution, and a second region including a second noise distribution, where the first noise distribution and the second noise distribution are different.

[0040] The input, which may include the noise parameters 204, may be propagated through multiple layers within the deep neural network 224 to map the intensity values ​​of the noisy medical image 206 to the intensity values ​​of the denoised image 220. For example, the deep neural network 224 may determine a set of weights to be applied based on the noise parameters 204. The deep neural network 224 includes a learned convolution filter 214 and a learned deconvolution filter 218. The deep neural network 224 may also include one or more densely connected layers (not shown) and one or more pooling layers (not shown), one or more upsampling layers (not shown), and one or more ReLU layers (not shown) or any layers conventional in the field of machine learning. The noise parameters 204 may be directly incorporated into the deep neural network 224 by setting the convolution filter 214 and / or the deconvolution filter 218 based on the noise parameters 204. In other embodiments, the noise parameters 204 may each be incorporated into the deep neural network 324 via multiple different mechanisms / channels. In other embodiments, the noise parameters 204 may not be provided to the deep neural network 224.

[0041] The output of the deep neural network 224 may be used to generate a denoised image 220 that includes an image of the same anatomical region as the noisy medical image 206 , but in which the colored noise is at least partially reduced.

[0042] Go to Figure 3 , an exemplary CNN architecture 300 for mapping a noisy medical image to a denoised medical image is shown. The CNN architecture 300 provides a more detailed description of a deep neural network (such as the deep neural network 224) that can perform colored noise reduction of noisy medical images. In some embodiments, a subset of parameters of the CNN architecture 300 can be selected / determined based on noise parameters. For example, Figure 2 As indicated in the illustrated colored noise reduction system 200 , the CNN architecture 300 may incorporate one or more noise parameters.

[0043] Figure 3The CNN architecture 300 shown represents a U-net architecture, which can be divided into an autoencoder portion (descending portion, elements 302b-330) and an autodecoder portion (ascending portion, elements 332-356a). The CNN architecture 300 is configured to receive a medical image including colored noise, which can be a magnetic resonance (MR) image. In one embodiment, the CNN architecture 300 is configured to receive data from a noisy medical image of an anatomical region including a plurality of pixels / voxels (such as the noisy medical image 302a), and map the input noisy medical image data to a denoised medical image of the same anatomical region (such as the denoised medical image 356b). The CNN architecture 300 includes a series of mappings from an input image tile 302b that can be received by an input layer through a plurality of feature mapping maps and finally to an output denoised medical image 356b that can be generated based on the output of an output layer 356a.

[0044] Various elements comprising the CNN architecture 300 are labeled in the legend 358. As indicated in the legend 358, the CNN architecture 300 includes multiple feature maps (and / or replicated feature maps) connected by one or more operations (indicated by arrows). The arrows / operations receive inputs from an external file or a previous feature map, and transform / map the received inputs to outputs to produce the next feature map. Each feature map may include multiple neurons, wherein in some embodiments, each neuron may receive inputs from a subset of neurons in a previous layer / feature map, and may compute a single output based on the received inputs, wherein the output may be propagated / mapped to a subset or all of the neurons in the next layer / feature map.

[0045] The feature map may be described using the terms length, width, and depth, where each term refers to the number of neurons that comprise the feature map (e.g., a given feature map is how many neurons long, how many neurons wide, and how many neurons deep). The length and width used by the reference feature map correspond to the spatial dimensions of the image being processed, and in some cases may correspond to the number of pixels / voxels of the image. The depth used by the reference feature map may correspond to the number of features in each feature channel.

[0046] The transformations / mappings performed between each feature map are indicated by arrows, where each different type of arrow corresponds to a different type of transformation, as indicated in legend 358. The solid black arrow pointing to the right indicates a 3×3 convolution with a stride of 1, where the output of the 3×3 grid of features from the immediately preceding feature map (where the 3×3 grid extends through all layers of the immediately preceding feature map) is mapped to a single feature at a single depth of the current feature map by performing a dot product between the output / activation of the 3×3 grid of feature channels and a 3×3 filter (where each layer / each unit depth of the immediately preceding feature map includes 9 weights). In some embodiments, the convolution filter weights may be selected based on the noise parameters 204. In some embodiments, the convolution filter weights may be learned during the training process. The filter used to perform the 3×3 convolution is referred to herein as a convolution filter (convolution filter / convolutional filter) or a convolution kernel (convolution kernel / convolutional kernel).

[0047] The downward pointing arrow indicates a 2×2 max pooling operation, where the maximum value of a 2×2 grid of feature channels at a single depth is propagated from the immediately previous feature map to a single feature at a single depth in the current feature map, resulting in an output feature map with a spatial resolution reduced by a factor of 3 compared to the immediately previous feature map. In one example, max pooling of a 2×2 activation grid from the immediately previous feature map, where the 2×2 activation grid includes (2, 1.3, 10, 3.3), produces an output of (10), since 10 is the maximum activation value within the 2×2 grid.

[0048] The upward pointing arrows indicate 2×2 up-convolutions with a stride of 2, which include performing a transposed convolution (also referred to herein as a deconvolution) using a deconvolution filter including a plurality of weights (the filter used to perform the transposed convolution is also referred to herein as a deconvolution filter (deconvolutional filter / deconvolution filter)), mapping the output of a single feature channel at each feature depth from the immediately previous feature map to a 2×2 feature grid at a single feature depth in the current feature map, thereby increasing the spatial resolution of the immediately previous feature map by a factor of 3.

[0049] The dashed arrow pointing to the right indicates copying and cropping of a feature map to be cascaded with another feature map that comes later. Cropping enables the dimensions of the copied feature map to match the dimensions of the feature channel to be cascaded with the copied feature map. It should be understood that when the size of the first feature map being copied and the size of the second feature map to be cascaded with the first feature map are equal, cropping may not be performed. For example, the cascaded feature maps may have the same size and no cropping may be performed.

[0050] The right-pointing arrow with a hollow head indicates a 1×1 convolution with a stride of 1, where each feature channel in the immediately previous feature map is mapped to a single feature channel of the current feature map, or in other words, where a 1-to-1 mapping of feature channels occurs between the immediately previous feature map and the current feature map. The processing at each feature map may include the convolution and deconvolution described above, as well as activation, where the activation function is a nonlinear function that limits the processed output values ​​to within a bounded range.

[0051] In addition to the operations indicated by the arrows in legend 358, CNN architecture 300 also includes a solid filled rectangle corresponding to a feature map, where the feature map includes a height (e.g., Figure 3 The length from top to bottom shown corresponds to the y spatial dimension in the xy plane), the width ( Figure 3 , assuming that the magnitude is equal to the height, corresponding to the x spatial dimension in the xy plane) and the depth (as Figure 3 , corresponding to the number of features in each feature channel). Similarly, CNN architecture 300 includes a hollow (unfilled) rectangle corresponding to the copied and cropped feature map, where the copied feature map includes a height (e.g. Figure 3 The length from top to bottom shown corresponds to the y spatial dimension in the xy plane), the width ( Figure 3 , assuming that the magnitude is equal to the height, corresponding to the x spatial dimension in the xy plane) and the depth (as Figure 3 The length from left to right shown corresponds to the number of features in each feature channel).

[0052] Starting at an input image tile 302b (also referred to herein as an input layer), data corresponding to a noisy medical image 302a is input and mapped to a first set of features. In some embodiments, the noisy medical image 302a includes one or more layers corresponding to one or more features of the image (such as each intensity value of a multi-color image). In some embodiments, a noise parameter layer cascaded with the noisy medical image 302a may indicate an expected / desired type or intensity of colored noise at each pixel location of the noisy medical image 302a. The noisy medical image 302a may include a two-dimensional (2D) or three-dimensional (3D) image / map of an anatomical region of a patient. In some embodiments, the input data from the noisy medical image 302a is pre-processed (e.g., normalized) before being processed by the neural network.

[0053] The output layer 356a may include an output layer of neurons, wherein each neuron may correspond to a pixel (or residual) of the predicted denoised medical image 356b, wherein the output of each neuron may correspond to the predicted pixel intensity at a specified location within the output denoised medical image 356b.

[0054] In this manner, the CNN architecture 300 may implement a mapping of multiple intensity values ​​from a noisy medical image 302a to multiple intensity values ​​of a denoised medical image 356b, wherein the degree of colored noise present in the noisy medical image 302a is reduced or eliminated in the denoised medical image 356b. In some embodiments, the CNN architecture 300 may implement a mapping of one or more features of a pixel / voxel of a noisy medical image to one or more attributes of a denoised medical image. The CNN architecture 300 illustrates the transformation of a feature map that occurs when an input image tile is propagated through a layer of neurons of a convolutional neural network to produce a denoised medical image. In one example, the CNN architecture 300 may implement a mapping of multiple pixel / voxel intensity values ​​of a noisy medical image to a residual map, wherein a denoised medical image may be produced by combining the residual map with the input noisy medical image 302a, such as by pixel-by-pixel addition of values.

[0055] The weights (and biases) of the convolutional layers in the CNN architecture 300 may be learned during training, which will be referred to below. Figure 7Detailed discussion. The CNN architecture 300 can be trained by calculating the difference between the predicted denoised medical image and the ground truth denoised medical image, wherein the ground truth denoised medical image may include a medical image without colored noise. In addition, in some embodiments, for example, the CNN architecture 300 can select between multiple weights learned in training based on the noise parameters 204 so as to select calibrated weights to perform accurate colored noise reduction using the noise parameters 204. When the noise parameters 204 are not provided and only the noisy medical image 206 is provided to the CNN architecture 300, the CNN architecture can select a set of weights learned and calibrated in training to perform accurate noise removal for various types of colored noise. The difference between the predicted denoised medical image and the ground truth denoised medical image can be used to determine the loss, and the gradient descent or any other parameter optimization method known in the field of machine learning can be used to back propagate the loss through the neural network to update the weights (and bias) of each feature map. During the training process of the CNN architecture 300, multiple training data pairs can be used, including noisy medical images and corresponding ground truth denoised medical images.

[0056] although Figure 3 Not shown, but it should be understood that the present disclosure encompasses neural network architectures that include one or more regularization layers, including batch normalization layers, dropout layers, Gaussian noise layers, and other regularization layers known in the field of machine learning, which can be used during training to mitigate overfitting and improve training efficiency while reducing training time.

[0057] It should be understood that Figure 3 The CNN architecture 300 shown in is for illustration and not limitation. Any suitable neural network (such as ResNet, recursive neural network, generalized regression neural network (GRNN), etc.) may be used herein to predict denoised medical images from noisy medical images. One or more specific embodiments of the present disclosure are described above to provide a thorough understanding. These described embodiments are merely examples of systems and methods for predicting denoised medical images from noisy medical images using deep neural networks and one or more noise parameters. Those skilled in the art will appreciate that the specific details described in the embodiments may be modified in implementation without departing from the essence of the present disclosure.

[0058] refer to Figure 4 , a flow chart of a method 400 for reducing colored noise in noisy medical images using a deep neural network according to an exemplary embodiment is shown. The method 400 may be implemented by the imaging processing system 100, an edge device connected to an imaging device, a cloud communicating with an imaging device, or any suitable combination thereof. In addition, the deep neural network may be Figure 2The deep neural network 224 shown in FIG. 1 may have the Figure 3 CNN architecture 300.

[0059] Method 400 begins at operation 402, in which a noisy medical image is acquired. In some embodiments, the image processing system acquires the noisy medical image from the imaging system via a communication connection (such as through a network). In some embodiments, the image processing system acquires the noisy medical image from a non-transitory memory. Although described with reference to a single noisy medical image for simplicity, it should be understood that the present disclosure provides mapping of multiple noisy medical images to multiple (or single) denoised medical images. For example, the number of input layers corresponding to the multiple noisy medical images can be increased to accommodate the number of noisy medical images to be denoised without departing from the disclosure provided herein.

[0060] At operation 404, method 400 optionally includes acquiring one or more noise parameters associated with one or more noisy medical images. The noise parameters associated with or corresponding to the noisy medical images may include one or more settings, parameters, or conditions used or present during the acquisition of the noisy medical images. Thus, in some embodiments, the noise parameters include settings of an imaging device used during scanning / image acquisition, such as a k-space sampling pattern. The noise parameters associated with or corresponding to the noisy medical images may include one or more parameters regarding the type and distribution of noise in the noisy medical images. The noise parameters may be stored or indexed with the medical images to which they correspond, so that fast and computationally efficient retrieval of one or more noise parameters associated with the noisy medical images may be achieved. Specifically, the noise parameters may include a k-space sampling pattern used to acquire the medical images, and the k-space sampling pattern may be one of a parallel line (PROPELLER) sampling pattern with enhanced reconstruction, a star stack sampling pattern, and a variable density spiral sampling pattern. Additional k-space sampling patterns will be in Figure 7 Described in more detail in .

[0061] At operation 406, the noisy medical image is mapped to an output using a deep neural network. Mapping the noisy medical image to the output includes: inputting data from the noisy medical image (including any additional cascaded data (e.g., noise parameters)) into an input layer / input tile of the deep neural network; and propagating the input data through each layer of the deep neural network until an output is generated by an output layer of the deep neural network. In some embodiments, the deep neural network includes a convolutional neural network, wherein one or more filters (convolution filters or deconvolution filters) are set based on the noise parameters, and the one or more filters are applied to the data from the noisy medical image as the data is propagated through the deep neural network.

[0062] In addition, when the noise parameter is acquired at 404, mapping the noisy medical image to the output at 406 may also include combining the noise parameter with a deep neural network. In some embodiments, combining the noise parameter with the deep neural network includes: concatenating the noise parameter with the noisy medical image; and inputting both the noisy image and the blurred medical image into an input layer of a trained deep neural network. In some embodiments, the noise parameter includes a plurality of values, and combining the noise parameter into the deep neural network includes setting a plurality of weights in the trained deep neural network based on the plurality of values, wherein in some embodiments, the deep neural network includes a CNN, and the plurality of weights include deconvolution filters of a deconvolution layer of a convolutional neural network or convolution filters of a convolution layer. In some embodiments, the noise parameter may be used for both the input layer and the weights of the filter.

[0063] At operation 408, the output from the deep neural network is used to generate a denoised medical image. In some embodiments, the output includes a residual map, and using the output to produce a denoised medical image from a noisy medical image includes combining the residual map with the noisy medical image to produce the denoised medical image. In other words, the residual map may include a plurality of values, i.e., one or each pixel or voxel of the input noisy medical image, describing the intensity difference between the intensity of each pixel or voxel of the noisy image and the intensity of each pixel or voxel of the corresponding denoised medical image. Combining the residual map with the noisy medical image to produce the denoised medical image may include pixel-by-pixel addition of the values ​​between the residual map and the noisy medical image. In some embodiments, the output from the deep neural network includes a map of pixel / voxel intensity values ​​of the denoised medical image.

[0064] At operation 410, the image processing system displays the denoised medical image via a display device. In some embodiments, the user may select the denoised image via a user input device for further image processing, such as image segmentation, pathology identification, super-resolution, etc., using a model trained on the original medical image. The difference between the denoised medical image output by the deep neural network and the corresponding noisy medical image is determined and back-propagated through the layers / feature maps of the deep neural network.

[0065] In this manner, method 400 enables reducing colored noise in noisy medical images in a time-efficient and more consistent manner via a deep neural network trained on training data with colored noise. Furthermore, by including noise parameters, the deep neural network can more accurately remove colored noise from noisy medical images by selecting a set of weights calibrated for the noise parameters. Furthermore, the denoised medical images can be more efficiently processed by additional downstream image processing models that may have been trained using the denoised medical images.

[0066] For example, a method includes: acquiring a medical image via an imaging system, wherein the medical image includes colored noise; mapping the medical image to a denoised medical image using a trained CNN; and displaying the denoised medical image via a display device. In the foregoing examples, additionally or optionally, mapping the medical image to a denoised medical image using a trained CNN also includes acquiring one or more noise parameters corresponding to a source of the colored noise, and incorporating the one or more noise parameters into the trained CNN. In one or both of the foregoing examples, incorporating the one or more noise parameters into the trained CNN includes selecting a set of pre-trained weights and biases of the trained CNN based on the one or more noise parameters, and the one or more noise parameters include one or more of a k-space sampling pattern for acquiring the medical image and a k-space sampling density for acquiring the medical image.

[0067] As another example, a method includes: acquiring a medical image via an imaging system, the medical image including colored noise; mapping the noisy medical image to an output by a deep neural network; and using the output to generate a denoised medical image. In the foregoing example, additionally or optionally, acquiring the noisy medical image also includes selecting one or more noise parameters, and incorporating the one or more noise parameters into the trained deep neural network. For example, the one or more noise parameters include one or more of a k-space sampling pattern for acquiring the medical image and a k-space sampling density for acquiring the medical image, and the medical image is a magnetic resonance (MR) image.

[0068] A deep neural network can be trained using multiple pairs of noisy medical images and corresponding original images (e.g., image pairs), referred to herein as training data. The noisy image in each image pair is used as input to the deep neural network, and the original image in each image pair is used as ground truth for reference. In some examples, an image pair can be generated by applying colored noise to the original image, thereby generating a corresponding noisy image. The colored noise can be synthesized by weighting the k-space representation of the white noise image by the noise power derived from the sampling density, and then transforming the noise back to the image space. For example, an image space representation of colored noise (e.g., a representation of colored noise in the spatial domain) can be generated by scaling the white noise field based on the k-space representation of the noise power of the k-space sampling pattern, and transforming the resulting noise from the k-space (frequency domain) back to the image space. In this way, a noisy image with colored noise based on the k-space sampling pattern can be generated for the original image.

[0069] therefore, Figure 5A flow chart of a method 500 for synthesizing colored noise generated by non-uniform sampling based on noise power of a k-space sampling pattern is shown. The method 500 may be implemented by the imaging processing system 100, an edge device connected to an imaging device, a cloud communicating with the imaging device, a computing device in wired communication with the imaging device, or any suitable combination thereof. The elements of the method 500 may be performed sequentially or simultaneously.

[0070] At operation 502, method 500 includes synthesizing white noise in image space (e.g., spatial domain). For example, the synthesized white noise can be a synthesized white noise image such that the noise in the synthesized white noise is uniformly distributed across all frequencies. In some examples, the white noise can be synthesized directly in k-space rather than in image space.

[0071] At operation 504, method 500 includes transforming the synthesized white noise image into k-space (e.g., frequency domain) via a two-dimensional (2D) Fourier transform. A 2D Fourier transform is a mathematical function that can transform data from a spatial domain (e.g., image space) into a frequency domain (also referred to as k-space). For example, each point in a 2D frequency domain representation of an image corresponds to the amplitude of a sine wave at a particular spatial frequency in a spatial domain representation of the image. A combination of digital and / or analytical methods can be applied to transform the synthesized white noise image into k-space via a Fourier transform, and the resulting synthesized white noise k-space can be stored in a memory.

[0072] At operation 506, method 500 includes determining a k-space sampling density. The k-space sampling density may be determined based on a k-space sampling pattern. Any suitable k-space sampling pattern may be used to generate the k-space sampling density, including a parallel line with enhanced reconstruction (PROPELLER) sampling pattern, a star stack sampling pattern, a ramp sampling pattern, a weighted average sampling pattern, and a variable density spiral sampling pattern. Figure 6 , an exemplary k-space sampling density 602 of a k-space sampling pattern is shown. Figure 6 The exemplary k-space sampling density 602 shown is the result of a PROPELLER sampling pattern characterized by multiple intersecting blades such that a central region has a relatively high k-space sampling density and an outer region has a relatively low k-space sampling density. For example, the sampling density of a medical image having a PROPELLER sampling pattern can be determined based on blade geometry, blade angle, rotational motion correction, and blade rejection. For a k-space sampling pattern, the k-space sampling density can be determined by calculation and / or analysis based on the characteristics of the expected sampling pattern.

[0073] At operation 508, method 500 includes determining a k-space representation of noise power (eg, a k-space noise power mask) of the k-space sampling pattern based on the k-space sampling density. Figure 6 The k-space sampling pattern may have a corresponding exemplary k-space noise power mask 604, such as Figure 6 As shown. Figure 6 As shown, the exemplary noise power mask 604 can be negatively correlated with the k-space sampling density. For example, a high sampling density in the central region of the exemplary k-space sampling density 602 is associated with a low noise power in the central region of the exemplary k-space noise power mask 604 because repeated rounds of sampling reduce random noise in the collected data. However, in the outer regions of the k-space representation, a low sampling density is associated with a high noise power.

[0074] At operation 510, method 500 includes weighting the synthesized white noise k-space based on the exemplary k-space noise power mask to generate colored noise k-space. Thus, the colored noise k-space (e.g., a synthesized colored noise k-space representation) may be weighted based on the k-space representation of the exemplary k-space noise power mask, as determined at 508. The output of weighting the synthesized white noise k-space based on the noise power mask is the colored noise k-space. Figure 6 An exemplary colored noise k-space 606 for a sampling pattern is shown in FIG.

[0075] At 512, method 500 includes transforming the colored noise k-space to image space via a 2D inverse Fourier transform. The 2D inverse Fourier transform is the mathematical inverse of the 2D Fourier transform used at 504, and can transform a signal from a k-space (e.g., frequency domain) representation to an image space (e.g., spatial domain) representation. For example, a combination of digital and / or analytical methods can be applied to transform the synthesized colored noise in k-space to image space via an inverse Fourier transform, and the resulting synthesized colored noise in image space can be stored in a memory. Figure 6 An exemplary representation of synthesized colored noise in image space 608 for an intended sampling pattern is shown in FIG.

[0076] In this way, colored noise can be synthesized based on a k-space sampling pattern. In other examples, similar methods can be applied to synthesize colored noise based on image processing methods, image reconstruction methods, noise colors, or noise patterns. In addition, in some examples, colored noise can be directly synthesized in k-space according to a noise power map based on noise parameters. By adding the synthesized colored noise to an original image (e.g., a noise-free image), a corresponding noisy image can be generated. Therefore, an image pair includes an original image and a corresponding noisy image, and the training data of the deep neural network may include multiple such image pairs. In addition, it should be noted that a variety of k-space sampling patterns can be used to generate colored noise. Therefore, multiple different noisy images can be generated from a single original image. For example, a first noisy image may include a first pattern of colored noise, and a second noisy image may include a second pattern of colored noise.

[0077] For example, synthesizing colored noise in image space includes: selecting a k-space sampling pattern, selecting a k-space sampling density, generating a k-space noise power mask based on the k-space sampling pattern and the k-space sampling density, synthesizing a white noise image, performing a two-dimensional Fourier transform on the white noise image to generate a white noise k-space, applying the k-space noise power mask to the white noise k-space to generate a colored noise k-space, and performing a two-dimensional inverse Fourier transform on the colored noise k-space to generate synthesized colored noise. For another example, for a non-Cartesian k-space sampling pattern, a Cartesian representation of the k-space noise can be generated based on a re-rasterization algorithm for resampling MRI data into a Cartesian coordinate system, rather than weighting the k-space representation of the white noise, wherein the re-rasterization algorithm includes one or more re-rasterization algorithms known in the field of image reconstruction.

[0078] For another example, synthesizing a colored noise image based on noise parameters includes: selecting a k-space sampling pattern based on the noise parameters, selecting a k-space sampling density, generating a k-space amplitude mask based on the k-space sampling pattern and the k-space sampling density, synthesizing a white noise image, generating a white noise k-space based on the white noise image, applying a k-space noise power mask to the white noise k-space to generate a colored noise k-space, generating a colored noise image based on the colored noise k-space, and indexing the k-space sampling pattern to the colored noise image.

[0079] although Figure 6 An example of synthesizing noise based on a PROPELLER sampling pattern is shown, but various other k-space sampling patterns may be used without departing from the scope of the present disclosure. Figure 7 , three exemplary k-space sampling patterns are shown. Figure 7The exemplary k-space sampling patterns shown can be used during image acquisition of MR images, and thus can be used to synthesize colored noise for training deep neural networks to denoise images. The first exemplary sampling pattern 702 shows a Cartesian sampling pattern. In the first exemplary sampling pattern 702, the k-space sampling pattern includes a series of parallel lines having a vertical orientation in the k-space representation. In addition, as shown, the sampling density in the central portion of the k-space representation is larger, so that a vertical band of higher sampling density spans the k-space representation. In a k-space sampling pattern such as the first exemplary sampling pattern 702, due to the larger sampling density, the noise power may have a first higher value in the outer portion of the k-space representation and may have a second lower value in the central portion. The second exemplary sampling pattern 704 shows a radial sampling pattern. In the second exemplary sampling pattern 704, sampling is performed in k-space along radial lines originating from the center of the k-space representation. In such a k-space sampling pattern, as the sampling density decreases, the noise power may increase as a function of the radius (e.g., the distance from the center of the k-space representation). The third exemplary sampling pattern 706 shows a variable density spiral sampling pattern. In a third exemplary sampling pattern 06, sampling is performed in k-space along a variable density spiral, wherein sampling is performed along the variable density spiral. In such a k-space sampling pattern, as the sampling density decreases, the noise power may increase as a function of the spiral curvature and the distance from the spiral center. For each expected sampling pattern, the sampling density may be determined by calculation or analysis.

[0080] although Figure 6 to Figure 7 The k-space sampling pattern shown is a two-dimensional (2D) k-space sampling pattern, but in some embodiments, colored noise of a three-dimensional (3D) sampling method can be synthesized. Specifically, the k-space sampling pattern can be a 3D sampling pattern, but image denoising can be performed on a 2D image, which is generated by a 3D inverse Fourier transform of the 3D k-space data. It should be noted that image noise is random, so the result of merging independent noise samples can be calculated by adding the square of the noise, the noise power of each sample, and then taking the square root. Therefore, colored noise in the 2D k-space of a 2D image can be calculated from the 3D k-space noise by calculating the square root of the projection or line integral of the noise power through the 3D volume onto the 2D image plane. Therefore, colored noise can be synthesized for 2D medical images acquired via both 2D k-space sampling patterns and 3D k-space sampling patterns.

[0081] Now go to Figure 8 , showing the use of deep neural networks such as Figure 3300) to reduce colored noise in a medical image. The method 800 may be performed by one or more of the systems discussed above. In some embodiments, the method 800 may be performed by Figure 1 The system 10 shown or Figure 2 The method 800 may be implemented by the system 200 shown. In some embodiments, the method 800 may be implemented by the training module 112 stored in the non-transitory memory 106 of the image processing system 31 .

[0082] At operation 802, original training images are acquired. For example, the original images may be stored in a memory, or may be acquired via a wired or wireless connection. The training images may be original (e.g., raw) images, and may include medical images, natural images, synthetic images, etc. For example, the training images may be selected to effectively train a neural network to recognize noise in a particular type of medical image.

[0083] At operation 804, according to Figure 5 The method synthesizes colored noise of multiple expected k-space sampling patterns. Synthesizing the colored noise of the multiple expected k-space sampling patterns includes weighting the synthesized white noise according to the noise power of each expected k-space sampling pattern in the multiple expected k-space sampling patterns. For example, colored noise can be synthesized for each expected k-space sampling pattern in the multiple expected k-space sampling patterns. In addition, additional colored noise can be synthesized to account for other sources of colored noise, such as image post-processing, image reconstruction, etc.

[0084] At operation 806, the method includes creating a noisy training image using the colored noise synthesized at 804. For example, the noisy image may be generated by applying the synthesized colored noise to the original image. The noisy image and the corresponding original image comprise an image pair. The training data may include a plurality of image pairs, each image pair comprising an original image and a noisy image. In addition, each image pair in the training data may be indexed as data recording a k-space sampling pattern or other noise parameters for generating a noisy image. For example, each image pair may be indexed to data recording a k-space sampling pattern of colored noise used to synthesize a noisy image. As another example, each image pair may be indexed to data recording a k-space sampling pattern of colored noise used to synthesize a noisy image. The training data may be stored, for example, in a memory.

[0085] At operation 808, the deep neural network is trained using the training images generated at 806. The deep neural network may be trained using the training images according to any method known in the art. In one embodiment, training the deep neural network includes mapping the noisy image in each image pair in the training data to the corresponding original image. For example, for each image pair in the training data, the noisy image is input into the input layer of the deep neural network together with information about the k-space sampling pattern used to generate the noisy image, and is mapped to the predicted denoised image. In addition, the difference metric between the predicted denoised image and the corresponding original image (e.g., the ground truth image) is calculated by the deep neural network to determine the error of the predicted denoised image relative to the corresponding original image. In some embodiments, the difference metric may include one or more of a DICE score, a mean square error, an absolute distance error, and an angular error, or a weighted combination thereof. In some embodiments, training the deep neural network also includes determining the difference between each output of each output neuron from the deep neural network and the corresponding value in the ground truth image. In addition, the weights and biases of the deep neural network are adjusted based on the difference metric determined between the denoised image and the original image. The difference metric can be back-propagated through the layers of the deep neural network to update the weights (and biases) of the layers. In some embodiments, the back-propagation of the loss can occur according to a gradient descent algorithm or another back-propagation method known in the art. In addition, in some embodiments, different sets of weights and biases can be adjusted based on the synthetic colored noise used to generate the noisy image in the image pair. For example, a deep neural network may include multiple sets of weights and biases, each set of weights and biases in the multiple sets of weights and biases corresponding to a specific sampling pattern and / or other noise parameters. In other embodiments, the deep neural network can be trained according to other methods known in the art.

[0086] After operation 808, method 800 may end. It should be noted that 808 may be repeated until the weights and biases of the deep neural network converge, a threshold difference metric is obtained (for the training data or on a separate validation data set), or the rate of change of the weights and / or biases of the deep neural network is below a threshold for each iteration. In this manner, method 800 enables a deep neural network to be trained to identify colored noise in medical images and produce denoised images with significantly less colored noise, thereby improving image resolution and diagnostic quality.

[0087] For example, a method for training a deep neural network to reduce colored noise in a medical image includes: selecting a first medical image without colored noise, synthesizing a colored noise image based on noise parameters, generating a second medical image by adding the synthesized colored noise to the first medical image, mapping the second medical image to a predicted denoised medical image via a deep neural network, determining a loss based on a difference between the predicted denoised image and the first medical image, and updating parameters of the deep neural network based on the loss. In the foregoing example, additionally or optionally, mapping the second medical image to the predicted denoised medical image via the deep neural network also includes indexing the noise parameters to the second medical image, and selecting pre-trained weights and biases for the deep neural network based on the noise parameters.

[0088] For another example, the medical image is a first medical image, and the method includes: training a convolutional neural network (CNN) to produce a trained CNN, using a second medical image including colored noise and an original medical image corresponding to the second medical image, wherein the original medical image does not contain colored noise, and wherein training the CNN includes: selecting the original medical image, synthesizing colored noise in an image space, generating a second medical image by adding the synthesized colored noise to the original medical image, mapping the second medical image to a predicted denoised medical image, determining a loss based on a difference between the predicted denoised medical image and the original medical image, and updating parameters of the CNN based on the loss.

[0089] Go to Fig. 9, shows an example of a noisy medical image and a corresponding denoised medical image generated according to an embodiment of the present disclosure. Specifically, the noisy medical image 902 includes an MR cross-sectional image of a human brain and includes one or more types of noise that reduce the clarity of the imaged anatomical region. The noisy medical image 902 may have been generated via a k-space sampling pattern (such as a PROPELLER sampling pattern) that includes an uneven sampling density and may contain colored noise. The medical image 904 without white noise includes the same anatomical region as shown in the noisy medical image 902, however, according to existing white noise removal methods (e.g., filtering, deep neural networks, etc.), the white original medical image 904 does not contain white noise. Although the medical image 904 without white noise has less noise than the noisy medical image 902, there are still visible noise areas in the medical image 904 without white noise. In contrast, according to the colored noise reduction methods disclosed herein (such as method 400), the medical image 906 without colored noise includes the same anatomical regions as those shown in the noisy medical image 902 and the medical image 904 without white noise, but without the colored noise. It can be seen that the medical image 906 without colored noise is clearer and has less noise than either of the noisy medical image 902 and the medical image 904 without white noise, which can enable more accurate analysis and / or diagnosis based on the imaged anatomical regions. For example, after colored noise reduction, the medical image can be viewed at a higher resolution or can be used for a more detailed analysis of the patient's condition.

[0090] Next, in Fig.10 , other examples of noisy medical images and corresponding denoised medical images generated according to embodiments of the present disclosure are shown. Specifically, the noisy medical image 1002 includes an MR cross-sectional image of a human abdomen, and includes one or more types of noise that reduce the clarity of the imaged anatomical region. The noisy medical image 1002 may have been generated via a k-space sampling pattern (such as a "star stacking" sampling pattern) that includes non-uniform sampling and may produce colored noise. In contrast, according to the colored noise reduction method disclosed herein (such as method 400), the noise-free medical image 1004 includes the same anatomical region as shown in the noisy medical image 902, but does not contain colored noise. It can be seen that the medical image 1004 without colored noise is clearer and less noisy than the noisy medical image 1002 and the medical image 904 without white noise, which can enable more accurate analysis and / or diagnosis based on the imaged anatomical region.

[0091] although Fig. 9 and Fig.10Two specific examples of anatomical regions imaged via MRI are provided that can be denoised using the systems and methods disclosed herein, but it should be understood that the present disclosure provides for denoising of substantially any medical image of any anatomical region. In one embodiment, a single deep neural network can be trained using training data pairs of substantially similar anatomical regions (these substantially similar anatomical regions are captured / acquired using a single medical imaging modality), and the deep neural network can be used to denoise noisy medical images of anatomical regions that are substantially similar to the training data pairs. In other embodiments, a single deep neural network can be trained using training data pairs of multiple different medical imaging modalities including different anatomical regions, thereby producing a more generalized deep neural network that can enable the use of a single deep neural network to denoise a wide range of medical images of various anatomical regions. In this way, colored noise in medical images can be significantly reduced via a deep neural network, thereby improving image resolution.

[0092] The technical effect of training a deep neural network to denoise images with synthetic colored noise based on a k-space sampling pattern is that the deep neural network can be enabled to accurately reduce colored noise in medical images with multiple types of colored noise.

[0093] One or more specific embodiments of the present disclosure are described above to provide a thorough understanding. These described embodiments are merely examples of systems and methods for selectively denoising medical images using deep neural networks. Those skilled in the art will appreciate that the specific details described in the embodiments may be modified in practice without departing from the essence of the present disclosure.

[0094] When introducing elements of various embodiments of the present disclosure, the words "one", "a kind of" and "the" are intended to mean that there are one or more elements in these elements. The terms "first", "second", etc. do not indicate any order, quantity or importance, but are used to distinguish one element from another. The terms "including", "comprising", and "having" are intended to be inclusive, and mean that additional elements may be present in addition to the listed elements. As the terms "connected to", "coupled to", etc. are used herein, an object (e.g., a material, element, structure, member, etc.) can be connected to or coupled to another object, regardless of whether the one object is directly connected or coupled to the other object, or whether there are one or more intervening objects between the one object and the other object. In addition, it should be understood that reference to "one embodiment" or "embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features.

[0095] In addition to any previously indicated modifications, numerous other variations and alternative arrangements may be devised by those skilled in the art without departing from the spirit and scope of the present description, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described above in detail in conjunction with what are currently considered to be the most practical and preferred aspects, it will be apparent to those of ordinary skill in the art that many modifications may be made, including but not limited to form, function, mode of operation, and use, without departing from the principles and concepts set forth herein. Likewise, as used herein, the examples and embodiments are intended to be illustrative in all respects only and should not be construed as limiting in any way.

Claims

1. A method for reducing colored noise in medical images using a deep neural network, comprising: receiving a medical image acquired by an imaging system, wherein the medical image includes colored noise; Mapping the medical image to a denoised medical image without colored noise using a trained convolutional neural network; as well as Displaying the denoised medical image via a display device; The method further comprises training a convolutional neural network using a training medical image including colored noise and an original medical image corresponding to the training medical image to generate the trained convolutional neural network, wherein the training medical image is generated by adding synthesized colored noise to the original medical image, and wherein synthesizing the colored noise comprises: Selecting a k-space sampling pattern; Select k-space sampling density; generating a k-space noise power mask based on one or more of the k-space sampling pattern, a re-rasterization algorithm, and the k-space sampling density; Synthesize white noise images; Performing Fourier transformation on the white noise image to generate a white noise k-space; applying the k-space noise power mask to the white noise k-space to generate colored noise k-space; and The colored noise k-space is inversely Fourier transformed to generate synthetic colored noise.

2. The method of claim 1, wherein mapping the medical image to the denoised medical image using the trained convolutional neural network further comprises: collecting one or more noise parameters corresponding to a source of the colored noise; and incorporating the one or more noise parameters into the trained convolutional neural network.

3. The method of claim 2, wherein incorporating the one or more noise parameters into the trained convolutional neural network comprises selecting a set of pre-trained weights and biases of the trained convolutional neural network based on the one or more noise parameters. 4 . The method according to claim 3 , wherein the one or more noise parameters include one or more of a k-space sampling pattern used to acquire the medical image and a k-space sampling density used to acquire the medical image. 5 . The method of claim 4 , wherein the k-space sampling pattern is one of a parallel line sampling pattern with enhanced reconstruction, a star stack sampling pattern, a ramp sampling pattern, a weighted average sampling pattern, and a variable density spiral sampling pattern.

6. The method of claim 1, wherein training the convolutional neural network comprises: mapping the training medical image to a predicted denoised medical image; determining a loss based on a difference between the predicted denoised medical image and the original medical image; as well as Parameters of the convolutional neural network are updated based on the loss.

7. The method of claim 6, wherein mapping the training medical image to the predicted denoised medical image comprises: indexing the k-space sampling pattern to the training medical image; as well as A set of weights and biases of the convolutional neural network are selected based on the k-space sampling pattern.

8. A method for training a deep neural network to reduce colored noise in a medical image, comprising: selecting a first medical image free of colored noise; synthesizing a colored noise image based on noise parameters; generating a second medical image by adding synthesized colored noise to the first medical image; mapping the second medical image to a predicted denoised medical image via the deep neural network; determining a loss based on a difference between the predicted denoised medical image and the first medical image; as well as updating parameters of the deep neural network based on the loss; Wherein synthesizing the colored noise image based on the noise parameter further comprises: selecting a k-space sampling pattern based on the noise parameter; Select k-space sampling density; generating a k-space noise power mask based on one or more of the k-space sampling pattern, a re-rasterization algorithm, and the k-space sampling density; Synthesize white noise images; generating a white noise k-space based on the white noise image; applying the k-space noise power mask to the white noise k-space to generate a colored noise k-space; generating the colored noise image based on the colored noise k-space; and indexing the k-space sampling pattern to the colored noise image.

9. The method of claim 8, wherein mapping the second medical image to the predicted denoised medical image via the deep neural network further comprises: indexing the noise parameter to the second medical image; as well as A set of weights and biases are selected for the deep neural network based on the noise parameters.

10. The method according to claim 9, further comprising: receiving a medical image acquired using an imaging system, the medical image comprising colored noise; mapping the medical image to an output by the deep neural network; as well as The output is used to generate a denoised medical image.

11. The method of claim 10, wherein receiving the medical image further comprises: Select one or more noise parameters; as well as The one or more noise parameters are incorporated into the trained deep neural network. 12 . The method of claim 11 , wherein the one or more noise parameters include one or more of a k-space sampling pattern used to acquire the medical image and a k-space sampling density used to acquire the medical image.

13. The method of claim 8, wherein the k-space sampling pattern is a three-dimensional k-space sampling pattern and the k-space noise power mask is a two-dimensional projection of the three-dimensional noise power in k-space.

14. An image processing system, comprising: A memory storing a deep neural network; Display devices; and a processor communicatively coupled to the memory and configured to: receiving a medical image acquired using a magnetic resonance imaging system, wherein the medical image includes colored noise; mapping the medical image to a denoised medical image using a trained deep neural network; as well as displaying the denoised medical image via the display device; The medical image is a first medical image, and the processor is further configured to train a deep neural network using an original medical image and a second medical image with colored noise to generate the trained deep neural network, wherein the second medical image is generated by adding synthesized colored noise to the original medical image, wherein synthesizing the colored noise comprises: Selecting a k-space sampling pattern; Select k-space sampling density; generating a k-space noise power mask based on one or more of the k-space sampling pattern, a re-rasterization algorithm, and the k-space sampling density; Synthesize white noise images; Performing Fourier transformation on the white noise image to generate a white noise k-space; applying the k-space noise power mask to the white noise k-space to generate colored noise k-space; and The colored noise k-space is inversely Fourier transformed to generate synthetic colored noise.

15. The system of claim 14, wherein the processor is further configured to: receiving one or more noise parameters associated with the colored noise; and The one or more noise parameters are incorporated into the trained deep neural network. 16 . The system of claim 15 , wherein the one or more noise parameters include one or more of a k-space sampling pattern used to acquire the medical image and a k-space sampling density used to acquire the medical image.

17. The system of claim 14, wherein the processor is configured to train the deep neural network using the original medical image and the second medical image with colored noise by: mapping the second medical image to a predicted denoised medical image; determining a loss based on a difference between the predicted denoised medical image and the original medical image; as well as Parameters of the deep neural network are updated based on the loss.

Images