Method and system for selectively removing streak artifacts and noise from images using deep neural networks

Abstract

The present invention is entitled "Methods and systems for selectively removing streak artifacts and noise from images using deep neural networks". Methods and systems for independently removing streak artifacts and noise from medical images using a trained deep neural network are provided. In one embodiment, streak artifacts and noise can be selectively and independently removed from a medical image by receiving a medical image including streak artifacts and noise, mapping the medical image to a streak residue and a noise residue using a trained deep neural network, subtracting the streak residue from the medical image to a first degree, and subtracting the noise residue from the medical image to a second degree to produce a denoised medical image, and displaying the denoised medical image via a display device.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to magnetic resonance imaging (MRI) and computed tomography (CT) imaging, and more particularly, to systems and methods for removing streak artifacts and noise from MRI and CT images using deep neural networks. Background Art

[0002] Medical imaging systems are often used to obtain internal physiological information of a subject (e.g., a patient). For example, medical imaging systems can be used to obtain images of a patient's skeletal structure, brain, heart, lungs, and various other features. Medical imaging systems can include magnetic resonance imaging (MRI) systems, computed tomography (CT) systems, x-ray systems, ultrasound systems, and various other imaging modalities.

[0003] One disadvantage associated with MRI systems and CT systems is the time-consuming nature of acquiring measurement data of an anatomical region of a patient and reconstructing a medical image of the anatomical region from the measurement data. Slow image acquisition and reconstruction speeds may cause patient discomfort because the patient may be required to remain within the imaging system for an unpleasantly long duration while the complete measurement data is acquired. Furthermore, slow image acquisition and reconstruction may be incompatible with some imaging applications, such as in real-time imaging, where image acquisition and reconstruction delays may result in poor temporal resolution.

[0004] One approach directed to improving image acquisition and reconstruction speed employs incomplete acquisition / sampling of measurement data (e.g., undersampled k-space sampling in MRI, or incomplete acquisition of x-ray projection data in CT). Although incomplete acquisition of measurement data may improve image acquisition speed, images reconstructed from incomplete measurement data may include imaging artifacts, such as streak artifacts, and may also exhibit a reduced signal-to-noise ratio (SNR). In addition to improving acquisition speed, some medical images may be reconstructed from incomplete measurement data if a portion of the measurement data is rejected or reweighted due to the presence of motion.

[0005] One approach to reducing or removing imaging artifacts in medical images reconstructed from incomplete measurement data utilizes sophisticated reconstruction techniques to produce artifact-free images. An example of such an approach is compressed sensing (CS). However, CS reconstruction is computationally intensive, and optimization of reconstruction parameters is time consuming. Therefore, CS can reduce imaging artifacts in medical images reconstructed from incomplete measurement data at the expense of a longer and more complex reconstruction process. In many cases, the increase in image reconstruction time introduced by CS may invalidate the time gained by incomplete sampling of the measurement data. Therefore, it is often desirable to explore new methods for more quickly removing imaging artifacts from medical images reconstructed using incomplete measurement data. Summary of the invention

[0006] The inventors herein have identified systems and methods for selectively and independently removing streak artifacts and noise from medical images using deep neural networks in a manner that is faster and computationally efficient than conventional methods. In one embodiment, streak artifacts and noise may be removed from a medical image by a method comprising the steps of receiving a medical image comprising streak artifacts and noise, mapping the medical image to a streak residual and a noise residual using a trained deep neural network, subtracting the streak residual from the medical image to a first degree, and subtracting the noise residual from the medical image to a second degree to produce a denoised medical image, and displaying the denoised medical image via a display device. By mapping the medical image to the streak residual and the noise residual, rather than mapping the medical image directly to the denoised medical image, the degree of streak removal and the degree of noise removal may be selected independently of each other, thereby providing greater flexibility and control over the appearance of the denoised medical image.

[0007] The above advantages and other advantages and features of the present specification will be apparent from the following detailed description when taken alone or in conjunction with the accompanying drawings. It should be understood that the above summary is provided to introduce a selection of concepts further described in the detailed description in a simplified form. This is not meant to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims that follow the detailed description. In addition, the claimed subject matter is not limited to embodiments that address any disadvantages mentioned above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Various aspects of the present disclosure may be better understood by reading the following detailed description and referring to the accompanying drawings, in which:

[0009] Figure 1 A block diagram of an exemplary embodiment of an MRI system is shown;

[0010] Figure 2 An image processing apparatus useful for selectively removing streak artifacts and noise from medical images is shown;

[0011] Figure 3 An example of a system employing a trained deep neural network to infer streak residuals and noise residuals of a streak noisy image is shown;

[0012] Figure 4 is a flow chart illustrating an exemplary method for selectively removing streak artifacts and noise from a medical image using a trained deep neural network;

[0013] Figure 5is a flow chart illustrating an exemplary method for generating training data triplets that can be used to train a deep neural network to infer streak residuals and noise residuals of medical images;

[0014] Figure 6 is a flow chart illustrating an exemplary method for training a deep neural network to infer streak residuals and noise residuals of medical images using training data triplets;

[0015] Figure 7 An exemplary process for generating streak residuals from CT measurement data is shown;

[0016] Figure 8 An exemplary process for generating streak residuals from MRI measurement data is shown;

[0017] Fig. 9 shows a comparison between a first fringe residue generated without contrast enhancement and a second fringe residue generated with contrast enhancement;

[0018] Fig.10 showing a comparison between a first denoised medical image denoised using a deep neural network trained on training data generated without contrast enhancement and a second denoised medical image denoised using a deep neural network trained on training data generated with contrast enhancement; and

[0019] Fig.11 A medical image including streak artifacts and noise, and a denoised medical image generated according to an exemplary embodiment of the present disclosure are shown.

[0020] The accompanying drawings illustrate specific aspects of the described systems and methods for selectively removing streak artifacts and noise from MRI and CT images using deep neural networks. Together with the following description, the accompanying drawings illustrate and explain the structures, methods, and principles described herein. In the accompanying drawings, the sizes of components may be exaggerated or otherwise modified for clarity. Well-known structures, materials, or operations are not shown or described in detail to avoid obscuring various aspects of the described components, systems, and methods. DETAILED DESCRIPTION

[0021] The following description relates to systems and methods for automatically and selectively removing streak artifacts and noise from images. Certain magnetic resonance imaging (MRI) sequences and computed tomography (CT) imaging protocols may be prone to noise and streak artifacts, which limit image resolution and reduce diagnostic quality. One source of streak artifacts is undersampling in k-space, which may be caused by non-Cartesian sampling modes (e.g., radial sampling modes, spiral sampling modes, etc.) used to achieve shorter scan times or mitigate the effects of motion-induced blur. In another example, k-space may also be undersampled if a portion of the measurement data is rejected or reweighted due to the presence of motion. In one example, in periodically rotated overlapping parallel lines with enhanced reconstruction (PROPELLER) imaging, if there is a large amount of motion-induced blur in the acquired k-space leaf, the leaf can be rejected to reduce motion artifacts, resulting in undersampled k-space. In another example, for low-dose CT imaging, sparse views or limited angles are acquired to reduce radiation exposure of the imaging subject, but the CT image reconstructed by the sparse or limited angles may be degraded by noise and streak artifacts. Rapid scanning and / or undersampling k-space may also result in reduced signal-to-noise ratio (SNR) and poor image quality. Conventional methods for mitigating streak artifacts and noise in images produced by undersampled measurement data (e.g., k-space or sinogram data) employ complex reconstruction techniques to produce artifact-free images from incomplete measurement data. One example is compressed sensing (CS). However, CS reconstruction is computationally intensive and optimization of reconstruction parameters is challenging.

[0022] In addition, conventional methods do not provide the ability to separately / independently control the degree of stripe artifact removal and the degree of noise removal, thereby providing the user with little control over the appearance of the denoised medical image. In one example, the conventional method may include training a deep neural network using a noisy medical image pair (e.g., a medical image that includes both stripe artifacts and noise) and a corresponding original medical image (e.g., the same medical image without stripe artifacts and noise). Therefore, the deep neural network trained according to the conventional method can learn a mapping from the noisy image space to the original / denoised image space. The deep neural networks generated according to conventional image denoising training schemes (such as the above schemes) do not enable the user to independently control the degree of removal of different types of artifacts, nor do they allow for variable removal of image artifacts. Therefore, the denoised medical images generated according to the conventional methods provide the user with little or no ability to customize / adjust the degree of removal of one or more types of noise present in the denoised medical image. In one example, in the case where the deep neural network incorrectly identifies a portion of the imaged anatomical region as noise / artifacts and thus removes / changes the portion, the user may not be able to adjust the degree of removal / change of the portion.

[0023] The present disclosure provides systems and methods that at least partially solve one or more of the above problems. In one embodiment, Figure 1 The MRI system 10 acquires an MRI image of the anatomical region of the patient and can be transmitted to Figure 2 The image processing device 202 is shown. The MRI image may include streak artifacts and noise. In some embodiments, the MRI system 10 may acquire the MRI image using a radial k-space undersampling mode, wherein the k-space is undersampled in order to reduce the duration of image acquisition. In some embodiments, the MRI system 10 may include the image processing device 202, or the image processing device 202 may be located outside the MRI system and may be communicatively coupled to the MRI system 10. The image processing device 202 may include a non-transitory memory 206, which includes a trained deep neural network, such as Figure 3 The deep neural network 324 is shown. The deep neural network 324 may be configured to map the received stripe noise medical image (such as the stripe noise image 304) to a stripe residual and a noise residual (such as the stripe residual 306 and the noise residual 308), respectively. In some embodiments, the deep neural network 324 may also be configured to receive one or more acquisition parameters as input at an input layer of the deep neural network 324, such as a k-space sampling pattern used to acquire the stripe noise medical image 304, thereby providing the deep neural network 324 with additional context information about the stripe noise medical image 304.

[0024] The image processing device 202 may also include instructions stored in the non-transitory memory 206, which when executed cause the processor 204 to perform Figure 4 One or more of the operations of the method 400 shown in the figure can selectively remove the streak residue and / or the noise residue output by the deep neural network from the MRI image based on the first weight and the second weight, respectively. In addition, the image processing device 202 may include instructions for further adjusting the degree of removal of the streak residue and the degree of removal of the noise residue based on the user input received via the user input device. Fig.11 A comparison between a striped noisy medical image 1102 and a denoised medical image 1104 is shown, wherein the denoised medical image 1104 is produced by mapping the striped noisy medical image 1102 to a stripe residual and a noise residual, and selectively subtracting the stripe residual and the noise residual from the striped noisy image to produce the denoised medical image 1104 according to one or more of the operations of method 400.

[0025] The present disclosure also provides systems and methods for training a deep neural network to map a stripe noise image to a stripe residue and a noise residue, wherein the deep neural network thus trained can be used in one or more of the methods described herein for removing stripe artifacts and noise. Figure 5 One or more of the operations of the method 500 shown in the figure may be used to generate training data triplets for training a deep neural network. In one example, the imaging processing device 202 may perform one or more of the operations of the method 500 to generate a plurality of training data triplets, wherein the plurality of training data triplets may be used in conjunction with a training method such as the method 600 to teach the deep neural network a mapping from an image space to a streak residue space and a noise space. In one embodiment, the trained deep neural network may include a plurality of parameters that enable the trained deep neural network to identify and extract streak residues and noise residues from imaging data. The parameters may be determined by executing one or more supervised training routines such as the training method 600 (e.g., Figure 6 The training method 600 includes: feeding a deep neural network training data triple as input data, the deep neural network training data triple including a stripe noise image and a corresponding true stripe residual and true noise residual; predicting the stripe residual and the noise residual based on the input data; and comparing the predicted stripe residual and the predicted noise residual with the true stripe residual and the true noise residual. The parameters of the deep neural network can then be adjusted / updated based on the comparison to make the deep neural network prediction closer to the true output.

[0026] Figure 7 An exemplary process by which streak residuals may be generated from CT measurement data (sinogram data) is shown, and Figure 8 An exemplary process by which streak residuals can be generated from MRI measurement data (k-space data) is shown. Fig. 9 A comparison between streak residues produced by contrast-enhanced images and non-contrast-enhanced medical images is shown, thereby illustrating the effect of contrast enhancement on streak residue generation. Fig.10 A comparison is shown between a first denoised medical image 1002 produced by a deep neural network trained on training data triplets generated without contrast enhancement and a second denoised medical image 1004 produced by a deep neural network trained on training data triplets generated with contrast enhancement.

[0027] As used herein, the term "denoised medical image" may refer to an image that is free of streak artifacts and noise, or to an image that has a significantly reduced intensity of streak artifacts and noise relative to an image that was originally acquired and reconstructed. In some cases, the term "partially denoised image" may be used to refer to an image that includes a relatively lower intensity or amount of streak artifacts and noise compared to a corresponding unprocessed image. As used herein, the term "striped noise image" will be understood to refer to an image that contains both streak artifacts and noise.

[0028] First turn Figure 1 , an MRI system 10 is shown. The MRI system 10 includes a static magnetic field magnet unit 12, a gradient coil unit 13, a local RF coil unit 14, an RF body or volume coil unit 15, a transmit / receive (T / R) switch 20, an RF driver unit 22, a gradient coil driver unit 23, a data acquisition unit 24, a controller unit 25, a patient table or bed 26, an image processing system 31, a user input device 32, and a display device 33. The RF coil is used to transmit RF excitation signals ("transmit coil") and receive MR signals emitted by an imaging subject ("receive coil"). In some embodiments, the RF coil unit 14 is a surface coil, which is a local coil that is generally placed near an anatomical structure of interest of the subject 16. Here, the RF body coil unit 15 is a transmit coil that transmits RF signals, and the local RF coil unit 14 receives MR signals. Therefore, the transmit body coil (e.g., the RF body coil unit 15) and the surface receive coil (e.g., the RF coil unit 14) are independent but electromagnetically coupled components. The MRI system 10 transmits electromagnetic pulse signals to a subject 16 placed in an imaging space 18, wherein a static magnetic field is formed to perform scanning to obtain magnetic resonance signals from the subject 16. One or more MR images of the subject 16 may be reconstructed based on the magnetic resonance signals thus obtained by the scanning.

[0029] The static magnetic field magnet unit 12 includes, for example, an annular superconducting magnet installed in an annular vacuum container. The magnet defines a cylindrical space around the subject 16 and generates a constant main static magnetic field B0.

[0030] The MRI system 10 further includes a gradient coil unit 13, which forms a gradient magnetic field in an imaging space 18 so as to provide three-dimensional position information for a magnetic resonance signal received by an RF coil array. The gradient coil unit 13 includes three gradient coil systems, each of which generates a gradient magnetic field along one of three spatial axes perpendicular to each other, and generates a gradient field in each of a frequency encoding direction, a phase encoding direction, and a slice selection direction according to an imaging condition. More specifically, the gradient coil unit 13 applies a gradient field in a slice selection direction (or a scanning direction) of the subject 16 to select a slice; and the RF body coil unit 15 or the local RF coil array can transmit an RF pulse to the selected slice of the subject 16. The gradient coil unit 13 also applies a gradient field in a phase encoding direction of the subject 16 to phase encode a magnetic resonance signal from a slice excited by an RF pulse. The gradient coil unit 13 also applies a gradient field in a frequency encoding direction of the subject 16 to frequency encode a magnetic resonance signal from a slice excited by an RF pulse.

[0031] The RF coil unit 14 is provided, for example, to surround a region to be imaged of the subject 16. In some examples, the RF coil unit 14 may be a receiving coil. In a static magnetic field space or an imaging space 18 in which a static magnetic field B0 is formed by the static magnetic field magnet unit 12, the RF coil unit 15 transmits RF pulses as electromagnetic waves to the subject 16 based on a control signal from the controller unit 25, and thereby generates a high-frequency magnetic field B1. This excites the proton spins in the slice to be imaged of the subject 16. The RF coil unit 14 receives the electromagnetic waves generated when the proton spins thus excited in the slice to be imaged of the subject 16 return to alignment with the initial magnetization as magnetic resonance signals. In some embodiments, the RF coil unit 14 may transmit RF pulses and receive MR signals. In other embodiments, the RF coil unit 14 may be used only to receive MR signals, and not to transmit RF pulses.

[0032] The RF body coil unit 15 is arranged, for example, to surround an imaging space 18, and generates RF magnetic field pulses orthogonal to the main magnetic field B0 generated by the static magnetic field magnet unit 12 in the imaging space 18 to excite the nucleus. The RF coil unit 14 can be disconnected from the MRI system 10 and replaced with another RF coil unit, while the RF body coil unit 15 is fixedly attached and connected to the MRI system 10. In addition, although local coils such as the RF coil unit 14 can transmit or receive signals only from a local area of ​​the subject 16, the RF body coil unit 15 generally has a larger coverage area. For example, the RF body coil unit 15 can be used to transmit or receive signals to the whole body of the subject 16. The use of a receive-only local coil and a transmission body coil provides uniform RF excitation and good image uniformity, at the expense of higher RF power deposited in the subject. For a transmit-receive local coil, the local coil provides RF excitation to the region of interest and receives MR signals, thereby reducing the RF power deposited in the subject. It should be understood that the specific use of the RF coil unit 14 and / or the RF body coil unit 15 depends on the imaging application.

[0033] When operating in the receive mode, the T / R switch 20 may selectively electrically connect the RF body coil unit 15 to the data acquisition unit 24, and when operating in the transmit mode, the T / R switch 20 may selectively electrically connect the RF body coil unit 15 to the RF driver unit 22. Similarly, when the RF coil unit 14 operates in the receive mode, the T / R switch 20 may selectively electrically connect the RF coil unit 14 to the data acquisition unit 24, and when operating in the transmit mode, the T / R switch 20 may selectively electrically connect the RF coil unit 14 to the RF driver unit 22. When both the RF coil unit 14 and the RF body coil unit 15 are used for a single scan, for example, if the RF coil unit 14 is configured to receive MR signals and the RF body coil unit 15 is configured to transmit RF signals, the T / R switch 20 may direct the control signal from the RF driver unit 22 to the RF body coil unit 15 while directing the received MR signal from the RF coil unit 14 to the data acquisition unit 24. The coils of the RF body coil unit 15 may be configured to operate in a transmit-only mode or a transmit-receive mode. The coils of the local RF coil unit 14 may be configured to operate in a transmit-receive mode or a receive-only mode.

[0034] The RF driver unit 22 includes a gate modulator (not shown), an RF power amplifier (not shown), and an RF oscillator (not shown), which are used to drive an RF coil (e.g., the RF coil unit 15) and form a high-frequency magnetic field in the imaging space 18. The RF driver unit 22 modulates the RF signal received from the RF oscillator into a signal of a predetermined timing with a predetermined envelope based on a control signal from the controller unit 25 and using the gate modulator. The RF signal modulated by the gate modulator is amplified by the RF power amplifier and then output to the RF coil unit 15.

[0035] The gradient coil driver unit 23 drives the gradient coil unit 13 based on a control signal from the controller unit 25, thereby generating a gradient magnetic field in the imaging space 18. The gradient coil driver unit 23 includes three driver circuit systems (not shown) corresponding to the three gradient coil systems included in the gradient coil unit 13.

[0036] The data acquisition unit 24 includes a preamplifier (not shown), a phase detector (not shown), and an analog / digital converter (not shown) for acquiring a magnetic resonance signal received by the RF coil unit 14. In the data acquisition unit 24, the phase detector phase uses the output of the RF oscillator from the RF driver unit 22 as a reference signal to detect the magnetic resonance signal received from the RF coil unit 14 and amplified by the preamplifier, and outputs the phase-detected analog magnetic resonance signal to the analog / digital converter to be converted into a digital signal. The digital signal thus obtained is output to the image processing system 31.

[0037] The MRI system 10 includes a couch 26 for placing the subject 16 thereon. By moving the couch 26 based on a control signal from the controller unit 25, the subject 16 can be moved inside and outside the imaging space 18.

[0038] The controller unit 25 includes a computer and a recording medium on which a program to be executed by the computer is recorded. The program, when executed by the computer, causes the various parts of the system to perform operations corresponding to a predetermined scan. The recording medium may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-transitory memory card. The controller unit 25 is connected to the user input device 32 and processes an operation signal input to the user input device 32, and also controls the examination bed 26, the RF driver unit 22, the gradient coil driver unit 23, and the data acquisition unit 24 by outputting control signals to them. The controller unit 25 also controls the image processing system 31 and the display device 33 based on the operation signal received from the user input device 32 to obtain a desired image.

[0039] The user input device 32 includes user input devices such as a touch screen, a keyboard, and a mouse. The MRI system operator uses the user input device 32, for example, to input such data as an imaging protocol, accept or reject a scan area preview, and in some embodiments set the area to be executed for an imaging sequence. The imaging protocol data, the scan area preview acceptance or rejection, and the imaging sequence execution area are output to the controller unit 25.

[0040] The image processing system 31 includes a processor and a non-transitory memory on which machine executable instructions may be stored, wherein the machine executable instructions may enable the processor to perform one or more of the steps of one or more of the methods disclosed herein. The image processing system 31 may be connected to the controller unit 25, and may perform data processing based on control signals received from the controller unit 25 or the user input device 32. The image processing system 31 is also connected to the data acquisition unit 24, and generates spectral data by applying various image processing operations to the magnetic resonance signals output from the data acquisition unit 24.

[0041] The MRI system 10 may acquire diagnostic images according to one or more imaging protocols. In some embodiments, the imaging protocol may indicate one or more k-space sampling modes and / or one or more k-space sampling densities for acquiring k-space data, wherein the k-space data may be reconstructed according to one or more image reconstruction methods known in the art to form a medical image. In one embodiment, the MRI system 10 may include instructions stored in a non-volatile memory that, when executed, cause the MRI system 10 to acquire k-space data according to one or more predetermined k-space sampling modes. In some embodiments, the MRI system 10 may be configured to acquire measurement data / k-space data by executing a PROPELLER imaging protocol, wherein one or more leaves of k-space centered on the k-space origin may be acquired. In some embodiments, the MRI system 10 may be configured to acquire MRI measurement data using a star stack imaging protocol. In addition, in some embodiments, the MRI system 10 may be configured to acquire measurement data using a reduced k-space sampling density (also referred to as an undersampling mode), wherein at least a portion of the k-space is sampled at a reduced density, thereby reducing acquisition time.

[0042] The display device 33 can display an image on a display screen of the display device 33 based on a control signal received from the controller unit 25. The display device 33 displays (for example) a denoised medical image. The display device 33 may include a graphical user interface (GUI), wherein a user may interact / input / change the one or more data fields via the user input device 32. In one embodiment, the display device 33 may display a GUI comprising an input field / slide bar configured to enable a user to adjust a first degree of removing streak residues and a second degree of removing noise residues from the acquired MRI image. The display device 33 may display a two-dimensional (2D) image, a three-dimensional (3D) image and / or a four-dimensional image (3D image in time) of the subject 16 generated by the image processing system 31.

[0043] During scanning, the RF coil array crossover cable ( Figure 1 ) can be used to transmit signals between RF coils (e.g., the RF coil unit 14 and the RF body coil unit 15) and other aspects of the processing system (e.g., the data acquisition unit 24, the controller unit 25, etc.), for example to control the RF coils and / or receive information from the RF coils.

[0044] See now Figure 2 , which shows an image processing system 200 according to an exemplary embodiment. In some embodiments, the image processing system 200 is incorporated into the MRI system 10. In some embodiments, at least a portion of the image processing system 200 is disposed at a device (e.g., an edge device, a server, etc.) that is communicatively coupled to the MRI system 10 via a wired connection and / or a wireless connection. In some embodiments, at least a portion of the image processing system 200 is disposed at a device (e.g., a workstation) located away from the MRI system 10, which is configured to receive images from the MRI system 10 or from a storage device configured to store images acquired by the MRI system 10. The image processing system 200 may include an image processing device 202, a user input device 230, and a display device 220. In some embodiments, the image processing system 200 may receive a CT image from a CT imaging system, wherein the image processing system 200 may be communicatively coupled to one or more CT imaging systems via a wired or wireless connection. In some embodiments, the image processing device 202 may be communicatively coupled to both the MRI imaging system and the CT imaging system, and may be configured to receive and process both the MRI image and the CT image. In some embodiments, the image processing device 202 may be communicatively coupled to a picture archiving and communication system (PACS) and may receive images from and / or send images to the PACS.

[0045] Image processing device 202 includes processor 204, and this processor is configured to execute the machine-readable instruction stored in non-transient memory 206.Processor 204 can be single-core or multi-core, and the program executed thereon can be configured to carry out parallel or distributed processing.In some embodiments, processor 204 can optionally include separate components distributed in two or more devices, and it can be remotely located and / or configured for coordinated processing.In some embodiments, one or more aspects of processor 204 can be virtualized and executed by a networked computing device capable of remote access configured with a cloud computing configuration.

[0046] The non-transitory memory 206 may store a deep neural network module 208, a training module 212, and image data 214. The deep neural network module 208 may include one or more deep neural networks including a plurality of weights and biases, activation functions, pooling functions, and instructions for implementing the one or more deep neural networks to identify and extract features within a medical image of interest and map the extracted features to streak residuals and noise residuals corresponding to streak artifacts and noise within the medical image of interest, respectively. In some embodiments, the deep neural network module 208 may include one or more trained deep neural networks, such as the deep neural network 324, and the trained deep neural network may be implemented according to one or more operations of the method 400 to selectively remove streak artifacts and noise based on the identified features in the image.

[0047] The deep neural network module 208 may include trained and / or untrained deep neural networks. In some embodiments, the deep neural network module 208 is not disposed at the image processing device 202, but is disposed at a remote device that is communicatively coupled to the image processing device 202 via a wired or wireless connection. The deep neural network module 208 may include various deep neural network metadata related to the trained and / or untrained networks. In some embodiments, the deep neural network metadata may include an indication of the training data used to train the deep neural network, the training method used to train the deep neural network, and the accuracy / validation score of the trained deep neural network. In some embodiments, the deep neural network module 208 may include metadata for the trained deep neural network, which indicates the type of anatomical structure and / or the type of imaging modality to which the trained deep neural network can be applied.

[0048] The non-volatile memory 206 also includes a training module 212, which includes machine executable instructions for training one or more deep neural networks in the deep neural network stored in the deep neural network module 208. In some embodiments, the training module 212 may include instructions for: generating training data triples by performing one or more operations of method 500, and using the training data triples to train the deep neural network to identify streak artifacts and noise within the medical image according to one or more operations of method 600, and mapping the streak artifacts and noise to streak residuals and noise residuals, respectively. In one embodiment, the training module 212 may include a gradient descent algorithm, a loss function, and machine executable rules for generating and / or selecting training data for training the deep neural network. The training module 212 may also include instructions that, when executed by the processor 204, cause the image processing device 102 to perform the following reference to the method 600. Figure 6 One or more of the operations of the method 600 discussed in more detail are used to train a deep neural network. In some embodiments, the training module 212 is not located at the image processing device 202, but is remotely located and communicatively coupled to the image processing device 202.

[0049] The non-transitory memory 206 may also include an image data module 214 that includes images / imaging data acquired by one or more imaging devices such as the MRI system 10. The images stored in the image data 214 may include medical images from various imaging modalities or from various medical imaging device configurations / models, and may include images of various views of anatomical regions of one or more patients. In some embodiments, the medical images stored in the image data module 214 may include information identifying the imaging modality and / or imaging device (e.g., the model and manufacturer of the imaging device) that acquired the medical image. In some embodiments, the images stored in the image data module 214 may include metadata indicating one or more acquisition parameters used to acquire the image. In one example, the metadata of the image may be stored in a DICOM header of the image. In some embodiments, the metadata may include a k-space sampling mode and a k-space sampling density used to acquire the image. In some embodiments, the metadata may indicate a plurality of projections acquired in a CT imaging protocol, and may also indicate an acquisition angle for each of the projections. In some embodiments, the image data module 214 may include x-ray images acquired by an x-ray device, MR images captured by an MRI system, CT images captured by a CT imaging system, PET images captured by a PET system, and / or one or more additional types of medical images.

[0050] In some embodiments, non-transitory memory 206 may include components disposed on two or more devices that may be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible networked storage device configured in a cloud computing configuration.

[0051] The image processing system 200 may also include a user input device 230. The user input device 230 may include one or more of a touch screen, a keyboard, a mouse, a touch pad, a motion sensing camera, or other devices configured to enable a user to interact with and manipulate data within the image processing system 200. In some embodiments, the user input device 230 enables a user to adjust the degree of streak residue removal and the degree of noise residue removal. In some embodiments, a user may input or select a first value indicating the degree of streak residue removal, a user may further input or select a second value indicating the degree of noise residue removal, and the image processing device 202 may independently adjust the degree of streak residue removal and the degree of noise residue removal in response to receiving the user input and based on the user input to produce a partially de-noised medical image.

[0052] The display device 220 may include one or more display devices utilizing almost any type of technology. In some embodiments, the display device 220 may include a computer monitor. The display device 220 may be configured to receive data from the image processing device 202 and display a denoised, partially denoised or non-denoised medical image based on the received data. The display device 220 may be combined in a shared housing with the processor 204, the non-transient memory 206 and / or the user input device 230, or may be a peripheral display device, and may include a monitor, a touch screen, a projector or other display devices known in the art, which may enable a user to view images, and / or interact with various data stored in the non-transient memory 206.

[0053] It should be understood that Figure 2 The image processing system 200 shown is for illustration and not limitation. Another suitable image processing system may include more, fewer, or different components.

[0054] Go to Figure 3, a block diagram of a system 300 for mapping a stripe noise image to a denoised medical image using a deep neural network is shown. The system 300 may be implemented by an image processing device (such as the image processing device 202) or other appropriately configured computing system. The system 300 shows an exemplary stripe noise image 304 mapped to an exemplary stripe residual 306 and an exemplary noise residual 308 via a deep neural network 324. The stripe residual 306 and the noise residual 308 are then weighted (via W1 and W2, respectively) and subtracted from the stripe noise image 304 to produce a denoised image 312. As used herein, the term "stripe noise" may refer to an image that includes both stripe artifacts and noise.

[0055] The stripe noise image 304 may include a plurality of pixel / voxel intensity values ​​and may be a 2D or 3D image. The stripe noise image 304 may include an MRI image or a CT image. In some embodiments, the stripe noise image 304 may include an MRI image acquired via undersampling of k-space (such as via a radial sampling pattern). In some embodiments, the stripe noise image 304 may include a CT image acquired via undersampled sinogram data.

[0056] The acquisition parameters 302 may include one or more pieces of background data related to the streak noise image 304. In some embodiments, the acquisition parameters 302 include a k-space sampling pattern used to acquire the streak noise image 304. In some embodiments, the acquisition parameters 302 include a k-space sampling density used to acquire the streak noise image 304. In some embodiments, the acquisition parameters 302 may include an indication of the number and orientation of x-ray projections used to construct the streak noise image 304.

[0057] The deep neural network 324 may include an encoding portion (e.g., encoder 314) and a decoding portion (decoder 318), wherein the encoder 314 is configured to identify and extract features from an input image such as the striped noise image 304, and wherein the decoder 318 is configured to map the extracted features output by the encoder 314 to corresponding striped residuals (e.g., striped residuals 306) and corresponding noise residuals (e.g., noise residuals 308). In some embodiments, separate deep neural networks may be used to generate the striped residuals and the noise residuals.

[0058] The deep neural network 324 may be configured to receive data from the striped noise image 304 via an input layer, and may optionally receive acquisition parameters 302. In some embodiments, the input layer may include a first plurality of nodes / neurons configured to receive pixel / voxel intensity data from the striped noise image 304. Optionally, the input layer may include a second plurality of nodes / neurons configured to receive acquisition parameters 302. In some embodiments, the acquisition parameters 302 may be concatenated with or embedded in the striped noise image 304, and both the striped noise image 304 and the acquisition parameters 302 may be received by the first plurality of nodes / neurons. The data received by the input layer may be passed to the encoder 314.

[0059] The encoder 314 may include a first plurality of layers / feature maps configured to identify and extract features embedded within the streak noise image 304. Each feature map receives an input from a file or a previous feature map and transforms / maps the received input to an output to produce the next feature map. In some embodiments, the transformation may include convolution using learned filters, pooling, activation functions (including rectified linear units), etc. Each feature map may include a plurality of neurons, wherein in some embodiments, each neuron may receive input from a subset of neurons in a previous layer / feature map and may compute a single output based on the received input, wherein the output may be propagated to a subset of neurons in the next layer / feature map. In some embodiments, the neurons of the feature map may compute outputs by performing a dot product of the received input using a set of learned weights (each set of learned weights may be referred to herein as a filter), wherein each received input has a unique corresponding learned weight, wherein the learned weight is learned during a training phase. The weights (and biases) of the deep neural network 324 may be learned during training, as will be described below with reference to Figure 5 and Figure 6 The output from the encoder 314 , which may include a plurality of identified and extracted features, is passed to the decoder 318 .

[0060] The decoder 318 may include a second plurality of layers / feature maps similar to the encoder 314. Each feature map may receive input from a previous feature map (or from the encoder 314) and may transform / map the received input to an output to produce the next feature map. In some embodiments, the transformation may include up-convolution using a learned deconvolution filter, an activation function (including a rectified linear unit), etc. Each feature map may include a plurality of neurons, wherein in some embodiments, each neuron may receive input from a subset of neurons of a previous layer / feature map and may compute a single output based on the received input, wherein the output may be propagated to a subset of neurons in the next layer / feature map. In some embodiments, the neurons of the feature map may compute outputs by performing a dot product of the received input using a set of learned weights (each set of learned weights may be referred to herein as a filter), wherein each received input has a unique corresponding learned weight, wherein the learned weight is learned during a training phase. The decoder 318 may also include one or more fully connected layers, wherein each node of the previous layer is connected to each node of the current layer.

[0061] The deep neural network 324 may also include an output layer configured to output both a streak residual (such as the streak residual 306) and a noise residual (such as the noise residual 308). In some embodiments, the output layer includes a first plurality of neurons configured to generate the streak residual 306 based on the input received from the decoder 318, and a second plurality of neurons configured to generate the noise residual 308 based on the input received from the decoder 318. Each neuron of the output layer may correspond to a pixel / voxel of the streak residual 306 or the noise residual 308. The dimensions of the streak residual 306 and the noise residual 308 may match the dimensions of the streak noise image 304. For example, the output of the neurons of the output layer may indicate the streak artifact intensity (or noise intensity) in the corresponding region of the streak noise image 304. In other words, the streak residual 306 may include a mapping of the spatial distribution and intensity of the streak artifacts present in the streak noise image 304, and the noise residual 308 may include a mapping of the spatial distribution and intensity of the noise present in the streak noise image 304.

[0062] It should be understood that the architecture and configuration of the deep neural network 324 are for illustration and not limitation. Any suitable neural network (including U-Net, ResNet, recursive neural network, generalized regression neural network (GRNN), etc.) may be used herein to infer streak residual and noise residual MR and / or CT images. One or more specific embodiments of the present disclosure are described above to provide a thorough understanding. However, these described embodiments are merely examples of systems and methods for inferring streak residual and noise in images using deep neural networks alone. Those skilled in the art will appreciate that the specific details described in the embodiments may be modified in implementation without departing from the essence of the present disclosure.

[0063] The streak residue 306 and the noise residue 308 may be weighted, as indicated by W1 and W2, as Figure 3 As shown. In one embodiment, each pixel / voxel value of the stripe residue 306 may be multiplied by W1 (a first degree) to produce a first weighted stripe residue, and similarly, each pixel / voxel value of the noise residue 308 may be multiplied by W2 (a second degree) to produce a weighted noise residue. The weighted stripe residue and the weighted noise residue may then be subtracted from the stripe noise image 304 to remove the stripe artifacts and noise present in the stripe noise image 304 to a first degree and a second degree, respectively, thereby producing a denoised image 312 (as indicated by the circled closed ∑). Since the stripe residue 306 includes a mapping of the stripe artifacts present in the stripe noise image 304, and since the noise residue 308 includes the noise present in the stripe noise image 304, the system 300 is able to substantially or to a given degree remove the stripe artifacts and noise (e.g., 0% to 100% removal). Furthermore, the system 300 may independently select the degree of streak artifact removal and noise removal (eg, streak artifact removal may be set to 50% by setting W1 to 0.5, and noise removal may be set to 95% by setting W2 to 0.95).

[0064] Go to Figure 4 , a flowchart of an exemplary method 400 for removing stripe artifacts and noise from a stripe noise image using a trained deep neural network is shown. The method 400 may be performed by one or more of the systems discussed above. In one embodiment, the image processing device 202 implementing the system 300 may perform one or more operations of the method 400.

[0065] At operation 402, an image processing device receives a stripe noise image, such as stripe noise image 420. In some embodiments, the stripe noise image may include a 2D or 3D image of an anatomical region of a patient, including a 2D array of pixel intensity values ​​in one or more channels or a 3D array of voxel intensity values ​​in one or more channels. In some embodiments, the stripe noise image includes an MR image of an anatomical region of a patient acquired by an MRI system (such as MRI system 10). In some embodiments, the stripe noise image includes a computed tomography (CT) image of an anatomical region of a patient acquired by a CT imaging system. In some embodiments, the received stripe noise image may include one or more metadata, in which one or more acquisition parameters may be stored. In some embodiments, the metadata may be included in a DICOM header of the received stripe noise image. The image metadata may include one or more of a k-space sampling mode and / or a k-space sampling density used during acquisition of the image, a sinogram sampling mode and / or a sinogram sampling density, an indication of a reconstruction algorithm for reconstructing an image from measurement data, and the like.

[0066] At operation 404, the image processing device selects a trained deep neural network. In some embodiments, the image processing device may select the trained deep neural network from a plurality of trained deep neural networks based on one or more metadata related to the trained deep neural network and further based on one or more metadata included with the received medical image. In some embodiments, the image processing device may compare a first piece of metadata included in the received streak noise image with a plurality of metadata related to the deep neural network, and may select the trained deep neural network in response to the first piece of metadata matching one or more of the plurality of metadata related to the trained deep neural network. In some embodiments, the first piece of metadata may include an indication of an imaging protocol used to acquire the streak noise image, wherein the imaging protocol may indicate one or more acquisition parameters used by the imaging device to acquire the streak noise image.

[0067] In some embodiments, operation 404 includes: the image processing device accessing a location of a non-transitory memory in which an index of a plurality of trained deep neural networks is stored; and comparing one or more metadata associated with the stripe noise image with one or more index parameters of the plurality of deep neural networks. In some embodiments, the image processing device may select the deep neural network based on the type of training data used to train the deep neural network. In some embodiments, the deep neural network may be trained to identify stripe residues and noise residues in an image acquired using one or more predetermined acquisition parameters (e.g., k-space sampling density, k-space sampling pattern, sinogram sampling density, sinogram sampling pattern, echo sequence, etc.), and the deep neural network may be indexed based on the predetermined acquisition parameters. In such embodiments, operation 404 may include the image processing device comparing one or more image acquisition parameters stored in metadata associated with the image used to acquire the stripe noise image to be denoised with metadata associated with the trained deep neural network indicating acquisition parameters of the image used to train the deep neural network.

[0068] At operation 406, the image processing device maps the stripe noise image to a stripe residual (e.g., stripe residual 422) and a noise residual (e.g., noise residual 424) using the trained deep neural network. Optionally, operation 406 may include the image processing device receiving one or more acquisition parameters related to acquisition of the stripe noise image, inputting the stripe noise image and the one or more acquisition parameters into an input layer of the trained deep neural network, and mapping the medical image and the one or more acquisition parameters to the stripe residual and the noise residual using the trained deep neural network, as described above with reference to Figure 3 Described in more detail.

[0069] At operation 408, the image processing device removes stripe residues from the stripe noise image to a first extent, and removes noise residues from the stripe noise image to a second extent to produce a denoised image. In some embodiments, the first extent includes a first value (and any quantiles between) between 0 and 1 including the end value, and wherein the second extent includes a second value (and any quantiles between) between 0 and 1 including the end value. In other words, the intensity of the stripe residue and the intensity of the noise residue can be removed independently to different degrees within the range of 0% to 100%. In some embodiments, the user can pre-select a preferred first degree and a preferred second degree, and the first degree and the second degree can be stored in a location of a non-transitory memory associated with a user preference, in such embodiments, at operation 408, the image processing device can access the location in the non-transitory memory and retrieve the predetermined first degree and second degree. In some embodiments, if the user has not pre-selected a preferred first degree and a preferred second degree, the image processing device can access the default first degree and the default second degree at operation 408. In some embodiments, the image processing device may include instructions for intelligently selecting the first degree and the second degree based on an input streak noise image and one or more metadata associated therewith.

[0070] In some embodiments, the streak residual comprises a 2D or 3D array of intensity values ​​representing intensities of streak artifacts identified by the trained deep neural network, and operation 408 comprises multiplying the 2D or 3D array of streak intensity values ​​by a first weight factor (also referred to herein as W1) to produce a weighted streak residual comprising a plurality of weighted streak intensity values. The weighted streak residual may be subtracted from the streak noise image by performing a pixel-by-pixel / voxel-by-voxel subtraction of the weighted streak intensity values ​​of the weighted streak residual from the intensity values ​​of the streak noise image, for example, by subtracting the weighted streak intensity value of a first pixel of the weighted streak residual from the intensity value of a second pixel of the streak noise image, wherein the first pixel and the second pixel represent the same region of imaging space.

[0071] Likewise, in some embodiments, the noise residual comprises a 2D or 3D array of intensity values ​​representing noise intensities identified by the trained deep neural network, and operation 408 comprises multiplying the 2D or 3D array of noise intensity values ​​by a second weighting factor (also referred to herein as W2) to produce a weighted noise residual comprising a plurality of weighted noise intensity values. The weighted noise residual may be subtracted from the streaky noise image by performing a pixel-by-pixel / voxel-by-voxel subtraction of the weighted noise intensity values ​​of the weighted noise residual from the intensity values ​​of the streaky noise image, e.g., by subtracting the weighted noise intensity value of a first pixel of the weighted noise residual from the intensity value of a second pixel of the streaky noise image, wherein the first pixel and the second pixel represent the same region of imaging space.

[0072] In some embodiments, operation 408 includes the image processing device retrieving W1 and W2 from a predetermined location in a non-transitory memory. In some embodiments, the predetermined location in the non-transitory memory includes a user preference file configured by a user, wherein the user preference file may include W1 and W2 selected by the user. In some embodiments, W1 and W2 may be set to 1.0 and 1.0, respectively, indicating 100% removal of both streak artifacts and noise. In some embodiments, W1 and W2 may be independently set to values ​​other than 1.0. Go to temporarily Fig.11 , shows a comparison between an exemplary striped noise image 1102 and an exemplary denoised medical image. Fig.11 As can be seen in FIG. 1 , the streak noise image 1102 includes a plurality of streak artifacts (elongated regions of higher intensity that extend beyond portions of the anatomical structure).

[0073] At operation 410, the image processing apparatus displays the denoised medical image. An example of the denoised medical image 426 is shown in FIG. Figure 4 4. In some embodiments, operation 410 includes displaying an indication of the degree of removal of streaky residues and an indication of the degree of removal of noise. In some embodiments, operation 410 includes displaying a first degree W1 of removal of streaky residues, and displaying a second degree W2 of removal of noise residues, together with a GUI configured to receive user input for adjusting the first degree W1 and / or adjusting the second degree W2.

[0074] At operation 412, method 400 optionally includes the image processing device adjusting the first degree and the second degree based on the user input received via the user input device to produce the third degree and the fourth degree. In some embodiments, the user can use the user input device to input the third degree and the fourth degree into the image processing device, and at operation 412, the image processing device can replace the first degree with the third degree, and can replace the second degree with the fourth degree. In some embodiments, at operation 412, the image processing device can receive the first scaling factor and the second scaling factor, and can adjust the first degree to produce the third degree by multiplying the first degree by the first scaling factor, and similarly, the image processing device can adjust the second degree to produce the fourth degree by multiplying the second degree by the second scaling factor. It should be understood that operation 412 covers embodiments in which the user adjusts the first degree but does not adjust the second degree, and the user adjusts the second degree but does not adjust the first degree.

[0075] At operation 414, method 400 optionally includes the image processing device removing the stripe residual from the stripe noise image to a third degree, and removing the noise residual from the stripe noise image to a fourth degree to produce a second denoised medical image. In some embodiments, the user may select or input the third degree and the fourth degree via the user input device, and in response, the image processing device may generate a second weighted stripe residual by multiplying the stripe residual determined at operation 406 by the third degree, and may generate a second weighted noise residual by multiplying the noise residual determined at operation 406 by the fourth degree. The image processing device may subtract the second weighted stripe residual and the second weighted noise residual from the stripe noise image to produce a second denoised medical image. In some embodiments, if the user selects the third degree or the fourth degree to be less than 1.0, the second denoised medical image may include a portion of the intensity of the stripe artifact and / or a portion of the intensity of the noise, and may therefore include a partially denoised image.

[0076] At operation 416, method 400 optionally includes displaying the second de-noised medical image by the image processing device. An example of the second de-noised medical image 428 is shown in FIG. Figure 4 4. After operation 416, method 400 may end.

[0077] By generating separate outputs for streak residues and noise residues, the extent of streak residue removal and noise residue removal can be independently controlled, thereby achieving different degrees of removal of streak artifacts relative to noise, thereby providing the user with greater control over the appearance of the displayed image. Further, compared to conventional methods such as CS, method 400 can increase the speed of image denoising after image acquisition by distributing a portion of the computational burden that would normally occur after image acquisition to a training phase performed before image acquisition. The training of a deep neural network can occupy a relatively larger portion of time and computational resources than an embodiment, and therefore, by pre-training a deep neural network to identify and extract streak residues and noise residues from image data, removal of streak artifacts and noise can occur faster before image acquisition than can be achieved by methods such as CS (which occur from scratch after image acquisition).

[0078] A technical effect of using a trained deep neural network to map a striped noise image to a stripe residual and a noise residual is that the stripe residual and the noise residual can be used to individually and variably remove stripe artifacts and noise, thereby providing a user with greater control over the displayed appearance of an acquired image. Further, by using a previously trained deep neural network to map a striped noise image to a stripe residual and a noise residual, the speed of image denoising can be increased by "preloading" the computational burden into a training phase that occurs prior to image acquisition. Additionally, by incorporating acquisition parameters related to the acquisition of the striped noise image, the trained deep neural network can be provided with background information related to the stripe artifacts and noise present in the striped noise image, enabling more accurate identification and more selective removal of both stripe artifacts and noise.

[0079] Go to Figure 5 , a flowchart of an exemplary method 500 for generating training data triplets for training a deep neural network to identify and extract stripe artifacts and noise from a stripe noise image is shown. In some embodiments, the training data triplets include three different elements; a true stripe residue, a true noise residue, and a stripe noise image including an image with both the true stripe residue and the true noise residue superimposed. The training data triplets generated by the method 500 can be trained according to a supervised training routine such as Figure 6 The method 600 shown is used to train a deep neural network to identify and extract streak artifacts and noise present in an image. The method 500 can be performed by one or more of the systems discussed above. In one embodiment, the image processing device 202 executing instructions stored in the training module 212 can perform one or more operations of the method 500.

[0080] At operation 502, the image processing device selects a high-resolution natural image without streak artifacts and noise. Natural images may include more types of images than medical images, and may include a larger amount of information than images generated based on mathematical models. Further, a known limitation in the field of machine learning is the lack of training data, and by utilizing relatively common natural images, rather than images in more constrained fields (e.g., medical images), a larger training data set can be generated. In some embodiments, at operation 502, the image processing device may access a repository of high-resolution natural images stored in a non-transitory memory, and may select an image therefrom. In some embodiments, the image processing device may randomly select a high-resolution natural image using a random number generator.

[0081] At operation 504, the image processing device enhances the contrast of the high-resolution natural image to produce a contrast-enhanced image. In some embodiments, enhancing the contrast of the high-resolution natural image selected at operation 504 includes selecting a narrower window width (WW) than the current window width (WW) for displaying the high-resolution natural image, generating a lookup table (LUT) including a mapping of pixel / voxel intensity values ​​of the high-resolution natural image to pixel / voxel intensity values ​​of the contrast-enhanced image. The image processing device may use the LUT to map / transform the pixel / voxel intensity values ​​of the high-resolution natural image to produce a contrast-enhanced image. In other words, in some embodiments, at operation 504, the image processing device increases a lower intensity threshold (below which the pixel / voxel will be displayed as black) and decreases a higher intensity threshold (above which the pixel / voxel will be displayed as white).

[0082] Brief reference Fig. 9 , the effect of contrast enhancement on the generation of streak artifact residues is illustrated by comparing a first artifact residue 906 generated using a non-contrast enhanced image 902 with an artifact residue 908 generated using a contrast enhanced image 904. As can be seen, the first artifact residue 906 includes a mixture of ringing artifacts and streak artifacts, while the second artifact residue 908 primarily includes streak artifacts. Therefore, the artifact residue generated by the contrast enhanced image according to one or more operations of the method 500 primarily includes streak artifacts, so that a deep neural network trained using artifact residues such as the second artifact residue 908 can learn to selectively identify and extract streak residues. In other words, by providing an example of streak artifacts to the deep neural network, generating realistic streak residues from the contrast enhanced image enables the deep neural network to selectively identify and extract streak artifacts, substantially without other types of artifacts / noise (e.g., ringing artifacts, white noise, blurring, colored noise, etc.).

[0083] Temporarily go to Fig.10 , shows a comparison between a denoised image 1002 and a denoised image 1004. The first denoised image 1002 is denoised using a deep neural network trained with training data generated from non-contrast enhanced images, while the second denoised image 1004 is denoised using a deep neural network trained with training data generated from contrast enhanced images. It can be seen that the first denoised image 1002 includes a large amount of streak artifacts, while the second denoised medical image 1004 includes significantly fewer streak artifacts than the first denoised image 1002.

[0084] Steering Figure 5 After operation 504, method 500 may proceed to operation 506. At operation 506, the image processing device may perform a Fourier transform on the contrast-enhanced image to generate a fully sampled k-space (an example of which is represented by Figure 8), and / or the contrast enhanced image may be subjected to a Radon transform to produce a fully sampled sinogram (an example of which is shown by Figure 7 702 in ). More specifically, in embodiments where method 500 is used to generate training data for training a deep neural network to remove streak artifacts and noise from MRI images, operation 506 may include performing a 2D or 3D Fourier transform (FT) on the contrast-enhanced image to produce a fully sampled 2D or 3D k-space, respectively, wherein a 2D FT is employed on the 2D contrast-enhanced image, and wherein a 3D FT is employed on the 3D contrast-enhanced image. In embodiments where method 500 is used to generate training data for training a deep neural network to remove streak artifacts and noise from CT images, operation 506 may include performing a plurality of Radon transforms (RTs) on the contrast-enhanced image to produce a fully sampled 2D or 3D sinogram.

[0085] At operation 508, the image processing apparatus downsamples the fully sampled k-space to generate a downsampled k-space (an example of which is represented by Figure 8 ), and / or the image processing device downsamples the fully sampled sinogram to produce a downsampled sinogram (an example of which is shown by Figure 7 704 of the downsampled sinogram in ). In some embodiments, the image processing device may automatically select a downsampling mode and / or a downsampling ratio to simulate a desired downsampling mode in a predetermined MRI or CT imaging protocol. In some embodiments, a k-space downsampling mode (e.g., PROPELLER, star stacking, etc.) may be selected, a k-space mask may be generated based on the k-space sampling mode (wherein the k-space mask includes an array of 1s and 0s, corresponding to unmasked areas and masked areas of a fully sampled k-space, respectively), and the k-space mask may be applied to the fully sampled k-space to produce a downsampled k-space. Similarly, in some embodiments, a sinogram downsampling mode may be selected to simulate a sparse sinogram sampling mode employed in a predetermined CT imaging protocol, a sinogram mask may be generated based on the sinogram sampling mode (wherein the sinogram mask includes an array of 1s and 0s, corresponding to unmasked areas and masked areas of a fully sampled sinogram, respectively), and the sinogram mask may be applied to the fully sampled sinogram to produce a downsampled sinogram. In some embodiments, a deep neural network trained on training data generated using a particular undersampling pattern may include an indication of the undersampling pattern in metadata associated with the deep neural network. Although several examples of sampling patterns are given herein, it should be understood that the present disclosure encompasses substantially any sampling pattern.

[0086] At operation 510, the image processing device may acquire an inverse FT (IFT) of the downsampled k-space and / or perform filtered backprojection on the downsampled sinogram to produce a fringe image (examples of which are shown by fringe image 708 and fringe image 808). In an embodiment where the undersampled k-space includes a 2D k-space, operation 510 includes performing a 2D IFT on the undersampled k-space to produce a 2D fringe image. In an embodiment where the undersampled k-space includes a 3D k-space, operation 510 includes performing a 3D IFT on the undersampled k-space to produce a 3D fringe image.

[0087] At operation 512, the image processing device subtracts the high-resolution natural image from the streak image to produce a true streak residue. Figure 7 , shows an example of subtracting a high-resolution natural image 706 from a streak image 708 to produce a true streak residue 710. As shown in Figure 7 As can be seen, after subtracting the intensity value of each pixel / voxel from the high-resolution natural image 706 from the intensity value of each corresponding pixel / voxel in the streak image 708, the remaining intensity value represents the streak artifact caused in the high-resolution natural image 706 by downsampling the fully sampled sinusoidal image 702.

[0088] Similarly, in Figure 8 , after the intensity value of each pixel / voxel from the high-resolution natural image 806 is subtracted from the intensity value of each corresponding pixel / voxel of the fringe image 808, a true fringe residue 810 may be generated. The true fringe residue 810 includes the remaining intensity values ​​(i.e., residual intensity values) representing the fringe artifacts caused in the high-resolution natural image 806 by downsampling the fully sampled k-space 802. Figure 8Also shown is a method of generating a channel-combined real streak residual 816 from MR data, wherein the real streak residual 816 simulates streak artifacts that may be produced in an MR image acquired using a multi-channel surface coil. In MR imaging, multi-channel surface coils are typically used for high SNR, large FOV, and / or parallel imaging. However, unlike larger body coils, multi-channel surface coils may further increase the inhomogeneity of signals, noise, and / or artifacts in the MR images acquired thereby. Depending on the channel combination method, the MR images acquired via the multi-channel surface coil may also include an inhomogeneous noise power distribution. The channel-combined high-resolution natural image 812 may be generated by applying a channel-combined algorithm simulation to the high-resolution natural image 806, and similarly, the channel-combined streak image 814 may be generated by applying a channel-combined algorithm simulation to the streak image 808. The channel-combined real streak residual 816 may be generated by subtracting the pixel / voxel intensity values ​​of the channel-combined high-resolution natural image 812 from the corresponding pixel / voxel intensity values ​​in the channel-combined streak image 814. Comparing the true fringe residual 816 of the channel combination with the true fringe residual 810, the distribution of the fringe artifacts is different due to the coil sensitivity distribution and the channel combination.

[0089] At operation 514, the image processing device generates a real noise residual. In some embodiments, at operation 514, the image processing device generates white noise in image space by randomly selecting intensity values ​​for each pixel / voxel in a pixel / voxel array that is equal in size / dimension to the high-resolution natural image according to a predetermined Gaussian distribution of intensity values. In some embodiments, at operation 514, the image processing device simulates color noise to generate a white noise k-space by acquiring the FT of the white noise image, and selectively attenuates the intensity of the white noise k-space according to a predetermined pattern (e.g., by multiplying the white noise k-space by a weighting matrix) to generate a colored noise k-space, wherein the noise intensity is not constant as a function of position in k-space. The colored noise k-space can then be converted to a real noise residual in image space by acquiring the IFT of the colored noise k-space. In some embodiments, operation 514 includes the image processing device generating a real noise residual by generating noise with a flat spatial frequency distribution in a blank image, wherein the first size of the blank image is equal to the second size of the high-resolution natural image. In some embodiments, the image processing device may generate a real noise residual by generating noise having a non-flat spatial frequency distribution in a blank image, wherein a first size of the blank image matches a second size of the high-resolution natural image.

[0090] At operation 516, the image processing device performs pixel-by-pixel / voxel-by-voxel intensity addition for each pixel / voxel in the real noise residual and each corresponding pixel / voxel in the streak image to generate a streak noise image. In other words, the streak noise image generated at operation 516 includes a linear combination of intensity values ​​of each pixel / voxel from the high-resolution natural image selected at operation 502, each pixel / voxel of the real streak residual generated at operation 512, and each pixel / voxel of the real noise residual generated at operation 514.

[0091] At operation 518, the image processing device stores the stripe noise image, the corresponding real stripe residual, and the corresponding real noise residual as a training data triplet. In some embodiments, the training data triplet may be stored with one or more metadata indicating one or more of a sampling mode used to generate the real stripe residual, a noise generation type used to generate the real noise residual, and an indication of the degree of contrast enhancement used to generate the contrast enhanced image at operation 504. After operation 518, method 500 may end.

[0092] Go to Figure 6 , showing the use of deep neural networks such as Figure 3 Flowchart of an exemplary method 600 for inferring streak residues and noise residues from an input streak noise image using a deep neural network 324 as shown. The method 600 may be performed by one or more of the systems discussed above. In some embodiments, the method 600 may be performed by Figure 1 The system 10 shown or Figure 2 The method 600 is implemented by the image processing device 202 shown. In some embodiments, the method 600 may be implemented by the training module 212 stored in the non-transitory memory 206 of the image processing device 202.

[0093] At operation 602, a training data triplet from a plurality of training data triplets is fed to a deep neural network, wherein the training data triplet includes a stripe noise image, and a true stripe residual and a true noise residual corresponding to the stripe noise image. The image processing device may intelligently select the training data triplet based on one or more metadata associated with the training data triplet. In one embodiment, the method 600 may be used to train a deep neural network to identify stripe residuals in images acquired using a specific k-space or sinogram sampling pattern, and operation 602 may include the image processing device selecting a training data triplet generated using the specific sampling pattern.

[0094] In some embodiments, the training data triples and the plurality of training data triples may be stored in the image processing device, such as in the image data module 214 of the image processing device 202. In other embodiments, the training data triples may be obtained via a communication coupling between the image processing device and an external storage device, such as via an Internet connection to a remote server.

[0095] At operation 604, the striped noise image of the training data triplet is mapped to a predicted striped residual and a predicted noise residual. In some embodiments, operation 604 may include inputting pixel / voxel intensity data of the striped noise image into an input layer of a deep neural network, identifying features present in the striped noise image by propagating the image data through one or more encoding layers of the deep neural network, wherein the encoding layer may include one or more convolutional filters; and predicting the striped residual and the noise residual by propagating the identified features through one or more decoding layers of the deep neural network, as described above with reference to Figure 3 discussed in more detail.

[0096] At operation 606, the image processing device calculates a first loss of the deep neural network based on the difference between the predicted streak residual and the true streak residual determined at operation 604. In other words, operation 606 includes the image processing device determining the error of the predicted streak residual using the true streak residual and the loss function. In some embodiments, operation 606 may include the image processing device determining the intensity difference between a plurality of pixels / voxels of the predicted streak residual and a plurality of pixels / voxels of the true streak residual, and inputting the plurality of intensity differences into a predetermined loss function (e.g., a mean square error function or other loss functions known in the field of machine learning). In some embodiments, the first loss may include one or more of a Sorensen-Dice score, a mean square error, an absolute distance error, or a weighted combination of one or more of the foregoing. In some embodiments, operation 606 may include determining the DICE score of the predicted streak residual using the true streak residual according to the following equation:

[0097]

[0098] Where S is the true streak residual and T is the predicted streak residual. In some embodiments, both the predicted streak residual and the true streak residual comprise 3D arrays of intensity values.

[0099] At operation 608, the image processing device may calculate a second loss of the deep neural network based on the difference between the predicted noise residual and the true noise residual determined at operation 604. In other words, operation 608 includes the image processing device determining the error of the predicted noise residual using the true noise residual and the loss function. In some embodiments, operation 608 may include the image processing device determining the intensity difference between a plurality of pixels / voxels of the predicted noise residual and a plurality of pixels / voxels of the true noise residual, and inputting the plurality of intensity differences into a predetermined loss function (e.g., an MSE function or other loss function known in the field of machine learning). In some embodiments, the second loss may include one or more of a DICE score, a mean square error, an absolute distance error, or a weighted combination of one or more of the foregoing. In some embodiments, operation 608 may include determining the DICE score of the predicted noise residual using the true noise residual according to the equation given above with reference to operation 606, where S is the true noise residual and T is the predicted noise residual. In some embodiments, both the predicted noise residual and the true noise residual include 3D intensity value arrays.

[0100] At operation 610, the weights and biases of the deep neural network are adjusted based on the first loss and the second loss calculated at operation 606 and at operation 608. In some embodiments, the first loss and the second loss may be alternately and individually back-propagated through the layers of the deep neural network, and the parameters of the deep neural network may be updated according to a gradient descent algorithm based on the back-propagated first loss and the second loss. In some embodiments, the first loss and the second loss may be aggregated to produce a cumulative loss, and operation 610 may include the image processing device updating the parameters of the deep neural network based on the cumulative loss. In some embodiments, the cumulative loss may be determined by the image processing device by multiplying the first loss and the second loss by the first weight and the second weight, respectively, to produce a first weighted loss and a second weighted loss, and summing the first weighted loss and the second weighted loss to produce a cumulative loss. The loss may be back-propagated through the layers of the deep neural network to update the weights (and biases) of the layers. In some embodiments, back-propagation of the loss may occur according to a gradient descent algorithm, wherein the gradient (first-order derivative or an approximation of the first-order derivative) of the loss function is determined for each weight and bias of the deep neural network. Each weight (and bias) of the deep neural network is then updated by adding the negative of the product of the gradient determined (or approximated) for the weight (or bias) with a predetermined step size according to the following equation:

[0101]

[0102] Where P i+1 is the updated parameter value, P i is the previous parameter value, Step is the step size, and is the partial derivative of the loss with respect to the previous parameters.

[0103] After operation 608, method 600 may end. It should be noted that method 600 may be repeated until the weights and biases of the deep neural network converge, a threshold difference metric is obtained (for the training data or on a separate validation data set), or the rate of change of the weights and / or biases of the deep neural network is below a threshold for each iteration of method 600. In this manner, method 600 enables a deep neural network to be trained to infer streak residuals and noise residuals present in a streak noisy image.

[0104] When introducing the elements of various embodiments of the present disclosure, the words "one", "a kind of" and "the" are intended to mean that there are one or more elements in these elements. The terms "first", "second", etc. do not represent any order, amount or importance, but are used to distinguish one element from another element. The terms "comprise", "include" and "have" are intended to be inclusive, and mean that additional elements may also exist in addition to the listed elements. As used herein, the terms "connected to", "connected to", etc., an object (e.g., material, element, structure, member, etc.) can be connected to or connected to another object, regardless of whether the one object is directly connected or connected to another object, or whether there are one or more intervening objects between the one object and another object. In addition, it should be understood that reference to "one embodiment" or "embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features.

[0105] In addition to any previously indicated modifications, numerous other variations and alternative arrangements may be devised by those skilled in the art without departing from the spirit and scope of the present description, and the appended claims are intended to cover such modifications and arrangements. Therefore, although the information has been described above in detail in conjunction with what are currently considered to be the most practical and preferred aspects, it will be apparent to those of ordinary skill in the art that many modifications may be made, including but not limited to form, function, mode of operation, and use, without departing from the principles and concepts set forth herein. Likewise, as used herein, the examples and embodiments are intended to be illustrative in all respects only and should not be construed as limiting in any way.

Claims

1. A method for selectively removing stripe artifacts and noise from an image using a deep neural network, comprising: receiving a medical image including streak artifacts and noise; The medical image is mapped into streak residues and noise residues using a trained deep neural network, which includes: inputting medical image data of the medical image into an input layer of the trained deep neural network; inputting the k-space sampling pattern used to acquire the medical image into the input layer of the trained deep neural network; and mapping both the medical image data and the k-space sampling pattern to the fringe residual and the noise residual; subtracting the streak residual from the medical image to a first extent and subtracting the noise residual from the medical image to a second extent to produce a denoised medical image; and The denoised medical image is displayed via a display device. 2 . The method according to claim 1 , wherein the medical image is a magnetic resonance image of an anatomical region of a patient acquired by a magnetic resonance imaging system. 3 . The method of claim 1 , wherein the medical image is a computed tomography image of an anatomical region of a patient acquired by a CT system.

4. The method of claim 1, wherein the denoised medical image is a first denoised medical image, and wherein the method further comprises: receiving a user input via a user input device, wherein the user input comprises a third degree and a fourth degree; subtracting the streak residue from the medical image to the third extent, and subtracting the noise residue from the medical image to the fourth extent to produce a second denoised medical image; as well as The second denoised medical image is displayed via the display device. 5 . The method of claim 1 , wherein the first degree comprises a first value between 0 and 1, inclusive, and wherein the second degree comprises a second value between 0 and 1, inclusive.

6. The method of claim 5, wherein subtracting the streak residue from the medical image to the first extent and subtracting the noise residue from the medical image to the second extent comprises: multiplying the streak residual by the first value to produce a weighted streak residual; multiplying the noise residual by the second value to produce a weighted noise residual; as well as The weighted streak residual and the weighted noise residual are subtracted from the medical image to produce the denoised medical image.

7. The method of claim 1, wherein mapping the medical image to the streak residual and the noise residual using the trained deep neural network comprises: receiving one or more acquisition parameters related to acquisition of the medical image; inputting the medical image and the one or more acquisition parameters into an input layer of the trained deep neural network; as well as The medical image and the one or more acquisition parameters are mapped to the streak residual and the noise residual using the trained deep neural network.

8. A method for training a deep neural network, comprising: Generate a training data triplet, wherein the training data triplet includes: images including streak artifacts and noise; True streak remnants; and True noise residual; mapping the image into a predicted streak residual and a predicted noise residual using a deep neural network; Calculating a first loss of the deep neural network based on a difference between the predicted stripe residual and the true stripe residual; Calculating a second loss of the deep neural network based on a difference between the predicted noise residual and the true noise residual; and Updating parameters of the deep neural network based on the first loss and the second loss; Generating the training data triplet includes: Select high-resolution natural images; enhancing the contrast of the high resolution natural image to produce a contrast enhanced image; performing a Fourier transform on the contrast-enhanced image to produce a fully sampled k-space, or performing a Radon transform on the contrast-enhanced image to produce a sinogram; Downsampling the fully sampled k-space to generate a downsampled k-space, or downsampling the sinogram to generate a downsampled sinogram; Obtaining an inverse Fourier transform of the downsampled k-space to generate a fringe image, or performing filtered backprojection on the downsampled sinogram to generate a fringe image; The high resolution natural image is subtracted from the fringe image to produce the true fringe residual.

9. The method according to claim 8, wherein generating the training data triples further comprises: generating the real noise residual; adding the true noise residual to the streak image to produce the image including streak artifacts and noise; The image, the true streak residual, and the true noise residual are stored as the training data triplet.

10. The method of claim 9, wherein generating the real noise residual comprises: Noise having a flat spatial frequency distribution is generated in a blank image, wherein a first size of the blank image is equal to a second size of the high resolution natural image.

11. The method of claim 9, wherein generating the real noise residual comprises: Noise having a non-flat spatial frequency distribution is generated in a blank image, wherein a first size of the blank image matches a second size of the high-resolution natural image.

12. The method of claim 8, wherein downsampling the fully sampled k-space to generate downsampled k-space comprises: Select the sampling mode; generating a k-space mask based on the sampling pattern; The k-space mask is applied to the fully sampled k-space to produce the downsampled k-space.

13. The method of claim 12, wherein the sampling pattern comprises radial undersampling of the fully sampled k-space.

14. The method of claim 8, wherein downsampling the sinusoidal graph to generate the downsampled sinusoidal graph comprises: Select the sampling mode; generating a sinusoidal mask based on the sampling pattern; The sinogram mask is applied to the sinogram to produce the downsampled sinogram.

15. A system for selectively removing streak artifacts and noise from an image using a deep neural network, comprising: a memory storing a trained deep neural network and instructions; and a processor, wherein the processor is communicatively coupled to the memory and when executing the instructions is configured to: receiving a medical image, wherein the medical image includes streak artifacts and noise; mapping the medical image into a streak residue and a noise residue using the trained deep neural network; removing the streak residue from the medical image to a first extent and removing the noise residue from the medical image to a second extent to produce a denoised medical image; as well as Displaying the denoised medical image via a display device; wherein the processor, when executing the instructions, is configured to use the trained deep neural network to map the medical image to the streak residual and the noise residual in the following manner: inputting medical image data of the medical image into an input layer of the trained deep neural network; Inputting a k-space sampling pattern used to acquire the medical image into the input layer of the trained deep neural network; as well as Both the medical image data and the k-space sampling pattern are mapped to the fringe residual and the noise residual.

16. The system of claim 15, wherein the system further comprises the display device, and wherein the memory further comprises user preferences, and wherein the processor, when executing the instructions, is configured to retrieve the first degree and the second degree from the user preferences.

17. The system according to claim 15, wherein the system is a magnetic resonance imaging system, and wherein the medical image is a magnetic resonance image acquired by the magnetic resonance imaging system via radial or helical sampling of k-space.

18. The system of claim 15, wherein the system further comprises a user input device communicatively coupled to the processor, wherein the de-noised medical image is a first de-noised medical image, and wherein the processor, when executing the instructions, is further configured to: receiving a user input via the user input device, wherein the user input comprises a third degree and a fourth degree; removing the streak residue from the medical image to the third extent, and removing the noise residue from the medical image to the fourth extent to produce a partially denoised medical image; and The partially denoised medical image is displayed via the display device.

Images