Method and system for flexible image denoising using disentangled feature representation viewing field

Abstract

A system and method for denoising an image is provided. A standard image module is configured to generate standard anatomical features and standard noise features from the standard image and reconstruct the standard image from the standard anatomical features and the standard noise features. A low quality image module is configured to generate low quality anatomical features and low quality noise features from the low quality image and reconstruct the low quality image from the low quality anatomical features and the low quality noise features. A loss calculation module is provided that calculates a loss metric based at least in part on 1) a comparison of the reconstructed standard image to the standard image and 2) a comparison of the reconstructed low quality image to the low quality image. Upon providing the low quality anatomical features to the standard image module, the standard image module outputs a reconstructed standard transferred image.

Description

[Technical field]

[0001]

[0001] The present disclosure generally relates to systems and methods for training and tuning neural network models that provide a flexible solution for denoising low-dose images using domain-independent learning with disentangled feature representations. [Background technology]

[0002]

[0002] Traditionally, most imaging modalities affect the physics of acquisition or reconstruction, resulting in artifacts such as noise in the final image. To train a denoising algorithm, such as a neural network model, pairs of noisy and noiseless image samples are presented to the neural network model, and the network attempts to minimize a cost function by denoising the noisy image and recovering the corresponding noiseless ground truth image.

[0003]

[0003] However, any change in the parameters used to acquire an image changes the shape or amount of artifacts in the corresponding image. Thus, denoising models used to denoise standard images are less effective when applied to images acquired using different acquisition parameters, such as reduced radiation dose for computed tomography (CT) scans.

[0004]

[0004] The increasing use of CT scans in modern medical practice has raised concerns about the associated radiation dose required, making dose reduction a clinical goal. However, lowering radiation dose tends to significantly increase noise and other artifacts in reconstructed images, compromising diagnostic information. Enormous efforts have been made to reduce the noise in low-dose CT scans, thereby converting them into higher quality images.

[0005]

[0005] Machine learning techniques have been investigated for image denoising, including the use of convolutional neural networks (CNNs). However, existing methods are usually tuned to a specific noise level and cannot generalize well to noise levels not covered by the training set used to train the corresponding CNN.

[0006]

[0006] In CT imaging, several factors, including kilovoltage peak (kVp), milliampere-seconds (mA-seconds), slice thickness, and patient size, all affect the noise level of the reconstructed image. Changing any of these imaging parameters results in different noise levels or different artifact profiles, and thus, in the past, different models were required to denoise images acquired with such different imaging parameters. This limits the applicability of CNN-based methods in practical denoising. Summary of the Invention [Problem to be solved by the invention]

[0007]

[0007] Therefore, there is a need for a method that can denoise images acquired with imaging parameters different from those used in the training set of the corresponding method. Furthermore, there is a need for a single trained model that can be used to denoise images with a variety of noise levels, including CT images acquired with a lower radiation dose than the images in the training set. [Means for solving the problem]

[0008]

[0008] A system and method for denoising medical images is provided. In one embodiment, a standard image module is configured to generate standard anatomical features and standard noise features from the standard image and reconstruct the standard image from the standard anatomical features and the standard noise features. A low quality image module is similarly configured to generate low quality anatomical features and low quality noise features from the low quality image and reconstruct the low quality image from the low quality anatomical features and the low quality noise features.

[0009]

[0009] A loss calculation module is provided that the systems and methods can train to calculate a loss metric based at least in part on 1) a comparison of the reconstructed standard image to the standard image, and 2) a comparison of the reconstructed low quality image to the low quality image.

[0010]

[0010] The loss metric calculated in the loss calculation module is incorporated into a loss function for adjusting the standard image module and the low quality image module using machine learning. Upon providing the standard image module with the low quality anatomical structure features, the standard image module outputs a reconstructed standard transfer image that includes the low quality anatomical structure features and a noise level that is lower than the noise level represented by the low quality noise features.

[0011]

[0011] In some embodiments, the standard image module has a standard anatomical structure encoder, a standard noise encoder, and a standard generator, and upon receiving a standard image, the standard anatomical structure encoder outputs standard anatomical structure features, the standard noise encoder outputs standard noise features, and the standard generator reconstructs the standard image from the standard anatomical structure features and the standard noise features.

[0012]

[0012] In some such embodiments, the low quality image module similarly has a low quality anatomical structure encoder, a low quality noise encoder, and a low quality generator, and upon receiving a low quality image, the low quality anatomical structure encoder outputs low quality anatomical structure features, the low quality noise encoder outputs low quality noise features, and the low quality generator reconstructs the low quality image from the low quality anatomical structure features and the low quality noise features.

[0013] In some such embodiments, the loss calculation module calculates a loss metric for the standard generator based at least in part on a comparison of the reconstructed standard image to the standard image. The loss calculation module similarly calculates a loss metric for the low quality generator based at least in part on a comparison of the reconstructed low quality image to the low quality image.

[0014]

[0014] The loss calculation module further calculates a loss metric for the standard anatomical structure encoder based on a comparison with the segmentation label for the standard image, and the loss calculation module further calculates a loss metric for the low quality anatomical structure encoder based on a comparison with the output of the standard anatomical structure encoder.

[0015]

[0015] In some embodiments, the loss criterion of the low-quality anatomical structure encoder is an adversarial loss criterion.

[0016]

[0016] In some embodiments, the system implementing the described method further comprises a segmentation network, and the segmentation mask of the reconstructed standard image is evaluated based on a comparison with a segmentation label for the standard image.

[0017] In some such embodiments, upon providing the low-quality generator with the standard anatomical features, the low-quality generator outputs a reconstructed low-quality transition image, the low-quality transition image including the standard anatomical features and a noise level higher than the noise level represented by the standard noise features. A segmentation mask of the low-quality transition image is then evaluated based on a comparison with the segmentation label for the standard image.

[0018]

[0018] In some embodiments, the loss criterion for the standard transferred image is evaluated based on a comparison with a standard image reconstruction, and the loss criterion for the standard transferred image is an adversarial loss criterion.

[0019]

[0019] In some embodiments, the standard image module and the low quality image module are trained simultaneously.

[0020] In other embodiments, the standard image module is trained before the training of the low quality image module, and the values ​​of the variables created during the training of the standard image module are held constant during the training of the low quality image module. In some such embodiments, after training the standard image module and the low quality image module, the system further trains the standard generator while holding constant values ​​for the standard anatomical structure coder, the standard noise coder, and the low quality anatomical structure coder.

[0021]

[0021] In some embodiments, the standard anatomical feature and the low quality anatomical feature each correspond to a single anatomical structure. [Brief description of the drawings]

[0022] [Figure 1]

[0022] FIG. 1 is a schematic diagram of a system according to one embodiment of the present disclosure. [Diagram 2]

[0023] FIG. 1 illustrates an imaging device according to one embodiment of the present disclosure. [Diagram 3]

[0024] FIG. 2 is a diagram of a training pipeline for use in one embodiment of the present disclosure. [Figure 4A]

[0025] FIG. 4 illustrates an exemplary training method used in the training pipeline of FIG. [Figure 4B] FIG. 4 illustrates an exemplary training method used in the training pipeline of FIG. [Figure 4C] FIG. 4 illustrates an exemplary training method used in the training pipeline of FIG. [Diagram 5]

[0026] FIG. 4 illustrates the use of a model trained using the training pipeline of FIG. 3 to denoise an image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023]

[0027] The description of exemplary embodiments according to the principles of the present disclosure is intended to be read in conjunction with the accompanying drawings, which are to be considered as part of the entire written description. Any reference to direction or orientation in the description of the disclosed embodiments disclosed herein is merely for convenience of description and is not intended to limit the scope of the present disclosure in any way. Relative terms such as "lower", "upper", "horizontal", "vertical", "above", "below", "up", "down", "top" and "bottom", as well as derivatives thereof (e.g., "horizontally", "downwards", "upwards", etc.) should be construed as referring to the orientation as illustrated in the drawings then being described or discussed. These relative terms are merely for convenience of description and do not require that the device be constructed or operated in a particular orientation unless expressly indicated as such. Terms such as "mounted," "attached," "connected," "coupled," "interconnected," and the like, unless otherwise specified, refer to both a movable or rigid attachment or relationship as well as a relationship in which structures are fixed or attached to one another, either directly or indirectly through intervening structures. Moreover, features and advantages of the present disclosure are illustrated with reference to the illustrated embodiments. The present disclosure should not therefore be specifically limited to such exemplary embodiments, which illustrate some possible non-limiting combinations of features, either alone or in other combinations of features. The scope of the present disclosure is defined by the claims appended hereto.

[0024]

[0028] This disclosure describes one or more currently contemplated best modes of carrying out the present disclosure. This description is not intended to be understood in a limiting sense, but rather provides an example of the present disclosure, presented merely for illustrative purposes, to inform those skilled in the art of the advantages and configurations of the present disclosure, by referring to the accompanying drawings. In the various figures of the drawings, like reference numerals represent like or similar parts.

[0025]

[0029] It is important to note that the disclosed embodiments are merely examples of the many advantageous uses of the innovative teachings herein. Statements made in the specification of this application generally do not necessarily limit any of the variously claimed disclosures. Moreover, some statements may apply to some inventive features but not to other features. Generally, singular elements may be plural and vice versa without loss of generality, unless otherwise indicated.

[0026]

[0030] Typically, image processors that use algorithms or models to denoise medical images are based on the level and form of noise that is expected to be present in the corresponding image, which is typically based on various parameters used to acquire the image.

[0027]

[0031] For computed tomography (CT) based medical imaging, various image processors are used to process images, such as machine learning algorithms in the form of convolutional neural networks (CNNs). These image processors are trained on various corresponding anatomical regions and structures, which in the case of machine learning algorithms have specific noise levels. The noise level of an image is a function of several factors, including, in this case, kilovolt peaks (kVp), milliamp seconds (mA seconds), slice thickness, and patient size.

[0028]

[0032] While a denoising image processor such as a CNN is formed based on standardised parameters, in this case based on expected noise levels and including a standardised radiation dose, the systems and methods disclosed herein effectively fit such models to images acquired using a range of acquisition parameters, such as low radiation dose.

[0029]

[0033] Although the following discussion is specific to CT-based medical imaging implementations, similar systems and methods are used in conjunction with other imaging modalities, such as magnetic resonance imaging (MRI) or positron emission tomography (PET).

[0030]

[0034] 1 is a schematic diagram of a system 100 according to one embodiment of the present disclosure. The system 100 generally includes a processing device 110 and an imaging device 120, as shown.

[0031]

[0035] The processing device 110 applies a processing routine to the received image. The processing device 110 comprises a memory 113 and a processor circuit 111. The memory 113 stores a plurality of instructions. The processor circuit 111 is coupled to the memory 113 and configured to execute the instructions. The instructions stored in the memory 113 include the processing routine as well as data related to a plurality of machine learning algorithms, such as various convolutional neural networks, for processing the image.

[0032]

[0036] The processing device 110 further comprises an input unit 115 and an output unit 117. The input unit 115 receives information, such as images, from an imaging device 120. The output unit 117 outputs information to a user or a user interface device. The output unit 117 comprises a monitor or display.

[0033]

[0037] In some embodiments, processing device 110 is in direct association with imaging device 120. In alternative embodiments, processing device 110 is separate from imaging device 120 and receives images for processing via a network or other interface at input 115.

[0034]

[0038] In some embodiments, the imaging device 120 comprises an image data processing device and a spectral or conventional CT scanning unit that generates CT projection data upon scanning an object (eg, a patient).

[0035]

[0039] 2 illustrates an exemplary imaging device according to one embodiment of the present disclosure. Although a CT imaging device is illustrated and the following discussion is with respect to CT images, it will be understood that similar methods apply with respect to other imaging devices and that the images to which these methods are applied may be acquired in a wide variety of ways.

[0036]

[0040] In an imaging device according to an embodiment of the present disclosure, the CT scanning unit is adapted to perform multiple axial and / or helical scans of the object to generate CT projection data. In an imaging device according to an embodiment of the present disclosure, the CT scanning unit comprises an energy-resolving photon counting image detector. The CT scanning unit comprises a radiation source that emits radiation that traverses the object when acquiring the projection data.

[0037]

[0041] In the imaging device according to an embodiment of the present disclosure, the CT scanning unit further performs a scout scan that is different from the primary scan, thereby generating separate images associated with the scout scan and the primary scan, which images are different but contain the same object.

[0038]

[0042] 2, comprises a stationary gantry 202 and a rotating gantry 204 rotatably supported by the stationary gantry 202. The rotating gantry 204 rotates about a longitudinal axis around an examination region 206 of the object when acquiring projection data. The CT scanning unit 200 comprises a support 207 for supporting a patient in the examination region 206 and is configured to move the patient through the examination region during the imaging process.

[0039]

[0043] The CT scanning unit 200 comprises a radiation source 208, such as an X-ray tube, supported by and configured to rotate with a rotating gantry 204. The radiation source 208 comprises an anode and a cathode. A source voltage applied between the anode and the cathode accelerates electrons from the cathode to the anode. The flow of electrons enables a current flow from the cathode to the anode to generate radiation across an examination region 206.

[0040]

[0044] The CT scanning unit 200 comprises a detector 210. The detector 210 defines an arc at an angle on the opposite side of the examination region 206 to the radiation source 208. The detector 210 comprises a one-dimensional or two-dimensional array of pixels, such as pixels of a direct conversion detector. The detector 210 is adapted to detect radiation traversing the examination region and to generate a signal indicative of the energy of the radiation.

[0041]

[0045] The CT scanning unit 200 further comprises generators 211 and 213. The generator 211 generates tomographic projection data 209 based on signals from the detector 210. The generator 213 receives the tomographic projection data 209 and generates a raw image of the object based on the tomographic projection data 209.

[0042]

[0046] 3 is a schematic diagram of a training pipeline for use in one embodiment of the present disclosure. The training pipeline 300 is typically implemented by the processing device 110 and the corresponding method is implemented by the processor circuit 111 based on instructions stored in the memory 113. The memory 113 further comprises a database structure that stores data required to implement the training pipeline 300 or a denoising method, such as an implementation of a model that utilizes data generated in the training pipeline. In some embodiments, the database is stored externally and accessed by the processing device 110 via a network interface.

[0043]

[0047] The method implemented in the training pipeline 300 includes training a learning algorithm, such as a CNN, to denoise low-dose CT images using domain-independent learning with a disentangled feature representation.

[0044]

[0048] Domain adaptation is done by applying the source domain D s From a given dataset, such as a source-labeled dataset belonging to a specific known target domain D t Domain-independent learning can be defined as transferring knowledge to a target unlabeled dataset belonging to a domain. There is no domain label to annotate each sample as to which domain it belongs to, and both the target unlabeled dataset and the source dataset can consist of data from multiple domains (e.g., the target domain is {D t1 ,D t2 ,...,D tn}, and {D s1 ,D s2 ,...,D sn}), domain-independent learning is achieved by using disentangled representation learning, which disentangles style from content.

[0045]

[0049] The training pipeline 300 can thus disentangle anatomical features a from noise features n for CT images processed by the pipeline, and use the anatomical and noise features to reconstruct the underlying image. When evaluating a model's denoising performance, radiologists score images on two types of quality: structural fidelity and image noise suppression. Structural fidelity is the ability of an image to accurately depict the anatomical structures in the field of view, while image noise appears as irregular patterns on the image and reduces image quality. By extracting anatomical features from low-quality images and combining them with low noise levels characteristic of higher quality images, models generated with the described training pipeline provide images that score highly on both metrics.

[0046]

[0050] The training pipeline 300 is shown to include a standard image module 310 and a low quality image module 320. The standard image module 310 is implemented using a standard anatomical structure encoder E a CT 330, Standard Noise Encoder E n CT 340, and the standard generator G CT 2 or an image database, then provides a standard image 370 to the standard image module 310. Once received, the standard image 370 is provided to a standard anatomy encoder 330 and a standard noise encoder 340.

[0047]

[0051] The standard anatomical structure encoder 330 then encodes the standard anatomical structure features a CT 380, the standard noise encoder 340 outputs the standard noise feature n CT Both the standard noise features 390 and the standard anatomical features 380 are then calculated from the standard anatomical features 380 and standard noise features 390 provided to the standard image X CT CTThe standard image module 310 thus decomposes the standard image 370 into component features 380, 390 and is then able to reconstruct the standard image 370' from these component features.

[0048]

[0052] The low quality image module 320 has components in parallel with those discussed with respect to the standard image module 310. The low quality image module 320 thus includes a low quality anatomical structure encoder E a LDCT 430, Low-quality Noise Encoder E n LDCT 440 and low quality generator G LDCT 450. The low quality image source 460 then provides the low quality image 470 to the low quality image module 320. The low quality image source 460 is the CT scan unit 200 of FIG. 2 where the parameters discussed above are reconfigured to reduce the image quality. The low quality image source is based on a reduced radiation dose compared to the acquisition of the standard image 370, for example resulting in a low dose CT scan (LDCT) image. Alternatively, the low quality image source 460 may be an image database. Once received, the low quality image 470 is provided to the low quality anatomy coder 430 and the low quality noise coder 440.

[0049]

[0053] The low-quality anatomical structure encoder then outputs the low-quality anatomical structure feature a LDCT 480, the low-quality noise encoder 440 outputs the low-quality noise feature n LDCT Both the low quality noise features 490 and the low quality anatomical features 440 are then combined to produce a low quality image X LDCT LDCT The low quality image module 320 thus decomposes the low quality image 470 into component features 480, 490 and can then reconstruct the low quality image 470' from these component features.

[0050]

[0054] A loss calculation module 500 is provided to calculate various loss functions when training the standard image module 310 and the low quality image module 320. Thus, the loss metric of the standard generator 350 is based at least in part on a comparison between the standard image 370 and the reconstructed standard image 370'. Such a loss metric is the reconstruction loss 510 of the standard generator 350, and is used during training to ensure that the reconstruction 370' generated by the standard generator is identical to the corresponding standard image 370 originally provided.

[0051]

[0055] The loss metric of the low quality generator 450 is similarly based at least in part on a comparison of the low quality image 470 and the reconstructed low quality image 470' generated by the low quality generator 450. The loss metric is again the reconstruction loss 520 of the low quality generator 450, and is used during training to ensure that the reconstruction 470' from the low quality generator is identical to the corresponding low quality image 470 that was originally fed in.

[0052]

[0056] The anatomy encoder 330, 430 is also evaluated by a loss calculation module 500. The loss metric of the standard anatomy encoder 330 is evaluated based on a comparison of the standard anatomy features 380 with the segmentation labels 540 generated for the corresponding standard image 370. Such segmentation labels are either generated manually when the standard image 370 is acquired or retrieved from a database. Such loss metric is the segmentation loss 530.

[0053]

[0057] The loss metric of the low-quality anatomy encoder 430 is evaluated based on a comparison of the low-quality anatomy features 480 to the standard anatomy features 380 generated by the standard anatomy encoder 330. This comparison is discussed in more detail below, but for the training method of Figures 4A-4C, such a loss metric would typically be an adversarial loss 550.

[0054]

[0058] To evaluate the anatomical structure encoders 330, 340, the loss calculation module 500 includes a segmentation mask M a CT 570a and M a LDCT A segmentation network 560 is provided that produces segmentation masks 570a and 570b for the standard anatomical features 480, respectively. The segmentation masks 570a and 570b for the low quality anatomical features 480 are then compared to the segmentation labels 540, and the segmentation masks 570b for the low quality anatomical features 480 are then compared to the segmentation masks 570a and 570b for the standard anatomical features 380.

[0055]

[0059] The loss metric is incorporated into a loss function to tune the respective generators 350, 450 and anatomy encoders 330, 430 using machine learning. Thus, the reconstruction losses 510, 520 for the generators 350, 450, respectively, can be used to improve the performance of the respective pipelines of the generators by adjusting the variables that determine the output of the corresponding modules. This is done by implementing standard machine learning techniques or by implementing an exemplary training method, discussed below with reference to Figures 4A-4C.

[0056]

[0060] In some embodiments, the loss calculation module further generates additional loss metrics, including a segmentation loss 580 for reconstructing the standard image 370′ generated by the standard generator 350. Thus, a segmentation network 590 is applied to the reconstructed image 370′ to generate a segmentation mask M CT CT 600, and then a segmentation mask M CT CT is evaluated based on the segmentation labels 540 for the corresponding images 370. The segmentation loss 580 is taken into account in every training process used in the training pipeline 500.

[0057]

[0061] As shown, a connection is made between the anatomical structure features 380 output from the standard anatomical structure coder 330 and the low-quality generator 450, and similarly, a connection is made between the anatomical structure features 480 output from the low-quality anatomical structure coder 430 and the standard generator 350. When the generator 350 is provided with the standard anatomical structure features 380 from the standard anatomical structure coder 330, it outputs a reconstructed standard image 370' based on the standard anatomical structure features and standard noise features 390 generated by the standard noise coder 340. On the other hand, when the standard generator 350 is provided with the low-quality anatomical structure features 480 generated by the low-quality anatomical structure coder 430, it outputs a reconstructed standard transition image X LDCT CT Outputs 470''.

[0058]

[0062] The reconstructed standard transfer image 470'' includes low quality anatomical features 480 and a noise level lower than the noise level represented by low quality noise features 490. The reconstructed standard transfer image is constructed by generator 350 based on the low quality anatomical features 480 and the noise features 390 generated by standard noise feature encoder 340. Such noise features 390, in the case of a transferred image, are, for example, an average of the standard noise features generated from the corresponding standard images 370 during training.

[0059]

[0063] Similarly, a transition image is generated using a low-quality generator 450. The generator 350 thus generates a reconstructed low-quality transition image X CT LDCT The reconstructed low-quality metastasis image 370'' includes standard anatomical features 380 and a noise level based on the low-quality noise features 490.

[0060]

[0064] To further improve the quality of any model implemented using the training pipeline 500, an additional loss metric is generated using the transferred images 370'', 470''. The reconstructed standard quality transferred image 470'' is therefore evaluated using an adversarial loss 610 that compares the transferred image to the reconstructed image 370' output from the standard generator 350.

[0061]

[0065] The reconstructed low-quality transition image 370″ is similarly evaluated using a loss criterion. Since the low-quality transition image 370″ contains standard anatomical features 380, the training pipeline 300 will usually have access to the corresponding segmentation labels 540. The loss criterion for the low-quality transition image 370″ is thus the segmentation loss 620, and the segmentation network 630 will be able to find a suitable segmentation mask M CT LDCT Generates 640.

[0062]

[0066] It should be further noted that in some embodiments, the segmentation network 560, 590, 630 itself is evaluated based on the segmentation loss 530, 580, 620, which is the result of comparing the resulting segmentation mask 570a, 580, 620 with the segmentation label 540.

[0063]

[0067] In some embodiments, the entire training pipeline 300 is trained simultaneously. Thus, both the standard image module 310 and the low quality image module 320 will be fed with various images for the purpose of training the respective modules. Once the network is improved, the training pipeline is instructed to generate transformed images 470'', 370'' based on the loss criterion, the anatomical features, and the reconstructed images, which will then be evaluated in parallel.

[0064]

[0068] In some embodiments, by contrast, the training pipeline 300 is trained sequentially, as discussed in more detail below with reference to FIGS. 4A-4C.

[0065]

[0069] The illustrated training pipeline 300 is used to generate a model that generates a transferred image 470'', which in this case can be used to denoise a previous low quality image 470. The training pipeline 300 is used to create modules for transferring anatomical features from a variety of original images acquired using different imaging parameters. For example, different encoders are trained to transfer images acquired using different amounts of radiation.

[0066]

[0070] Additionally, although the same model created using the training pipeline 300 may be used across different anatomical features, in some embodiments, the use of a model trained using the pipeline is limited to a particular anatomical structure. The low quality image 470 may not be an image of the same patient or the same organ as the standard image 370, but in this case may pertain to the same anatomical structure in a different image. The model may be trained, for example, on images of the head, abdomen, or a particular organ.

[0067]

[0071] 4A-4C illustrate an exemplary training method used in the training pipeline 300 of FIG.

[0068]

[0072] In some embodiments, as shown in Figure 4A, the standard image module 310 is trained before training the low quality image module 320. In such embodiments, as shown in Figure 4B, when training the low quality image module 320, the variables associated with the standard image module are kept constant, including the variables built into the standard image encoders 330, 340, the variables of the standard generator 350, and the variables of the segmentation network 560 associated with the standard anatomical features 380.

[0069]

[0073] In such an embodiment, the training pipeline 300 trains the standard image module 310 and the low quality image module 320 separately, and then further trains the standard generator 350 while retaining values ​​for the standard anatomical structure coder 330, the standard noise coder 340, and the low quality anatomical structure coder 430.

[0070]

[0074] Thus, when training a model using the claimed method, a method for implementing training of a pipeline 300 according to this disclosure first provides a standard image module 310. The standard image module 310 has a standard anatomical structure encoder 330 that extracts standard anatomical structure features 380 from a standard image 370, and a standard noise encoder 340 that extracts standard noise features 390 from the standard image.

[0071]

[0075] The standard image module 310 also comprises a standard generator 350 which generates a reconstructed standard image 370 ′ from the standard anatomical features 380 and the standard noise features 390 .

[0072]

[0076] The method then provides a low quality image module 320. The low quality image module 320 comprises a low quality anatomy encoder 430 for extracting low quality anatomical features 480 from the low quality image 470, and a low quality noise encoder 440 for extracting low quality noise features 490 from the low quality image.

[0073]

[0077] The low quality image module 410 also includes a low quality generator 450 that generates a reconstructed low quality image 470 ′ from the low quality anatomical features 480 and the low quality quasi-noise features 490 .

[0074]

[0078] The standard image module 310 is then trained by receiving a number of standard images 370 at the standard image module. The standard images will be received from an image source 360, which may be the CT scan unit 200 or alternatively a database of images. For each received standard image 370, the system then compares the reconstructed standard image 370' output from the standard image generator 350 with the corresponding standard image and extracts a standard reconstruction loss metric. The variables of the standard image module 310 are then updated based on such loss metric, thereby adjusting the standard encoders 330, 340 and the standard generator 350.

[0075]

[0079] The method generates, in a segmentation network 560, segmentation masks 570a corresponding to the standard anatomical features 380 extracted from each standard image 370 by the standard anatomy encoder 330 while still training the standard image module 310. The segmentation masks 570a for each standard image 370 are then compared to the segmentation labels 540 associated with the corresponding standard image 370 to generate a standard anatomical loss metric 530. At least one variable of the standard anatomy encoder 330 is then updated based on the standard anatomical loss metric.

[0076]

[0080] As shown in Fig. 4A, these initial training steps are performed in a part of the training pipeline 300 related to the standard image 370, independent of the low quality image module 320. Additional training elements are performed during this first part of training to improve the models used for the standard encoders 330, 340 and the standard generator 350. For example, the reconstructed standard image 370' is submitted to an additional segmentation network 590 to generate a segmentation mask 600, which is then compared to the known segmentation labels 540 for the corresponding image 370 to generate an additional segmentation loss metric.

[0077]

[0081] After the standard image module 510 has been trained and has produced acceptable results, the low quality image module 320 is trained, as shown in Figure 4B. To that end, a number of low quality images 470 are received at the low quality image module 320 from an image source 460. As discussed above, the image source may be the CT scanning unit 200 or an image database, or some combination thereof.

[0078]

[0082] The system then compares, for each received low quality image 470, the reconstructed low quality image 470′ output from the low quality image generator 450 with the corresponding low quality image and extracts the low quality reconstruction loss 520. The low quality encoders 430, 440 and the low quality generator 450 are then adjusted by keeping the variables of the standard image module 310 constant while the variables of the low quality image module 320 are updated based on the loss metric.

[0079]

[0083] The segmentation network 560 that is initially trained when training the standard image module 310 is similarly kept constant when training the low quality image module 320. The segmentation network 560 is then used to generate a segmentation mask 570b that corresponds to the low quality anatomical features 480 extracted from each low quality image 470 by the low quality anatomy encoder 430. The segmentation mask 570b is then compared to at least one of the segmentation masks 570a previously generated by the standard image module 310, typically the average of the segmentation masks 570a, to generate an adversarial loss metric.

[0080]

[0084] The adversarial loss metric is then used to update at least one variable of the low-quality anatomy encoder 430.

[0081]

[0085] Once the low quality image module 320 is trained in this manner, the low quality anatomical features 480 generated by the low quality anatomy encoder 430 are then fed to the standard generator 350 of the standard image module 310. The standard generator 350 then outputs a reconstructed standard transferred image 470'' based on the low quality anatomical features 480 and at least one of the standard quality noise features 390, possibly an average of the standard quality noise features 390.

[0082]

[0086] The reconstructed standard transfer image 470'' is then evaluated by comparing it to the standard image reconstruction 370' or an average of multiple such reconstructions to generate an adversarial loss 610. The standard generator 350 is kept constant and thus no adjustments are made based on the adversarial loss criterion, while the low-quality anatomical structure encoder 430 is further adjusted based on the adversarial loss criterion.

[0083]

[0087] Further, as discussed above, a reconstructed low-quality transition image 370'' is generated using the low-quality generator 450. Such transition image 370'' may be analyzed by a segmentation network 630 based on the standard anatomical features 380 generated by the standard image module 320 to generate a corresponding segmentation map 640, which may then be evaluated against the segmentation labels 540 to generate a segmentation loss 620. The segmentation loss evaluation metric may then be used to further tune the low-quality module 320.

[0084]

[0088] In some embodiments, the models created in the described training pipeline 300 are then used to create standard transferred images 470″, which can then be denoised. In other embodiments, as shown in FIG. 4C, the training pipeline is then further trained. As shown, the associated encoders 330, 340, and 430 are held constant while the standard generator 350 is further tuned. Training is performed by creating additional reconstructed standard transferred images 470″, which can then be compared to one or more standard reconstructions 370′ to generate additional adversarial loss data 610, which can then be used to further tune the standard generator 350.

[0085]

[0089] Thus, by training the described training pipeline 300, there are several checks in the design that encourage the incorporation of domain-independent anatomical structures and the preservation of anatomical structures.

[0086]

[0090] Figure 5 illustrates the use of a model trained using the training pipeline 330 of Figure 3 to denoise an image. In denoising an image, the model first encodes low-dose anatomical data from a low-quality image 470 into low-quality anatomical features 480 using a low-quality anatomical encoder 430. The low-quality anatomical features 480 are then fed to a standard generator 350 along with standard noise features 390 or an average of such noise features to generate a reconstructed standard transition image 470''.

[0087]

[0091] This training pipeline 300 can be used to train a transformation from low-dose CT or other low-quality images to multiple normal-dose CT or standard images with different noise levels. In the case of CT images, this noise level can represent the average of the entire normal-dose CT data, or the average of a set of CT data with specific characteristics reconstructed with a specific algorithm, such as filtered back projection or iterative reconstruction. The trained CT noise encoder E n CT To obtain these noise features, the training set 330 is used on all or a subset of the normal dose CT images 370 in the training set to encode the normal dose CT images into CT noise features 390. The representative CT noise features, n CT1 , n CT2 etc., each of the CT noise features can then be used as the average noise feature 390 discussed above. A user can select from predefined CT noise features 390 based on specific needs to denoise low-dose CT data. The training pipeline 300 also performs interpolation of the CT noise features extracted from various groups so that a user can continuously adjust the CT noise.

[0088]

[0092] Although the methods described herein are described with respect to CT scan images, a variety of imaging techniques are contemplated, including a variety of medical imaging techniques, and it will be understood that images generated using a wide variety of imaging techniques can be effectively denoised using the methods described herein.

[0089]

[0093] The method according to the present disclosure is implemented on a computer as a computer-implemented method, on dedicated hardware, or a combination of both. The executable code of the method according to the present disclosure is stored in a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. The computer program product preferably includes non-transitory program code stored in a computer readable medium for performing the method according to the present disclosure when the program product is executed on a computer. In one embodiment, the computer program includes computer program code adapted to perform all steps of the method according to the present disclosure when the computer program is executed on a computer. The computer program is embodied on a computer readable medium.

[0090]

[0094] While the present disclosure has been described in some detail and with a certain degree of particularity with respect to certain illustrated embodiments, it is not intended to be limited to any such details, embodiments, or to any particular embodiment, but should be interpreted with reference to the appended claims so as to give the broadest possible interpretation in view of the prior art, thereby effectively encompassing the intended scope of the present disclosure.

[0091]

[0095] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the present disclosure and concepts that the inventors have contributed to furthering the art, and should not be construed as limiting such specifically recited examples and conditions. Furthermore, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples of the present disclosure, are intended to encompass both structural and functional equivalents of the present disclosure. Moreover, such equivalents are intended to include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

Claims

1. 1. A system for denoising a medical image, the system comprising: a standard image module for generating standard anatomical features and standard noise features from a standard image and reconstructing the standard image from the standard anatomical features and the standard noise features; a low quality image module for generating low quality anatomical structure features and low quality noise features from the low quality image and for reconstructing the low quality image from the low quality anatomical structure features and the low quality noise features; a loss calculation module that calculates a loss metric based at least in part on 1) a comparison of the reconstructed standard image to the standard image, and 2) a comparison of the reconstructed low quality image to the low quality image; wherein the loss criterion is incorporated into a loss function for adjusting the standard image module and the low quality image module using machine learning, and when the standard image module is provided with the low quality anatomical structure features, the standard image module outputs a reconstructed standard transferred image including the low quality anatomical structure features and a noise level lower than the noise level represented by the low quality noise features.

2. the standard image module has a standard anatomical structure encoder, a standard noise encoder, and a standard generator, and when receiving the standard image, the standard anatomical structure encoder outputs the standard anatomical structure features, the standard noise encoder outputs the standard noise features, and the standard generator reconstructs the standard image from the standard anatomical structure features and the standard noise features; the low quality image module comprises a low quality anatomical structure encoder, a low quality noise encoder, and a low quality generator, and when receiving the low quality image, the low quality anatomical structure encoder outputs the low quality anatomical structure features, the low quality noise encoder outputs the low quality noise features, and the low quality generator reconstructs the low quality image from the low quality anatomical structure features and the low quality noise features; the loss calculation module calculates a loss metric for the standard generator based at least in part on a comparison of the reconstructed standard image and the standard image; the loss calculation module calculates a loss metric for the loss generator based at least in part on a comparison of the reconstructed loss image and the loss image; the loss calculation module calculates a loss metric for the standard anatomical structure encoder based on a comparison with a segmentation label for the standard image; the loss calculation module calculates a loss metric for the reduced quality anatomical structure encoder based on a comparison with the output of the standard anatomical structure encoder. The system of claim 1 .

3. The system of claim 2 , wherein the loss metric of the low-quality anatomical encoder is an adversarial loss metric.

4. The system of claim 1 , further comprising a segmentation network, wherein a segmentation mask of the reconstructed standard image is evaluated based on a comparison with a segmentation label for the standard image.

5. 5. The system of claim 4, wherein upon providing the standard anatomical features to a low-quality generator, the low-quality generator outputs a reconstructed low-quality transition image, the reconstructed low-quality transition image including the standard anatomical features and a noise level higher than a noise level represented by the standard noise features, and a segmentation mask of the reconstructed low-quality transition image is evaluated based on a comparison with the segmentation label for the standard image.

6. The system of claim 1 , wherein a loss metric for the reconstructed standard transferred image is evaluated based on a comparison with the standard image reconstruction, and the loss metric for the reconstructed standard transferred image is an adversarial loss metric.

7. The system of claim 1 , wherein the standard image module and the low quality image module are trained simultaneously.

8. The system of claim 1 , wherein the standard image module is trained before training the low quality image module, and values ​​of variables created during training of the standard image module are kept constant during training of the low quality image module.

9. 10. The system of claim 8, wherein after training the standard image module and the low quality image module, the system further trains a standard generator while keeping constant values ​​for a standard anatomical structure coder, a standard noise coder, and a low quality anatomical structure coder.

10. The system of claim 1 , wherein the standard anatomical feature and the low quality anatomical feature each correspond to a single anatomical structure.

11. 1. A method for denoising a medical image, the method comprising: generating standard anatomical features and standard noise features from the standard image by a standard image module; reconstructing, by the standard image module, the standard image from the standard anatomical features and the standard noise features; generating, by a low quality image module, low quality anatomical features and low quality noise features from the low quality image; reconstructing, by the low quality image module, the low quality image from the low quality anatomical structure features and the low quality noise features; calculating, by a loss calculation module, a loss metric based at least in part on: 1) a comparison of the reconstructed standard image to the standard image; and 2) a comparison of the reconstructed low quality image to the low quality image. wherein the loss criterion is incorporated into a loss function for adjusting the standard image module and the low quality image module using machine learning, and when the standard image module is provided with the low quality anatomical structure features, the standard image module outputs a reconstructed standard transferred image including the low quality anatomical structure features and a noise level lower than the noise level represented by the low quality noise features.

12. the standard image module has a standard anatomical structure encoder, a standard noise encoder, and a standard generator, and when receiving the standard image, the standard anatomical structure encoder outputs the standard anatomical structure features, the standard noise encoder outputs the standard noise features, and the standard generator reconstructs the standard image from the standard anatomical structure features and the standard noise features; the low quality image module has a low quality anatomical structure encoder, a low quality noise encoder, and a low quality generator, and when receiving the low quality image, the low quality anatomical structure encoder outputs the low quality anatomical structure features, the low quality noise encoder outputs the low quality noise features, and the low quality generator reconstructs the low quality image from the low quality anatomical structure features and the low quality noise features; the loss calculation module calculates a loss metric for the standard generator based at least in part on a comparison of the reconstructed standard image and the standard image; the loss calculation module calculates a loss metric for the loss generator based at least in part on a comparison of the reconstructed loss image and the loss image; the loss calculation module calculates a loss metric for the standard anatomical structure encoder based on a comparison with a segmentation label for the standard image; the loss calculation module calculates a loss metric for the reduced quality anatomical structure encoder based on a comparison with the output of the standard anatomical structure encoder. The method of claim 11.

13. The method of claim 12 , wherein the loss metric of the low-quality anatomical encoder is an adversarial loss metric.

14. The method of claim 11 further comprising the steps of: generating, in a segmentation network, a segmentation mask of the reconstructed standard image; and evaluating the segmentation mask based on a comparison with a segmentation label for the standard image.

15. 15. The method of claim 14, wherein upon providing the standard anatomical features to a low-quality generator, the low-quality generator outputs a reconstructed low-quality transition image, the reconstructed low-quality transition image including the standard anatomical features and a noise level higher than a noise level represented by the standard noise features, and a segmentation mask of the reconstructed low-quality transition image is evaluated based on a comparison with the segmentation label for the standard image.

16. The method of claim 11 , wherein a loss metric for the reconstructed standard transferred image is evaluated based on a comparison with the standard image reconstruction, and the loss metric for the reconstructed standard transferred image is an adversarial loss metric.

17. The method of claim 11 , wherein the standard image module and the low quality image module are trained simultaneously.

18. The method of claim 11 , wherein the standard image module is trained before training the low quality image module, and values ​​of variables created during training of the standard image module are kept constant during training of the low quality image module.

19. 20. The method of claim 18, after training the standard image module and the low quality image module, the method further trains a standard generator while keeping a standard anatomical structure coder, a standard noise coder, and a low quality anatomical structure coder constant.

20. The method of claim 11 , wherein the standard anatomical feature and the low quality anatomical feature each correspond to a single anatomical structure.

Images