Controllable reference-free denoising of medical images

Abstract

The invention provides a method for training a denoising machine learning model. The method comprises the following steps of: retrieving a target image data frame; the target image data frame is one image data frame in a sequence of image data containing an object. The method also includes retrieving at least one previous image data frame and at least one subsequent image data frame of the sequence. The content of the previous image data frame and the content of the subsequent image data frame are partially overlapped with the content of the target image data frame. The method also includes retrieving acquisition parameters associated with image data frames in the sequence, and generating a prediction of a de-noised target image data frame based on the previous image data frame and the subsequent image data frame. The method trains a machine learning algorithm based on prediction, and trains a noise model based on acquisition parameters. A system and a denoising method are also provided.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for training and using a neural network model to denoise an image without using a reference image during training. In particular, the present disclosure relates to systems and methods for training and using such a neural network model in the context of computed tomography (CT) images. Background Art

[0002] Traditionally, in imaging modalities such as computed tomography, there are some effects in the acquisition physics or reconstruction that lead to artifacts (e.g., noise) in the final image. Although low-dose computed tomography (LDCT) is widely used in radiology, the reduction in X-ray dose increases the noise level, which affects diagnostic performance.

[0003] The best performing denoising techniques are based on deep learning models trained on pairs of clean-noise images of the same anatomical structure. Thus, to train a denoising algorithm that utilizes machine learning, such as a neural network model, pairs of noisy and noise-free image samples are typically fed to the neural network model, and the network attempts to minimize a cost function by denoising the noisy image to recover the corresponding noise-free ground-truth image.

[0004] Noise-free or clear images are difficult to obtain because they usually require high radiation doses to produce high-quality images. Acquiring such data in a clinical setting typically takes at least twice as long as a conventional examination and may increase the patient's radiation exposure. In addition, even with this approach, paired images may not be perfectly aligned because patient movement can cause separate artifacts to appear in the denoised image. Therefore, image pairs that are useful for training purposes may be difficult to obtain, especially in a clinical setting.

[0005] Currently, paired datasets are created by adding synthetic noise to real high-quality images. However, such noise models can fail when the underlying mathematical assumptions do not match the acquisition settings. Therefore, a denoising technique is needed that can be trained without the availability of clear high-dose images.

[0006] Furthermore, the training of neural network models used for denoising is often specific to a particular image quality, where the quality of the image depends on the image acquisition parameters. Thus, an image acquired with half the typical radiation dose will have a different amount of noise than an image acquired with a quarter of the typical radiation dose.

[0007] Therefore, neural network models used for denoising are usually specific to a set of acquisition parameters, and changes in the parameters used to acquire the image may result in changes in the form or amount of artifacts in the corresponding image. Therefore, when a denoising model that denoises images obtained using a first set of parameters is applied to images acquired using different acquisition parameters (such as reducing the radiation dose in the case of CT scans), its effect is poor. Therefore, the noise level of the images used during training limits the generalization ability of the trained denoising model. For example, if the training set contains images with a moderate noise level, the algorithm will not be able to accurately denoise images with a high noise level. Typically, the application of the algorithm results in over-smoothing or incomplete noise removal.

[0008] In CT imaging, multiple factors, including peak tube voltage in kilovolt peak (kVp), tube current in milliampere-seconds (mAs), slice thickness, column position, and patient size can affect the noise level in the reconstructed image. As a result, changing any of these imaging parameters can result in different noise levels or different artifact profiles, traditionally preventing the generalization of denoising capabilities and requiring different models to denoise images acquired with these different imaging parameters. This limits the applicability of CNN-based methods in practical denoising.

[0009] Supervised denoising methods rely on the properties of the data used for training. Therefore, the degree of denoising can only be adjusted by selecting appropriate data for training or other post-processing techniques (e.g., a weighted combination of the initial noisy image and its noisy residual). This technique is called overcorrection.

[0010] Current self-supervised denoising methods do not require low-noise reference images, but they have several weaknesses. They do not use the information contained in a sequence of connected images (e.g., CT projections). Instead, each image is denoised individually, which results in suboptimal image quality and longer run times. Some of the methods require additional data, which complicates the performance of the denoising method. Furthermore, like supervised methods, they do not allow for adjustment of the denoising level.

[0011] Therefore, there is a need for a controllable denoising method that can be trained without clean reference images and can be used to denoise images acquired with a variety of imaging parameters. There is also a need for a single trained model that can be used to denoise images with different noise levels, including CT images acquired with lower radiation doses than the images used for training. Summary of the invention

[0012] A system and method for denoising medical images are provided, wherein no reference images to be used as clean images are used for training. The proposed method relies instead on the similarities present in the sequence of connected images and on the modeled distribution of clean and noisy data. Special noise modules may allow the denoising algorithm to generalize better to different noise levels and to adjust the degree of denoising by manually or automatically adjusting interpretable parameters. The possibility to adjust the noise module and to train only on noisy data makes the proposed reference-free denoising method quite flexible for application in clinical settings.

[0013] In some embodiments, a method for training a machine learning model for denoising is provided. The method includes retrieving a target image data frame. The target image data frame is an image data frame in a sequence of image data frames containing image data of an object.

[0014] The method further includes retrieving at least one previous image data frame in the sequence of image data frames that precedes the target image data frame in the sequence. The content of the at least one previous image data frame at least partially overlaps with the content of the target image data frame. The method further includes retrieving at least one subsequent image data frame in the sequence of image data frames that follows the target image data frame in the sequence. The content of the at least one subsequent image data frame at least partially overlaps with the content of the target image data frame.

[0015] The method also includes retrieving acquisition parameters associated with acquisition of an image data frame in the sequence of image data frames and generating a prediction for a denoising target image data frame based on the at least one previous image data frame and the at least one subsequent image data frame.

[0016] Then, the method trains a machine learning algorithm to denoise the target image data frame based on the prediction of the denoised target image data frame and the noise model based on the acquisition parameters.

[0017] In some embodiments, the prediction for the denoised target image data frame is based on an estimate of the mean and standard deviation of the representation of at least one anatomical feature extracted from each of at least one previous image data frame and at least one subsequent image data frame. The representations from at least one previous image data frame and at least one subsequent image data frame are then fused to form a prediction for the denoised target image data frame.

[0018] In some such embodiments, a convolutional memory unit is used to transfer the representation of the at least one anatomical feature between frames. Such a convolutional memory unit may be a convolutional long short-term memory unit for carrying information between frames of a sequence of image data frames.

[0019] In some embodiments, the at least one previous image data frame is a plurality of image data frames in a sequence of image data frames preceding the target image data frame. Similarly, the at least one subsequent image data frame may be a plurality of image data frames in a sequence of image data frames following the target image data frame.

[0020] In some embodiments, a prediction of a denoised target image data frame is output by a trained convolutional neural network provided using the at least one previous image data frame and the at least one subsequent image data frame.

[0021] In some embodiments, the loss function used to train the machine learning algorithm is based on the expected noise distribution based on the acquisition parameters. Such a noise distribution may be based on a Poisson-Gaussian distribution.

[0022] In some such embodiments, the training method is repeated for a sequence of projection frames obtained using different acquisition parameters, and tuning variables are extracted or generated for the machine learning algorithm based on the results associated with the different acquisition parameters. In some such embodiments, the tuning variables are trained based on changes in tube current associated with the acquisition of an associated sequence of image data frames.

[0023] In some other such embodiments, the tuning variable is a scaling factor that determines how much noise the machine learning algorithm identifies to remove.

[0024] In some embodiments, the acquisition parameters are extracted from a DICOM file associated with the sequence of projection frames. In some embodiments, the machine learning algorithm is a convolutional neural network.

[0025] In some embodiments, the image data is CT image data, and each image data frame in the sequence of image data frames is a projection frame. Each projection frame contains imaging data of the same object acquired from different angles.

[0026] A denoising method may also be provided in which a machine learning algorithm is trained in the manner discussed above. The method then retrieves a sequence of noisy image data frames including a noisy target image data frame to be denoised. The method then includes selecting values ​​for tuning variables based on acquisition parameters of the sequence of noisy image data frames.

[0027] The method then includes applying the trained machine learning algorithm to the sequence of noisy image data frames using the selected values ​​for the tuning variables. The method then generates a first denoised image data frame based on an estimate of a mean and a standard deviation of the sequence of noisy image data frames, a noise distribution based on acquisition parameters of the sequence of noisy image data frames, and a noisy target image data frame.

[0028] In some such embodiments, the tuning variables are trained to correspond to acquisition parameters in the training data, and the selected values ​​for the tuning variables differ from actual values ​​of the corresponding acquisition parameters for the noisy image data frames. The machine learning algorithm then identifies more noise in the noisy image data frames when using the selected values ​​than when using the actual values.

[0029] In some embodiments, the denoising method further includes: reconstructing an image based on a plurality of denoised image data frames including the first denoised image data frame, and outputting the reconstructed image to a user.

[0030] A machine learning training system is also provided, which includes a memory storing a plurality of instructions and a processor circuit coupled to the memory and configured to execute the instructions to implement the training method discussed above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a system schematic diagram according to an embodiment of the present disclosure.

[0032] Figure 2 An exemplary imaging device according to one embodiment of the present disclosure is illustrated.

[0033] Figure 3 A process for training a model for denoising an image according to the present disclosure is illustrated.

[0034] Figure 4 The use of a model for denoising an image in accordance with the present disclosure is illustrated.

[0035] Figure 5A A flow chart illustrating a method for training a model for denoising an image according to the present disclosure is provided.

[0036] Figure 5B A flow chart illustrating an image denoising method according to the present disclosure is provided.

[0037] Figure 6 A schematic diagram of the use of tuning variables in a model for denoising an image according to the present disclosure is provided. DETAILED DESCRIPTION

[0038] The description of the illustrative embodiments according to the principles of the present invention should be read in conjunction with the accompanying drawings, which should be considered as part of the entire written description. In the description of the embodiments of the present disclosure disclosed herein, any reference to direction or orientation is only for the convenience of description and is not intended to limit the scope of the present disclosure in any way. Relative terms such as "below", "above", "horizontal", "vertical", "above", "below", "up", "down", "top" and "bottom" and their derivatives (such as "horizontal", "down", "up", etc.) should be interpreted as referring to the orientation shown in the drawings described at the time or discussed. These relative terms are only for the convenience of description and do not require the device to be constructed or operated in a specific direction unless explicitly stated. Terms such as "attach", "fix", "connect", "couple", "interconnect" refer to a relationship in which, unless otherwise explicitly stated, the structure is fixed or connected together directly or indirectly through an intermediate structure, as well as a movable or rigid connection or relationship. In addition, the features and advantages of the present invention will be explained by reference to exemplary embodiments. Therefore, the present disclosure should obviously not be limited to such exemplary embodiments, which show certain possible non-limiting combinations of features that may exist alone or in other combinations of features; the scope of the present disclosure is defined by the appended claims.

[0039] The present disclosure describes the best mode or modes currently contemplated for implementing the present disclosure. This specification should not be construed as limiting, but rather provides examples of the present disclosure by reference to the accompanying drawings, which are presented for illustrative purposes only, so as to suggest the advantages and configurations of the present disclosure to those of ordinary skill in the art. In the various views of the accompanying drawings, the same reference numerals represent the same or similar components.

[0040] It is noteworthy that the disclosed embodiments are merely examples of the numerous advantageous uses of the innovative teachings herein. In general, statements in the specification of the present application do not necessarily limit any claimed disclosure. In addition, some statements may apply to some inventive features but not to other inventive features. In general, unless otherwise indicated, singular elements may be plural, and vice versa, without loss of generality.

[0041] Generally, images acquired for use in a medical setting require some processing to remove image noise. Such denoising is necessary in a medical setting because the images may be used for diagnosis and treatment, as the precision and accuracy of such images can improve their usefulness. Such denoising is often achieved using machine learning based algorithms, such as convolutional neural networks (CNNs).

[0042] CNNs used for denoising require training to correctly identify noise in medical imaging. Traditionally, such CNNs are trained using pairs of images, where each pair contains a first "noisy" image and a second clean image, where the clean image is used as the ground truth. The CNN is then trained to compare the noisy image with the clean image and process the noisy image so that the output image approximates the clean image. In order to train a CNN in this way, a training set containing a large number of image pairs is required. In addition, in order to obtain consistent results, the training set usually includes images acquired using a consistent set of acquisition parameters, and the resulting CNN is usually limited to new images acquired using the same or similar acquisition parameters.

[0043] As described above, generating such a training set is very difficult and time consuming. Therefore, the systems and methods described herein do not require paired images, but rely on adjacent frames in a sequence of image data frames. Such methods allow the image data in the target frame to be recreated based on the image data of adjacent frames in the sequence of image data frames. These adjacent frames can provide different representations of the content of the target frame with different noise distributions. By combining multiple adjacent frames, predictions can then be generated for the target frame. Such predictions may be estimates of the mean and standard deviation of a denoised version of the target frame. The noisy target frame can then be used as the real basis data to train the CNN, where the loss function of the training depends on the predictions generated based on multiple adjacent frames.

[0044] For example, when training a CNN based on an image sequence, the noise in the CT projections depends on the technical acquisition parameters that can be extracted from the acquisition description from, for example, a DICOM file. The method can then consider a set of acquisition parameters for acquiring a sequence of image data frames. The acquisition parameters can be used to model the expected noise distribution, which is then used to generate a prediction for the target frame. Alternatively, or in combination with this approach, a model of the expected noise distribution can be used to further inform the loss function in conjunction with the prediction for the target frame.

[0045] Using acquisition parameters can make the model more adaptive and, therefore, less dependent on the properties of the data set used during training. The systems and methods described herein can then be used to create a tunable CNN to denoise image data. In such an embodiment, the denoising model can create an estimate of the noise variance based on the acquisition parameters. The noise estimation module can then be trained using the CNN to account for the acquisition parameters in the denoising process. By training the method using a sequence of frames acquired using different acquisition parameters, the CNN can then be trained to account for the acquisition parameters in the denoising process, allowing the CNN to function over a range of acquisition parameters.

[0046] Furthermore, tunable variables can be extracted from the model by which the noise estimate based on the acquisition parameters can be manipulated. During CNN-based denoising, the tunable variables can be used to increase or decrease the amount of noise to be removed from the image.

[0047] Figure 1 FIG. 1 is a schematic diagram of a system 100 according to an embodiment of the present disclosure. As shown in the figure, the system 100 generally includes a processing device 110 and an imaging device 120 .

[0048] The processing device 110 may apply a processing routine to images or measurement data, such as projection data, received from the image device 120. The processing device 110 may include a memory 113 and a processor circuit 111. The memory 113 may store a plurality of instructions. The processor circuit 111 may be coupled to the memory 113 and may be configured to execute instructions. The instructions stored in the memory 113 may include a processing routine, and data associated with the processing routine, such as a machine learning algorithm and various filters for processing images.

[0049] The processing device 110 may also include an input 115 and an output 117. The input 115 may receive information, such as images or measurement data, from the imaging device 120. The output 117 may output information, such as filtered images, to a user or user interface device. The output may include a monitor or display.

[0050] In some embodiments, processing device 110 may be directly associated with imaging device 120. In alternative embodiments, processing device 110 may be distinct from imaging device 120, such that processing device 110 receives image or measurement data through a network or other interface at input 115 for processing.

[0051] In some embodiments, the imaging device 120 may include an image data processing device, and a spectral or conventional CT scanning unit for generating CT projection data when scanning an object (eg, a patient).

[0052] Figure 2 An exemplary imaging device 200 is illustrated in accordance with one embodiment of the present disclosure. It should be understood that while a CT imaging device is illustrated and the following discussion is generally conducted in the context of CT images, similar methods may also be applied in the context of other imaging devices, and the images to which these methods may be applied may be acquired in a variety of ways.

[0053] In an imaging device according to an embodiment of the present disclosure, the CT scanning unit may be adapted to perform multiple axial scans and / or spiral scans on an object to generate CT projection data. Thus, the multiple scans may be recorded as a series of scans, each scan containing image data and recorded as an image data frame. In an imaging device according to an embodiment of the present disclosure, the CT scanning unit may include an energy-resolving photon counting image detector. The CT scanning unit may include a radiation source that emits radiation to pass through an object when acquiring projection data.

[0054] exist Figure 2 In the example shown in , a CT scanning unit 200, such as a computed tomography (CT) scanner, may include a fixed gantry 202 and a rotating gantry 204, which may be rotatably supported by the fixed gantry 202. The rotating gantry 204 may rotate around a longitudinal axis around an examination region 206 of an object when acquiring projection data. The CT scanning unit 200 may include a support 207 for supporting a patient in the examination region 206 and configured to move the patient through the examination region during imaging.

[0055] The CT scanning unit 200 may include a radiation source 208, such as an X-ray tube, which may be supported by the rotating gantry 204 and configured to rotate with the rotating gantry 204. The radiation source 208 may include an anode and a cathode. A source voltage applied between the anode and the cathode may accelerate the movement of electrons from the cathode to the anode. The electron flow may provide a current from the cathode to the anode to generate radiation that passes through the examination region 206.

[0056] The CT scanning unit 200 may include a detector 210. The detector 210 may subtend an angular arc relative to the radiation source 208, opposite the examination region 206. The detector 210 may include a one-dimensional or two-dimensional array of pixels, such as direct conversion detector pixels. The detector 210 may be adapted to detect radiation that passes through the examination region 206 and to generate a signal indicative of its energy.

[0057] Typically, the CT scanning unit acquires a sequence of projection frames as the rotating gantry 204 rotates around the patient. Thus, each acquired frame of projection data overlaps adjacent frames to some extent, depending on the amount of gantry movement between frames, and includes image data of the same object (i.e., the patient) acquired at different angles. In the context of the present application, a frame may contain different types of data, such as raw signal data, tomographic projection data, or image data at various stages of processing. All of these types of data contain image data, and therefore each frame in the sequence of frames contains some type of image data.

[0058] Thus, in some embodiments, the imaging data in the frames may be processed to varying degrees before the data is transmitted to the processing device 110. Thus, the CT scanning unit 200 may include generators 211 and 213. The generator 211 may generate tomographic projection data 209 based on signals from the detector 210. The generator 213 may receive the tomographic projection data 209 and, in some embodiments, generate a sequence of raw image data frames 311 of the object based on the tomographic projection data 209. In some embodiments, the tomographic projection data 209 may be provided to the input 115 of the processing device 110, while in other embodiments, the sequence of raw image data frames 311 is provided to the input of the processing device.

[0059] As discussed in more detail below, in addition to the sequence of image data frames provided to the input 115 of the processing device 110, the image device 120 may also provide data defining acquisition parameters associated with the sequence of image frames. These acquisition parameters may include incident photon flux distribution at detector column positions and tube current and / or voltage used. For example, such acquisition parameters may be provided in the context of the Digital Imaging and Communications in Medicine (DICOM) standard and may be transmitted with the underlying sequence of image data frames. Thus, the acquisition parameters may be extracted from the DICOM file.

[0060] Figure 3 A process 300 for training a model for denoising an image according to the present disclosure is illustrated. Figure 4 The use of a model for denoising an image according to the present disclosure is illustrated 400. As shown, and as discussed in more detail below with respect to FIG. 5, a method for training a model will first retrieve or be provided with a sequence 310 of image data frames that can then be used to denoise any of those image data frames. The target image data frame 315 is then provided directly to a model training module 320, while a separate prediction module 330 predicts the content of the target image data frame.

[0061] The prediction module 330 receives image data from at least one image data frame before the target image data frame 315 and at least one image data frame after the target image data frame 315, and generates a prediction of the content of the target image data frame based on these frames. In doing so, the prediction module 330 extracts the content of the image data frame, such as anatomical features of the image data object, from the image data frame provided to it by the feature extractor module 340. The feature extractor module 340 can be a separate, separately trained learning algorithm, such as a CNN.

[0062] The content extracted from the image data frames is then considered across multiple frames using, for example, a convolutional memory unit such as a convolutional long short-term memory unit (ConvLSTM) 350. The extracted features are then combined along the time axis (at 360) and fused by a feature combination module 370.

[0063] The fused features may be anatomical features of the object of the image data and may be used to predict the content of the target image data frame that has been provided to the model training module 320. The output of the prediction module 330 is then a prediction of the target image data frame. For example, the prediction may be an estimate of the mean and standard deviation of the denoised target frame. The model training module 320 then trains a machine learning algorithm, such as a CNN, using the noisy target image data frame 315 as the basic truth. In the illustrated embodiment, the CNN trained by the model training module 320 is the denoising module 330.

[0064] In some embodiments, such as the illustrated embodiment, a noise estimation module 380 may be provided to predict the noise distribution in the prediction results. The noise estimation module 380 (discussed in detail below) may then provide a prediction in the form of a noise model based on acquisition parameters 390 associated with the sequence of image data frames 310, which may be in the form of a DICOM file or metadata associated with the underlying image data. As discussed in more detail below, the noise estimation module 380 itself may be a machine learning algorithm, such as a CNN, and the model training module 320 may also train the noise estimation module 380.

[0065] In such an embodiment, the predictions provided by the noise estimation module 380 are provided to the model training module 320 along with the predictions for the target image data frame in order to better define the loss function for using the target image data frame 315 as the true ground truth data.

[0066] Once trained, model 400 can be used as Figure 4 Thus, optionally, a target image data frame 410 and at least one previous image data frame and one subsequent image data frame extracted from a sequence of image data frames 415 may be provided to a denoising model 420, and a noise estimation module 430 may generate a noise estimate based on acquisition parameters 440, and the noise estimate may then be used by a prediction engine 470 to output a denoised image 450 derived from the target image data frame 410.

[0067] As discussed in detail below, the noise estimation module 430 may be provided with tuning variables 460 derived during training 300. Such tuning variables may be similar to the physical characteristics of the image acquisition, as recorded in the acquisition parameters 440, and may be user selectable. In this way, the user may select tuning variable values ​​that differ from the actual characteristics stored in the corresponding acquisition parameters 440 to scale the output of the noise estimation module. Alternatively, the tuning variables 460 may be general values ​​associated with the noise level in the model, and may be scaled without regard to the corresponding physical characteristics.

[0068] Figure 5A A flow chart illustrating a method for training a model for denoising an image according to the present disclosure is provided.

[0069] As shown in the figure, and as in Figure 3 As discussed in the context of , the method retrieves a target image data frame to be considered (500). The target image data frame is one of a sequence 310 of image data frames containing image data of an object. The target image data frame may be obtained from a CT scanning system (e.g., as described above with respect to Figure 2 The CT scanning system discussed herein extracts image data. In such an embodiment, the image data frame is a projection frame and is part of a CT projection frame sequence. Therefore, each projection frame generally includes imaging data of the same object acquired from different angles.

[0070] The method then proceeds to retrieving at least one previous image data frame in the sequence of image data frames that precedes the target image data frame in the sequence (retrieved at 500) (510). The content of the at least one previous image data frame at least partially overlaps with the content of the target image data frame. The method also retrieves at least one subsequent image data frame in the sequence of image data frames that follows the target image data frame in the sequence (520). The content of the at least one subsequent image data frame at least partially overlaps with the content of the target image data frame.

[0071] In this way, the method sequentially acquires image data frames preceding and following the target frame. The previous frame and the subsequent frame are usually adjacent frames, and in addition to at least partially overlapping with the target frame, they also overlap each other at least partially. In this way, the previous frame and the subsequent frame carry at least some of the same content. When the frame is described as overlapping, it should be understood that the overlap can be associated with the image data frame or the actual object of the image data frame or the image data frame. The image generated by the actual object. For example, when the image is acquired in a linear process, the image associated with the image data frame may include adjacent images with some overlaps and therefore have the same image content. However, in the case of axial or spiral acquisition images, such as CT projection frames, the image content will not overlap, but the object of the image will overlap because the image usually contains the same object acquired from different angles. When the angle between the acquired projections is small, the content of the image data frame can be said to overlap. For example, in some embodiments, as long as the same feature side is visible, the image data frame can be said to overlap, and in such embodiments, as long as the angle between the projections is less than 90 degrees, the image data frame can be said to overlap. In other embodiments, the degree of overlap may be measured by evaluating the difference between the images. For example, this may be achieved by using the structural similarity index measure (SSIM) or the mean square error (MSE).

[0072] In some embodiments, the method retrieves a plurality of previous image data frames and retrieves a plurality of image data frames in the sequence of image data frames that precede the target image data frame. Similarly, the method may retrieve a plurality of subsequent image data frames, thereby retrieving a plurality of image data frames in the sequence of image data frames that follow the target image data frame. Each frame retrieved in this manner may have at least some overlapping content, and the number of adjacent frames retrieved may depend on the degree of overlap in the content of the frames, and the computational and storage capabilities of the device implementing the method. These values ​​may also be adjustable by the user.

[0073] Although the flow chart illustrates the individual retrieval of each frame identified, it should be understood that the method may also retrieve a sequence of image data frames 310 from the imaging system as a file, or may access a database containing such a sequence, and the method would then identify the target image data frame 315 as well as previous and subsequent image data frames in that data.

[0074] The method then retrieves 530 acquisition parameters associated with the acquisition of the image data frames of the sequence of image data frames 310. As described above, such acquisition parameters may be provided as a DICOM file 390 associated with or extracted from the sequence of image data frames 310, or may be included as metadata in the image data frames themselves.

[0075] The method then proceeds to generate (540) a prediction of a denoised target image data frame based on at least one previous image data frame (retrieved at 510) and at least one subsequent image data frame (retrieved at 520). This prediction may take the form of an estimate of the mean and standard deviation of a clean version of the target data frame 315.

[0076] In some embodiments, the method generates a prediction for a denoised target image data frame based on a representation of at least one anatomical feature extracted from each of at least one previous image data frame and at least one subsequent image data frame (at 540). The representations may be extracted from the previous image data frame and the subsequent image data frame by the feature extractor module 340 (550), and the representations extracted from each image data frame may be fused (560) by the feature combiner module 370 to form a prediction for the denoised target image data frame.

[0077] In some such embodiments, a representation of at least one anatomical feature is transmitted between frames using a convolutional memory unit, which may be a convolutional long short-term memory unit 350 for carrying information between frames in a sequence of image data frames.

[0078] In some embodiments, a variety of machine learning algorithms, such as CNNs, may be implemented in different steps of the method. For example, the feature extractor module 340 may implement a CNN to extract features (at 550), such as anatomical features, from various image data frames. Similarly, the feature combination module 340 may implement a CNN to fuse feature representations together (at 560) to form a prediction. Similarly, a prediction of a denoised target image data frame may be generated and output by a single trained convolutional neural network provided with at least one previous image data frame and at least one subsequent image data frame. The prediction of the denoised target image data frame may take the form of a mean and standard deviation of a denoised version of the target frame.

[0079] The prediction of the denoised target image data frame based on the acquisition parameters (retrieved at 530), the target image data frame and the noise model are then provided to the self-supervised model training module 320, and the prediction module 330 (570) is trained to denoise the target image data frame 315 based on the provided data. The prediction module 330 can be a CNN. The training process can use the noisy target data frame 315 as a ground truth, and can use the prediction (generated at 540) as the basis of the loss function.

[0080] In some embodiments, the loss function used to train the machine learning algorithm is based on the noise distribution expected from the acquisition parameters. In such embodiments, after retrieving the acquisition parameters (at 530), a noise model including a prediction of the noise distribution is generated (580) at the noise estimation module 380 based on these parameters. This prediction of the noise distribution can be used to create a loss function for training (at 570), and can be trained or fine-tuned simultaneously through the training process.

[0081] The following will refer to Figure 6 Additional details related to the noise estimation module generating the noise model are discussed in more detail.

[0082] The training methods described herein are typically repeated for a large number of sequences of image data frames. Such repeated sequences may include sequences obtained using different acquisition parameters. Therefore, after training the algorithm using the sequence 310 of image data frames, the method determines whether there are additional sequences available for training (580). If so, the method continues to retrieve the target image data frame (at 500), retrieve the previous frame (at 510), and retrieve the subsequent frame (at 520) from the sequence 310 of additional image data frames.

[0083] Each sequence 310 of image data frames is typically provided with a different DICOM file 390 defining associated acquisition parameters. Thus, the method may retrieve such acquisition parameters at each iteration (at 530). In this way, the training of the algorithm (at 580) may take into account the acquisition parameters detailed in the DICOM file 390.

[0084] As part of the generation of the noise prediction (at 580), the generated model is typically based on details extracted from the DICOM file, as described below with reference to Figure 6 In this way, each detail extracted from the DICOM file effectively tunes the model output to the noise prediction (at 580). During training of the algorithm, the method can therefore associate at least one detail extracted from the DICOM file with a specified tuning or scaling variable, which can be manually tuned during model inference after the model is trained.

[0085] Thus, tuning variables may then be extracted or generated for the machine learning algorithm based on the results associated with the different acquisition parameters. As described above, the tuning variables may be trained based on, and thereby associated with, specific variables identified in the DICOM file. For example, the tuning variables may be based on changes in tube current associated with an image acquisition or a sequence of associated image data frames.

[0086] The tuning variable may be a scaling factor that determines how much of the noise identified by the machine learning algorithm is to be removed. Thus, the model output as noise prediction (580) may generate a prediction of the form that the noise may take, and the tuning variable may then determine how much of the noise in that form is to be removed from the target image data frame.

[0087] Such tuning and the expected form of the noise predicted by the noise estimation module 380 will be described below with respect to Figure 6 for a more detailed discussion.

[0088] Figure 5B A flow chart illustrating an image denoising method according to the present disclosure is provided.

[0089] Once the machine learning algorithm or CNN is trained and no additional sequences are provided for training (at 580), the machine learning algorithm can be used to denoise the image.

[0090] Thus, the method can retrieve a target noisy image data frame 410 to be denoised (590) as well as at least one previous image data frame and at least one subsequent image data frame (595). The method separately retrieves acquisition parameters associated with the noisy image data frame (600). Such acquisition parameters can be extracted from an associated DICOM file 440 associated with the image data frame 410 or extracted from the image data frame 410.

[0091] As mentioned above, in Figure 5A Tuning variables may be extracted or generated during the training process described above. Thus, during model inference, the tuning variables may be applied to or associated with acquisition parameters (610). Values ​​of the tuning variables may be selected and applied (at 610) based on the acquisition parameters of the noisy image data frame. For example, the tuning variables may be associated with details extracted from a DICOM file and may be set accordingly (at 610).

[0092] Once the tuning variables are set (at 610), the method may proceed to generate a noise prediction based thereon (620) and apply the trained machine learning denoising algorithm (630) to a sequence of noisy image data frames, optionally including the target noisy image data frame 410 and at least one previous image data frame and at least one subsequent image data frame. The method then generates estimates of the mean and standard deviation of the denoised target image data frame and operates on the estimates in combination with the noise prediction and the target image data frame using the selected values ​​for the tuning variables.

[0093] The method can then predict a clean image data frame of the target image data frame to generate a first denoised image data frame (640).

[0094] In some embodiments, the first denoised image data frame (generated at 640) can be initially reviewed before presentation to the user to determine whether the quality of the denoising method is satisfactory (650). If so, the first denoised image data frame (generated at 640) can be output to the user. If not, the reviewer can modify the tuning variables (at 610) to improve the quality of the first denoised image data frame. In some embodiments, the user reviews the first denoised image data frame themselves and determines whether to modify the applied tuning variables (at 610) and process the image data frame again.

[0095] As described above, the tuning variables may be trained to correspond to acquisition parameters in the training data. Thus, after modification (at 610), the selected values ​​for the tuning variables may differ from the actual values ​​of the corresponding acquisition parameters associated with the noisy image data frames. When the tuning variables act to apply the scaling factors, the selected variables may cause the machine learning algorithm to identify and remove more noise in the noisy image data frames when using the selected values ​​than when using the actual values ​​for the corresponding acquisition parameters.

[0096] For example, when the tuning variable is a proxy for acquisition tube current, lower tube currents generally correspond to higher noise levels in the image. Thus, if a higher value is used for the tuning variable (at 610), less noise is removed from the image data frame and the resulting data is more accurate. However, if a lower value is used as the tuning variable, the denoising will be more stringent and the resulting data will be cleaner, but at the expense of some anatomical detail that may be smoothed out. Thus, a user (e.g., a clinician) can determine whether a clearer or more accurate output is desired based on the specific diagnostic task at hand.

[0097] In some embodiments, the user may initially apply the selected tuning variables (at 610) to the method without first examining the outputs. This may remove more noise from the noisy image data frame being processed than the default values.

[0098] Once it is determined that the quality of the denoising method is satisfactory (at 650), the method can proceed to reconstructing the image from the image data file (660). Such reconstruction may be necessary, for example, when the image data frame is a projection frame of a CT scan that was processed prior to reconstruction. In some embodiments, the reconstruction of the image may be based on a plurality of denoised image data frames including the first denoised image data frame. Additionally, the reconstruction may be a volume or other combination of images rather than a single image. The method may then end by outputting the reconstructed image or volume to the user (670).

[0099] Figure 6 Picture shows Figure 3The noise estimation module 380 estimates the noise in the image.

[0100] The following discussion concerns noise in CT projections, but similar noise estimation methods can be implemented in other imaging modalities. The noise in CT projections can be approximated by a mixed Poisson-Gaussian distribution whose parameters can be obtained from the description of the acquisition process. This distribution has the form:

[0101]

[0102] Where p is the normalized projection data without noise, and T is the transmission data without noise. T can be represented by a Gaussian distribution. to approximate and the noise can be normally distributed To model,

[0103] is noisy transmission data,

[0104] λ is the number of incident photons,

[0105] is the electronic noise variance.

[0106] The electronic noise variance can be obtained by measuring the dark current, while the number of incident photons can be estimated by air scanning. However, these values ​​are generally not easy to obtain. Therefore, in the method described in this article, the noise estimation module 380 provided does not need to know these parameters accurately. Alternatively, they can be estimated by an optimization algorithm when training the main denoising model. Nevertheless, the theory helps to make a preliminary estimate. If the CT scanner uses bow-tie filtering and automatic exposure control, the noise parameter λ corresponding to the incident photon flux should depend on the detector column position 700 and the tube current 710. The trainable noise estimation module 380 then takes the CT acquisition attributes (for example from the above-mentioned DICOM file 390) as input and produces an estimate of the noise variance In our main embodiment, the noise estimation module is implemented as a small neural network that calculates the noise from the average μ of the clean transmission data. T 720, detector column position 700 and tube current (mAs) 710 used to predict noise variance.

[0107] The tube current 710 can then be used to extract the slope and bias coefficients, and the column positions 700 can be used to extract a prediction of the photon number distribution. The extracted slope and bias coefficients can then be used to scale the expected photon number distribution to derive a prediction of the number of incident photons λ. This method can estimate the electronic noise variance alone to derive a prediction of which can then be used to estimate the noise variance

[0108] The main denoising model 320 and the noise estimation module 380 can be trained together, or the parameters of the noise estimation module can be pre-calculated. The training is performed using the available noisy CT projections 310 and their acquisition parameters. If the test data has other noise properties (e.g., bow-tie filtering is not used), the noise estimation module 380 can be tuned according to these properties, while the main denoising model 320 can remain unchanged. In the case of such acquisition parameter changes, the denoising model 320 and the noise estimation module 380 can be retrained directly on the new data, or the noise estimation module 380 can be retrained separately.

[0109] It should be understood that although the methods described herein are described in the context of CT scan images, various imaging techniques, including various medical imaging techniques, may be considered, and images generated using various imaging techniques may be effectively denoised using the methods described herein. Since the methods include a discrete noise model, modifications to the noise model allow the methods to be used to process other medical image sequences. Thus, the relevant modalities may include, for example, dynamic positron emission tomography (PET), dynamic magnetic resonance (MR), microfluorescence images, ultrasound scans, and the like.

[0110] The method according to the present disclosure can be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or implemented in a combination of the two. The executable code of the method according to the present disclosure can be stored on a computer program product. Examples of computer program products include storage devices, optical storage devices, integrated circuits, servers, online software, etc. Preferably, the computer program product may include a non-transient program code stored on a computer-readable medium for executing the method according to the present disclosure when the program product is executed on a computer. In one embodiment, the computer program may include a computer program code, and the computer program code is suitable for executing all steps of the method according to the present disclosure when running on a computer. The computer program may be embodied on a computer-readable medium.

[0111] Although the present disclosure has been described in considerable detail with respect to several described embodiments, it is not intended that the present disclosure should be limited to any such details or embodiments or to any particular embodiment, but rather should be read with reference to the appended claims so as to provide such claims with the broadest possible interpretation in light of the prior art so as to effectively encompass the intended scope of the present disclosure.

[0112] All examples and conditional language described herein are intended to be used for teaching purposes, to help the reader understand the principles of the present disclosure and the concepts contributed by the inventor to promote this area, and should be interpreted as not being limited to these specifically described examples and conditions. In addition, all statements of principles, aspects and embodiments of the present disclosure and their specific examples mentioned herein are intended to cover their structural and functional equivalents. In addition, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, that is, the elements developed to perform the same function, regardless of their structure.

Claims

1. A method for training a machine learning model for denoising, comprising: Retrieving a target image data frame, the target image data frame being one of the sequence of image data frames containing imaging data of the object; retrieving at least one previous image data frame in the sequence of image data frames that precedes the target image data frame in the sequence, wherein content of the at least one previous image data frame at least partially overlaps content of the target image data frame; retrieving at least one subsequent image data frame in the sequence of image data frames that follows the target image data frame in the sequence, wherein content of the at least one subsequent image data frame at least partially overlaps content of the target image data frame; retrieving acquisition parameters associated with acquisition of an image data frame in said sequence of image data frames; generating a prediction of a denoised target image data frame based on the at least one previous image data frame and the at least one subsequent image data frame; Based on the prediction of the denoised target image data frame and based on the noise model of the acquisition parameters, a machine learning algorithm is trained to denoise the target image data frame.

2. The method according to claim 1, wherein: The prediction of the denoised target image data frame is based on an estimate of a mean and a standard deviation of a representation of at least one anatomical feature extracted from each of the at least one previous image data frame and the at least one subsequent image data frame, and wherein the representations from the at least one previous image data frame and the at least one subsequent image data frame are fused to form the prediction of the denoised target image data frame.

3. The method according to claim 2, wherein: The representation of the at least one anatomical feature is transferred between frames using a convolutional memory unit.

4. The method according to claim 3, wherein: The convolutional memory unit is a convolutional long short-term memory unit, which is used to carry information between the frames of the image data frame sequence.

5. The method according to claim 2, wherein: The at least one previous image data frame is a plurality of image data frames in the sequence of image data frames that precede the target image data frame, and wherein the at least one subsequent image data frame is a plurality of image data frames in the sequence of image data frames that follow the target image data frame.

6. The method according to claim 2, wherein: The prediction of the denoised target image data frame is output by a trained convolutional neural network provided using the at least one previous image data frame and the at least one subsequent image data frame.

7. The method according to claim 1, wherein: The loss function used to train the machine learning algorithm is based on the noise distribution expected according to the acquisition parameters.

8. The method according to claim 7, wherein: The training method is repeated for sequences of projection frames obtained using different acquisition parameters, and wherein tuning variables are extracted or generated for the machine learning algorithm based on the results associated with the different acquisition parameters.

9. The method according to claim 8, wherein: The tuning variables are trained based on changes in tube current associated with acquisition of an associated sequence of image data frames.

10. The method according to claim 8, wherein: The tuning variable is a scaling factor that determines how much noise identified by the machine learning algorithm is removed.

11. The method according to claim 7, wherein: The expected noise distribution is based on a Poisson-Gaussian distribution.

12. The method according to claim 1, wherein: The acquisition parameters are extracted from a DICOM file associated with the sequence of projection frames.

13. The method according to claim 1, wherein: The machine learning algorithm is a convolutional neural network.

14. The method according to claim 1, wherein: The imaging data is CT imaging data, and wherein each image data frame in the sequence of image data frames is a projection frame, and wherein each projection frame comprises imaging data of the same object acquired from a different angle.

15. A denoising method, comprising: Execute the method according to claim 8; retrieving a sequence of noisy image data frames including a noisy target image data frame to be denoised; selecting a value for the tuning variable based on acquisition parameters of the sequence of noisy image data frames; applying the trained machine learning algorithm to the sequence of noisy image data frames using the selected values ​​for the tuning variables; A first denoised image data frame is generated based on a mean and a standard deviation estimated based on the sequence of noisy image data frames, a noise distribution based on the acquisition parameters of the sequence of noisy image data frames, and the noisy target image data frame.

16. The denoising method according to claim 15, wherein: The tuning variables are trained to correspond to acquisition parameters in training data, and wherein the values ​​selected for the tuning variables are different from actual values ​​of the corresponding acquisition parameters for the noisy image data frames, and wherein the machine learning algorithm identifies more noise in the noisy image data frames when using the selected values ​​than when using the actual values.

17. A machine learning training system comprising: a memory storing a plurality of instructions; as well as A processor circuit is coupled to the memory and is configured to execute instructions to perform the following operations: retrieving a plurality of image data frames comprising a sequence of image data frames, the sequence of image data frames containing imaging data of the object; identifying a target image data frame in the sequence of image data frames; generating a prediction of a denoised target image data frame based on at least one previous image data frame in the sequence of image data frames that precedes the target image data frame in the sequence and at least one subsequent image data frame in the sequence of image data frames that follows the target image data frame in the sequence, wherein each of the at least one previous image data frame and the at least one subsequent image data frame at least partially overlaps with the target image data frame; retrieving acquisition parameters associated with acquisition of said image data frame in said sequence of image data frames; Based on the prediction of the denoised target image data frame and based on the noise model of the acquisition parameters, a machine learning algorithm is trained to denoise the target image data frame.

18. The system of claim 17, wherein: The loss function used to train the machine learning algorithm is based on the noise distribution expected according to the acquisition parameters.

19. The system of claim 18, wherein: The training method is repeated for sequences of image data frames obtained using different acquisition parameters, and wherein tuning variables are extracted or generated for the machine learning algorithm based on the results associated with the different acquisition parameters.

20. The system of claim 18, wherein: The imaging data is CT imaging data, and wherein each image data frame in the sequence of image data frames is a projection frame, and wherein each projection frame comprises imaging data of the same object acquired from a different angle.

Images