Cone beam artifact reduction

Abstract

Systems and methods for training machine learning models to reduce artifacts are provided. Such a method includes retrieving a three-dimensional digital phantom reconstructed from CT imaging data. The method then selects a first Z position along the central axis and simulates a first set of forward projections from the digital phantom acquired along the axial trajectory at the first Z position along the central axis. The first set of forward projections has a first analog collimation in the axial direction. The method then reconstructs a first analog image from the first set of forward projections, and identifies a plurality of secondary Z-positions along the central axis other than the first Z-position. For each of the first Z-position itself and the secondary Z-position, the method then simulates a set of secondary forward projections from the digital phantom acquired along the respective axial trajectory at the respective secondary Z-position.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for training and using neural network models to reduce artifacts in cone beam computed tomography (CT) images. Specifically, the present disclosure relates to systems and methods for training and using 3D neural network models to correct artifacts in the context of cone beam derived CT images. Background Art

[0002] Conventionally, in imaging modalities such as computed tomography, there are effects in the acquisition physics or reconstruction that result in noise or artifacts in the final image. To train a denoising or artifact reduction algorithm using machine learning (such as a neural network model), pairs of noisy and noise-free image samples or artifact-prone and clean image samples are typically presented to the neural network model, and the network attempts to minimize a cost function by denoising or removing artifacts from the sample noisy or artifact-prone images to recover the corresponding clean ground truth image.

[0003] Noise-free images or clean images are difficult to obtain as they usually require high radiation doses to produce high-quality images. Therefore, it may be difficult to obtain image pairs that can be used for training purposes, especially in clinical settings. In addition, some types of image artifacts have a considerable spatial extent, requiring a large amount of background data to classify and remove such artifacts.

[0004] As an example, cone beam computed tomography (CBCT) is an imaging category that plays an important and growing role in clinical applications, but suffers from significant artifacts. Artifacts associated with cone beam CT imaging are often in the form of large streaks, which require both image and model background to be consistently identified and corrected.

[0005] These cone beam artifacts arise due to the data insufficiency inherent in axial data acquisition and become more pronounced as coverage along the Z-axis increases. In modern CBCT, there is a trend to increase the cone angle, which increases the Z-axis coverage in the scan. This trend exacerbates the artifacts in such images and makes the problem of correcting such artifacts more challenging.

[0006] Many methods have been proposed to solve this problem. In addition to requiring changes to the hardware or changing the method of data acquisition, there are several software-based methods. For example, iterative reconstruction or second-pass methods can be used, which utilizes computationally expensive forward and back projections.

[0007] There have been several approaches that aim to use deep learning to solve the CBCT artifact correction problem. However, such approaches rely on 2D neural networks or pseudo 3D networks implemented on 3D data. Such approaches usually require large datasets to be available and / or are computationally intensive.

[0008] Therefore, in order to solve the cone beam artifact problem using traditional methods, it is necessary to change the hardware or use computationally intensive forward and back-projection operations. Existing AI methods do not solve the problem of 3D neural networks. Instead, they utilize 2D neural networks or apply pseudo-3D methods.

[0009] There is a need for a deep learning based approach that can be more easily trained and can directly address cone beam artifacts using a 3D convolutional neural network (CNN). There is also a need for an approach that can generalize across a variety of CBCT cone angles and spiral artifacts. Summary of the invention

[0010] Systems and methods for training machine learning models to reduce artifacts are provided. Such methods include first retrieving a three-dimensional digital phantom reconstructed from computed tomography (CT) imaging data. The CT imaging data includes projection data acquired from multiple angles around a central axis. In some embodiments, the digital phantom is reconstructed by spiral scanning.

[0011] Then, the method selects a first Z position along the central axis and simulates a first set of forward projections from the digital phantom acquired along an axial trajectory at the first Z position along the central axis. The first set of forward projections has a first simulated collimation in the axial direction.

[0012] The method then reconstructs a first simulated image from the first set of forward projections. The first simulated image includes a three-dimensional volume surrounding a first segment of the central axis containing the first Z position. The method then identifies a first plurality of secondary Z positions along the central axis in addition to the first Z position within the first segment of the central axis.

[0013] For the first Z position itself and each of the first plurality of secondary Z positions, the method then simulates a first set of secondary forward projections from the digital phantom acquired along a corresponding axial trajectory at the corresponding secondary Z position. The first set of secondary forward projections has a second simulated collimation in the axial direction that is less than the first simulated collimation.

[0014] The method then reconstructs the forward projections associated with the first Z position and each of the first plurality of secondary Z positions into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis.

[0015] The method then combines the two-dimensional images associated with the first Z position and each of the first plurality of secondary Z positions to create a second simulated image comprising a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image.

[0016] Then, the method trains a machine learning algorithm by providing the first simulated image as a sample artifact-prone image and providing the second simulated image as a ground truth. The machine learning algorithm may be a three-dimensional convolutional neural network (CNN).

[0017] In some embodiments, the first segment of the central axis is centered at the first Z position.

[0018] The first simulated image may be reconstructed using a three-dimensional filtered back projection process, and the two-dimensional image corresponding to an axial slice of the digital phantom may be reconstructed using a two-dimensional filtered back projection process.

[0019] In some embodiments, each of the first simulated image and the second simulated image can be segmented into three-dimensional tiles. Then, each tile of the first simulated image can have a corresponding tile of the second simulated image, and the corresponding tiles can be provided to the machine learning algorithm.

[0020] In some embodiments, the machine learning algorithm comprises at least one first convolution step applied to each tile of the first simulated image provided, followed by at least one downsampling operation. At least one additional convolution step may then be applied after the downsampling, and the downsampled tiles may then be upsampled after the at least one additional convolution step. The upsampled tiles may then be cascaded with the output of the first convolution step. In such an embodiment, the machine learning algorithm may be constructed as a three-dimensional U-net model, and each tile of the first simulated image may then be provided to the U-net model, and the output may then be compared with the corresponding tile of the second simulated image.

[0021] In some such embodiments, a forward pass through the U-net model may include conversion of data to half precision, and a subsequent backward pass through the U-net model may include loss scaling at half precision.

[0022] In some embodiments, the mean square error between the output of the U-net model and the corresponding patch of the second simulated image can be defined as a loss function for training the machine learning algorithm.

[0023] In some embodiments, prior to dividing the first simulated image into tiles, data corresponding to the first simulated image is normalized according to a sample mean and standard deviation calculated across a plurality of corrupted scans.

[0024] In some embodiments, the first simulated image and the second simulated image each include a discrete photo image layer, a scattered image layer, and a combined image layer. In such embodiments, each three-dimensional tile of the first simulated image and the second simulated image may then include a corresponding discrete photo image layer, a scattered layer image, and a combined image layer, each layer being provided to the machine learning algorithm as a discrete channel. Each image layer is then processed with a discrete loss function, and each channel is normalized independently of the other channels.

[0025] In some embodiments, each tile further comprises a position encoding such that the machine learning algorithm is provided with position data associated with the respective tile.

[0026] In some embodiments, the method further comprises incorporating an artifact generating feature into the three-dimensional digital phantom prior to selecting the first Z position.

[0027] In some embodiments, the method continues to generate additional training images from the digital phantom. In such embodiments, the method may continue to select a second Z position along the central axis of the digital phantom and simulate a second set of forward projections from the digital phantom acquired along an axial trajectory at the second Z position along the central axis. The second set of forward projections has the first simulated collimation.

[0028] The method then continues by reconstructing a third simulated image from the second set of forward projections. The third simulated image is a three-dimensional volume enclosing a second segment of the central axis containing the second Z position, the second segment being different from the first segment of the central axis.

[0029] The method then identifies a second plurality of secondary Z positions along the central axis other than the second Z position within the second segment of the central axis, and for each of the second Z position and the second plurality of secondary Z positions, simulates a second set of secondary forward projections from the digital phantom acquired along an axial trajectory at the corresponding secondary Z position. The second set of secondary forward projections has the second simulated collimation.

[0030] The method then proceeds to reconstruct the forward projection associated with the second Z position and each of the second plurality of secondary Z positions into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis.

[0031] The method then combines the two-dimensional images to create a fourth simulated image comprising a three-dimensional volume corresponding to the three-dimensional volume of the third simulated image.

[0032] The method then continues to train the machine learning algorithm by providing the third simulated image as a sample artifact-prone image and providing the fourth simulated image as a ground truth.

[0033] In some such embodiments, the first simulated image, the second simulated image, the third simulated image, and the fourth simulated image are all provided to the machine learning algorithm as a batch.

[0034] In some embodiments, the three-dimensional digital phantom varies along a time dimension. Then, the first simulated image and the second simulated image are drawn from the digital phantom at a first time along the time dimension, and the method continues by simulating a second set of forward projections from the digital phantom at a second time along the time dimension acquired along an axial trajectory at the first Z position. The second set of forward projections has the first simulated collimation.

[0035] The method then reconstructs a third simulated image from the second set of forward projections, the third simulated image being a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image.

[0036] For each of the first Z position and the first plurality of secondary Z positions, the method then simulates a second set of secondary forward projections from the digital phantom at a second time along the time dimension acquired along the axial trajectory at the corresponding secondary Z position. The second set of secondary forward projections each have the second simulated collimation.

[0037] The method then reconstructs the forward projections associated with the first Z position and each of the first plurality of secondary Z positions into two-dimensional images corresponding to axial slices of the digital phantom at corresponding Z positions along the central axis, and combines the two-dimensional images to create a fourth simulated image comprising a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image.

[0038] The method then continues to train the machine learning algorithm by providing the third simulated image as a sample artifact-prone image and providing the fourth simulated image as a ground truth.

[0039] In some embodiments, the method continues to implement an artifact reduction method. In such embodiments, the method retrieves cone beam CT imaging data acquired using a cone beam computed tomography process. The method then applies the trained machine learning algorithm to the cone beam CT imaging data and generates an artifact reduced image comprising a three-dimensional volume. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of a system according to one embodiment of the present disclosure.

[0041] Figure 2 An exemplary imaging device according to one embodiment of the present disclosure is shown.

[0042] Figure 3 A method for generating a training set for training a model for artifact reduction in images according to the present disclosure is shown.

[0043] Figure 4 A schematic pipeline for training a model for artifact reduction in images according to the present disclosure is shown.

[0044] Figure 5 is a flow chart illustrating a method for artifact reduction according to the present disclosure.

[0045] Figure 6 An alternative schematic pipeline for training a model for artifact reduction in images according to the present disclosure is shown. DETAILED DESCRIPTION

[0046] The description of the illustrative embodiments according to the principles of the present disclosure is intended to be read in conjunction with the accompanying drawings, which should be considered as part of the entire written description. In the description of the embodiments of the present disclosure disclosed herein, any reference to direction or orientation is intended only to facilitate description and does not limit the scope of the present disclosure in any way. Relative terms such as "lower", "upper", "horizontal", "vertical", "above", "below", "up", "down", "upward" and "downward" and their derivatives (e.g., "horizontally", "downward", "upward", etc.) should be interpreted as referring to the orientation shown in the drawings described or under discussion at the time. These relative terms are only for convenience of description, and unless explicitly stated, the device is not required to be constructed or operated in a specific orientation. Unless otherwise explicitly stated, terms such as "attachment", "attachment", "connection", "coupling", "interconnection" and the like refer to a kind of, wherein the structures are directly or indirectly fixed or attached to each other through intermediate structures, and the two are removable or rigidly attached or related. In addition, the features and benefits of the present disclosure are explained with reference to exemplary embodiments. Therefore, the present disclosure is clearly not limited to such exemplary embodiments, which show some possible non-limiting feature combinations that can exist alone or in other feature combinations; the scope of the present disclosure is defined by the claims.

[0047] The present disclosure describes one or more best modes of practicing the present disclosure currently contemplated. This specification is not intended to be understood in a limiting sense, but rather to provide examples of the present disclosure presented for illustrative purposes only, by reference to the accompanying drawings, to inform those of ordinary skill in the art of the advantages and configurations of the present disclosure. In the various views of the accompanying drawings, the same reference numerals represent the same or similar components.

[0048] It is important to note that the disclosed embodiments are merely examples of the many advantageous uses of the innovative teachings herein. In general, statements in the specification of this application do not necessarily limit any claimed disclosure. Furthermore, some statements may apply to some inventive features but not to others. Generally, unless otherwise stated, singular elements may be plural and vice versa without loss of generality.

[0049] Often, images acquired for use in a medical setting require some processing in order to denoise or remove artifacts from the image. This artifact removal is necessary in a medical setting where images may be used for diagnosis and treatment, as the precision and accuracy of these images can improve their usefulness. This artifact removal can be achieved using machine learning based algorithms such as convolutional neural networks (CNNs).

[0050] In the context of cone beam computed tomography (CBCT) imaging, imaging artifacts are often quite large and require a model background for artifact reduction. For example, artifacts may appear as streaks on portions of the image. In addition, cone beam artifacts may be due to data insufficiency inherent in axial data acquisition. In some cases, cone beam images can be obtained from only one axial rotation of the radiation source around the object. Therefore, while some artifact reduction can be achieved by filtering the raw data or the final image, CBCT artifact reduction may require a more nuanced approach to account for this data insufficiency.

[0051] Figure 1 1 is a schematic diagram of a system 100 according to an embodiment of the present disclosure. As shown in the figure, the system 100 generally includes a processing device 110 and an imaging device 120.

[0052] The processing device 110 may apply a processing routine to images or measurement data, such as projection data, received from the image device 120. The processing device 110 may include a memory 113 and a processor circuit 111. The memory 113 may store a plurality of instructions. The processor circuit 111 may be coupled to the memory 113 and may be configured to execute instructions. The instructions stored in the memory 113 may include a processing routine and data associated with the processing routine, such as a machine learning algorithm and various filters for processing images.

[0053] The processing device 110 may also include an input 115 and an output 117. The input 115 may receive information, such as images or measurement data, from the imaging device 120. The output 117 may output information, such as filtered images, to a user or user interface device. The output may include a monitor or display.

[0054] In some embodiments, processing device 110 may be directly associated with imaging device 120. In alternative embodiments, processing device 110 may be distinct from imaging device 120, such that processing device 110 receives image or measurement data at input 115 through a network or other interface for processing.

[0055] In some embodiments, the imaging device 120 may include an image data processing device and a spectral or conventional CT scanning unit for generating CT projection data when scanning an object (e.g., a patient). In some embodiments, the imaging device 120 may be a conventional CT scanning unit configured to generate spiral scans for generating training data, as described below. In some embodiments, the imaging device 120 may be a cone beam CT unit configured to obtain a cone beam image from a single axial scan of the subject.

[0056] Figure 2An exemplary imaging device 200 according to one embodiment of the present disclosure is shown. It should be understood that although a CT imaging device is shown and the following discussion is generally conducted in the context of CT images, similar methods can be applied in the context of other imaging devices, and the images to which these methods are applicable can be acquired in a variety of ways.

[0057] In the imaging device 200 according to an embodiment of the present disclosure, the CT scanning unit may be adapted to perform one or more axial scans and / or spiral scans on the object to generate CT projection data. In the imaging device 200 according to an embodiment of the present disclosure, the CT scanning unit may include an energy-resolved photon counting or spectral double-layer image detector. Other detector settings may also be used to acquire spectral content. The CT scanning unit may include a radiation source that emits radiation for passing through the object when acquiring projection data.

[0058] exist Figure 2 In the illustrated example, a CT scanning unit 200 (e.g., a computed tomography (CT) scanner) may include a fixed gantry 202 and a rotating gantry 204, which may be rotatably supported by the fixed gantry 202. The rotating gantry 204 may rotate about a longitudinal axis around an examination region 206 of a subject while acquiring projection data. The CT scanning unit 200 may include a support 207 to support a patient in the examination region 206 and is configured to allow the patient to pass through the examination region during an imaging procedure.

[0059] The CT scanning unit 200 may include a radiation source 208, such as an X-ray tube, which may be supported by the rotating gantry 204 and configured to rotate with the rotating gantry 204. The radiation source 208 may include an anode and a cathode. A source voltage applied between the anode and the cathode may accelerate electrons from the cathode to the anode. The electron flow may provide an electric current from the cathode to the anode, such as generating radiation for passing through the examination region 206.

[0060] The CT scanning unit 200 may include a detector 210. The detector 210 may subtend an angular arc opposite the radiation source 208 relative to the examination region 206. The detector 210 may include a one-dimensional or two-dimensional array of pixels, such as direct conversion detector pixels. The detector 210 may be adapted to detect radiation passing through the examination region 206 and to generate a signal indicative of its energy.

[0061] Typically, a CT scanning unit acquires a series of projection frames as the rotating gantry 204 rotates around the patient. Therefore, each acquired frame of projection data overlaps adjacent frames to some extent, depending on the amount of gantry movement between frames, and includes imaging data of the same subject (i.e., patient) acquired at different angles.

[0062] The CT scanning unit 200 may include generators 211 and 213. The generator 211 may generate tomographic projection data 209 based on the signal from the detector 210. The generator 213 may receive the tomographic projection data 209 and, in some embodiments, generate a sequence of raw image data frames 311 of the object based on the tomographic projection data 209. In some embodiments, the tomographic projection data 209 may be provided to an input 115 of the processing device 110, while in other embodiments, the sequence of raw image data frames 311 is provided to the input of the processing device.

[0063] In some embodiments, the first CT scanning unit 200 can be used during training of the model for artifact reduction described below, while the second CT scanning unit 200 can be used to collect imaging data requiring artifact reduction. For example, the first CT scanning unit 200 can be used to collect imaging data for creating a three-dimensional digital phantom for training. Such imaging data can be collected by the first CT scanning unit 200 through spiral scanning. In contrast, the second CT scanning unit 200 can be a cone beam CT unit configured to collect imaging data requiring artifact reduction.

[0064] Thus, the first CT scanning unit 200 may be provided with a one-dimensional or two-dimensional pixel array in the detector 210, and a conventional axial or spiral scanning process may generate a two-dimensional projection. In contrast, the second CT scanning unit 200 may be provided with a two-dimensional pixel array in the corresponding detector 210, and the unit may then implement a cone-beam image acquisition process. In some embodiments, the cone-beam image acquisition process includes only a single axial scan, which includes a set of projections acquired along an axial trajectory around the axis of the subject, which axis generally corresponds to the longitudinal axis of the examination region 206.

[0065] In some embodiments, the size of the pixel array in the detector 210 defines the collimation size of the image data acquired by the array. Thus, a one-dimensional pixel array may be used to acquire only two-dimensional projections acquired along the axial direction, while a two-dimensional pixel array may be used to acquire three-dimensional projections with a certain collimation size along the axial direction. A CT scanning unit 200 configured to acquire cone beam CT images may have a larger or wider two-dimensional pixel array, and thus may provide greater collimation in the axial direction.

[0066] In the methods described herein, the first step is typically to acquire training data and then train a machine learning model for artifact reduction. As described below, the method provides for training a three-dimensional neural network to reduce artifacts typical in the context of cone beam computed tomography (CBCT). To support the training of such a model, the method first requires a dataset comprising registered corrupted and clean image pairs, which can then be used for such training.

[0067] In a clinical setting, registered corrupted and clean image pairs are not easily available. Clean images that can be used as ground truth typically require a full dose of radiation, while the acquisition of a registered pair typically requires multiple scans, where one scan is performed with a full dose of radiation using a conventional modality (such as a standard axial or spiral scan) and a second scan is performed using a modality that seeks artifact reduction. Furthermore, even in the case where two true scans are performed, such scans are not easily correlated to each other, as the positions may be offset from each other and there may be a resolution mismatch between the paired images. Furthermore, for anatomical structures with complex motion patterns, such as the heart or lungs, the registration between the two scans will also involve non-rigid deformations corresponding to the different cardiac or respiratory states associated with the two scans. In practice, it is often difficult or impossible to achieve registered image pairs with the accuracy required for use as training data in a neural network.

[0068] Thus, in some embodiments, the method begins by generating a simulated data set.

[0069] Figure 3 A method for generating a training set to train a model for artifact reduction in an image according to the present disclosure is shown. As shown, the method can begin by scanning a patient (at 300) using a conventional modality using a CT scanning unit 200. Thus, a conventional detector 210 having a one-dimensional or two-dimensional sensor array 310 can be provided, which can then be used to implement a spiral acquisition (300). The projections 320 acquired using the spiral acquisition process (300) can then be reconstructed (330) using conventional methods to generate a three-dimensional digital phantom 340.

[0070] The digital phantom 340 is a three-dimensional digital model that can be used to simulate the imaging process. The digital phantom in this case is a three-dimensional image or model reconstructed from a conventional scan, and can be a helical image. The digital phantom can then be used to simulate different methods for imaging scans.

[0071] Although the method described herein begins with scanning a patient using the CT scanning unit 200 (at 300), such a scan may equally be replaced by a simulated scan of an existing digital phantom or a scan of a physical phantom 345 or human body model extracted from a database. Thus, the scan or simulated scan of the physical phantom 345 (at 300) may then be used to simulate a helical acquisition (310) of a human subject, such that the resulting digital phantom 340 is in the form expected by the training of the machine learning model.

[0072] In some embodiments, no such conversion is required, and the digital phantom 340 available for training is itself extracted from the database.Any such digital phantom 340 is initially created from imaging data, and such imaging data initially includes projection data acquired from multiple angles around the central axis of the corresponding subject.

[0073] The digital phantom 340 is typically assumed to be a complete model of the subject for training, and can be used to generate clean images that are free of noise or artifacts and can be used as ground truth. Alternatively, the digital phantom 340 can be used to simulate imaging modalities that are known to introduce artifacts.

[0074] Thus, as shown, the digital phantom 340 is used to simulate an axial acquisition at a specified Z position along the central axis of the subject. Such an axial acquisition may include a single axial rotation and may then include simulating a first set of forward projections 370 (350) from the digital phantom 340 acquired along the axial trajectory at the first Z position.

[0075] The first set of forward projections 370 has a first simulated collimation 360 in the axial direction. The first simulated collimation 360 can be based on a simulated two-dimensional pixel array corresponding to a detector that can be used for cone beam CT imaging. Thus, the first simulated collimation 360 can be greater than the collimation expected in conventional axial or spiral CT imaging in the axial direction, but can alternatively correspond to the collimation expected in the context of cone beam CT image acquisition. For example, the first simulated collimation 360 can be a 16 cm axial simulation.

[0076] The first set of forward projections 370 may then be used to reconstruct 380 a first simulated image 390. For example, such a reconstruction may be achieved by a standard filtered backprojection performed in three dimensions. The first simulated image 390 may then include a three-dimensional volume surrounding a first segment containing a first Z position in the central axis, and may therefore contain typical artifacts of cone beam CT acquisition.

[0077] The digital phantom 340 alone can then be used to simulate a conventional axial scan. Thus, the method can identify a plurality of secondary Z positions along the central axis in addition to the first Z position within a first segment of the central axis, and can then simulate a slice-by-slice scan (400) of the digital phantom 340. This will then result in a first set of secondary forward projections 410, each acquired at a corresponding Z position along a corresponding axial trajectory.

[0078] The slice-by-slice scan (at 400) has a second simulated collimation in the axial direction that is less than the first simulated collimation. In some embodiments, the second simulated collimation is based on a simulated one-dimensional pixel array 410 in the detector. In such embodiments, each slice will include a one-dimensional projection 420.

[0079] The forward projection 420 associated with each Z position is then reconstructed 430 into a corresponding two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis. This reconstruction is repeated for the forward projection 420 associated with each secondary Z position and the forward projection associated with the first Z position.

[0080] The reconstructed two-dimensional images associated with each Z position are then combined along the Z direction to obtain a three-dimensional second simulated image 440. The second simulated image 440 has the same geometry as the first simulated image 390. However, while the first simulated image 390 has artifacts associated with cone beam CT image acquisition, the second simulated image 440 is based on a two-dimensional image reconstruction in the axial acquisition plane and therefore has no such artifacts. This is because the effective cone angle of the second simulated image 440 is zero if compared to the cone beam acquisition process, thereby eliminating the problem of insufficient data for the axial scan.

[0081] Therefore, the second simulated image 440 can be used as a reference truth image for network training while training the network to remove artifacts from the first simulated image 390.

[0082] In some embodiments, the digital phantom 340 or spiral scan 300 discussed above can be used directly as ground truth. However, by using the second simulated image 440, there will be no resolution mismatch between the first simulated image 390 and the ground truth, since both have undergone one iteration of forward and back projection. In this way, a neural network trained on such an image pair will focus on the task of removing cone beam artifacts without being subject to correcting for resolution mismatch.

[0083] Figure 4 A schematic pipeline 500 for training a model for artifact reduction in an image according to the present disclosure is shown. As shown, the machine learning algorithm may be a three-dimensional convolutional neural network (CNN) 510 implemented using a U-net class architecture.

[0084] When training the CNN 510, the method may start with a set of corrupted scans, such as the first simulated image 390 discussed above, and a set of corresponding ground truth scans of the same object, such as the second simulated image 440 discussed above. To implement the three-dimensional CNN 510, the method discussed in more detail below must first segment each three-dimensional image 390, 440 into corresponding tiles 520 from the corrupted first simulated image 390 and corresponding tiles 530 from the second simulated image 440 used as ground truth.

[0085] For ease of understanding, the method is described here and below based on a single pair of a first simulated image 390 and a second simulated image 440. However, it should be understood that the CNN 510 is trained on a large number of indexed image pairs. Such image pairs 340 can be generated by selecting different Z positions as starting points and from multiple digital phantoms containing different content.

[0086] Once the pairs of corresponding tiles 520, 530 are provided to the CNN 510, each corrupted tile 520 is provided to the network 510. In the case where the CNN 510 has a U-net or similar architecture, as shown, the machine learning algorithm includes at least one first convolution step 540 applied to each tile, followed by at least one downsampling operation 550. After downsampling 550, at least one additional convolution step 560 is implemented, followed by upsampling 570.

[0087] After both the downsampling 550 and upsampling steps 570 and the intermediate convolution step 560, the output of the first convolution step 540 is cascaded 580 with the upsampling tile 590. As shown, the downsampling 550 and upsampling 570 may be repeated multiple times, with additional convolutions being implemented between each level. In some embodiments, the described cascade is implemented at each level so that the CNN 510 works symmetrically.

[0088] The resulting output is a prediction 600 corresponding to each corrupted tile 520, which can then be compared to the corresponding tile 530 of the ground truth simulated image 440. The CNN 510 can then be trained by evaluating how successfully the prediction 600 corresponds to the tile 530 of the simulated image 440 according to a loss function (e.g., calculating the mean square error between the two).

[0089] CNN 510 can be implemented in both forward 610 and reverse 620, and can be repeated with image pairs until the results converge and the loss function is minimized. For example, the reverse pass 620 can be a back propagation of the output of the loss function to improve the accuracy of the variable weights in the model. Therefore, after each pass, the weights within CNN 510 can be updated before further training.

[0090] Various techniques are implemented to reduce memory usage of the CNN 510 during implementation. As described above, the CNN 510 is provided with three-dimensional tiles 520, 530 rather than full simulated images 390, 440 drawn from these tiles. This approach avoids the need to store the entire CT volume in GPU memory. In addition, mixed precision training can be implemented so that the forward pass 610 and the backward pass 620 use half-precision floating point numbers, thereby utilizing 16 bits instead of 32 bits. Similarly, the use of the U-net architecture itself reduces the need for memory because the feature maps are downsampled during processing and take up less memory.

[0091] Figure 5 is a flow chart showing a method for artifact reduction according to the present disclosure. Figure 3 As discussed, the method first generates pairs of simulated images 390, 440 for use in a training set. Thus, the method first retrieves (700) a three-dimensional digital phantom 340 for use in generating the pairs of images. The three-dimensional digital phantom 340 is typically reconstructed from previously acquired computed tomography (CT) data. The CT data includes projection data acquired from multiple angles around a central axis. As described above, the digital phantom 340 can be constructed from a spiral scan 300.

[0092] In some embodiments, the method can be used to address potential artifacts generated by discrete objects or features in the image that are known to cause artifacts. For example, the method can be used to address artifacts generated by external objects such as metal implants. Therefore, before generating a simulated image, artifact-causing features such as simulated metal plates can be incorporated into the three-dimensional digital phantom 340 before proceeding. Similarly, motion can be simulated during the creation of the simulated images 390, 440.

[0093] The method then selects 710 a first Z position along the central axis and simulates 720 a first set of forward projections from the digital phantom 340 acquired along the axial trajectory at the first Z position. The first set of forward projections has a first simulated collimation in the axial direction.

[0094] In some embodiments, the first set of forward projections is used to simulate a cone beam CT procedure. Thus, the first simulated collimation may be quite large, for example, may be 16 cm. Furthermore, the forward projections may be collected in a single simulated pass along an axial trajectory around the digital phantom 340 at the first Z position. Therefore, the data collected in the first set of forward projections is limited.

[0095] The method then continues to reconstruct (730) a first simulated image 390 from the first set of forward projections. The reconstruction (at 730) can be accomplished using a three-dimensional filtered backprojection process. The first simulated image 390 includes a three-dimensional volume surrounding a first segment of the central axis containing a first Z position. In some embodiments, the first segment of the central axis is centered at the first Z position.

[0096] The method then proceeds to identify (740) a first plurality of secondary Z positions along the central axis in addition to the first Z position within a first segment of the central axis. For each of the first plurality of secondary Z positions and the first Z position, a first set of secondary forward projections from the digital phantom is simulated (750). Each first set of secondary forward projections is acquired at a corresponding secondary Z position along a corresponding axial trajectory. Each set of secondary forward projections acquired in this manner has a second simulated collimation in the axial direction that is less than the first simulated collimation.

[0097] In this manner, each first set of secondary forward projections corresponds to an axial slice of the digital phantom 340 that is less thick than the first simulated image 390. In some embodiments, each set of secondary forward projections is obtained using simulation of a detector having a one-dimensional array of pixels. Thus, each slice generated by a set of secondary forward projections is two-dimensional.

[0098] After acquiring the secondary forward projections, each first set of forward projections associated with the corresponding secondary Z position or the first Z position is reconstructed (760) into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis. A two-dimensional filtered backprojection process can be used to reconstruct each axial slice of the digital phantom 340. The two-dimensional images are then combined (770) along the central axis to create a second simulated image 440 that includes a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image.

[0099] The method then proceeds to provide the first simulated image 390 as a sample artifact-prone image and the second simulated image 440 as a ground truth to train 780 a machine learning algorithm, such as the three-dimensional CNN 510 described above.

[0100] As described above, embodiments are described in terms of the generation of a single matching image pair. However, in use, the matching image pair created is one of many image pairs in the sample used in training. In some embodiments, before proceeding to segment (790) each image into tiles, the data corresponding to the first simulated image 390 may be normalized according to the sample mean and standard deviation calculated over multiple damage scans.

[0101] In some embodiments, multiple matching image pairs can be generated from a single digital phantom 340. Thus, after generating the first simulated image 390 and the second simulated image 440 (at 770), the method can then create additional pairs of images (at 710) by selecting a second Z position along the central axis of the digital phantom that is different from the first Z position. The method then simulates (720) a second set of forward projections from the digital phantom 340 acquired along the axial trajectory at the second Z position along the central axis. The second set of forward projections has the same first simulated collimation in the axial direction as the first set of forward projections.

[0102] The method then reconstructs (730) a third simulated image from the second set of forward projections. The third simulated image is a three-dimensional volume surrounding a second segment of the central axis containing a second Z position, the second segment being different from the first segment of the central axis.

[0103] The method then continues by identifying (740) a second plurality of secondary Z positions along the central axis in addition to the second Z position within a second segment of the central axis. For each of the second Z position and the second plurality of secondary Z positions, the method then simulates (750) a second set of secondary forward projections from the digital phantom 340 acquired along the axial trajectory at the corresponding secondary Z position. The second set of secondary forward projections has a second simulated collimation.

[0104] The method then reconstructs (760) the forward projection into two-dimensional images corresponding to transverse slices of the digital phantom at corresponding Z positions along the central axis, and combines (770) the two-dimensional images to create a fourth simulated image that includes a three-dimensional volume corresponding to the three-dimensional volume of the third simulated image.

[0105] In this manner, a third simulated image can be formed by selecting a second Z position along the central axis that is different from the first Z position in a manner similar to the first simulated image 390. A fourth simulated image can then be formed to pair with the third simulated image. Such a process can be repeated for additional Z positions to create a large data set from a limited number or even a single digital phantom 340.

[0106] This method of simulating cone beam artifacts from a given 3D volume allows the method to create perfectly registered pairs of corrupted and clean images. Selecting different Z positions during simulation allows for an increase in the dataset size, since each Z position will produce slightly different artifacts.

[0107] In some embodiments, instead of or in addition to changing the Z position used to generate the first simulated image and the third simulated image, the time dimension can be changed. In such an embodiment, the digital phantom 340 can vary along the time dimension. For example, the digital phantom can correspond to data from one or more CT scans acquired over a period of time or acquired at different time periods. Such acquisitions across time can be used to capture the motion of the subject, such as a gated cardiac scan. Therefore, the three-dimensional digital phantom 340 can contain data from different time periods, and thus the digital phantom itself can vary along the time dimension.

[0108] In such an embodiment, a first simulated image 390 and a second simulated image 440 may be drawn from the digital phantom at a first time along the time dimension. The method may then repeat the method of generating the first simulated image 390 and the second simulated image 440 at a second time along the time dimension, thereby generating a third simulated image and a fourth simulated image. As discussed above with respect to changing the Z position, this technique may be used to generate additional training data from a single digital phantom.

[0109] Once the training set is available, the method proceeds to train (780) the machine learning algorithm using the available data set.

[0110] As described above, as part of the training process, the method can segment (790) each of the first simulated image 390 and the second simulated image 440 into three-dimensional tiles 520, 530. Therefore, each tile 520 of the first simulated image 390 has a corresponding tile 530 of the second simulated image 440. During such a training process, the three-dimensional tiles are provided to the machine learning algorithm. In some embodiments, each tile 520, 530 includes a position encoding. For example, each voxel can be provided with a (Z, X, Y) position. Therefore, the position data associated with the corresponding tile is provided to the machine learning algorithm. This can provide the model with information about the Z position of each tile, thereby allowing better control of the behavior of the network.

[0111] The tiles can be random and significantly smaller than the image drawn from them. For example, in an image of size (256, 512, 512) corresponding to the (Z, X, Y) dimensions, the tiles can be of size (64, 128, 128).

[0112] Furthermore, as described above, the three-dimensional CNN may include at least one first convolution step (800) applied to each tile 520 of the first simulated image 390, followed by at least one downsampling operation (810). At least one additional convolution step (820) is then applied after the downsampling, and then the downsampled tile is upsampled (830) after the at least one additional convolution step. The upsampled tile is then concatenated (840) with the output of the first convolution step (at 800).

[0113] In some embodiments, various downsampling, upsampling, and convolution operations may be performed on the Figure 4 The 3D U-net model described and shown in more detail is implemented in the 3D U-net model. Therefore, each tile 520 of the first simulated image 390 is provided to the 3D U-net model, and the output is compared with the corresponding tile 530 of the second simulated image 440 (850). The comparison can be based on a loss function used to train the CNN, for example, the loss function can be defined as the mean square error between the output of the U-net model and the corresponding tile 530 of the second simulated image 440. The output of this loss function can then be back-propagated through the model in a backward traversal 620.

[0114] In some embodiments, as described above, the forward pass 610 through the U-net model includes conversion of data to half precision, and the backward pass 620 through the U-net model includes loss scaling at half precision.

[0115] After training of the CNN 510, in some embodiments, the trained model can be used to reduce artifacts in an image. In such an embodiment, the method can retrieve cone beam computed tomography imaging data acquired using a cone beam computed tomography process. The trained CNN 510 can then be applied to the cone beam computed tomography imaging data to generate an artifact-reduced image comprising a three-dimensional volume.

[0116] Figure 6 An alternative schematic pipeline for training a model for artifact reduction in images according to the present disclosure is shown.

[0117] In some embodiments, the first simulated image 390 and the second simulated image 440 both simulate spectral scanning, so each simulated image includes a discrete photo image layer 900, a scattered image layer 910, and a combined image layer 920. Each three-dimensional image tile 520 of the first simulated image 390 and each three-dimensional image tile 530 of the second simulated image 440 similarly includes a corresponding discrete photo image layer 900, a scattered image layer 910, and a combined image layer 920. Then, each layer of each image tile is provided to the CNN 510 as a discrete channel, such as Figure 6In simplified form, in order to generate the corresponding prediction tile 600 layers, each layer is processed using a discrete loss function. In such an embodiment, each channel can be normalized independently of other channels.

[0118] In some embodiments, the loss function is the sum of the mean squared error values ​​calculated for each channel. In such embodiments, it is important to balance the performance between predictions. Therefore, each channel can then have a different normalization value. Instead of using the sample mean and standard deviation in such embodiments, the method can shift and scale the data according to the level and window values ​​used for visualization later. For example, if the scatter is usually visualized with level -50 and window 400, the method can shift the data by -50 and scale it by 200, which is half of the window. This technique helps to evenly distribute the performance of the model between different channels and achieve visually equal results.

[0119] The methods discussed herein can be used to combine artifact reduction with denoising and / or super-resolution processing and other image-to-image problems. Therefore, when creating simulated images, the problem to be solved should be simulated. For example, in order to combine artifact reduction with denoising, the simulation of the axial acquisition of the first simulated image 390 should be combined with the simulation of the low-dose acquisition.

[0120] In some embodiments, 3D natural images can be used to train artifact removal. Due to the huge structural variability, models trained using natural images have all the prerequisites to be generalizable to the medical image domain.

[0121] The method according to the present disclosure can be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or implemented in a combination of the two. The executable code of the method according to the present disclosure can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Preferably, the computer program product may include a non-transitory program code stored on a computer-readable medium for performing the method according to the present disclosure when the program product is executed on a computer. In one embodiment, the computer program may include a computer program code suitable for performing all steps of the method according to the present disclosure when the computer program is running on a computer. The computer program may be embodied on a computer-readable medium.

[0122] While the present disclosure has been described at some length and with a certain particularity with respect to several described embodiments, it is not intended that the present disclosure be limited to any such details or embodiments or to any particular embodiment, but rather be interpreted with reference to the claims so as to provide such claims with the broadest possible interpretation in light of the prior art so as to effectively encompass the intended scope of the present disclosure.

[0123] All examples and conditional language described herein are intended to be used for teaching purposes, to help readers understand the principles of the present disclosure and the concepts contributed by the inventors to promote this area, and will be interpreted as not being limited to these specific examples and conditions of narration. In addition, all statements describing the principles, aspects and embodiments of the present disclosure and their specific examples herein are intended to cover their structural and functional equivalents. In addition, these equivalents are intended to include currently known equivalents and equivalents developed in the future, that is, any element of the performance of the same function developed, regardless of its structure.

Claims

1. A method for training a machine learning model to reduce artifacts, comprising: retrieving a three-dimensional digital phantom reconstructed from computed tomography imaging data including projection data acquired from a plurality of angles about a central axis; selecting a first Z position along the central axis; simulating a first set of forward projections from the digital phantom acquired along an axial trajectory at the first Z position along the central axis, the first set of forward projections having a first simulated collimation in the axial direction; reconstructing a first simulated image from the first set of forward projections, the first simulated image comprising a three-dimensional volume surrounding a first segment of the central axis containing the first Z position; identifying a first plurality of secondary Z positions along the central axis in addition to the first Z position within the first segment of the central axis; for each of the first Z position and the first plurality of secondary Z positions, simulating a first set of secondary forward projections from the digital phantom acquired at the corresponding secondary Z position along a corresponding axial trajectory, the first set of secondary forward projections having a second simulated collimation in the axial direction that is less than the first simulated collimation; reconstructing a forward projection associated with the first Z position and each of the first plurality of secondary Z positions into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis; combining the two-dimensional images associated with the first Z position and each of the first plurality of secondary Z positions to create a second simulated image including a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image; A machine learning algorithm is trained by providing the first simulated image as a sample artifact-prone image and providing the second simulated image as a ground truth.

2. The method according to claim 1, wherein: The first segment of the central axis is centered at the first Z position.

3. The method according to claim 1, wherein: The digital phantom is reconstructed by spiral scanning.

4. The method according to claim 1, wherein: The first simulated image is reconstructed using a three-dimensional filtered back projection process, and wherein the two-dimensional image corresponding to an axial slice of the digital phantom is reconstructed using a two-dimensional filtered back projection process.

5. The method according to claim 1, wherein: The machine learning algorithm is a three-dimensional convolutional neural network.

6. The method according to claim 1, wherein: Each of the first simulated image and the second simulated image is segmented into three-dimensional tiles such that each tile of the first simulated image has a corresponding tile of the second simulated image, and wherein the three-dimensional tiles are provided to the machine learning algorithm.

7. The method according to claim 6, wherein: The machine learning algorithm comprises at least one first convolution step applied to each tile of the first simulated image provided, followed by at least one downsampling operation, and wherein at least one additional convolution step is applied after the downsampling, and wherein after the at least one additional convolution step the downsampled tile is upsampled, and wherein the upsampled tile is concatenated with the output of the first convolution step.

8. The method according to claim 7, wherein: The machine learning algorithm is a three-dimensional U-net model, and each tile of the first simulated image is provided to the three-dimensional U-net model and the output is compared with a corresponding tile of the second simulated image.

9. The method according to claim 8, wherein: The mean squared error between the output of the U-net model and the corresponding patch of the second simulated image is defined as the loss function for training the machine learning algorithm.

10. The method according to claim 8, wherein: The forward pass through the U-net model includes conversion of the data to half precision, and the subsequent backward pass through the U-net model includes loss scaling at half precision.

11. The method according to claim 6, wherein: Prior to segmenting the first simulated image into tiles, data corresponding to the first simulated image is normalized according to a sample mean and standard deviation calculated across a plurality of corrupted scans.

12. The method according to claim 6, wherein: Both the first simulated image and the second simulated image include discrete photo image layers, scattered image layers, and combined image layers, and wherein each three-dimensional tile of the first simulated image and the second simulated image includes corresponding discrete photo image layers, scattered image layers, and combined image layers, each image layer is provided to the machine learning algorithm as a discrete channel, each image layer is processed with a discrete loss function, and wherein each channel is normalized independently of other channels.

13. The method according to claim 6, wherein: Each tile also includes a position encoding so that the machine learning algorithm is provided with position data associated with the corresponding tile.

14. The method of claim 1, further comprising incorporating an artifact generating feature into the three-dimensional digital phantom prior to selecting the first Z position.

15. The method according to claim 1, further comprising: selecting a second Z position along the central axis of the digital phantom; simulating a second set of forward projections from the digital phantom acquired along an axial trajectory at the second Z position along the central axis, the second set of forward projections having the first simulated collimation; reconstructing a third simulated image from the second set of forward projections, the third simulated image being a three-dimensional volume surrounding a second segment of the central axis containing the second Z position, and the second segment being different from the first segment of the central axis; identifying a second plurality of secondary Z positions along the central axis in addition to the second Z position within the second segment of the central axis; for each of the second Z position and the second plurality of secondary Z positions, simulating a second set of secondary forward projections from the digital phantom acquired along an axial trajectory at the corresponding secondary Z position, the second set of secondary forward projections having the second simulated collimation; reconstructing a forward projection associated with the second Z position and each of the second plurality of secondary Z positions into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis; combining the two-dimensional images to create a fourth simulated image, the fourth simulated image comprising a three-dimensional volume corresponding to the three-dimensional volume of the third simulated image; The machine learning algorithm continues to be trained by providing the third simulated image as a sample artifact-prone image and providing the fourth simulated image as a ground truth.

16. The method according to claim 15, wherein: The first simulated image, the second simulated image, the third simulated image, and the fourth simulated image are all provided to the machine learning algorithm as a batch.

17. The method according to claim 1, wherein: The three-dimensional digital phantom varies along a time dimension, and wherein the first simulated image and the second simulated image are drawn from the digital phantom at a first time along the time dimension, the method further comprising: simulating a second set of forward projections from the digital phantom at a second time along the time dimension acquired along an axial trajectory at the first Z position, the second set of forward projections having the first simulated collimation; reconstructing a third simulated image from the second set of forward projections, the third simulated image being a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image; for each of the first Z position and the first plurality of secondary Z positions, simulating a second set of secondary forward projections from the digital phantom at the second time along the time dimension acquired at the corresponding secondary Z position along the axial trajectory, the second set of secondary forward projections having the second simulated collimation; reconstructing a forward projection associated with the first Z position and each of the first plurality of secondary Z positions into a two-dimensional image corresponding to an axial slice of the digital phantom at the corresponding Z position along the central axis; combining the two-dimensional images to create a fourth simulated image, the fourth simulated image comprising a three-dimensional volume corresponding to the three-dimensional volume of the first simulated image; The machine learning algorithm continues to be trained by providing the third simulated image as a sample artifact-prone image and providing the fourth simulated image as a ground truth.

18. A method for reducing artifacts, comprising: Execute the method according to claim 1; retrieving cone beam computed tomography imaging data acquired using a cone beam computed tomography procedure; applying the trained machine learning algorithm to the cone-beam computed tomography imaging data; An artifact-reduced image comprising a three-dimensional volume is generated.

Images