Optimizing CT image formation in simulated X-rays

Abstract

Medical computed tomography (CT) data is processed as follows. Projection data obtained by scanning the object is retrieved. The retrieved projection data is processed to reconstruct a three-dimensional image. A two-dimensional image is generated based on the reconstructed three-dimensional image. Identifying a disease in a two-dimensional image or uncertainty in identification of a disease. The projection data or the three-dimensional image is reprocessed into an updated three-dimensional image such that at least one parameter of the reprocessing is based on the disease or an uncertainty in the identification of the disease. An updated two-dimensional image is then generated based on the updated three-dimensional image.

Description

Technical Field

[0001] The present invention generally relates to systems and methods for simulating conventional X-ray images from computed tomography (CT) data. In particular, the present invention relates to optimizing simulated X-ray images based on disease or condition classification using tunable parameters in a model of an image forming chain. Background Art

[0002] CT imaging offers advantages over conventional planar X-ray (CXR) imaging. As a result, CT imaging has replaced X-ray imaging in various clinical settings and is increasingly being used in additional clinical settings to replace such X-ray imaging.

[0003] This is particularly true in the case of low-dose and ultra-low-dose CT imaging (ULDCT), which are now aimed at replacing CXR in additional settings, such as routine chest imaging in the outpatient setting.

[0004] A major advantage of CT imaging over CXR is that CT provides additional information, especially three-dimensional spatial information. In addition, in many clinical scenarios, CXR has relatively low sensitivity and high false negative rates. Due to the additional information associated with CT imaging, CT imaging is also more suitable for various image processing and artificial intelligence (AI)-based diagnostic techniques.

[0005] On the other hand, CXR has higher spatial resolution than conventional CT imaging (and specifically ULDCT imaging), and suffers from less noise.

[0006] One reason CT imaging has not been more widely used in routine clinical settings is that the read time for CT imaging is substantially higher than for CXR. This is in part because radiologists are more familiar with CXR images and are therefore more comfortable reading and basing diagnoses on such conventional planar X-ray images.

[0007] However, to the extent that radiologists continue to rely on CXR images, they are giving up the advantages of CT imaging in terms of both additional information and analytical capabilities.

[0008] Therefore, it is known to create digitally reconstructed radiographic (DRR) images in order to incorporate some of the advantages associated with CT images into a synthetic CXR image. Such DDR images can be constructed by digitally tracking X-rays from a 3D CT volume. In a clinical setting, such reconstructions can be used by radiologists in a variety of settings where CXR images would traditionally be used.

[0009] Classification models for diseases, conditions, disorders, etc. (hereinafter collectively referred to as "disease classification models") are known in the context of CXR images. However, such models cannot be applied to CT images themselves, and therefore, when image processing is applied to the CT data prior to generating a DDR, information related to disease classification cannot be utilized to optimize reconstruction.

[0010] There is a need for a CT imaging system and method, particularly a ULDCT imaging system and method, which can present data to a radiologist in a form that is easier to interpret and more likely to be adopted. There is also a need for such a system in which disease classification can be used to optimize image reconstruction. Summary of the invention

[0011] Systems and methods for converting three-dimensional CT data into two-dimensional images are provided. For example, such methods and systems include retrieving projection data acquired by scanning an object from multiple angles around a central axis.

[0012] In some embodiments, a computer-implemented method for processing medical CT data includes retrieving projection data acquired by scanning an object.

[0013] The method then includes processing the projection data to reconstruct a three-dimensional image.Once processed, the method generates a two-dimensional image based on the three-dimensional image and identifies a disease or uncertainty in the identification of a disease in the two-dimensional image.

[0014] The method then reprocesses the projection data or the three-dimensional image into an updated three-dimensional image, wherein at least one parameter of the reprocessing is based on the disease or uncertainty in the identification of the disease. The method then generates an updated two-dimensional image based on the updated three-dimensional image.

[0015] In some embodiments, generating the two-dimensional image is performed by a neural network.

[0016] In some embodiments, reprocessing the projection data or the three-dimensional image is performed by a neural network.

[0017] In some embodiments, the three-dimensional image is denoised using a first value for at least one parameter. In some such embodiments, the updated three-dimensional image is denoised using a second value for at least one parameter. The second value is based on uncertainty in the disease or identification of the disease.

[0018] In some embodiments, artificial intelligence (AI)-based super-resolution is applied to the three-dimensional image using a first value for at least one parameter so that the three-dimensional image produces a high-resolution image. In some such embodiments, AI-based super-resolution is applied to an updated three-dimensional image using a second value for at least one parameter so that the updated three-dimensional image produces another high-resolution image. The second value is based on the disease or uncertainty in the identification of the disease.

[0019] In some embodiments, the three-dimensional image and / or the updated three-dimensional image are denoised by a trained convolutional neural network (CNN). In some embodiments, a two-dimensional image is generated by a DRR network using a first value for at least one parameter. In some such embodiments, an updated two-dimensional image is generated by using a DRR network using a second value for at least one parameter. The second value is based on uncertainty in the disease or the identification of the disease.

[0020] In some embodiments, at least one physical element is identified in the three-dimensional image and / or the updated three-dimensional image. The at least one physical element is removed or masked before generating the output image. In some such embodiments, the at least one physical element is a plurality of ribs or a heart. In some such embodiments, the at least one physical element is removed or masked based on a disease in the two-dimensional image or uncertainty in the identification of the disease.

[0021] In some embodiments, the disease is identified by using a trained disease classification model.

[0022] In some embodiments, a system for processing medical CT data includes: a memory storing multiple instructions; and a processor coupled to the memory and configured to execute the multiple instructions to: retrieve projection data acquired by scanning an object; process the projection data to reconstruct a three-dimensional image; generate a two-dimensional image based on the three-dimensional image; identify a disease in the two-dimensional image or uncertainty in the identification of the disease; reprocess the projection data or the three-dimensional image into an updated three-dimensional image, wherein at least one parameter of the reprocessing is based on the disease or the uncertainty in the identification of the disease; and generate an updated two-dimensional image based on the updated three-dimensional image.

[0023] In some embodiments, generating the two-dimensional image and reprocessing the projection data or the three-dimensional image is performed by a neural network.

[0024] In some embodiments, the three-dimensional image is denoised using a first value for at least one parameter. In some such embodiments, the updated three-dimensional image is denoised using a second value for at least one parameter. The second value is based on uncertainty in the disease or identification of the disease.

[0025] In some embodiments, the three-dimensional image and / or the updated three-dimensional image is denoised by a trained convolutional neural network (CNN).

[0026] In some embodiments, the DRR network generates a two-dimensional image using a first value for at least one parameter. In some such embodiments, the DRR network generates an updated two-dimensional image using a second value for at least one parameter. The second value is based on uncertainty in the disease or identification of the disease.

[0027] In some embodiments, AI-based super-resolution is applied to the three-dimensional image using a first value for at least one parameter such that the three-dimensional image produces a high-resolution image. In some such embodiments, AI-based super-resolution is applied to an updated three-dimensional image using a second value for at least one parameter such that the updated three-dimensional image produces another high-resolution image. The second value is based on the disease or uncertainty in the identification of the disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a system according to an embodiment of the present invention.

[0029] Figure 2 An exemplary imaging device according to one embodiment of the present invention is illustrated.

[0030] Figure 3 The image formation chain used in the method of one embodiment of the present invention is illustrated.

[0031] Figure 4 A schematic overview of a model structure for implementing a method of one embodiment of the present invention is illustrated.

[0032] Figure 5 is a flow chart illustrating a method according to one embodiment of the present invention.

[0033] Figure 6 A ray tracing process applied to a three-dimensional image useful in the context of one embodiment of the present invention is schematically illustrated.

[0034] Figure 7 is a flow chart illustrating a method for training an image forming chain according to one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The description of the illustrative embodiments according to the principles of the present invention is intended to be read in conjunction with the accompanying drawings, which are considered to be part of the entire written description. In the description of the embodiments of the present invention disclosed herein, any reference to direction or orientation is intended only to facilitate description and is not intended to limit the scope of the present invention in any way. Relative terms such as "lower", "upper", "horizontal", "vertical", "above", "below", "up", "down", "top" and "bottom" and their derivatives (e.g., "horizontally", "downwardly", "upwardly", etc.) should be interpreted as referring to the orientation as then described or as shown in the drawings discussed. These relative terms are only for convenience of description and do not require the device to be constructed or operated in a specific orientation unless explicitly indicated. Unless otherwise explicitly described, terms such as "attach", "fix", "connect", "couple", "interconnect" and similar terms refer to a relationship in which structures are fixed or attached to each other directly or indirectly through intermediate structures, as well as a removable or rigid attachment or relationship. In addition, the features and benefits of the present invention are explained by reference to exemplary embodiments. Therefore, the present invention should clearly not be limited to such exemplary embodiments that illustrate some possible non-limiting feature combinations that may exist alone or in other feature combinations; the scope of the present invention is defined by the appended claims.

[0036] The present invention describes one or more best modes of practicing the present invention currently contemplated. This specification is not intended to be understood in a limiting sense, but rather provides examples of the present invention presented for illustrative purposes only, with reference to the accompanying drawings, to suggest advantages and configurations of the present invention to those of ordinary skill in the art. In the various views of the accompanying drawings, the same reference numerals represent the same or similar parts.

[0037] It is important to note that the disclosed embodiments are merely examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. In addition, some statements may apply to some inventive features but not to other features. In general, unless otherwise stated, a single element may be plural and vice versa without loss of generality.

[0038] Computed tomography (CT) and conventional planar X-ray (CXR) are both used for medical imaging. However, CT imaging, especially ultra-low-dose CT imaging (ULDCT), is intended to replace CXR in many clinical settings, such as chest imaging in routine outpatient settings.

[0039] Some of the major advantages of ULDCT imaging are immediately apparent. CT imaging, including ULDCT, provides three-dimensional spatial information, which allows for sophisticated analysis techniques. In addition, ULDCT avoids the relatively low sensitivity and high false-negative rates associated with CXR in many clinical scenarios.

[0040] However, the reading time of ULDCT is slower than that of CXR, and radiologists are less familiar with ULDCT and more uncomfortable. Therefore, radiologists prefer to face more familiar CXR images and make diagnoses based on more familiar CXR images. Therefore, an embodiment of the present invention provides a workflow for generating artificial CXR images or images stylized to have the appearance of CXR images based on ULDCT data. Such embodiments can be implemented or enhanced using artificial intelligence (AI) techniques, including using learning algorithms in the form of neural networks, such as convolutional neural networks (CNNs).

[0041] Thus, methods for transforming three-dimensional CT data into two-dimensional images are provided. In this way, CXR-style images can be generated from ULDCT data and presented to radiologists. Such presentation can be after implementation of analysis techniques for the underlying ULDCT data in raw or three-dimensional image format, and can be presented to radiologists as a proxy for a CXR image or in the context of a corresponding ULDCT-based image interface.

[0042] Therefore, one can use Figure 1 The system shown and through Figure 2 The imaging device shown in the figure generates ULDCT imaging data as three-dimensional CT imaging data. Then, the imaging data can be used Figure 1 The system's processing equipment processes the retrieved data.

[0043] Once the ULDCT imaging data is generated, it can be processed into a three-dimensional image by passing the CT imaging data through an image formation chain. Such an image formation chain may include a DRR simulation for a denoising process, a super-resolution process, and converting a three-dimensional image into a two-dimensional image. The image formation chain may include too many parameters that can be tuned and affect the resulting DRR image. From a practical point of view, it is not feasible to iterate over an infinite number of possible images in order to identify which image is optimal for a given situation. In addition, it is not always clear what makes the image optimal in a certain situation.

[0044] This is particularly relevant in the setting of evaluating medical images, as different radiologists may have different opinions on which aspects of the image they find relevant. Furthermore, this expertise is not always easy to obtain. The problem of “expensive” medical expertise is well known when using AI for disease classification.

[0045] Therefore, one embodiment of the present invention uses an AI model that links an image formation chain to a pre-trained disease classification model. The effect of modifying the tunable parameters in the image formation chain can then be evaluated and optimized. Thus, the pre-trained disease classification model can evaluate the first attempt to transform a three-dimensional CT image into a two-dimensional image and classify the image accordingly. The method can then modify the tunable parameters in the image formation chain and reprocess the projection data or 3-D image using the tuned parameters associated with the identified disease.

[0046] Figure 1 FIG. 1 is a schematic diagram of a system 100 according to an embodiment of the present invention. As shown in the figure, the system 100 generally includes a processing device 110 and an imaging device 120 .

[0047] The processing device 110 may apply a processing routine to images or measurement data, such as projection data, received from the image device 120. The processing device 110 may include a memory 113 and a processor circuit 111. The memory 113 may store a plurality of instructions. The processor circuit 111 may be coupled to the memory 113 and may be configured to execute instructions. The instructions stored in the memory 113 may include a processing routine and data associated with the processing routine, such as a machine learning algorithm and various filters for processing images.

[0048] The processing device 110 may also include an input 115 and an output 117. The input 115 may receive information, such as a 3-D image or projection data, from the imaging device 120. The output 117 may output information, such as a filtered image or a converted two-dimensional image, to a user or user interface device. The output may include a monitor or display.

[0049] In some embodiments, processing device 110 may be directly related to imaging device 120. In alternative embodiments, processing device 110 may be distinct from imaging device 120, such that processing device 110 receives image or measurement data for processing through a network or other interface at input 115.

[0050] In some embodiments, the imaging device 120 may include an image data processing device, ie, a spectral or conventional CT scanning unit for generating projection data when scanning an object (eg, a patient). In some embodiments, the imaging device 120 may be a conventional CT scanning unit configured to generate a helical scan.

[0051] Figure 2An exemplary imaging device 200 is illustrated according to one embodiment of the present invention. It should be understood that although a CT imaging device 200 is shown and the following discussion is generally in the context of CT images, similar methods can be applied in the context of other imaging devices, and images to which these methods can be applied can be acquired in a variety of ways.

[0052] In an imaging device 200 according to an embodiment of the present invention, the CT scanning unit may be adapted to perform one or more axial scans and / or spiral scans of an object to generate CT projection data. In an imaging device 200 according to an embodiment of the present invention, the CT scanning unit may include an energy-resolved photon counting and / or spectral double-layer image detector. Other detector settings may also be used to collect spectral content. The CT scanning unit may include a radiation source that emits radiation for traversing an object when collecting projection data.

[0053] exist Figure 2 In the example shown, a CT scanning unit 200 (e.g., a CT scanner) may include a fixed gantry 202 and a rotating gantry 204, which may be rotatably supported by the fixed gantry 202. The rotating gantry 204 may rotate about a longitudinal axis around an examination region 206 of an object (patient) when acquiring projection data. The CT scanning unit 200 may include a table or support 207 to support the patient in the examination region 206 and is configured to move the patient through the examination region during an imaging procedure.

[0054] The CT scanning unit 200 may include a radiation source 208, such as an X-ray tube, which may be supported by the rotating gantry 204 and configured to rotate with the rotating gantry 204. The radiation source 208 may include an anode and a cathode. A source voltage applied across the anode and cathode may accelerate electrons from the cathode to the anode. The electron flow may provide a current from the cathode to the anode to generate radiation for traversing the examination region 206.

[0055] The CT scanning unit 200 may include a detector 210. The detector 210 may subtend an angular arc opposite the examination region 206 relative to the radiation source 208. The detector 210 may include a one-dimensional or two-dimensional pixel array, such as direct conversion detector pixels. The detector 210 may be adapted to detect radiation traversing the examination region 206 and to generate a signal indicative of its energy.

[0056] The CT scanning unit 200 may include generators 211 and 213. The generator 211 may generate projection data 209 based on the signal from the detector 210. The generator 213 may receive the projection data 209 and, in some embodiments, generate a three-dimensional image 311 of the object based on the projection data 209. In some embodiments, the projection data 209 may be provided to the input 115 of the processing device 110, while in other embodiments, the 3-D image 311 is provided to the input of the processing device.

[0057] Figure 3 The image formation chain used in the method / apparatus of one embodiment of the present invention is illustrated. Figure 4 A schematic overview of a model structure for implementing a method of one embodiment of the present invention is illustrated. Figure 5 is a flow chart illustrating a method according to one embodiment of the present invention.

[0058] As shown, according to a method of one embodiment, CT data is processed into images and generally includes retrieving (500) projection data acquired from multiple angles about a central axis.

[0059] Therefore, in Figure 2 In the context of an imaging device 200, the object may be a patient on a support 207, and the central axis may be an axis passing through an examination region. In the case of acquiring projection data from the imaging device 200, the rotating gantry 204 may then rotate about the central axis of the object, thereby acquiring projection data from various angles.

[0060] Once acquired, the projection data is processed to reconstruct (510) a 3D image 300, which is then processed (520).

[0061] It should be understood that although reconstruction (510) and processing (520) are indicated as different processes, the reconstruction itself can be the actual processing of the projection data into a 3D image. Similarly, the reconstruction can be part of such processing. Such reconstruction (510) can be done using standard reconstruction techniques, such as by filtered backprojection.

[0062] like Figure 3 and Figure 5 As shown, the processing (520) can include denoising 310 (523), which can be, for example, by a neural network or other algorithm, such as an AI-based learning algorithm. In the example shown, the denoising 310 (523) is by a convolutional neural network (CNN) previously trained on appropriate images. For example, in the case of CT imaging with noise, such as in the case of ULDCT images, such a denoising process 310 (523) can be utilized. The denoising process (523) can then produce a denoised or partially denoised 3-D image 320.

[0063] The denoising process 310 (523) can be a process that incorporates features that allow it to generalize well to different contrasts, anatomical structures, reconstruction filters, and noise levels. Such a denoising process 310 can compensate for the high noise levels inherent in the ULDCT images. The denoising process 310 (523) includes denoising the imaging data using a denoising algorithm having at least one tunable parameter. During the initial processing (520), the tunable parameter is typically set to a first value. The first value can be based on a generic or non-disease image, or it can be based on an expected disease associated with CT imaging.

[0064] In the example shown in the figure, the processing of the three-dimensional image (520) can also include the implementation of a super-resolution process 330 (526). As in the case of the denoising process 310, the super-resolution process 330 (526) can be implemented by a neural network or other algorithm, such as an AI-based learning algorithm, such as a CNN. In some embodiments, the super-resolution process 330 can include deblurring of the image. The super-resolution process 330 produces a higher resolution three-dimensional image 340.

[0065] The super-resolution process 330 typically interpolates the image to a smaller voxel size while maintaining or improving perceived image clarity. Alternatively, the super-resolution process 330 can operate on a fixed voxel size and perform a deconvolution-like operation to recover a higher resolution from a blurred image. The AI-based super-resolution process 330 can be trained on real CT images (including ULDCT images) or more general image material (such as natural high-resolution photographs). The super-resolution process 330 (526) can include performing the AI-based super-resolution process using a super-resolution algorithm having at least one tunable parameter. During initial processing (520), the tunable parameter is typically set to a first value. The first value can be based on a generic or non-disease image, or it can be based on an expected disease associated with CT imaging.

[0066] In the embodiment shown in the figures, both the denoising process 310 (523) and the super-resolution process 330 (526) are applied sequentially. However, it should be understood that both processes can be incorporated into a single neural network, such as a CNN. In addition, although the two processes 310, 330 are shown as being applied to the 3D image 300, in some embodiments, the processes can be applied directly to the projection data before reconstruction (510). In addition, in some embodiments, one or both processes 310, 330 can be applied to a two-dimensional plane in a three-dimensional CT imaging data set perpendicular to the projection direction for generating a two-dimensional image discussed below.

[0067] In some embodiments, processing may also include identifying (530) at least one physical element in the 3D image. Once identified (530), the physical element of the 3-D image may then be removed or masked (535). By removing or masking (535) the element prior to generating the 2-D image, such physical element may be removed from a simulated x-ray generated from the projection data.

[0068] The identified physical element (530) may be an anatomical element, such as one or more ribs or a heart. For example, by removing such anatomical elements from the simulated X-ray, a radiologist viewing the image may more easily see other anatomical elements of interest, and the simulated X-ray may show a cut in the patient's chest without obstructing the ribs.

[0069] Alternatively, the identified physical element (530) may be a table or support 207 or an implant. CT imaging data is typically acquired from a patient lying on the table or support 207, as in the imaging device 200 described above. In contrast, conventional planar X-rays are typically acquired from a standing patient. Thus, by removing the support 207, the simulated X-rays appear more natural to the radiologist viewing the image. Similarly, removing the implant may provide a better view of the patient's anatomy.

[0070] In some embodiments, rather than removing the identified physical elements (530), the physical elements may be weighted. Similarly, different portions of the three-dimensional image 300 may be weighted differently. The removal, masking, or weighting of the identified (530) physical elements may be based on an algorithm having tunable parameters. Therefore, and as discussed in the context of the denoising (523) and super-resolution (526) processes, a first value may be provided for the tunable parameter prior to initial processing.

[0071] After processing the 3-D image (520), the method continues to generate a two-dimensional image 350 based on the three-dimensional image (540). Although this can be performed in a variety of ways, in some embodiments, such generation of the two-dimensional image 350 is by tracking X-rays from a simulated radiation source external to the object of the three-dimensional image. For example, such an X-ray tracing process can be performed by implementing a Siddons-Jacobs ray tracing algorithm.

[0072] Once the two-dimensional image 350 is generated, the method then proceeds to identify the disease represented in the two-dimensional image (550). Such identification can be performed using a pre-trained disease classification network 360. Such a network can be configured to distinguish a specific set of diseases associated with a specific anatomical structure that appears in the two-dimensional image 350. For example, chest imaging can be processed using a disease classification network 360 that is trained to identify cardiac hypertrophy 370, emphysema 380, and individual nodules 380. Therefore, a disease classification model can be previously trained based on potential diseases. In addition to identifying specific diseases, the network 360 can be designed to output additional parameters of the identification, such as the degree of uncertainty in the result. Therefore, in an embodiment where a portion of the three-dimensional image is masked before generating an updated two-dimensional image 540, it can be meaningful to base the parameters on this uncertainty in the result because the nodules can be identified but partially hidden.

[0073] Upon identifying a disease represented in the two-dimensional image 350, the method recalibrates (560) the image formation chain by 1) reconstructing the projection data into a 3-D image or 2) processing the original reconstructed 3-D image in the image domain (without reconstruction in the projection domain). This produces an updated 3-D image. This recalibration is based on the identified disease (550) and / or the output of the pre-trained disease classification network 360. This recalibration includes modifying some portion of the processing (520) of the image so that at least one parameter of the reprocessing is based on the identified disease (550) and / or the output of the network 360. For example, in some embodiments, the recalibration can take into account the reported uncertainty of the disease classification network in addition to the actual identified disease.

[0074] As described above, the processing (520) of the image may include denoising (523) and / or a super-resolution process (526). As described above, the denoising (523) may utilize a denoising algorithm having tunable parameters. Then, the recalibration (560) of the image formation chain may include providing a second value for the tunable parameter of the denoising algorithm (523) that is different from the first value. The second value may then be based on the identified disease (550). Similarly, the super-resolution process (526) may utilize a super-resolution algorithm having tunable parameters. The recalibration (560) of the image formation chain may then include providing a second value for the tunable parameter of the super-resolution algorithm (526) that is different from the first value. As in the case of the denoising algorithm, the second value may be based on the identified disease (550).

[0075] Similarly, in embodiments where physical elements are identified (530) and removed, masked, or weighted (535), calibration (560) of the image formation chain can set a tunable parameter for an algorithm controlling the identification, removal, masking, or weighting. Thus, while the tunable parameter may initially be provided with a first value, the calibration (560) may set it to a second value different from the first value. Thus, removal or masking of at least one physical element may be based on identifying (550) a disease represented in the two-dimensional image.

[0076] Once the 2-D image (540) is generated (as calibrated via the image formation chain and based on the at least one updated parameter), the method proceeds to presenting the updated 2-D image to the user (570).

[0077] Figure 6 An embodiment of a ray tracing process applied to a three-dimensional image 300 to generate a two-dimensional image 350 (540) is illustrated. The ray tracing process can then be performed by a process of simulating x-rays 345 by causing incident x-ray photons from a simulated radiation source 600 to propagate through the reconstructed three-dimensional image 300. The two-dimensional image 350 can be generated by means of a neural network (such as a CNN), and in this case, the CNN can be combined with one or more of the denoising and super-resolution processes discussed herein. Such a neural network can be a generative adversarial network (GAN). In such an embodiment, many or all of the steps described herein can be incorporated into a single network so that the network is provided with CT volume data and a simulated CXR projection is output.

[0078] In some embodiments, the two-dimensional image (540) may be generated based on an algorithm having tunable parameters, such as an algorithm utilizing a CNN. Such a CNN may be a network configured to generate a digitally reconstructed radiography (DRR). As in the case of tunable parameters of a denoising algorithm (523) and a super-resolution algorithm (526), ​​a first value may be provided to such tunable parameter during initial generation (540) of the two-dimensional image. In such embodiments, after identifying the disease (550) and calibrating the image formation chain (560) and during reprocessing of the image, the method may generate an updated two-dimensional image based on a second value for the tunable parameter, the second value being based on the identification of the disease represented in the two-dimensional image.

[0079] In some embodiments, the projection angle or orientation of the ray tracing process (540) may be adjusted to improve the resulting two-dimensional image 350. Similarly, the weighting of physical elements in the three-dimensional image 300 may be adjusted to improve the resulting two-dimensional image 350. Such characteristics may be controlled by at least one tunable parameter, which may be set by a calibration process (560).

[0080] Figure 7 is a flow chart illustrating a method for training an image forming chain according to the present invention. The method begins by retrieving projection data collected from multiple angles about a central axis (700). Thus, the collection of such training data is similar to the above description of Figure 5 (500) discussed.

[0081] The training data also includes an indicator of a disease state associated therewith. Such an indicator of a disease state may indicate a specific disease, such as cardiac hypertrophy 370, emphysema 380, and the presence of an individual nodule 380. Alternatively, the indicator of a disease state may indicate that the training data corresponds to a healthy anatomical structure.

[0082] The method then reconstructs (710) and processes the reconstructed 3D image (720). The processing of the image (720) is similar to that described above with respect to Figure 5 The processing discussed, and proceeds along a similar chain of image formation. Thus, the processing of the image (720) includes denoising (723) and a super-resolution process (726). This processing is followed by the generation of a two-dimensional image (730) based on the training image data.

[0083] As mentioned above Figure 5 As discussed, the processes incorporated into the image formation chain, including the denoising process (723), the super-resolution process (726), and the generation of the two-dimensional image (730), can each incorporate tunable parameters that can initially be provided with default or first values. The methods described herein for training the image formation chain can be used to derive disease-specific second values ​​for those tunable parameters to enhance the output of the image formation chain after the disease is identified.

[0084] After generating the two-dimensional image (730), a previously trained disease classification network is applied (740) to the two-dimensional image output and used to generate a disease classification prediction associated with the training image data. A loss metric is then defined (750) based on the output of the previously trained disease classification network and an indicator of a disease state associated with the training image data.

[0085] The loss metric may correspond to a classification error 400, which may then be provided to a pre-trained disease classification network. The image may then be chained backpropagated 410 ( Figure 4) such classification error in order to identify and refine values ​​for tunable parameters of the image formation chain. Accordingly, the loss metric (750) is then used to tune (760) at least one parameter of the processing of the training images to minimize the loss metric. Thus, since the loss metric (750) is defined based on the disease state indicator, the values ​​for one or more tunable parameters identified by the training method are optimized for a specific disease state.

[0086] The training process can be repeated multiple times on a large number of training images for various specific disease states. In addition, the tunable parameter can be a single tunable parameter in one of the denoising process 310, the super-resolution process 330, or the DRR simulation process 345, or it can be, for example, a first tunable parameter in the denoising process 310 and a second tunable parameter in the super-resolution state, each of which will be tuned by the training method to find the ideal value.

[0087] In some embodiments, at least one parameter affects suppression of anatomical structures in an output image generated by processing the three-dimensional image using a tuned value for the at least one parameter. Thus, an indicator of a disease state can indicate the presence of an identified disease, and the loss metric can optimize visibility of the indicator of the identified disease. For example, the loss metric can optimize the tunable parameter to increase visibility of a nodule.

[0088] In some such embodiments, the loss metric optimizes visibility of nodules that are otherwise occluded by adjacent anatomical structures. In this way, the tuned parameters can produce masking of obstructed anatomical structures, such as, for example, the thorax.

[0089] For example, in the context of chest imaging, lung nodules are notoriously difficult to identify in chest X-rays due to the resolution of the film, the size of the nodules, and the likelihood of nodules being occluded by adjacent anatomical structures. In this embodiment, the image formation chain is then optimized toward suppressing occlusion of adjacent anatomical structures in DRR image simulation and enhancing nodule detail and granularity through super-resolution and denoising modules, respectively.

[0090] Each model within the image formation chain can usually accept the output of each other model.

[0091] Pre-trained disease classification networks expect images (or in this case DRRs) from the domain of the images they have been trained on. Specifically, this means that if the network has been trained on chest X-rays, etc., then DRR images are provided from chest CT scans. The added value of combining these features is to create a model optimized for real-world applications such as disease classification, which can simultaneously generate and optimize 2-D representations from 3-D images.

[0092] In some embodiments, the image formation chain can be extended in two directions. In one embodiment, the image formation chain can include a disease classification model based on the underlying three-dimensional image. This can supplement the two-dimensional pre-trained classification model by incorporating contextual information and spatial information into the classification of a given case. Similarly, the image formation chain can be extended to include reconstruction of CT images so that learnable parameters can be utilized in the reconstruction process. For example, the visualization of individual CT slices can be improved for specific disease categories.

[0093] Various embodiments of the present invention may be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or in a combination of the two. The executable code of the method according to one embodiment of the present invention may be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Preferably, the computer program product may include a non-transitory program code stored on a computer-readable medium for performing the method according to the present invention when the program product is executed on a computer. In one embodiment, the computer program may include a computer program code suitable for performing the method according to various embodiments of the present invention when the computer program is running on a computer. The computer program may be embodied on a computer-readable medium.

[0094] While the present invention has been described at some length and with a certain degree of particularity with respect to several described embodiments, it is not intended to be limited to any such details or embodiments or to any particular embodiment, but rather is to be interpreted with reference to the appended claims so as to provide the broadest possible interpretation of such claims in light of the prior art and so as to effectively encompass the intended scope of the invention.

[0095] All examples and conditional language described herein are intended to be used for teaching purposes, to help readers understand the principles of the present invention and the concepts contributed by the inventors to promote this area, and should be interpreted as not being limited to such specific examples and conditions. In addition, all statements describing the principles, aspects and embodiments of the present invention and their specific examples herein are intended to cover their structural and functional equivalents. In addition, these equivalents are intended to include both currently known equivalents and equivalents developed in the future, that is, any element developed to perform the same function, regardless of the structure.

Claims

1. A computer-implemented method for processing medical computed tomography (CT) data, comprising: retrieving projection data acquired by scanning the object; processing the projection data to reconstruct a three-dimensional image; generating a two-dimensional image based on the three-dimensional image; identifying a disease in the two-dimensional image or uncertainty in the identification of the disease; reprocessing the projection data or the three-dimensional image into an updated three-dimensional image, wherein at least one parameter of the reprocessing is based on the uncertainty in the disease or the identification of the disease; as well as An updated two-dimensional image is generated based on the updated three-dimensional image.

2. The method according to claim 1, wherein: The two-dimensional image is generated by a neural network.

3. The method according to claim 1, wherein: Reprocessing the projection data or the three-dimensional image is performed by a neural network. 4 . The method of claim 1 , further comprising denoising the three-dimensional image using the first value for the at least one parameter. 5 . The method of claim 4 , further comprising denoising the updated three-dimensional image using a second value for the at least one parameter, the second value being based on the uncertainty in the disease or the identification of the disease. 6 . The method of claim 1 , further comprising applying artificial intelligence (AI)-based super-resolution to the three-dimensional image using the first value for the at least one parameter so that the three-dimensional image produces a high-resolution image.

7. The method of claim 6, further comprising applying the AI-based super-resolution to the updated three-dimensional image using a second value for the at least one parameter so that the updated three-dimensional image produces another high-resolution image, wherein The second value is based on the uncertainty in the disease or the identification of the disease.

8. The method of claim 1, further comprising denoising the 3D image and / or the updated 3D image by a trained convolutional neural network (CNN).

9. The method according to claim 1, wherein: Generating the two-dimensional image includes using a digitally reconstructed radiography (DRR) network that uses a first value for the at least one parameter.

10. The method according to claim 9, wherein: Generating the updated two-dimensional image includes using the DRR network, the DRR network using a second value for the at least one parameter, the second value being based on the uncertainty in the disease or the identification of the disease.

11. The method of claim 1 , further comprising identifying at least one physical element in the three-dimensional image and / or the updated three-dimensional image, and removing or masking the at least one physical element before generating an output image.

12. The method according to claim 11, wherein: The at least one physical element is a plurality of ribs or a heart.

13. The method according to claim 11, wherein: Removing or masking the at least one physical element is based on the uncertainty of the disease or the identification of the disease in the two-dimensional image.

14. The method according to claim 1, wherein: Identifying the disease includes using a trained disease classification model.

15. A system for processing medical computed tomography (CT) data, comprising: a memory storing a plurality of instructions; as well as a processor coupled to the memory and configured to execute the plurality of instructions to: retrieving projection data acquired by scanning the object; processing the projection data to reconstruct a three-dimensional image; generating a two-dimensional image based on the three-dimensional image; identifying a disease in the two-dimensional image or uncertainty in the identification of the disease; reprocessing the projection data or the three-dimensional image into an updated three-dimensional image, wherein at least one parameter of the reprocessing is based on the uncertainty in the disease or the identification of the disease; as well as An updated two-dimensional image is generated based on the updated three-dimensional image.

16. The system of claim 15, wherein: Generating the two-dimensional image and reprocessing the projection data or the three-dimensional image are performed by a neural network.

17. The system of claim 15, wherein: The three-dimensional image is denoised using a first value for the at least one parameter, and wherein the updated three-dimensional image is denoised using a second value for the at least one parameter, the second value being based on the disease or the uncertainty in the identification of the disease.

18. The system of claim 15, wherein: The three-dimensional image and / or the updated three-dimensional image are denoised by a trained convolutional neural network (CNN).

19. The system of claim 15, wherein: The two-dimensional image is generated by a digitally reconstructed radiography (DRR) network using a first value for the at least one parameter, and wherein the updated two-dimensional image is generated by the DRR network using a second value for the at least one parameter, the second value being based on the uncertainty in the disease or the identification of the disease.

20. The system of claim 15, wherein: Artificial intelligence (AI)-based super-resolution is applied to the three-dimensional image using a first value for the at least one parameter so that the three-dimensional image produces a high-resolution image, and wherein the AI-based super-resolution is applied to the updated three-dimensional image using a second value for the at least one parameter so that the updated three-dimensional image produces another high-resolution image, wherein the second value is based on the disease or the uncertainty in the identification of the disease.

Images