System and method for medical imaging

Abstract

A method includes receiving a plurality of fluoroscopic images of a lung from a C-arm device, where each fluoroscopic image is obtained with the C-arm device positioned at a particular pose of a plurality of poses traversed by the C-arm device as the C-arm device moves through a range of rotation including a sweep angle between 45 degrees and 120 degrees; generating an enhanced tomographic image of the lung by utilizing: the trained machine learning model and the plurality of fluoroscopic images; and outputting a representation of the enhanced tomographic image, where the lung comprises a lesion of less than 30 millimeters, and the representation is an axial slice showing the boundary of the lesion having a contrast noise value of at least 5 compared to the background of the representation, when tested by the method.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for medical imaging. More particularly, the present disclosure relates to systems and methods for obtaining CT-like medical images using a C-arm-based fluoroscopic imaging device. Background Art

[0002] A computed tomography ("CT") scan is a type of medical imaging that uses a rotating X-ray tube to obtain detailed internal images. CT scans are used as the "gold standard" for diagnosing a variety of diseases, including cancers such as lung cancer. However, CT scanning equipment is expensive, requires a licensed radiologist to operate, and even in facilities where such equipment is available, the number of scans that can be performed in a given period of time is limited. Furthermore, CT scans involve delivering a large radiation dose to the patient and are therefore performed only when the diagnostic benefit outweighs the patient's radiation-related cancer risk.

[0003] C-arm mounted fluoroscopic imaging devices, such as X-ray imaging devices, are widely used for diagnostic and therapeutic procedures, are easily accessible to a variety of professionals operating within a typical hospital, and are associated with low radiation doses. In some cases, the C-arm mounted imaging device is used to obtain a two-dimensional image sequence as the C-arm moves through a range of rotation. Such an image sequence can be used to "reconstruct" a three-dimensional volume or tomographic image. However, the image quality of such a reconstructed volume is uneven and may not be sufficient for certain types of clinical applications, such as the diagnosis of early stage lung cancer presenting small (e.g., less than 10 mm) lesions or low density lesions (e.g., having a density of less than -300 Hounsfield units) - such as the type referred to in the medical literature as semisolid or ground glass opacities. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Some embodiments of the present invention are described herein by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings, it should be emphasized that the details shown are by way of example only and are provided for purposes of illustrative discussion of the embodiments of the present invention. In this regard, the description taken in conjunction with the drawings will provide those skilled in the art with a clear understanding of how the embodiments of the present invention may be practiced.

[0005] Figure 1 An exemplary medical imaging system is shown.

[0006] Figure 2 An exemplary process for generating CT-like images is shown.

[0007] Figure 3A An exemplary sequence of fluoroscopic images is shown.

[0008] Figure 3B Shows the use Figure 3AExample tomographic images reconstructed from fluoroscopic images are shown.

[0009] Figure 3C Shown for Figure 3B The tomographic images shown are compared to an exemplary reference CT image.

[0010] Figure 3D According to an exemplary embodiment, a Figure 3B An exemplary enhanced tomographic image (eg, a CT-like image) generated from a tomographic image is shown.

[0011] Figure 4 An exemplary process for training a machine learning model to enhance tomographic images is shown.

[0012] Figure 5A An exemplary ground truth tomographic image is shown.

[0013] Figure 5B Shown based on Figure 5A An exemplary sequence of simulated fluoroscopic images generated from ground truth tomographic images is shown.

[0014] Figure 5C Shows the use Figure 5B An exemplary simulated tomographic image reconstructed from a simulated fluoroscopic image is shown.

[0015] Figure 5D An exemplary enhanced simulated tomographic image generated using a trained tomographic image enhancement machine learning model is shown.

[0016] Figure 6A Shown are exemplary enhanced simulated tomographic images marked for evaluation using the test method.

[0017] Figure 6B shows a further stage in the test method Figure 6A Example enhanced simulated tomographic image.

[0018] Figure 6C Shown are various background arcs marked at a later stage of the test method Figure 6A Example simulated tomographic images of .

[0019] Figure 7A Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0020] Figure 7B Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0021] Figure 7CRepresentative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0022] Figure 8A Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0023] Figure 8B Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0024] Figure 8C Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0025] Figure 9A Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0026] Figure 9B Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0027] Figure 9C Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0028] Figure 10A Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0029] Figure 10B Representative images with sufficient contrast-to-noise ratio to discern lesions are shown.

[0030] Figure 10C Representative images with sufficient contrast-to-noise ratio to discern lesions are shown. Summary of the Invention

[0031] In some embodiments, a method includes: a) receiving, by a controller unit, a plurality of fluoroscopic images of at least a portion of a lung of a patient from a C-arm device, wherein each of the plurality of fluoroscopic images is obtained with the C-arm device positioned in a particular posture among a plurality of postures traversed by the C-arm device as the C-arm device moves through a range of motion, wherein the range of motion includes at least a range of rotation, and wherein the range of rotation includes a sweep angle between 45 degrees and 120 degrees; b) generating, by the controller unit, an enhanced tomographic image of at least a portion of the lung by utilizing at least: a trained machine learning model and the plurality of fluoroscopic images; and c) outputting, by the controller unit, a representation of the enhanced tomographic image, wherein, when tested by a testing method under the following circumstances: (a) at least a portion of the lung of the patient includes at least one lesion having a size less than 30 mm, and (b) the representation of the enhanced tomographic image is an axial slice showing defined boundaries of the at least one lesion, the at least one lesion having a contrast-to-noise value of at least 5 compared to a background of the representation.

[0032] In some embodiments, the step of generating an enhanced tomographic image comprises reconstructing the tomographic image based on a plurality of fluoroscopic images; and enhancing the tomographic image using a trained machine learning model to generate the enhanced tomographic image. In some embodiments, the step of reconstructing the tomographic image based on the plurality of fluoroscopic images comprises reconstructing the tomographic image using filtered back projection. In some embodiments, the step of reconstructing the tomographic image based on the plurality of fluoroscopic images comprises determining a pose of each of the plurality of fluoroscopic images. In some embodiments, determining the pose of each of the plurality of fluoroscopic images comprises image-based pose estimation. In some embodiments, the image-based pose estimation comprises identifying at least one of an anatomical feature or a radiopaque marker.

[0033] In some embodiments, the representation of the enhanced tomographic image includes axial slices.

[0034] In some embodiments, the sweep angle is between 45 degrees and 90 degrees.

[0035] In some embodiments, when tested by the testing method under the following circumstances: (a) at least a portion of a lung of the patient includes at least one lesion having a size less than 10 mm, and (b) the representation of the enhanced tomographic image is an axial slice showing defined boundaries of the at least one lesion, the at least one lesion has a contrast-to-noise value of at least 5 compared to a background of the representation.

[0036] In some embodiments, the trained machine learning model comprises a gradient descent machine learning model.

[0037] In some embodiments, the range of motion also includes a translational range of motion.

[0038] In some embodiments, a method includes: obtaining, by a controller unit, a plurality of fluoroscopic images of a region of interest of tissue of a patient, wherein each of the plurality of fluoroscopic images is obtained by the C-arm device when the C-arm device is positioned in a particular posture among a plurality of postures traversed by the C-arm device as the C-arm device moves through a rotation range, and wherein the rotation range includes a rotation of less than 180 degrees; reconstructing, by the controller unit, a tomographic image including the region of interest using the plurality of fluoroscopic images; and enhancing, by the controller unit, the tomographic image using a trained tomographic image enhancement machine learning model to generate an enhanced tomographic image, wherein the trained tomographic image enhancement machine learning model has been trained by a training process comprising: receiving CT image data of a plurality of patients, wherein the CT image data comprises a ground truth tomographic image of each of the plurality of patients; receiving a plurality of fluoroscopic images of each of the plurality of patients, generating a plurality of simulated fluoroscopic images based on the CT image data of each of the plurality of patients, wherein each of the plurality of simulated fluoroscopic images The fluoroscopic image corresponds to a specific posture of the C-arm device at a specific angle, and the multiple fluoroscopic images correspond to multiple angles spanning a rotation range between 45 degrees and 120 degrees; based on the multiple fluoroscopic images of each patient in the multiple patients, reconstruct a simulated tomographic image of each patient in the multiple patients, wherein the simulated tomographic image includes multiple artifacts; using the tomographic image enhancement machine learning model to perform an enhancement process to enhance the simulated tomographic image of each patient in the multiple patients to reduce the multiple artifacts to obtain an enhanced simulated tomographic image of each patient in the multiple patients; scoring each enhanced simulated tomographic image based on the multiple artifacts and the corresponding ground truth tomographic image to obtain a corresponding performance score of the tomographic image enhancement machine learning model; when the performance score of the tomographic image enhancement machine learning model is less than a predetermined performance score threshold, updating the parameters of the tomographic image machine learning model; and iteratively repeating the enhancement process until the corresponding performance score is equal to or higher than the predetermined performance score threshold to produce a trained tomographic image enhancement machine learning model.

[0039] In some embodiments, the plurality of fluoroscopic images of each patient in the plurality of patients includes a plurality of actual fluoroscopic images of at least some of the plurality of patients.

[0040] In some embodiments, the plurality of fluoroscopic images of each patient in the plurality of patients includes a plurality of simulated fluoroscopic images of at least some of the plurality of patients. In some embodiments, the plurality of simulated fluoroscopic images are generated by projecting the at least one tomographic image into a plurality of poses.

[0041] In some embodiments, the step of reconstructing the tomographic image comprises reconstructing the tomographic image using filtered back projection.

[0042] In some embodiments, the range of rotation includes rotation between 45 degrees and 120 degrees. DETAILED DESCRIPTION

[0043] In conjunction with the accompanying drawings, various detailed embodiments of the present disclosure are disclosed herein; however, it will be understood that the disclosed embodiments are only illustrative. In addition, each of the examples given in conjunction with the various embodiments of the present disclosure is intended to be illustrative rather than restrictive.

[0044] Throughout this specification, the following terms have the meanings explicitly associated herein, unless the context clearly dictates otherwise. As used herein, the phrases "in one embodiment" and "in some embodiments" do not necessarily refer to the same embodiment, although they may refer to the same embodiment. Furthermore, as used herein, the phrases "in another embodiment" and "in some other embodiments" do not necessarily refer to different embodiments, although they may refer to different embodiments. Thus, as described below, the various embodiments can be readily combined without departing from the scope or spirit of the present disclosure.

[0045] In addition, the term "based on" is not exclusive and allows for being based on additional factors that are not described unless the context clearly dictates otherwise. In addition, throughout the specification, the meanings of "a," "an," and "the" include plural references. The meaning of "in" includes "in" and "on."

[0046] It should be understood that at least one aspect / functionality of the various embodiments described herein can be performed in real time and / or dynamically. As used herein, the term "real time" refers to when another event / action has occurred, the event / action can occur simultaneously or nearly simultaneously. For example, "real-time processing," "real-time computing," and "real-time execution" all involve performing calculations during the actual time that a related physical process (e.g., a user interacting with an application on a mobile device) occurs, so that the results of the calculations can be used to guide the physical process.

[0047] As used herein, the terms "dynamically" and "automatically" and logical and / or linguistic related words and / or derivatives thereof mean that certain events and / or actions can be triggered and / or occur without any human intervention. In some embodiments, events and / or actions according to the present disclosure can be real-time and / or based on a predetermined period of at least one of the following: nanoseconds, nanoseconds, milliseconds, milliseconds, seconds, seconds, minutes, minutes, hours, days, weeks, months, etc.

[0048] Exemplary embodiments relate to techniques for generating CT-like images. More specifically, exemplary embodiments relate to techniques for generating CT-like images using a standard C-arm mounted fluoroscopic imaging device by using reconstruction techniques and machine learning enhancement techniques, as will be described below.

[0049] Figure 1 An example of an implementation of a medical imaging system 100 according to one or more exemplary embodiments of the present disclosure is shown. In some embodiments, the medical imaging system 100 includes a computing device 110 for generating CT-like images according to one or more embodiments of the present disclosure. In some embodiments, the computing device 110 may include hardware components such as a processor 112, which may include local or remote processing components. In some embodiments, the processor 112 may include any type of data processing capability, such as hardware logic circuits, such as application specific integrated circuits (ASICs) and programmable logic, or such as a computing device, such as a microcomputer or microcontroller including a programmable microprocessor. In some embodiments, the processor 112 may include data processing capabilities provided by a microprocessor. In some embodiments, the microprocessor may include memory, processing, interface resources, controllers, and counters. In some embodiments, the microprocessor may also include one or more programs stored in the memory.

[0050] Similarly, the computing device 110 may include a storage device 114, such as one or more local and / or remote data storage solutions, such as, for example, a local hard drive, a solid-state drive, a flash drive, a database, or other local data storage solutions, or any combination thereof; and / or a remote data storage solution, such as a server, a mainframe, a database or cloud service, a distributed database, or other suitable data storage solutions, or any combination thereof. In some embodiments, the storage device 114 may include, for example, a suitable non-transitory computer-readable medium, such as, for example, a random access memory (RAM), a read-only memory (ROM), one or more buffers and / or caches, and other memory devices, or any combination thereof.

[0051] In some embodiments, the computing device 110 may implement a computer engine for generating CT-like images based on fluoroscopic images obtained using a C-arm-based imaging device according to exemplary embodiments described herein. In some embodiments, the terms "computer engine" and "engine" identify at least one software component and / or a combination of at least one software component and at least one hardware component that is designed / programmed / configured to manage / control other software and / or hardware components (such as a library, software development kit (SDK), object, etc.).

[0052] Examples of hardware elements that may be included in the computing device 110 may include a processor, a microprocessor, a circuit, a circuit element (e.g., a transistor, a resistor, a capacitor, an inductor, etc.), an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a digital signal processor (DSP), a field programmable gate array (FPGA), a logic gate, a register, a semiconductor device, a chip, a microchip, a chipset, a graphics processing unit (GPU), etc. In some embodiments, one or more processors may be implemented as a complex instruction set computer (CISC) or a reduced instruction set computer (RISC) processor; an x86 instruction set compatible processor, a multi-core or any other microprocessor or central processing unit (CPU). In various implementations, one or more processors may be a dual-core processor, a dual-core mobile processor, etc.

[0053] Examples of software that can be executed by computing device 110 may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. Determining whether to implement an embodiment using hardware elements and / or software elements can vary depending on any number of factors, such as desired computing rate, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance constraints.

[0054] In some embodiments, to generate CT-like images according to the exemplary embodiments described herein, the computing device 110 may include a computer engine, such as, for example, a CT-like image generation engine 116. In some embodiments, the CT-like image generation engine 116 may include dedicated and / or shared software components, hardware components, or a combination thereof. For example, the CT-like image generation engine 116 may include a dedicated processor and storage device. However, in some embodiments, the CT-like image generation engine 116 may share hardware resources, including the processor 112 and storage device 114 of the computing device 110 via, for example, a bus 118. Thus, the CT-like image generation engine 116 may include a memory including software and software instructions, such as, for example, machine learning models and / or logic, for generating CT-like images using fluoroscopic images obtained from a C-arm mounted imaging device.

[0055] In some embodiments, the medical imaging system 100 includes a C-arm assembly 120. In some embodiments, the C-arm assembly 120 includes a radiation source 122 and an imaging device 124 (e.g., a fluoroscopic imaging device, such as an X-ray imaging device), which are mounted to a C-arm 126 to allow the radiation source 122 and the imaging device 124 to move relative to the patient P through a rotation range to obtain a two-dimensional image sequence of the patient P from various viewing angles (e.g., postures). In some embodiments, the C-arm assembly 120 is a fixed assembly (e.g., in a fixed position relative to the room and / or relative to the bed). In some embodiments, the C-arm assembly 120 is a movable assembly (e.g., movable from one room to another and / or from one bed to another).

[0056] Figure 2 2 shows an example of an implementation of method 200 according to one or more exemplary embodiments of the present disclosure. In some embodiments, Figure 2 The method shown is a method for generating a CT-like image based on a medical image obtained using a conventional C-arm. Figure 1 The exemplary methods are described using elements of the exemplary medical imaging system 100 described below. In other embodiments, the exemplary methods described below can be practiced using other system arrangements. The exemplary method 200 is described with reference to a process for imaging a patient's lungs. In other embodiments, substantially the same method can be used to image a patient's lungs or liver, perform an image-guided biopsy procedure, administer an injection of pain medication, visualize a tool proximate a patient's spine, deliver a drug therapy or ablation therapy to a target location (e.g., a lesion) in the body, or for any other purpose for which CT imaging is generally utilized.

[0057] In step 210, the computing device 110 receives a sequence of fluoroscopic images from the C-arm apparatus 120. In some embodiments, the sequence of fluoroscopic images is an image of at least a portion of a lung of a patient. In some embodiments, each fluoroscopic image in the sequence of fluoroscopic images is obtained by the imaging device 124 positioned at a particular pose among a plurality of poses traversed by the C-arm 126 as the C-arm 126 moves through a range of motion. In some embodiments, the range of motion includes rotational motion through a range of rotation (e.g., the C-arm 126 rotates, thereby rotating the imaging device 124 about the patient P). In some embodiments, the range of motion includes both rotational motion through a range of rotation and translational motion through a range of translation (e.g., the C-arm 126 moves along a rotational axis, thereby linearly translating the imaging device 124 relative to the patient P). In some embodiments, a range of motion that includes both rotational motion and translational motion is advantageous in order to avoid physical obstacles to rotational motion (e.g., a table or patient's body blocking the movement of the C-arm 126).

[0058] Figure 3A An exemplary sequence of fluoroscopic images is shown. Figure 3A Six fluoroscopic images are shown as a representative sampling. In some embodiments, the sequence of fluoroscopic images received in step 210 varies based on factors such as frame rate (e.g., in a range between 5 images per second and 20 images per second, such as 8 images per second or 15 images per second) and acquisition duration (e.g., in a range between 10 seconds and 120 seconds, such as between 30 seconds and 60 seconds). For example, in some embodiments, the sequence of fluoroscopic images received in step 210 includes 80 to 1800 images.

[0059] In some embodiments, the rotation range is less than 180 degrees. In some embodiments, the rotation range is within the range of 0 to 180 degrees. In some embodiments, the rotation range is within the range of 15 to 180 degrees. In some embodiments, the rotation range is within the range of 30 to 180 degrees. In some embodiments, the rotation range is within the range of 45 to 180 degrees. In some embodiments, the rotation range is within the range of 60 to 180 degrees. In some embodiments, the rotation range is within the range of 75 to 180 degrees. In some embodiments, the rotation range is within the range of 90 to 180 degrees. In some embodiments, the rotation range is within the range of 105 to 180 degrees. In some embodiments, the rotation range is within the range of 120 to 180 degrees. In some embodiments, the rotation range is within the range of 135 to 180 degrees. In some embodiments, the rotation range is within the range of 150 to 180 degrees. In some embodiments, the rotation range is within the range of 165 to 180 degrees.

[0060] In some embodiments, the rotation range is within the range of 0 to 165 degrees. In some embodiments, the rotation range is within the range of 15 to 165 degrees. In some embodiments, the rotation range is within the range of 30 to 165 degrees. In some embodiments, the rotation range is within the range of 45 to 165 degrees. In some embodiments, the rotation range is within the range of 60 to 165 degrees. In some embodiments, the rotation range is within the range of 75 to 165 degrees. In some embodiments, the rotation range is within the range of 90 to 165 degrees. In some embodiments, the rotation range is within the range of 105 to 165 degrees. In some embodiments, the rotation range is within the range of 120 to 165 degrees. In some embodiments, the rotation range is within the range of 135 to 165 degrees. In some embodiments, the rotation range is within the range of 150 to 165 degrees.

[0061] In some embodiments, the rotation range is in the range of 0 to 150 degrees. In some embodiments, the rotation range is in the range of 15 to 150 degrees. In some embodiments, the rotation range is in the range of 30 to 150 degrees. In some embodiments, the rotation range is in the range of 45 to 150 degrees. In some embodiments, the rotation range is in the range of 60 to 150 degrees. In some embodiments, the rotation range is in the range of 75 to 150 degrees. In some embodiments, the rotation range is in the range of 90 to 150 degrees. In some embodiments, the rotation range is in the range of 105 to 150 degrees. In some embodiments, the rotation range is in the range of 120 to 150 degrees. In some embodiments, the rotation range is in the range of 135 to 150 degrees.

[0062] In some embodiments, the rotation range is within the range of 0 to 135 degrees. In some embodiments, the rotation range is within the range of 15 to 135 degrees. In some embodiments, the rotation range is within the range of 30 to 135 degrees. In some embodiments, the rotation range is within the range of 45 to 135 degrees. In some embodiments, the rotation range is within the range of 60 to 135 degrees. In some embodiments, the rotation range is within the range of 75 to 135 degrees. In some embodiments, the rotation range is within the range of 90 to 135 degrees. In some embodiments, the rotation range is within the range of 105 to 135 degrees. In some embodiments, the rotation range is within the range of 120 to 135 degrees. In some embodiments, the rotation range is within the range of 0 to 120 degrees. In some embodiments, the rotation range is within the range of 15 to 120 degrees. In some embodiments, the rotation range is within the range of 30 to 120 degrees. In some embodiments, the rotation range is within the range of 45 to 120 degrees. In some embodiments, the range of rotation is in the range of 60 to 120 degrees. In some embodiments, the range of rotation is in the range of 75 to 120 degrees. In some embodiments, the range of rotation is in the range of 90 to 120 degrees. In some embodiments, the range of rotation is in the range of 105 to 120 degrees.

[0063] In some embodiments, the rotation range is in the range of 0 to 105 degrees. In some embodiments, the rotation range is in the range of 15 to 105 degrees. In some embodiments, the rotation range is in the range of 30 to 105 degrees. In some embodiments, the rotation range is in the range of 45 to 105 degrees. In some embodiments, the rotation range is in the range of 60 to 105 degrees. In some embodiments, the rotation range is in the range of 75 to 105 degrees. In some embodiments, the rotation range is in the range of 90 to 105 degrees. In some embodiments, the rotation range is in the range of 0 to 90 degrees. In some embodiments, the rotation range is in the range of 15 to 90 degrees. In some embodiments, the rotation range is in the range of 30 to 90 degrees. In some embodiments, the rotation range is in the range of 45 to 90 degrees. In some embodiments, the rotation range is in the range of 60 to 90 degrees. In some embodiments, the rotation range is in the range of 75 to 90 degrees.

[0064] In some embodiments, the rotation range is in the range of 0 to 75 degrees. In some embodiments, the rotation range is in the range of 15 to 75 degrees. In some embodiments, the rotation range is in the range of 30 to 75 degrees. In some embodiments, the rotation range is in the range of 45 to 75 degrees. In some embodiments, the rotation range is in the range of 60 to 75 degrees. In some embodiments, the rotation range is in the range of 0 to 60 degrees. In some embodiments, the rotation range is in the range of 15 to 60 degrees. In some embodiments, the rotation range is in the range of 30 to 60 degrees. In some embodiments, the rotation range is in the range of 45 to 60 degrees. In some embodiments, the rotation range is in the range of 0 to 45 degrees. In some embodiments, the rotation range is in the range of 15 to 45 degrees. In some embodiments, the rotation range is in the range of 30 to 45 degrees. In some embodiments, the rotation range is in the range of 0 to 30 degrees. In some embodiments, the rotation range is in the range of 15 to 30 degrees. In some embodiments, the range of rotation is in the range of 0 degrees to 15 degrees.

[0065] In step 220, the computing device 110 applies a reconstruction process to the sequence of fluoroscopic images to generate a tomographic image (e.g., a non-enhanced tomographic image) (e.g., a three-dimensional image). In some embodiments, the reconstruction is performed based at least in part on the known pose of each image in the sequence of fluoroscopic images. In some embodiments, the pose of each image is determined using image-based pose estimation (e.g., based on recognition of objects such as anatomical features or radiopaque markers shown in each image). In some embodiments, the image-based pose estimation is performed in a manner described in U.S. Patent No. 10,674,970, the contents of which are incorporated herein by reference in their entirety. In some embodiments, the reconstruction process includes filtered back projection ("FBP"), algebraic reconstruction technique ("ART"), simultaneous algebraic reconstruction technique ("SART"), or simultaneous iterative reconstruction technique ("SIRT"). Figure 3B Coronal, axial, and sagittal slices of exemplary tomographic images are shown.

[0066] In some embodiments, the tomographic image generated in step 220 is a similar type of image as a "gold standard" reference CT image that would be obtained for the same patient represented by the image received in step 210, but of lower quality. Figure 3CCoronal, axial, and sagittal slices of an exemplary reference CT image are shown. For example, in some embodiments, the tomographic image generated in step 220 has lower resolution, lower fidelity, or is otherwise of lower quality than the reference CT image. In some embodiments, the quality of the tomographic image generated in step 220 is insufficient for a clinician (e.g., a radiologist) to discern lesions or other objects that are less than 30 mm in size. In some embodiments, the quality of the tomographic image generated in step 220 is insufficient for a clinician (e.g., a radiologist) to discern lesions or other objects that are less than 10 mm in size. In some embodiments, the tomographic image generated in step 220 includes one or more artifacts.

[0067] In step 230, the computing device 110 applies the trained tomographic image enhancement machine learning model to the tomographic image generated in step 220, thereby generating an enhanced tomographic image. In some embodiments, the trained tomographic image enhancement machine learning model is trained, as will be described in further detail below with respect to exemplary method 400. In some embodiments, the enhanced tomographic image generated in step 230 is of a similar type to a "gold standard" reference CT image that would be obtained for the same patient represented by the image received in step 210, and is of comparable quality. For example, in some embodiments, the enhanced tomographic image generated in step 230 includes fewer artifacts than the simulated tomographic image generated in step 220. Figure 3D An exemplary enhanced tomographic image is shown.

[0068] In step 240, computing device 110 outputs a representation of the enhanced tomographic image. In some embodiments, the representation is a two-dimensional slice of the enhanced tomographic image, such as an axial slice. In some embodiments, the output is to a display (e.g., a display communicatively coupled to computing device 110). In some embodiments, the output is to a further software program (e.g., a program that generates enhanced images, a surgical planning program, etc.). As in Figure 3C and Figure 3D As can be seen, in some embodiments, the quality of the enhanced tomographic image generated in step 230 is sufficient for a clinician (e.g., a radiologist) to discern lesions or other objects that are less than 30 mm in size. In some embodiments, the quality of the enhanced tomographic image generated in step 230 is sufficient for a clinician (e.g., a radiologist) to discern lesions or other objects that are less than 10 mm in size.

[0069] Figure 4 An example of an implementation of a method according to one or more exemplary embodiments of the present disclosure is shown. In some embodiments, Figure 4 The method 400 shown is a method for training a trained tomographic image enhancement machine learning model for generating CT-like images based on medical images obtained using a conventional C-arm. Figure 1 The exemplary method 400 is described with reference to the elements of the exemplary system 100, but one skilled in the art will appreciate that other suitable system arrangements are possible. For example, in the following description, the method 400 is described with reference to training a machine learning model at the computing device 110 of the exemplary medical imaging system 100. However, in other embodiments, Figure 4 The illustrated exemplary training method 400 is performed in a separate computing environment and then provided to the medical imaging system 100 to perform the exemplary method 200 as described above.

[0070] In step 410, computing device 110 receives CT image data including a CT image of each patient in a plurality of patients. In some embodiments, the CT image data is used as a ground truth tomographic image for each patient. Figure 5A Coronal, axial, and sagittal slices of exemplary CT images of a patient are shown.

[0071] In step 420, the sequence of fluoroscopic images for each patient is provided to the computing device 110. In some embodiments, the computing device 110 receives a sequence of actual fluoroscopic images for each of the plurality of patients for which CT image data was received in step 410. In some embodiments, the computing device 110 generates a sequence of simulated fluoroscopic images for the CT images of each patient. In some embodiments, each simulated fluoroscopic image is generated by projecting the CT image to a pose of a desired simulated fluoroscopic image. In some embodiments, each simulated fluoroscopic image is generated using a machine learning model such as a DeepDRR machine learning model. In some embodiments, each simulated fluoroscopic image in a particular sequence of simulated fluoroscopic images for a particular patient corresponds to a particular pose of the C-arm assembly at a particular angle, such that each simulated fluoroscopic image simulates a fluoroscopic image that would be obtained using a fluoroscopic imaging device mounted on the C-arm assembly if the C-arm assembly was positioned at the particular angle. In some embodiments, the plurality of simulated fluoroscopic images for each particular patient corresponds to a plurality of angles spanning a rotational range, such that the sequence of simulated fluoroscopic images for each particular patient simulates a sequence of fluoroscopic images that would be acquired during a C-arm imaging study of the particular patient, wherein the C-arm apparatus is moved through the rotational range. In some embodiments, each particular rotational range simulated in step 420 is any of the rotational ranges discussed above with reference to step 210 of method 200. Figure 5B Shown based on Figure 5A The exemplary simulated fluoroscopic images generated from the exemplary CT images shown are similar to those discussed above. Figure 3A similar, Figure 5B Six exemplary simulated fluoroscopic images are shown; in some embodiments, between 80 and 1800 fluoroscopic images are provided in step 420 .

[0072] In step 430, computing device 110 reconstructs a simulated tomographic image for each of the plurality of patients based on each actual or simulated fluoroscopic image sequence provided in step 420. In some embodiments, each simulated tomographic image is generated using one of the techniques described above with reference to step 220 of method 200. Similar to the discussion above with reference to step 220 of method 200, in some embodiments, the simulated tomographic images generated in step 430 are generally comparable to the ground-truth CT images on which they are based, but of lower quality. For example, in some embodiments, the simulated tomographic images generated in step 430 have lower resolution, lower fidelity, or are otherwise of lower quality than the corresponding ground-truth CT images. In some embodiments, the quality of the simulated tomographic images generated in step 430 is insufficient for a clinician (e.g., a radiologist) to discern lesions or other objects less than 10 mm in size. In some embodiments, the simulated tomographic images generated in step 430 include one or more artifacts. Figure 5C Shown based on Figure 5B Example simulated fluoroscopic images are shown, along with example simulated tomographic images generated.

[0073] As described herein, exemplary embodiments relate to training a tomographic image enhancement machine learning model to produce a trained tomographic image enhancement machine learning model. In some embodiments, the tomographic image enhancement machine learning model is a gradient descent machine learning model employing a suitable loss function, such as projected gradient descent, fast gradient sign method, stochastic gradient descent, batch gradient descent, mini-batch gradient descent, or other suitable gradient descent techniques. In some embodiments, the tomographic image enhancement machine learning model comprises a regression model. In some embodiments, the tomographic image enhancement machine learning model comprises a neural network. In step 440, the computing device 110 performs an enhancement process using the tomographic image enhancement machine learning model to enhance the quality of at least some of the simulated tomographic images generated in step 430, thereby obtaining corresponding enhanced simulated tomographic images for each of a plurality of patients. In some embodiments, the enhancement process is performed on a randomly selected subset of the simulated tomographic images. In some embodiments, each of the enhanced simulated tomographic images generated in step 440 may include one or more artifacts. In some embodiments, the artifacts may be caused by factors such as, for example, reconstruction performed using simulated fluoroscopic images that lack some data.

[0074] In step 450, a score is assigned to each of the enhanced simulated tomographic images generated in step 440. In some embodiments, the score is assigned based on each enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on a comparison of each enhanced simulated tomographic image with the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on one or more artifacts in each enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on an automated (e.g., algorithmic) comparison performed by computing device 110. In some embodiments, the score is assigned by a user. In some embodiments, a performance score of the tomographic image enhancement machine learning model is calculated based on the score of each of the enhanced simulated tomographic images.

[0075] In step 460 , the computing device 110 determines whether the performance score of the tomographic image enhancement machine learning model exceeds a predetermined performance score threshold.

[0076] If the performance score of the tomographic image enhancement machine learning model does not exceed the predetermined performance score threshold, the method 400 proceeds to step 480. In step 480, the parameters (e.g., weights) of the tomographic image enhancement machine learning model are updated based on the performance of the gradient descent or other tomographic image enhancement machine learning model, for example, using backpropagation. After step 480, the method 400 returns to step 440 and repeats the enhancement process of step 440.

[0077] If the performance score of the tomographic image enhancement machine learning model exceeds the predetermined performance score threshold, the method 400 is completed, and the output of the method is a trained tomographic image enhancement machine learning model in step 470. In other words, in some embodiments, the enhancement process is iteratively repeated (e.g., by repeating steps 440, 450, 460, and 480) until the performance score of the tomographic image enhancement machine learning model exceeds the predetermined performance score threshold, thereby generating a trained tomographic image enhancement machine learning model. Figure 5D It shows that after completing the above training process, based on Figure 5C Example enhanced simulated tomographic images generated from example simulated tomographic images are shown.

[0078] In some embodiments, the exemplary tomographic image enhancement machine learning model can be trained until the loss function reaches an acceptable value / threshold (e.g., 0.99 (1%), 0.98 (2%), 0.97 (3%), 0.96 (4%), 0.95 (5%), ..., 0.90 (10%), ..., 0.85 (15%), etc.). In some embodiments, the loss function can measure the error between the enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the error can be calculated as the L2 and / or L1 norm distance.

[0079] In some embodiments, the CT-like images generated according to the exemplary techniques described above are comparable to "gold standard" CT images obtained using a CT scanning device. In some embodiments, the CT-like images generated according to the exemplary techniques described above have similar quality to "gold standard" CT images obtained using a CT scanning device and are generated using source data derived from a standard C-arm mounted fluoroscopic imaging device. Therefore, the CT-like images can be generated without the use of a CT scanning device, which is expensive and is generally in high demand where they are available. In some embodiments, the CT-like images generated as described above can be used to identify lesions or other objects that are, for example, less than 10 mm in size, thereby enabling early diagnosis of diseases such as lung cancer.

[0080] In some embodiments, the recognizability of objects displayed in exemplary CT-like images is defined based on the contrast-to-noise ratio ("CNR"). As used herein, contrast refers to the brightness difference between an object and its surroundings (as represented in an image or display) that makes the object distinguishable. As used herein, CNR can be calculated according to the following expression:

[0081] CNR=(mean(ObjectMask)-mean(BackgroundMaskPortion)) / STD(BackgroundMaskPortion)

[0082] In this expression, mean(ObjectMask) refers to the average brightness value within the region defined as the lesion, mean(BackgroundMaskPortion) refers to the average brightness value within the region defined as the background, and std(backgroundmaskportion) refers to the standard deviation of the brightness value within the region defined as the background. The regions used in this article are identified as follows.

[0083] The lesion region is identified by marking the lesion in the exemplary CT-like image in a manner deemed appropriate by a person of ordinary skill in the art, and the lesion region may be referred to as InputObjectMask. Figure 6A An exemplary image 600 is shown with an InputObjectMask, which is obtained using user input as described above, depicted by a ring 610. As used herein, a lesion region (also referred to as an ObjectMask) is defined based on the InputObjectMask by the following expression:

[0084] ObjectMask=erosion(InputObjectMask,2)

[0085] In this expression, erosion(mask,N) means eroding the mask by N pixels. In other words, ObjectMask is generated by eroding (e.g., reducing the size of) InputObjectMask by two (2) pixels. Return to Reference Figure 6A , the ObjectMask is depicted by a ring 620 that is smaller than the ring 610. For the above values, the background region is defined by dilating the object mask (e.g., increasing its size) by the following expression:

[0086] BackgroundMask=dilation(ObjectMask,10)-dilation(ObjectMask,5)

[0087] In this expression, dilation(mask,N) means dilating the mask (i.e., increasing its size) by N pixels. Return to Reference Figure 6A , BackgroundMask is depicted by the area between the inner ring 630 and the outer ring 640 which is larger than the ring 610 .

[0088] As used herein, BackgroundMaskPortion refers to a 180 degree arc of the background region BackgroundMask selected to maximize the calculated value CNR. Figure 6B Shown based on Figure 6A The image 600 shows the creation of a possible 180 degree arc. Figure 6B In the figure, arc 650 is defined by a vector V 660 extending from the centroid C 670 of the ObjectMask depicted in 620, with vector V 660 extending in a direction defined by an angle α 680 measured from a reference direction 690. As used herein, reference direction 690 is horizontal when viewed in image 600, but this is merely an arbitrary reference point, and a range of potential 180-degree arcs can be defined based on any given reference point. According to the test method described herein, all possible 180-degree arcs are computationally evaluated. Figure 6C An exemplary sequence of images is shown, each image of which shows a different 180 degree arc 650 corresponding to image 600 . Figure 6C Six (6) such images are shown in FIG as a representative sampling, but the actual number of possible arcs to be evaluated will be larger (eg, the number of such arcs depends on the resolution of the images being evaluated).

[0089] As described above, based on the lesion ObjectMask (e.g., Figure 6B The ring 620 depicts the ObjectMask) and the selected arc (e.g., by Figure 6B The BackgroundMaskPortion depicted by arc 650 is shown to calculate the contrast-to-noise ratio CNR using the following expression:

[0090] CNR=(mean(ObjectMask)-mean(BackgroundMaskPortion)) / STD(BackgroundMaskPortion)

[0091] In this expression, mean(ObjectMask) refers to the mean value of the brightness within the area defined as the lesion, mean(BackgroundMaskPortion) refers to the mean value of the brightness within the selected arc, and std(backgroundmaskportion) refers to the standard deviation of the brightness values ​​within the selected arc. As used herein, an object is identifiable in an image if its CNR (as determined based on the arc that produces the maximum CNR) exceeds 5.

[0092] 7A to 10C Representative images are shown, which illustrate the suitability of a threshold CNR value of 5 to assess the identifiability of lesions in images. Figure 7A 、 Figure 7B and Figure 7C Images 710, 720, 730 are shown having CNR values ​​of 12.7, 7.0, and 3.9, respectively. It can be seen that images 710 and 720 are of sufficient quality so that lesion 740 is discernible, while image 730 is not. Figure 8A 、 Figure 8B and Figure 8C Images 810, 820, 830 are shown having CNR values ​​of 9.9, 4.4, and 2.6, respectively. It can be seen that image 810 is of sufficient quality so that lesion 840 is discernible, while images 820 and 830 are not. Figure 9A 、 Figure 9B and Figure 9C Images 910, 920, 930 are shown having CNR values ​​of 11.7, 6.2, and 3.4, respectively. It can be seen that images 910 and 920 are of sufficient quality so that lesion 940 is discernible, while image 930 is not. Figure 10A 、 Figure 10B and Figure 10C Images 1010, 1020, 1030 are shown having CNR values ​​of 7.3, 4.5, and 3.0, respectively. It can be seen that image 1010 is of sufficient quality so that lesion 1040 is discernible, while images 1020 and 1030 are not.

[0093] In some embodiments, the exemplary techniques described above are applied to generate a CT-like image of at least one organ (e.g., lung, kidney, liver, etc.) of a patient. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least one lesion in at least one organ (e.g., lung, kidney, liver, etc.) of a patient. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least one lesion in at least one organ (e.g., lung, kidney, liver, etc.) of a patient to enable diagnosis and / or treatment of the at least one lesion. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least a portion of a patient's spine. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image for performing an image-guided biopsy procedure. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image for performing an injection of analgesic. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image for delivering therapy (e.g., drug therapy or ablation therapy) to the lesion shown in the CT-like image.

[0094] In some embodiments, the exemplary techniques are capable of producing CT-like images in which lesions less than 30 mm in size are distinguishable from their surroundings based on the tests described above. In some embodiments, the exemplary techniques are capable of producing CT-like images in which lesions less than 10 mm in size are distinguishable from their surroundings based on the tests described above. In some embodiments, the exemplary techniques are capable of producing CT-like images in which lesions with a density less than -300 Hounsfield units (HU) are distinguishable from their surroundings based on the tests described above. In some embodiments, the exemplary techniques are capable of producing CT-like images without the involvement of a licensed radiologist who would be required to operate a CT scanner. In some embodiments, the exemplary techniques are capable of producing CT-like images without exposing the patient to the high radiation doses that would be delivered by a CT scanner.

[0095] Although several embodiments of the present invention have been described, it should be understood that these embodiments are illustrative only and not restrictive, and that many modifications will be apparent to those skilled in the art. For example, all dimensions discussed herein are provided as examples only and are intended to be illustrative rather than restrictive.

Claims

1. A method comprising: a) receiving, by a controller unit, a plurality of fluoroscopic images of at least a portion of a lung of a patient from a C-arm device, wherein each of the plurality of fluoroscopic images is acquired with the C-arm assembly positioned in a particular pose among a plurality of poses traversed by the C-arm assembly as the C-arm assembly moves through a range of motion, wherein the range of motion includes at least a range of rotation, and wherein the range of rotation includes a sweep angle between 45 degrees and 120 degrees; b) generating, by the controller unit, an enhanced tomographic image of the at least a portion of the lung by utilizing at least: A trained machine learning model, and the plurality of fluoroscopic images; and c) outputting, by said controller unit, a representation of said enhanced tomographic image, Wherein, when tested by the test method under the following conditions: (a) said at least a portion of said lung of said patient comprises at least one lesion having a size less than 30 mm, and (b) said representation of said enhanced tomographic image is an axial slice showing defined boundaries of said at least one lesion, The at least one lesion has a contrast-to-noise value of at least 5 compared to a background of the representation.

2. The method of claim 1 , wherein the step of generating the enhanced tomographic image comprises: reconstructing a tomographic image based on the plurality of fluoroscopic images; as well as The tomographic image is enhanced using the trained machine learning model to generate the enhanced tomographic image. 3 . The method of claim 2 , wherein reconstructing the tomographic image based on the plurality of fluoroscopic images comprises reconstructing the tomographic image using filtered back projection. 4 . The method of claim 2 , wherein the step of reconstructing the tomographic image based on the plurality of fluoroscopic images comprises determining a pose of each of the plurality of fluoroscopic images. The method of claim 4 , wherein determining the pose of each of the plurality of fluoroscopic images comprises image-based pose estimation. The method of claim 5 , wherein the image-based pose estimation comprises recognition of at least one of anatomical features or radiopaque markers.

7. The method of claim 1, wherein the representation of the enhanced tomographic image comprises axial slices. The method of claim 1 , wherein the sweep angle is between 45 degrees and 90 degrees.

9. The method of claim 1, wherein when tested by the test method under the following conditions: (a) said at least a portion of said lung of said patient comprises at least one lesion having a size less than 10 mm, and (b) said representation of said enhanced tomographic image is an axial slice showing defined boundaries of said at least one lesion, The at least one lesion has a contrast-to-noise value of at least 5 compared to a background of the representation.

10. The method of claim 1, wherein the trained machine learning model comprises a gradient descent machine learning model. The method of claim 1 , wherein the range of motion further comprises a translational range of motion.

12. A method comprising: obtaining, by a controller unit, a plurality of fluoroscopic images of a region of interest of a patient's tissue, wherein each of the plurality of fluoroscopic images is acquired by the C-arm apparatus with the C-arm apparatus positioned in a particular pose among a plurality of poses traversed by the C-arm apparatus as the C-arm apparatus moves through a range of rotation, and wherein the rotation range includes rotations less than 180 degrees; reconstructing, by the controller unit, a tomographic image including the region of interest using the plurality of fluoroscopic images; as well as enhancing, by the controller unit, the tomographic image using the trained tomographic image enhancement machine learning model to generate an enhanced tomographic image, wherein the trained tomographic image enhancement machine learning model has been trained through a training process comprising: receiving CT image data of multiple patients, wherein the CT image data comprises a ground truth tomographic image of each patient of the plurality of patients; receiving a plurality of fluoroscopic images of each patient of the plurality of patients, generating a plurality of simulated fluoroscopic images based on the CT image data of each of the plurality of patients, wherein each of the plurality of simulated fluoroscopic images corresponds to a specific posture of the C-arm device at a specific angle, and wherein the plurality of fluoroscopic images correspond to a plurality of angles spanning a rotation range between 45 degrees and 120 degrees; reconstructing a simulated tomographic image of each of the plurality of patients based on the plurality of fluoroscopic images of each of the plurality of patients, wherein the simulated tomographic image includes a plurality of artifacts; performing an enhancement process using a tomographic image enhancement machine learning model to enhance the simulated tomographic image of each of the plurality of patients to reduce the plurality of artifacts to obtain an enhanced simulated tomographic image of each of the plurality of patients; scoring each enhanced simulated tomographic image based on the plurality of artifacts and the corresponding ground truth tomographic image to obtain a corresponding performance score of the tomographic image enhancement machine learning model; updating parameters of the tomographic image machine learning model when the performance score of the tomographic image enhancement machine learning model is less than a predetermined performance score threshold; and The enhancement process is iteratively repeated until the corresponding performance score is equal to or higher than the predetermined performance score threshold to generate the trained tomographic image enhancement machine learning model. 13 . The method of claim 12 , wherein the plurality of fluoroscopic images of each of the plurality of patients comprises a plurality of actual fluoroscopic images of at least some of the plurality of patients.

14. The method of claim 12, wherein the plurality of fluoroscopic images of each of the plurality of patients comprises a plurality of simulated fluoroscopic images of at least some of the plurality of patients.

15. The method of claim 14, wherein the plurality of simulated fluoroscopic images are generated by projecting at least one tomographic image into a plurality of poses.

16. The method of claim 12, wherein reconstructing the tomographic image comprises reconstructing the tomographic image using filtered back projection.

17. The method of claim 12, wherein the range of rotation includes rotation between 45 degrees and 120 degrees.

Images