Systems and methods for medical imaging

Abstract

1. A method comprising: receiving a plurality of fluoroscopic images of the lungs from a C-arm device, each fluoroscopic image being acquired with the C-arm device positioned in a particular one of a plurality of postures assumed by the C-arm device while being moved through a range of motion including a rotational range, the rotational range encompassing a sweep angle between 45 degrees and 120 degrees; generating an enhanced tomographic image of the lungs by utilizing a trained machine learning model and the plurality of fluoroscopic images; and outputting a representation of the enhanced tomographic image, wherein the lungs contain a lesion smaller than 30 millimeters, and the lesion has a contrast-to-noise value of at least 5 compared to a background of the representation when tested by the following method: the lungs contain a lesion smaller than 30 millimeters, and the representation is an axial section showing the boundary of the lesion.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to systems and methods for medical imaging, and more particularly, to systems and methods for acquiring CT-like medical images using a C-arm based fluoroscopic imaging device. [Background technology]

[0002] Computed tomography (CT) scanning is a type of medical imaging that uses a rotating X-ray tube to obtain detailed internal images. CT scanning serves as the "gold standard" for diagnosing many medical conditions, including cancer, such as lung lesions. However, CT scanning devices are expensive, require qualified radiologists to operate them, and even in facilities where such devices are available, the number of scans that can be performed in a given period of time is limited. Furthermore, CT scanning involves exposing patients to large amounts of radiation, so it is only performed when the diagnostic benefit outweighs the patient's radiation-related cancer risk.

[0003] C-arm-mounted fluoroscopic imaging devices, such as X-ray imaging devices, are widely used for diagnostic and therapeutic procedures, are easily accessible by a variety of specialists working within a typical hospital, and involve low radiation doses. In some cases, C-arm-mounted imaging devices are used to acquire a series of two-dimensional images while the C-arm is moved through a rotational range. Such a series of images can be used to "reconstruct" a three-dimensional volumetric or tomographic image. However, the image quality of such reconstructed volumes is not uniform and may be insufficient for some types of clinical applications, such as the diagnosis of early-stage lung cancer, which presents with small (e.g., less than 10 millimeters) or low-density (e.g., density less than -300 Hounsfield units) lesions, such as those types known in the medical literature as semisolid or ground-glass opacity lesions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,674,970

[0005] Certain embodiments of the present invention are herein described, by way of example only, with reference to the accompanying drawings. Reference will now be made in detail to the drawings, it being stressed that the particulars shown are exemplary and are for illustrative purposes only to explain embodiments of the invention. In this regard, it will become apparent to those skilled in the art how embodiments of the invention may be practiced, with reference to the drawings and the description. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an exemplary medical imaging system. [Figure 2] FIG. 1 illustrates an exemplary process for generating a CT-like image. [Figure 3A] FIG. 1 illustrates an exemplary sequence of perspective images. [Figure 3B] 3B shows an exemplary tomographic image reconstructed using the fluoroscopic image shown in FIG. 3A. FIG. [Figure 3C] FIG. 3C shows an exemplary reference CT image for comparison with the tomographic image shown in FIG. 3B. [Figure 3D] 3C illustrates an exemplary enhanced tomographic image (eg, a CT-like image) generated based on the tomographic image shown in FIG. 3B, according to an exemplary embodiment. [Figure 4] FIG. 1 illustrates an exemplary process for training a machine learning model for enhancing tomographic images. [Figure 5A] FIG. 1 shows an exemplary ground truth tomographic image. [Figure 5B] Figure 5B shows an exemplary series of simulated fluoroscopic images generated based on the ground truth tomographic images shown in Figure 5A. [Figure 5C]FIG. 5C shows an exemplary simulated tomographic image reconstructed using the simulated fluoroscopic image shown in FIG. 5B. [Figure 5D] FIG. 1 illustrates an exemplary enhanced simulated tomographic image generated using a trained tomographic image enhancement machine learning model. [Figure 6A] FIG. 10 shows an exemplary enhanced simulated tomographic image labeled for evaluation using the test method. [Figure 6B] 6B shows the exemplary enhanced simulated tomographic image of FIG. 6A at a further stage of the testing method. [Figure 6C] 6B shows the example simulated tomographic image of FIG. 6A labeled with various background arcs at a later stage in the testing method. [Figure 7A] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 7B] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 7C] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. [Figure 8A] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 8B] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. [Figure 8C] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. [Figure 9A] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 9B] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 9C] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. [Figure 10A] FIG. 1 shows a representative image with sufficient contrast-to-noise ratio to distinguish the lesions. [Figure 10B] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. [Figure 10C] FIG. 1 shows a representative image with insufficient contrast-to-noise ratio to distinguish the lesions. Summary of the Invention

[0007] In some embodiments, the method includes: a) receiving, by the controller unit, from a C-arm device, a plurality of fluoroscopic images of at least a portion of a patient's lungs, wherein each image of the plurality of fluoroscopic images is acquired with the C-arm device positioned in a particular one of a plurality of postures assumed by the C-arm device while the C-arm device is moved through a range of motion, the range of motion including at least a rotational range, the rotational range encompassing a sweep angle between 45 degrees and 120 degrees; b) generating, by the controller unit, an enhanced tomographic image of the at least portion of the lungs by utilizing at least the trained machine learning model and the plurality of fluoroscopic images; and c) outputting, by the controller unit, a representation of the enhanced tomographic image, wherein (a) a region of the at least portion of the patient's lungs includes at least one lesion less than 30 millimeters in size, and (b) the at least one lesion has a contrast-to-noise value of at least 5 compared to a background of the representation when tested by a testing method in which (a) a region of the at least portion of the patient's lungs includes at least one lesion less than 30 millimeters in size, and (b) the enhanced tomographic image representation is an axial slice showing a clear boundary of the at least one lesion.

[0008] In some examples, generating the enhanced tomographic image includes reconstructing the tomographic image based on a plurality of fluoroscopic images and enhancing the tomographic image using a trained machine learning model to generate the enhanced tomographic image. In some examples, reconstructing the tomographic image based on the plurality of fluoroscopic images includes reconstructing the tomographic image using filtered back projection. In some examples, reconstructing the tomographic image based on the plurality of fluoroscopic images includes determining a pose of each of the plurality of fluoroscopic images. In some examples, determining a pose of each of the plurality of fluoroscopic images includes image-based pose estimation. In some examples, the image-based pose estimation includes recognizing at least one of anatomical features or radiopaque markers.

[0009] In some embodiments, the enhanced tomographic image representation includes an axial slice.

[0010] In some embodiments, the sweep angle is between 45 degrees and 90 degrees.

[0011] In some embodiments, when tested by a testing method in which (a) at least a portion of an area of ​​a patient's lung contains at least one lesion less than 10 millimeters in size, and (b) the enhanced tomographic image representation is an axial slice showing a clear boundary of the at least one lesion, the at least one lesion has a contrast-to-noise value of at least 5 compared to the background of the representation.

[0012] In some examples, the trained machine learning model includes a gradient descent machine learning model.

[0013] In some embodiments, the range of motion further includes a translational range of motion.

[0014] In some embodiments, the method includes the steps of: acquiring, by a controller unit, a plurality of fluoroscopic images of a region of interest of tissue of a patient, wherein each image of the plurality of fluoroscopic images is acquired by the C-arm device with the C-arm device positioned at a particular one of a plurality of postures assumed by the C-arm device while the C-arm device is moved through a range of rotation, the range of rotation encompassing a rotation of less than 180 degrees; reconstructing, by the controller unit, a tomographic image including the region of interest using the plurality of fluoroscopic images; enhancing, by the controller unit, the tomographic image using a trained tomographic image enhancement machine learning model to generate enhanced tomographic images, wherein the trained tomographic image enhancement machine learning model receives CT image data of a plurality of patients, the CT image data including ground truth tomographic images for each of the plurality of patients; receiving the plurality of fluoroscopic images for each of the plurality of patients; and generating a plurality of simulated fluoroscopic images based on the CT image data for each of the plurality of patients. the plurality of simulated fluoroscopic images corresponding to a particular posture of the C-arm device at a particular angle, the plurality of fluoroscopic images corresponding to a plurality of angles spanning a rotation range between 45 degrees and 120 degrees; reconstructing a simulated tomographic image for each of a plurality of patients based on the plurality of fluoroscopic images for each of the plurality of patients, the simulated tomographic image including a plurality of artifacts; performing an enhancement process using a tomographic image enhancement machine learning model to enhance the simulated tomographic image for each of the plurality of patients to reduce the plurality of artifacts and obtain an enhanced simulated tomographic image for each of the plurality of patients; scoring each enhanced simulated tomographic image based on the plurality of artifacts and the corresponding ground truth tomographic image to obtain a corresponding performance score of the tomographic image enhancement machine learning model; and updating parameters of the tomographic image enhancement machine learning model while the performance score of the tomographic image enhancement machine learning model is less than a predetermined performance score threshold.and training the tomographic image enhancement machine learning model by a training process that includes iteratively repeating the enhancement process until a corresponding performance score is equal to or greater than a predetermined performance score threshold to obtain a trained tomographic image enhancement machine learning model.

[0015] In some examples, the plurality of fluoroscopic images for each of the plurality of patients includes a plurality of actual fluoroscopic images for at least some of the plurality of patients.

[0016] In some embodiments, the plurality of fluoroscopic images for each of the plurality of patients includes a plurality of simulated fluoroscopic images for at least some of the plurality of patients, hi some embodiments, the plurality of simulated fluoroscopic images are generated by projecting at least one tomographic image in a plurality of poses.

[0017] In some embodiments, reconstructing the tomographic image includes reconstructing the tomographic image using filtered backprojection.

[0018] In some embodiments, the range of rotation includes rotation between 45 degrees and 120 degrees. DETAILED DESCRIPTION OF THE INVENTION

[0019] Detailed Description of the Drawings While various detailed embodiments of the present disclosure are disclosed herein in conjunction with the accompanying drawings, it should be understood that the disclosed embodiments are merely exemplary. Additionally, each of the examples given in connection with the various embodiments of the present disclosure are intended to be illustrative, not limiting.

[0020] Throughout this specification, the following terms have the meanings expressly associated therewith, unless the context dictates otherwise. As used herein, the phrases "in one embodiment" and "in some embodiments" do not necessarily refer to the same embodiment, but may. Additionally, as used herein, the phrases "in another embodiment" and "in some other embodiments" do not necessarily refer to different embodiments, but may. Thus, as explained below, various embodiments may be readily combined without departing from the scope or spirit of the present disclosure.

[0021] Additionally, the term "based on" is not exclusive and allows for based on additional unrecited factors unless the context dictates otherwise. Additionally, throughout this specification, the meanings of "a," "an," and "the" include plural referents. The meaning of "in" includes "in" and "on."

[0022] It is understood that at least one aspect / functionality of various embodiments described herein may be performed in real time and / or dynamically. The term "real time," as used herein, covers an event / action that may occur instantaneously or near-instantaneously in time when another event / action occurs. For example, "real-time processing," "real-time computation," and "real-time execution" all relate to performing a computation during the actual time that an associated physical process (e.g., a user operating an application on a mobile device) occurs, such that the results of the computation can be used to direct the physical process.

[0023] The terms "dynamically" and "automatically," as well as their logical and / or linguistic relatives and / or derivatives, as used herein, mean that a particular event and / or action may be triggered and / or occur without human intervention. In some examples, the event and / or action according to the present disclosure may be real-time and / or based on at least one predetermined period of nanoseconds, nanoseconds, milliseconds, milliseconds, seconds, seconds, minutes, minutes, hours, hours, days, days, weeks, months, etc.

[0024] Exemplary embodiments relate to techniques for generating CT-like images, and more particularly, to techniques for generating CT-like images using standard C-arm mounted fluoroscopic imaging devices with the reconstruction and machine learning enhancement techniques described below.

[0025] FIG. 1 illustrates an example implementation of a medical imaging system 100 according to one or more exemplary embodiments of the present disclosure. In some embodiments, the medical imaging system 100 includes a computing device 110 for generating CT-like images according to one or more exemplary embodiments of the present disclosure. In some embodiments, the computing device 110 may include hardware components such as a processor 112, which may include local or remote processing components. In some embodiments, the processor 112 may include any type of data processing capability, such as hardware logic, e.g., an application specific integrated circuit (ASIC) and programmable logic, or a computing device, e.g., a microcomputer or microcontroller, including a programmable microprocessor. In some embodiments, the processor 112 may include data processing capability provided by a microprocessor. In some embodiments, the microprocessor may include memory, processing, interface resources, a controller, and a counter. In some embodiments, the microprocessor may also include one or more programs stored in memory.

[0026] Similarly, computing device 110 may include storage 114, such as one or more local and / or remote data storage solutions, e.g., a local hard drive, solid-state drive, flash drive, database, or other local data storage solution, or any combination thereof, and / or a remote data storage solution, such as a server, mainframe, database or cloud service, distributed database, or other suitable data storage solution, or any combination thereof. In some embodiments, storage 114 may include a suitable non-transitory computer-readable medium, such as, for example, random access memory (RAM), read-only memory (ROM), one or more buffers and / or caches, or any combination thereof, among other memory devices.

[0027] In some embodiments, computing device 110 may implement a computer engine for generating CT-like images based on fluoroscopic images acquired using a C-arm-based imaging device according to example embodiments described herein. In some embodiments, the terms “computer engine” and “engine” refer to at least one software component and / or a combination of at least one software component and at least one hardware component designed, programmed, or configured to manage or control other software and / or hardware components (e.g., libraries, software development kits (SDKs), objects, etc.).

[0028] Examples of hardware elements that may be included in computing device 110 may include a processor, microprocessor, circuit, circuit element (e.g., transistor, resistor, capacitor, inductor, etc.), integrated circuit, application specific integrated circuit (ASIC), programmable logic device (PLD), digital signal processor (DSP), field programmable gate array (FPGA), logic gate, register, semiconductor device, chip, microchip, chip set, graphical processing unit (GPU). In some embodiments, one or more processors may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processor, an x86 instruction set compatible processor, a multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, one or more processors may be a dual-core processor, a dual-core mobile processor, etc.

[0029] Examples of software that may be executed by computing device 110 may include a software component, program, application, computer program, application program, system program, machine program, operating system software, middleware, firmware, software module, routine, subroutine, function, method, procedure, software interface, application program interface (API), instruction set, computing code, computer code, code segment, computer code segment, word, value, symbol, or any combination thereof. The decision whether an embodiment is implemented using hardware and / or software elements may depend on any number of factors, such as desired computation speed, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance constraints.

[0030] In some embodiments, to generate CT-like images in accordance with the example embodiments described herein, the computing device 110 may include a computer engine, including, for example, a CT-like image generation engine 116. In some embodiments, the CT-like image generation engine 116 may include dedicated and / or shared software components, hardware components, or a combination thereof. For example, the CT-like image generation engine 116 may include a dedicated processor and storage. However, in some embodiments, the CT-like image generation engine 116 may share hardware resources, including the processor 112 and storage 114 of the computing device 110, for example, via a bus 118. Thus, the CT-like image generation engine 116 may include software instructions, such as machine learning models and / or logic, for generating CT-like images using fluoroscopic images acquired from a C-arm-mounted imaging device, as well as a memory containing the software instructions.

[0031] In some embodiments, medical imaging system 100 includes a C-arm device 120. In some embodiments, C-arm device 120 includes a radiation source 122 and an imaging device 124 (e.g., a fluoroscopic imaging device such as an X-ray imaging device), with imaging device 124 mounted on a C-arm 126 such that radiation source 122 and imaging device 124 can be moved through a range of rotation relative to patient P to thereby acquire a series of two-dimensional images of patient P from various perspectives (e.g., postures). In some embodiments, C-arm device 120 is a fixed device (e.g., in a fixed position relative to a room and / or bed). In some embodiments, C-arm device 120 is a mobile device (e.g., capable of being moved from one room to another and / or from one bed to another).

[0032] FIG. 2 illustrates an example implementation of a method 200 according to one or more exemplary embodiments of the present disclosure. In some embodiments, the method illustrated in FIG. 2 is a method for generating a CT-like image based on medical images acquired using a conventional C-arm. The exemplary method is described below with reference to elements of the exemplary medical imaging system 100 described above with reference to FIG. 1. In other embodiments, the exemplary method described below is practiced using other system configurations. The exemplary method 200 is described with reference to a process for imaging a patient's lungs. In other embodiments, substantially the same method is used to image a patient's lungs or liver, perform an image-guided biopsy procedure, administer pain medication, visualize a tool adjacent a patient's spine, administer drug or ablation therapy to a target location (e.g., a lesion) within the body, or for any other purpose for which CT imaging is typically utilized.

[0033] In step 210, computing device 110 receives a series of fluoroscopic images from C-arm device 120. In some embodiments, the series of fluoroscopic images are images of at least a portion of a patient's lungs. In some embodiments, each fluoroscopic image in the series is acquired by imaging device 124 positioned in a particular position among multiple positions assumed by C-arm 126 while C-arm 126 is moved through a range of motion. In some embodiments, the range of motion includes rotational motion through a rotational range (e.g., rotation of C-arm 126 to rotate imaging device 124 around patient P). In some embodiments, the range of motion includes both rotational motion through a rotational range and translational motion through a translational range (e.g., movement of C-arm 126 along an axis of rotation to translate imaging device 124 linearly relative to patient P). In some embodiments, a range of motion that includes both rotational motion and translational motion is advantageous for avoiding physical obstacles to rotational motion (e.g., a table or the patient's body blocking movement of C-arm 126).

[0034] An exemplary series of fluoroscopic images is shown in FIG. 3A. FIG. 3A shows six fluoroscopic images as a representative sample. In some embodiments, the series of fluoroscopic images received in step 210 varies based on factors such as frame rate (e.g., ranging from 5 images per second to 20 images per second, such as 8 images per second or 15 images per second) and acquisition duration (e.g., ranging from 10 seconds to 120 seconds, such as ranging from 30 seconds to 60 seconds). For example, in some embodiments, the series of fluoroscopic images received in step 210 includes between 80 and 1800 images.

[0035] In some embodiments, the range of rotation is less than 180 degrees. In some embodiments, the range of rotation is between 0 degrees and 180 degrees. In some embodiments, the range of rotation is between 15 degrees and 180 degrees. In some embodiments, the range of rotation is between 30 degrees and 180 degrees. In some embodiments, the range of rotation is between 45 degrees and 180 degrees. In some embodiments, the range of rotation is between 60 degrees and 180 degrees. In some embodiments, the range of rotation is between 75 degrees and 180 degrees. In some embodiments, the range of rotation is between 90 degrees and 180 degrees. In some embodiments, the range of rotation is between 105 degrees and 180 degrees. In some embodiments, the range of rotation is between 120 degrees and 180 degrees. In some embodiments, the range of rotation is between 135 degrees and 180 degrees. In some embodiments, the range of rotation is between 150 degrees and 180 degrees. In some embodiments, the range of rotation is between 165 degrees and 180 degrees.

[0036] In some embodiments, the rotation range is from 0 degrees to 165 degrees. In some embodiments, the rotation range is from 15 degrees to 165 degrees. In some embodiments, the rotation range is from 30 degrees to 165 degrees. In some embodiments, the rotation range is from 45 degrees to 165 degrees. In some embodiments, the rotation range is from 60 degrees to 165 degrees. In some embodiments, the rotation range is from 75 degrees to 165 degrees. In some embodiments, the rotation range is from 90 degrees to 165 degrees. In some embodiments, the rotation range is from 105 degrees to 165 degrees. In some embodiments, the rotation range is from 120 degrees to 165 degrees. In some embodiments, the rotation range is from 135 degrees to 165 degrees. In some embodiments, the rotation range is from 150 degrees to 165 degrees.

[0037] In some embodiments, the rotation range is from 0 degrees to 150 degrees. In some embodiments, the rotation range is from 15 degrees to 150 degrees. In some embodiments, the rotation range is from 30 degrees to 150 degrees. In some embodiments, the rotation range is from 45 degrees to 150 degrees. In some embodiments, the rotation range is from 60 degrees to 150 degrees. In some embodiments, the rotation range is from 75 degrees to 150 degrees. In some embodiments, the rotation range is from 90 degrees to 150 degrees. In some embodiments, the rotation range is from 105 degrees to 150 degrees. In some embodiments, the rotation range is from 120 degrees to 150 degrees. In some embodiments, the rotation range is from 135 degrees to 150 degrees.

[0038] In some embodiments, the rotation ranges from 0 degrees to 135 degrees. In some embodiments, the rotation ranges from 15 degrees to 135 degrees. In some embodiments, the rotation ranges from 30 degrees to 135 degrees. In some embodiments, the rotation ranges from 45 degrees to 135 degrees. In some embodiments, the rotation ranges from 60 degrees to 135 degrees. In some embodiments, the rotation ranges from 75 degrees to 135 degrees. In some embodiments, the rotation ranges from 90 degrees to 135 degrees. In some embodiments, the rotation ranges from 105 degrees to 135 degrees. In some embodiments, the rotation ranges from 120 degrees to 135 degrees. In some embodiments, the rotation ranges from 0 degrees to 120 degrees. In some embodiments, the rotation ranges from 15 degrees to 120 degrees. In some embodiments, the rotation ranges from 30 degrees to 120 degrees. In some embodiments, the rotation ranges from 45 degrees to 120 degrees. In some embodiments, the rotation ranges from 60 degrees to 120 degrees. In some embodiments, the range of rotation is between 75 degrees and 120 degrees. In some embodiments, the range of rotation is between 90 degrees and 120 degrees. In some embodiments, the range of rotation is between 105 degrees and 120 degrees.

[0039] In some embodiments, the range of rotation is from 0 degrees to 105 degrees. In some embodiments, the range of rotation is from 15 degrees to 105 degrees. In some embodiments, the range of rotation is from 30 degrees to 105 degrees. In some embodiments, the range of rotation is from 45 degrees to 105 degrees. In some embodiments, the range of rotation is from 60 degrees to 105 degrees. In some embodiments, the range of rotation is from 75 degrees to 105 degrees. In some embodiments, the range of rotation is from 90 degrees to 105 degrees. In some embodiments, the range of rotation is from 0 degrees to 90 degrees. In some embodiments, the range of rotation is from 15 degrees to 90 degrees. In some embodiments, the range of rotation is from 30 degrees to 90 degrees. In some embodiments, the range of rotation is from 45 degrees to 90 degrees. In some embodiments, the range of rotation is from 60 degrees to 90 degrees. In some embodiments, the range of rotation is from 75 degrees to 90 degrees.

[0040] In some embodiments, the range of rotation is from 0 degrees to 75 degrees. In some embodiments, the range of rotation is from 15 degrees to 75 degrees. In some embodiments, the range of rotation is from 30 degrees to 75 degrees. In some embodiments, the range of rotation is from 45 degrees to 75 degrees. In some embodiments, the range of rotation is from 60 degrees to 75 degrees. In some embodiments, the range of rotation is from 0 degrees to 60 degrees. In some embodiments, the range of rotation is from 15 degrees to 60 degrees. In some embodiments, the range of rotation is from 30 degrees to 60 degrees. In some embodiments, the range of rotation is from 45 degrees to 60 degrees. In some embodiments, the range of rotation is from 0 degrees to 45 degrees. In some embodiments, the range of rotation is from 15 degrees to 45 degrees. In some embodiments, the range of rotation is from 30 degrees to 45 degrees. In some embodiments, the range of rotation is from 0 degrees to 30 degrees. In some embodiments, the range of rotation is from 15 degrees to 30 degrees. In some embodiments, the range of rotation is from 0 degrees to 15 degrees.

[0041] In step 220, the computing device 110 applies a reconstruction process to the series of fluoroscopic images to generate tomographic images (e.g., unenhanced tomographic images) (e.g., three-dimensional images). In some embodiments, the reconstruction is performed based at least in part on the known pose of each image in the series of fluoroscopic images. In some embodiments, the pose of each image is determined using image-based pose estimation (e.g., based on recognition of objects depicted in each image, such as anatomical features or radiopaque markers). In some embodiments, the image-based pose estimation is performed as described in U.S. Pat. No. 10,674,970, the contents of which are incorporated herein by reference in their entirety. In some embodiments, the reconstruction process includes filtered back projection (FBP), algebraic reconstruction technique (ART), simultaneous algebraic reconstruction technique (SART), or simultaneous iterative reconstruction technique (SIRT). FIG. 3B shows coronal, axial, and sagittal slices of an exemplary tomographic image.

[0042] In some embodiments, the tomographic images generated in step 220 are similar in type to the “gold standard” reference CT image acquired for the same patient represented by the image received in step 210, but are of lower quality. FIG. 3C illustrates coronal, axial, and sagittal slices of an exemplary reference CT image. For example, in some embodiments, the tomographic images generated in step 220 have lower resolution, lower fidelity, or are otherwise of lower quality than the reference CT image. In some embodiments, the tomographic images generated in step 220 are not of sufficient quality to enable a clinician (e.g., a radiologist) to distinguish lesions or other objects less than 30 millimeters in size. In some embodiments, the tomographic images generated in step 220 are not of sufficient quality to enable a clinician (e.g., a radiologist) to distinguish lesions or other objects less than 10 millimeters in size. In some embodiments, the tomographic images generated in step 220 include one or more artifacts.

[0043] In step 230, computing device 110 applies the trained tomographic image enhancement machine learning model to the tomographic image generated in step 220, thereby generating an enhanced tomographic image. In some embodiments, the trained tomographic image enhancement machine learning model is trained as described in further detail below in connection with exemplary method 400. In some embodiments, the enhanced tomographic image generated in step 230 is of a similar type and quality to the “gold standard” reference CT image acquired for the same patient represented by the image received in step 210. For example, in some embodiments, the enhanced tomographic image generated in step 230 contains fewer artifacts than the simulated tomographic image generated in step 220. An exemplary enhanced tomographic image is shown in FIG. 3D.

[0044] In step 240, computing device 110 outputs a representation of the enhanced tomographic image. In some embodiments, the representation is a two-dimensional slice of the enhanced tomographic image, such as an axial slice. In some embodiments, the output is to a display (e.g., a display communicatively coupled to computing device 110). In some embodiments, the output is to an additional software program (e.g., a program that generates the enhanced image, a surgical planning program, etc.). As can be seen in FIGS. 3C and 3D , in some embodiments, the enhanced tomographic image generated in step 230 is of sufficient quality to enable a clinician (e.g., a radiologist) to distinguish lesions or other objects less than 30 millimeters in size. In some embodiments, the enhanced tomographic image generated in step 230 is of sufficient quality to enable a clinician (e.g., a radiologist) to distinguish lesions or other objects less than 10 millimeters in size.

[0045] FIG. 4 illustrates an implementation of a method according to one or more exemplary embodiments of the present disclosure. In some embodiments, the method 400 illustrated in FIG. 4 is a method for training a trained tomographic image-enhanced machine learning model for use in generating CT-like images based on medical images acquired using a conventional C-arm. The exemplary method 400 is described below with reference to elements of the exemplary system 100 described above with reference to FIG. 1, although it will be apparent to those skilled in the art that other suitable configurations of the system are possible. For example, in the following description, the method 400 is described with reference to training a machine learning model on the computing device 110 of the exemplary medical imaging system 100. However, in other embodiments, the exemplary training method 400 illustrated in FIG. 4 is executed in a separate computing environment and then provided to the medical imaging system 100 to perform the exemplary method 200 described above.

[0046] In step 410, the computing device 110 receives CT image data including a CT image for each patient of a plurality of patients. In some embodiments, the CT image data is used as a ground truth tomographic image for each patient. Figure 5A shows coronal, axial, and sagittal slices of an exemplary CT image of a patient.

[0047] In step 420, the computing device 110 is provided with a series of fluoroscopic images for each patient. In some embodiments, the computing device 110 receives a series of actual fluoroscopic images for each patient among the multiple patients for whom the CT image data was received in step 410. In some embodiments, the computing device 110 generates a series of simulated fluoroscopic images for the CT images for each patient. In some embodiments, each simulated fluoroscopic image is generated by projecting the CT image into a desired simulated fluoroscopic image pose. In some embodiments, each simulated fluoroscopic image is generated using a machine learning model, such as the DeepDRR machine learning model. In some embodiments, each simulated fluoroscopic image in a particular series of simulated fluoroscopic images for a particular patient corresponds to a particular pose of the C-arm device at a particular angle, such that each simulated fluoroscopic image simulates a fluoroscopic image acquired using a fluoroscopic imaging device attached to the C-arm device with the C-arm device positioned at the particular angle. In some embodiments, the plurality of simulated fluoroscopic images for each particular patient corresponds to a plurality of angles across a rotational range, such that the series of simulated fluoroscopic images for each particular patient simulates a series of fluoroscopic images acquired during a C-arm imaging examination of the particular patient while the C-arm device is moved across a rotational range. In some embodiments, each particular rotational range simulated in step 420 is one of the rotational ranges described above with reference to step 210 of method 200. FIG. 5B illustrates an exemplary simulated fluoroscopic image generated based on the exemplary CT image shown in FIG. 5A. Similar to FIG. 3A described above, FIG. 5B illustrates six exemplary simulated fluoroscopic images, and in some embodiments, between 80 and 1800 fluoroscopic images are provided in step 420.

[0048] In step 430, computing device 110 reconstructs a simulated tomographic image for each of the plurality of patients based on each series of actual or simulated fluoroscopic images provided in step 420. In some embodiments, each simulated tomographic image is generated using one of the techniques described above with reference to step 220 of method 200. Similar to the description above with reference to step 220 of method 200, in some embodiments, the simulated tomographic images generated in step 430 are generally equivalent to, but lesser in quality than, the ground truth CT images on which they are based. For example, in some embodiments, the simulated tomographic images generated in step 430 have lower resolution, lower fidelity, or are otherwise of lower quality than the corresponding ground truth CT images. In some embodiments, the simulated tomographic images generated in step 430 are not of sufficient quality to enable a clinician (e.g., a radiologist) to distinguish lesions or other objects less than 10 millimeters in size. In some embodiments, the simulated tomographic images generated in step 430 include one or more artifacts. FIG. 5C shows an exemplary simulated tomographic image generated based on the exemplary simulated fluoroscopic image shown in FIG. 5B.

[0049] As described herein, exemplary embodiments relate to training a tomographic image enhancement machine learning model to create a trained tomographic image enhancement machine learning model. In some embodiments, the tomographic image enhancement machine learning model is a gradient descent model using a suitable loss function, such as projected gradient descent, fast gradient sign, stochastic gradient descent, batch gradient descent, mini-batch gradient descent, or other suitable gradient descent technique. In some embodiments, the tomographic image enhancement machine learning model includes a regression model. In some embodiments, the tomographic image enhancement machine learning model includes a neural network. At step 440, computing device 110 performs an enhancement process using the tomographic image enhancement machine learning model to improve the quality of at least some of the simulated tomographic images generated at step 430, thereby obtaining corresponding enhanced simulated tomographic images for each of a plurality of patients. In some embodiments, the enhancement process is performed on a randomly selected subset of the simulated tomographic images. In some embodiments, each of the enhanced simulated tomographic images generated at step 440 may include one or more artifacts. In some embodiments, the artifacts may result from factors such as reconstruction being performed using simulated fluoroscopic images with some missing data.

[0050] In step 450, a score is assigned to each of the enhanced simulated tomographic images generated in step 440. In some embodiments, the score is assigned based on each enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on a comparison of each enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on one or more artifacts in each enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the score is assigned based on an automated (e.g., algorithmic) comparison performed by computing device 110. In some embodiments, the score is assigned by a user. In some embodiments, a performance score of the tomographic image enhancement machine learning model is calculated based on the score of each enhanced simulated tomographic image.

[0051] In step 460, the computing device 110 determines whether the performance score of the tomographic image enhancement machine learning model exceeds a predetermined performance score threshold.

[0052] If the performance score of the tomographic image enhancement machine learning model does not exceed the predetermined performance score threshold, method 400 proceeds to step 480. In step 480, the parameters (e.g., weights) of the tomographic image enhancement machine learning model are updated based on the performance of the gradient descent or other tomographic image enhancement machine learning model, for example, using backpropagation. After step 480, method 400 returns to step 440, where the enhancement process of step 440 is repeated.

[0053] If the performance score of the tomographic image enhancement machine learning model exceeds the predetermined performance score threshold, method 400 is complete, and the output of the method is a trained tomographic image enhancement machine learning model at step 470. In other words, in some embodiments, the enhancement process is repeated iteratively (e.g., by repeating steps 440, 450, 460, and 480) until the performance score of the tomographic image enhancement machine learning model exceeds the predetermined performance score threshold, thereby creating a trained tomographic image enhancement machine learning model. Figure 5D shows an exemplary enhanced simulated tomographic image generated based on the exemplary simulated tomographic image shown in Figure 5C after completion of the training process described above.

[0054] In some embodiments, the exemplary tomographic image enhancement machine learning model may be trained until the loss function reaches an acceptable value / threshold (e.g., 0.99 (1%), 0.98 (2%), 0.97 (3%), 0.96 (4%), 0.95 (5%), ..., 0.90 (10%), ..., 0.85 (15%), etc.). In some embodiments, the loss function may measure the error between the enhanced simulated tomographic image and the corresponding ground truth tomographic image. In some embodiments, the error may be calculated as an L2-norm distance and / or an L1-norm distance.

[0055] In some embodiments, CT-like images generated according to the exemplary techniques described above are comparable to "gold standard" CT images acquired using a CT scanning device. In some embodiments, CT-like images generated according to the exemplary techniques described above have comparable quality to "gold standard" CT images acquired using a CT scanning device and are generated using source data originating from a standard C-arm mounted fluoroscopic imaging device. Thus, CT-like images may be generated without requiring access to a CT scanning device, which is expensive and typically in high demand where available. In some embodiments, CT-like images generated as described above can be used to identify lesions or other objects, e.g., less than 10 millimeters in size, thereby enabling early diagnosis of conditions such as lung cancer.

[0056] In some examples, the discernibility of objects shown in an exemplary CT-like image is defined based on the contrast-to-noise ratio ("CNR"). Contrast, as used herein, refers to the brightness difference between an object (as shown in an image or display) and its surroundings, which brightness difference makes the object distinguishable. CNR, as used herein, is calculated using the following formula: CNR=(mean(ObjectMask)-mean(BackgroundMaskPortion)) / STD(BackgroundMaskPortion) It can be calculated according to:

[0057] In this formula, mean(ObjectMask) refers to the mean value of the luminance in the region defined as the lesion, mean(BackgroundMaskPortion) refers to the mean value of the luminance in the region defined as the background, and STD(BackgroundMaskPortion) refers to the standard deviation of the luminance values ​​in the region defined as the background. Regions as used herein are identified as described below.

[0058] The lesion region, which may be referred to as an InputObjectMask, is identified by labeling the lesion in the exemplary CT-like image in a manner deemed suitable by one skilled in the art. Figure 6A shows an exemplary image 600 in which the InputObjectMask obtained using user input as described above is depicted by a ring 610. The lesion region, which may also be referred to as an ObjectMask, as used herein, is determined based on the InputObjectMask using the following formula: ObjectMask=erosion(InputObjectMask,2) is defined by

[0059] In this formula, erosion(mask, N) means to erode the mask by N pixels. In other words, the ObjectMask is generated by eroding (e.g., reducing its size) the InputObjectMask by 2 pixels. Referring again to FIG. 6A, the ObjectMask is depicted by a ring 620 that is smaller than ring 610. For the values ​​mentioned above, the background region is calculated by the formula: BackgroundMask=dilation(ObjectMask,10)-dilation(ObjectMask,5) It is defined by dilating (e.g. increasing the size of) the object mask by

[0060] In this formula, dilation(mask, N) means to dilate (e.g., increase the size of) the mask by N pixels. Referring again to Figure 6A, BackgroundMask is depicted by the area between inner ring 630 and outer ring 640, which is larger than ring 610.

[0061] As used herein, BackgroundMaskPortion refers to a 180-degree arc of the background region BackgroundMask selected to maximize the calculated value CNR. FIG. 6B illustrates the creation of one possible 180-degree arc based on the image 600 shown in FIG. 6A. In FIG. 6B, arc 650 is defined by a vector V 660 extending from the center of gravity C 670 of the ObjectMask depicted by 620, and vector V 660 extends in a direction defined by an angle α 680 measured from a reference direction 690. While reference direction 690, as used herein, is horizontal as viewed in image 600, this is merely an arbitrary reference point, and a series of potential 180-degree arcs can be defined based on any given reference point. According to the testing method described herein, all possible 180-degree arcs are computationally evaluated. FIG. 6C illustrates an exemplary series of images illustrating different 180-degree arcs 650 corresponding to image 600. While six such images are shown in FIG. 6C as a representative sampling, the actual number of possible arcs evaluated will be much larger (e.g., the number of such arcs will vary depending on the resolution of the images evaluated).

[0062] As explained above, the contrast-to-noise ratio (CNR) is calculated by calculating the lesion ObjectMask (e.g., the ObjectMask depicted by the ring 620 shown in FIG. 6B) using the formula: CNR=(mean(ObjectMask)-mean(BackgroundMaskPortion)) / STD(BackgroundMaskPortion) The BackgroundMaskPortion is calculated based on the selected arc (e.g., the BackgroundMaskPortion depicted by arc 650 shown in FIG. 6B) using the

[0063] In this formula, mean(ObjectMask) refers to the mean value of the intensity within the region defined as the lesion, mean(BackgroundMaskPortion) refers to the mean value of the intensity within the selected arc, and STD(BackgroundMaskPortion) refers to the standard deviation of the intensity within the selected arc. As used herein, an object is identifiable in an image if its CNR (determined based on the arc resulting in the maximum CNR) is greater than 5.

[0064] Figures 7A-10C show representative images demonstrating the applicability of a threshold CNR value of 5 for assessing the identifiability of lesions within images. Figures 7A, 7B, and 7C show images 710, 720, and 730 with CNR values ​​of 12.7, 7.0, and 3.9, respectively. It can be seen that images 710 and 720 are of sufficient quality to make lesion 740 distinguishable, while image 730 is not. Figures 8A, 8B, and 8C show images 810, 820, and 830 with CNR values ​​of 9.9, 4.4, and 2.6, respectively. It can be seen that image 810 is of sufficient quality to make lesion 840 distinguishable, while images 820 and 830 are not. Figures 9A, 9B, and 9C show images 910, 920, and 930 with CNR values ​​of 11.7, 6.2, and 3.4, respectively. It can be seen that images 910 and 920 are of sufficient quality to make lesion 940 distinguishable, while image 930 is not. Figures 10A, 10B, and 10C show images 1010, 1020, and 1030 with CNR values ​​of 7.3, 4.5, and 3.0, respectively. It can be seen that image 1010 is of sufficient quality to make lesion 1040 distinguishable, while images 1020 and 1030 are not.

[0065] In some embodiments, the exemplary techniques described above are applied to generate a CT-like image of at least one organ (e.g., lung, kidney, liver, etc.) of a patient. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least one lesion in at least one organ (e.g., lung, kidney, liver, etc.) of a patient. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least one lesion in at least one organ (e.g., lung, kidney, liver, etc.) of a patient to enable diagnosis and / or treatment of the at least one lesion. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image showing at least a portion of a patient's spine. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image used to perform an image-guided biopsy procedure. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image used to administer an analgesic. In some embodiments, the exemplary techniques described above are applied to generate a CT-like image used to administer a therapy (e.g., a drug therapy or an ablation therapy) to a lesion shown in the CT-like image.

[0066] In some examples, the exemplary technique is capable of generating CT-like images capable of distinguishing lesions less than 30 millimeters in size from their surroundings based on tests such as those described above. In some examples, the exemplary technique is capable of generating CT-like images capable of distinguishing lesions less than 10 millimeters in size from their surroundings based on tests such as those described above. In some examples, the exemplary technique is capable of generating CT-like images capable of distinguishing lesions with densities less than -300 Hounsfield Units (HU) from their surroundings based on tests such as those described above. In some examples, the exemplary technique is capable of generating CT-like images without the involvement of a licensed radiologist required to operate a CT scanner. In some examples, the exemplary technique is capable of generating CT-like images without exposing patients to the high radiation doses delivered by a CT scanner.

[0067] While several embodiments of the present invention have been described, it will be understood that these embodiments are for illustrative purposes only and are not limiting, and that many variations will be apparent to those skilled in the art. For example, all dimensions set forth herein are provided by way of example only and are intended to be illustrative and not limiting.

Claims

1. a) receiving, by a controller unit, from a C-arm device, a plurality of fluoroscopic images of at least a portion of a patient's lungs; each of the plurality of fluoroscopic images is acquired with the C-arm device positioned in a particular position among a plurality of positions assumed by the C-arm device while the C-arm device is moved through a range of motion; the range of motion includes at least a rotational range; the rotation range encompassing a sweep angle between 45 degrees and 120 degrees; b) said controller unit controls at least A trained machine learning model, and the plurality of perspective images generating an enhanced tomographic image of the at least a portion of the lung by utilizing c) outputting, by said controller unit, a representation of said enhanced tomographic image, (a) the at least a portion of the lung of the patient includes at least one lesion less than 30 millimeters in size; (b) the representation of the enhanced tomographic image is an axial slice showing a clear boundary of the at least one lesion. When tested by the test method the at least one lesion having a contrast-to-noise value of at least 5 compared to a background of the representation; A method comprising:

2. generating the enhanced tomographic image, reconstructing a tomographic image based on the plurality of perspective images; and enhancing the tomographic image using the trained machine learning model to generate the enhanced tomographic image. The method of claim 1 , comprising:

3. The method of claim 2 , wherein the step of reconstructing the tomographic image based on the plurality of perspective images comprises reconstructing the tomographic image using filtered back projection.

4. The method of claim 2 , wherein the step of reconstructing the tomographic image based on the plurality of perspective images includes the step of determining a pose of each of the plurality of perspective images.

5. The method of claim 4 , wherein determining the pose of each of the plurality of perspective images comprises image-based pose estimation.

6. The method of claim 5 , wherein the image-based pose estimation includes recognition of at least one of anatomical features or radiopaque markers.

7. The method of claim 1 , wherein the representation of the enhanced tomographic image comprises an axial slice.

8. The method of claim 1 , wherein the sweep angle is between 45 and 90 degrees.

9. (a) the at least a portion of the lung of the patient includes at least one lesion less than 10 millimeters in size; (b) the representation of the enhanced tomographic image is an axial slice showing a clear boundary of the at least one lesion.

2. The method of claim 1, wherein the at least one lesion has a contrast-to-noise value of at least 5 compared to a background of the representation when tested by a testing method:

10. The method of claim 1 , wherein the trained machine learning model comprises a gradient descent machine learning model.

11. The method of claim 1 , wherein the range of motion further comprises a translational range of motion.

12. acquiring, by the controller unit, a plurality of fluoroscopic images of a region of interest of the patient's tissue; each of the plurality of fluoroscopic images is acquired by the C-arm device while the C-arm device is positioned in a particular position among a plurality of positions assumed by the C-arm device while the C-arm device is moved through a rotational range; the range of rotation encompasses rotation of less than 180 degrees; reconstructing, by the controller unit, a tomographic image including the region of interest using the plurality of perspective images; enhancing, by the controller unit, the tomographic image using a trained tomographic image enhancement machine learning model to generate an enhanced tomographic image; the trained tomographic image enhancement machine learning model: receiving CT image data of a plurality of patients; the CT image data includes ground truth tomographic images for each of the plurality of patients; receiving a plurality of fluoroscopic images for each of the plurality of patients; generating a plurality of simulated fluoroscopic images based on the CT image data for each of the plurality of patients, each fluoroscopic image of the plurality of simulated fluoroscopic images corresponds to a particular posture of a C-arm device at a particular angle; the plurality of perspective images corresponding to a plurality of angles spanning a rotational range between 45 degrees and 120 degrees; reconstructing a simulation tomographic image for each of the plurality of patients based on the plurality of fluoroscopic images for each of the plurality of patients, the simulation tomographic image including a plurality of artifacts; performing an enhancement process utilizing a tomographic image enhancement machine learning model to enhance the simulated tomographic image for each of the plurality of patients to reduce the plurality of artifacts and obtain an enhanced simulated tomographic image for each of the plurality of patients; scoring each enhanced simulated tomographic image based on the plurality of artifacts and corresponding ground truth tomographic images to obtain a corresponding performance score of the tomographic image enhancement machine learning model; updating parameters of the tomographic image machine learning model while the performance score of the tomographic image enhancement machine learning model is below a predetermined performance score threshold; and repeating the enhancement process iteratively until the corresponding performance score is greater than or equal to the predetermined performance score threshold to obtain the trained tomographic image enhancement machine learning model. The training process includes the steps A method comprising:

13. The method of claim 12 , wherein the plurality of fluoroscopic images for each of the plurality of patients comprises a plurality of actual fluoroscopic images for at least some of the plurality of patients.

14. The method of claim 12 , wherein the plurality of fluoroscopic images for each of the plurality of patients comprises a plurality of simulated fluoroscopic images for at least some of the plurality of patients.

15. The method of claim 14 , wherein the plurality of simulated fluoroscopic images are generated by projecting at least one tomographic image into a plurality of poses.

16. The method of claim 12 , wherein the step of reconstructing the tomographic image comprises reconstructing the tomographic image using filtered backprojection.

17. The method of claim 12 , wherein the range of rotation includes rotation between 45 degrees and 120 degrees.

Images