Imaging reconstruction systems and methods

Abstract

3D imaging systems and methods at least minimize image artifacts caused by radiopaque fiducials. The methods include performing a 3D Fourier transform on 2D images of a 3D medical scan, filtering the Fourier images with a background filter, yielding a background image, segmenting the Fourier images based on the background image, modifying the Fourier images based on the segmented images, and reconstructing the volume based on the modified Fourier images. The systems may include a motorized location board which, after a tomographic scan, causes fiducials to travel from outside a field of view to inside the field of view. Alternatively, a fiducial board may be inserted into a receptacle of a location board after the tomographic scan. Thereafter, a non-tomographic scan is performed. The non-tomographic scan and fiducials are used to locate features in the non-tomographic scan and display information on the non-tomographic scan based on the located features.

Description

Attorney Docket No. A0009847WD00 (00017-01221PCT00)IMAGING RECONSTRUCTION SYSTEMS AND METHODSFIELD

[0001] The technology of this disclosure generally relates to systems and methods for addressing artifacts in radiographic images.BACKGROUND

[0002] There are several commonly applied methods for treating various maladies affecting tissues and organs including the liver, brain, heart, lung, and kidney. Often, one or more imaging modalities, such as magnetic resonance imaging (MRI), ultrasound imaging, computed tomography (CT), cone-beam computed tomography (CBCT), fluoroscopic imaging as well as others are employed by clinicians to identify areas of interest within a patient and ultimately targets for treatment.

[0003] An endoscopic approach has proven useful in navigating to areas of interest within a patient, and particularly so for areas within luminal networks of the body such as the lungs. To enable the endoscopic, and more particularly the bronchoscopic, approach in the lungs, endobronchial navigation systems have been developed that use previously acquired MRI data or CT image data to generate a three-dimensional (3D) rendering or volume of the particular body part such as the lungs. In particular, previously acquired images, acquired from an MRI scan or CT scan of the patient, are utilized to generate a 3D or volumetric rendering of the patient.

[0004] The resulting volume generated from the MRI scan or CT scan is then utilized to create a navigation plan to facilitate the advancement of a navigation catheter (or other suitable tool) through a bronchoscope and a branch of the bronchus of a patient to an area of interest. Electromagnetic tracking may be utilized in conjunction with the CT data to facilitate guidance of the navigation catheter through the branch of the bronchus to the area of interest. In certain instances, the navigation catheter may be positioned within one of the airways of the branched luminal networks adjacent to, or within, the area of interest to provide access for one or more medical instruments.

[0005] Thus, to generate a navigation plan, or to generate a three-dimensional (3D) or volumetric rendering of the patient’s anatomy, such as the lung, a clinician may utilize an MRI system or CT system to acquire the necessary image data for reconstruction of the volume. An MRI system or CT-based imaging system is extremely costly, and in many cases not availableAttorney Docket No. A0009847WG00 (00017-01221PCT00) in the same location as the location where a navigation plan is generated or where a navigation procedure is conducted.

[0006] A fluoroscopic imaging system is commonly located in the operating room during navigation procedures. A clinician may use the fluoroscopic imaging system to visualize and confirm the placement of a tool after the tool has been navigated to a desired location. However, although standard fluoroscopic images display highly dense objects such as metal tools and bones as well as large soft-tissue objects such as the heart, the fluoroscopic images may have difficulty resolving small soft-tissue objects of interest such as lesions. Further, the fluoroscopic images are only two-dimensional projections. To help see small soft-tissue objects in 3D space, an X-ray volumetric reconstruction is needed.

[0007] X-ray volumetric reconstruction may be achieved by back projecting fluoroscopic images from multiple angles. However, metal treatment and monitoring systems, such as bronchoscopes, catheters, electrocardiograph (ECG) components, patient sensor triplets (PSTs), and metal spheres on angle measurement jigs may produce strong artifacts on the order of several hundred Hounsfield units (HUs) in captured fluoroscopic images. Artifacts in the captured fluoroscopic images may appear as streaks and broad bright or dark bands, severely degrading image quality and drastically reducing the diagnostic value of images. Additionally, the artifacts may obscure important anatomical information such as lesions, major blood vessels, airways, or pleura borders. A CBCT scanner may be used; however, the location board may include metal beads, which may also produce strong artifacts in CBCT images leading to reduced image quality.

[0008] Algorithms developed to reduce artifacts can be classified into project! on- interpolation -based methods, iterative reconstruction methods, or their combination. Projection interpolation methods treat parts of the projections affected by metal (the so-called metal shadow) as unreliable. Metal shadow data is complemented by interpolation between neighboring reliable data. Iterative reconstruction methods model the main causes for metal artifacts, such as noise and beam hardening. Although the image quality obtained from these methods is often better than that obtained through use of project! on-interpolation -based methods, a drawback is the high computational complexity. In particular, iterative methods have trouble dealing with data for a metal so dense that it stops almost all beams passing through it.

[0009] Some algorithms combining projection completion and iterative reconstruction have been proposed. These methods create a model image using classification prior information and then forward-project the model image to fill the gaps of the metal shadow. The modelAttorney Docket No. A0009847WD00 (00017-01221PCT00) image classifies the pixels into several kinds of tissue and diminishes the density contrast of soft tissues. The region close to the metal is often not well corrected and some residual shadow artifacts may remain.

[0010] In recent years, many algorithms have been developed for 3D reconstruction and motion estimation, which can be divided into several categories, namely: methods using bundle adjustment (BA), or methods based on factorization and hierarchical methods. In the first group, multi-view structure from motion started from estimating the geometry of two views. The structure is used to estimate the pose of the adjacent camera, and the quality of the reconstruction is strongly dependent on the initial structure of first camera pairs. Additional disadvantages of methods using bundle adjustment include the drift problem, expensive computation cost, and accumulated errors when increasing the image number. In the second group of methods based on factorization and hierarchical methods, missing data and sensitiveness to outliers is a significant drawback. In the third group, input images must be arranged in the hierarchical tree processed from root to the top.SUMMARY

[0011] The techniques of this disclosure generally relate to systems and methods for addressing artifacts in radiographic images.

[0012] In one aspect, the disclosure provides a method of correcting for artifacts. The method includes receiving two-dimensional (2D) radiographic images of a patient and performing a Fourier transform on the 2D radiographic images, yielding Fourier images. The method may also include filtering the Fourier images with at least one background filter, yielding at least one background image, and segmenting the Fourier images based on the at least one background image, yielding segmented images. The method may also include modifying the Fourier images based on the segmented images, yielding modified Fourier images, and reconstructing a volume based on the modified Fourier images.

[0013] In aspects, implementations of the method of correcting for artifacts may include one or more of the following features.

[0014] In aspects, the 2D radiographic images may be captured by a tomographic imaging system or a fluoroscopic imaging system.

[0015] In aspects, filtering the Fourier images may include filtering the Fourier images with at least three background filters.

[0016] In aspects, the artifacts may be caused by a pattern of markers or a periodic grid of markers.Attorney Docket No. A0009847WD00 (00017-01221PCT00)

[0017] In aspects, the method may include removing pixels at the center of the segmented images, at a margin of the segmented images, or at a center and a margin of the segmented images.

[0018] In aspects, the method may include retaining only connected components with eight to twelve pixels in the segmented images. The method may include dilating the connected components by two pixels. The method may include generating D images according to the expression D = I > I * F, where I is an original Fourier image, * is the convolution operator, and F is a background filter, and applying the D images to the segmented images.

[0019] In another aspect, the disclosure provides a system for generating a medical image volume of a patient. The system includes a pattern of markers, a processor, and a memory. The memory has stored thereon instructions, which, when executed by the processor, cause the processor to receive two-dimensional (2D) images of a patient and the pattern of markers, identify the pattern of markers in the 2D images, yielding an identified pattern of markers, and perform a Fourier transform on the 2D images, yielding Fourier images. The instructions, when executed by the processor, may also cause the processor to filter the Fourier images with at least one background filter, yielding at least one background image, segment the Fourier images based on the at least one background image, yielding segmented images, and modify the Fourier images based on the segmented images, yielding modified Fourier images. The instructions, when executed by the processor, may also cause the processor to reconstruct a medical image volume based on the modified Fourier images and the identified pattern of markers.

[0020] In aspects, implementations of the system for generating a medical image volume of a patient may include one or more of the following features.

[0021] In aspects, the instructions, when executed by the processor, may cause the processor to remove pixels at a center of the segmented images, at a margin of the segmented images, or at a center and a margin of the segmented images. The pattern of markers may be positioned outside of a region of interest.

[0022] In aspects, the instructions, when executed by the processor, may cause the processor to generate D images according to the expression D = I > I * F, where I is an original Fourier image, * is the convolution operator, and F is a background filter, and apply the D images to the segmented images.

[0023] In another aspect, the disclosure provides a system, which includes a fiducial board, a processor, and a memory. The fiducial board includes moveable fiducials, and at least one motor operably coupled to the moveable fiducials. The memory has stored thereon instructions,Attorney Docket No. A0009847WG00 (00017-01221PCT00) which, when executed by the processor, cause the processor to determine that a tomographic scan is to be performed, and in response to determining that the tomographic scan is to be performed: receive first images of the tomographic scan of a patient with the moveable fiducials located in a marginal region of the fiducial board, reconstruct an image volume of the patient based on the first images, and display the image volume. The instructions, when executed by the processor, may also cause the processor to determine that a fluoroscopic scan is to be performed, and in response to determining that the fluoroscopic scan is to be performed: control the at least one motor to move the moveable fiducials to a central region of the fiducial board, receive second images captured by a fluoroscopic imaging system in a fluoroscopic scan of a patient and the moveable fiducials located in the central region of the fiducial board, identify the moveable fiducials in the second images, yielding identified moveable fiducials, and estimate a pose of the fluoroscopic imaging system based on the identified moveable fiducials and the second images, yielding an estimated pose

[0024] In aspects, implementations of the system for generating a medical image volume of a patient may include one or more of the following features.

[0025] In aspects, the instructions, when executed by the processor, may cause the processor to determine a position of a feature in the second images based on the estimated pose, yielding a determined position of the feature, and overlay information on the second images based on the determined position of the feature. The feature may be a target, and the instructions, when executed by the processor, may cause the processor to generate a representation of the target based on the image volume, and augment the second images with the representation of the target.

[0026] In aspects, the marginal regions may be two opposing sides of the fiducial board.

[0027] In aspects, the instructions, when executed by the processor, may cause the processor to determine a subset of the moveable fiducials that have a high probability of generating artifacts in the second images at a region including a target, and control the at least one motor to move the subset of the moveable fiducials out of the region including the target.

[0028] In aspects, the fiducial board may be incorporated into a location board.

[0029] In another aspect, the disclosure provides another system for generating an image volume of a patient. The system includes a fiducial board, an electromagnetic (EM) emitter configured to emit an EM signal, a receptacle configured to receive the fiducial board, a processor, and a memory. The memory has stored thereon instructions, which, when executed by the processor, cause the processor to, in a tomographic imaging mode, receive first two- dimensional (2D) radiographic images of a patient at different angles without the fiducial boardAttorney Docket No. A0009847WD00 (00017-01221PCT00) disposed in the receptacle, and reconstruct an image volume of the patient based on the first 2D radiographic images. The instructions, when executed by the processor, may also cause the processor to, in a non-tomographic imaging mode, receive second 2D radiographic images of a patient at different angles with the first fiducial board disposed in the receptacle, identify fiducials of the fiducial board in the second 2D radiographic images, yielding identified fiducials, and estimate a pose of a non-tomographic imaging system based on the identified fiducials and the second 2D radiographic images.

[0030] In aspects, implementations of the other system for generating an image volume of a patient may include one or more of the following features.

[0031] In aspects, in the tomographic imaging mode, the instructions, when executed by the processor, may cause the processor to reconstruct a second image volume of the patient based on the estimated pose and the second 2D radiographic images.

[0032] In aspects, the receptacle may be a stand-alone receptacle configured to be placed between a patient and an operating table or the receptacle may be incorporated or coupled to the operating table, and the fiducial board may be configured as a drawer having a handle for manually removing or inserting the fiducial board from or to the receptacle.

[0033] In aspects, the EM emitter may be located in a side area adjacent to a patient area or the EM emitter may be coupled to a side portion of an operating table.

[0034] In aspects, the tomographic imaging mode may be a computed tomography (CT) mode, a cone-beam computed tomography (CBCT) mode, a positron emission tomography (PET) mode, or a single photon emission computed tomography (SPECT) mode.

[0035] In aspects, the system may include at least one EM receiver, and the instructions, when executed by the processor, may cause the processor to calculate a position of the at least one EM receiver based on the EM signal received by the at least one EM receiver.

[0036] In another aspect, the disclosure provides another method. The method includes generating, by at least one electromagnetic (EM) transmitter, an EM signal and calculating position information of at least one EM receiver based on the EM signal received by the at least one EM receiver. The method may also include receiving instructions to operate according to a navigation phase and, in response to receiving the instructions to operate according to a navigation phase, capturing tomographic images with a tomographic imaging system, reconstructing an image volume based on the tomographic images, and displaying the position information and the image volume.

[0037] The method may also include determining that the navigation phase is complete and, in response to determining that the navigation phase is complete, moving fiducial markersAttorney Docket No. A0009847WG00 (00017-01221PCT00) into a field of view, capturing fluoroscopic images with a fluoroscopic imaging system, identifying fiducial markers in the fluoroscopic images, yielding identified fiducial markers, determining a position of the fluoroscopic imaging system relative to the EM transmitter based on the identified fiducial markers, determining a position of a feature in the fluoroscopic images based on the determined position of the fluoroscopic imaging system relative to the EM transmitter, displaying the fluoroscopic images, and displaying information on the fluoroscopic images based on the determined position of the feature in the fluoroscopic images.

[0038] In aspects, implementations of the other method may include one or more of the following features.

[0039] In aspects, the tomographic imaging system may be a computed tomography (CT) system, a cone-beam computed tomography (CBCT) system, a positron emission tomography (PET) system, or a single photon emission computed tomography (SPECT) system.

[0040] In aspects, the fluoroscopic imaging system is a C-arm fluoroscope.

[0041] In aspects, the method may include transitioning to a target treatment phase in response to determining that the navigation phase is complete.

[0042] In aspects, the method may include receiving instructions to change from a tomographic imaging mode to a fluoroscopic imaging mode and, in response to receiving the instructions to change from the tomographic imaging mode to the fluoroscopic imaging mode, displaying instructions to insert a fiducial board into a receptacle of a location board.

[0043] In aspects, the method may include detecting a presence of a fiducial board including the fiducial markers and determining that the navigation phase is complete in response to detecting the presence of the fiducial board.

[0044] In aspects, the method may include detecting a presence of a fiducial board including the fiducial markers, in response to detecting the presence of the fiducial board, prompting a user to confirm transition to a target treatment phase, receiving information confirming transition to the target treatment phase, and, in response to receiving the information confirming transition to the target treatment phase, determining that the navigation phase is complete.

[0045] In aspects, the feature may be a target, and the method may include generating a representation of the target based on the image volume and augmenting the second images with the representation of the target.

[0046] In aspects, the method may include, in response to receiving the instructions to operate according to a navigation phase, removing fiducial markers from the field of view.

[0047] In aspects, the fluoroscopic images may be live images.Attorney Docket No. A0009847WD00 (00017-01221PCT00)

[0048] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF DRAWINGS

[0049] FIG. 1 is a block diagram that illustrates a system for acquiring and processing CBCT scans of a patient and a location board;

[0050] FIG. 2 is a circuit block diagram that illustrates the workstation of the system of FIG. 1;

[0051] FIG. 3 is a flowchart that illustrates a method for reconstructing a volume from a CBCT scan of a patient with artifacts;

[0052] FIG. 4A is an image that illustrates an axial slice from a CBCT volume with the location board beads;

[0053] FIG. 4B is an image that illustrates an axial slice from a CBCT volume without the location board beads;

[0054] FIG. 5A is an image that illustrates an axial slice of a 3D Fourier transform of a CBCT scan volume without the location board beads;

[0055] FIG. 5B is an image that illustrates an axial slice of a 3D Fourier transform of a CBCT scan volume with the location board beads;

[0056] FIG. 5C is an image that illustrates an example of a 3D render of the Fourier transform volume based on a CBCT scan volume with the location board beads.

[0057] FIG. 6A is an image that illustrates an axial slice of a 3D Fourier transform of a CBCT scan volume with the location board beads;

[0058] FIG. 6B is an image that illustrates a filtered image resulting from filtering the Fourier image of FIG. 6 A with a single background filter;

[0059] FIG. 7 is an image that illustrates a filtered image R resulting from filtering the Fourier image of FIG. 6 A with multiple background filters;

[0060] FIG. 8 is an image that illustrates a filtered image D resulting from filtering the Fourier image of FIG. 6 A with the multiple background filters;

[0061] FIG. 9 is an image that illustrates a mask to be applied to the segmented image;

[0062] FIG. 10 is an image that illustrates a masked image resulting from applying the mask of FIG. 9 to the segmented image;

[0063] FIG. 11 A is an image that illustrates an axial slice of a volume reconstructed from masked images including the masked image of FIG. 10;Attorney Docket No. A0009847WD00 (00017-01221PCT00)

[0064] FIG. 1 IB is an image that illustrates an axial slice from a CBCT volume without the location board beads;

[0065] FIG. 12 is a flowchart that illustrates a method for reconstructing and processing a volume based on the imaging mode;

[0066] FIG. 13 is a perspective view that illustrates a location board with a receptacle configured to removably receive a fiducial board;

[0067] FIG. 14 is a flowchart that illustrates a method for reconstructing a volume using a motorized fiducial board configured to drive movement of fiducials;

[0068] FIGS. 15A and 15B are perspective views that illustrate a motorized location board configured to drive movement of fiducials; and

[0069] FIG. 16 is a flowchart that illustrates a method for reconstructing a volume using the motorized fiducial board of FIGS. 15A and 15B.DETAILED DESCRIPTION

[0070] When metal beads are present during a tomographic scan, e.g., a three-dimensional (3D) cone beam computed tomography (CBCT) scan, artifacts may appear in the volume. For example, FIG. 4A shows an axial slice from a CBCT volume with the location board beads, and FIG. 4B shows a corresponding axial slice from a CBCT volume without the location board beads. The artifacts may obscure important anatomical information such as lesions, major blood vessels, airways, pleura borders, etc. A 3D Fourier transform reveals a pattern for a significant portion of the interference. The interference may stem from a periodic grid of markers, which may cause a high frequency response at certain frequencies. The pattern appears as lines in axial and sagittal slices, and circles in coronal slices. For example, FIG. 5A shows an axial slice of a 3D Fourier transform of a CBCT scan volume without the location board beads. And FIG. 5B shows an axial slice of a 3D Fourier transform of a CBCT scan volume with the location board beads. Thus, FIGS. 5A and 5B illustrate that a CBCT scan volume including the location board beads introduces interference lines in the axial slices of the 3D Fourier transform. FIG. 5C shows an example of a 3D render of the Fourier transform volume based on a CBCT scan volume including the location board beads. The 3D render makes clear that the interference pattern is significant and must be removed or minimized to improve the accuracy of a volume reconstructed from the CBCT scan volume.

[0071] The disclosure is directed to systems and methods for removing or at least minimizing artifacts, which are caused by the metal beads or other fiducials incorporated into a fiducial board or a location board, from a volume reconstructed from a tomographic scan. The systems and methods involve identifying and filtering out the artifacts.Attorney Docket No. A0009847WD00 (00017-01221PCT00)

[0072] In one aspect, the fiducials may be removed from the CBCT images used to create the 3D reconstruction. The method of removal may include inpainting. Once the fiducials are removed from the CBCT images, a 3D reconstruction can be generated from the new CBCT images, without the fiducials and therefore without the artifacts caused by the fiducials.

[0073] In another aspect, the system includes a user-configurable location board, which enables a user to manually configure the fiducials, e.g., via one or more fiducial boards in the form of drawers configured to slide into a receptacle of the location board. In a CBCT or other tomographic configuration, the location board may have no fiducials or a limited number of fiducials. The limited number of fiducials may be sufficient to determine an orientation of the CBCT scanner in relation to the location board. In a non-CBCT configuration, the location board or the fiducial board may have a full fiducial grid needed for a navigation or treatment procedure using a fluoroscopic imaging system. For the purposes of this disclosure, “tomographic imaging” refers to modalities used for 3D volume reconstruction (e.g., CBCT), while “fluoroscopic imaging” is an example of a 2D projection imaging mode, sometimes referred to as “non-tomographic ” The fiducial board with fiducials may slide on top of or on the bottom of the location board via a dedicated drawer. Accordingly, the location board configuration may be changed without moving the patient.

[0074] In another aspect, the system includes a software-configurable location board with moveable fiducials. For example, in a CBCT configuration, fiducials that generate artifacts on the CBCT images at or near the location of the target, can be automatically moved to a different location. In aspects, at least four fiducials are needed to determine orientations of the CBCT scanner. In aspects, a larger number of fiducials may be used to make the orientation determination more robust. Additionally, an automatic process that includes a first phase during which an AP image is acquired with a full fiducial grid, and a second stage during which the CBCT scanner is spun without any fiducials in the relevant field of view to acquire a series of CBCT images.

[0075] In still another aspect, fiducial artifacts may be removed from the 3D reconstruction using a Fourier transform -based image processing method. The method may include performing a 3D Fourier transform on 2D images of a 3D medical scan, filtering the Fourier images with a background filter, yielding a background image, segmenting the Fourier images based on the background image, modifying the Fourier images based on the segmented images, and reconstructing a volume based on the modified Fourier images.

[0076] Accordingly, the systems and methods of the disclosure facilitate generation of an accurate reconstructed volume of tissue treatment targets within a patient that may aid aAttorney Docket No. A0009847WD00 (00017-01221PCT00) clinician in performing medical treatment on the tissue treatment target. Example medical treatments that may be performed using the reconstructed volume for guidance include biopsy and ablation treatment. During these procedures, a treatment tool, often metallic in nature, is guided through a luminal network or inserted percutaneously into the patient. The treatment tool is then moved to a position proximate to the treatment target. Once the treatment tool is proximate the treatment target, a 3D volume may be developed to either update a previous model of the patient to establish a more precise or more recent location of the treatment target or, alternatively, to develop a reconstructed volume for the first time that may be displayed to aid a clinician in guiding a medical treatment tool to a treatment target. The reconstructed volume is accurate to ensure the medical tool is placed at the treatment target so that the treatment target receives treatment, and no other tissue is damaged. The systems and methods for generating a reconstructed volume of a target removes interference caused by artifacts, such as those from a fiducial board, and creates clear, accurate reconstructed volumes.

[0077] FIG. 1 is a block diagram of an example of a system 100 for facilitating navigation of a medical tool, e.g., a catheter, to a target via airways of the lungs. The system 100 may be further configured to construct radiographic-based volumetric data of the target area from intraprocedural 2D radiographic images, e.g., intraprocedural fluoroscopic and / or CBCT images, to confirm navigation of a navigation catheter 102, e.g., an extended working channel (EWC) or a smart EWC (sEWC), to a desired location near the target area, where a tool may be placed through and extending out of the navigation catheter. In aspects, the imaging system 124 of the system 100 may include one or more of a C-arm fluoroscope, a 3D CBCT imaging system (including but not limited to Medtronic PLC’s 0-arm™), and a fluoroscopic imaging system.

[0078] The system 100 may be further configured to facilitate approach of a medical tool or tool to the target area and to determine the location of the medical tool with respect to the target by using electromagnetic navigation (EMN) of the navigation catheter. One such EMN system is the ILLUMISITE system currently sold by Medtronic PLC, though other systems for intraluminal navigation are considered within the scope of this disclosure.

[0079] One aspect of the system 100 is a software component for reviewing computed tomography (CT) image scan data that has been acquired separately from the system 100. The review of the CT image data allows a user to identify one or more targets, plan a pathway to an identified target (planning phase), navigate the navigation catheter 102 to the target (navigation phase) using a user interface running on a computer system 122, and confirming placement of a distal end portion of the navigation catheter 102 near the target using one orAttorney Docket No. A0009847WG00 (00017-01221PCT00) more electromagnetic (EM) sensors 104b, 126 disposed in or on the navigation catheter 102 at a predetermined position at or near the distal end portion of the navigation catheter 102. The target may be tissue of interest identified by review of the CT image data during the planning phase. Following navigation of the navigation catheter 102 near the target, a medical tool, such as a biopsy tool, an access tool, or a therapy tool, e.g., a flexible microwave ablation catheter, is inserted into and fixed in place with respect to the navigation catheter 102 such that a distal end portion of the medical tool extends a desired distance 107 beyond the distal end of the navigation catheter 102 and the navigation catheter 102 is further navigated using EM navigation to obtain a tissue sample, enable access to a target site, or apply therapy to the target using the medical tool.

[0080] As shown in FIG. 1, the navigation catheter 102 is part of a catheter guide assembly 110. In practice, the navigation catheter 102 is inserted into a bronchoscope 108 for access to a luminal network of the patient P. Specifically, the navigation catheter 102 of catheter guide assembly 110 may be inserted into a working channel of bronchoscope 108 for navigation through a patient’s luminal network. A bronchoscope adapter 109 is coupled to the proximal end portion of the bronchoscope. The bronchoscope adapter 109 may be the EDGE™ Bronchoscope Adapter, which is currently marketed and sold by Medtronic PLC. The bronchoscope adapter 109 is configured either to allow motion of the navigation catheter 102 through the working channel of the bronchoscope 108 (which may be referred to as an unlocked state of the bronchoscope adapter 109) or prevent motion of the navigation catheter 102 through the working channel of the bronchoscope (which may be referred to as a locked state of the bronchoscope adapter 109).

[0081] A locatable guide (LG) 101a, which may be a catheter, and which may include a sensor 104a similar to the sensor 104b, is inserted into the navigation catheter 102 and locked into position such that the LG sensor 104a extends a predetermined distance beyond the distal end portion of the navigation catheter 102. The tools lOla-lOld of the same lengths which include a fixing member 103a-d such that when the fixing member 103a-103d of the tools lOla-lOld engages, e.g., snaps in, with the proximal end portion of the handle 106 of the catheter guide assembly 110, the LG 101a extends a predetermined distance 107 beyond a distal tip or end portion of the navigation catheter 102. The predetermined distance 107 may be based on the length of the navigation catheter 102 and a length between the end portion of the handle 105a-d or the fixing member 103a-103d and the distal end portion of the LG 101a or the other medical tools lOlb-lOld. In aspects, the handles 105a-105d may include control objects, e.g., a button or a lever, for controlling operation of the medical tools lOla-lOld.Attorney Docket No. A0009847WG00 (00017-01221PCT00)

[0082] In some aspects, the position of the fixing member 105a-105d along the length of the medical tools lOla-lOld may be adjustable so that the user can adjust the distance by which the distal end portion of the LG 101a or the medical tools lOlb-lOld extend beyond the distal end portion of the navigation catheter 102. The position and orientation of the LG sensor 104a relative to a reference coordinate system within an electromagnetic field can be derived using an application executed by the computer system 122. In some aspects, the navigation catheter 102 may function as the LG 101a, in which case the LG 101a may not be used. In other aspects, the navigation catheter 102 and the LG 101a may be used together. For example, data from the sensors 104a and 104b may be fused together. Catheter guide assemblies 110 are currently marketed and sold by Medtronic PLC under the brand names SUPERDIMENSION® Procedure Kits, or EDGE™ Procedure Kits, and are contemplated as useable with the disclosure.

[0083] The system 100 generally includes an operating table 112 configured to support a patient P; a bronchoscope 108 configured for insertion through patient P’s mouth into patient P’s airways; monitoring equipment 114 coupled to bronchoscope 108 (e.g., a video display for displaying the video images received from the video imaging system of bronchoscope 108); and a tracking system 115 including a tracking module 116, reference sensors 118, and a location board 120, which may be implemented by a transmitter mat or board. The location board 120 includes one or more EM transmitters for generating an EM field.

[0084] The location board 120 may also include fiducials, which may be embedded or otherwise incorporated into the location board 120, and which are designed and / or arranged to appear in radiographic images for the purpose of creating a 3D reconstruction from the radiographic images. Since the fiducials may be radiographically dense, the fiducials create artifacts on the radiographic images, e.g., intraoperative CBCT images. The system 100 further includes a computer system 122 on which software and / or hardware are used to facilitate identification of a target, planning a pathway to the target, navigating a medical tool to the target, and / or confirmation and / or determination of placement of the navigation catheter 102, or a suitable tool therethrough, relative to the target.

[0085] As noted above, an imaging system 124 capable of acquiring CBCT images or fluoroscopic images of the patient P is also included in the system 100. The images, sequence of images, or video captured by the imaging system 124 may be stored within the imaging system 124 or transmitted to the computer system 122 for storage, processing, and display. Additionally, the imaging system 124 may move relative to the patient P so that images mayAttorney Docket No. A0009847WD00 (00017-01221PCT00) be acquired from different angles or perspectives relative to patient P to create a series of CBCT images.

[0086] The pose of the imaging system 124 relative to patient P and while capturing the images may be estimated via markers incorporated with the location board 120, in the operating table 112, or a pad (not shown) placed between the patient and the operating table 112. The markers are positioned under patient P, between patient P and operating table 112 and between patient P and a radiation source or a sensing unit of the imaging system 124. The markers may have a symmetrical spacing or may have an asymmetrical spacing, a repeating pattern, or no pattern at all. The imaging system 124 may include a single imaging system or more than one imaging system. When a CBCT system is employed, the captured images can be employed to confirm the location of the navigation catheter 102 and / or one of the medical tools lOla-lOld within the patient, update CT-based 3D modeling, or replaced pre-procedural 3D modeling with intraprocedural modeling of the patient’s airways and the position of the navigation catheter 102 within the patient.

[0087] The computer system 122 may be any suitable computer system including a processor and storage medium, such that the processor is capable of executing instructions stored on the storage medium. The computer system 122 may further include a database configured to store patient data, CT data sets including CT images, CBCT images and data sets, fluoroscopic data sets including fluoroscopic images and video, 3D reconstructions, navigation plans, and any other such data. Although not explicitly illustrated, the computer system 122 may include inputs, or may otherwise be configured to receive, CT data sets, CBCT or fluoroscopic images or video, and other suitable imaging data. Additionally, the computer system 122 includes a display configured to display graphical user interfaces. The computer system 122 may be connected to one or more networks through which one or more databases may be accessed by the computer system 122.

[0088] With respect to the navigation phase, a six degrees-of-freedom electromagnetic locating or tracking system 115, or other suitable system for determining position and orientation of a distal portion of the navigation catheter 102 (e.g., Fiber-Bragg flex sensors), is utilized for performing registration of pre-procedure images (e.g., a CT image data set and 3D models derived therefrom) and the pathway for navigation with the patient as they are located on operating table 112.

[0089] In an EMN-type system, the tracking system 115 may include the tracking module 116, the reference sensors 118, and the location board 120 (including the markers). The tracking system 115 is configured for use with a locatable guide, and particularly the LG sensor.Attorney Docket No. A0009847WD00 (00017-01221PCT00)As described above, the medical tools, e.g., the locatable guide 101a with the LG sensor 104a, are configured for insertion through the navigation catheter 102 into patient P’s airways (either with or without the bronchoscope 108) and are selectively lockable relative to one another via a locking mechanism, e.g., the bronchoscope adapter 109. The location board 120 is positioned beneath patient P. The location board 120 generates an electromagnetic field around at least a portion of the patient P within which the position of the LG sensor 104a, the sensor 104b, and the reference sensors 118 can be determined through use of a tracking module 116. An additional electromagnetic sensor 126 may also be incorporated into the end of the navigation catheter 102. The additional electromagnetic sensor 126 may be a five degree-of-freedom sensor or a six degree-of-freedom sensor. One or more of the reference sensors 118 are attached to the chest of the patient P.

[0090] Registration refers to a method of correlating the coordinate systems of the preprocedure images, and particularly a 3D model derived therefrom, with the patient P’s airways as, for example, observed through the bronchoscope 108 and allow for the navigation to be undertaken with accurate knowledge of the location of the LG sensor within the patient and an accurate depiction of that position in the 3D model. Registration may be performed by moving the LG sensor through the airways of the patient P. More specifically, data pertaining to locations of the LG sensor, while the locatable guide is moving through the airways, is recorded using the location board 120, the reference sensors 118, and the tracking system 115. A shape resulting from this location data is compared to an interior geometry of passages of the 3D model generated in the planning phase, and a location correlation between the shape and the 3D model based on the comparison is determined, e.g., utilizing the software on the computer system 122. The software aligns, or registers, an image representing a location of LG sensor with the 3D model and / or two-dimensional images generated from the 3D model, which are based on the recorded location data and an assumption that LG remains located in non-tissue space in patient P’s airways. Alternatively, a manual registration technique may be employed by navigating the bronchoscope 108 with the LG sensor to pre-specified locations in the lungs of the patient P, and manually correlating the images from the bronchoscope 108 to the model data of the 3D model.

[0091] Though described herein with respect to EMN systems using EM sensors, the instant disclosure is not so limited and may be used in conjunction with flexible sensor, shape sensors such as Fiber-Bragg gratings, ultrasonic sensors, or any other suitable sensor that does not emit harmful radiation. Additionally, the methods described herein may be used inAttorney Docket No. A0009847WD00 (00017-01221PCT00) conjunction with robotic systems such that robotic actuators drive the navigation catheter 102 or bronchoscope 108 proximate the target.

[0092] At any point during the navigation process, tools such as a locatable guide 101a, a therapy tool (e.g., a microwave ablation tool 101b or a forceps lOld), a biopsy tool (e.g., a biopsy needle 101c), may be inserted into and fixed in place relative to the navigation catheter 102 to place one of the tools lOla-lOld proximate the target using position information from the navigation catheter 102. The position information from the sensors 104b and / or 126 of the navigation catheter 102 may be used to calculate the position of the distal tip or distal end portion of any of the tools lOla-lOld.

[0093] To ensure the accuracy of the position calculations, the tools lOla-lOld are each designed to extend a predetermined distance from the distal end of the navigation catheter 102 and at least the distal portions of the tools lOla-lOld that extend from the navigation catheter 102 are designed to be rigid or substantially rigid. The predetermined distance may be different depending on one or more design of the tools lOla-lOld, the stiffnesses of the tools 101a- lOld, or how each of the tools lOla-lOld interact with different types of tissue. The tools lOla-lOld may be designed or characterized to set the predetermined distance to ensure deflection is managed (e.g., minimized) so that the virtual tools and environment displayed to a clinician are an accurate representation of the actual clinical tools and environment.

[0094] Calculating the position of the distal end portion of any of the tools lOla-lOld may include distally projecting the position information from the sensors 104b and / or 126 according to tool information. The tool information may include one or more of the shape of the tool, the type of tool, the stiffness of the tool, the type or characteristics of the tissue to be treated by the tool, or the dimensions of the tool.

[0095] With respect to the planning phase, the computer system 122, or a separate the computer system not shown, utilizes previously acquired CT image data for generating and viewing a 3D model or rendering of patient P’s airways, enables the identification of a target (automatically, semi-automatically, or manually), and allows for determining a pathway through patient P’s airways to tissue located at and around the target. More specifically, CT images acquired from CT scans are processed and assembled into a 3D CT volume, which is then utilized to generate a 3D model of patient P’s airways. The 3D model may be displayed on a display associated with the computer system 122, or in any other suitable fashion. Using the computer system 122, various views of the 3D model or enhanced two-dimensional images generated from the 3D model are presented. The enhanced two-dimensional images may possess some 3D capabilities because they are generated from 3D data. The 3D model may beAttorney Docket No. A0009847WD00 (00017-01221PCT00) manipulated to facilitate identification of target on the 3D model or two-dimensional images, and selection of a suitable pathway through patient P’s airways to access tissue located at the target can be made. Once selected, the pathway plan, the 3D model, and the images derived therefrom, can be saved, and exported to a navigation system for use during the navigation phase(s). The ILLUMISITE software suite currently sold by Medtronic PLC includes one such planning software.

[0096] Reference is now made to FIG. 2, which is a schematic diagram of the computer system 122 of FIG. 1 configured for implementing the methods of the disclosure including the methods of FIGS. 3, 12, 14, and 16. The computer system 122 may include a workstation. In some aspects, the computer system 122 may be coupled with the imaging system, directly or indirectly, e.g., by wireless communication. The computer system 122 may include a memory 202, a processor 204, a display 206 and an input device 210. The processor 204 may include one or more hardware processors. The computer system 122 may optionally include an output module 212 and a network interface 208. The memory 202 may store an application 218 and image data 214. The application 218 may include instructions executable by the processor 204 for executing the methods of the disclosure including the methods of FIGS. 3, 12, 14, and 16.

[0097] The application 218 may further include a user interface 216. The image data 214 may include preoperative CT image data, intraoperative fluoroscopic image data, preoperative or intraoperative CBCT image data, and / or 3D reconstruction data. The processor 204 may be coupled with the memory 202, the display 206, the input device 210, the output module 212, the network interface 208, and the imaging system. The computer system 122 may be a stationary computer system, such as a personal computer, or a portable computer system such as a tablet computer. The computer system 122 may embed multiple computers.

[0098] The memory 202 may include any non-transitory computer-readable storage media for storing data and / or software including instructions that are executable by the processor 204 and which control the operation of the computer system 122, process data from one or more EM sensors disposed in or on the navigation catheter, e.g., at a distal end portion of the navigation catheter, to track the position of the navigation catheter and calculate or project the position of a distal end portion of a medical tool at a fixed position within the navigation catheter, and, in some aspects, may also control the operation of the imaging system. The imaging system may be used to capture a series of preoperative CT images of a portion of a patient’s body, e.g., the lungs, as the portion of the patient’s body moves, e.g., as the lungs move during a respiratory cycle. Optionally, the imaging system may include a CBCT imaging system and / or a fluoroscopic imaging system that captures a series of images based on whichAttorney Docket No. A0009847WD00 (00017-01221PCT00) a 3D reconstruction is generated and / or to capture a live 2D view to confirm placement of the navigation catheter and / or the medical tool. In one aspect, the memory 202 may include one or more storage devices such as solid-state storage devices, e.g., flash memory chips. Alternatively, or in addition to the one or more solid-state storage devices, the memory 202 may include one or more mass storage devices connected to the processor 204 through a mass storage controller (not shown) and a communications bus (not shown).

[0099] Although the description of computer-readable media contained herein refers to solid-state storage, it should be appreciated by those skilled in the art that computer-readable storage media can be any available media that can be accessed by the processor 204. That is, computer readable storage media may include non-transitory, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media may include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, Blu-Ray or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information, and which may be accessed by computer system 122.

[0100] The application 218 may, when executed by the processor 204, cause the display 206 to present the user interface 216. The user interface 216 may be configured to present to the user a single screen including a three-dimensional (3D) view of a 3D model of a target from the perspective of a tip of a medical tool, a live two-dimensional (2D) view showing the medical tool, and a target mark, which corresponds to the 3D model of the target, overlaid on the live 2D view. The user interface 216 may be further configured to display the target mark in different colors depending on whether the medical tool tip is aligned with the target in three dimensions.

[0101] The network interface 208 may be configured to connect to a network such as a local area network (LAN) consisting of a wired network and / or a wireless network, a wide area network (WAN), a wireless mobile network, a Bluetooth network, and / or the Internet. The network interface 208 may be used to connect between the computer system 122 and the imaging system 515. The network interface 208 may be also used to receive the image data 214. The input device 210 may be any device by which a user may interact with the computer system 122, such as, for example, a mouse, keyboard, foot pedal, touch screen, and / or voice interface. The output module 212 may include any connectivity port or bus, such as, for example, parallel ports, serial ports, universal serial busses (USB), or any other similarAttorney Docket No. A0009847WD00 (00017-01221PCT00) connectivity port known to those skilled in the art. From the foregoing and with reference to the various figures, those skilled in the art will appreciate that certain modifications can be made to the disclosure without departing from the scope of the disclosure.

[0102] Three-dimensional (3D) CBCT reconstruction is the process of generating a 3D representation of a patient’s anatomy from a series of 2D X-ray images acquired through CBCT scanning. The reconstruction algorithm takes the 2D X-ray images, which are captured from different angles around the patient, and combines them to create a comprehensive reconstructed volume. The different angles may be determined based on a pattern of fiducials, which may be incorporated into a fiducial or location board, appearing in the 2D X-ray images. In general, the 3D reconstruction process involves several stages. First, the acquired 2D X-ray images are preprocessed to correct for any artifacts, distortions, or noise that may have occurred during the acquisition process including artifacts resulting from the fiducials. This stage ensures that the images are of optimal quality for reconstruction.

[0103] Next, the aligned 2D images are used to estimate the attenuation coefficients of the tissues within the patient's anatomy. Attenuation coefficients represent how X-rays are absorbed and scattered within different tissues. The 3D reconstruction algorithm applies suitable techniques, such as filtered back projection or iterative reconstruction methods, to estimate the attenuation coefficients at each voxel within the volume of interest. Once the attenuation coefficients are estimated, the 3D reconstruction algorithm assigns them to their corresponding voxels in the 3D space, creating a volumetric representation of the patient's anatomy, e.g., a patient’s lung. This reconstructed volume may then be displayed, allowing clinicians to examine and analyze the structures of interest from various perspectives.

[0104] Fig. 3 illustrates a flowchart of a method for removing artifacts to generate an improved reconstructed volume in accordance with the disclosure. The method may depend on two assumptions. The first assumption is that the interference appears as lines in axial slices. The second assumption is that the lines are distinguishable from the background of the axial slices. At block 301, a CBCT scan of a patient with location board beads is received. Then, at block 302, a 3D Fourier transform is performed on the CBCT scan with the location board beads resulting in Fourier images I. FIG. 6A illustrates a Fourier image I, which is an axial slice of a 3D Fourier transform of a CBCT scan volume.

[0105] Next, each Fourier image I may be filtered with background filters Fn at block 303 resulting in a composite filtered image R according to the following example filter expressions: Rn= I > 2 * FnAttorney Docket No. A0009847WD00 (00017-01221PCT00)R — R-^ + R2+ R3 whereFi = [1, 1, 1, 1, 0, 1, 1, 1, 1] / 8F2= [1, 1, 1, 1, 0, 0, 1, 1, 1, 1] / 8F3= [1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1] / 8FIG. 6B illustrates an example of a filtered image resulting from filtering the Fourier image of FIG. 6A with the background filter F3. And FIG. 7 illustrates a filtered image R resulting from filtering the Fourier image of FIG. 6A with multiple background filters Fy, 2, and F3.

[0106] Next, at block 304, in a slice, e.g., an axial slice, of the 3D Fourier transform volume connected components with at least a predetermined number of pixels are retained. In aspects, the predetermined number of pixels may be at least ten pixels.

[0107] At block 306, remaining connected components in the slice are dilated by two pixels and the resulting image is multiplied by the filtered image D, resulting in a segmented image. The filtered image D may be generated by applying the background filters Fn to the original Fourier image I as follows:Dn= I > I * Fnwhere * is the convolution operator and D = £)-£ + D2 + D3.FIG. 8 illustrates a filtered image D resulting from filtering the Fourier image of FIG. 6A with multiple background filters Fy, F2, and F3 according to the filter expression Dn.

[0108] At block 308, pixels at the middle, vertical strip and at the rims of the segmented image are removed using a mask. FIG. 9 illustrates a mask configured to remove pixels at a middle, vertical strip and at rims of the segmented image. And FIG. 10 illustrates a masked image resulting from applying the mask of FIG. 9 to the segmented image. In aspects, other suitable masks may be used. For example, the mask may remove pixels at the rims of the segmented image.

[0109] Next, at block 309, the method 300 determines whether all axial slices are processed. If the method 300 determines that all axial slices are not processed, the method 300 repeats blocks 304-308 for another axial slice of the 3D Fourier transform. If, on the other hand, the method 300 determines that all axial slices are processed, a modified FourierAttomey Docket No. A0009847WD00 (00017-01221PCT00) transform volume I’ is generated at block 310 from the segmented images, which may be referred to as segmentation BI’. The modified Fourier transform volume I’ may be generated according to the following expressions:

[0110] Before ending at block 314, a volume is reconstructed using the modified Fourier transform volume I’ at block 312. The volume may be reconstructed using a suitable reconstruction method such as filtered back projection, iterative reconstruction, Fourier-based reconstruction employing the Fast Fourier Transform (FFT), model-based iterative reconstruction, or machine learning and / or Al-based reconstruction. The machine learning and / or Al-based reconstruction may employ deep learning to enhance existing reconstruction methods or learn the mappings from 2D projection data to a 3D volume space.

[0111] FIG. 11A illustrates an axial slice of an example of a volume reconstructed from masked images including the masked image of FIG. 10. As compared to FIG. 11B, which illustrates an axial slice from a CBCT volume without the location board beads, the axial slice of the example of the reconstructed volume of FIG. 11 A closely corresponds to the axial slice from the CBCT volume without artifacts from the location board beads.

[0112] In aspects, a 3D inpainting method may be applied to the 3D reconstruction. The inpainting method may include identifying artifacts in the 3D reconstruction. The artifacts may be identified by a segmentation method and / or a manual identification method, in which marking tools are provided enabling a user to mark artifacts and / or confirm or modify segmented artifacts in a 3D reconstruction displayed on a display to the user. The 3D inpainting method may include preprocessing the reconstruction volume. The preprocessing may include reducing artifacts to increase the performance, e.g., efficiency and / or speed, of the 3D inpainting method applied to the 3D reconstruction.

[0113] The 3D inpainting method also includes applying a 3D inpainting algorithm. The 3D inpainting algorithm may be a patch-based, structural, texture synthesis, and / or a partial differential equation (PDE)-based algorithm. The PDE-based inpainting algorithm may diffuse voxel values into a voxel region needing inpainting while preserving voxel edges. The patch-Attorney Docket No. A0009847WD00 (00017-01221PCT00) based algorithm, for example, may search for similar 3D voxel patches within the reconstruction volume to use as replacements in the one or more voxel regions needing inpainting. The 3D inpainting method may optionally include iterative refining of the voxel region needing inpainting by, for example, repeatedly applying the inpainting method to the 3D reconstruction. In some aspects, applying the inpainting algorithm may include generating a 3D mask and applying the 3D mask to the voxels of the 3D reconstruction.

[0114] The 3D inpainting method may include post-processing to further refine the inpainted 3D reconstruction. The post-processing may include smoothing discontinuities and / or enhancing voxel edges and / or other features of anatomy. The 3D inpainting method may include validating the inpainted 3D reconstruction. Validating the inpainted 3D reconstruction may involve displaying the inpainted 3D reconstruction to a clinician for inspection and / or automatically comparing the inpainted 3D reconstruction with other 3D reconstructions, e.g., 3D reconstructions including expected anatomy.

[0115] In aspects, artificial intelligence (Al) and / or machine learning (ML) methods may be incorporated into and / or employed by the systems and methods of this disclosure. The Al and / or ML methods may generate, update, and use a model to identify and remove artifacts from medical images or volumetric reconstructions based on the medical images. The artifact removal model may include a deep learning model, such as a convolutional neural network (CNN), which may be used to identify artifacts in the medical images or volumetric reconstruction and process the medical images or volumetric reconstruction to remove or otherwise address the identified artifacts. The medical image or volumetric reconstruction datasets may include image datasets with and without artifacts.

[0116] The AI / ML methods may involve image preprocessing, which may include preprocessing the CBCT and / or non-CBCT images to enhance their quality and make them suitable for analysis. The image preprocessing may include noise reduction, normalization to standardize the intensity levels across images, resizing, and / or augmentation to increase the diversity of the dataset. The augmentation may involve flipping, rotation, zooming, and / or any other technique suitable for processing the images to optimize the model used to remove artifacts from the images or the volumetric reconstruction.

[0117] The medical images or volumetric reconstructions may be manually or automatically labelled, marked, annotated, or otherwise associated with relevant information (e.g., metadata) to facilitate the generation, updating, and use of the artifact removal model. The system and methods of this disclosure may include an application that displays the medical images or volumetric reconstructions to a clinician and enables the clinician to label, mark,Attorney Docket No. A0009847WG00 (00017-01221PCT00) annotate, or associate information with the medical images or volumetric reconstructions, e.g., via a virtual marking or drawing tool.

[0118] The AI / ML methods may include dividing the image dataset into training, validation, and testing image datasets. The training dataset may be used to train the artifact removal model, the validation set may be used to tune the artifact removal model parameters and prevent overfitting, and the test image dataset may be used to evaluate the artifact removal model’s performance.

[0119] Training the artifact removal model may involve learning from the preprocessed and marked images and updating the artifact removal model based on the learning. Learning may include analyzing preprocessed and marked images with and without artifacts. The artifact removal model’s parameters may be adjusted based on the accuracy of predicting the presence of artifacts and predicting a medical image and / or volume reconstruction without artifacts.

[0120] Validating the artifact removal model may involve fine tuning the artifact removal model parameters and hyperparameters based on the validation image dataset. This may improve the artifact removal model’s generalization capability and prevent the artifact removal model from overfitting the training image dataset.

[0121] The AI / ML methods may include evaluating the trained model’ s performance using the test image dataset. Metrics such as accuracy, precision, recall, and Fl score may be used to assess the artifact removal model’s effectiveness in accurately identifying and removing artifacts.

[0122] Once the artifact removal model is trained and evaluated, the artifact removal model may be deployed in a clinical setting. This may involve integrating the artifact removal model into medical imaging systems of the disclosure to assist clinicians in diagnosing and planning treatment for patients.

[0123] The AI / ML methods may include continuously monitoring the artifact removal model’s performance and updating the artifact removal model as needed. Updating the artifact removal model may include retraining the artifact removal model with new image and / or volumetric reconstruction datasets to ensure and / or improve accuracy.

[0124] FIG. 12 illustrates a method of generating reconstructed volumes to present an accurate reconstructed volume to a clinician to help the clinician accurately visualize and navigate a patient’s anatomy during a medical procedure. At block 1201, the method 1200 determines the imaging mode. Block 1201 may include determining whether the system is in a fluoroscopic or tomographic imaging mode based on information regarding, for example, a commanded or current state of the system. For example, the method 1200 may determine atAttorney Docket No. A0009847WG00 (00017-01221PCT00) block 1201 that the system is in a tomographic imaging mode, e.g., a CBCT imaging mode, based on user input, e.g., a command, to start the system in the tomographic imaging mode. Block 1201 may include determining that the system is in a fluoroscopic imaging mode in response to detecting the presence of a fiducial board or a grid of radiopaque markers.

[0125] If, at block 1201, the method 1200 determines that the system is in a tomographic imaging mode, e.g., a CBCT imaging mode, a command or any suitable signal to configure radiopaque markers for the tomographic imaging mode is sent to the system, e.g., a command or any suitable signal to move the radiopaque markers outside of the imaging field of view at block 1212. Then, images of a tomographic scan of a patient with the radiopaque markers configured for the tomographic imaging mode are received, e.g., images of a tomographic scan of a patient without radiopaque markers at block 1214. For tomographic imaging mode, there may be a configuration without radiopaque markers or a configuration with a limited number of radiopaque markers, e.g., 3-6 radiopaque markers. At block 1218, an image volume of the patient is reconstructed based on the images of the tomographic scan, and then the method 1200 ends at block 1224.

[0126] If, at block 1201, the method 1200 determines that the system is in a nontom ographic imaging mode, e.g., a fluoroscopic imaging mode, a command or any suitable signal to configure radiopaque markers for the fluoroscopic imaging mode is sent to the system, e.g., a command or any suitable signal to move radiopaque markers within the field of view at block 1202. The radiopaque markers may be arranged in a grid pattern or any suitable pattern, e.g., a pattern suitable for determining a position and / or orientation of a fluoroscopic imaging system. Then, images of a non-tomographic scan, e.g., a fluoroscopic scan, of a patient with the radiopaque markers are received at block 1204. The non-tomographic imaging mode may be a C-arm fluoroscope imaging mode.

[0127] At block 1206, radiopaque markers are identified in the received images. Then, at block 1208, an image volume of the patient is reconstructed based on the identified radiopaque markers and the received images of the non-tomographic scan. At block 1220, the method 1200 determines whether to remove artifacts. If the method 1200 determines to remove artifacts, then, before ending at block 1224, an artifact removal method, the same as or similar to the artifact removal methods described herein, is applied to the reconstructed image volume, at block 1222. Otherwise, if the method 1200 determines not to remove artifacts, the method 1200 ends at block 1224. Block 1220 may involve determining the quality of the reconstructed volume. For example, if the reconstructed volume is less than a threshold amount of quality asAttorney Docket No. A0009847WD00 (00017-01221PCT00) determined by one or more quality metrics, then the method 1200 may determine that the artifact removal method should be applied to the reconstructed volume at block 1222.

[0128] In aspects of the disclosure, artifacts appearing in 3D reconstructions may be at least minimized by relocating metallic markers. In a non-tomographic imaging mode, e.g., a fluoroscopic imaging mode, a fiducial board 1310 may be inserted in a receptacle 1304 of a location board 1300, as illustrated in FIG. 13. And, in a tomographic imaging mode, e.g., a CBCT imaging mode, the fiducial board 1310 may not be inserted in or may be removed from the receptacle 1304 of the location board 1300 in response to a command to a robotic system or instructions displayed to the clinician. Alternatively, or additionally, in a tomographic imaging mode, a fiducial board having a limited number of markers or fiducials, e.g., three to six markers or fiducials, may be inserted in the receptacle 1304 of the location board 1300. FIG. 14 is a flowchart that illustrates a method 1400 for reconstructing a volume using a location board with a receptacle configured to removably receive a fiducial board.

[0129] FIG. 13 shows a location board 1300 with a receptacle 1304 configured to removably receive a fiducial board 1310. The fiducial board 1310 includes a pattern of radiopaque fiducials 1312 and a handle enabling a user to manually insert the fiducial board 1310 into or remove the fiducial board 1310 from the receptacle 1304. In aspect, there may be multiple fiducial boards with different patterns of radiopaque fiducials 1312, which may be selectively used depending on, for example, the anatomy being imaged, one or more characteristics of the patient, and / or the type of procedure being performed. While FIG. 13 illustrates a pattern of nine radiopaque fiducials 1312, it is contemplated that a fiducial board 1310 may include any number of radiopaque fiducials 1312 in any pattern suitable for accurately generating a 3D reconstruction of a target area of the patient.

[0130] In aspects, the fiducial board 1310 may be automatically inserted into or removed from the receptacle 1304 using a robot system. The robot system may be a local robot system (not shown) incorporated into the location board 1300 or a global robot system (not shown) residing in the operating room. The global robot system may include at least one arm, and an end-effector configured to grasp the fiducial board 1310, e.g., via the handle 1314, and to move the fiducial board 1310. The robot system may be configured to place the fiducial board 1310 in a tool holding receptacle or area incorporated into the operating table, forming part of the robot system, or at a suitable location in the operating room.

[0131] FIG. 14 is an example of a method 1400 of visualizing a catheter-based medical procedure using different imaging modalities, the location board 1300, and the fiducial board 1310. At block 1402, the method 1400 determines whether the medical procedure has enteredAttorney Docket No. A0009847WG00 (00017-01221PCT00) a navigation phase of the catheter-based medical procedure. If the method 1400 determines that the medical procedure has entered the navigation phase, the method 1400 advances to optional block 1406 orblock 1408. Otherwise, the method 1400 remains at block 1402. In some aspects, the medical procedure may automatically transition to optional block 1406 or block 1408 from a pre-navigation phase, e.g., an imaging or planning phase.

[0132] At optional block 1406, instructions to remove the fiducial board from a receptacle of the location board may be displayed. Alternatively, the method 1400 may determine whether the fiducial board is present within the location board. And if the method 1400 determines that the fiducial board is present within the location board, optional block 1406 is performed. Otherwise, the method 1400 may skip optional block 1406 and advance to block 1408. As another alternative to optional block 1406, the method 1400 may display instructions to confirm that the fiducial board is not present within the location board before advancing to block 1408.

[0133] At block 1408, tomographic images of the patient are received. At block 1410, an image volume is reconstructed based on the tomographic images and the reconstructed image volume is displayed. At block 1411, the method 1400 determines whether the navigation phase is complete to determine whether to continue with another phase of the medical procedure, e.g., a treatment or biopsy phase. If the method 1400 determines that the navigation phase is not complete, blocks 1408 and 1410 are repeated. Blocks 1408 and 1410 may be repeated at predetermined or preset intervals until the method 1400 determines that the navigation phase is complete.

[0134] If the method 1400 determines that the navigation phase is complete, a message including instructions to insert the fiducial board into the receptacle of the location board is displayed at block 1412. Alternatively, the instructions may be provided to a robotic system, e.g., in the form of commands, drive signals, or any suitable control data, causing the robotic system to insert the fiducial board into the receptacle. Block 1412 may include displaying a message indicating a particular fiducial board to insert into the receptacle of the location board. The fiducial board may include a limited number of fiducials, e.g., 3-6, to avoid the effects of artifacts, or a full grid of fiducials in which the fiducials are arranged equidistant from each other.

[0135] At block 1414, fluoroscopic images of a patient and the fiducials of the fiducial board are received. At block 1416, a position of a feature or medical device in the fluoroscopic images is determined based on the fiducials in the fluoroscopic images. Block 1416 may include identifying fiducials of the fiducial board in the received fluoroscopic images. BeforeAttorney Docket No. A0009847WD00 (00017-01221PCT00) ending at block 1420, fluoroscopic images are displayed and information is displayed on the fluoroscopic images based on the determined position of the feature, at block 1418.

[0136] In aspects of the disclosure, artifacts appearing in 3D reconstructions may be at least minimized by relocating metallic markers without removing a fiducial board. FIG. 16 illustrates a method 1600 for reconstructing a volume using a motorized location board configured to drive movement of fiducials during imaging to at least minimize the artifacts caused by the radiopaque fiducials. In general, the imaging process first involves performing a first medical scan, e.g., with a CBCT scanner or a fluoroscopy scanner. Next, the imaging process involves driving the radiopaque fiducials to the periphery of the location board. Then, the imaging process involves performing a second 3D scan and combining the first and second 3D scans to reconstruct a scan volume of the patient. In aspects, the image processing described herein may be applied to the second medical scan images to filter artifacts appearing in the periphery of the medical scan images.

[0137] FIGS. 15A and 15B are perspective views illustrating example structures and functions of a motorized location board 1500 configured to drive movement of radiopaque fiducials 1512. The motorized location board 1500 may include electric drive motors 1502, which drive respective drive shafts 1504. One or more radiopaque fiducials 1512 may operatively couple to each drive shaft 1506. In operation, after the first 3D scan is performed, the electric drive motors 1502 are controlled to rotate the drive shafts 1504 in a first direction, which causes the one or more radiopaque fiducials 1512 to travel from a center region 1521 to either a first peripheral region 1522a or a second peripheral region 1522b. After the one or more radiopaque fiducials 1512 reach either the first peripheral region 1522a or the second peripheral region 1522b, the second 3D scan is performed.

[0138] In aspects, the motorized location board 1500 may include any suitable number of electric drive motors, respective drive shafts, radiopaque fiducials, and peripheral regions. For example, the motorized location board 1500 may include a peripheral region on each side of the motorized location board 1500, three electric drive motors and three respective drive shafts associated with each peripheral region, and a single radiopaque fiducial operatively coupled to each drive shaft. The electric drive motors and the respective drive shafts may be configured in the motorized location board 1500 to define a desired or suitable pattern of twelve radiopaque fiducials.

[0139] FIG. 16 illustrates a method 1600 of visualizing a catheter-based medical procedure, in which at least one motor is used to move fiducials into and out of a radiographic field of view depending on the phase of the medical procedure and / or the type of radiographic modalityAttorney Docket No. A0009847WG00 (00017-01221PCT00) being used. As shown in FIG. 16 the method 1600 determines at block 1402 whether the medical procedure has entered a navigation phase of the catheter-based medical procedure. If the method 1600 determines that the medical procedure has entered the navigation phase, the method 1600 advances to optional block 1602 or block 1408. Otherwise, the method 1600 remains at block 1402. In some aspects, the medical procedure may automatically transition to optional block 1602 or block 1408 from a pre-navigation phase, e.g., an imaging or planning phase.

[0140] At optional block 1602, at least one motor is controlled to move fiducials outside a radiographic field of view of a radiographic system, e.g., a CBCT system. In aspects, the at least one motor may be controlled to move fiducials to marginal or peripheral regions of the location board. Alternatively, the method 1600 may determine whether the fiducials are outside a radiographic field of view of the radiographic system. And if the method 1600 determines that the fiducials are partially or fully inside a radiographic field of view, optional block 1602 is performed. Otherwise, the method 1600 may automatically skip optional block 1602 and advance to block 1408.

[0141] At block 1408, the method 1600 includes receiving tomographic images of a patient. At block 1410, an image volume is reconstructed based on the tomographic images and the reconstructed image volume is displayed. At block 1604, the method 1600 determines whether to transition to a target treatment phase, e.g., an energy or medication application phase or a biopsy phase. If the method 1600 determines that the target treatment phase has not started, blocks 1408 and 1410 are repeated. Blocks 1408 and 1410 may be repeated at predetermined or preset intervals until the method 1400 determines that the target treatment phase has started.

[0142] If the method 1600 determines that the target treatment phase has started, the at least one motor is controlled to move the fiducials into a radioscopic field of view, e.g., into a central region of the location board, at block 1606. For example, the at least one motor may be controlled to move fiducials inside the radioscopic field of view in a symmetrical manner. As another example, the at least one motor may be controlled to arrange the fiducials equidistant from each other or in a diagonal pattern.

[0143] At block 1414, fluoroscopic images of a patient and the fiducials in the radiographic field of view are received. At block 1608, fiducials in the fluoroscopic images are identified. At block 1610, a position of the fluoroscopic imaging system relative to EM transmitter is determined based on identified fiducials. At block 1612, the position of the feature in fluoroscopic images is determined based on determined position of the fluoroscopic imaging system relative to EM transmitter. Then, before ending at block 1420, the fluoroscopic imagesAttorney Docket No. A0009847WG00 (00017-01221PCT00) are displayed and information is displayed, e.g., overlayed, on the fluoroscopic images based on the determined position of the feature, at block 1418.

[0144] The systems and methods of the disclosure may include two or more operating modes. For example, the operating modes may include at least two of a C-arm fluoroscope imaging mode, a CBCT imaging mode, or an O-arm imaging mode. The modes may include common methods, some of which may be optional methods a clinician may select based on the particular situation.

[0145] All of, portions of, or a portion of the methods described herein may be performed, enabled, or facilitated by a computer system. For example, the computer system may include a display on which messages, prompts, indicators, or other user interface features suitable for guiding a clinician through all or a portion of the methods described herein are displayed.

[0146] While several aspects of the disclosure have been shown in the drawings, it is not intended that the disclosure be limited thereto, as it is intended that the disclosure be as broad in scope as the art will allow and that the specification be read likewise. Therefore, the above description should not be construed as limiting, but merely as exemplifications of particular aspects. Those skilled in the art will envision other modifications within the scope and spirit of the claims appended hereto.

[0147] Specifically, while aspects of the disclosure have been described with respect to CBCT and fluoroscopic scanners, it is not intended that the disclosure be limited thereto. The current disclosure contemplates use of the systems and methods described herein to plan a path to a target that avoids obstructions that may be present during the performance of various surgical procedures. Those skilled in the art would envision numerous other obstructions.

[0148] Detailed aspects of such tools, systems incorporating such tools, and methods using the same are described above. However, these detailed aspects are merely examples of the disclosure, which may be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting but merely as a basis for the claims and as a representative basis for allowing one skilled in the art to variously employ the disclosure in virtually any appropriately detailed structure.

[0149] Although aspects have been described in detail with reference to the accompanying drawings for the purpose of illustration and description, it is to be understood that the inventive processes and apparatus are not to be construed as limited thereby. It will be apparent to those of ordinary skill in the art that various modifications to the foregoing aspects may be made without departing from the scope of the disclosure as set forth in the following claims.Attorney Docket No. A0009847WD00 (00017-01221PCT00)

[0150] Various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the techniques). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a medical tool.

[0151] In one or more examples, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non- transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0152] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0153] The invention may be further described by reference to the following numbered paragraphs:1. A method of correcting for artifacts, comprising: receiving two-dimensional (2D) radiographic images of a patient; performing a Fourier transform on the 2D radiographic images, yielding Fourier images; filtering the Fourier images with at least one background filter, yielding at least one background image; segmenting the Fourier images based on the at least one background image, yielding segmented images;Attorney Docket No. A0009847WD00 (00017-01221PCT00) modifying the Fourier images based on the segmented images, yielding modified Fourier images; and reconstructing a volume based on the modified Fourier images.2. The method according to any of the preceding paragraphs, wherein the 2D radiographic images are captured by a tomographic imaging system or a fluoroscopic imaging system.3. The method according to any of the preceding paragraphs, wherein filtering the Fourier images includes filtering the Fourier images with at least three background filters.4. The method according to any of the preceding paragraphs, wherein the artifacts are caused by a pattern of markers or a periodic grid of markers.5. The method according to any of the preceding paragraphs, further comprising removing pixels at a center of the segmented images, at a margin of the segmented images, or at a center and a margin of the segmented images.6. The method according to any of the preceding paragraphs, further comprising retaining only connected components with eight to twelve pixels in the segmented images.7. The method according to paragraph 6, further comprising dilating the connected components by two pixels.8. The method according to paragraph 7, further comprising: generating D images according to an expression D = I > I * F, where I is an original Fourier image, * is a convolution operator, and F is a background filter; and applying the D images to the segmented images.9. A system for generating a medical image volume of a patient, the system comprising: a pattern of markers; a processor; and a memory having stored thereon instructions, which, when executed by the processor, cause the processor to: receive two-dimensional (2D) images of a patient and the pattern of markers;Attorney Docket No. A0009847WD00 (00017-01221PCT00) identify the pattern of markers in the 2D images, yielding an identified pattern of markers; perform a Fourier transform on the 2D images, yielding Fourier images; filter the Fourier images with at least one background filter, yielding at least one background image; segment the Fourier images based on the at least one background image, yielding segmented images; modify the Fourier images based on the segmented images, yielding modified Fourier images; and reconstruct a medical image volume based on the modified Fourier images and the identified pattern of markers.10. The system according to paragraph 9, wherein the instructions, when executed by the processor, further cause the processor to remove pixels at a center of the segmented images, at a margin of the segmented images, or at a center and a margin of the segmented images.11. The system according to any of the preceding paragraphs, wherein the pattern of markers are positioned outside of a region of interest.12. The system according to any of the preceding paragraphs, wherein the instructions, when executed by the processor, further cause the processor to: generate D images according to an expression D = I > I * F, where I is an original Fourier image, * is a convolution operator, and F is a background filter; and apply the D images to the segmented images.13. A system comprising: a fiducial board including: moveable fiducials; and at least one motor operably coupled to the moveable fiducials; a processor; and a memory having stored thereon instructions, which, when executed by the processor, cause the processor to: determine that a tomographic scan is to be performed; in response to determining that the tomographic scan is to be performed:Attorney Docket No. A0009847WG00 (00017-01221PCT00) receive first images of the tomographic scan of a patient with the moveable fiducials located in a marginal region of the fiducial board; reconstruct an image volume of the patient based on the first images; and display the image volume. determine that a fluoroscopic scan is to be performed; and in response to determining that the fluoroscopic scan is to be performed: control the at least one motor to move the moveable fiducials to a central region of the fiducial board; receive second images captured by a fluoroscopic imaging system in a fluoroscopic scan of a patient and the moveable fiducials located in the central region of the fiducial board; identify the moveable fiducials in the second images, yielding identified moveable fiducials; and estimate a pose of the fluoroscopic imaging system based on the identified moveable fiducials and the second images, yielding an estimated pose.14. The system according to paragraph 13, wherein the instructions, when executed by the processor, further cause the processor to: determine a position of a feature in the second images based on the estimated pose, yielding a determined position of the feature; and overlay information on the second images based on the determined position of the feature.15. The system according to paragraph 14, wherein the feature is a target, and wherein the instructions, when executed by the processor, further cause the processor to: generate a representation of the target based on the image volume; and augment the second images with the representation of the target.16. The system according to any of the preceding paragraphs, wherein the marginal regions are two opposing sides of the fiducial board.Attorney Docket No. A0009847WD00 (00017-01221PCT00)17. The system according to any of the preceding paragraphs, wherein the instructions, when executed by the processor, further cause the processor to: determine a subset of the moveable fiducials that have a high probability of generating artifacts in the second images at a region including a target; and control the at least one motor to move the subset of the moveable fiducials out of the region including the target.18. The system according to any of the preceding paragraphs, wherein the fiducial board is incorporated into a location board.19. A system for generating an image volume of a patient, the system comprising: a fiducial board; an electromagnetic (EM) emitter configured to emit an EM signal; a receptacle configured to receive the fiducial board; a processor; and a memory having stored thereon instructions, which, when executed by the processor, cause the processor to: in a tomographic imaging mode, receive first two-dimensional (2D) radiographic images of a patient at different angles without the fiducial board disposed in the receptacle; reconstruct an image volume of the patient based on the first 2D radiographic images; and in a non-tomographic imaging mode, receive second 2D radiographic images of a patient at different angles with the first fiducial board disposed in the receptacle; identify fiducials of the fiducial board in the second 2D radiographic images, yielding identified fiducials; and estimate a pose of a non-tomographic imaging system based on the identified fiducials and the second 2D radiographic images.20. The system according to paragraph 19, wherein, in the tomographic imaging mode, the instructions, which when executed by the processor, further cause the processor to reconstruct a second image volume of the patient based on the estimated pose and the second 2D radiographic images.Attorney Docket No. A0009847WG00 (00017-01221PCT00)21. The system according to any of the preceding paragraphs, wherein the receptacle is a stand-alone receptacle configured to be placed between a patient and an operating table or the receptacle is incorporated or coupled to the operating table, and wherein the fiducial board is configured as a drawer having a handle for manually removing or inserting the fiducial board from or to the receptacle.22. The system according to any of the preceding paragraphs, wherein the EM emitter is located in a side area adjacent to a patient area or in a location board including the receptacle; or the EM emitter is coupled to a side portion of an operating table.23. The system according to any of the preceding paragraphs, wherein the tomographic imaging mode is a computed tomography mode, a cone-beam computed tomography mode, a positron emission tomography mode, or a single photon emission computed tomography mode.24. The system according to any of the preceding paragraphs, further comprising at least one EM receiver, wherein the instructions, when executed by the processor, cause the processor to calculate a position of the at least one EM receiver based on the EM signal received by the at least one EM receiver.25. A method comprising: generating, by at least one electromagnetic (EM) transmitter, an EM signal; calculating position information of at least one EM receiver based on the EM signal received by the at least one EM receiver; receiving instructions to operate according to a navigation phase; in response to receiving the instructions to operate according to a navigation phase: capturing tomographic images with a tomographic imaging system; reconstructing an image volume based on the tomographic images; and displaying the position information and the image volume; determining that the navigation phase is complete; in response to determining that the navigation phase is complete:Attorney Docket No. A0009847WG00 (00017-01221PCT00) moving fiducial markers into a field of view; capturing fluoroscopic images with a fluoroscopic imaging system; identifying fiducial markers in the fluoroscopic images, yielding identified fiducial markers; determining a position of the fluoroscopic imaging system relative to the EM transmitter based on the identified fiducial markers; determining a position of a feature in the fluoroscopic images based on the determined position of the fluoroscopic imaging system relative to the EM transmitter; displaying the fluoroscopic images; and displaying information on the fluoroscopic images based on the determined position of the feature in the fluoroscopic images.26. The method according to paragraph 25, wherein the tomographic imaging system is a computed tomography system, a cone-beam computed tomography system, a positron emission tomography system, or a single photon emission computed tomography system.27. The method according to any of the preceding paragraphs, wherein the fluoroscopic imaging system is a C-arm fluoroscope.28. The method according to any of the preceding paragraphs, further comprising transitioning to a target treatment phase in response to determining that the navigation phase is complete.29. The method according to any of the preceding paragraphs, further comprising: receiving instructions to change from a tomographic imaging mode to a fluoroscopic imaging mode; and in response to receiving the instructions to change from the tomographic imaging mode to the fluoroscopic imaging mode, displaying instructions to insert a fiducial board into a receptacle of a location board.30. The method according to any of the preceding paragraphs, further comprising: detecting a presence of a fiducial board including the fiducial markers; andAttorney Docket No. A0009847WG00 (00017-01221PCT00) determining that the navigation phase is complete in response to detecting the presence of the fiducial board.31. The method according to any of the preceding paragraphs, further comprising: detecting a presence of a fiducial board including the fiducial markers; in response to detecting the presence of the fiducial board, prompting a user to confirm transition to a target treatment phase; receiving information confirming transition to the target treatment phase; and in response to receiving the information confirming transition to the target treatment phase, determining that the navigation phase is complete.32. The method according to any of the preceding paragraphs, wherein the feature is a target, further comprising: generating a representation of the target based on the image volume; and augmenting the second images with the representation of the target.33. The method according to any of the preceding paragraphs, further comprising, in response to receiving the instructions to operate according to a navigation phase, removing fiducial markers from the field of view.34. The method according to any of the preceding paragraphs, wherein the fluoroscopic images are live images.

Claims

Attorney Docket No. A0009847WD00 (00017-01221PCT00)WHAT IS CLAIMED IS:

1. A method of correcting for artifacts, comprising: receiving two-dimensional (2D) radiographic images of a patient; performing a Fourier transform on the 2D radiographic images, yielding Fourier images; filtering the Fourier images with at least one background filter, yielding at least one background image; segmenting the Fourier images based on the at least one background image, yielding segmented images; modifying the Fourier images based on the segmented images, yielding modified Fourier images; and reconstructing a volume based on the modified Fourier images.

2. The method according to claim 1, wherein filtering the Fourier images includes filtering the Fourier images with at least three background filters.

3. The method according to any of the preceding claims, further comprising retaining only connected components with eight to twelve pixels in the segmented images.

4. The method according to claim 3, further comprising dilating the connected components by two pixels.

5. The method according to claim 4, further comprising: generating D images according to an expression D = I > I * F, where I is an original Fourier image, * is a convolution operator, and F is a background filter; and applying the D images to the segmented images.

6. A system for generating a medical image volume of a patient, the system comprising: a pattern of markers; a processor; and a memory having stored thereon instructions, which, when executed by the processor, cause the processor to: receive two-dimensional (2D) images of a patient and the pattern of markers;Attorney Docket No. A0009847WD00 (00017-01221PCT00) identify the pattern of markers in the 2D images, yielding an identified pattern of markers; perform a Fourier transform on the 2D images, yielding Fourier images; filter the Fourier images with at least one background filter, yielding at least one background image; segment the Fourier images based on the at least one background image, yielding segmented images; modify the Fourier images based on the segmented images, yielding modified Fourier images; and reconstruct a medical image volume based on the modified Fourier images and the identified pattern of markers.

7. The system according to claim 6, wherein the instructions, when executed by the processor, further cause the processor to remove pixels at a center of the segmented images, at a margin of the segmented images, or at a center and a margin of the segmented images.

8. The system according to any of the preceding claims, wherein the instructions, when executed by the processor, further cause the processor to: generate D images according to an expression D = I > I * F, where I is an original Fourier image, * is a convolution operator, and F is a background filter; and apply the D images to the segmented images.

9. A system comprising: a fiducial board including: moveable fiducials; and at least one motor operably coupled to the moveable fiducials; a processor; and a memory having stored thereon instructions, which, when executed by the processor, cause the processor to: determine that a tomographic scan is to be performed; in response to determining that the tomographic scan is to be performed: receive first images of the tomographic scan of a patient with the moveable fiducials located in a marginal region of the fiducial board;Attorney Docket No. A0009847WG00 (00017-01221PCT00) reconstruct an image volume of the patient based on the first images; and display the image volume. determine that a fluoroscopic scan is to be performed; and in response to determining that the fluoroscopic scan is to be performed: control the at least one motor to move the moveable fiducials to a central region of the fiducial board; receive second images captured by a fluoroscopic imaging system in a fluoroscopic scan of a patient and the moveable fiducials located in the central region of the fiducial board; identify the moveable fiducials in the second images, yielding identified moveable fiducials; and estimate a pose of the fluoroscopic imaging system based on the identified moveable fiducials and the second images, yielding an estimated pose.

10. The system according to claim 9, wherein the instructions, when executed by the processor, further cause the processor to: determine a position of a feature in the second images based on the estimated pose, yielding a determined position of the feature; and overlay information on the second images based on the determined position of the feature.

11. The system according to claim 10, wherein the feature is a target, and wherein the instructions, when executed by the processor, further cause the processor to: generate a representation of the target based on the image volume; and augment the second images with the representation of the target.

12. A method comprising: generating, by at least one electromagnetic (EM) transmitter, an EM signal; calculating position information of at least one EM receiver based on the EM signal received by the at least one EM receiver; receiving instructions to operate according to a navigation phase;Attorney Docket No. A0009847WG00 (00017-01221PCT00) in response to receiving the instructions to operate according to a navigation phase: capturing tomographic images with a tomographic imaging system; reconstructing an image volume based on the tomographic images; and displaying the position information and the image volume; determining that the navigation phase is complete; in response to determining that the navigation phase is complete: moving fiducial markers into a field of view; capturing fluoroscopic images with a fluoroscopic imaging system; identifying fiducial markers in the fluoroscopic images, yielding identified fiducial markers; determining a position of the fluoroscopic imaging system relative to the EM transmitter based on the identified fiducial markers; determining a position of a feature in the fluoroscopic images based on the determined position of the fluoroscopic imaging system relative to the EM transmitter; displaying the fluoroscopic images; and displaying information on the fluoroscopic images based on the determined position of the feature in the fluoroscopic images.

13. The method according to claim 12, further comprising: receiving instructions to change from a tomographic imaging mode to a fluoroscopic imaging mode; and in response to receiving the instructions to change from the tomographic imaging mode to the fluoroscopic imaging mode, displaying instructions to insert a fiducial board into a receptacle of a location board.

14. The method according to any of the preceding claims, further comprising: detecting a presence of a fiducial board including the fiducial markers; and determining that the navigation phase is complete in response to detecting the presence of the fiducial board.

15. The method according to any of the preceding claims, further comprising: detecting a presence of a fiducial board including the fiducial markers;Attorney Docket No. A0009847WG00 (00017-01221PCT00) in response to detecting the presence of the fiducial board, prompting a user to confirm transition to a target treatment phase; receiving information confirming transition to the target treatment phase; and in response to receiving the information confirming transition to the target treatment phase, determining that the navigation phase is complete.

Images