Cone beam and 3D fluoroscopic lung navigation

Abstract

The present disclosure relates to cone-beam and 3D fluoroscope lung navigation. A method and system for reducing the deviation between computed tomography images and a patient using three-dimensional reconstruction. The method utilizes cone-beam imaging or three-dimensional fluoroscopy to supplement or replace preoperative computed tomography imaging.

Description

Technical Field

[0001] This disclosure relates to methods and systems for reducing the deviation between computed tomographic images and a patient by using cone-beam computed tomography imaging. Background Technology

[0002] Lung diseases can cause one or more parts of a patient's lungs to lose their normal function, and therefore may require treatment. Lung surgery can be very complex, and it is greatly aided if the surgeon performing the procedure can visualize how the airways and other structures in the patient's lungs are reshaped and how the tools are positioned. Traditional preoperative imaging is helpful to some extent for the former, but it does not provide guidance for the latter.

[0003] Systems for displaying images and tracking tools in a patient's lungs typically rely on preoperative data, such as computed tomography (CT) scans performed before the start of the treatment procedure, sometimes days or weeks prior. However, such systems do not account for changes that may have occurred after the CT scan or movement that may occur during the treatment procedure. Systems, apparatus, and methods for improving the process of identifying and visualizing a patient's lungs and the structures and tools located therein are described below. Summary of the Invention

[0004] This disclosure relates to a system and method for registering an image with a cavity network, the method comprising detecting the location of a sensor in the cavity network. The registration method further comprises: receiving an image for 3D reconstruction of the cavity network using the sensor within the cavity network; presenting the 3D reconstructed image on a user interface; receiving an indication of the location of a target in the 3D reconstructed image; generating a path through the cavity network to the target; and determining whether the sensor has moved from a detected location after receiving the image for 3D reconstruction, wherein the cavity network and the 3D reconstruction are registered when it is determined that the location of the sensor is the same as the detected location. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs, each configured to perform the actions of the method, on one or more computer storage devices.

[0005] The implementation may include one or more of the following: receiving measurement data when it is determined that the positioning of the sensor has changed; registering the cavity network with the 3D reconstruction or generating a 3D model of the cavity network based on the measurement data. The method may further include displaying the 3D reconstruction, a 2D slice image derived from the 3D reconstruction, a 3D model derived from the 3D reconstruction, or the path during a virtual bronchoscopy. Alternatively or additionally, the method may further include displaying the sensor's positioning along the path in a user interface.

[0006] Another aspect of this disclosure is a method for registering an image with a cavity network, the method comprising: receiving a preoperative computed tomography (CT) image of the cavity network; receiving an indication of a target within the cavity network; and generating a path through the cavity network to the target. The registration method further comprises receiving an image for 3D reconstruction of the cavity network; transforming the coordinates of the preoperative CT image to the coordinates of the 3D reconstruction to register the preoperative CT image with the 3D reconstruction; and updating the positioning of the catheter in the 3D reconstruction image or 3D model upon detection of catheter movement.

[0007] The method may further include displaying the 3D reconstruction, a 2D slice image derived from the 3D reconstruction, a 3D model derived from the 3D reconstruction, or a virtual bronchoscopy on a user interface. On the other hand, the method includes generating a 3D model from the 3D reconstructed image before transforming the preoperative CT coordinates and the 3D reconstructed coordinates, and may also include matching features from the CT image with the 3D reconstruction and the 3D model derived from the 3D reconstruction. Alternatively, the method includes generating a 3D model from the 3D reconstruction after transferring the target and the path from the preoperative CT image to the 3D reconstruction. The method may include receiving measurement data, wherein the measurement data is received before receiving the 3D reconstruction, or the measurement data is received after transferring the target and the path from the preoperative CT image to the 3D reconstruction to register the 3D reconstruction with the cavity network.

[0008] Another aspect of this disclosure is a method for registering an image with a cavity network, the method comprising: receiving a preoperative computed tomography (CT) image of the cavity network; receiving an indication of a target within the cavity network; generating a path through the cavity network to the target; generating a CT 3D model; detecting the location of a catheter within the cavity network; registering the preoperative CT image with the detected location of the catheter; receiving an indication of the location of a sensor in the preoperative CT or the CT 3D model and updating the location in a user interface until the target is near; receiving an image for 3D reconstruction of the cavity network; and detecting the location of the catheter and updating the location in a user interface.

[0009] The method may further include generating a 3D model from the 3D reconstruction. Still further, the method may include retrieving measurement data from memory; presenting the 3D reconstruction on a user interface; receiving an indication of the target's position in the 3D reconstruction; and generating a path in the 3D reconstruction or the 3D model. Still further, the method may further include determining the relative positions of the target and the catheter in the 3D reconstruction, and updating the relative positions of the target and the catheter in the preoperative CT image and the CT 3D model based on the determined relative positions in the 3D reconstruction or the 3D model. Additionally, the method may include: registering the preoperative CT image with the 3D reconstruction, and transferring the target and the path from the preoperative CT image and the 3D model to the 3D reconstruction and the 3D model. Attached Figure Description

[0010] Various aspects and features of this disclosure are described below with reference to the accompanying drawings, in which:

[0011] Figure 1 It is a schematic diagram depicting an imaging and navigation system according to this disclosure;

[0012] Figure 1A It describes various aspects according to this disclosure. Figure 1 A schematic diagram of the end view of the imaging and navigation system;

[0013] Figure 2 It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0014] Figure 3 It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0015] Figure 4A This is a partial flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0016] Figure 4B This is a partial flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0017] Figure 5 It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0018] Figure 6 It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0019] Figure 7 It is a block diagram depicting the features and components of a computing device according to various aspects of this disclosure;

[0020] Figure 8It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0021] Figure 9 It is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure;

[0022] Figure 10 This is a flowchart of the imaging and navigation procedures according to various aspects of this disclosure. Detailed Implementation

[0023] This disclosure relates to systems and methods for using cone-beam computed tomography (CBCT) images or 3D fluorescence microscopy images in conjunction with intracavitary navigation techniques and systems.

[0024] For the purpose of identifying areas of interest or targets to which an endoscope or catheter is to be navigated, many systems exist that utilize the output from preoperative computed tomography (CT) scans (e.g., CT image data). Typically, this navigation will be a network of cavities, such as the airways of the lungs or biliary tract, but it can also be spatial, such as the pleural cavity or other locations within the patient's body. These systems typically have two phases. The first phase is the planning phase, in which the target is identified and a three-dimensional (3D) model is generated. The second phase is the navigation phase, in which the position of the catheter within the patient's body is detected and depicted on the 3D model or other images to allow the clinician to navigate to the identified target. By updating the catheter's positioning within the 3D model, the clinician is able to perform procedures such as biopsies or treatments at the target location. One such system is the ILLUMISITE system, marketed by Medtronic PLC, which is an electromagnetic navigation (EMN) system.

[0025] Figure 1 A system 100 suitable for implementing the methods described herein is depicted. For example... Figure 1 As shown, system 100 is used to perform one or more surgeries on a patient supported on operating table 40. In this regard, system 100 typically includes a bronchoscope 50, a monitoring device 30, a tracking system 70, and a computing device 80.

[0026] A bronchoscope 50 is configured for insertion into a patient's airway through the mouth and / or nose. The bronchoscope 50 includes an illumination source and a video imaging system (not explicitly shown) and is coupled to a monitoring device 30, such as a video display, for displaying video images received from the video imaging system of the bronchoscope 50. In one embodiment, the bronchoscope 50 may operate in conjunction with a catheter guidance assembly 90. The catheter guidance assembly 90 includes a positionable guide (LG) 92 and a catheter 96. The catheter 96 may act as an extended working channel (EWC) and is configured for insertion into the patient's airway through the working channel of the bronchoscope 50 (although the catheter guidance assembly 90 may alternatively be used without the bronchoscope 50). The catheter guidance assembly 90 includes a handle 91 connected to the catheter 96, and the LG 92 and catheter 96 can be manipulated by rotating and compressing the handle. The catheter 96 is sized to fit into the working channel of the bronchoscope 50. During the operation of the catheter guiding assembly 90, the LG 92, which includes the EM sensor 94, is inserted into the catheter 96 and locked in position such that the EM sensor 94 extends beyond the desired distance of the distal tip 93 of the catheter 96. The position of the EM sensor 94, and therefore the distal tip 93 of the catheter 96, within the EM field generated by the EM field generator 76 can be determined by the tracking module 72 and the computing device 80.

[0027] LG 92 and catheter 96 can be selectively locked relative to each other via locking mechanism 99. A six-degree-of-freedom tracking system 70 is used to perform navigation, but other configurations are also envisioned. The tracking system 70 can be configured to be used with the catheter guidance assembly 90 to track the positioning of the EM sensor 94 as the combined catheter 96 moves through the patient's airway, as described in detail below. In one embodiment, the tracking system 70 includes a tracking module 72, a plurality of reference sensors 74, and an EM field generator 76. Figure 1 As shown, the EM field generator 76 is positioned below the patient. The EM field generator 76 and the plurality of reference sensors 74 are interconnected with a tracking module 72, which determines the position of each reference sensor 74 in six degrees of freedom. One or more reference sensors 74 are attached to the patient's chest. The six-degree-of-freedom coordinates of the reference sensors 74 are sent as data to a computing device 80 containing an application 81, where the data from the reference sensors 74 is used to calculate the patient's reference coordinate system.

[0028] Although the EM sensor 94 is described above as being included in the LG 92, it is also contemplated that the EM sensor 94 may be embedded or incorporated into a treatment tool such as the biopsy tool 62 or the treatment tool 64 (e.g., an ablation catheter), which may be used alternatively for navigation without requiring the LG 92 or the necessary tool exchanges required to use the LG 92. The EM sensor 94 may also be embedded or incorporated into the catheter 96, such as at the distal portion of the catheter 96, thereby enabling tracking of the distal portion of the catheter 96 without requiring the LG 92.

[0029] According to an embodiment, biopsy and treatment tools 62, 64 are configured to be inserted into the catheter guidance assembly 90 after navigation to the target location and removal of LG 92. Biopsy tool 62 can be used to collect one or more tissue samples from the target location, and in one embodiment, is further configured to be used in conjunction with a tracking system 70 to facilitate navigation of biopsy tool 62 to the target location and to track the position of the biopsy tool as it is manipulated relative to the target location to obtain a tissue sample. Treatment tool 64 is configured to operate in conjunction with a generator 66 (such as a radiofrequency generator or a microwave generator) and may comprise any of various ablation tools and / or catheters. Although in Figure 1 The illustrations depict biopsy tools and microwave ablation tools. Those skilled in the art will recognize that other tools, including, for example, RF ablation tools, brachytherapy tools, etc., can be similarly deployed and tracked without departing from the scope of this disclosure. Additionally, piercing tools and / or puncture tools can be used with and / or incorporated into the LG 92 to create an exit point, wherein the LG 92 and the resulting catheter 96 are navigated outside the patient's airway toward a target location, as further described below.

[0030] A radiographic imaging apparatus 20, such as a C-arm imaging apparatus capable of capturing images of at least a portion of a patient's lungs, is used in conjunction with system 100. The radiographic imaging apparatus 20 captures images from which 3D reconstructions, such as those from a CBCT apparatus or a 3D fluorescein microscope, can be generated. Typically, both CBCT images and 3D fluorescein images are captured by sweeping the radiographic imaging apparatus 20 across a defined sweep angle (e.g., 30–180 degrees and any integer value within that range). By processing the individual images or videos captured during the sweep, 3D reconstructions similar to conventional CT images can be generated. As will be understood, CBCT images have a resolution similar to CT images, while fluorescein images have a lower resolution.

[0031] like Figure 1As shown, the radiographic imaging apparatus 20 is connected to the computing device 80, allowing the application 81 to receive and process image data acquired by the radiographic imaging apparatus 20. However, the radiographic imaging apparatus 20 may also have a separate computing device located within itself, in a treatment room, or in a separate control room to first receive the image data acquired by the radiographic imaging apparatus 20 and relay such image data to the computing device 80. In one example, the radiographic imaging apparatus 20 is connected to a Picture Archiving and Communication System (PACS) server, which in turn is connected to the computing device 80 and the application 81. To avoid exposing clinicians to unnecessary radiation from repeated radiographic scans, when the radiographic imaging apparatus 20 performs CBCT and / or fluorescence microscopy scans, the clinician can leave the treatment room and wait in an adjacent room (such as a control room). Figure 1A An end view of a radiographic imaging apparatus according to the present disclosure is depicted, which can be used to image a patient when the patient is lying on an operating table 40.

[0032] The computing device 80 includes software and / or hardware, such as application 81, to facilitate various stages of EMN surgery, including generating a 3D model, identifying target locations, planning paths to the target locations, registering the 3D model with the patient's actual airway, navigating to the target location, and performing treatment at the target location. For example, the computing device 80 utilizes data acquired from CT scans, CBCT scans, magnetic resonance imaging (MRI) scans, positron emission tomography (PET) scans, and / or any other suitable imaging modality to generate and display a 3D model of the patient's airway, enabling the identification of target locations on the 3D model (automatically, semi-automatically, or manually) by analyzing the image data and / or the 3D model, and allowing the determination and selection of paths through the patient's airway to the target location. While image data may have gaps, omissions, and / or other defects contained within the image data, the 3D model is a smooth representation of the patient's airway, wherein any such gaps, omissions, and / or defects in the image data are filled or corrected. The 3D model can be displayed on a display monitor associated with the computing device 80, or in any other suitable manner.

[0033] Although described herein as generating 3D models from preoperative CT images, CBCT images, or 3D fluorescein images, application 81 may not require the generation of 3D models or even 3D reconstructions. Instead, the functionality may reside in a computing device associated with the radiographic imaging apparatus 20 or the PACS server. In this scenario, application 81 only needs to import the 3D reconstructions or 3D models generated by the radiographic imaging apparatus 20 or the PACS server from CT images, CBCT images, or fluorescein images.

[0034] Using the computing device 80, various views of image data and / or 3D models can be displayed to and manipulated by the clinician to facilitate the identification of target locations. As mentioned above, the target location can be the site within the patient's lungs where treatment is to be performed. For example, the treatment target can be located in lung tissue adjacent to the airways. Among other things, the 3D model can include a model airway tree corresponding to the actual airways of the patient's lungs, and show the various channels, branches, and bifurcations of the patient's actual airway tree. Additionally, the 3D model can include 3D renderings of lesions, landmarks, blood vessels and vascular structures, lymphatic vessels and structures, organs, other physiological structures, and / or the pleural surface and fissures of the patient's lungs. Some or all of the aforementioned elements can be selectively displayed, allowing the clinician to choose which elements should be displayed when viewing the 3D model.

[0035] After identifying the target location, the application 81 can be used to determine the path between the patient's trachea and the target location across the patient's airway. If the target location is located in lung tissue not directly adjacent to the airway, at least a portion of the path will lie outside the patient's airway to connect an exit point on the airway wall to the target location. In this case, LG 92 and catheter 96 will first be navigated along the first portion of the path across the patient's airway to the exit point on the airway wall. LG 94 can then be removed from catheter 96, and an entry tool, such as a puncture or insertion tool, can be inserted into catheter 96 to create an opening in the airway wall at the exit point. Catheter 96 can then be advanced through the airway wall into the parenchyma surrounding the airway. The entry tool can then be removed from catheter 96, and LG 92 and / or tools 62, 64 can be reinserted into catheter 96 to navigate catheter 96 to the target location along the second portion of the path outside the airway.

[0036] During the procedure, in conjunction with the tracking system 70, the EM sensor 94 enables tracking of the EM sensor 94 (and thus the distal tip 93 of the catheter 96 or the tools 62, 64) as the EM sensor 94 is advanced through the patient's airway following the path planned during the planning phase. Although generally described herein in conjunction with the EM sensor 94, this disclosure is not limited thereto. Instead, the positioning of the bronchoscope 50, catheter 96, or tools 62, 64 can be determined using a flexure sensor (e.g., a fiber-Bragg sensor) used to match the shape of the catheter 96 to the shape of the airway in a 3D model. By sensing the shape of the sensor and matching it to the airway, the accurate positioning of the sensor or the distal portion of the bronchoscope 50, catheter 96, or tools 62, 64 can be determined and displayed on the 3D model.

[0037] As an initial step in the surgery, when using a 3D model generated from a CT scan, the 3D model must be registered with the patient's actual airway so that the application 81 can display an indication on the 3D model of the position of the EM sensor 94 corresponding to its position within the patient's airway. Registration is necessary because the CT scan may be performed days, weeks, or even months before the actual surgery. Even if the CT scan is performed on the same day, it is not performed in the operating room, so registration is still necessary.

[0038] One possible registration method involves performing measurements of the patient's lungs by navigating the LG 92 to at least a second bifurcation of the airway in each lobe of the patient's lung. During this registration phase, the positioning of the LG 92 is tracked, and the 3D model is iteratively updated based on the tracked position of the sensor 94 within the actual airway of the patient's lung. While the registration process focuses on aligning the patient's actual airway with the airway of the 3D model, registration also ensures that the location of the lung's vascular structures, pleural surfaces, and clefts is accurately determined.

[0039] However, registration does not achieve a perfect match between the patient's lung localization and the 3D model. This mismatch has many causes and is commonly referred to as CT-to-body divergence. As an initial problem, conventional CT images are taken under full breath-holding conditions. That is, the patient is required to expand their lungs to their maximum extent and maintain this localization during imaging. This has the benefit of inflating the airways and increasing their visibility in the CT images, and makes it easier to generate highly detailed 3D models. However, when performing surgery, the patient is not under full breath-holding; instead, they are typically sedated and breathing tidal volumes. This results in differences in the shape and localization of the airways in the patient's lungs during surgery compared to during CT imaging. Therefore, even when the airways have been registered with the 3D model (e.g., using airway sweeping or another method), there will be a difference between the relative positions of the airways or targets identified in the lungs of the model and the actual relative positions of the patient's airways and targets.

[0040] One approach to addressing CT discrepancies with the body is to utilize a CBCT image dataset from radiographic imaging apparatus 20, rather than a traditional CT scan, as the starting point for surgery. In this process, the CBCT image data is used to generate and display a 3D model of the patient's airway, enabling the identification of target locations on the 3D model (automatically, semi-automatically, or manually) by analyzing the image data and / or the 3D model, and allowing the determination and selection of paths through the patient's airway to reach the target location. Although the following techniques are described in conjunction with CBCT images, those skilled in the art will understand that these techniques are equally applicable to any imaging technique capable of generating 3D reconstructions as described above (such as 3D fluorescein microscopy).

[0041] Figure 2 Presented for combination Figure 1 System 100 employs CBCT to enable the planning phase to be integrated with patient navigation and treatment occurrence methods 200. As will be understood, the patient is positioned on operating table 40, and reference sensor 74 is located on the patient's chest and connected to EM tracking system 70. Bronchoscope 50 and / or catheter 96 are inserted into the patient's airway, and images can be displayed on monitoring device 30. At step 202, the positioning of sensor 94 (e.g., a sensor associated with bronchoscope 50 or catheter 96 or another tool) can be detected, and an indication that the sensor positioning has been received by tracking system 70 can be presented on a user interface on computing device 80.

[0042] Then, at step 204, the radiographic imaging apparatus 20 can be engaged, and the computing device 80 receives the CBCT. The computing device 80 includes one or more applications for processing the CBCT data and presenting it on one or more user interfaces for manipulation and evaluation. At step 206, the applications analyze the CBCT data and generate a 3D model of the airway. This 3D model can be manipulated by the user through the user interface to ensure it has sufficient resolution and adequately captures the patient's airway (e.g., to a specific bifurcation point). The applications can receive a rejection of the 3D model, at which point additional CBCT images can be acquired, and the process restarts at step 204. The rejection can be based, for example, on a clinician's dissatisfaction with the 3D model (e.g., insufficient bifurcation generation, missing lobes or other important parts of the lungs), or alternatively, on the clinician's experience with the patient's physiology, where the 3D model simply appears incorrect. These types of deficiencies may be caused by incorrect or insufficient CBCT imaging or incorrect settings on the radiographic imaging apparatus 120.

[0043] At step 208, the application receives the 3D model, and at step 210, the user interface presents CBCT images or virtual CBCT images of the patient's lungs from a CBCT scan. These CBCT images are slice images taken at different points in cross-section of the patient's lungs or generated for different points in the patient's lungs. The user interface allows the user to scroll through these images showing one or more (but other planes are also possible) of the lungs in the axial, coronal, or sagittal planes, and allows the user to identify targets within the images. At step 212, the application receives indications of the target, and at step 214, a path through the airway to the target is generated. The target indication can be a manual marker provided by a clinician via the user interface on computing device 80. Alternatively, application 81 can perform image analysis and automatically detect the target and provide indications of its location. At step 216, the user interface then displays the path through the airway in one or more CBCT images, virtual CBCT images, or virtual bronchoscopy views of the 3D model. The user interface can additionally or alternatively display the path on one or more of the CBCT images or virtual CBCT images.

[0044] The CBCT images will also show the presence of the bronchoscope 50 or catheter 96 previously inserted into the patient. At step 217, the application can determine whether sensor 94 has moved since the CBCT images were acquired. If sensor 94 has not moved since the CBCT images were acquired in step 204, the location of the EM sensor detected in step 202 corresponds to the location of the distal end of the bronchoscope 50 or catheter 96 in the image. Thus, the EM coordinate system and the CBCT image system are registered to each other, and no further steps are required to register the patient's lungs with the 3D model generated from the CBCT images. Further navigation can be confidently performed across the 3D model following the planned path.

[0045] If it is determined at step 217 that sensor 94 has moved or moved beyond a certain threshold, an indicator can be presented on the user interface to suggest the user perform a measurement as described above, and at step 218, application 81 receives the measurement data. The measurement involves inserting EM sensor 94 into a lobe of the lung, and the tracking system 70 receives the EM sensor's location as it moves across the airway. A point cloud of locations (EMN coordinates) is created when hundreds or thousands of these locations are collected. Assuming all points are taken from a point cloud within the cavity network with a 3D dimensional shape, this 3D shape can later be matched with the 3D shape of the airway for registration with the patient's 3D model and airway. Once registered, the location of the detected EM sensor can be used to follow a path in the 3D model to reach the identified target. At step 220, the location of the detected EM sensor relative to the path and target is continuously updated on the user interface until it is determined at step 222 that the target has been reached and surgery is performed at step 224. The procedure can be a biopsy or treatment of the target, such as ablation (e.g., RF, microwave, cryotherapy, thermal, chemical, immunotherapy, or a combination thereof).

[0046] Regardless of whether the patient and CBCT images are registered due to sensor 94 not moving after imaging (step 216) or through registration using measurements (step 218), this registration using CBCT images should essentially eliminate any CT-to-body deviation issues, as the CBCT images are acquired with the patient in the exact same position as at the start of the navigation process. Furthermore, the CBCT images are taken while the patient is tidal breathing rather than fully holding their breath; therefore, when using conventional CT images while the patient is fully holding their breath, there will be discrepancies between the patient's lungs and the 3D model.

[0047] Although not described in detail here, the positioning and navigation of the EM sensor 94 (e.g., on the bronchoscope 50, catheter 96 or other tools) can be done manually in conjunction with the catheter guidance assembly 90 as described above, or can be achieved using a robot-driven catheter guidance assembly.

[0048] Combination Figure 3 An additional method 300 that can be used with system 100 is described. In method 300, at step 302, a CT image and / or a CT 3D model are received and stored in a memory associated with computing device 80. This is a standard preoperative CT image taken with a conventional CT imaging system while the patient is in a state of complete breath-holding, as described above. This preoperative CT image is processed by application 81, and at step 304, a path is generated to a target (e.g., the airways of the lungs) within the already imaged cavity network. Steps 302 and 304 implement the planning phase.

[0049] At optional step 306, which can occur at any time after the planning phase is completed, the patient is on operating table 40, and data from measurements (e.g., EM sensor 94 inserted into the airway) is received by tracking system 70 and processed by application 81 in computing device 80. At step 308, a CBCT image of the desired portion of the patient is acquired using radiographic imaging device 20 via application 81. This CBCT image may include bronchoscope 50 or another device including EM sensor 94. Optionally, at step 310, a CBCT 3D model can be generated from the CBCT image. Alternatively, the acquired CBCT image received at step 308 may include a 3D model generated by software residing on radiographic imaging device 20 or a PACS server and provided along with the CBCT image to computing device 80 and application 81.

[0050] Both the preoperative CT image used in the planning phase and the CBCT image acquired in step 308 are in Medical Digital Imaging and Communication (DICOMM) format. The DICOMM format contains a coordinate system referenced for image acquisition. Therefore, at step 312, application 81 transforms the coordinate system of the preoperative CT image to the coordinate system of the CBCT image captured by the radiographic imaging apparatus 20. Step 312 effectively registers the preoperative CT image with the CBCT image.

[0051] Alternatively, at step 311, application 81 aligns the CBCT 3D model generated at step 310 with the 3D model generated from the preoperative CT image and received at step 302. The alignment of the two 3D models registers the preoperative CT image with the CBCT image. The application may present either or both of the preoperative CT 3D model and the CBCT 3D model on a user interface and request user confirmation of the alignment or allow user interaction to complete the orientation of the two 3D models relative to each other, thereby completing the registration of the two 3D models. Alternatively, this may be performed automatically by application 81.

[0052] Another alternative to registration is to assume that the patient is aligned in the preoperative CT and CBCT images. This process relies on the fact that during imaging with the radiographic imaging device 20, the patient is always lying supine on the operating table 40, with the patient's chest extending away from the operating table 40 along its length, and the patient will be substantially in this position during the acquisition of the preoperative CT. During this registration process, application 81 can request a clinician via a user interface to identify common points in both the preoperative CT and CBCT images. This point can be a target, as described above with respect to method 200, or it can be a point appearing in both image datasets, such as the main carina of the lung or a rib, or some other feature. Alternatively, application 81 can utilize various image processing techniques to identify these common features in the two image datasets and register them with each other. Once identified manually or automatically, the two image datasets (e.g., preoperative CT and CBCT images) are registered with each other because it is assumed that the patient is substantially aligned in the same position on the operating table 40 in both images. As will be understood, automatically or by clinicians using a user interface to identify 2, 3, 4, 5, 10 points will improve registration as desired, or even more. In some respects, this can be achieved using image brightness matching inter-information technology. This can be aided by various deep learning methods, where empirical algorithms are developed by processing hundreds or thousands or more images and performing registration.

[0053] At step 314, once the two CT images or 3D models are registered to each other, all planning data generated using the preoperative CT images can be transferred to the CBCT image acquired at step 308 or the 3D model acquired at step 310. By transferring features from the preoperative CT images to the CBCT images, such as targets and paths to those targets, if a 3D model of the CBCT image was not generated at step 310, the model can now be generated at step 316, and it will contain the targets and paths already transferred from the preoperative CT images to the CBCT images at step 312. Alternatively, if the CBCT 3D model is generated at step 310, but the preoperative CT 3D model and the CBCT 3D model are not registered to each other at step 311, the transferred features can be matched to the CBCT 3D model at optional step 318. Regardless of when the transfer to features occurs, at step 320, application 81 can display the CBCT 3D model and the CBCT images on the user interface, and can display features from the planning stage identified in the preoperative CT images on the user interface.

[0054] If no measurement is performed at step 306, a measurement can be performed at step 322. This measurement registers the CBCT image and CBCT 3D model with the patient's lungs by navigating sensor 94 into the patient's airway, the sensor being embodied in bronchoscope 50, catheter 96, or another tool, thereby generating the point cloud discussed above. As will be understood, other registration methods may also be employed without departing from the scope of this disclosure. If a measurement was performed in step 306 above, the application can continue to perform EM sensor 94 movement analysis as described above in step 216 during CBCT image acquisition at step 308, thereby registering the patient's airway with the CBCT image and the 3D model generated therefrom. Once registered, the localization of the detected EM sensor can be used to follow the path in the CBCT 3D model to reach the target initially identified in the preoperative CT image. At step 324, the localization of the detected EM sensor 94 relative to the path and target is continuously updated on the user interface until the application 81 determines at step 326 that the target has been reached and surgery can be performed at step 328 upon arrival. Alternatively, to utilize the EM sensor 94 and detect its location, the radiographic imaging apparatus 20 may be able to generate fluorescence microscopy images. The location of the catheter 96 can be detected in one or more fluorescence microscopy images acquired by the radiographic imaging apparatus 20. This detection can be performed manually by a clinician using a user interface on the computing device 80, or it can be performed by the application 81 using image processing techniques. Because the coordinate system is identical between the CBCT image and the fluorescence microscopy image acquired by the same apparatus, the detected location of the catheter 96 in the fluorescence microscopy image can be transferred to the CBCT image or a CBCT 3D model. Fluorescence microscopy images can be acquired periodically as the catheter 96 navigates toward the target. The procedure can be a biopsy or treatment of the target, such as ablation (e.g., RF, microwave, cryotherapy, thermal, chemical, immunotherapy, or a combination thereof).

[0055] and Figure 2 The method is the same. Figure 3 This method eliminates CT-body bias because the CBCT images and model are generated when the patient is in the same position as they were during navigation. Furthermore, the target and path are shown in the CBCT 3D model and CBCT images. Further, any discrepancies in registration are minimized through the DICOMM registration process, acquiring CBCT images using sensor 94 in the images and / or receiving measurement data and matching it with the airway in the CBCT images and CBCT 3D model.

[0056] See Figure 4A and 4BMethod 400 is described. According to method 400, at step 402, a preoperative CT image is acquired and stored in the memory of the computing device 80. At step 404, application 81 processes the CT image, generates a 3D model, presents the CT image on a user interface (receiving an indication of the target on the image), and generates a path through the airway of the patient (or another cavity network) to reach the target. These steps complete the planning phase using the preoperative CT image.

[0057] After the planning phase is completed, the patient can be placed on the operating table 40, and the bronchoscope 50 or catheter 96 can be inserted, allowing the tracking system 70 to detect sensor 94 at step 406 and provide the data to application 81. Next, at step 408, during measurement, a point cloud of the sensor 94's location can be measured and received by the tracking system 70. Using the point cloud, at step 410, the patient and preoperative CT images, along with the 3D model generated therefrom, are registered to each other. As mentioned above, sensor 94 can be an EM sensor, a flexure sensor, or other sensors used to determine the location of catheter 96 or bronchoscope within the patient and to depict the preoperative location, thus registering the patient and preoperative CT images and 3D model.

[0058] At step 412, with the patient and preoperative CT images registered, navigation may begin when the tracking system 70 receives an indication of the new position of the sensor 94 as it moves through the patient's airway, and updates the detected location on the user interface as the path is followed to the identified target.

[0059] Once sensor 94, and more specifically bronchoscope 50, catheter 96, or other tool containing sensor 94, approaches the target, a CBCT image can be generated at step 414 using radiographic imaging apparatus 20. At this point, at least two different options are available. According to one option, at step 416, a CBCT 3D model is generated from the CBCT image. Next, at step 418, the point cloud can be retrieved from computerized apparatus 80 storing the point cloud generated by the measurements at step 408, and fitted with respect to the CBCT image and the CBCT 3D model. Alternatively, the method can jump forward to step 420, where the CBCT model and CBCT image are registered using any of the methods described herein and can be presented on a user interface. Because the CBCT image and 3D model are registered with the patient based on the measurements from step 408, in Figure 4B At step 422, the path and target identified in step 404 can be transferred from the preoperative CT image 3D model to the CBCT image and CBCT 3D model. Alternatively, in Figure 4BIn step 424, CBCT images and a CBCT 3D model can be presented on the user interface, allowing the target to be identified in the CBCT images. Once the application receives the target identification, it generates a path from the position of sensor 94 to the target, as depicted in the CBCT images and the CBCT model. Again, because the CBCT images are generated around the patient while the patient is on the operating table 40 for navigation, there is no CT-to-body deviation. As a result, navigation to the “last mile” of the target (e.g., the final 3mm to the target) can be performed with increased confidence that the target will be properly reached for biopsy or treatment. In step 426, subsequent movement of sensor 94 is detected, and the positioning of sensor 94 in the CT 3D model can be updated, and the surgery can be performed in step 428.

[0060] As described above, alternative methods can be followed after step 414. At step 430, the CBCT image containing the bronchoscope 50 or catheter 96 (or other tool) having sensor 94 is within the CBCT image, and the CBCT image and / or CBCT 3D model can be analyzed to determine the relative location of the target and the distal end of the bronchoscope 50 or catheter 96. This relative location determination can be automatically derived by application 81. Alternatively, the relative location can be determined by receiving an indication of the position of the bronchoscope 50 or catheter 96 via a user interface, wherein one or more of the target and the distal end of the bronchoscope 50 or catheter 96 are shown in the 2D image or 3D model. In both the preoperative CT image and the CBCT image, it can be assumed that the location of the target is the same. Then at step 432, application 81 can use the relative location data to update the location of the detected sensor 94 in the preoperative CT image and the 3D model derived from the preoperative CT. This location update will take into account CT and body deviations resulting from the use of the preoperative CT image and the 3D model for navigation. Again, at step 426, the movement of sensor 94 to the target in the last mile can be detected, and the surgery can be performed at step 428.

[0061] As will be understood, system 100, and specifically application 81 running on computing device 80, can be configured to control the operation of radiographic imaging apparatus 20. This control can be performed via user input to a user interface. Accordingly, alternatively, after registering preoperative CT or initial CBCT images with the patient (if desired) and identifying the target, the application can be configured to adjust the CBCT imaging field to focus on the target. Application 81 can focus all future CBCT images on the target using the target's location within the patient. This can be done without any user intervention. Similarly, application 81 can initiate CBCT imaging at points during any of the methods described with respect to methods 200-400 without user interaction. For example, in conjunction with the descriptions... Figure 5 In method 500, when the bronchoscope 50, catheter 96, or any other tool containing sensor 94 is navigated to the target and detected within a predetermined distance from the target at step 502, application 81 signals the radiographic imaging device 20 to initiate the CBCT image acquisition process at step 504. Alerts can be provided to clinicians and surgical personnel, allowing them to move away from the patient and limiting their exposure to radiation emitted by the radiographic imaging device 20. These alerts can be audible or visual via a user interface.

[0062] At step 506, the CBCT image is acquired by the radiographic imaging apparatus 20 and received by the application 81. This CBCT image can be used as described specifically with respect to CBCT imaging in method 400. Alternatively, the use of CBCT imaging can be examined and considered entirely independently of methods 200-400, and simply as another visual tool for clinicians to confirm placement, location, and other clinical observations.

[0063] Another aspect of this disclosure relates to respiratory detection to aid CBCT imaging. Regarding Figure 6 Method 600 is described. Using the output from reference sensor 74, at step 602, tracking system 70 and application 81 having said tracking system can detect the stage of the patient's breathing. At step 604, this stage can be monitored throughout the procedure. At step 606, whenever a request for CBCT images is received (directly or through an automated process), application 81 can determine the patient's breathing stage. At this point, the application has various options depending on which method 200-400 is performed by system 100.

[0064] If the CBCT image is the first CT image acquired for surgery, then at step 608, application 81 can instruct the radiographic imaging apparatus to acquire images only when the reference sensor is within the tolerance of the desired portion of the respiratory phase (e.g., near the end of the expiratory phase or near the end of the inspiratory phase). For example, near the end of the inspiratory phase can allow the airway to be in a dilated state, resulting in potentially clearer images, which can generate a more accurate 3D model due to the airway contrast produced by the airway dilation. Alternatively, a longer duration of respiratory cycles can exist near the end of the expiratory phase, where the lungs have essentially not moved, thereby allowing more images to be captured and enhancing the stability of the images acquired in the CBCT image.

[0065] At step 610, application 81 signals radiographic imaging device 20 to begin imaging. When application 81 determines at step 612 that the desired portion of the respiratory phase is about to end, the application signals radiographic imaging device 20 at step 614 to stop imaging the patient. At step 616, the application can determine whether CBCT imaging is complete. If not, the method continues to step 618, where application 81 determines the approaching desired respiratory phase by monitoring the positioning of reference sensor 74, and the method returns to step 610, where radiographic imaging device 20 acquires images again during the desired portion of the respiratory phase. If CBCT imaging is complete at step 616, application 81 stops radiographic imaging device 20 at step 6192 and proceeds to the subsequent steps in methods 200-400.

[0066] In cases where registration is desired between preoperative CT images or previously acquired CBCT images, application 81 can signal at step 608 to the radiographic imaging apparatus 20 to acquire images only during those portions of the respiratory cycle that most closely match the respiratory cycle of the previously acquired CT or CBCT images. By matching the respiratory cycles as closely as possible, the two image datasets will be more similar to each other, making registration between the two image datasets easier and transferring features such as targets or paths from the first CT image to the second CT image. For example, in cases where registration with preoperative CT images is desired, the radiographic imaging apparatus 20 can be guided by application 81 to acquire CBCT images only during the portion of the respiratory cycle that is close to the maximal inspiration of normal tidal breathing positioning. When two CBCT images are to be acquired, application 81 can store the respiratory phase of the first CBCT image in memory and instruct the radiographic imaging apparatus 20 to acquire images at the same respiratory phase at step 608.

[0067] As will be understood, limiting imaging to a specific portion of the respiratory phase can increase the time required to acquire CBCT images and may require several respiratory cycles to complete. This can minimize the time required to acquire CBCT images, but results in greater fidelity of the captured images because the lungs are always imaged in approximately the same location during the respiratory cycle. Furthermore, by monitoring the reference sensor 74, the application 81, which monitors the location of the reference sensor 74 throughout the respiratory cycle, can detect rapid movement of the sensor 74 if the patient coughs or moves on the operating table 40 during the imaging process. If such movement is detected at step 622 during imaging by the radiographic imaging apparatus 20, then at step 624, the application 81 can reject the most recently acquired portion of the CBCT image. Then at step 626, the application 81 can instruct the radiographic imaging apparatus 20 to reposition itself to reacquire the portion of the CBCT image corresponding to the image rejected in a subsequent respiratory phase, and the method proceeds back to step 610.

[0068] Now go to Figure 7 A simplified block diagram of a computing device 80 is shown. The computing device 80 may include a memory 702, a processor 704, a display 706, a network interface 708, an input device 710, and / or an output module 712. The memory 702 may store an application 81 and / or image data 514. When executed by the processor 704, the application 81 may cause the display 706 to display a user interface 716. The application 81 may also provide an interface between the sensed location of the EM sensor 94 and the image and planning data developed during the path planning phase as described above.

[0069] Memory 702 may include any non-transitory computer-readable storage medium for storing data and / or software that can be executed by processor 704 and control the operation of computing device 80. In embodiments, memory 507 may include one or more solid-state storage devices, such as flash memory chips. Alternatively, or in addition to the one or more solid-state storage devices, memory 702 may include one or more mass storage devices connected to processor 704 via a mass storage controller (not shown) and a communication bus (not shown). Although the description of computer-readable medium contained herein refers to solid-state storage devices, those skilled in the art will understand that computer-readable storage medium can be any available medium accessible by processor 704. That is, computer-readable storage medium includes non-transitory, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technologies, CD-ROM, DVD, Blu-ray or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by computing device 80.

[0070] Network interface 708 can be configured to connect to a network, such as a local area network (LAN), wide area network (WAN), wireless mobile network, Bluetooth network, and / or the Internet, consisting of wired and / or wireless networks. Input device 710 can be any device by which a user interacts with computing device 80, such as a mouse, keyboard, foot pedal, touchscreen, and / or voice interface. Output module 712 can include any connection port or bus, such as a parallel port, serial port, universal serial bus (USB), or any other similar connection port known to those skilled in the art.

[0071] Figure 8 A method 800 that does not require preoperative CT scan data is described. Figure 8In this method 800, the steps 800 are as follows: setting up system 100, placing the patient on operating table 40, and initially navigating catheter guide assembly 90, alone or in conjunction with bronchoscope 50, to a location within the lung. The location within the lung may be a target lobe or other anatomical point. As an example, the location may be the third bifurcation in the desired lobe of the lung. Once in this location, method 800 begins at step 802 by capturing a CBCT scan using radiographic imaging device 20. Computing device 80 receives the CBCT scan at step 804. For example, application 81 may retrieve the CBCT scan from a database where the scan is stored after capture, or the user may instruct radiographic imaging device 20 to output the CBCT scan directly to computing device 80. At step 806, the distal end of catheter 96 and the target (e.g., a lesion or other location for treatment) are identified in one or more images of the CBCT scan. This identification may be manual, where the user marks the distal end of catheter 96 and the target in one or more images of the CBCT scan. These images may be displayed in a user interface on computing device 80. Alternatively, application 81 can be configured to perform image analysis and automatically identify the distal portion of catheter 96 and the target. If either or both of the distal portion of the catheter or the target cannot be identified in the CBCT images from the scan, the process can return to step 802 for another CBCT scan. This may require repositioning the patient, radiographic imaging device 20, or catheter 96.

[0072] Following identification, at step 808, the computing device 80 can register the CBCT scan data using the electromagnetic field generated by the tracking system 70. Registration can be performed in various ways. If, for example, the coordinate system of the radiographic imaging device 20 is perpendicular to the operating table 40, then all that is needed is to translate the CBCT coordinates to match the EM coordinates of the tracking system (e.g., the EM coordinates of the field generated by the EM field generator 76). Alternatively, registration can be achieved using pose estimation techniques.

[0073] To determine the orientation of each slice that makes up a CBCT scan, reference markers formed in or on an EM field generator 76 placed beneath the patient are analyzed. These markers may be uniformly spaced or spaced in a known pattern. Regardless of the spacing, the orientation and placement of the markers are known, and the spacing and positioning of the markers in any CBCT slice can be analyzed to determine the angle of the device relative to the radiographic imaging apparatus 20 relative to the EM field generator 76. A mathematical transformation is performed from the coordinate system of the CBCT scan data to the coordinate system of the tracking system 70 (e.g., EM coordinates), utilizing both the known positioning of the markers and the positioning of the distal portion of the marked catheter 96 and the positioning of the detected catheter, such as that identified by the tracking system 70.

[0074] Once registration is complete, at step 810, a 3D model of the patient's lungs can be generated from a CBCT scan, similar to the process described above using preoperative CT images. At step 312, a path is generated through the 3D model from the marker location of the distal portion of catheter 96 to the marker location of the target. This path can be created manually, semi-automatically, or automatically by the user, just as it might be in the 3D model from the preoperative CT scan. Navigation to the target can now proceed. If the user wishes to perform another CBCT scan at any time during navigation, a decision can be made at step 814, and the process can return to step 302. For example, multiple CBCT scans may be desired when performing a microwave or RF ablation procedure within the lung to ensure accurate placement of the ablation catheter in the desired location within the target. Once navigated to the appropriate location, the user or robot can remove LG 92 to allow placement of the ablation catheter or other tools (e.g., biopsy tools) for the procedure to be performed at step 816.

[0075] Figure 9Method 900, which does not require preoperative CT scan data, is described. Similar to method 800, method 900 follows these steps: setting up system 100 and placing the patient on operating table 40. Instead of initiating navigation as described in method 800, a CBCT scan is performed on the patient at step 902. Computing device 80 receives the CBCT scan at step 904. For example, application 81 can retrieve the CBCT scan from a database where the scan is stored after capture, or the user can instruct radiographic imaging device 20 to output the CBCT scan directly to computing device 80. At step 906, a 3D model of the airway is generated from the CBCT scan. At step 908, the 3D model or slice images from the CBCT images are analyzed to identify targets (e.g., lesions) and a path through the airway to the target is generated in the same manner as that which can be accomplished using preoperative CT images. After target identification and path planning, airway sweeping can be performed at step 910. As described above, during airway sweeping, sensors 94 of catheter 96 are inserted into the airway and a point cloud of positioning data is generated. At step 912, the point cloud of the data is matched with the internal features of the 3D model, and the coordinate system of the radiographic imaging device 20 is registered with the electromagnetic field coordinate system of the EM field output by the EM field generator 76. At step 914, once the navigation of the catheter 96 is registered, it can be performed manually or robotically. Once the target is approached, a second CBCT scan can be performed at step 916. Examining the slice images from the CBCT scan, at step 918, the location of the distal end of the catheter 96 and the target can be marked. Based on the marked location of the distal portion of the catheter 96, and at step 920, the target and offset can be calculated. Since the target's location is unlikely to move significantly during surgery, this offset is essentially an indication of the positioning error detected in the EM field by the sensor 94 at the distal end of the catheter 96. By calculating this offset, at step 922, the displayed location of the distal portion of the catheter 96 in the 3D model can be updated to accurately depict the relative location of the catheter and the target in the 3D model, as well as their relative location in other views provided by the user interface of the application 81 described above. The combination of steps 920 and 922 is a local registration of the CBCT and EM field coordinate systems, and again provides the higher accuracy that may be expected when performing procedures (such as microwave ablation or diagnostic procedures, such as biopsy of lesions) at step 924. If further catheter movement is desired, further navigation can be performed at step 926, and the method can return to step 916 to update the local registration.

[0076] Figure 10Further methods according to this disclosure are provided. In method 1000, at step 1002, preoperative planning is performed using a preoperative CT scan (as described above). Sometime after preoperative planning, system 100 is initialized, which may require placing the patient on operating table 40 and initializing tracking system 70, as well as the other steps described above. Once the patient is in place, at step 1004, a CBCT scan of a relevant portion of the patient (e.g., the lungs) is acquired using radiographic imaging device 20. At step 1006, application 81, operating on computing device 80, receives the CBCT scan. For example, application 81 may retrieve the CBCT scan from a database where CBCT scans are stored after capture, or the user may instruct radiographic imaging device 20 to output the CBCT scan directly to computing device 80. At step 1008, the CBCT scan is registered with the preoperative scan. Various methods can be used for this registration, such as image or 3D model matching, to substantially match the preoperative CT scan with the CBCT scan. This registration allows the planned path and target to be transferred from the preoperative plan generated by the preoperative CT scan to the CBCT scan. As a result of this registration, the user interface on the computing device 80 can display the path to the target through the 3D model and other views generated by the CBCT scan, which is now also presented in the CBCT scan image. At step 1010, airway sweeping can be performed. As described above, during airway sweeping, the sensor 94 of the catheter 96 is inserted into the airway and a point cloud of positioning data is generated. At step 1012, the point cloud of data is matched with the internal features of the 3D model generated by the CBCT scan, and the coordinate system of the radiographic imaging device 20 is registered with the electromagnetic field coordinate system of the EM field output by the EM field generator 76 (or other tracking system 70 described herein). At step 1014, once the navigation of the catheter 96 is registered, it can be performed manually or robotically. Once the target is approached, a second CBCT scan can be performed at step 1016. Examining the sliced ​​images from the CBCT scan, at step 1018, the location of the distal end of catheter 96 and the target can be marked. Based on the marked location of the distal portion of catheter 96, and at step 1020, the target and offset can be calculated. This offset is used to update the location of the detected sensor 94 relative to the target in the EM field. By calculating this offset, at step 1022, the location of the displayed distal portion of catheter 96 in the 3D model generated from the CBCT scan can be updated to accurately depict the relative location of the catheter and target in the 3D model, as well as other views provided by the user interface of application 81 described above. The combination of steps 1020 and 1022 is a local registration of the CBCT and EM field coordinate systems, and again provides the higher accuracy that may be desired when performing surgery (such as microwave ablation or diagnostic treatment such as biopsy of lesions) at step 1024.If further movement of catheter 96 is desired, further navigation can be performed at step 1026, and the method can return to step 1016 to update the local registration.

[0077] Although several aspects of this disclosure have been shown in the accompanying drawings, this disclosure is not intended to be limited thereto, as the scope of this disclosure is intended to be as broad as the field would allow, and the specification should be read in the same manner. Therefore, the above description should not be construed as limiting, but rather as an example of certain aspects only.

Claims

1. An apparatus for registering an image with a cavity network, the apparatus comprising: The memory is configured to store computer-readable instructions, and The processor, coupled to the memory and configured to execute the computer-readable instructions, The computer-readable instructions, when executed by the processor, cause the processor to perform the following steps: Detect the positioning of the sensor in the cavity network; The sensors within the cavity network receive cone-beam computed tomography (CBCT) images of the cavity network. Present CBCT images on the user interface; Receive indication of the target's location in the CBCT image; A 3D model of the cavity network is generated from the CBCT image; Generate a path through the cavity network from the detected location by the sensor to the target in the CBCT image and 3D model; as well as Determine whether the sensor has moved from the detected location after receiving the CBCT image, wherein the CBCT image and the 3D model are registered when it is determined that the sensor's location is the same as the detected location.

2. The device according to claim 1, wherein when it is determined that the position of the sensor has changed, measurement data is received.

3. The device of claim 2, further comprising registering the cavity network with the CBCT image based on the measurement data.

4. The device of claim 1, further comprising displaying the CBCT image, a 2D slice image derived from the CBCT image, a 3D model derived from the CBCT image, or the path in a virtual bronchoscopy.

5. The device of claim 1, further comprising displaying the positioning of the sensor along the path in a user interface.

6. An apparatus for registering an image with a cavity network, the apparatus comprising: The memory is configured to store computer-readable instructions, and The processor, coupled to the memory and configured to execute the computer-readable instructions, The computer-readable instructions, when executed by the processor, cause the processor to perform the following steps: Receive preoperative computed tomography (CT) images of the cavity network; Receive indication of a target within the cavity network in the CT image; Generate a path through the cavity network to reach the target; Receive cone-beam computed tomography (CBCT) images from the cavity network; Detect the positioning of the sensor in the cavity network; A 3D model of the cavity network is generated from the CBCT image; The coordinates of the preoperative CT image are transformed into the coordinates of the CBCT image to register the preoperative CT image with the CBCT image; Features from CT images are matched with features from CBCT images and features from 3D models derived from CBCT images; as well as Displays the path from the sensor-detected location to the target in the CBCT image or 3D model.

7. The device of claim 6, further comprising displaying the CBCT image, a 2D slice image derived from the CBCT image, a 3D model derived from the CBCT image, or a virtual bronchoscopy on a user interface.

8. The device of claim 6, further comprising generating a 3D model from the CBCT image before transforming the coordinates of the preoperative CT image.

9. The device of claim 6, further comprising generating a 3D model from the CBCT image after transferring the target and the path from the preoperative CT image to the CBCT image.

10. The device of claim 6, further comprising receiving measurement data, wherein the measurement data is received prior to receiving the CBCT image, or the measurement data is received after transferring the target and the path from the preoperative CT image to the CBCT image to register the CBCT image with the cavity network.

Images