Electromagnetic navigation and multi-rigid registration of computed tomography volumes

Abstract

An apparatus, system, method and computer-readable medium for electromagnetically registering a luminal network with a 3D model of the luminal network include: accessing a 3D model of the luminal network based on a computed tomography (CT) image of the luminal network; selecting multiple reference points within the 3D model of the luminal network; obtaining multiple survey points within the luminal network; dividing the 3D model of the luminal network and the luminal network into multiple regions, assigning multiple weights to the multiple regions, determining an alignment of the multiple reference points with the multiple survey points based on the multiple weights, and generating a registration based on the alignment, wherein the registration enables the multiple survey points within the luminal network to be converted into points within the 3D model of the luminal network.

Description

Technical Field

[0001] The present invention relates to bronchial registration, and more particularly to devices, systems, methods and computer-readable media for automatically registering a model of a patient's airway generated by three-dimensional (3D) electromagnetic navigation with a model generated by a 3D computed tomography (CT) scan of the patient's airway based on a weighted metric for each region of the patient's airway. Background Art

[0002] A common device used to examine a patient's airway is a bronchoscope. Typically, a bronchoscope is inserted into the patient's airway through the patient's nose or mouth and can extend into the patient's lungs. A typical bronchoscope includes an elongated flexible tube having an illumination assembly for illuminating an area distal to the bronchoscope tip, an imaging assembly for providing a video image from the bronchoscope tip, and a working channel through which an instrument (e.g., a diagnostic instrument such as a biopsy instrument, a therapeutic instrument) can be inserted.

[0003] However, the distance a bronchoscope can be advanced through the airway is limited by its size. In situations where a bronchoscope is too large to reach a target location deep in the lung, clinicians can utilize certain real-time imaging modalities, such as computed tomography or fluoroscopy. CT or fluoroscopic images, while useful, have certain deficiencies for navigation, as it is often difficult to distinguish luminal passages from solid tissue. Additionally, the images generated are two-dimensional, and navigating a patient's airway requires the ability to maneuver in three dimensions.

[0004] To address these problems, systems have been developed that are capable of developing a three-dimensional model of an airway or other luminal network, typically based on a series of computed tomography (CT) images. One such system has been developed as the 3D-MODEL® currently sold by Medtronic PLC. ELECTROMAGNETIC NAVIGATION (ENB TM ) system. Details of such a system are described in commonly assigned U.S. Pat. No. 7,213,820, filed by Gilboa on Mar. 29, 2004, entitled "ENDOSCOPE STRUCTURES AND TECHNIQUES FOR NAVIGATING TO A TARGET IN BRANCHED STRUCTURE," the entire contents of which are incorporated herein by reference.

[0005] During the ENB process, it is important to register the magnetic navigation space with the patient's CT scan. As understood in the art, "registration" refers to the conversion, mapping, transformation, etc. of a position or coordinate in one space into a position or coordinate in another space. In order to perform such registration, several methods have been developed. A method called manual registration is based on visually identifying the main airway branch points with a bronchoscope and associating them with the points visually identified on the CT scan. During the ENB process, the user navigates the bronchoscope to the branch point in the airway using a catheter equipped with a magnetic induction coil, and manually generates an association between the branch point and the position in the magnetic navigation space. Another method, called automatic registration, is based on collecting a cluster of points (called a survey) from the airway using a catheter with a magnetic sensor at its tip, and then using a segmentation algorithm to fit the survey to an image of the airway tree derived from a CT scan, and is further described in commonly owned U.S. Patent Publication No. 2011 / 0085720, entitled “AUTOMATIC REGISTRATION TECHNIQUE,” filed by Averbuch et al. on May 14, 2010, the entire contents of which are incorporated herein by reference.

[0006] While the systems described in US Pat. No. 7,213,820 and US Publication No. 2011 / 0085720 are perfectly viable, there is always a need for improvements and additions to such systems. Summary of the invention

[0007] According to the present disclosure, apparatus, systems and methods are provided for generating multiple registrations of electromagnetic navigation with a CT scan volume model of a patient and dynamically determining which of the multiple registrations to use.

[0008] In one aspect of the present disclosure, the present disclosure includes a method for registering electromagnetic navigation of a luminal network with a 3D model of the luminal network. The method includes: accessing a 3D model of the luminal network, wherein the 3D model is based on a computed tomography (CT) image of the luminal network, selecting a reference point within the 3D model of the luminal network, and obtaining a survey point within the luminal network; dividing the 3D model of the luminal network and the luminal network into a plurality of regions, assigning weights to the regions, determining an alignment of the reference point with the survey point based on the weights, and generating a registration based on the alignment, wherein the registration enables a position within the luminal network to be converted into a position within the 3D model of the luminal network.

[0009] In various embodiments, the luminal network is an airway of a patient.In various embodiments, the 3D model is a model of the airway of a patient.

[0010] In various embodiments, the method further includes determining from which region each of the plurality of survey points was obtained, and assigning a corresponding weight to each of the survey points based on the region from which each of the survey points was obtained.

[0011] In various embodiments, determining the alignment of the reference point with the survey point includes minimizing a divergence between the reference point and the survey point based on a respective weight assigned to each of the survey points.

[0012] In various embodiments, for each specific area in the area, the method includes: assigning a specific weight to the area, wherein the specific weight favors the specific area, minimizing the divergence between the reference point and the survey point based on the specific weight, wherein the minimized divergence corresponds to a specific alignment, and generating a specific registration based on the specific alignment, wherein the specific registration is associated with the specific area and enables the position within the lumen network to be converted into a position within the 3D model of the lumen network.

[0013] In various embodiments, the region comprises a transition region. In various embodiments, the region comprises a right lung region, a central region, and a left lung region, and the transition region comprises a left central region and a right central region.

[0014] According to another aspect, the present disclosure includes a system for updating a 3D model of a luminal network. The system includes a position sensor capable of navigating in a luminal network in a patient, an electromagnetic field generator configured to detect the position of the position sensor when navigating in the luminal network, and a computing device including a processor and a memory storing instructions. When the instructions are executed by the processor, it causes the computing device to: access a 3D model of the luminal network based on a computed tomography (CT) image of the luminal network, select a reference point within the 3D model of the luminal network, obtain a survey point within the luminal network, wherein the survey point is generated by the position sensor and the electromagnetic field generator, divide the 3D model of the luminal network and the luminal network into a plurality of regions, assign weights to the plurality of regions, determine an alignment of the reference point with the survey point based on the weights, and generate a registration based on the alignment, wherein the registration enables the survey point within the luminal network to be converted into a point within the 3D model of the luminal network.

[0015] In various embodiments, the instructions, when executed by the processor, further cause the computing device to determine from which region of the region each of the measurement points is obtained based on the detected position of the position sensor, and assign a corresponding weight to each of the measurement points based on the region from which each of the measurement points is obtained. The instructions, when executed by the processor, further cause the computing device to minimize the divergence between the reference point and the survey point based on the corresponding weight assigned to each of the plurality of survey points.

[0016] In various embodiments, the instructions, when executed by the processor, further cause the computing device to, for each specific region among the multiple regions: assign a specific weight to the region, wherein the specific weight favors the specific region, minimize a divergence between a reference point and a survey point based on the specific weight, wherein the minimized divergence corresponds to a specific alignment, and generate a specific registration based on the specific alignment, wherein the specific registration is associated with the specific region and enables a position within the lumen network to be converted into a position within a 3D model of the lumen network.

[0017] In various embodiments, the region comprises a transition region. In various embodiments, the region comprises a right lung region, a central region, and a left lung region, and the transition region comprises a left central region and a right central region.

[0018] In various embodiments, the instructions, when executed by the processor, also cause the computing device to determine that the position of the position sensor is within a transition region, determine which of these regions the position sensor transitions between, and interpolate the position in a 3D model of the luminal network based on the alignment associated with the regions between which the position sensor transitions.

[0019] Any of the above-described aspects and embodiments of the present disclosure may be combined without departing from the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The objects and features of the systems and methods of the present disclosure will become apparent to those of ordinary skill in the art when reading the description of the various embodiments of the present invention with reference to the accompanying drawings, in which:

[0021] Figure 1 is a 3D body model view of a lumen network and divided into multiple regions provided according to an embodiment of the present disclosure;

[0022] Figures 2A-2D is an illustration of a 3D volume model covering survey points provided according to an embodiment of the present disclosure;

[0023] Figure 3A and 3B is a flow chart of a method for registering electromagnetic navigation within a luminal network with a 3D model of the luminal network according to an embodiment of the present disclosure; and

[0024] Figure 4 is a perspective view of an electromagnetic navigation system and a schematic diagram of a workstation provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The present disclosure relates to devices, systems, methods and computer-readable media that align a 3D bronchial tree model (hereinafter referred to as a "3D model") with a patient's airway based on electromagnetic navigation. "Registration" refers to the conversion, mapping, transformation, etc. of a position or coordinate in one space into a position or coordinate in another space. The registration disclosed in the co-owned U.S. Patent Publication No. 2011 / 0085720 entitled "Automatic Registration" is referred to as rigid. In this article, the technology of the present disclosure is referred to as "multi-rigid" registration and "optimized regional registration". As further described herein, the present disclosure provides increased registration accuracy by taking into account the flexibility of the lungs and focusing on one area of ​​the lungs at a time. Using the technology of the present disclosure, multiple registrations can be generated, each corresponding to a different area of ​​the lungs and airways.

[0026] Due to the flexibility of the lung, the actual shape of the lung during surgery may be distorted or different from the shape at the time of the CT scan and / or initial registration, resulting in reduced navigation accuracy. This deformation due to the flexibility of the lung may be due to several differences, such as: (1) the CT scan is performed while the patient holds his breath after a full inhalation, while during the shell surgery, including during registration and navigation, the patient remains sedated and breathes normally; (2) during the surgery, the patient may lie flat for a longer time, resulting in increased deformation; (3) during the surgery, the head, neck, and upper chest may also be deformed while the bronchoscope is in the patient's body; and / or (4) CT scans are often performed on a concave bed, while surgical procedures are usually performed with the patient lying flat. Therefore, the present disclosure includes devices, systems, methods and computer-readable media for performing magnetic navigation and CT scan volume model registration (referred to as antenna-to-CT registration), particularly when magnetic navigation is at or around the navigation target, using a weighted metric to improve the accuracy of the 3D model of the airway to account for the increased deformation of the airway during bronchoscopy. Furthermore, the present disclosure relates to registering multiple rigid registrations based on electromagnetic navigation, and dynamically determining which of the multiple rigid registrations to use during navigation.

[0027] Figure 1 is an illustration of a 3D model 100 of a patient's airway generated based on a computed tomography (CT) scan. Figure 1 As shown, the 3D model 100 includes an airway tree 102, a target 104, and airway branches 108 originating from "G1"-"G3" level bifurcations in the airway tree 102. The 3D model 100 is divided into a plurality of regions. A "G1"-"G3" level bifurcation is a location within the 3D model 100 where one airway branch 108 splits into two or more smaller airways. Although the 3D model 100 only shows airway branches 108 up to the third level of bifurcation, it should be understood that in practice, the 3D model 100 may have up to 23 levels of bifurcations.

[0028] like Figure 1 As shown, the central region 115 includes the trachea and surrounding airways, the left lung region 120 includes the airways of the left lung, and the right lung region 130 includes the airways of the right lung. In addition, the 3D model 100 includes two transition regions: the right central region 135 and the left central region 125. It is also contemplated that, although these five specific regions are listed, additional regions and transition regions may be included, if desired, in order to increase the number of multiple registrations and, therefore, improve the accuracy of the 3D model 100.

[0029] In a multiple rigid registration, each of the multiple regions is associated with a separate registration. For example, in one embodiment, the central region 115, the left lung region 120, and the right lung region 130 each have a separate registration ( Figures 2A-2C In other embodiments, each registration may be merged into a single registration ( Figure 2D ).

[0030] The right central region 135 is the airway volume where navigation is required to transfer between the right lung region 130 and the central region 115. The left central region 125 is the airway volume where navigation is required to transfer between the left lung region 125 and the central region 115. Figure 3A and 3B As further described in the detailed description of , it is contemplated that within the right center region 135 and the left center region 125, when the EM sensor 94 and / or surgical instrument 402 transitions between the two regions, the registration of adjacent regions is used to interpolate the position of the EM sensor 94 and / or surgical instrument 402. The number of regions and the location and arrangement of the regions are exemplary, and other numbers, locations, and arrangements of the regions are contemplated to be within the scope of the present disclosure. For example, a particular arrangement of regions may include regions for target lobe regions, regions for other lobes, and transition regions of the lobes. Other arrangements are also contemplated.

[0031] Additionally, included within the 3D model 100 are a plurality of reference points 110 that are selected within the airway tree 102 and that create a reference airway tree 122. The plurality of reference points 110 serve as the EM sensors 94 ( Figure 4 ) and / or surgical instrument 402 ( Figure 4 ), the EM sensor 94 and / or surgical instrument 402 is used to generate a plurality of survey points 210 (e.g., Figures 2A-2D In various embodiments, each of the plurality of reference points 110 includes a plurality of reference points recorded and stored in the memory 403 ( Figure 4) in the x, y, and z coordinates. Although a Cartesian coordinate system is used in the present disclosure for simplicity, it is contemplated that a different coordinate system may be used, such as a spherical coordinate system, a cylindrical coordinate system, or other coordinate systems. Such other coordinate systems are contemplated within the scope of the present disclosure.

[0032] Reference now Figures 2A-2D , four differently registered views of a patient's airway are shown, 100a, 100b, 100c, and 100d, each view including a central region 115, a left lung region 120, and a right lung region 130, and a plurality of survey points 210 obtained from the positions of the EM sensor 94 and / or the surgical instrument 402, the plurality of survey points 210 creating a survey airway tree 222. Also refer to Figure 4 During electromagnetic navigation, the EM sensor 94 and / or surgical instrument 402 together with the tracking system 70 enable tracking of the EM sensor 94 and / or surgical instrument 402 as it is advanced through the patient's airway. The EM sensor 94 may be incorporated into the surgical instrument 402 or a different type of instrument, such as a biopsy instrument, a microwave ablation instrument, a surgical stapling instrument, a radiotherapy ink application instrument, etc., and enable determination of the current position of the instrument. As the EM sensor 94 and / or surgical instrument 402 is advanced through the patient's airway, the x, y, and z coordinates of the EM sensor 94 and / or surgical instrument 402 are obtained at predetermined distance intervals and stored in the memory 403. In closer proximity to one or more targets 104 ( Figure 1 ), the predetermined distance interval can be shortened, thereby increasing the number of the obtained multiple survey points 210. All multiple survey points 210 are obtained along the electromagnetic navigation and can be overlaid on the 3D model 100. For the rigid registration to be performed, the multiple survey points 210 are used in combination with the multiple reference points 110.

[0033] During rigid registration, the goal is to find an alignment between a survey airway tree 222 created by the locations of multiple survey points 210 and a reference airway tree 122 created by multiple reference points 110, thereby achieving a minimum "divergence" between local airways, as explained below. Rigid registration essentially corrects for the overall displacement between the reference airway tree 122 and the survey airway tree 222. In rigid registration, the minimum distance of the expected transformation from each survey point 210 to the reference airway tree 122 is determined. In the case of rigid registration, the "divergence" can be the average of the minimum distances. (Divergence in the case of multiple rigid registrations will be described below.) In rigid registration, the divergence metrics of the expected transformations are compared, and the transformation corresponding to the minimum divergence is selected as the transformation for rigid registration. That is, the transformation enables the conversion of a position in the patient's airway into a position or coordinate in a 3D model based on a CT scan. By applying the automatic registration techniques described in co-owned U.S. Patent Publication No. 2011 / 0085720, and using multiple reference points 110 and multiple survey points, the reference airway tree 122 can become associated with the survey airway tree 222 so that the EM sensor coordinates can be converted into reference airway tree coordinates (also referred to herein as CT coordinates). For example, a rigid transformation can provide a slight rotation of the reference airway tree 122 so that the reference airway tree 122 is more closely aligned with the survey airway tree 222 after the rotation. After rigid registration, the x, y and / or z coordinates of the multiple reference points 110 will be effectively mapped to the multiple survey points 210 without changing the 3D model from the CT scan. The specific manner of determining divergence disclosed above is exemplary. In general, divergence as used herein refers to a measure that quantifies the degree to which the survey airway tree is aligned or misaligned with the reference airway tree. Other manners of determining divergence should be considered within the scope of the present disclosure.

[0034] After the rigid registration, a multi-rigid registration is performed. As used herein, a multi-rigid registration refers to a collection of optimized regional registrations. As described below, optimized regional registration seeks to align a specific region of the reference airway tree with a corresponding region of the survey airway tree as closely as possible, while to a lesser extent considering the overall alignment of other regions of the airway tree. An optimized regional registration is performed for each region of the airway tree, and this entire process is referred to as a multi-rigid registration.

[0035] According to aspects of the present disclosure, during multiple rigid registration and optimized regional registration, as further described herein, it is expected that each of the multiple regions will have an assigned weight that is used to determine the amount by which each region of the reference airway tree 122 can be aligned with a corresponding region of the survey airway tree 222. The weight assigned to each of the multiple regions is also applied to the multiple survey points 210 located within the region, as described below. Typically, during rigid registration, the alignment between the reference airway tree 122 and the survey airway tree 222 minimizes the above-mentioned divergence. In contrast, during multiple rigid registration / optimized regional registration, different regions of the airway tree are weighted differently, so that regions with larger weights are favored in the process of minimizing divergence. For example, assume that the survey airway tree has two regions A and B, and the regional weight of region A is 0.9 and the regional weight of region B is 0.1. The weight of 0.9 is evenly distributed across all survey points in region A, and the weight of 0.1 is evenly distributed across all survey points in region B. For example, if region A has three (3) survey points, each survey point in region A is assigned a weight of 0.9 / 3 = 0.3, and if region B has two (2) survey points, each survey point in region B is assigned a weight of 0.1 / 2 = 0.05. In the calculation of the divergence, the minimum distance of the expected transformation from each survey point to the reference airway tree is multiplied by the weight assigned to the corresponding survey point, and the weighted minimum distances are averaged. Therefore, in the case of multi-rigid registration / optimized regional registration, the divergence is essentially a weighted average of the minimum distances, where survey points with larger weights will have a larger impact on the divergence. The divergence metrics of the expected transformations are compared, and the transformation corresponding to the minimum divergence is selected as the transformation to be used. As described above, this process for a particular region is called optimized regional registration. And as shown below in combination Figures 2A-2C As described, the result of the optimized regional registration is a specific registration that closely aligns the survey airway tree in a specific region with the reference airway tree. Therefore, when there are multiple regions, the optimized regional registration is performed on each region, and several separate registrations are obtained. The entire process of performing optimized regional registration on all regions is referred to herein as multi-rigid registration. The specific manner of determining the divergence described above is exemplary, and other manners of determining the divergence may be considered within the scope of the present disclosure.

[0036] Figure 2A 100a shows the registration of the patient's airway EM sensor readings with the 3D model of the airway after performing an optimized regional registration on the more heavily weighted right lung region 130. Included within the 3D model 100 are a central region 115, a left lung region 120, and a right lung region 130. Additionally, the 3D model 100 is shown along with a plurality of survey points 210. Figure 2A, the right lung region 130 is assigned a greater weight than both the central region 115 and the left lung region 120, and based on the greater weight assigned to the right lung region 130, the airways of the right lung region 130 of the 3D model 100 are more closely aligned with the survey points 210 within the right lung region 130 than with the survey points 210 within the central lung region 115 and the left lung region 120. Therefore, when performing optimized regional registration when the right lung region 130 is more heavily weighted, the alignment of the plurality of reference points 110 with the 3D model 100 having the plurality of survey points 210 minimizes the alignment deviation within the right lung region 130, while taking into account the alignment deviation of the left lung region 120 and the central lung region 115 to a lesser extent.

[0037] and Figure 2A In comparison, Figure 2B , the left lung region 120 is depicted as being assigned a greater weight than both the right lung region 130 and the central region 115, and therefore, the patient's airway in the left lung region 120 of the 3D model 100 is more closely aligned with the survey points 210 within the left lung region 120 than with the survey points 210 within the central region 115 and the right lung region 130. Similarly, in Figure 2C , the central region 115 is assigned a higher weight than the left lung region 120 and the right lung region 130 , and the airways of the central region 115 of the 3D model 100 are more closely aligned with the survey points 210 within the central region 115 than with the survey points 210 within the left lung region 120 and the right lung region 130 .

[0038] It is contemplated that each registration 100a, 100b, and 100c may be used, respectively, during navigation within the right lung region 130, the left lung region 120, and the central region 115. Thus, the corresponding registration 100a, 100b, or 100c may be used when advancing the EM sensor 94 and / or surgical instrument 402 through the patient's airway and determining the position of the EM sensor 94 and / or surgical instrument 402.

[0039] View now Figure 2D , shown is a “merged” registration 100d. Figure 2D Shown as Figure 2C The registration of the central region 115 shown in Figure 2B The registration of the left lung region 120 shown in FIG. Figure 2A Thus, the updated 3D model 100d utilizes Figures 2A-2C The partial registration provides a combined registration of the patient's airway and provides a more accurate registration of the entire area of ​​the airway.

[0040] Figure 3A and 3B403 ( FIG. 4 ) shows a flowchart of a method 300. Flowchart 300 details the use of a weighted metric to align multiple electromagnetic alignments of a luminal network with a 3D model of the luminal network and dynamically determine which of the multiple alignments to use based on the position of the EM sensor 94 and / or the surgical instrument 402 as navigation proceeds. Method 300 may be performed at least in part by executing a program stored in memory 403 ( Figure 4 ) is implemented by the processor 404 of the instructions in the method 300. In addition, the specific order of steps shown in the method 300 is provided as an example and not as a limitation. Therefore, without departing from the scope of the present disclosure, the steps of the method 300 may be performed in an order different from the order shown in FIG. 3. In addition, some of the steps shown in the method 300 may be performed simultaneously with respect to each other, rather than being performed sequentially with respect to each other.

[0041] Reference now Figure 3A , the method 300 begins at step 305, where a 3D model 100 is generated or accessed based on a CT scan of a patient, and a plurality of reference points 110 are selected within the 3D model 100. At step 307, the clinician identifies one or more targets 104 on the 3D model 100, and generates a navigation plan based on the 3D model 100. At steps 309-340, the clinician begins the registration process. At step 309, the clinician loads the navigation plan generated at step 307. Next, at step 310, the registration begins by inserting the EM sensor and bronchoscope into the patient to survey the patient's airway. At step 313, the EM sensor 94, together with the tracking system 70, enables tracking of the EM sensor 94 and / or the surgical instrument 402 as it is advanced through the patient's airway. Next, at step 315, the processor 404 obtains a plurality of survey points 210 within the patient's airway during navigation based on data from the tracking system 70.

[0042] Next, at step 317, after all of the plurality of survey points 210 are obtained, rigid registration is performed between the plurality of reference points 110 and the reference airway tree 122, and between the plurality of survey points 210 and the reference airway tree 222, as described above. Based on the rigid registration, a mapping or transformation between the positions of the plurality of reference points 110 and the plurality of survey points is generated. The rigid registration allows the survey points 210 to be associated with the reference airway tree 122, such that each survey point 210 can be associated with a specific region of the reference airway tree 122 in preparation for multiple rigid registrations. Next, at step 320, after the 3D model 100 is updated, the 3D model 100 is divided into a plurality of regions, such as the central region 115, the left lung region 120, the right lung region 130, the right central region 135, and the left central region 125, as shown in FIG. Figure 1 In step 323, a weight is assigned to each region, as shown in Figures 2A-2D As described in the detailed description of , and in step 325, weights are assigned to the plurality of survey points 210 based on the weights assigned to the regions where each survey point 210 is located. Next, in step 327, optimized regional registration is performed based on the weights assigned to each region, as in Figures 2A-2D At step 329, based on the optimized regional registration, the registration is associated with the specific region. Next, at step 330, based on performing the optimized regional registration with weights that favor different regions each time, a separate registration is generated for each of the plurality of regions, such as registrations 100a, 100b, and 100c, as shown in FIG. Figures 2A-2C The expected registrations 100a, 100b and 100c may be stored in the memory 403 ( Figure 4 )middle.

[0043] Optionally, at step 340, each of the registrations generated at steps 329 and 330 may be merged into a single registration, such as registration 100d. Thus, after the multiple rigid registration process of steps 310-340, a plurality of registrations are generated.

[0044] Reference now Figure 3B , the method 300 continues from step 345 to step 378, where navigation and use of the 3D model 100 are performed during the surgical procedure. In step 345, the 3D model 100 is loaded and presented to the user. In addition, the clinician then loads a navigation plan based on the 3D model loaded in step 345, the navigation plan including one or more targets 104. Thereafter, the clinician loads the navigation plan into the application 81 from the memory 403, the USB device, or the network interface 208. The navigation plan may require all areas of the patient's lungs or only some areas to be registered. After the navigation plan, in step 350, electromagnetic navigation is performed toward one or more targets 104.

[0045] Turning now to step 355, the EM sensor 94, together with the tracking system 70, enables tracking of the EM sensor 94 and / or surgical instrument 402 as it is advanced through the patient's airway and the processor 404 tracks the position of the EM sensor 94 and / or surgical instrument 402. At step 360, the processor 404 and tracking system 70, together with the EM sensor 94 and / or surgical instrument 402, determine in which region of the patient's airway the EM sensor 94 and / or surgical instrument 402 is located based on the region divisions described in step 320. In various embodiments, the processor 404 is configured to assume that the EM sensor 94 is in the same region as it was previously located. Next, at step 365, based on the region of the patient's airway where the EM sensor 94 and / or surgical instrument 402 is located, a corresponding registration ( Figures 2A-2C ) to determine the location of the EM sensor 94 within the 3D model. For example, if the processor 404 determines that the EM sensor 94 is in the right lung region, then select and use Figure 2A The registration 100a is then performed, and the processor 404 uses the registration 100a to determine the position of the EM sensor 94 within the 3D model 100. If the sensor position determined in step 365 is in a different region than the region selected in step 360, step 365 is performed again to determine the sensor position using the registration for the new region.

[0046] Next, at step 367, processor 404 determines whether one or more targets 104 have been reached during navigation. If, at step 367, processor 404 determines that EM sensor 94 and / or surgical instrument 402 have not reached target 104, method 300 proceeds to step 370, where processor 404 determines whether the position of EM sensor 94 and / or surgical instrument 402 transitions between different regions of the patient's airway. For example, if processor 404 determines that the position of EM sensor 94 and / or surgical instrument 402 transitions between different regions of the patient's airway, the method 300 proceeds to step 371, where processor 404 determines whether the position of EM sensor 94 and / or surgical instrument 402 transitions between different regions of the patient's airway. Figure 1 Within the right center area 135 or the left center area 125 shown.

[0047] If, at step 370, the processor 404 determines that the position of the EM sensor 94 and / or surgical instrument 402 is not transitioning between different regions of the patient's airway (i.e., located in the central region 115, the left lung region 120, or the right lung region 130), the method 300 returns to step 355 and continues to track the position of the EM sensor 94 and / or surgical instrument 402. If, at step 370, the processor 404 determines that the EM sensor 94 and / or surgical instrument 402 is transitioning between regions (i.e., located in the right central region 135 and the left central region 125), the method 300 proceeds to step 372, where, similar to step 355, the tracking system 70 tracks the position of the EM sensor 94 and / or surgical instrument 402.

[0048] Next, at step 374, the processor 404 determines in which transition zone the EM sensor 94 and / or surgical instrument 402 is located, thereby determining which zone the EM sensor 94 and / or surgical instrument 402 is transitioning out of and into. At step 376, the processor 404 selects the registrations of the zones that the EM sensor 94 and / or surgical instrument 402 is transitioning out of and into, and at step 378, interpolates the position of the EM sensor using the two selected registrations. In various embodiments, the processor 404 interpolates the position of the EM sensor by calculating the position L1 of the EM sensor using the first registration and also calculating the position L2 of the EM sensor using the second registration. The two calculated positions serve as outer limits for the interpolation, such that the interpolated position of the EM sensor is between the two positions L1 and L2. In various embodiments, to determine the interpolated position, the processor 404 determines the distance d from the first position to the first zone. 11 , the distance d from the first position to the second area 12 , the distance d from the second position to the first area 21 , and the distance d from the second position to the second area 22 In various embodiments, d 11 and d 21 The average value of may be the average distance to the first region, and d 12 and d 22 The average value of can be the average distance to the second area. If the average distance to the first area and the average distance to the second area are approximately the same, the interpolated position of the EM sensor can be approximately halfway between the two positions L1 and L2. Otherwise, the interpolated position of the EM sensor between L1 and L2 can be proportional to the average distance to the first area and the average distance to the second area. For example, if the average distance to the first area is 10 (arbitrary units) and the average distance to the second area is 20, the interpolated position can be 10 / (10+20)=1 / 3 of the path from L1 to L2, or equivalently, 2 / 3 of the path from L2 to L1. The specific manner of determining the interpolated position is exemplary, and other manners of interpolating the position of the EM sensor should be considered within the scope of the present disclosure.

[0049] After step 378, method 300 returns to step 367, where processor 404 determines whether EM sensor 94 and / or surgical instrument 402 have reached target 104. If, at step 367, processor 404 determines that EM sensor 94 and / or surgical instrument 402 have reached target 104, the currently generated 3D model is used and method 300 ends.

[0050] Described above is a method for registering bronchoscope sensor survey points with reference points in a 3D model and using the registration during navigation of an airway lumen network. According to aspects of the present disclosure, the disclosed method 300 can update the registration during navigation of an airway lumen network. In various embodiments, Figure 3B Between steps 360 and 365, processor 404 may perform optimized regional registration using the locations of the EM sensors determined in step 355. The weights used to optimize regional registration may favor the regions where the EM sensors determined in step 360 are located.

[0051] Reference now Figure 4 According to the present disclosure, an electromagnetic navigation system 10 is provided. One such electromagnetic navigation system is the ELECTROMAGNETIC NAVIGATION SYSTEM currently sold by Medtronic PLC. Other tasks that may be performed using the electromagnetic navigation system 10 are: planning a path to a target tissue, navigating a positioning assembly to the target tissue, navigating a biopsy instrument to the target tissue to obtain a tissue sample from the target tissue using the biopsy instrument, digitally marking the location where the tissue sample is obtained, and placing one or more echogenic markers at or around the target.

[0052] The electromagnetic navigation system 10 generally includes an operating table 40 configured to support a patient; a bronchoscope 50 configured to be inserted into the patient's airway through the patient's mouth and / or nose; a monitoring device 60 coupled to the bronchoscope 50 for displaying video images received from the bronchoscope 50; a tracking system 70 including a tracking module 72, a plurality of reference sensors 74, and an electromagnetic (EM) field generator 76; and a workstation 80 including software and / or hardware for facilitating path planning, identifying target tissue, navigating to target tissue, and digitally marking a biopsy location.

[0053] Figure 4Also shown are two types of catheter guide assemblies 90, 100. Both catheter guide assemblies 90, 100 can be used with the electromagnetic navigation system 10 and share a number of common components. Each catheter guide assembly 90, 100 includes a handle 91 connected to an extended working channel (EWC) 96. The size of the EWC 96 is suitable for placement in the working channel of the bronchoscope 50. In operation, a locatable guide (LG) 92 including an EM sensor 94 is inserted into the EWC 96 and locked in place so that the sensor 94 extends beyond the desired distance of the distal end 93 of the EWC 96. Within the electromagnetic field generated by the electromagnetic field generator 76, the position of the EM sensor 94 and therefore the distal end of the EWC 96 can be obtained by the tracking module 72 and the workstation 80. The catheter guide assemblies 90, 100 have different operating mechanisms, but each includes a handle 91 that can manipulate the distal tip 93 of the LG 92 and the EWC 96 by rotation and compression. Catheter guide assembly 90 is currently marketed by Medtronic PLC under the trade name Similarly, catheter guide assembly 100 is currently marketed by Medtronic PLC under the trade name EDGE TM Both sets include Handle 91, EWC 96 and LG 92.

[0054] like Figure 4 As shown, the patient lies on an operating table 40, and a bronchoscope 50 is inserted into the patient's airway through the patient's mouth. The bronchoscope 50 includes an illumination source and a video imaging system (not explicitly shown), and is connected to a monitoring device 60 (e.g., a video display) for displaying video images received from the video imaging system of the bronchoscope 50.

[0055] The catheter guide assembly 90, 400 including the LG 92 and the EWC 96 is configured for insertion into the airway of a patient through the working channel of a bronchoscope 50 (although the catheter guide assembly 90, 400 may be used instead without the bronchoscope 50). The LG 92 and the EWC 96 may be selectively locked relative to each other via a locking mechanism 99. Navigation is performed using, for example, a six-degree-of-freedom electromagnetic tracking system 70 similar to that disclosed in U.S. Pat. No. 6,188,355, and U.S. Patent Publication No. 2014 / 0281961, or any other suitable positioning survey system, each of which is incorporated herein by reference in its entirety, but other configurations are also contemplated. The tracking system 70 is configured for use with the catheter guide assembly 90, 400 to track the position of the EM sensor 94 as it moves through the airway of a patient together with the EWC 96, as described below.

[0056] An electromagnetic field generator 76 is located below the patient. The electromagnetic field generator 76 and a plurality of reference sensors 74 are interconnected with a tracking module 72, which obtains the position of each reference sensor 74 in six degrees of freedom. One or more reference sensors 74 are attached to the patient's chest. The six degrees of freedom coordinates of the reference sensors 74 are sent to a workstation 80 including an application 81, which uses the data collected by the sensors 74 to calculate a patient coordinate reference frame.

[0057] Figure 4 Also shown is a surgical instrument 402, which can be inserted into the catheter guide assembly 90, 400 after navigating to the target and removing the LG 92. The surgical instrument 402 may include, but is not limited to, a tissue collection instrument, a microwave ablation instrument, a surgical stapling instrument, a radiotherapy ink application instrument, etc. The surgical instrument 402 is also configured to be used in conjunction with the tracking system 70 to facilitate navigating the surgical instrument 402 to the target tissue, thereby tracking the position of the surgical instrument 402 when the surgical instrument 402 is manipulated relative to one or more targets 404.

[0058] During process planning, workstation 80 utilizes computed tomography (CT) image data to generate and observe the 3D model of the patient's airway, makes it possible to identify (automatically, semi-automatically or manually) the target tissue on the 3D model, and allows the path to the target tissue through the patient's airway to be selected. More specifically, the CT scan is processed and assembled into a 3D volume, and then the 3D volume is used to generate the 3D model of the patient's airway. The 3D model can be presented on a display monitor associated with the workstation 80, or presented in any other suitable manner. Using the workstation 80, the views of various 3D volume slices and 3D models can be presented and / or manipulated by the clinician to help identify the target and select the appropriate path to enter the target through the patient's airway. The 3D model can also display the mark of the location where the previous biopsy was performed, including data, time, and other identification information related to the tissue sample obtained. These marks can also be selected as the target that can be planned to reach the path. Once selected, the path is saved to use during the navigation process. Examples of suitable path planning systems and methods are described in U.S. patent application publications US 2014 / 0281961, US 2014 / 0270441, and US 2014 / 0282216, all filed by Baker on March 15, 2013, each entitled PATHWAY PLANNING SYSTEM AND METHOD, the entire contents of each of which are incorporated herein by reference.

[0059] System diagram 400 of workstation 80 includes memory 403, processor 404, display 406, network interface 408, input device 410 and / or output module 412. Memory 403 includes any non-transitory computer-readable storage medium for storing data and / or software that can be executed by processor 404 and control the operation of workstation 80. In an embodiment, memory 403 may include one or more solid-state storage devices, such as flash memory chips. Alternatively or in addition to one or more solid-state storage devices, memory 403 may include one or more mass storage devices connected to processor 404 through a mass storage controller (not shown) and a communication bus (not shown). Although the description of computer-readable media contained herein relates to solid-state storage devices, it should be understood by those skilled in the art that computer-readable storage media can be any available media that can be accessed by processor 404. That is, computer-readable storage media include non-transitory, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). For example, computer-readable storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, Blu-Ray or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the workstation 80.

[0060] The memory 403 may store the application 81 and / or the CT data 414. The application 81, when executed by the processor 404, may cause the display 406 to present a user interface 616. The network interface 208 may be configured to connect to a network, such as a local area network (LAN), a wide area network (WAN), a wireless mobile network, a Bluetooth network, and / or the Internet, which may be composed of a wired network and / or a wireless network. The input device 510 may be any device through which a user may interact with the workstation 50, such as a mouse, a keyboard, a foot pedal, a touch screen, and / or a voice interface. The output module 412 may include any connectivity port or bus, such as a parallel port, a serial port, a universal serial bus (USB), or any other similar connectivity port known to those skilled in the art.

[0061] The memory 403 includes any non-transient computer-readable storage medium for storing data and / or software that can be executed by the processor 404 and control the operation of the workstation 80. In an embodiment, the memory 403 may include one or more solid-state storage devices, such as flash memory chips. Alternatively or in addition to one or more solid-state storage devices, the memory 403 may include one or more mass storage devices connected to the processor 404 through a mass storage controller (not shown) and a communication bus (not shown). Although the description of computer-readable media contained herein relates to solid-state storage devices, it should be understood by those skilled in the art that computer-readable storage media can be any available media that can be accessed by the processor 404. That is, computer-readable storage media include non-transient, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). For example, computer-readable storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, Blu-Ray or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the workstation 80.

Claims

1. A system for updating a 3D model of a lumen network, the system comprising: Position sensors capable of navigating the luminal network within a patient's body, an electromagnetic field generator configured to detect the position of the position sensor while navigating within the luminal network, and A computing device comprising a processor and a memory storing instructions which, when executed by the processor, cause the computing device to: accessing a 3D model of a luminal network, the 3D model being based on a computed tomography (CT) image of the luminal network; selecting a plurality of reference points within the 3D model of the luminal network; Acquiring a plurality of survey points within the lumen network; dividing the 3D model of the luminal network and the luminal network into a plurality of regions; assigning a weight to each of the plurality of regions; determining an alignment of the plurality of reference points with the plurality of survey points based on the weights; as well as generating an electromagnetic registration based on the alignment, the electromagnetic registration enabling a position within the luminal network to be converted into a position within the 3D model of the luminal network, When the instructions are executed by the processor, the computing device is further caused to: assigning a specific weight to each of the plurality of regions, the specific weight favoring the specific region; minimizing a divergence between the plurality of reference points and the plurality of survey points based on the specific weights, the minimized divergence corresponding to a specific alignment; as well as A specific registration is generated based on the specific alignment, the specific registration being associated with the specific region and enabling a position within the luminal network to be converted into a position within the 3D model of the luminal network.

2. The system of claim 1, wherein the luminal network is an airway of a patient.

3. The system according to claim 2, wherein: The 3D model is a model of the airway of the patient.

4. The system according to claim 1, when the instructions are executed by the processor, further causes the computing device to: determining from which region of the plurality of regions each of the plurality of survey points is obtained; and A corresponding weight is assigned to each of the plurality of survey points based on a region in which each of the plurality of survey points is obtained.

5. The system of claim 4, wherein when the instructions are executed by the processor, the computing device is further caused to: A divergence between the plurality of reference points and the plurality of survey points is minimized based on the respective weights assigned to each of the plurality of survey points. The system of claim 1 , wherein the plurality of regions include a transition region.

7. The system of claim 6, wherein the plurality of regions comprises a right lung region, a central region, and a left lung region; and The transition area includes a left center area and a right center area.

Images