System for assessing registrability of an anatomical model and associated methods

Abstract

Disclosed herein are devices, systems, methods, and computer program products for planning a medical procedure. In some embodiments, a system for planning a medical procedure includes a processor and a memory operably coupled to the processor. The memory stores instructions that, when executed by the processor, cause the system to perform operations including receiving a three-dimensional model of an anatomical region of a patient and evaluating whether the three-dimensional model is suitable for a registration procedure to be performed in the anatomical region. The evaluating includes analyzing a span of the three-dimensional model along at least two different directions, determining whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and performing a registration between the three-dimensional model and a virtual registration dataset.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 046,584, filed June 30, 2020, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to systems, methods, and computer program products for evaluating the registerability of models for anatomical regions and planning other aspects of registration procedures. Background Technology

[0004] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during medical procedures, thereby reducing patient recovery time, discomfort, and adverse side effects. These techniques can be performed through natural openings in the patient's anatomy or through one or more surgical incisions. Through these natural openings or incisions, the operator can insert minimally invasive medical instruments to reach the target tissue location. Minimally invasive medical instruments include devices such as therapeutic, diagnostic, biopsy, and surgical instruments. These instruments can be inserted into anatomical pathways and navigate to regions of interest within the patient's anatomy. Imagery of the anatomical pathway can be used to aid navigation. Improved systems and methods are needed to accurately perform registration between medical instruments and images of the anatomical pathway. Summary of the Invention

[0005] This document discloses devices, systems, methods, and computer program products for planning medical procedures, including evaluating whether a model of an anatomical region is suitable for a registration procedure and / or determining a survey trajectory to be navigated by a medical device during the registration procedure. In some embodiments, a system for planning a medical procedure includes a processor and a memory operatively coupled to the processor. The memory may store instructions that, when executed by the processor, cause the system to perform operations including: receiving a three-dimensional model of a patient's anatomical region and evaluating whether the three-dimensional model is suitable for a registration procedure to be performed in the anatomical region. The evaluation may include: analyzing the span of the three-dimensional model along at least two different directions, determining whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and performing registration between the three-dimensional model and a virtual registration dataset.

[0006] In these and other embodiments, a non-transitory computer-readable medium stores instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations including: receiving a three-dimensional model of a patient's anatomical region and evaluating whether the three-dimensional model is suitable for a registration procedure to be performed in the anatomical region. The evaluation may include: analyzing the span of the three-dimensional model along at least two different directions, determining whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and performing registration between the three-dimensional model and a virtual registration dataset.

[0007] In these and other embodiments, a method may include receiving a three-dimensional model of a patient's anatomical region and evaluating whether the three-dimensional model is suitable for a registration procedure to be performed in the anatomical region. The evaluation may include: analyzing the span of the three-dimensional model along at least two different directions; determining whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region; and performing registration between the three-dimensional model and a virtual registration dataset.

[0008] In these and other embodiments, the system for planning medical procedures includes a processor and a memory operatively coupled to the processor. The memory may store instructions that, when executed by the processor, cause the system to perform operations including: receiving a three-dimensional model of a patient's anatomical region; generating an investigation trajectory through a portion of the three-dimensional model; generating a virtual registration dataset including multiple data points along or near the investigation trajectory; performing multiple registrations between the three-dimensional model and the virtual registration dataset; and, based on the multiple registrations, evaluating whether the investigation trajectory is suitable for a registration procedure to be performed in the anatomical region.

[0009] In these and other embodiments, a non-transitory computer-readable medium stores instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations including: receiving a three-dimensional model of an anatomical region of a patient; generating an investigation trajectory through a portion of the three-dimensional model; generating a virtual registration dataset including multiple data points along or near the investigation trajectory; performing multiple registrations between the three-dimensional model and the virtual registration dataset; and, based on the multiple registrations, evaluating whether the investigation trajectory is suitable for a registration procedure to be performed in the anatomical region.

[0010] In these and other embodiments, a method may include: receiving a three-dimensional model of a patient's anatomical region; generating an investigation trajectory through a portion of the three-dimensional model; generating a virtual registration dataset including multiple data points along or near the investigation trajectory; performing multiple registrations between the three-dimensional model and the virtual registration dataset; and evaluating, based on the multiple registrations, whether the investigation trajectory is suitable for a registration procedure to be performed in the anatomical region. Attached Figure Description

[0011] Many aspects of this disclosure can be better understood with reference to the following accompanying drawings. The components in the drawings are not necessarily drawn to scale. Rather, the focus is on clearly illustrating the principles of this disclosure. The drawings should not be construed as limiting this disclosure to the specific embodiments depicted, but are for explanation and understanding only.

[0012] Figure 1 This is a flowchart illustrating a method for evaluating the registrationability of a model of an anatomical region according to various embodiments of the present technology.

[0013] Figure 2A This is a schematic diagram of the airways of the lungs.

[0014] Figure 2B It has insufficient horizontal span. Figure 2A A schematic diagram of the airway model.

[0015] Figure 2C It has a sufficient horizontal span. Figure 2A A schematic diagram of the airway model.

[0016] Figure 3 This is a flowchart illustrating various embodiments of the present technology for performing registration between a 3D model and a virtual registration dataset.

[0017] Figure 4 This is a flowchart illustrating various embodiments of the present technology for performing registration between a 3D model and multiple virtual registration datasets.

[0018] Figure 5 This is a flowchart illustrating a method for planning a registration procedure according to various embodiments of the present technology.

[0019] Figure 6 This is a schematic diagram of a robot or remote-controlled medical system configured according to various embodiments of the present technology.

[0020] Figure 7 This is a schematic diagram of a manipulator assembly, medical device system, and imaging system configured according to various embodiments of the present technology.

[0021] Figure 8It extends within the anatomical region of the patient according to various embodiments of this technology. Figure 7 A schematic diagram of a part of a medical device system.

[0022] Figure 9 The diagram illustrates multiple coordinate points forming a point cloud, which represents a configuration according to various embodiments of the present technology. Figure 8 The shape of a part of a medical device system.

[0023] Figure 10 Various embodiments of the present technology are shown. Figure 8 Medical device systems in Figure 8 A real navigation image of the patient's anatomical structure from the perspective of the extended portion of the anatomical region.

[0024] Figure 11 Various embodiments of the present technology are shown. Figure 8 When a part of a medical device system extends into the anatomical region Figure 8 Intraoperative images of a portion of the anatomical region.

[0025] Figure 12 This is a schematic diagram of a display system according to various embodiments of the present technology, the display showing a composite virtual navigation image (wherein) Figure 7 and Figure 8 The medical device system was registered to Figure 8 The anatomical model of the anatomical region, the virtual navigation image of the virtual patient's anatomical structure, and the real navigation image of the real patient's anatomical structure within the anatomical region. Detailed Implementation

[0026] This disclosure relates to devices, systems, methods, and computer program products for planning medical procedures to be performed within a patient's anatomical region. In some embodiments, image-guided medical procedures use a three-dimensional model of the anatomical region to assist an operator in navigating a medical device or instrument within the patient's body. The three-dimensional model is registered to the patient's anatomy such that the device's position within the patient's body can be tracked and mapped to a corresponding location within the model. Accurate registration can be particularly important for medical procedures performed within complex, dense, and / or tortuous anatomical regions, such as the airways of the lungs. However, if the three-dimensional model cannot be accurately registered to the anatomical structure (e.g., due to insufficient model data, the complexity of the anatomical region, etc.), the operator may find it difficult or impossible to navigate the medical device to the correct location within the anatomical region.

[0027] Therefore, in some embodiments, the system described herein is configured to evaluate an anatomical model during preoperative planning to determine whether the model is suitable for registration of anatomical regions (i.e., accurate and consistent registration to at least a portion of the patient's anatomical structures). For example, the evaluation may include analyzing whether the model has sufficient span along at least two distinct directions and / or determining whether the model includes key anatomical structures (e.g., trachea, main carina, left main bronchus, right main bronchus, and / or subsegmental bronchus). In some embodiments, the evaluation includes performing one or more registrations between a 3D model and one or more virtual registration datasets. The results of these simulated registrations can be used to evaluate whether the model is suitable for use in actual registration procedures or whether the model should be modified. By providing feedback on the model during the preoperative planning phase, this technology aims to reduce the likelihood of delays, cancellations, or other setbacks due to inaccurate registration during actual medical procedures.

[0028] Furthermore, a challenge during registration procedures can be determining how to survey anatomical regions using medical devices to obtain sufficient data for accurate registration of the model to the anatomical regions. Therefore, in some embodiments, the systems disclosed herein assist in planning registration procedures by determining which survey trajectories(s) are likely to produce accurate registration. For example, the systems disclosed herein can generate suggested survey trajectories from at least a portion of a 3D model and then generate a virtual registration dataset that simulates survey data obtained by a medical device following the survey trajectories. The systems disclosed herein can then perform multiple registrations between the 3D model and the virtual registration dataset to evaluate whether the survey trajectories are likely to produce successful registration. If the survey trajectories are determined to be appropriate, they can be displayed to the operator for visual guidance during the actual registration procedure. Therefore, this technique promises to improve the efficiency and accuracy of registration procedures.

[0029] A. Examples of techniques for evaluating the registrationability of anatomical models and planning registration procedures

[0030] This technology relates to the preoperative planning of medical procedures to be performed in a patient's anatomical region. In some embodiments, such as the system described herein, it is configured to assess the registerability of a three-dimensional model of the anatomical region—that is, whether the model is suitable for subsequent registration procedures. Registration procedures may involve determining transformations that map the model's reference frame to the reference frame of the patient's anatomy. As discussed above, accurate registration allows the location of a medical device to be mapped to a corresponding location in the three-dimensional model, so that the operator can refer to the model for visual guidance when navigating the device to a target location within the patient's anatomy. However, the accuracy of registration can be affected if the model does not include sufficient anatomical region data (e.g., insufficient span along multiple directions, lack of certain anatomical structures), if the fit between the model and the anatomical region is ambiguous (e.g., due to the high density of pathways within the anatomical region), or if the model otherwise fails to support stable and consistent registration. These problems can be mitigated by assessing the registerability of the model during the preoperative planning phase, allowing the model to be modified as needed before the actual registration procedure is performed on the patient.

[0031] Figure 1 This is a flowchart illustrating a method 100 for evaluating the registerability of a model of an anatomical region according to various embodiments of the present technology. Method 100 is shown as a set of steps or processes 110-150. All steps or subsets of steps of method 100 may be implemented by a computing system or device, such as a workstation, including a portable computing system (such as a laptop computer) configured to perform preoperative planning of medical procedures. Alternatively, all steps or subsets of steps of method 100 may be implemented by a control system of a medical device system or device (including various components or devices of a robot or remote control system). The computing system for implementing method 100 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform the operations according to steps 110-150. The following cross-references... Figure 2A , Figure 2B , Figure 2C , Figure 3 and Figure 4 To illustrate method 100 from various aspects.

[0032] Method 100 begins at step 110, wherein a three-dimensional model of the patient's anatomical region is received. The model may represent the anatomical region in which medical procedures are to be performed (e.g., the airways of the patient's lungs) and may represent the location, shape, and connectivity of pathways and other structures within the region. In some embodiments, the three-dimensional model is generated from preoperative image data of the anatomical region, such as computed tomography (CT) data, magnetic resonance imaging (MRI) data, fluoroscopy data, thermography data, ultrasound data, optical coherence tomography (OCT) data, thermal image data, impedance data, laser image data, nanotube X-ray image data, and / or other suitable data representing the patient's anatomy. The image data may correspond to two-dimensional, three-dimensional, or four-dimensional (e.g., time-based or velocity-based information) images. In some embodiments, for example, the image data includes two-dimensional images from multiple viewpoints that can be combined to form a pseudo-three-dimensional image.

[0033] A 3D model can be generated by segmenting graphical elements representing anatomical features in image data. During the segmentation process, pixels or voxels generated from the image data can be divided into segments or elements and / or labeled to indicate that they share certain properties or computational attributes, such as color, density, intensity, and texture. The segments or elements associated with the patient's anatomical features are then converted into a segmented anatomical model, which is generated in a model or image reference frame. To represent the model, the segmentation process can depict a set of voxels representing anatomical regions, and then apply functions such as the traveling cube function to generate a 3D surface surrounding the voxels. The model can be made by generating meshes, volumes, or voxel maps. Additionally, the model may include a centerline model comprising a set of interconnecting line segments or points extending through the center of the modeling pathway. In the case where the model includes a centerline model containing a set of interconnecting line segments, these line segments can be converted into a point cloud or point set. By converting the line segments, the desired number of points corresponding to the interconnecting line segments can be selected manually or automatically.

[0034] At step 120, the 3D model is evaluated to determine its suitability for subsequent registration procedures. As discussed above, a model is considered suitable if it can produce accurate registration with, or at least a portion thereof, of a patient's anatomy. The evaluation procedure may involve analyzing various aspects of the model to assess its registerability. In some embodiments, the evaluation procedure is used to assess the registerability of the entire model. However, in other embodiments, the evaluation procedure may be used to assess the registerability of one or more selected portions of the model. For example, registerability may be assessed only for the portion of the model corresponding to an anatomical location where accurate registration is expected to be particularly important, such as a location of interest for subsequent medical procedures or a location near such a location (e.g., a target lesion to be biopsied).

[0035] For example, at step 122, the span of the 3D model is analyzed. The span of the model can correspond to the extent of the anatomical region covered by the model. If the model does not have a sufficient span (i.e., the model does not cover enough anatomical region to allow for accurate registration of that region), the model can be considered unsuitable for registration. As discussed above, the span can be evaluated across the entire model or a specific anatomical sub-region (e.g., a lung lobe), where the site of interest (e.g., a lesion) may be located and / or accurate registration is most critical.

[0036] In some embodiments, for example, the analysis includes determining whether the model has sufficient span along at least two distinct dimensions or directions (e.g., up / down and inside / out). A model that includes sufficient span only along a first direction (e.g., up / down) and lacks sufficient span along a second direction (e.g., inside / out) may be considered unsuitable for registration, while a model that includes sufficient span along two or more distinct directions may be considered suitable for registration. In some embodiments, the two or more distinct directions correspond to known basic directions (e.g., up / down, inside / out, etc.). However, in other embodiments, the two or more distinct directions may be randomized directions, for example, to ensure that the model provides sufficient coverage in three-dimensional space.

[0037] Figures 2A-2C This is a schematic diagram illustrating the concept of a 200-degree span relative to the patient's lung airway. First, refer to... Figure 2A The airway 200 includes multiple branches or bifurcations extending in at least two different directions, such as a vertical (up-down) direction V0 and a horizontal (inward-outward) direction H0. The span of the model of the airway 200 may correspond to the amount of branches / bifurcations represented in the model. (Reference) Figure 2B For example, model 210 does not cover enough branching / bifurcation points of the airway 200 (e.g. Figure 2B (Indicated by the dashed line in the diagram), this results in model 210 having sufficient vertical span V1 but insufficient horizontal span H1. Therefore, model 210 may produce poor registration with airway 200. (See next for reference.) Figure 2C Model 220 covers sufficient branches / bifurcations of airway 200, giving it a sufficient vertical span V2 and a sufficient horizontal span H2. Therefore, model 220 can produce accurate registration with airway 200.

[0038] In some embodiments, the span of a 3D model is analyzed by calculating a first span value along a first direction and a second span value along a second direction different from (e.g., orthogonal to) the first direction. If both the first and second span values ​​exceed predetermined thresholds, the model is considered to have sufficient span along both the first and second directions. According to techniques known to those skilled in the art, span values ​​can be calculated in various ways, such as by decomposing the model into multiple feature vectors (corresponding to different directions) and multiple feature values ​​(corresponding to spans along different directions) using feature analysis. When calculating the model span, feature analysis can provide the orientation and / or magnitude of the planar anatomical structure represented by the model. Feature analysis can be performed on a surface model or a centerline model of the anatomical region. Data from a centerline model better represents the orientation and span of the model compared to other types of models. In some embodiments, feature analysis involves calculating a first feature value corresponding to the span along a first direction (e.g., the direction of maximum span, such as up-down) and a second feature value corresponding to the span along a second direction (e.g., the direction of minimum span, such as inside-out). If both the first and second feature values ​​exceed thresholds, the model is considered to have sufficient span along both the first and second directions. If the first eigenvalue or the second eigenvalue is less than the threshold, the model is considered to have insufficient span along the first or second direction, respectively. Alternatively or in combination, other techniques may be used to assess model span and / or coverage, such as principal component analysis (PCA), or to assess the area and / or volume contained in the convex hull of a 3D model.

[0039] Refer again Figure 1 At step 124, the 3D model is analyzed to determine whether it includes key anatomical structures. Specifically, the model should include one or more segmental components corresponding to one or more anatomical structures that are necessary or at least highly beneficial for successful registration. Key anatomical structures for registration may vary based on specific anatomical structures and / or patients, and may be determined based on location, size, shape, uniqueness, proximity to the target site, and / or other characteristics of the anatomical structure. For example, in embodiments where the anatomical region includes the airways of the lungs, key anatomical structures may include the trachea, main carina, left main bronchus, right main bronchus, and / or subsegmental bronchi. A model lacking segmental components corresponding to some or all of these key anatomical structures may be considered unsuitable for registration. In other embodiments, various other anatomical structures may be utilized.

[0040] In addition to determining the presence of certain key anatomical structures in the model, the analysis may also include detecting whether the dimensions (e.g., length, diameter, etc.) of these structures represented in the model are greater than or equal to a predetermined minimum dimension. For example, in an embodiment where the model includes a bronchus of the airway, the analysis may include determining whether a segmented model component corresponding to the bronchus has a length greater than or equal to 0.5 cm, 1 cm, 1.5 cm, 2 cm, 2.5 cm, 3 cm, 3.5 cm, 4 cm, 4.5 cm, or 5 cm. Optionally, the analysis may include evaluating other characteristics of the anatomical structures represented in the model. For example, in an embodiment where the anatomical structure is a branching structure with multiple generations (e.g., a bronchus), the analysis may include detecting whether the model includes a predetermined minimum number of generations (e.g., at least two, three, four, five, six, seven, eight, or more generations).

[0041] At step 126, one or more registrations are performed between the 3D model and one or more virtual registration datasets. Each virtual registration dataset can be configured to simulate survey data collected by a medical device driven within an anatomical region during an actual registration procedure. For example, when the medical device is driven within different pathways in an anatomical structure, the virtual registration dataset can simulate sensor data (e.g., position and / or shape data) generated by one or more sensors in the medical device. Thus, the registration between the virtual registration dataset and the model can be used to simulate the results of an actual registration performed between a real patient anatomy and the model.

[0042] Simulated registration can be performed in various different ways, as shown below relative to Figure 3 and Figure 4 In a more detailed description, in some embodiments, for example, one or more simulated registrations are performed between a model and a single virtual registration dataset, wherein each registration is performed with a different randomized initial seed. Alternatively or in combination, multiple simulated registrations may be performed between a model and multiple different virtual registration datasets. To register the model to one or more virtual registration datasets, the data points of the virtual registration datasets may be rotated, translated, or otherwise manipulated by rigid and / or non-rigid transformations to align them with the data points of the model. Simulated registration may be performed, for example, using a point-based Iterative Closest Point (ICP) technique, as described in U.S. Provisional Patent Applications Nos. 62 / 205,440 and 62 / 205,433, the entire contents of which are incorporated herein by reference. However, in other embodiments, simulated registration may be performed using other registration techniques.

[0043] The model's registerability can be evaluated based on whether it can be accurately and consistently registered to a virtual registration dataset. If the model cannot be accurately registered to the virtual registration dataset and / or if the registration results are unstable or inconsistent, the model may be considered unsuitable for registration. As discussed above, the accuracy and stability of registration can be evaluated across the entire model or only at selected parts of the model (e.g., parts of the model representing anatomical locations containing the target site, or other locations where accurate registration is particularly important).

[0044] At step 130, the evaluation results are output to the operator (e.g., a surgeon or clinician who will perform the registration procedure). The evaluation results may include the results of each analysis in steps 122-126, as well as feedback indicating whether the 3D model is suitable for use in the registration procedure. In some embodiments, the 3D model is considered suitable for registration if each of steps 122-126 yields a successful result (e.g., the model has sufficient span to include all key anatomical structures and can be successfully registered to the virtual registration dataset). However, in other embodiments, the model may be considered suitable even if some steps in steps 122-126 are unsuccessful (e.g., the model does not have sufficient span but includes all key anatomical structures and can be successfully registered to the virtual registration dataset). Optionally, the analyses in steps 122-126 may be given different weights when evaluating the overall registerability of the model. In a further embodiment, the output may simply include the results of the analyses in steps 122-126, so that the operator evaluates the registerability of the model (e.g., based on their own knowledge and experience) without any specific recommendations or guidance from the system.

[0045] If the model is deemed suitable, the system can provide feedback instructing the operator that the model is ready for use in medical procedures. If the model is deemed unsuitable, the feedback can indicate the need for corrective actions before proceeding with the procedure and can provide instructions or suggestions regarding the corrective actions to be taken. For example, the feedback can instruct the operator to collect additional image data of a portion or the entire anatomical region. Alternatively or in combination, the feedback can instruct the operator to modify the model, for example, by segmenting additional anatomical structures from the image data, modifying existing model components, etc. In some embodiments, the feedback can instruct the operator to extend, refine, or otherwise improve specific parts of the model, for example, if these parts do not include sufficient data (e.g., the model lacks span along a certain direction and / or is missing certain anatomical structures, etc.) and / or if the registration of the simulation is particularly inaccurate or inconsistent at these parts. Subsequently, the analysis of one or more steps 122-126 can be repeated to evaluate the registerability of the modified model. The evaluation, feedback, and revision process can be repeated multiple times to iteratively arrive at a suitable model.

[0046] Once the 3D model is approved, method 100 can proceed to step 140, where the model is registered to the anatomical region during the actual registration procedure performed in the patient. In some embodiments, the model is saved (e.g., as one or more digital files) as part of the planning of a medical procedure that includes the registration procedure. Optionally, the approved model can be used for other preoperative planning steps of the medical procedure (e.g., determining the registration investigation trajectory, determining the navigation path for the medical device to reach the target site, etc.). In embodiments where the plan is created on a preoperative planning workstation, the plan can be transferred to the medical device system used to perform the registration procedure.

[0047] During the registration procedure, a 3D model (e.g., via a graphical user interface) can be displayed to the operator for visual guidance as they navigate the medical device within the anatomical region to obtain survey data. The model can then be registered to the survey data, and thus to the actual patient anatomy, using ICP technology or any other techniques described herein. In some embodiments, the operator can review the registration results and accept the registration or take action to adjust it. For example, the operator can obtain additional survey data and rerun the registration algorithm (e.g., using only the new survey data or using a combination of survey data across multiple survey channels). This process can be repeated multiple times until a satisfactory registration is achieved. Optionally, in embodiments where multiple registrations are performed, sensitivity analysis can be performed to assess the stability and consistency of the registration results.

[0048] Once registration is approved, method 100 can proceed to step 150, in which the three-dimensional model is displayed during medical procedures. The model can be used to guide the operator in performing medical procedures (e.g., navigating a biopsy instrument to a target lesion), as described in more detail below.

[0049] Although the steps of method 100 are discussed and described in a specific order, those skilled in the art will recognize that method 100 can be modified while still remaining within these and other embodiments of the present technology. In other embodiments, for example, method 100 may be performed in a different order; for instance, any step of method 100 may be performed before, during, and / or after any other step of method 100. For example, step 124 may be performed before step 122, step 126 may be performed before step 122 and / or step 124, etc. Additionally, steps may be omitted. Figure 1 One or more steps of the method 100 shown. For example, steps 140 and 150 may be omitted, or may be performed by a different system or device than the system or device used to perform steps 110-130. Optionally, one or more steps of method 100 may be repeated.

[0050] Figure 3This is a flowchart illustrating a method 300 for performing registration between a 3D model and a virtual registration dataset according to various embodiments of the present technology. Method 300 is shown as a set of steps or processes 310-330. In some embodiments, some or all of the steps of method 300 are performed as part of a method for evaluating the registerability of the model (e.g., as part of...). Figure 1 (As part of step 126 of method 100). Method 300 may be performed by a suitable computing system or device (e.g., a preoperative planning workstation, a medical device system, etc.). The computing system for implementing method 300 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform the operations according to steps 310-330.

[0051] Method 300 begins at step 310, where a virtual registration dataset is generated. The virtual registration dataset can simulate survey data collected by a medical device during a registration procedure. In some embodiments, the virtual registration dataset is generated from a 3D model and includes data points randomized along multiple parameters (e.g., pathways, noise, deformation, depth, etc.). For example, the data points can be generated by selecting a subset of pathways or a structure of the model (e.g., all airways up to the nth generation, airways near the target lesion, or other sites of interest), creating a skeletonized representation of the centerlines of these pathways / structures, and then adding randomized noise to the skeletonized representation to simulate survey data obtained from driving the device within the selected pathways / structures. In other embodiments, the virtual registration dataset can be generated by determining a simulated survey trajectory within the 3D model and then selecting randomized data points along or near the survey trajectory to simulate survey data obtained from driving the device along the survey trajectory. Subsequently, the data points may be perturbed (e.g., translated and / or rotated) relative to their original positions, causing the virtual registration dataset to be misaligned with the model. As another example, data points can be generated by applying deformations to a model representing observed or anticipated anatomical movements at a known location (e.g., the airway). This approach can be used to produce data points that reflect anticipated movements or misalignments that may occur during actual registration procedures (e.g., due to breathing, physiological factors, patient movement, etc.).

[0052] At step 320, multiple registrations are performed between the virtual registration dataset and the 3D model. The result of each registration may be a set of registration parameters (e.g., translation and / or rotation parameters) that align the model with the virtual registration dataset when applied to points on the model. Because the virtual registration dataset is generated from the model, the “real” registration parameters used to align the virtual registration dataset and the model are known and can be used to evaluate the accuracy of the simulated registration. In some embodiments, step 320 involves using Monte Carlo analysis, sensitivity analysis, and / or other suitable techniques to evaluate whether the model can be consistently and accurately registered to the virtual registration dataset.

[0053] At step 322, for example, a randomized initial seed is generated. The initial seed can be an initial guess of the registration parameters used to align the 3D model with the virtual registration dataset. For example, in an embodiment where the 3D model is a patient airway model, the initial seed can be an estimated transformation used to align the main carina in the model with the main carina in the virtual registration dataset. The randomized initial seed can be generated in various ways, such as by first determining the “true” registration parameters used to align the model with the registration dataset and then adding randomized noise to the parameters. At step 324, the virtual registration dataset is registered to the 3D model using the initial seed as a starting point. Registration can be performed using the same or similar techniques as those used in practical registration procedures (e.g., ICP techniques) as previously described herein. At step 326, the registration parameters for the registration are determined. For example, the output of the registration algorithm can be a transformation matrix describing translation, rotation, and / or other transformations or operations used to map the model onto the virtual registration dataset. In some embodiments, steps 322-326 are repeated multiple times (e.g., at least 5, 10, 20, or 50 times) to create a sufficient result set for Monte Carlo analysis or other statistical analysis, wherein a different randomization initial seed is used for each simulation registration.

[0054] At step 330, method 300 determines the amount of variance among the multiple registrations. For example, variance can be determined by performing statistical analysis on the registration parameters from each simulated registration to assess the spread between the parameters. Alternatively or in combination, statistical analysis can be used to assess the spread between the accuracy of each registration. The amount of variance may be related to the overall registration stability of the model. For example, if the variance between registrations is relatively low (e.g., less than or equal to a threshold), the model is more likely to produce stable and successful registrations during actual registration procedures. Conversely, if the variance between registrations is relatively large (e.g., greater than a threshold), the model is less likely to produce stable and successful registrations. These results can be used to assess the overall registerability of the model, as previously relative to... Figure 1The method described in step 120 of method 100. The results may also be output to the operator as feedback for modifying the model. Statistical measures of the differences may be calculated as a whole to determine the registrationability of the entire model, or they may be calculated for specific locations within the model (e.g., locations corresponding to lesions or other sites of interest in an anatomical region).

[0055] Although the steps of method 300 are discussed and described in a specific order, those skilled in the art will recognize that method 300 can be modified while still remaining within these and other embodiments of the present technology. In other embodiments, for example, method 300 may be performed in a different order; for instance, any step of method 300 may be performed before, during, and / or after any other step of method 300. Additionally, steps may be omitted or repeated. Figure 3 One or more steps of the method 300 shown.

[0056] Figure 4 This is a flowchart illustrating a method 400 for performing registration between a 3D model and multiple virtual registration datasets according to various embodiments of the present technology. Method 400 is shown as a set of steps or processes 410-430. In some embodiments, some or all steps of method 400 are performed as part of a method for evaluating the registerability of the model (e.g., as part of...). Figure 1 (A portion of step 126 of method 100). For example, method 400 can be used with... Figure 3 Method 300 may be performed in combination with or as an alternative to it. Method 400 differs from Method 300 in that Method 300 involves repeatedly registering the model to a single virtual registration dataset (which has a different initial seed for each registration run), while Method 400 involves registering the model to multiple different virtual registration datasets. Method 400 may be performed by a suitable computing system or device (e.g., a preoperative planning workstation, a medical device system, etc.). A computing system for implementing Method 400 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform the operations according to steps 410-430.

[0057] Method 400 begins at step 410, in which multiple virtual registration datasets are generated. Each virtual registration dataset can simulate survey data collected by a medical device during a registration procedure. In some embodiments, each virtual registration dataset is different from the others. For example, step 410 involves generating at least 5, 10, 20, 50, or more distinct virtual registration datasets. This can be used in relation to... Figure 3The process described in step 310 of method 300 uses similar or identical techniques to generate different virtual registration datasets from the model. In some embodiments, for example, each virtual registration dataset is generated from a different part of the model (e.g., different airways). Alternatively, some or all of the virtual registration datasets may be generated from the same part of the model, but randomized noise is added so that each dataset includes at least some unique data points. Optionally, some or all of the virtual registration datasets may be generated based on different simulated survey trajectories, as previously described relative to... Figure 3 As described in step 310.

[0058] At step 420, multiple registrations are performed between the virtual registration datasets and the 3D model. In some embodiments, each registration is performed between the model and different virtual registration datasets. The result of each registration may be a set of registration parameters that align the model with the corresponding virtual registration dataset. For example, at step 422, the virtual registration datasets are registered to the 3D model. Registration can be performed using techniques that are the same as or similar to those used in practical registration procedures as previously described herein (e.g., ICP techniques). At step 424, as previously described, the registration parameters for the registration are determined. Steps 422 and 424 may be repeated multiple times until all virtual registration datasets have been registered with the model.

[0059] At step 430, method 400 determines the number of virtual registration datasets that generate accurate registrations through the model. As previously described, because each virtual registration dataset is generated from the model, the “true” registration parameters for each virtual registration dataset are known. Therefore, the registration parameters of each virtual registration dataset can be compared with the “true” parameters to determine the accuracy of the registration. In some embodiments, the accuracy may be expressed as a score or other quantitative measure. Method 400 then determines the number and / or proportion of simulated registrations that produce sufficiently accurate results (e.g., a registration accuracy score greater than or equal to a threshold). The number and / or proportion of accurate registrations may be related to the overall probability that the model will be successfully registered to the anatomical structure during an actual registration procedure. For example, if at least one virtual registration dataset produces accurate registrations, or if at least 50%, 75%, 80%, 90%, 95%, or 99% of the simulated registration results are sufficiently accurate, the model may be considered suitable for registration. Conversely, if no virtual registration dataset produces accurate registration, or if the number and / or proportion of accurate registrations is too small (e.g., less than 50%, 25%, 10%, or 5% accurate registration), the model may be considered unsuitable for registration. As discussed above, the accuracy of simulated registration can be evaluated across the entire model or only for selected portions of the model (e.g., portions corresponding to the target anatomical site of interest). These results can be used to evaluate the overall registerability of the model, as previously assessed relative to… Figure 1The method described in step 120 of method 100. The result can also be output to the operator as feedback for modifying the model.

[0060] Optionally, method 400 may include other techniques for evaluating the overall stability and accuracy of simulated registration. In some embodiments, for example, method 400 includes performing statistical analysis on the registration results to determine differences or gaps in accuracy across all registrations. Method 400 may also include identifying common characteristics in virtual registration datasets that produce successful or unsuccessful registrations (e.g., if virtual registration datasets generated from certain portions of the model tend to produce more accurate or inaccurate registrations). This information may be provided to the operator as additional feedback for adjusting the model.

[0061] Although the steps of method 400 are discussed and described in a specific order, those skilled in the art will recognize that method 400 can be modified while still remaining within these and other embodiments of the present technology. In other embodiments, for example, method 400 may be performed in a different order; for instance, any step of method 400 may be performed before, during, and / or after any other step of method 400. Additionally, steps may be omitted or repeated. Figure 4 One or more steps of the method 400 shown.

[0062] In some embodiments, the three-dimensional anatomical model described herein is used to plan a registration procedure. As previously mentioned, a registration procedure may involve driving a medical device within pathways in an anatomical region to collect survey data, and then using the survey data to register the model to the anatomical region. However, the operator may not know the optimal trajectory or route for driving the medical device to obtain survey data. Even if the system instructs the operator to drive the device within certain areas of the anatomical structure, it may be difficult to determine the specific pathway to which the device should be driven, especially when the pathways are extremely dense or tortuous (e.g., in the airways of the lungs). Furthermore, the operator may not know whether they have collected enough survey data to generate an accurate and consistent registration through the model. In some cases, even if the model itself is suitable for registration, registration may fail if insufficient survey data is obtained and / or if the locations covered by the survey data are not ideal. Therefore, in some embodiments, the system described herein is configured to plan a survey trajectory that may produce a successful registration with the model. The survey trajectory may be displayed to guide the operator in collecting survey data during the registration procedure.

[0063] Figure 5This is a flowchart illustrating a method 500 for planning a registration procedure according to various embodiments of the present technology. Method 500 is shown as a set of steps or procedures 510-570. Method 500 may be performed by a suitable computing system or device (e.g., a preoperative planning workstation, a medical device system, etc.). A computing system for implementing method 500 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform the operations according to steps 510-570.

[0064] Method 500 begins at step 510, where a three-dimensional model of the patient's anatomical region is received. This model may be a segmented model generated from preoperative imaging data, as previously described relative to... Figure 1 The method 100 is described in step 110. In some embodiments, the model has been determined to be suitable for registration, as previously described relative to... Figure 1 , Figure 3 and Figure 4 Methods 100, 300, and / or 400 are described. In other embodiments, the registerability of the model has not been evaluated, such that some or all of the subsequent steps of method 500 (e.g., steps 520-550) may be performed before or concurrently with some or all of the steps in methods 100, 300, and / or 400.

[0065] At step 520, a survey trajectory is generated through a portion of the 3D model. The survey trajectory may be a suggested route for driving the medical device within the anatomical region to collect survey data during registration procedures. The model can be used to generate the survey trajectory automatically, semi-automatically, or manually. For example, an operator can manually create some or all of the survey trajectory by selecting pathways (e.g., airways) in the model via a suitable graphical user interface. Alternatively or in combination, some or all of the survey trajectory may be automatically generated by the system. In some embodiments, the system may automatically generate the trajectory, and the operator may approve the trajectory or manually modify it (e.g., by adding, deleting, or otherwise modifying portions of the trajectory). Conversely, the operator may manually create the trajectory, and the system may automatically modify the trajectory or propose modifications for operator approval.

[0066] In some embodiments, the system automatically identifies pathways in a model that are likely to produce accurate registration when investigated via a medical device, and generates investigation trajectories traversing some or all of these pathways. For example, the system may select pathways based on the accessibility of the medical device (e.g., a sufficiently large diameter, not too tortuous), depth within the anatomical region (e.g., at least x-generation deep), and / or uniqueness (e.g., having a shape and / or location easily distinguishable from other pathways). Optionally, investigation trajectories may be generated at least in part based on patient-specific factors such as the location of the target site (e.g., the target lesion or biopsy site) of the medical procedure. For example, investigation trajectories may be configured to traverse pathways relatively close to the target location to increase the likelihood that registration will be accurate at or near the target location. Optionally, if the target site is at a specific depth within the anatomical region (e.g., at or near the n-generation pathway), the investigation trajectory may be configured to be at least as deep as the target site to increase the likelihood that registration will be accurate at the target depth.

[0067] At step 530, a virtual registration dataset is generated for the survey trajectory. The virtual registration dataset can simulate survey data generated by a medical device driven along the survey trajectory and may include multiple data points along or near the survey trajectory. It can be used in conjunction with previously generated datasets relative to the survey trajectory. Figure 3 and Figure 4 The techniques described in steps 310 and / or 410 are the same or similar to those used to generate the virtual registration dataset. In some embodiments, for example, the system creates a skeletal representation of the pathway centerline traversed by the survey trajectory, and then adds randomized noise to that representation to generate data points for the virtual registration dataset. In other embodiments, deformations may be applied to a model simulating or representing anticipated anatomical motion at certain locations (e.g., the airway) to generate data points reflecting anticipated motion or misalignment (e.g., due to breathing, physiological factors, or patient movement).

[0068] At step 540, multiple registrations (e.g., at least 5, 10, 20, 50, or more registrations) are performed between the 3D model and the virtual registration dataset. Similar to... Figure 3 Method 300, this step may use Monte Carlo analysis, sensitivity analysis, and / or other statistical techniques to evaluate whether the virtual registration dataset generated from the survey trajectory consistently produces accurate registration with the model. In some embodiments, the registration algorithm includes relative to... Figure 3 Method 300 describes some or all of the procedures in steps 322-326. For example, each registration can be performed using the same virtual registration dataset and different (e.g., randomized) initial seeds. Each registration can produce a set of registration parameters representing an estimated transformation used to map the 3D model to the virtual registration dataset.

[0069] At step 550, the system evaluates whether the survey trajectory is suitable for the registration procedure based on the registration results from step 540. The evaluation may include determining the amount of difference between registrations, similar to the above relative to... Figure 3 The process described in step 330 of method 300. If the variance is relatively low (e.g., less than or equal to a threshold), the survey trajectory is more likely to produce an accurate registration with the model. On the other hand, if the variance is relatively high (e.g., greater than a threshold), the survey trajectory is less likely to produce an accurate registration.

[0070] If the survey trajectory is determined to be unsuitable for registration, the system may output feedback to the operator indicating the need for corrective action, and optionally including instructions or suggestions regarding the specific corrective actions to be taken. For example, the feedback may instruct the operator to manually add, delete, or otherwise modify portions of the survey trajectory. In some embodiments, if the registration accuracy is particularly poor at certain locations in the model, the system may instruct the operator to extend the survey trajectory to those locations. The analysis in steps 530-550 may then be repeated to evaluate whether the corrected survey trajectory produces a satisfactory registration result. The evaluation, feedback, and revision process may be repeated multiple times to iteratively arrive at a suitable survey trajectory.

[0071] Once the investigation trajectory is determined to be appropriate, method 500 can proceed to step 560, where the investigation trajectory is displayed during the registration procedure. In some embodiments, the investigation trajectory (e.g., as one or more digital files) is saved as part of preoperative planning for a medical procedure that includes the registration procedure. The planning may also include a three-dimensional model for generating the investigation trajectory. The planning may be transferred to a medical device system for performing the medical procedure. Examples of medical device systems suitable for use with the embodiments herein are described in more detail below.

[0072] During the registration procedure, a survey trajectory may be displayed to the operator (e.g., via a graphical user interface) to provide visual guidance for navigating the medical device within the patient's anatomy to collect survey data. For example, the system may output a graphical representation of the survey trajectory, such as a path overlaid on a model and / or image of the actual patient's anatomy. Alternatively or in combination, the system may output text, audio, or other instructions that guide the operator to navigate the device along the survey trajectory (e.g., in a specific direction and / or for a specific distance, relative to specific anatomical landmarks, etc.). Optionally, the system may display the survey trajectory along with location data of the medical device collected by the device so that the operator can track the device's position relative to the trajectory. In some embodiments, the system also displays the survey data collected by the medical device along with the survey trajectory, allowing the author to assess whether sufficient data has been obtained, whether any gaps or incomplete sampling areas exist, etc.

[0073] At step 570, the 3D model is registered to the anatomical region using the survey data collected during the registration procedure (e.g., as previously relative to the anatomical region). Figure 1 (as described in step 140 of method 100). In some embodiments, the survey trajectory is also used to calculate the registration. For example, when calculating the registration parameters, the registration algorithm may assume that the survey data is located along or near the survey trajectory. Optionally, the algorithm may ignore survey data points located far from the survey trajectory when calculating the registration, for example, if such data points are considered unnecessary for accurate registration and / or may introduce errors. In some embodiments, the operator examines the registration results and, if necessary, collects additional survey data to correct the registration, as described above relative to... Figure 1 The steps discussed in step 140. Once the model has been successfully registered to the patient's anatomy, it can be used to guide the operator in performing medical procedures (e.g., navigating biopsy instruments to the target lesion), as described in more detail below.

[0074] Although the steps of method 500 are discussed and described in a specific order, those skilled in the art will recognize that method 500 can be modified while still remaining within these and other embodiments of the present technology. In other embodiments, for example, method 500 may be performed in a different order; for instance, any step of method 500 may be performed before, during, and / or after any other step of method 100. Additionally, steps may be omitted. Figure 1 One or more steps of the method 500 shown. For example, steps 560 and 570 may be omitted, or may be performed by a different system or device than the system or device used to perform steps 510-550.

[0075] Optionally, one or more steps of method 500 can be repeated. For example, steps 520-550 can be repeated multiple times to generate and evaluate multiple different investigation trajectories (e.g., two, three, four, five, or more different investigation trajectories). Different investigation trajectories may differ from one another in terms of location within the anatomical region, proximity to the target site within the anatomical region, length, depth, shape, uniqueness, and / or any other suitable characteristics. For example, the system described herein can evaluate multiple investigation trajectories and suggest different trajectories depending on the location of the target lesion in a subsequent biopsy procedure. A virtual registration dataset can be generated for each investigation trajectory and used in simulated registration to evaluate the suitability of the investigation trajectory, as described above. In such embodiments, method 500 may also include selecting a subset of investigation trajectories for a registration procedure, for example, based on whether the investigation trajectories are likely to produce accurate and stable registration. The selected trajectories may be displayed to the operator sequentially or simultaneously at step 560, allowing the operator to choose between different options of the actual registration procedure.

[0076] Furthermore, in other embodiments, instead of generating a single virtual registration dataset for each survey trajectory, steps 530 and 540 can be repeated multiple times for a single survey trajectory to generate multiple different virtual registration datasets for that trajectory (e.g., at least 5, 10, 20, 50, or more datasets). Previously used data can be used relative to... Figure 4 Method 400, step 410, describes the techniques used to create different virtual registration datasets. The suitability of the survey trajectory for registration can then be determined by evaluating the number and / or proportion of virtual registration datasets that can be successfully registered to the model, similar to previous methods relative to... Figure 4 The technique described in steps 420-430 of method 400. This method can provide additional information about the quality and stability of registrations generated via survey trajectories.

[0077] In some embodiments, techniques for determining the survey trajectory of a registration procedure can be combined with techniques for evaluating the registerability of a three-dimensional anatomical model. For example, Figure 5 Method 500 has some or all of the steps that can be compared with Figure 1 Method 100 may involve the combination or simultaneous execution of some or all of its steps (e.g., step 110 of method 100 may be combined with step 510 of method 500, step 126 of method 100 may be combined with steps 520-540 of method 500, step 140 of method 100 may be combined with steps 560 and 570 of method 500, etc.). In such embodiments, method 500 may be performed partially or entirely by the same system or device used to perform method 100, and vice versa. However, in other embodiments, method 500 and method 100 may be different processes performed at different times and / or by different systems or devices.

[0078] In some embodiments, this text is relative to Figures 1-5 The described techniques are performed as part of the preoperative planning of a medical protocol to perform biopsies on one or more target sites (e.g., lesions or other tissues of interest). Therefore, the techniques described herein can be used to (i) assess the registerability of an anatomical model with anatomical structures at or near the target site, and (ii) determine one or more investigation trajectories that may produce accurate and consistent registration with anatomical structures at or near the target site. In embodiments where the medical protocol involves biopsies of multiple target sites, the system described herein can assess the registerability of the model relative to each target site (e.g., by analyzing model span, key anatomical structures, and / or simulated registration results at or near each site), and / or can suggest investigation trajectories tailored for each site (e.g., by evaluating different investigation trajectories through portions of the anatomical structures at or near each site).

[0079] B. Examples of robotic or remote-controlled medical systems and associated devices, systems, and methods

[0080] Figure 6 This is a schematic diagram of a robot or remote-controlled medical system 600 (“Medical System 600”) configured according to various embodiments of the present technology. The Medical System 600 may be related to the above description. Figures 1-5 Any procedures or methods described herein may be used together. As shown, medical system 600 includes a manipulator assembly 602, a medical device system 604, a main assembly 606, and a control system 612. The manipulator assembly 602 supports the medical device system 604 and drives the medical device system 604 under the guidance of the main assembly 606 and / or the control system 612 to perform various medical procedures on a patient 603 positioned on a table 607 in a surgical setting 601. In this respect, the main assembly 606 typically includes one or more control devices that can be operated by an operator 605 (e.g., a physician) to control the manipulator assembly 602. Furthermore, or alternatively, the control system 612 includes a computer processor 614 and at least one memory 616 for implementing control between the medical device system 604, the main assembly 606, and / or other components of the medical system 600. The control system 612 may also include programming instructions (e.g., stored on a non-transitory computer-readable medium) to implement any one or more of the methods described herein, including instructions for providing information to the display system 610 and / or processing data for registering the medical device system 604 with an anatomical model of the patient 603 (as described in more detail below). The manipulator assembly 602 may be a remote-controlled, non-remote-controlled, or hybrid remote-controlled and non-remote-controlled assembly. Therefore, all or part of the main assembly 606 and / or all or part of the control system 612 may be located inside or outside the surgical environment 601.

[0081] To assist operator 605 in controlling manipulator assembly 602 and / or medical device system 604 during image-guided medical procedures, medical system 600 may also include position sensor system 608, endoscopic imaging system 609, imaging system 618, and / or virtual visualization system 615. In some embodiments, position sensor system 608 includes a position sensor system (e.g., an electromagnetic (EM) sensor system) and / or shape sensor system for capturing position sensor data (e.g., position, orientation, velocity, rate, posture, shape, etc.) of medical device system 604. In these and other embodiments, endoscopic imaging system 609 includes one or more image capture devices (not shown) that record endoscopic image data including concurrent or real-time images (e.g., video, still images, etc.) of patient anatomy. For example, the images captured by endoscopic imaging system 609 may be two-dimensional or three-dimensional images of patient anatomy captured by an image capture device located within patient 603, and are referred to below as “real-world navigation images.”

[0082] In some embodiments, the medical device system 604 may include components of the position sensor system 608 and / or components of the endoscopic imaging system 609. For example, components of the position sensor system 608 and / or components of the endoscopic imaging system 609 may be integrally or removably coupled to the medical device system 604. Additionally or alternatively, the endoscopic imaging system 609 may include a separate endoscope (not shown) attached to a separate manipulator assembly (not shown), which may be used in conjunction with the medical device system 604 to image patient anatomy. The position sensor system 608 and / or the endoscopic imaging system 609 may be implemented as hardware, firmware, software, or a combination thereof, which interacts with or is otherwise executed by one or more computer processors, wherein the one or more computer processors are one or more computer processors 614, such as control system 612.

[0083] The imaging system 618 of the medical system 600 may be positioned in a surgical environment 601 near the patient 603 to obtain real-time and / or near-real-time images of the patient 603 before, during, and / or after medical procedures. In some embodiments, the imaging system 618 includes a mobile C-arm cone-beam CT imaging system for generating three-dimensional images. For example, the imaging system 618 may include a Siemens DynaCT imaging system, or other suitable imaging systems. In these and other embodiments, the imaging system 618 may include other imaging techniques, including MRI, fluoroscopy, temperature recording, ultrasound, OCT, thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like.

[0084] When controlling the medical device system 604 during an image-guided medical procedure, the virtual visualization system 615 of the control system 612 provides navigation and / or anatomical interaction assistance to the operator 605. As described in more detail below, virtual navigation using the virtual visualization system 615 may be based at least in part on a preoperative or intraoperative dataset of the acquired anatomical pathways of the patient 603 (e.g., at least in part on a reference to data generated by the position sensor system 608, the endoscopic imaging system 609, and / or the imaging system 618). In some specific implementations, for example, the virtual visualization system 615 processes preoperative and / or intraoperative image data of the anatomical regions of the patient 603 captured by the imaging system 618 to generate an anatomical model of the anatomical regions (not shown). The virtual visualization system 615 then registers the anatomical model with position sensor data generated by the position sensor system 608 and / or endoscopic image data generated by the endoscopic imaging system 609 to (i) map the tracking position, orientation, posture, shape and / or movement of the medical device system 604 within the anatomical region to the correct position within the anatomical model, and / or (ii) determine a virtual navigation image of the virtual patient anatomy of the anatomical region from the perspective of the medical device system 604, wherein the position of the medical device system 604 within the anatomical model corresponds to the position of the medical device system 604 within the patient 603.

[0085] Display system 610 may display various images or representations of patient anatomy and / or medical device system 604 generated by position sensor system 608, endoscopic imaging system 609, imaging system 618, and / or virtual visualization system 615. In some embodiments, display system 610 and / or main component 606 may be oriented such that operator 605 can control manipulator component 602, medical device system 604, main component 606, and / or control system 612 through perceptual telepresentation.

[0086] As discussed above, the manipulator assembly 602 drives the medical device system 604 under the guidance of the main assembly 606 and / or the control system 612. In this regard, the manipulator assembly 602 may select degrees of freedom that can be motorized and / or remotely controlled, and may also select degrees of freedom that can be non-motorized and / or non-remotely controlled. For example, the manipulator assembly 602 may include multiple actuators or motors (not shown) that drive inputs on the medical device system 604 in response to commands received from the control system 612. The actuators may include a drive system (not shown) that, when coupled to the medical device system 604, can advance the medical device system 604 into a naturally or surgically generated anatomical opening. Other drive systems may move a distal portion (not shown) of the medical device system 604 with multiple degrees of freedom, including three linear degrees of freedom (e.g., linear motion along the X, Y, Z Cartesian axes) and three rotational degrees of freedom (e.g., rotation about the X, Y, Z Cartesian axes). Additionally or alternatively, the actuator may be used to actuate the articulated end effector of the medical device system 604 (e.g., for grasping tissue in the jaws of a biopsy device and / or the like).

[0087] Figure 7 It is configured within the surgical environment 601 and according to various embodiments of the present technology. Figure 6 A schematic diagram of the manipulator assembly 602, the medical device system 604, and the imaging system 618. (See attached diagram.) Figure 7 As shown, the surgical environment 601 has a surgical reference frame (X). S ,Y S Z S In this system, patient 603 is located on platform 607 and medical device system 604 has a medical device reference system (X) within surgical environment 601. M ,Y M Z M During medical procedures, patient 603 may remain still within surgical environment 601 in the sense that the patient's overall movement can be restricted by sedation, restraint, and / or other means. In these and other embodiments, periodic anatomical movements of patient 603 (including respiratory and cardiac movements) may continue unless patient 603 is instructed to hold his or her breath to temporarily cease respiratory movements.

[0088] The manipulator assembly 602 includes an instrument holder 726 mounted to the insertion stage 728. In the illustrated embodiment, the insertion stage 728 is linear, while in other embodiments, the insertion stage 728 is curved or has a combination of curved and linear portions. In some embodiments, the insertion stage 728 is fixed within the surgical environment 601. Alternatively, the insertion stage 728 may be movable within the surgical environment 601 but has a known position within the surgical environment 601 (e.g., via a tracking sensor (not shown) or other tracking device). In these alternatives, a medical device reference frame (X...) M ,Y M Z M ) relative to the surgical reference frame (X S ,Y S Z S () is fixed or otherwise known.

[0089] Figure 7 The medical device system 604 includes an elongated device 731, a medical device 732, a device body 735, at least a portion of a position sensor system 608, and at least a portion of an endoscopic imaging system 609. In some embodiments, the elongated device 731 is a flexible catheter or other biomedical device defining a channel or lumen 744. The channel 744 may be sized and shaped to receive the medical device 732 (e.g., via a proximal end 736 of the elongated device 731 and / or an instrument port (not shown)) and facilitate delivery of the medical device 732 to a distal portion 738 of the elongated device 731. The elongated device 731 is coupled to the device body 735, which is in turn coupled and secured relative to the instrument holder 726 of the manipulator assembly 602.

[0090] In operation, the manipulator assembly 602 can control the insertion movement (e.g., proximal and / or distal movement along axis A) of the elongated device 731 into the patient 603 via a natural or surgically generated anatomical orifice of the patient 603 to facilitate navigation of the elongated device 731 through anatomical pathways of the anatomical region of the patient 603 and / or facilitate delivery of the distal portion 738 of the elongated device 731 to or near a target location within the patient 603. For example, the instrument holder 726 and / or insertion stage 728 may include actuators (not shown), such as servo motors, which help control the movement of the instrument holder 726 along the insertion stage 728. Furthermore, or alternatively, in some embodiments, the manipulator assembly 602 can control the movement of the distal portion 738 of the elongated device 731 in multiple directions, including yaw, pitch, and roll rotation (e.g., to navigate patient anatomy). For this purpose, the elongated device 731 may accommodate or include cables, linkages, and / or other steering controls (not shown) that the manipulator assembly 602 may use to controllably bend the distal portion 738 of the elongated device 731. For example, the elongated device 731 may accommodate at least four cables that can be used by the manipulator assembly 602 to provide (i) independent "up and down" steering to control the pitch of the distal portion 738 of the elongated device 731, and (ii) independent "left and right" steering of the elongated device 731 to control the yaw of the distal portion 738 of the elongated device 731.

[0091] Medical device 732 of medical device system 604 can be used in medical procedures such as investigation of anatomical pathways, surgery, biopsy, ablation, illumination, irrigation, and / or aspiration. Therefore, medical device 732 may include image-capturing probes, biopsy instruments, laser ablation fibers, and / or other surgical, diagnostic, and / or therapeutic tools. For example, medical device 732 may include an endoscope or other biomedical device having one or more image-capturing devices 747 located at a distal portion 737 of medical device 732 and / or at other locations along the medical device. In these embodiments, when medical device 732 is in the anatomical region of patient 603, image-capturing device 747 may capture one or more real navigation images or videos (e.g., a sequence of one or more real navigation image frames) of the anatomical pathway and / or other real patient anatomy.

[0092] As discussed above, the medical device 732 can be deployed to and / or delivered to a target location within the patient 603 via the channel 744 defined by the elongated device 731. In embodiments where the medical device 732 includes an endoscope or other biomedical device with an image capture device 747 at its distal portion 737, the image capture device 747 may be advanced to the distal portion 738 of the elongated device 731 before, during, and / or after the manipulator assembly 602 navigates the distal portion 738 of the elongated device 731 to the target location within the patient 603. In these embodiments, the medical device 732 can be used as an investigation instrument to capture realistic navigation images of anatomical pathways and / or other real patient anatomy, and / or assist an operator (not shown) in navigating the distal portion 738 of the elongated device 731 through the anatomical pathway to the target location.

[0093] As another example, after the manipulator assembly 602 positions the distal portion 738 of the elongated device 731 near a target location within the patient 603, the medical device 732 may be advanced beyond the distal portion 738 of the elongated device 731 to perform a medical procedure at the target location. Continuing this example, after all or part of the medical procedure at the target location has been completed, the medical device 732 may be retracted into the elongated device 731 and additionally or alternatively removed along the elongated device 731 from its proximal end 736 or from another device port (not shown).

[0094] like Figure 7 As shown, the position sensor system 608 of the medical device system 604 includes a shape sensor 733 and a position measuring device 739. In these and other embodiments, in addition to or in place of the shape sensor 733 and / or the position measuring device 739, the position sensor system 608 may include other position sensors (e.g., accelerometers, rotary encoders, etc.).

[0095] The shape sensor 733 of the position sensor system 608 includes an optical fiber extending within and aligned with an elongated device 731. In one embodiment, the optical fiber of the shape sensor 733 has a diameter of approximately 200 μm. In other embodiments, the diameter of the optical fiber may be larger or smaller. The optical fiber of the shape sensor 733 forms an optical fiber bending sensor for determining the shape, orientation, and / or posture of the elongated device 731. In some embodiments, an optical fiber having a fiber Bragg grating (FBG) may be used to provide strain measurements in one-dimensional or multi-dimensional structures. Various systems and methods for monitoring the shape and relative position of optical fibers in three dimensions are further described in detail below: U.S. Patent Application Publication No. 2006 / 0013523 (filed July 13, 2005) (disclosing an optical fiber position and shape sensing device and related methods); U.S. Patent No. 7,781,724 (filed September 26, 2006) (disclosing an optical fiber position and shape sensing device and related methods); U.S. Patent No. 7,772,541 (filed March 12, 2008) (disclosing optical fiber position and / or shape sensing based on Rayleigh scattering); and U.S. Patent No. 6,389,187 (filed June 17, 1998) (disclosing an optical fiber bending sensor), the entire contents of which are incorporated herein by reference. In these and other embodiments, the sensor of this technology may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. In these and other embodiments, other techniques may be used to determine the shape of the elongated device 731. For example, the history of the orientation of the distal portion 738 of the elongated device 731 can be used to reconstruct the shape of the elongated device 731 over a certain time interval.

[0096] In some embodiments, the shape sensor 733 is fixed at a proximal point 734 on the device body 735 of the medical device system 604. In operation, for example, the shape sensor 733 measures a medical device reference frame (X) from the proximal point 734 along an optical fiber to another point. M ,Y M Z M The shape in the device, such as the distal portion 738 of the elongated device 731. The proximal point 734 of the shape sensor 733 may move together with the device body 735, but the position of the proximal point 734 may be known (e.g., via a tracking sensor (not shown) or other tracking device).

[0097] The position measuring device 739 of the position sensor system 608 provides information about the position of the instrument body 735 as it moves along the insertion axis A on the insertion stage 728 of the manipulator assembly 602. In some embodiments, the position measuring device 739 includes a resolver, encoder, potentiometer, and / or other sensors that determine the rotation and / or orientation of the actuator (not shown) that controls the movement of the instrument carrier 726 of the manipulator assembly 602 and thus the movement of the instrument body 735 of the medical device system 604.

[0098] Figure 8 It extends within the anatomical region 850 (e.g., human lung) of patient 603 according to various embodiments of the present technology. Figure 7 A schematic diagram of a portion of the medical device system 604. Specifically, Figure 8 An elongated device 731 of a medical device system 604 extending within a branching anatomical pathway 852 of an anatomical region 850 is shown. The anatomical pathway 852 includes a trachea 854 and multiple bronchi 856.

[0099] like Figure 8 As shown, the elongated device 731 has a specific position, orientation, posture, and shape within the anatomical region 850, wherein all or part thereof (except for or in lieu of movement, such as velocity or rate) can be generated by Figure 6 and Figure 7 Position sensor system 608 (e.g., consisting of shape sensor 733 and / or position measuring device 739) Figure 7 The system captures data as a position sensor to investigate anatomical pathways 852 within anatomical region 850. Specifically, the position sensor system 608 can capture data via a medical device reference frame (X-ray). M ,Y M Z M Position sensor data of the medical device system 604 is collected within the anatomical region 850 to investigate the anatomical pathway 852. The position sensor data may be recorded at least partially as a set of two-dimensional or three-dimensional coordinate points. In an example where the anatomical region 850 is a human lung, the coordinate points may represent the position of the distal portion 738 of the elongated device 731 and / or other portions of the elongated device 731 as it is advanced through the trachea 854 and bronchi 856. In these and other embodiments, the set of coordinate points may represent the shape of the elongated device 731 as it is advanced through the anatomical region 850. In these and other embodiments, the coordinate points may represent other portions of the medical device system 604 (e.g., medical device 732). Figure 7 Location data.

[0100] Coordinate points can be combined to form a point cloud. For example, Figure 9Multiple coordinate points 962 according to various embodiments of the present technology are shown, their formation representing Figure 8 The slender device 731 is located in the anatomical region 850. Figure 8 Point cloud shape within 960. Specifically, Figure 9 The point cloud 960 is from the position sensor system 608 ( Figure 7 In the slender device 731 Figure 8 The union of all or a subset of the coordinates 962 recorded at the static position shown is generated.

[0101] In some embodiments, a point cloud (e.g., point cloud 960) may include the union of all or a subset of coordinate points recorded by the position sensor system 608 during an image capture period across an anatomical region 850, representing multiple shapes, positions, orientations, and / or poses of the elongated device 731. In these embodiments, the point cloud may include coordinate points captured by the position sensor system 608 representing multiple shapes of the elongated device 731 as it is advanced or moved through the patient's anatomy during the image capture period. Additionally or alternatively, because the configuration (including shape and position) of the elongated device 731 within the patient 603 may change due to anatomical motion during the image capture period, in some embodiments, the point cloud may include multiple coordinate points 962 captured by the position sensor system 608 representing the shape of the elongated device 731 as it is passively moved within the patient 603. As described in more detail below, the point cloud of coordinate points captured by the position sensor system 608 may be registered to different models or datasets of the patient's anatomy.

[0102] Refer again Figure 7 The endoscopic imaging system 609 of the medical device system 604 includes one or more image capture devices 747, which are configured to capture images when the elongated device 731 and / or the medical device 732 are within the anatomical region of the patient 603 (e.g., Figure 8 When capturing the anatomical region (850) of the real patient's anatomical structure (e.g., the anatomical region 850), the actual patient's anatomical structure is captured. Figure 8 One or more real navigation images of the anatomical pathway 852. For example, the endoscopic imaging system 609 may include an image capturing device 747 located at the distal portion 737 of the medical device 732. In these and other embodiments, the endoscopic imaging system 609 may include one or more image capturing devices (not shown) located at other locations along the medical device 732 and / or along the elongated device 731 (e.g., at the distal portion 738 of the elongated device 731).

[0103] exist Figure 8 In the illustrated embodiment, medical device 732 ( Figure 7The image capturing device 747 is advanced and positioned at the distal portion 738 of the elongated device 731. In this embodiment, the image capturing device 747 can investigate the anatomical pathway 852 by capturing real navigation images of the anatomical pathway 852 as the elongated device 731 is navigated through the trachea 854 and bronchus 856 of the anatomical region 850.

[0104] Figure 10 It is via image capture device 747 ( Figure 8 ) captured Figure 8 An example of a real navigation image 1070 (e.g., a still image, a video frame, etc.) of the patient's anatomical structure of the anatomical region 850 (such as one of the anatomical pathways 852). As shown, the real navigation image 1070 illustrates the patient's anatomical structure from the medical device 732 ( Figure 7 From the viewpoint of the anatomical region 850, the branching points or ridges 1071 of the two anatomical pathways 852 are viewed. In this example, because the image capturing device 747 is positioned at the medical device 732 and the elongated device 731 respectively. Figure 8 The viewpoint of the actual navigation image 1070 is from the distal portion 737 of the medical device 732, so that the medical device 732 and the elongated device 731 are not visible in the actual navigation image 1070. In other embodiments, the image capturing device 747 may be along the medical device 732 and / or along the elongated device 731. Figure 7 and Figure 8 The endoscopic imaging system 109 is positioned in another location. In these embodiments, the endoscopic imaging system 109 is positioned in another location. Figure 7 A true navigation image can be captured from the corresponding viewpoint of the medical device 732 and / or the elongated device 731. A portion of the medical device 732 and / or the elongated device 731 may be visible in these true navigation images, depending on the position of the medical device 732 and the elongated device 731 relative to each other.

[0105] Refer again Figure 7 The real navigation images captured by the endoscopic imaging system 609 can facilitate navigation of the distal portion 738 of the slender device 731 through the patient's anatomy (e.g., via...). Figure 8The anatomical pathway 852) and / or delivery of the distal portion 738 of the elongated device 731 to a target location within the patient 603. In these and other embodiments, the real-world navigation images captured by the endoscopic imaging system 609 facilitate (i) navigation of the distal portion 737 of the medical device 732 beyond the distal portion 738 of the elongated device 731, (ii) delivery of the distal portion 737 of the medical device 732 to a target location within the patient 603, and / or (iii) visualization of the patient's anatomy during medical procedures. In some embodiments, each real-world navigation image captured by the endoscopic imaging system 609 may be timestamped and / or recorded in a medical device reference frame (X). M ,Y M Z M The location is associated with the position in the image. The real navigation image captured by the endoscope imaging system 609 can optionally be used to improve the point cloud of coordinate points generated by the position sensor system 608 (e.g., Figure 9 The registration between the point cloud (960) and the image data captured by the imaging system (618).

[0106] like Figure 7 As shown, the imaging system 618 is positioned near the patient 603 to obtain images of the patient 603 (e.g., Figure 8 The imaging system 618 provides a three-dimensional image of the anatomical region 850. In some embodiments, the imaging system 618 includes one or more imaging techniques, including CT, MRI, fluoroscopy, temperature recording, ultrasound, OCT, thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like. The imaging system 618 is configured to generate image data of the patient's anatomy before, during, and / or after the elongated device 731 extends within the patient 603. Thus, the imaging system 618 can be configured to capture preoperative, intraoperative, and / or postoperative three-dimensional images of the patient's anatomy. In these and other embodiments, the imaging system 618 can provide real-time or near-real-time images of the patient's anatomy.

[0107] Figure 11 This illustrates when the elongated device 731 of the medical device system 604 is in Figure 8 When the anatomical region extends within 850, it is covered by imaging system 618 ( Figure 7 Captured during the image capture period Figure 8 An example of intraoperative image data 1180 of a portion 1155 of the anatomical region 850. As shown, image data 1180 includes a graphic element 1181 representing the elongated device 731 and a graphic element 1182 representing the anatomical pathway 852 of the anatomical region 850.

[0108] All or part of the graphic elements 1181 and 1182 of image data 1180 may be segmented and / or filtered to generate a virtual three-dimensional model of the anatomical pathway 852 within a portion 1155 of the anatomical region 850 (with or without the medical device system 604). In some embodiments, graphic elements 1181 and 1182 may additionally or alternatively be segmented and / or filtered to be at least partially based on the imaging system 118 when the medical device system 604 is within the anatomical region 850. Figure 7 The captured images are used to generate an image point cloud (not shown) of the medical device system 604. During the segmentation process, pixels or voxels generated from the image data 1180 may be divided into segments or elements or labeled to indicate that they share certain characteristics or computational properties, such as color, density, intensity, and texture. The segments or elements can then be converted into an image point cloud of the anatomical model and / or the medical device system 604. Additionally or alternatively, segments or elements may be used to locate (e.g., compute) and / or define centerlines or other points extending along the anatomical pathway 852. The generated anatomical model and / or image point cloud may be two-dimensional or three-dimensional and may be displayed in an image reference frame (X). I ,Y I Z I Generated in ).

[0109] As mentioned above, relative to Figure 6 The healthcare system under discussion is 600 ( Figure 6 The display system 610 ( Figure 6 The system can display various images or representations of patient anatomy and / or medical device system 604, at least in part, based on data captured and / or generated by position sensor system 608, endoscopic imaging system 609, imaging system 618, and / or virtual visualization system 615. In various specific implementations, the system may utilize images and / or representations to assist operator 605. Figure 6 Image-guided medical procedures.

[0110] Figure 12 The display system 610 is based on various embodiments of the present technology. Figure 6 A schematic diagram of an exemplary display 1210 generated by [the diagram is missing]. As shown, display 1210 includes a real navigation image 1270, a synthesized virtual navigation image 1291 (also referred to as "synthetic virtual image 1291"), and a virtual navigation image 1292. The real navigation image 1270 can be [the diagram is missing]. Figure 10 The actual navigation image 1070 is substantially the same. Therefore, for example, the actual navigation image 1270 can be generated by the endoscopic imaging system 609 ( Figure 7 ) capture and provide to the display system 610 ( Figure 6The image is displayed on the display 1210 in real-time or near real-time. In the illustrated embodiment, the real navigation image 1270 shows the view from a distance of 732 ( Figure 7 The real patient anatomy (e.g., the ridge 1271 marking the branching points of the two anatomical pathways 852) is viewed from a distally oriented viewpoint 737.

[0111] Figure 12 The synthesized virtual image 1291 is displayed in the image reference frame (X). I ,Y I Z I ) and includes from the imaging system 618 ( Figure 7 ) captured Figure 8 The anatomical model 1250 is generated from image data of the anatomical region 850. The anatomical model 1250 is integrated with the position sensor system 608 ( Figure 7 The point cloud of coordinate points generated (e.g., Figure 9 The point cloud (960) was registered (i.e., dynamically referenced) to display in patient 603 ( Figure 7 Medical device systems within 604 (e.g., Figure 7 The elongated device 731) tracks the position, shape, pose, orientation, and / or movement of the anatomical model 1250 represented by the representation 1204. In some embodiments, the synthesized virtual image 1291 is controlled by the control system 612 ( Figure 6 ) virtual visualization system 615 ( Figure 6 The generation of a synthetic virtual image 1291 involves using an image reference frame (X) to generate the image. I ,Y I Z I ) and surgical reference system (X) S ,Y S Z S Registration and / or with the medical device reference system (X) M ,Y M Z M Registration. This registration can be performed by rotating, translating, or otherwise manipulating the coordinates of the point cloud captured by the position sensor system 608 through rigid and / or non-rigid transformations (e.g., Figure 9 The coordinates of point cloud 960 (point 962) are used to align the coordinates with the anatomical model 1250. Registration between the image and the surgical / instrument reference frame can be achieved, for example, by using point-based ICP techniques, as described in U.S. Provisional Patent Applications Nos. 62 / 205,440 and 62 / 205,433, the entire contents of which are incorporated herein by reference. In other embodiments, another point cloud registration technique may be used to achieve registration.

[0112] Based at least in part on registration, the virtual visualization system 615 may additionally or alternatively generate virtual navigation images (e.g., Figure 12 The virtual navigation image 1292 includes the medical device system 604 within the anatomical model 1250. Figure 8 The representation of 1204 is a virtual depiction of the patient's anatomy from the viewpoint of a virtual camera. Figure 12 In the illustrated embodiment, the virtual camera of the virtual navigation image 1292 is positioned at the distal portion 1237 of representation 1204, such that (i) the virtual viewpoint of the virtual navigation image 1292 points distally away from the distal portion 1237 of representation 1204, and (ii) representation 1204 is not visible within the virtual navigation image 1292. In other embodiments, the virtual visualization system 615 may (a) position the virtual camera at another location along representation 1204 and / or (b) with different orientations, such that the virtual navigation image 1292 has a corresponding virtual viewpoint. In some embodiments, depending on the position and orientation of the virtual camera and the position of the elongated device 731 and / or medical device 732 relative to each other within the patient 603, the virtual visualization system 615 may render at least a portion of the elongated device 731 and / or medical device 732 into a virtual representation (not shown) in the virtual navigation image 1292.

[0113] In some embodiments, the virtual visualization system 615 can be integrated with the image capture device 747 at the patient 603 ( Figure 7 The virtual camera is placed within the anatomical model 1250, with its position and orientation corresponding to the location and orientation within the model. For example... Figure 12 As further shown, the virtual navigation image 1292 is captured from the image capture device 747. Figure 7 The virtual patient's anatomical structures, such as the ridges 1201 marking the branching points of the two anatomical pathways 1252 of the anatomical model 1250, are shown in substantially the same locations as the real navigation image 1270. Therefore, the virtual navigation image 1292 provides... Figure 8 The rendering estimate of the patient's anatomical structures visible to the image capture device 747 at a given location within the anatomical region 850. Because the virtual navigation image 1292 is at least partially based on the registration of point cloud generated by the position sensor system 608 and image data captured by the imaging system 618, the correspondence between the virtual navigation image 1292 and the real navigation image 1270 provides insights into registration accuracy and can be used to improve registration. Furthermore, the endoscopic imaging system 609 ( Figure 7 The captured real navigation image (e.g., real navigation image 1270) can (a) provide information about the medical device system 604. Figure 6(a) information on the location and orientation within the patient 603, (b) information on portions of the anatomical region actually accessed by the medical device system, and / or (c) assistance in identifying patient anatomical structures (e.g., branching points of anatomical pathways) adjacent to the medical device system 604, any one or more of which may be used to improve the accuracy of registration.

[0114] like Figure 12 As further shown, the virtual navigation image 1292 may optionally include a navigation path overlay 1299. In some embodiments, the navigation path overlay 1299 is used to assist the operator 605 ( Figure 6 ) Medical device system 604 ( Figure 6 Navigation path 1299 can be used to navigate to a target location within the patient 603 via anatomical pathways through the anatomical region. For example, the navigation path cover 1299 may indicate an “optimal” path through the anatomical region for the operator 605 to follow in order to deliver the distal portions 737 and / or 738 of the medical device 732 and / or elongated device 731 to the target location within the patient 603, respectively. In some embodiments, the navigation path cover 1299 may be aligned with the centerline of the corresponding anatomical pathway or with another line along the corresponding anatomical pathway (e.g., its bottom).

[0115] C. Example

[0116] Several aspects of the present technology are illustrated in the following examples. Although several aspects of the present technology are illustrated in the examples relating to systems, computer-readable media, and methods, in other embodiments, any of these aspects of the present technology may be illustrated in the examples relating to any of the systems, computer-readable media, and methods.

[0117] 1. A system for planning medical procedures, the system comprising:

[0118] Processor; and

[0119] A memory, operatively coupled to the processor and storing instructions, which, when executed by the processor, cause the system to perform operations including—

[0120] Receive a three-dimensional model of the patient's anatomical region, and

[0121] Evaluating whether the three-dimensional model is suitable for the registration procedure to be performed in the anatomical region, wherein the evaluation includes—

[0122] Analyze the span of the three-dimensional model along at least two different directions.

[0123] Determine whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and

[0124] Registration is performed between the 3D model and the virtual registration dataset.

[0125] 2. The system of claim 1, wherein the at least two different directions include a first direction and a second direction, and wherein the operation further includes:

[0126] Calculate the first span value of the three-dimensional model along the first direction;

[0127] Calculate the second span value of the three-dimensional model along the second direction; and

[0128] Determine whether one or more of the first span value or the second span value exceeds a threshold.

[0129] 3. The system according to Embodiment 1 or Embodiment 2, wherein the at least one structure includes one or more of the following: trachea, main carina, left main bronchus, right main bronchus and subsegmental bronchus.

[0130] 4. The system according to Embodiment 1 or Embodiment 2, wherein the operation further includes determining whether the length of the at least one segmented component is greater than or equal to a minimum length.

[0131] 5. The system according to Embodiment 1 or Embodiment 2, wherein the at least one structure is a branch structure having multiple generations, and wherein the operation further includes determining whether the at least one segmented component covers a minimum number of generations of the branch structure.

[0132] 6. The system according to any one of embodiments 1 to 5, wherein the virtual registration dataset comprises multiple data points within the three-dimensional model along or near the survey trajectory.

[0133] 7. The system according to any one of embodiments 1 to 6, wherein the operation further includes performing multiple registrations between the three-dimensional model and the virtual registration dataset, wherein each registration is performed using a different initial seed.

[0134] 8. The system according to Embodiment 7, wherein the operation further includes determining the amount of difference between the registrations.

[0135] 9. The system according to any one of embodiments 1 to 6, wherein the operation further includes performing multiple registrations, and wherein each of the registrations is performed between the three-dimensional model and different virtual registration datasets.

[0136] 10. The system according to Example 9, wherein the operation further includes determining the accuracy of each registration.

[0137] 11. The system according to Embodiment 10, wherein the accuracy of each registration is evaluated at one or more portions of the three-dimensional model corresponding to one or more target sites in the anatomical region.

[0138] 12. The system according to any one of Embodiments 1 to 11, wherein the operation further includes outputting feedback indicating whether the three-dimensional model is suitable for the registration procedure.

[0139] 13. A non-transitory computer-readable medium having instructions stored thereon, the instructions causing the computing system to perform operations including: when executed by one or more processors of a computing system.

[0140] Receive a three-dimensional model of the patient's anatomical region; and

[0141] Evaluating whether the three-dimensional model is suitable for the registration procedure to be performed in the anatomical region, wherein the evaluation includes—

[0142] Analyze the span of the three-dimensional model along at least two different directions.

[0143] Determine whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and

[0144] Registration is performed between the 3D model and the virtual registration dataset.

[0145] 14. The non-transitory computer-readable medium according to embodiment 13, wherein the at least two different directions include a first direction and a second direction, and wherein the operation further includes calculating a first feature value and a second feature value, the first feature value corresponding to a first span of the three-dimensional model along the first direction, and the second feature value corresponding to a second span of the three-dimensional model along the second direction.

[0146] 15. The non-transitory computer-readable medium according to Embodiment 13 or Embodiment 14, wherein the at least one structure comprises one or more of the following: trachea, main carina, left main bronchus, right main bronchus, and subsegmental bronchus.

[0147] 16. The non-transitory computer-readable medium according to Embodiment 13 or Embodiment 14, wherein the operation further includes determining whether the length of the at least one segmented component is greater than or equal to a minimum length.

[0148] 17. The non-transitory computer-readable medium according to Embodiment 13 or Embodiment 14, wherein the at least one structure is a branch structure having multiple generations, and the operation further includes determining whether the at least one segmented component covers a minimum number of generations of the branch structure.

[0149] 18. The non-transitory computer-readable medium according to any one of embodiments 13 to 17, wherein the operation further includes generating the virtual registration dataset by:

[0150] Select a subset of the pathways in the 3D model;

[0151] Create a representation of the centerline of the subset of the pathway; and

[0152] Randomized noise is added to the representation.

[0153] 19. The non-transitory computer-readable medium according to any one of embodiments 13 to 18, wherein the operation further includes performing a plurality of registrations between the three-dimensional model and the virtual registration dataset, wherein each registration is performed using a randomized initial seed.

[0154] 20. The non-transitory computer-readable medium according to Embodiment 19, wherein the operation further includes determining the amount of difference between the registrations.

[0155] 21. The non-transitory computer-readable medium according to any one of embodiments 13 to 18, wherein the operation further includes performing a plurality of registrations, wherein each of the registrations is performed between the three-dimensional model and different virtual registration datasets.

[0156] 22. The non-transitory computer-readable medium according to Example 21, wherein the operation further includes determining the accuracy of each registration.

[0157] 23. The non-transitory computer-readable medium according to Example 22, wherein the accuracy of each registration is assessed at or near the location of the target lesion in the biopsy procedure.

[0158] 24. The non-transitory computer-readable medium according to any one of embodiments 13 to 23, wherein the operation further includes outputting feedback indicating whether the three-dimensional model is suitable for the registration procedure.

[0159] 25. A method comprising:

[0160] Receive a three-dimensional model of the patient's anatomical region; and

[0161] Evaluating whether the three-dimensional model is suitable for the registration procedure to be performed in the anatomical region, wherein the evaluation includes—

[0162] Analyze the span of the three-dimensional model along at least two different directions.

[0163] Determine whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and

[0164] Registration is performed between the 3D model and the virtual registration dataset.

[0165] 26. A system for planning medical procedures, the system comprising:

[0166] Processor; and

[0167] A memory operatively coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including:

[0168] Receive a three-dimensional model of the patient's anatomical region;

[0169] Generate a survey trajectory through a portion of the 3D model;

[0170] Generate a virtual registration dataset comprising multiple data points along or near the survey trajectory;

[0171] Perform multiple registrations between the 3D model and the virtual registration dataset; and

[0172] Based on the multiple registrations, assess whether the survey trajectory is suitable for the registration procedure to be performed in the anatomical region.

[0173] 27. The system according to embodiment 26, wherein the investigation trajectory is generated by selecting one or more pathways within the three-dimensional model, wherein the pathways are selected based on one or more of depth, accessibility, and uniqueness.

[0174] 28. The system according to embodiment 26, wherein the investigation trajectory is generated based on the location of the target site within the anatomical region.

[0175] 29. The system according to any one of embodiments 26 to 28, wherein the virtual registration dataset is configured to simulate location data generated by a medical device traveling along the survey trajectory.

[0176] 30. The system according to any one of embodiments 26 to 29, wherein each registration is performed using a different initial seed.

[0177] 31. The system according to any one of embodiments 26 to 30, wherein the operation further includes determining the amount of difference between the registrations.

[0178] 32. The system according to embodiment 31, wherein if the difference is less than or equal to a threshold, the survey trajectory is applicable to the registration procedure.

[0179] 33. The system according to embodiment 26, wherein the operation further includes:

[0180] Determine multiple survey trajectories through the portion of the three-dimensional model; and

[0181] For each of the aforementioned survey trajectories—

[0182] Generate a corresponding virtual registration dataset including multiple data points along or near the survey trajectory, and

[0183] Multiple registrations are performed between the 3D model and the corresponding virtual registration dataset.

[0184] 34. The system according to embodiment 33, wherein the operation further includes selecting a subset of the survey trajectories for use in the registration procedure.

[0185] 35. The system according to any one of embodiments 26 to 32, wherein the operation further includes outputting a graphical representation of the survey trajectory, wherein the graphical representation is configured to be displayed to the operator during the registration procedure.

[0186] 36. A non-transitory computer-readable medium having instructions stored thereon, the instructions causing the computing system to perform operations including: when executed by one or more processors of a computing system.

[0187] Receive a three-dimensional model of the patient's anatomical region;

[0188] Generate a survey trajectory through a portion of the 3D model;

[0189] Generate a virtual registration dataset comprising multiple data points along or near the survey trajectory;

[0190] Perform multiple registrations between the 3D model and the virtual registration dataset; and

[0191] Based on the multiple registrations, assess whether the survey trajectory is suitable for the registration procedure to be performed in the anatomical region.

[0192] 37. The non-transitory computer-readable medium according to Example 36, wherein the investigation trajectory is generated by selecting one or more pathways within the three-dimensional model.

[0193] 38. The non-transitory computer-readable medium according to embodiment 37, wherein the selection of the one or more pathways is based at least in part on input from an operator.

[0194] 39. The non-transitory computer-readable medium according to any one of embodiments 36 to 38, wherein the virtual registration dataset is configured to simulate survey data generated by a medical device traveling along the survey trajectory.

[0195] 40. The non-transitory computer-readable medium according to any one of embodiments 36 to 39, wherein each registration is performed using a randomized initial seed.

[0196] 41. The non-transitory computer-readable medium according to any one of embodiments 36 to 40, wherein the operation further includes determining the amount of difference between the registrations.

[0197] 42. The non-transitory computer-readable medium according to Example 41, wherein the survey trajectory is applicable to the registration procedure if the difference is less than or equal to a threshold.

[0198] 43. The non-transitory computer-readable medium according to Embodiment 36, wherein the operation further includes:

[0199] Determine multiple survey trajectories through the portion of the three-dimensional model; and

[0200] For each of the aforementioned survey trajectories—

[0201] Generate a corresponding virtual registration dataset including multiple data points along or near the survey trajectory, and

[0202] Multiple registrations are performed between the 3D model and the corresponding virtual registration dataset.

[0203] 44. The non-transitory computer-readable medium according to embodiment 43, wherein the operation further includes selecting a subset of the survey trajectories for use in the registration procedure.

[0204] 45. The non-transitory computer-readable medium according to any one of embodiments 36 to 42, wherein the operation further includes outputting a graphical representation of the subset of the survey trajectory, wherein the graphical representation is configured to be displayed to the operator during the registration procedure.

[0205] 46. ​​A method comprising:

[0206] Receive a three-dimensional model of the patient's anatomical region;

[0207] Generate a survey trajectory through a portion of the 3D model;

[0208] Generate a virtual registration dataset comprising multiple data points along or near the survey trajectory;

[0209] Perform multiple registrations between the 3D model and the virtual registration dataset; and

[0210] Based on the multiple registrations, assess whether the survey trajectory is suitable for the registration procedure to be performed in the anatomical region.

[0211] D. Conclusion

[0212] The systems and methods described herein may be provided in the form of a tangible and non-transitory machine-readable medium (such as a hard disk drive, hardware memory, etc.) on which instructions are recorded for execution by a processor or computer. The instruction set may include various commands that instruct a computer or processor to perform specific operations (such as the methods and processes of the various embodiments described herein). The instruction set may be in the form of a software program or application. Computer storage media may include volatile and non-volatile media, as well as removable and non-removable media, for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media may include, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technologies, CD-ROM, DVD, or other optical storage devices, disk storage devices, or any other hardware medium that can be used to store desired information and is accessible by components of the system. Components of the system may communicate with each other via wired or wireless communication. Components may be separate from each other, or various combinations of components may be integrated together into a monitor or processor or contained within a workstation having standard computer hardware (e.g., processors, circuits, logic circuits, memory, etc.). The system may include processing devices such as microprocessors, microcontrollers, integrated circuits, control units, storage media, and other hardware.

[0213] While many embodiments have been described above in the context of intrapulmonary navigation and medical procedure execution in patients, other applications and embodiments beyond those described herein are also within the scope of this technology. For example, unless otherwise stated or clearly indicated from the context, the devices, systems, methods, and computer program products of this technology can be used for a variety of image-guided medical procedures, such as those performed on, in, or near hollow patient anatomy structures, and more specifically, procedures for investigating, biopsiing, ablating, or otherwise treating tissue within and / or proximal to hollow patient anatomy structures. Thus, for example, the systems, devices, methods, and computer program products of this disclosure can be used for one or more medical procedures associated with other patient anatomy structures such as the bladder, urinary tract, GI system, and / or heart.

[0214] As used herein, the term “operator” should be understood to include any type of person capable of performing or assisting medical procedures, and therefore includes physicians, surgeons, doctors, nurses, medical technicians, other persons or users of the technologies disclosed herein, and any combination thereof. Furthermore, or alternatively, the term “patient” should be considered to include human and / or non-human (e.g., animal) patients on whom medical procedures are being performed.

[0215] Based on the description, it should be understood that, for illustrative purposes, specific embodiments of the present technology have been described herein, but well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the present technology. If any material incorporated herein by reference conflicts with this disclosure, this disclosure shall prevail. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, unless the word “or” is explicitly limited to referring only to a single item different from other items in a list of two or more items, its use in such lists shall be interpreted as including (a) any single item in the list, (b) all items in the list, or (c) any combination of items in the list. As used herein, the phrase “and / or”, such as in “A and / or B”, refers to A alone, B alone, and A and B. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, the terms “comprising,” “including,” “having,” and “with” are used throughout to indicate that at least the listed features are included, such that any larger number of the same features and / or other features of additional types are not excluded.

[0216] Furthermore, as used herein, the term "substantially" refers to the extent or degree of completeness or near-completeness of an action, characteristic, attribute, state, structure, item, or result. For example, a "substantially" closed object would mean that the object is completely or almost completely closed. In some cases, the exact permissible deviation from absolute completeness may depend on the specific circumstances. However, in general, near-completeness will have the same overall result as achieving absolute and complete completeness. When used in a negative sense, the use of "substantially" also applies to indicating the complete or near-complete absence of an action, characteristic, attribute, state, structure, item, or result.

[0217] The above detailed description of embodiments of this technology is not intended to be exhaustive or to limit the technology to the precise forms disclosed above. While specific embodiments and examples of this technology have been described above for illustrative purposes, various equivalent modifications can be made within the scope of this technology, as will be recognized by those skilled in the art. For example, although steps are presented in a given order, alternative embodiments may perform the steps in a different order. As another example, components of various technologies may be further divided into sub-components, and / or various components and / or functions of the technology may be combined and / or integrated. Furthermore, although advantages associated with certain embodiments of this technology have been described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments must exhibit such advantages to fall within the scope of this technology.

[0218] It should also be noted that other embodiments besides those disclosed herein are also within the scope of this technology. For example, embodiments of this technology may have different configurations, components, and / or procedures than those shown or described herein. Furthermore, those skilled in the art will understand that these and other embodiments may lack certain configurations, components, and / or procedures shown or described herein without departing from this technology. Therefore, this disclosure and associated technologies may cover other embodiments not explicitly shown or described herein.

Claims

1. A system for planning medical procedures, the system comprising: processor; as well as A memory operatively coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including: Receive a three-dimensional model of the patient's anatomical region, and Evaluating whether the three-dimensional model is suitable for use in a registration procedure to be performed in the anatomical region, wherein the evaluation includes— The analysis covers the span of the three-dimensional model along at least two different directions, where the span corresponds to the extent of the anatomical region covered by the three-dimensional model. Determine whether the three-dimensional model includes at least one segmental component corresponding to at least one structure of the anatomical region, and Registration is performed between the 3D model and the virtual registration dataset.

2. The system of claim 1, wherein the at least two different directions include a first direction and a second direction, and wherein the operation further includes: Calculate the first span value of the three-dimensional model along the first direction; Calculate the second span value of the three-dimensional model along the second direction; as well as Determine whether one or more of the first span value or the second span value exceeds a threshold.

3. The system according to claim 1, wherein the at least one structure comprises one or more of the following: trachea, main carina, left main bronchus, right main bronchus and subsegmental bronchus.

4. The system of claim 1, wherein the operation further includes determining whether the length of the at least one segmented component is greater than or equal to a minimum length.

5. The system of claim 1, wherein the at least one structure is a branch structure having multiple generations, and wherein the operation further includes determining whether the at least one segmented component covers a minimum number of generations of the branch structure.

6. The system of claim 1, wherein the virtual registration dataset comprises a plurality of data points within the three-dimensional model along or near the survey trajectory.

7. The system of claim 1, wherein the operation further comprises performing multiple registrations between the 3D model and the virtual registration dataset, wherein each registration is performed using a different initial seed.

8. The system of claim 7, wherein the operation further includes determining the amount of difference between the registrations.

9. The system of claim 1, wherein the operation further comprises performing a plurality of registrations, and wherein each of the registrations is performed between the 3D model and different virtual registration datasets.

10. The system of claim 9, wherein the operation further includes determining the accuracy of each registration.

11. The system of claim 10, wherein the accuracy of each registration is evaluated at one or more portions of the three-dimensional model corresponding to one or more target sites in the anatomical region.

12. The system of claim 1, wherein the operation further includes outputting feedback indicating whether the three-dimensional model is suitable for the registration procedure.

13. A system for planning medical procedures, the system comprising: processor; as well as A memory operatively coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including: Receive a three-dimensional model of the patient's anatomical region; Generate a survey trajectory through a portion of the 3D model; Generate a virtual registration dataset comprising multiple data points along or near the survey trajectory; Perform multiple registrations between the 3D model and the virtual registration dataset; as well as Based on the multiple registrations, assess whether the survey trajectory is suitable for use in the registration procedure to be performed in the anatomical region.

14. The system of claim 13, wherein the survey trajectory is generated by selecting one or more pathways within the three-dimensional model, wherein the pathways are selected based on one or more of depth, accessibility, and uniqueness.

15. The system of claim 13, wherein the investigation trajectory is generated based on the location of the target site within the anatomical region.

16. The system of claim 13, wherein the virtual registration dataset is configured to simulate location data generated by a medical device traveling along the survey trajectory.

17. The system of claim 13, wherein each registration is performed using a different initial seed.

18. The system of claim 13, wherein the operation further includes determining a difference amount between the registrations.

19. The system of claim 18, wherein the survey trajectory is suitable for use in the registration procedure if the difference is less than or equal to a threshold.

20. The system of claim 13, wherein the operation further comprises: Determine multiple survey trajectories through the portion of the three-dimensional model; as well as For each of the aforementioned survey trajectories— Generate a corresponding virtual registration dataset including multiple data points along or near the survey trajectory, and Multiple registrations are performed between the 3D model and the corresponding virtual registration dataset.

21. The system of claim 20, wherein the operation further includes selecting a subset of the survey trajectories for use in the registration procedure.

22. The system of claim 13, wherein the operation further includes outputting a graphical representation of the survey trajectory, wherein the graphical representation is configured to be displayed to the operator during the registration procedure.

Images