Systems and related methods for planning and performing a biopsy procedure

Abstract

Disclosed herein are devices, systems, methods, and computer program products for planning and performing medical procedures. In some embodiments, a system for planning a medical procedure includes a processor and a memory operably coupled to the processor. The memory can store instructions that, when executed by the processor, cause the system to perform operations including receiving image data of an anatomical region including a plurality of lymph nodes and a target lesion, generating a three-dimensional model of the anatomical region by segmenting the image data, selecting a subset of lymph nodes to be biopsied during the medical procedure based at least in part on a location of the target lesion in the three-dimensional model, and determining a sequence of locations for navigating a biopsy device to the subset of lymph nodes during the medical procedure.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 064,111, filed on August 11, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to systems, methods, and computer program products for planning and performing biopsy procedures. Background Technology

[0004] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects. Such techniques can be performed through natural openings in the patient's anatomy or through one or more surgical incisions. Through these natural openings or incisions, the operator can insert minimally invasive medical instruments to reach target tissue locations. Minimally invasive medical instruments include devices such as therapeutic instruments, diagnostic instruments, biopsy instruments, and surgical instruments. Medical instruments can be inserted through anatomical channels and navigate toward regions of interest within the patient's anatomy. Image-aided navigation can be used with respect to the anatomical channels. Improved systems and methods are needed to accurately perform registration between images of the medical instruments and the anatomical channels. Summary of the Invention

[0005] This document discloses apparatus, systems, methods, and computer program products for planning medical procedures, including selecting lymph nodes and / or groups of lymph node stations to be biopsied and determining sequences for biopsiing the selected lymph nodes and / or groups of lymph node stations. In some embodiments, the system for planning the medical procedure includes a processor and a memory operatively coupled to the processor. The memory may store instructions that, when executed by the processor, cause the system to perform operations including receiving image data of an anatomical region of a patient. The anatomical region may include a plurality of lymph nodes and a target lesion. These operations may further include generating a three-dimensional model of the anatomical region by segmenting the image data. The three-dimensional model may include a plurality of segmented components corresponding to the plurality of lymph nodes and the target lesion. The operations may further include: selecting a subset of lymph nodes to be biopsied during the medical procedure based at least in part on the location of the target lesion in the three-dimensional model; and determining sequences for navigating a biopsy device to the subset of lymph nodes during the medical procedure.

[0006] In these and other embodiments, a non-transitory computer-readable medium may store instructions thereon that, when executed by one or more processors of a computing system, cause the computing system to perform operations including receiving image data of an anatomical region of a patient, the anatomical region having multiple lymph nodes and a target lesion. The operations may further include generating a three-dimensional model of the anatomical region by segmenting the image data, the three-dimensional model having multiple segmented components corresponding to the multiple lymph nodes and the target lesion. The operations may further include: selecting a subset of lymph nodes to be biopsied during the medical procedure based at least in part on the location of the target lesion in the three-dimensional model; and determining a sequence for navigating a biopsy device to the subset of lymph nodes during the medical procedure.

[0007] In these and other embodiments, a method may include: receiving image data of an anatomical region of a patient, the anatomical region having multiple lymph nodes and a target lesion; generating a three-dimensional model of the anatomical region by segmenting the image data, the three-dimensional model having multiple segmented components corresponding to the multiple lymph nodes and the target lesion; selecting a subset of lymph nodes to be biopsied during the medical procedure based at least in part on the location of the target lesion in the three-dimensional model; and determining a sequence for navigating a biopsy device to the subset of lymph nodes during the medical procedure. Attached Figure Description

[0008] Many aspects of this disclosure can be better understood by referring to the following accompanying drawings. The components in the drawings are not necessarily drawn to scale. Instead, the focus is on clearly illustrating the principles of this disclosure. The drawings should not be construed as limiting this disclosure to the specific embodiments depicted, but are intended for explanation and understanding only.

[0009] Figures 1A-1C The illustration shows the lymph node stations in the patient's thoracic region.

[0010] Figure 2 This is a flowchart illustrating various embodiments of the present technology for planning biopsy procedures.

[0011] Figure 3 These are computed tomographic images of the patient's thoracic region.

[0012] Figure 4 This is a flowchart illustrating various embodiments of the present technology for performing a biopsy procedure.

[0013] Figure 5 This is a schematic representation of a robot or remotely operated medical system configured according to various embodiments of the present technology.

[0014] Figure 6This is a schematic representation of a manipulator assembly, medical device system, and imaging system configured according to various embodiments of the present technology.

[0015] Figure 7 It extends within the patient's anatomical region according to various embodiments of this technology. Figure 6 A schematic representation of a part of a medical device system.

[0016] Figure 8 The illustration shows multiple coordinate points forming a point cloud, which represents a configuration according to various embodiments of the present technology. Figure 7 The shape of said part of the medical device system.

[0017] Figure 9 The illustrations depict various embodiments of the present technology, from which... Figure 8 Extending within the anatomical region Figure 7 A true navigation image of the patient's anatomy at the viewing point of the medical device system.

[0018] Figure 10 The illustrations depict various embodiments of the present technology. Figure 7 The aforementioned part of the medical device system in Figure 8 Intraoperative image of a portion of the anatomical region as it extends within the anatomical region.

[0019] Figure 11 This is a schematic representation of a display system for displaying composite virtual navigation images according to various embodiments of the present technology, wherein... Figure 6 and Figure 7 The medical device system was registered to Figure 7 The anatomical model of the anatomical region, the virtual navigation image of the virtual patient anatomy, and the real navigation image of the real patient anatomy within the anatomical region. Detailed Implementation

[0020] This disclosure relates to apparatus, systems, methods, and computer program products for planning and / or performing biopsy procedures within the anatomical region of a patient. In some embodiments, for example, a biopsy procedure is performed on a cancer patient to determine the anatomical extent of the cancer (“cancer staging”). The cancer staging process may include determining how much cancer is in the patient’s body, where the cancer is located, and / or whether the cancer has spread from its original site (e.g., to the lymphatic drainage system, vascular system, other organ systems, etc.). For example, in the context of lung cancer (e.g., non-small cell lung cancer), the staging process may include biopsies of multiple lymph nodes and / or groups of lymph node stations near the airways of the lung (i.e., clinically defined lymph node groups). Accurate staging is important for assessing patient prognosis and selecting appropriate treatment. For example, in some cases, surgery may be recommended for early-stage lung cancer, but may be contraindicated for advanced-stage lung cancer. However, operators performing biopsy procedures may not be aware of clinically recommended staging guidelines (e.g., which lymph nodes and / or lymph node stations should be biopsied), may not know how to apply the guidelines to the pathology of a particular patient, and / or may not know how to effectively perform biopsy procedures while adhering to the guidelines.

[0021] Therefore, this technology can assist operators in planning and / or performing biopsy procedures by: (i) identifying which lymph nodes and / or groups of lymph node stations (collectively, “lymph site”) should be biopsied, and (ii) determining the sequence for biopsiing the selected lymph node sites. In some implementations, for example, a three-dimensional (3D) model of the patient’s anatomical region is generated and segmented into components representing anatomical structures such as airways, lungs, lymph nodes, blood vessels, and / or target lesions (e.g., from preoperative image data). The segmented model can be used to select lymph node sites to be biopsied (e.g., based on the location of the target lesion, the location of the lymph node site, lymphatic drainage pathways, clinical staging guidelines, etc.). The model can also be used to determine the sequence for biopsiing the selected lymph node sites in an efficient manner while minimizing the risk of cross-contamination. During the biopsy procedure, the selected lymph node sites and biopsy sequence can be displayed to provide visual guidance to the operator and facilitate navigation within the patient’s anatomy. This technology is expected to increase operator adherence to clinical staging guidelines and improve the efficiency and accuracy of the staging process, which may help improve patient outcomes.

[0022] A. Implementation of technologies for planning and performing biopsy procedures

[0023] This technology generally relates to planning and / or performing medical procedures, such as biopsy procedures for diagnosing a patient's disease or condition. In some implementations, for example, the system described herein is configured to plan a biopsy procedure for staging lung cancer (e.g., non-small cell lung cancer). Lung cancer staging can be defined as follows:

[0024] Stage 0: The cancer has not yet spread from its original site (“in situ”).

[0025] Stage I: A small primary tumor located in only one lung, which has not spread to any lymph nodes and has not metastasized.

[0026] • Stage II: A large primary tumor that has not yet spread to any lymph nodes, or a small tumor in the lung that has spread to nearby lymph nodes.

[0027] • Stage III: Cancer is present in the lungs and mid-chest lymph nodes (“locally advanced disease”). There are two subtypes of Stage III: IIIA (cancer has spread only to the lymph nodes on the same side of the chest where the cancer originated) and IIIB (cancer has spread to the lymph nodes on the opposite side of the chest and / or above the clavicle).

[0028] Stage IV: The cancer has spread to both lungs, the fluid surrounding the lungs, or other parts of the body, such as the liver or other organs (“advanced disease”).

[0029] As discussed above, accurate lung cancer staging can be important for assessing patient prognosis and / or determining appropriate treatment options. For example, surgery may be recommended for patients with stage 0 or I cancer; for patients with stage II cancer, treatment (e.g., chemotherapy, radiotherapy, chemoradiotherapy, immunotherapy) may be recommended before surgery; and for patients with stage III or IV cancer, treatment without surgery (e.g., chemotherapy, radiotherapy, chemoradiotherapy, immunotherapy) may be recommended.

[0030] In some implementations, lung cancer staging involves obtaining tissue samples from one or more lymph node sites within the patient's thoracic region. As mentioned above, the presence of cancer cells in certain lymph node sites (e.g., lymph node stations in the mid-chest, lymph node stations on one side of the chest opposite the original cancer site, etc.) may be associated with a more advanced stage of cancer. Therefore, the extent and severity of cancer can be assessed by systematically sampling lymph nodes from different lymph node stations within the thoracic region.

[0031] Figures 1A-1C The diagram illustrates the lymph node station groupings in the patient's thoracic region (100 mm). From... Figures 1A-1C As can be seen from Table 1 below, the lymph nodes in the thoracic region 100 can be grouped into 14 different lymph node station groups (station groups 1R-14). These lymph node station groups can be further divided into 7 anatomical regions (102-114, by...). Figures 1A-1C (The dashed line indicates this).

[0032] Table 1: Thoracic Lymph Node Station Groups

[0033]

[0034]

[0035] As described in more detail below, the system described herein can be configured to select Figures 1A-1C One or more lymph node sites, as shown in Table 1, are used for biopsies during medical procedures for lung cancer staging.

[0036] Figure 2 This is a flowchart illustrating a method 200 for planning a biopsy procedure according to various embodiments of the present technology. Method 200 is illustrated as a set of steps or processes 210-240. All or a subset of the steps of method 200 may be implemented by a computing system or device, such as a workstation configured to perform preoperative planning of a medical procedure. Alternatively or in combination, all or a subset of the steps of method 200 may be implemented by a control system of a medical device system or device, including various components or devices of a robot or remote operating system, as described in more detail below. The computing system for implementing method 200 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform operations according to steps 210-240.

[0037] Method 200 begins at step 210 with receiving image data of an anatomical region of the patient. The image data may include, for example, computed tomography (CT) data, magnetic resonance imaging (MRI) data, fluorescence examination data, thermal imaging data, ultrasound data, optical coherence tomography (OCT) data, thermal image data, impedance data, laser image data, nanotube X-ray image data, and / or other suitable data representing the anatomical region to be biopsied. The image data may correspond to two-dimensional (2D), 3D, or four-dimensional (e.g., time-based or velocity-based information) images. In some embodiments, for example, the image data includes 2D images from multiple viewpoints, which may be combined to form a pseudo-3D image. The image data may be preoperative image data obtained prior to performing a biopsy on the patient.

[0038] At step 220, a 3D model of the anatomical region is generated by segmenting the image data. This model can represent the anatomical region in which a biopsy procedure will be performed (e.g., the airways of a patient's lungs) and can represent the location, shape, and connectivity of passageways and other structures (e.g., lymph nodes, target lesions, blood vessels, etc.) within the region. In some embodiments, the model includes multiple segmented components corresponding to anatomical structures or features within the anatomical region. Examples of anatomical structures or features that may be included in the model include one or more of the following: airways (e.g., trachea, main carina, left main bronchus, right main bronchus, and / or subsegmental bronchi), lymph nodes (e.g., related to...). Figure 1A-1CThe target lesions include any lymph node sites described in Table 1, blood vessels (e.g., aorta, superior vena cava, pulmonary trunk), lungs, and / or target lesions (e.g., known or suspected cancerous tumors or other tissue sites).

[0039] A 3D model can be generated by segmenting graphical elements in image data that represent or otherwise correspond to anatomical structures or features. During segmentation, pixels or voxels generated from the image data can be divided into segments or elements and / or labeled to indicate that they share certain features or computational properties such as color, density, intensity, and material. The segments or elements associated with the patient's anatomical features are then transformed into a segmented anatomical model generated in a model or image reference frame. To represent this model, the segmentation process can delineate the groups of voxels representing the anatomical structures and then apply functions such as the traveling cube function to generate a 3D surface surrounding the voxels. The model can be created by generating meshes, volumes, or voxel maps. Additionally or alternatively, the model can include a centerline model comprising a set of interconnected line segments or points extending through the center of the modeling channel. In the case where the model includes a centerline model containing a set of interconnected line segments, these line segments can be transformed into a cloud or a set of points. By transforming the line segments, the desired number of points corresponding to the interconnected line segments can be selected manually or automatically. U.S. Patent Application Publication No. 2020 / 0030044 (filed April 18, 2018) (discloses a graphical user interface for planning procedures); and U.S. Patent No. 10,373,719 (filed September 3, 2015) (discloses a system and method for preoperative modeling) further describe in detail various systems and methods for segmenting anatomical structures from image data; both patents are incorporated herein by reference in their entirety.

[0040] In some embodiments, step 220 includes multiple lymph nodes in segmented image data. Segmentation of lymph nodes may include, for example, analyzing image data to identify graphic elements corresponding to lymph nodes rather than other anatomical structures such as airways, blood vessels, etc. In some embodiments, lymph nodes are identified based on features such as shape (e.g., oval or circular, rather than tubular), size (e.g., approximately 1 cm in diameter), and / or location (e.g., near airways and / or within anatomical areas corresponding to lymph node station groups). Once identified, the lymph nodes can be segmented into individual model components as discussed above. Optionally, step 220 may also include assigning the lymph nodes of each segment to lymph node station groups (e.g., based on the location of the lymph nodes relative to other anatomical structures).

[0041] Lymph node segmentation procedures can be performed in various ways, such as automatically (e.g., without any operator input to identify lymph nodes), semi-automatically (e.g., with some operator input), or manually by an operator. For example, machine learning algorithms can be used to perform automatic lymph node segmentation, such as deep learning algorithms (e.g., convolutional neural networks or other types of neural networks)—which have been trained to identify and segment individual lymph nodes from CT scans or other image data of a patient's anatomy. Training the machine learning algorithm can be performed, for example, via supervised learning techniques using large image datasets in which lymph nodes have already been identified. Once trained, the machine learning algorithm can automatically identify graphic elements in the image data that may correspond to lymph nodes and can segment those graphic elements to create individual model parts, as discussed above. Optionally, the machine learning algorithm can also be trained to automatically identify and segment other anatomical structures (e.g., airways, lesions, blood vessels, etc.).

[0042] A semi-automatic lymph node segmentation process may involve some steps performed automatically and others based on input from the operator. For example, the operator may select one or more locations in image data that include lymph nodes and / or correspond to lymph node stations, and the computational system may analyze the selected locations to identify and segment the lymph nodes at those locations. In some implementations, the operator provides input indicating the selected locations (e.g., via a suitable graphical user interface), such as by clicking or otherwise marking points corresponding to lymph nodes, drawing boundaries around the edges or surfaces of lymph nodes, selecting regions of image data that include lymph nodes and / or lymph node stations, or any other suitable process. The system can then use the input from the operator as a starting point to automatically detect one or more lymph nodes in the image data. For example, the system may use edge detection algorithms, machine learning algorithms, etc., to search for objects in the image locations indicated by the operator that may correspond to lymph nodes. The results may be displayed to the operator for approval, rejection, modification, or other feedback. Alternatively or in combination, the system may automatically provide an initial selection of potential lymph nodes, which the operator may accept, reject, or modify. Optionally, if identifying individual lymph nodes is challenging (e.g., due to low signal-to-noise ratio in the image data), the system and / or operator may instead identify and segment the image data corresponding to lymph node stations (e.g., as mentioned above regarding...). Figures 1A-1C The location of the lymph node station group and / or anatomical area is described, rather than identifying and segmenting individual lymph nodes.

[0043] For example, Figure 3This is a CT image 300 of the patient's thoracic region. Image 300 includes graphic elements representing various anatomical structures such as: airways (e.g., left main bronchus 302, right main bronchus 304), left lung 306, right lung 308, blood vessels (e.g., ascending aorta 310, descending aorta 312, superior vena cava 314, pulmonary trunk 316), and multiple lymph nodes (e.g., prevascular lymph nodes 318 (station group 3A), subaortic lymph nodes 320 (station group 5), subcarinal lymph nodes 322 (station group 7), and hilar lymph nodes 324 (station group 10)). The above is about... Figure 2 The process described in step 220 can be used to automatically, semi-automatically, or manually segment the image 300 to create a 3D model of the thoracic region. This model may include segmented model components representing airways (e.g., left main bronchus 302, right main bronchus 304) and lymph nodes 318-324. Optionally, the model may also include segmented model components corresponding to lungs (e.g., left lung 306, right lung 308), blood vessels (e.g., ascending aorta 310, descending aorta 312, superior vena cava 314, pulmonary trunk 316), and / or other anatomical structures such as the target lesion (not shown).

[0044] Return to reference Figure 2 At step 230, one or more lymph node sites are selected. As discussed above, biopsies can be taken from the selected lymph node sites to determine the stage of the patient's cancer, for example, by assessing whether the cancer has spread from the initial site (e.g., the target lesion) to the lymph node sites. For example, if a lymph node located near the target lesion tests negative for malignant cells, the cancer may be early-stage. Conversely, if a lymph node located far from the target tests positive for malignant cells, the cancer may be late-stage. The number of positive lymph node sites may also be related to the degree of cancer; for example, if few or no positive results are found in the biopsied lymph node sites, the cancer may be early-stage, while if most or all of the biopsied lymph node sites test positive, the cancer may be late-stage. Therefore, step 230 may involve selecting which lymph node sites should be biopsied to accurately stage the patient's cancer. Step 230 may involve selecting one or more individual lymph nodes, one or more lymph node stations, or a combination thereof. In some embodiments, for example, one or more lymph node stations are selected without specifying any particular lymph node within that station group to be biopsied. In other implementations, one or more specific lymph nodes may be selected, with or without specifying a corresponding lymph node station group.

[0045] The selection of lymph node sites may be based on the specific pathology of the patient, such as the location of the target lesion. For example, step 230 may include selecting one or more lymph node sites (e.g., signal lymph nodes) located downstream of the lymphatic drainage pathway along the target lesion. Drainage pathways within the patient's anatomical region may be determined based on clinical guidelines or studies, the patient's specific anatomy and physiology, and / or any other suitable considerations. For example, in the thoracic region, the drainage pathway may generally run from a group of lymph node stations in the peripulmonary portion of the lung to a group of lymph node stations in the mediastinum between the lungs. Drainage pathways generally do not cross lobes or reach the other side of the surrounding lung. Therefore, if the target lesion is located in the peripulmonary portion of the lung (e.g., in the mediastinum), the drainage pathway may be chosen based on the patient's specific pathology, such as the location of the target lesion. Figure 1A If the lymph nodes are located near the 13R station group, then step 230 may involve selecting some or all of the lymph node sites located between the target lesion and the mediastinum (e.g., near the 13R station group). Figure 1A The 12R, 11R, and 10R station groups), and the mediastinal lymph node sites (e.g., Figure 1A (4R station group and 7 station group). Optionally, the lymph node site upstream of the target lesion can be omitted (e.g., Figure 1A (14R station group) and / or lymph node sites not located along the drainage path from the target lesion (e.g., Figure 1A (The 14L, 13L, and 12L station groups). Those skilled in the art will understand that the drainage path can vary to some extent from patient to patient and may skip lymph node station groups and / or pass between lymph node station groups. Therefore, the selection process in step 230 can be customized based on each patient's specific anatomy and / or other clinical considerations.

[0046] Alternatively or in combination, lymph node sites can be selected based on their location, such as their proximity to the target lesion. Proximity can be assessed quantitatively (e.g., based on the measured distance between the target lesion and the lymph node site) and / or qualitatively (e.g., whether the lymph node site is on the same side of the chest as the target lesion, in the middle of the chest, or on the opposite side of the chest). In some embodiments, step 230 involves selecting lymph node sites with different proximities to the target lesion. For example, the selection may include at least one lymph node site on the same side of the chest as the target lesion (ipsilateral or "N1 node"), at least one lymph node site in the middle of the chest ("N2 node"), and at least one lymph node site on the opposite side of the chest as the target lesion (contralateral or "N3 node"). For example, if the target lesion is close to station group 13R ( Figure 1A If the step 230 involves selecting the ipsilateral lymph node site (e.g., Figure 1A (12R station group, 11R station group, 10R station group and 4R station group) and / or contralateral lymph node sites (e.g., Figure 1A(12L station group, 11L station group, 10L station group, and 4L station group). Optionally, step 230 may include selecting more lymph node sites close to the target lesion (e.g., N1 node) and fewer lymph node sites far from the target lesion (e.g., N2 node, N3 node). Lymph node sites that are very far from the target lesion or otherwise unlikely to have metastasis may be excluded.

[0047] In some implementations, lymph node sites are selected based on predictive modeling. For example, predictive modeling can be used to predict which lymph node sites are likely to have metastases based on the location of the target lesion, and can be performed using statistical models, machine learning models, or any other suitable techniques. The predictive model can generate a risk score for each lymph node site, representing the probability that cancer has metastasized to that particular site. Lymph node sites associated with higher risk scores can be selected for biopsy, while lymph node sites associated with lower risk scores can be excluded.

[0048] Alternatively or in combination, lymph node sites may be selected based on other parameters, such as one or more of the following: accessibility of the biopsy apparatus (e.g., lymph nodes located near airways that are too narrow, too tortuous, or otherwise inaccessible to the biopsy apparatus), lymph node size (e.g., lymph nodes with a diameter greater than 1 cm and / or a short axis diameter greater than 5 mm), lymph node shape (e.g., lymph nodes with abnormal shapes), proximity to vulnerable anatomical structures (e.g., lymph node sites that are too close to major blood vessels, pleura, alveoli, etc.), spatial relationship between the lymph node site and other anatomical structures (e.g., airways, lungs, pulmonary nodules), patient-specific physiology, clinical guidelines or studies (e.g., regarding cancer staging procedures), etc.

[0049] Any suitable number and combination of lymph node sites can be selected. In some embodiments, step 230 involves selecting at least one, two, three, four, five, or more different lymph node sites to be biopsied (e.g., at least one, two, three, four, five, or more different lymph node stations). Optionally, step 230 may involve selecting a certain number of lymph nodes to be biopsied for each station (e.g., at least one, two, three, or more lymph nodes). In some embodiments, certain lymph node sites are always selected, such as mediastinal lymph nodes (e.g., Figure 1A (Some or all of the lymph node station groups 2L, 2R, 3A, 3P, 4L, 4R, 8L, 8R, 9L, and 9R).

[0050] Lymph node site selection can be performed automatically, semi-automatically, or manually. For example, the system can automatically analyze the location of the target lesion, lymph nodes, and / or other segmented anatomical structures in a 3D model and apply any of the selection parameters described above to select a subset of lymph nodes for biopsy. Selection parameters can be determined by the system (e.g., coded in the system software) or can be manually set by the operator (e.g., the operator can choose which selection parameters should be applied). Once the system has selected the lymph node sites, the selection can be output to the operator for approval, rejection, or modification. Optionally, the operator can provide user input indicating which lymph node sites should be biopsied (e.g., via a graphical user interface). For example, the operator can manually select certain anatomical areas or regions, and the system can subsequently identify and select lymph node sites within these areas. The operator can also manually select specific lymph node sites to be biopsied.

[0051] At step 240, a sequence for biopsiing the selected lymph node sites is determined. This sequence may indicate the order in which the selected lymph node sites should be biopsied during the procedure. In some embodiments, the sequence is configured to reduce the likelihood of cross-contamination between lymph node sites (e.g., the transfer of malignant cells to non-cancerous lymph nodes). Therefore, the sequence may include biopsiing less likely malignant lymph node sites (e.g., sites further from and / or upstream of the target lesion) before more likely positive lymph node sites (e.g., sites closer to and / or downstream of the target lesion). For example, the biopsy sequence may include sampling the N3 segment before the N2 segment and sampling the N2 segment before the N1 segment. As another example, the biopsy sequence may include sampling peripheral lymph node sites before central lymph node sites.

[0052] In some implementations, the sequence is at least partially based on a trajectory used to navigate the biopsy device to the target lesion. This trajectory may be a predetermined route traversing anatomical pathways to the target lesion and may be generated automatically, semi-automatically, or manually during preoperative planning of the biopsy procedure. In such implementations, if the trajectory passes near one or more selected lymph node sites, biopsy samples can be collected from these sites in the order they are encountered (e.g., as the biopsy device moves toward the target lesion). Alternatively or in combination, the sequence may also be determined based on any of the following considerations: minimizing backtracking, minimizing the total distance traversed by the biopsy device, minimizing the total time of the biopsy procedure, avoiding pathways that are inaccessible or otherwise difficult to navigate for the biopsy device, and / or avoiding areas near vulnerable anatomical structures (e.g., large blood vessels, pleura, alveoli).

[0053] In some embodiments, step 240 further includes generating at least one suggested path for navigating the biopsy device within the anatomical region to reach each selected lymph node site. This path can be configured to traverse anatomical pathways between the selected lymph node sites such that the biopsy device reaches these sites in the correct sequence simply by following the path. Optionally, the path may also include a route for navigating the biopsy device to the target lesion, as discussed above. The path can be generated in various ways, such as automatically, semi-automatically, or manually. For example, an operator can manually create some or all of the path by selecting channels (e.g., airways) within the model via a suitable graphical user interface. Alternatively or in combination, some or all of the path can be automatically generated by the system. For example, the system can use the model to identify and select channels (e.g., those with a sufficiently large diameter) located near the selected lymph node sites and accessible to the biopsy device. In some embodiments, the system automatically generates the suggested path, and the operator can approve the path or manually revise it (e.g., by adding, deleting, or otherwise modifying portions of the path). Conversely, the operator can manually create the path, and the system can automatically revise the path or propose revisions for operator approval.

[0054] Optionally, the path may include an exit location near each selected lymph node site. The exit location may correspond to the point where the biopsy device leaves the channel and reaches the lymph node site (e.g., through the channel lumen at the exit location). For example, the exit location may be the point in the channel closest to the site. The path may also include path segments connecting the exit location to the lymph node site (referred to herein as "exit segments"). The length of the exit segment may be configured to be less than or equal to the maximum insertion depth of the biopsy device. For example, some biopsy needles may not be able to biopsies targets more than 3 cm from the exit location.

[0055] In some implementations, the path is configured to avoid one or more vulnerable anatomical structures, such as blood vessels, pleura, alveoli, etc. For example, puncture of the pleura during a biopsy procedure may result in pneumothorax and / or other conditions that are dangerous to the patient. Therefore, the path, exit location, and / or exit segmentation can be constrained to avoid vulnerable anatomical structures. This can be achieved, for example, by defining one or more hazard fences around vulnerable anatomical structures to indicate locations where the path cannot reach and / or overlap. Hazard fences can be created automatically, semi-automatically, or manually by the operator. Other techniques for creating paths within anatomical regions are described in further detail in U.S. Patent Application Publication No. 2020 / 0030044 (filed April 18, 2018) (disclosing a graphical user interface for planning procedures), the entire contents of which are incorporated herein by reference.

[0056] The output of method 200 (e.g., a 3D model, selected lymph node site, biopsy sequence, and / or path) can be saved as part of the planning for the biopsy procedure (e.g., as one or more digital files). In an embodiment where the plan is created on a preoperative planning workstation, the plan can be transferred to the medical device system to be used to perform the biopsy procedure. Subsequently, during the biopsy procedure, the 3D model, selected lymph node site, and / or biopsy sequence can be displayed to the operator (e.g., via a graphical user interface) to provide visual guidance and instructions for navigating to the selected biopsy site, as described in more detail below.

[0057] Although the steps of method 200 are discussed and described in a specific order, those skilled in the art will recognize that method 200 can be modified and still fall within the scope of these and other embodiments of the present technology. In other embodiments, for example, method 200 can be performed in a different order; for example, any step of method 200 can be performed before, during, and / or after any other step of method 200. Furthermore, steps can be omitted. Figure 2 One or more steps of method 200 (e.g., step 220 or 240) are shown. Optionally, one or more steps of method 200 may be repeated. For example, step 240 may be performed multiple times to generate multiple different sequences and / or paths for navigating the biopsy device to the selected lymph node site. The operator can then select the sequence and / or path to use, for example, during the preoperative planning phase or during the actual biopsy procedure.

[0058] Figure 4 This is a flowchart illustrating a method 400 for performing a biopsy procedure according to various embodiments of the present technology. In some embodiments, the biopsy procedure is an image-guided process that uses anatomical models to assist the operator in navigating the biopsy device to one or more target locations within the patient's body (e.g., navigating to one or more lymph node sites). In some embodiments, method 400 is performed after a preoperative plan for the biopsy procedure has been generated. For example, method 400 may be performed after... Figure 2 The method is executed after some or all of the steps of method 200.

[0059] Method 400 is illustrated as a set of steps or processes 410-430. Method 400 can be performed by a suitable computing system or device (e.g., a medical device system, etc.). For example, all or a subset of the steps of method 400 can be implemented by a control system of a medical device system or device, including various components or devices of a robot or remote operating system as further described below. The computing system for implementing method 400 may include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform operations according to steps 410-430.

[0060] Method 400 begins at step 410 with the registration of the biopsy device to a 3D model of the patient's anatomy. Registration of the biopsy device to the 3D model allows tracking and mapping of the biopsy device's orientation within the patient's body to a corresponding orientation within the model, thereby providing visual guidance for navigating the biopsy device within the anatomy. When the biopsy device is driven within different channels within the anatomy, registration can be performed using survey data (e.g., orientation data and / or shape data) generated by one or more sensors in the biopsy device. The survey data can be rotated, translated, or otherwise manipulated by rigid and / or non-rigid transformations to align it with data points in the model. For example, registration can be performed using a point-based Iterative Closest Point (ICP) technique—as described in U.S. Provisional Patent Applications Nos. 62 / 205,440 and 62 / 205,433, which are incorporated herein by reference in their entirety. However, in other embodiments, other registration techniques may be used to perform registration.

[0061] At step 420, the system displays instructions for navigating the biopsy device to one or more lymph node sites. (As previously stated...) Figure 2 As described in method 200, the lymph node site can be selected during the preoperative planning of the biopsy procedure. During the biopsy procedure, the system can output a graphical representation of the lymph node site via a suitable graphical user interface. For example, the lymph node site can be rendered as an opaque object within a 3D model, while other anatomical structures in the model (e.g., airways, blood vessels) can be rendered as transparent or translucent objects so that the lymph node site remains visible. As another example, the lymph node site can be displayed as a point or image on a 2D map of the anatomical region. Alternatively or in combination, the system can output text, audio, or other instructions that guide the operator to navigate the biopsy device to the selected lymph node site. For example, the system can instruct the operator to biopsy lymph nodes within certain lymph node station groups, to biopsy lymph nodes located in a specific anatomical region or area of ​​the anatomy, and so on.

[0062] In some implementations, step 420 further includes displaying instructions for performing a biopsy on the selected lymph node site according to a specified sequence. (As previously mentioned...) Figure 2 As described in method 200, the sequence can be determined during preoperative planning of the biopsy procedure. This sequence can be output to the operator in various ways, such as through the same graphical user interface used to display lymph node sites. For example, the sequence can be graphically represented as a path connecting selected lymph node sites in the desired order, as described above regarding... Figure 2Method 200 discusses this approach. The path can be overlaid on a model and / or image of the actual patient's anatomy to provide visual guidance as the operator navigates the biopsy device within the anatomical region. Alternatively or in combination, the system can output text, audio, or other instructions that guide the operator to perform biopsies on lymph node sites in a specific order and / or guide the operator to navigate the biopsy device along a path (e.g., in a specific direction and / or at a specific distance—relative to specific anatomical landmarks, etc.).

[0063] At step 430, method 400 displays orientation data of the biopsy device relative to the selected lymph node site. As discussed above, once the biopsy device is registered to the anatomical model, the orientation of the biopsy device within the patient can be mapped to the corresponding orientation within the model. Therefore, the orientation of the biopsy device relative to the lymph node site can also be tracked and displayed. For example, the orientation of the biopsy device can be graphically represented as an object within the 3D anatomical model, allowing the operator to visualize the orientation of the biopsy device relative to the lymph node site and / or the planned path. Orientation data can also be used to monitor the progress of the biopsy procedure, track which lymph node sites have been biopsied or not, or otherwise provide instructions and / or feedback to assist the operator in performing the procedure. For example, the operator can be alerted if a lymph node site is missed, if lymph node sites are not biopsied sequentially, or if the biopsy device is no longer on the correct path.

[0064] Optionally, method 400 may also include receiving feedback from the operator during the biopsy procedure. For example, the operator may provide input indicating that one or more selected lymph node sites cannot be biopsied, one or more other lymph node sites cannot be biopsied, a portion of the planned path is unreachable, an alternative path should be used, etc. The biopsy procedure plan can be adjusted based on operator feedback. For example, the system may change the lymph node site to be biopsied (e.g., add or remove a lymph node site), change the biopsy sequence, change the path to the lymph node site, or make any other suitable modifications. In embodiments where the operator can obtain biopsy results immediately (e.g., via rapid onsite cytopathology evaluation (ROSE)), the biopsy plan can be updated based on whether a particular lymph node site is malignant positive or negative. For example, the system may omit lymph node sites downstream of negative biopsy sites. The instructions displayed to the operator can be updated to reflect any changes in the biopsy plan.

[0065] Although the steps of method 400 are discussed and described in a specific order, those skilled in the art will recognize that method 400 can be modified and still fall within the scope of these and other embodiments of the present technology. In other embodiments, for example, method 400 can be performed in a different order; for example, any step of method 400 can be performed before, during, and / or after any other step of method 400. Furthermore, steps can be omitted. Figure 4 One or more steps of method 400 are shown. Optionally, one or more steps of method 400 may be repeated.

[0066] B. Implementation methods of robotic or remotely operated medical systems and related devices, systems and methods

[0067] Figure 5 This is a schematic representation of a robot or remotely operated medical system 500 (“medical system 500”) configured according to various embodiments of the present technology. Medical system 500 can be related to the above-mentioned… Figures 1A-4 Any procedures or methods described may be used in conjunction. For example, medical system 500 may be used to plan and / or perform biopsy procedures on one or more lymph node sites, as previously discussed. As shown, medical system 500 includes a manipulator assembly 502, a medical device system 504, a main assembly 506, and a control system 512. Manipulator assembly 502 supports medical device system 504 and drives medical device system 504 in the direction of main assembly 506 and / or control system 512 to perform various medical procedures on patient 503 positioned on table 507 in surgical environment 501. In this respect, main assembly 506 typically includes one or more control devices that can be operated by an operator 505 (e.g., a physician) to control manipulator assembly 502. Additionally or alternatively, control system 512 includes computer processor 514 and at least one memory 516 for implementing control between medical device system 504, main assembly 506, and / or other components of medical system 500. The control system 512 may also include programming instructions (e.g., a non-transitory computer-readable medium storing instructions) to implement any one or more methods described herein, including instructions for providing information to the display system 510 and / or processing data for registering the medical device system 504 with an anatomical model of the patient 503 (as described in more detail below). The manipulator component 502 may be a remotely operated, non-remotely operated, or a hybrid of both. Therefore, all or part of the main component 506 and / or all or part of the control system 512 may be located inside or outside the surgical environment 501.

[0068] To assist operator 505 in controlling manipulator assembly 502 and / or medical device system 504 during image-guided medical procedures, medical system 500 may further include orientation sensor system 508, endoscopic imaging system 509, imaging system 518, and / or virtual visualization system 515. In some embodiments, orientation sensor system 508 includes position sensor system (e.g., electromagnetic (EM) sensor system) and / or shape sensor system for capturing orientation sensor data (e.g., orientation, orientation, velocity, rate, posture, shape, etc.) of medical device system 504. In these and other embodiments, endoscopic imaging system 509 includes one or more image capture devices (not shown) that record endoscopic image data including concurrent or real-time images of the patient's anatomy (e.g., video images, still images, etc.). Images captured by endoscopic imaging system 509 may be, for example, 2D or 3D images of the patient's anatomy captured by an image capture device positioned within the patient 503, and are referred to below as "real-time navigation images".

[0069] In some embodiments, the medical device system 504 may include components of the orientation sensor system 508 and / or components of the endoscopic imaging system 509. For example, components of the orientation sensor system 508 and / or components of the endoscopic imaging system 509 may be integrally or removably coupled to the medical device system 504. Additionally or alternatively, the endoscopic imaging system 509 may include a separate endoscope (not shown) attached to a separate manipulator assembly (not shown), which can be used in conjunction with the medical device system 504 to image a patient's anatomy. The orientation sensor system 508 and / or the endoscopic imaging system 509 may be implemented as hardware, firmware, software, or a combination thereof, which interact with or are otherwise performed by one or more computer processors, such as the computer processor(s) 514 of the control system 512.

[0070] The imaging system 518 of the medical system 500 may be positioned within the surgical environment 501 close to the patient 503 to acquire real-time and / or near-real-time images of the patient 503 before, during, and / or after medical procedures. In some embodiments, the imaging system 518 includes a mobile C-arm cone-beam CT imaging system for generating 3D images. For example, the imaging system 518 may include a Siemens Corporation DynaCT imaging system, or other suitable imaging systems. In these and other embodiments, the imaging system 518 may include other imaging techniques, including MRI, fluorescein imaging, thermal imaging, ultrasound, OCT, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like.

[0071] When the medical device system 504 is controlled during an image-guided medical procedure, the virtual visualization system 515 of the control system 512 provides navigation and / or anatomical-interactive assistance to the operator 505. As described in more detail below, virtual navigation using the virtual visualization system 515 may be based, at least in part, on a reference to a preoperative or intraoperative dataset of the acquired anatomical pathways of the patient 503 (e.g., at least in part on a reference to data generated by the orientation sensor system 508, the endoscopic imaging system 509, and / or the imaging system 518). In some embodiments, for example, the virtual visualization system 515 processes preoperative and / or intraoperative image data of the anatomical regions of the patient 503 captured by the imaging system 518 to generate an anatomical model of the anatomical regions (not shown). Then, the virtual visualization system 515 registers the anatomical model with orientation sensor data generated by the orientation sensor system 508 and / or endoscopic image data generated by the endoscopic imaging system 509 to (i) map the tracked orientation, orientation, posture, shape and / or motion of the medical device system 504 within the anatomical region to the correct orientation within the anatomical model, and / or (ii) determine a virtual navigation image of the virtual patient anatomy within the anatomical region from the viewing point of the medical device system 504 at a location within the anatomical model corresponding to the position of the medical device system 504 within the patient 503.

[0072] Display system 510 may display various images or representations of the patient's anatomy generated by orientation sensor system 508, endoscopic imaging system 509, imaging system 518, and / or virtual visualization system 515, and / or medical device system 504. In some embodiments, display system 510 and / or main component 506 may be oriented such that operator 505 can use telepresent perception to control manipulator component 502, medical device system 504, main component 506, and / or control system 512.

[0073] As discussed above, the manipulator assembly 502 drives the medical device system 504 in the direction of the main assembly 506 and / or the control system 512. In this respect, the manipulator assembly 502 may include selected degrees of freedom of motion, which may be motorized and / or remotely operated, and selected degrees of freedom of motion, which may be non-motorized and / or non-remotely operated. For example, the manipulator assembly 502 may include a plurality of actuators or motors (not shown) that drive inputs on the medical device system 504 in response to commands received from the control system 512. The actuators may include a drive system (not shown) that, when coupled to the medical device system 504, can advance the medical device system 504 into a naturally occurring or surgically generated anatomical opening. Other drive systems may move a distal portion (not shown) of the medical device system 504 with multiple degrees of freedom, which may include three linear degrees of freedom (e.g., linear motion along the X, Y, Z Cartesian axes) and three rotational degrees of freedom (e.g., rotation about the X, Y, Z Cartesian axes). Additionally or alternatively, the actuator may be used to actuate the articulated end effector of the medical device system 504 (e.g., for grasping tissue in the jaws of a biopsy device and / or similar device).

[0074] Figure 6 yes Figure 5 A schematic representation of the manipulator assembly 502, medical device system 504, and imaging system 518 configured within the surgical environment 501 according to various embodiments of the present technology. Figure 6 As shown, the surgical environment 501 has a surgical reference frame (X). S Y S Z S In this system, patient 503 is positioned on stage 507, and medical device system 504 has a medical device reference system (X) within the surgical environment 501. M Y M Z M During medical procedures, patient 503 may be immobile within surgical environment 501 in the sense that the patient's overall movement is restricted by sedation, restraint, and / or other means. In these and other embodiments, circulatory anatomical movements, including the patient 503's breathing and cardiac movements, may continue unless the patient 503 is asked to hold his or her breath to temporarily suspend respiratory movements.

[0075] The manipulator assembly 502 includes an instrument holder 626 mounted to the insertion stage 628. In the illustrated embodiment, the insertion stage 628 is linear, while in other embodiments, the insertion stage 628 is curved or has a combination of curved and linear segments. In some embodiments, the insertion stage 628 is fixed within the surgical environment 501. Alternatively, the insertion stage 628 is movable within the surgical environment 501 but has a known position within the surgical environment 501 (e.g., via a tracking sensor (not shown) or other tracking device). In these alternatives, a medical device reference frame (X... M Y M Z M ) relative to the surgical reference frame (X) S Y S Z S () is fixed or otherwise known.

[0076] Figure 6 The medical device system 504 includes an elongated device 631, a medical device 632, an device body 635, at least a portion of an orientation sensor system 508, and at least a portion of an endoscopic imaging system 509. In some embodiments, the elongated device 631 is a flexible catheter or other biomedical device defining a channel or cavity 644. The channel 644 may be sized and shaped to receive the medical device 632 (e.g., via the proximal end 636 of the elongated device 631 and / or the device port (not shown)) and facilitate delivery of the medical device 632 to a distal portion 638 of the elongated device 631. The elongated device 631 is coupled to the device body 635, which in turn is coupled to and fixed relative to the device holder 626 of the manipulator assembly 502.

[0077] During operation, the manipulator assembly 502 can control the insertion movement of the elongated device 631 into the patient 503 via a naturally occurring or surgically created anatomical opening (e.g., proximal and / or distal movement along axis A) to facilitate navigation of the elongated device 631 through anatomical pathways in the anatomical region of the patient 503 and / or to facilitate delivery of the distal portion 638 of the elongated device 631 to a target location within or near the patient 503. For example, the instrument holder 626 and / or insertion stage 628 may include actuators (not shown), such as servo motors, that facilitate control of movement of the instrument holder 626 along the insertion stage 628. Additionally or alternatively, in some embodiments, the manipulator assembly 502 can control movement of the distal portion 638 of the elongated device 631 in multiple directions, including yaw, pitch, and roll rotation (e.g., thereby navigating the patient anatomy). For this purpose, the elongated device 631 may accommodate or include cables, linkages, and / or other steering controls (not shown) that the manipulator assembly 502 may use to controllably bend the distal portion 638 of the elongated device 631. For example, the elongated device 631 may accommodate at least four cables that the manipulator assembly 502 may use to provide (i) independent up-down steering to control the pitch of the distal portion 638 of the elongated device 631, and (ii) independent left-right steering of the elongated device 631 to control the yaw of the distal portion 638 of the elongated device 631.

[0078] Medical device 632 of medical device system 504 can be used for medical procedures such as exploration, surgery, biopsy, ablation, illumination, irrigation, and / or aspiration of anatomical access. Therefore, medical device 632 may include image-capturing probes, biopsy instruments or devices (e.g., biopsy needles, endobronchial ultrasound (EBUS) probes), laser ablation fibers and / or other surgical tools, diagnostic tools, and / or therapeutic tools. For example, medical device 632 may include an endoscope or other biomedical device having one or more image-capturing devices 647 located at a distal portion 637 of medical device 632 and / or at other locations along medical device 632. In these embodiments, when medical device 632 is within the anatomical region of patient 503, image-capturing device 647 may capture one or more real navigation images or videos (e.g., a sequence of one or more real navigation image frames) of the anatomical access and / or other real patient anatomy.

[0079] As discussed above, the medical device 632 can be deployed into and / or delivered to a target location within the patient 503 via the channel 644 defined by the elongated device 631. In embodiments where the medical device 632 includes an endoscope or other biomedical device having an image capturing device 647 at its distal portion 637, the image capturing device 647 may be advanced into the distal portion 638 of the elongated device 631 before, during, and / or after the manipulator assembly 502 navigates the distal portion 638 of the elongated device 631 to the target location within the patient 503. In these embodiments, the medical device 632 can be used as an exploration instrument to capture realistic navigation images of anatomical channels and / or other real patient anatomy, and / or to assist an operator (not shown) in navigating the distal portion 638 of the elongated device 631 through the anatomical channels to the target location.

[0080] As another example, after the manipulator assembly 502 positions the distal portion 638 of the elongated device 631 near a target location within the patient 503, the medical device 632 can be advanced beyond the distal portion 638 of the elongated device 631 to perform a medical procedure at the target location. Continuing this example, after all or part of the medical procedure at the target location has been completed, the medical device 632 can be retracted into the elongated device 631 and, additionally or alternatively, can be removed from the proximal end 636 of the elongated device 631 or from another instrument port (not shown) along the elongated device 631.

[0081] like Figure 6 As shown, the orientation sensor system 508 of the medical device system 504 includes a shape sensor 633 and an orientation measuring device 639. In these and other embodiments, in addition to, or in place of, the shape sensor 633 and / or the orientation measuring device 639, the orientation sensor system 508 may include other orientation sensors (e.g., accelerometers, rotary encoders, etc.).

[0082] The shape sensor 633 of the orientation sensor system 508 includes an optical fiber extending within and aligned with an elongated device 631. In one embodiment, the optical fiber of the shape sensor 633 has a diameter of approximately 200 μm. In other embodiments, the diameter of the optical fiber may be larger or smaller. The optical fiber of the shape sensor 633 forms an optical fiber bending sensor for determining the shape, orientation, and / or attitude of the elongated device 631. In some embodiments, the optical fiber having a fiber Bragg grating (FBG) can be used to provide stress measurements in a structure of one or more dimensions. U.S. Patent Application Publication No. 2006 / 0013523 (filed July 13, 2005) (disclosing an optical fiber orientation and shape sensing device and related methods thereof); U.S. Patent No. 7,781,724 (filed September 26, 2006) (disclosing an optical fiber orientation and shape sensing device and related methods thereof); U.S. Patent No. 7,772,541 (filed March 12, 2008) (disclosing optical fiber orientation and / or shape sensing based on Rayleigh scattering); and U.S. Patent No. 6,389,187 (filed June 17, 1998) (disclosing an optical fiber bending sensor) further describe in detail various systems and methods for monitoring the three-dimensional shape and relative orientation of optical fibers, which are incorporated herein by reference in their entirety. In these and other embodiments, the sensor of this technology may employ other suitable stress sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. In these and other embodiments, the shape of the elongated device 631 may be determined using other techniques. For example, the historical record of the orientation of the distal portion 638 of the elongated device 631 can be used to reconstruct the shape of the elongated device 631 over time intervals.

[0083] In some embodiments, the shape sensor 633 is fixed at a proximal point 634 on the device body 635 of the medical device system 504. During operation, for example, the shape sensor 633 measures the medical device reference frame (X). M Y M Z M The shape sensor 633 is defined as the shape from the proximal point 634 to another point along the optical fiber (such as the distal portion 638 of the elongated device 631). The proximal point 634 of the shape sensor 633 can move together with the device body 635, but the position of the proximal point 634 can be known (e.g., via a tracking sensor (not shown) or other tracking device).

[0084] As the instrument body 635 moves along insertion axis A on the insertion stage 628 of the manipulator assembly 502, the orientation measuring device 639 of the orientation sensor system 508 provides information about the orientation of the instrument body 635. In some embodiments, the orientation measuring device 639 includes a resolver, encoder, potentiometer, and / or other sensors that determine the rotation and orientation of the actuator (not shown) that controls the movement of the instrument carrier 626 of the manipulator assembly 502 and thus determines the movement of the instrument body 635 of the medical device system 504.

[0085] Figure 7 It extends within the anatomical region 750 (e.g., human lung) of patient 503 according to various embodiments of this technology. Figure 6 A schematic representation of a portion of medical device system 504. Specifically, Figure 7 The illustration shows an elongated device 631 of a medical device system 504 extending within a branch anatomical passage 752 of an anatomical region 750. The anatomical passage 752 includes a trachea 754 and multiple bronchi 756.

[0086] like Figure 7 As shown, the elongated device 631 has orientation, orientation, posture, and shape within the anatomical region 750, and all or part of it (other than motion such as velocity or rate, or in lieu of motion such as velocity or rate) can serve as orientation sensor data from Figure 5 and Figure 6 The orientation sensor system 508 (e.g., consisting of a shape sensor 633 and / or an orientation measuring device 639) Figure 6 The system captures and explores the anatomical passage 752 of the anatomical region 750. Specifically, the orientation sensor system 508 can collect data from the medical device reference frame (X-ray). M Y M Z M The medical device system 504 uses orientation sensor data within the anatomical region 750 to survey the anatomical passage 752. The orientation sensor data can be recorded at least partially as a set of 2D or 3D coordinate points. In an example where the anatomical region 750 is a human lung, the coordinate points can represent the position of the distal portion 638 of the elongated device 631 and / or the position of other portions of the elongated device 631 as it is advanced through the trachea 754 and bronchi 756. In these and other embodiments, the set of coordinate points can represent the shape of the elongated device 631 as it is advanced through the anatomical region 750. In these and other embodiments, the coordinate points can represent other portions of the medical device system 504 (e.g., medical device 632). Figure 6 The directional data.

[0087] Coordinate points can be combined to form a point cloud. For example, Figure 8The illustration shows a plurality of coordinate points 862 forming a point cloud 860 according to various embodiments of the present invention, wherein the point cloud 860 represents Figure 7 The slender device 631 is located in the anatomical region 750 ( Figure 7 The shape of the elongated device 631 when it is inside the body. Specifically, when the elongated device 631 is in the position of... Figure 7 When the position shown is stationary, Figure 8 The point cloud of 860 is obtained from the orientation sensor system 508 ( Figure 6 Generate the union of all or a subset of the recorded coordinate points 862.

[0088] In some embodiments, the point cloud (e.g., point cloud 860) may include the union of all or a subset of coordinate points recorded by the orientation sensor system 508 during an image capture period of multiple shapes, orientations, and / or poses of the elongated device 631 across the anatomical region 750. In these embodiments, the point cloud may include coordinate points captured by the orientation sensor system 508 representing multiple shapes of the elongated device 631 as it is advanced or moved through the patient anatomy during the image capture period. Additionally or alternatively, because the configuration (including shape and position) of the elongated device 631 within the patient 503 may change due to anatomical movement during the image capture period, the point cloud in some embodiments may include multiple coordinate points 862 captured by the orientation sensor system 508 representing the shape of the elongated device 631 as it is passively moved within the patient 503. As described in more detail below, the point cloud of coordinate points captured by the orientation sensor system 508 may be registered to different models or datasets of the patient anatomy.

[0089] Refer again Figure 6 The endoscopic imaging system 509 of the medical device system 504 includes one or more image capturing devices 647, which are configured to be positioned when the elongated device 631 and / or the medical device 632 are located in the anatomical region of the patient 503 (e.g., Figure 7 Capture the real patient's anatomy within the anatomical region (750) (e.g.) Figure 7 One or more real navigation images of the anatomical passage 752. For example, the endoscopic imaging system 509 may include an image capturing device 647 located at the distal portion 637 of the medical device 632. In these and other embodiments, the endoscopic imaging system 509 may include one or more image capturing devices (not shown) positioned along the medical device 632 and / or along other locations (e.g., at the distal portion 638 of the elongated device 631).

[0090] exist Figure 7 In the embodiment shown, medical device 632 ( Figure 6The image capturing device 647 is advanced to the distal portion 638 of the elongated device 631 and positioned thereon. In this embodiment, the image capturing device 647 can survey the anatomical passage 752 by capturing a real navigation image of the anatomical passage 752 while the elongated device 631 is being navigated through the trachea 754 and bronchus 756 of the anatomical region 750.

[0091] Figure 9 It is via image capture device 647 ( Figure 7 ) captured Figure 7 An example of a real navigation image 970 (e.g., a still image, a video frame, etc.) of the patient's anatomy in the anatomical region 750 (such as one of the anatomical channels 752). As shown, the real navigation image 970 shows the patient's anatomy from the medical device 632 ( Figure 6 The viewing point is located at the branching point or protuberance 971 of the two anatomical channels 752 within the anatomical region 750. In this example, because the image capturing device 647 is positioned at the medical device 632 and the elongated device 631 respectively. Figure 7 The image capture device 647 is located at the distal portions 637 and 638 of the medical device 632, so the viewing point of the real navigation image 970 is from the distal portion 637 of the medical device 632, making the medical device 632 and the elongated device 631 invisible within the real navigation image 970. In other embodiments, the image capture device 647 may be positioned at another location along the medical device 632 and / or along the elongated device 631. Figure 6 and Figure 7 In these embodiments, the endoscopic imaging system 99 ( Figure 6 A true navigation image can be captured from the corresponding viewing point of the medical device 632 and / or the elongated device 631. Depending on the orientation of the medical device 632 and the elongated device 631 relative to each other, a portion of the medical device 632 and / or the elongated device 631 may be visible within these true navigation images.

[0092] Refer again Figure 6 The real navigation images captured by the endoscopic imaging system 509 facilitate the passage of the distal portion 638 of the elongated device 631 through the patient's anatomy (e.g., through...). Figure 7Navigation of the anatomical passage 752 and / or delivery of the distal portion 638 of the elongated device 631 to a target location within the patient 503. In these and other embodiments, the real navigation images captured by the endoscopic imaging system 509 can facilitate (i) navigation of the distal portion 637 of the medical device 632 beyond the distal portion 638 of the elongated device 631, (ii) delivery of the distal portion 637 of the medical device 632 to a target location within the patient 503, and / or (iii) visualization of the patient's anatomy during medical procedures. In some embodiments, each real navigation image captured by the endoscopic imaging system 509 can be compared with images recorded in the medical device reference frame (X). M Y M Z M The timestamps and / or orientations in the image are associated. The real navigation images captured by the endoscopic imaging system 509 can optionally be used to improve the point cloud of coordinate points generated by the orientation sensor system 508 (e.g., ...). Figure 8 The registration between the point cloud (860) and the image data captured by the imaging system (518).

[0093] like Figure 6 As shown, the imaging system 518 is positioned near the patient 503 to obtain images of the patient 503 (e.g., Figure 7 The imaging system 518 provides 3D images of the anatomical region 750. In some embodiments, the imaging system 518 includes one or more imaging techniques, including CT, MRI, fluorescein imaging, thermal imaging, ultrasound, OCT, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like. The imaging system 518 is configured to generate image data of the patient's anatomy before, during, and / or after the elongated device 631 extends within the patient 503. Thus, the imaging system 518 can be configured to capture preoperative, intraoperative, and / or postoperative 3D images of the patient's anatomy. In these and other embodiments, the imaging system 518 can provide real-time or near-real-time images of the patient's anatomy.

[0094] Figure 10 The illustration shows the imaging system 518 during the image capture period as the elongated device 631 of the medical device system 504 extends within the anatomical region 750. Figure 6 ) captured Figure 7 An example of intraoperative image data 1080 for a portion 1055 of the anatomical region 750. As shown, image data 1080 includes a graphic element 1081 representing an elongated device 631 and a graphic element 1082 representing an anatomical passage 752 of the anatomical region 750.

[0095] All or a portion of the graphic elements 1081 and 1082 of image data 1080 may be segmented and / or filtered to generate a virtual 3D model (with or without the medical device system 504) of the anatomical passage 752 within said portion 1055 of the anatomical region 750. In some embodiments, graphic elements 1081 and 1082 may additionally or alternatively be segmented and / or filtered to at least in part based on the imaging system 108 when the medical device system 504 is within the anatomical region 750. Figure 6 The captured images are used to generate an image point cloud (not shown) of the medical device system 504. During the segmentation process, pixels or voxels generated from the image data 1080 can be divided into segments or elements or labeled to indicate that they share certain features or computational properties such as color, density, intensity, and material. These segments or elements can then be converted into an anatomical model and / or image point cloud of the medical device system 504. Additionally or alternatively, these segments or elements can be used to locate (e.g., calculate) and / or define a centerline or other points extending along the anatomical passage 752. The generated anatomical model and / or image point cloud can be 2D or 3D and can be displayed in an image reference frame (X). I Y I Z I Generated in ).

[0096] As mentioned above Figure 5 The discussion focused on the healthcare system (500). Figure 5 The display system 510 ( Figure 5 The system can display various images or representations of the patient's anatomy and / or the medical device system 504, at least in part, based on data captured and / or generated by the orientation sensor system 508, the endoscopic imaging system 509, the imaging system 518, and / or the virtual visualization system 515. In various embodiments, the system can utilize these images and / or representations to assist the operator 505. Figure 5 ) Perform image-guided medical procedures.

[0097] Figure 11 The display system 510 is based on various embodiments of this technology. Figure 5 The diagram illustrates an example display 1110. As shown, display 1110 includes a real navigation image 1170, a composite virtual navigation image 1191 (also referred to as "composite virtual image 1191"), and a virtual navigation image 1192. The real navigation image 1170 can be substantially similar to... Figure 9 The actual navigation image 970 is the same. Therefore, for example, the actual navigation image 1170 can be generated by the endoscopic imaging system 509 ( Figure 6 ) capture and provide to the display system 510 ( Figure 5The image is displayed on the display 1110 in real time or near real time. In the illustrated embodiment, the real navigation image 1170 illustrates the view from a distance of 632 ( Figure 6 The distal portion 637 of the viewpoint is the actual patient anatomy (e.g., the ridge 1171 marking the branching points of the two anatomical channels 752).

[0098] Figure 11 The composite virtual image 1191 is displayed in the image reference frame (X). I Y I Z I ), and includes from the imaging system 518 ( Figure 6 ) captured Figure 7 Anatomical model 1150 is generated from image data of anatomical region 750. Anatomical model 1150 is integrated with orientation sensor system 508. Figure 6 The point cloud of coordinate points generated (e.g., Figure 8 The point cloud 860) is registered (i.e., dynamically referenced) to display the medical device system 504 within the anatomical model 1150 (e.g., Figure 6 The slender device 631) in patient 503 ( Figure 6 The representation 1104 of the tracked orientation, shape, pose, orientation, and / or motion within the system. In some embodiments, the composite virtual image 1191 is controlled by the control system 512. Figure 5 ) virtual visualization system 515 ( Figure 5 Generating a composite virtual image 1191 involves using an image reference frame (X...) I Y I Z I ) and surgical reference system (X) S Y S Z S Registration and / or registration to the medical device reference system (X) M Y M Z M This registration can be performed using the coordinates of the point cloud captured by the orientation sensor system 508 (e.g., Figure 8 Rigid and / or non-rigid transformations of the coordinate points 862 of the point cloud 860 can be used to rotate, translate, or otherwise manipulate them to align the coordinate points with the anatomical model 1150. Registration between the image and the surgical / instrument reference frame can be achieved, for example, by using point-based ICP techniques, as described in U.S. Provisional Patent Applications Nos. 62 / 205,440 and 62 / 205,433, which are incorporated herein by reference in their entirety. In other embodiments, registration can be achieved using a different point cloud registration technique.

[0099] Based at least in part on registration, the virtual visualization system 515 can additionally or alternatively generate virtual navigation images (e.g., Figure 11 The virtual navigation image 1192 includes a medical device system 504 within the anatomical model 1150 viewed from a virtual camera. Figure 7 The virtual depiction of the patient's anatomy, representing 1104 viewing points. Figure 11 In the illustrated embodiment, the virtual camera of the virtual navigation image 1192 is positioned at the distal portion 1137 of representation 1104 such that (i) the virtual viewing point of the virtual navigation image 1192 points distally—away from the distal portion 1137 of representation 1104, and (ii) representation 1104 is not visible within the virtual navigation image 1192. In other embodiments, the virtual visualization system 515 may position the virtual camera at (a) another location along representation 1104 and / or (b) in different orientations such that the virtual navigation image 1192 has a corresponding virtual viewing point. In some embodiments, depending on the orientation and position of the virtual camera and the orientation of the elongated device 631 and the medical device 632 relative to each other within the patient 503, the virtual visualization system 515 may render a virtual representation (not shown) of at least a portion of the elongated device 631 and / or the medical device 632 into the virtual navigation image 1192.

[0100] In some implementations, the virtual visualization system 515 can place a virtual camera within the anatomical model 1150 and project it onto the patient 503. Figure 6 The orientation and orientation of the image capturing device 647 within the image capture device 647 correspond to the orientation and orientation of the image capture device 647. For example... Figure 11 As further shown, the virtual navigation image 1192 illustrates the virtual patient anatomy, such as the ridges 1101 marking the branching points of the two anatomical passages 1152 of the anatomical model 1150, which are obtained from the image capture device 647. Figure 6 The virtual navigation image 1192 captures essentially the same location as the real navigation image 1170. Therefore, the virtual navigation image 1192 provides a view of the location... Figure 7 The rendering estimate of the patient's anatomy visible at a given location within the anatomical region 750 by the image capture device 647. Because the virtual navigation image 1192 is at least partially based on the registration of the point cloud generated by the orientation sensor system 508 and the image data captured by the imaging system 518, the correspondence between the virtual navigation image 1192 and the real navigation image 1170 provides insight into the registration accuracy and can be used to improve registration. Furthermore, the endoscopic imaging system 509 ( Figure 6 The captured real navigation image (e.g., real navigation image 1170) can (a) provide information about the medical device system 504. Figure 5(a) information on the orientation and orientation within the patient 503, (b) information on portions of the anatomical region actually accessed by the medical device system, and / or (c) assistance in identifying patient anatomy (e.g., branch points of anatomical pathways) near the medical device system 504, any one or more of which may be used to improve the accuracy of registration.

[0101] like Figure 11 As further shown, the virtual navigation image 1192 may optionally include a navigation path overlay 1199. In some embodiments, the navigation path overlay 1199 is used to assist the operator 505 ( Figure 5 Navigation Medical Device System 504 Figure 5 The navigation path overlay 1199 can be used to reach a target location within the patient 503 via anatomical pathways through the anatomical region. For example, the navigation path overlay 1199 can illustrate an “optimal” path through the anatomical region for the operator 505 to follow, thereby delivering the distal portions 637 and / or 638 of the medical device 632 and / or the elongated device 631 to the target location within the patient 503, respectively. In some embodiments, the navigation path overlay 1199 can be aligned with the centerline of the corresponding anatomical pathway or with another line along the (e.g., the bottom) of the corresponding anatomical pathway.

[0102] C. Example

[0103] Several aspects of this technology will be described in the following embodiments. Although several aspects of this technology have been described with reference to embodiments relating to systems, computer-readable media, and methods, any of these aspects of this technology may be similarly described in other embodiments relating to any system, computer-readable medium, and method.

[0104] 1. A system for planning medical procedures, the system comprising:

[0105] A processor; and a memory operatively coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including the following—

[0106] Receive image data of the patient's anatomical regions, including multiple lymph nodes and target lesions.

[0107] A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion.

[0108] The subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model.

[0109] A sequence for determining the location of a subset of the lymph nodes during the medical procedure.

[0110] 2. The system of claim 1, wherein the plurality of lymph nodes comprises lymph nodes from a plurality of different lymph node stations.

[0111] 3. The system of Embodiment 1 or Embodiment 2, wherein the segmentation of the image data is performed at least in part based on input from the operator.

[0112] 4. The system of Example 3, wherein the input from the operator includes selecting one or more locations in the image data corresponding to one or more lymph nodes.

[0113] 5. The system of Embodiment 3, wherein the input from the operator includes accepting or rejecting one or more of the segmented components.

[0114] 6. The system of Embodiment 1 or Embodiment 2, wherein the segmentation of the image data is performed at least in part using a machine learning algorithm.

[0115] 7. The system of any one of Examples 1-6, wherein the subset of lymph nodes includes one or more mediastinal lymph nodes.

[0116] 8. The system of any one of Examples 1-6, wherein the subset of lymph nodes includes one or more lymph nodes located downstream of the lymphatic drainage path originating from the target lesion.

[0117] 9. The system of any one of Examples 1-6, wherein the subset of lymph nodes includes at least one lymph node located on the same side of the anatomical region as the target lesion and at least one lymph node located on the opposite side of the anatomical region as the target lesion.

[0118] 10. The system of any one of Examples 1-6, wherein the subset of lymph nodes is selected based on one or more of the following: lymph node size, lymph node shape, lymph node location, location of the target lesion, the patient's physiology, a predicted metastasis risk score, input from the operator, or clinical guidelines.

[0119] 11. The system of any one of Examples 1-10, wherein the determined sequence for navigating the biopsy device is configured to reduce cross-contamination between different lymph nodes.

[0120] 12. The system of any one of Examples 1-10, wherein the determined sequence for navigating the biopsy device includes biopsy of lymph nodes with a lower probability of malignancy prior to lymph nodes with a higher probability of malignancy.

[0121] 13. The system of any one of Examples 1-10, wherein the determined sequence for navigating the biopsy device includes performing a biopsy on a lymph node distant from the target lesion location prior to a lymph node located near the target lesion location.

[0122] 14. The system of any one of Examples 1-13, wherein the operation further includes generating, at least in part, a path for navigating the biopsy device to the target lesion based on the three-dimensional model.

[0123] 15. The system of any one of Embodiments 1-14 further includes a display configured to output a graphical representation of the three-dimensional model and a subset of the lymph nodes.

[0124] 16. The system of Example 15, wherein the operation further includes outputting instructions via the display for navigating the biopsy device to a subset of the lymph nodes according to the sequence.

[0125] 17. The system of Embodiment 15 or 16 further includes a sensor configured to generate orientation data of the biopsy device, wherein the operation further includes outputting a graphical representation of the orientation data along with a graphical representation of a subset of the lymph nodes via the display.

[0126] 18. A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors of a computing system, causing the computing system to perform operations including:

[0127] Receive image data of the patient’s anatomical region, which includes multiple lymph nodes and target lesions;

[0128] A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion;

[0129] A subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model; and

[0130] A sequence for determining the location of a subset of the lymph nodes during the medical procedure.

[0131] 19. The non-transitory computer-readable medium of Example 18, wherein the plurality of lymph nodes comprises lymph nodes from a plurality of different lymph node stations.

[0132] 20. A non-transitory computer-readable medium of Embodiment 18 or Embodiment 19, wherein the operation further includes receiving user input for performing segmentation of the image data.

[0133] 21. The non-transitory computer-readable medium of Example 20, wherein the user input includes selection of one or more locations in the image data corresponding to one or more locations of the lymph node.

[0134] 22. The non-transitory computer-readable medium of Embodiment 20, wherein the input from the operator includes accepting or rejecting one or more of the segmented components.

[0135] 23. A non-transitory computer-readable medium of Example 18 or Example 19, wherein the segmentation is performed using at least in part a machine learning algorithm.

[0136] 24. A non-transitory computer-readable medium of any of Examples 18-23, wherein the subset of said lymph nodes includes one or more mediastinal lymph nodes.

[0137] 25. A non-transitory computer-readable medium of any of Examples 18-23, wherein the subset of lymph nodes comprises one or more lymph nodes located downstream of a lymphatic drainage pathway originating from the target lesion.

[0138] 26. A non-transitory computer-readable medium of any one of Examples 18-23, wherein the subset of lymph nodes includes at least one lymph node located on the same side of the anatomical region as the target lesion and at least one lymph node located on the opposite side of the anatomical region as the target lesion.

[0139] 27. A non-transitory computer-readable medium of any one of Examples 18-23, wherein the subset of lymph nodes is selected based on one or more of the following: lymph node size, lymph node shape, lymph node location, location of the target lesion, the patient's physiology, a predicted metastasis risk score, input from an operator, or clinical guidelines.

[0140] 28. A non-transitory computer-readable medium of any of Examples 18-27, wherein the determined sequence for navigating the biopsy device is configured to reduce the likelihood of cross-contamination between lymph nodes.

[0141] 29. A non-transitory computer-readable medium of any of Examples 18-27, wherein the determined sequence for navigating the biopsy device includes biopsiing a lymph node with a lower probability of malignancy prior to a lymph node with a higher probability of malignancy.

[0142] 30. A non-transitory computer-readable medium of any one of Examples 18-27, wherein the determined sequence for navigating the biopsy device includes biopsiing lymph nodes distant from the target lesion location prior to lymph nodes near the target lesion location.

[0143] 31. A non-transitory computer-readable medium of any of Examples 18-30, wherein the operation further includes generating, at least in part, a path for navigating the biopsy device to the target lesion based on the three-dimensional model.

[0144] 32. A non-transitory computer-readable medium of any of Examples 18-31, wherein the operation further includes outputting a graphical representation of the three-dimensional model and a subset of the lymph nodes.

[0145] 33. A non-transitory computer-readable medium of any of Examples 18-31, wherein the operation further includes outputting instructions for navigating the biopsy device to a subset of the lymph nodes according to the sequence.

[0146] 34. A non-transitory computer-readable medium of any one of Examples 18-31, wherein said operation further includes:

[0147] Receive the orientation data of the biopsy device; and

[0148] Output a graphical representation of the location data along with a graphical representation of a subset of the lymph nodes.

[0149] 35. A non-transitory computer-readable medium of any of Examples 18-34, wherein the three-dimensional model further includes other segmented components corresponding to the patient's airway and lungs.

[0150] 36. A method comprising:

[0151] Receive image data of the patient’s anatomical region, which includes multiple lymph nodes and target lesions;

[0152] A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion;

[0153] A subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model; and

[0154] A sequence for determining the location of a subset of the lymph nodes during the medical procedure.

[0155] D. in conclusion

[0156] The systems and methods described herein may be provided in the form of a tangible and non-transitory machine-readable medium or medium (such as a hard disk drive, hardware memory, etc.) on which instructions for execution by a processor or computer are recorded. The instruction set may include various commands instructing a computer or processor to perform specific operations, such as the methods and procedures of the various embodiments described herein. The instruction set may be in the form of a software program or application. Computer storage media may include volatile and non-volatile media, as well as removable and non-removable media, for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media may include, but are not limited to, RAM, ROM, EPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, disk storage, or any other hardware medium that can be used to store desired information and is accessible by components of the system. Components of the system may communicate with each other via wired or wireless communication. These components may be separate from each other, or various combinations of components may be integrated together into a monitor or processor, or housed in a workstation with standard computer hardware (e.g., processor, circuitry, logic circuitry, memory, etc.). The system may include processing devices such as microprocessors, microcontrollers, integrated circuits, control units, storage media, and other hardware.

[0157] Although many embodiments have been described above in the context of intrapulmonary navigation and medical procedure execution in patients, other applications and embodiments besides those described herein are also within the scope of this technology. For example, unless otherwise specified or clearly understood from the context, the apparatus, system, method, and computer program product of this technology can be used in a variety of image-guided medical procedures, such as those performed on, within, or near a hollow patient anatomy, and more specifically, in procedures for the exploration, biopsy, ablation, or other treatment of tissue within and / or near a hollow patient anatomy. Therefore, for example, the systems, apparatus, methods, and computer program products of this disclosure can be used in one or more medical procedures relating to other patient anatomy such as a patient's bladder, urethra, GI system, and / or heart.

[0158] As used herein, the term "operator" should be understood to include any type of person who may be performing or assisting in a medical procedure, and therefore includes physicians, surgeons, doctors, nurses, medical technicians, other persons or users of the technologies disclosed herein, and any combination thereof. Additionally or alternatively, the term "patient" should be considered to include human and / or non-human (e.g., animal) patients to whom medical procedures are performed.

[0159] As will be understood from the foregoing, for illustrative purposes, this document has described specific embodiments of the technology, but has not shown or described in detail well-known structures and functions in order to avoid unnecessarily obscuring the description of the embodiments of the technology. In the event of any conflict between any material incorporated herein by reference and this disclosure, this disclosure shall prevail. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, unless the word “or” is explicitly limited to referring only to a single item excluding other items when referring to an enumeration of two or more items, its use in such an enumeration shall be construed as including (a) any single item in the enumeration, (b) all items in the enumeration, or (c) any combination of items in the enumeration. As used herein, “and / or” in the phrase “A and / or B” means only A, only B, and both A and B. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, the terms “including,” “containing,” “having,” and “with” are used throughout to indicate that at least one or more of the described features are included, without excluding any additional number of the same features and / or other features of additional types.

[0160] Furthermore, as used herein, the term "substantially" refers to the degree or extent to which an action, characteristic, property, state, structure, item, or result is complete or nearly complete. For example, a "substantially" closed object would mean that the object is either completely closed or almost completely closed. In some cases, the exact degree of deviation from absolute completeness may depend on the specific circumstances. However, in general, near-completeness will have the same overall result as if absolute and overall completion had been achieved. The use of "substantially" also applies when used in a negative sense, referring to the complete or near-complete lack of an action, characteristic, property, state, structure, item, or result.

[0161] The above detailed description of the embodiments of this technology is not intended to be exhaustive or to limit the technology to the precise forms disclosed above. Although specific embodiments and examples of the technology have been described above for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the technology. For example, although the steps are presented in a given order, alternative embodiments may perform the steps in a different order. As another embodiment, various components of the technology may be further divided into sub-components, and / or various components and / or functions of the technology may be combined and / or integrated. Furthermore, although advantages associated with certain embodiments of the technology have been described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments need to exhibit such advantages to fall within the scope of the technology.

[0162] It should also be noted that other embodiments besides those disclosed herein are also within the scope of this technology. For example, embodiments of this technology may have different configurations, components, and / or procedures than those shown or described herein. Furthermore, those skilled in the art will understand that these and other embodiments may be established without having certain configurations, components, and / or procedures shown or described herein, without departing from this technology. Therefore, this disclosure and related technologies may cover other embodiments not explicitly shown or described herein.

Claims

1. A system for planning medical procedures, the system comprising: processor; and A memory operatively coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including the following— Receive image data of the patient's anatomical regions, including multiple lymph nodes and target lesions. A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion. The subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model. A sequence for navigating a biopsy device to a subset of the lymph nodes during the medical procedure is determined, wherein, in the sequence for navigating the biopsy device, a first lymph node of the subset of the lymph nodes precedes a second lymph node of the subset of the lymph nodes, wherein the location of the first lymph node is further away from the target lesion than the location of the second lymph node, and wherein the determined sequence for navigating the biopsy device is configured to reduce cross-contamination between different lymph nodes.

2. The system of claim 1, wherein the plurality of lymph nodes comprises lymph nodes from a plurality of different lymph node stations.

3. The system of claim 1, wherein the segmentation of the image data is performed at least in part based on input from the operator.

4. The system of claim 3, wherein the input from the operator includes selecting one or more locations in the image data corresponding to one or more lymph nodes.

5. The system of claim 3, wherein the input from the operator includes accepting or rejecting one or more of the segmented components.

6. The system of claim 1, wherein the segmentation of the image data is performed at least in part using a machine learning algorithm.

7. The system of claim 1, wherein the subset of lymph nodes comprises one or more mediastinal lymph nodes.

8. The system of claim 1, wherein the subset of lymph nodes comprises one or more lymph nodes located downstream of the lymphatic drainage path originating from the target lesion.

9. The system of claim 1, wherein the subset of lymph nodes comprises at least one lymph node located on the same side of the anatomical region as the target lesion and at least one lymph node located on the opposite side of the anatomical region as the target lesion.

10. The system of claim 1, wherein the subset of lymph nodes is selected based on one or more of the following: lymph node size, lymph node shape, lymph node location, location of the target lesion, the patient's physiology, a predicted metastasis risk score, input from the operator, or clinical guidelines.

11. The system of claim 1, wherein the determined sequence for navigating the biopsy device includes biopsiing lymph nodes with a lower probability of malignancy before lymph nodes with a higher probability of malignancy.

12. The system of claim 1, wherein the operation further comprises generating, at least in part, a path for navigating the biopsy device to the target lesion based on the three-dimensional model.

13. The system of claim 1, further comprising a display configured to output a graphical representation of the three-dimensional model and the subset of the lymph nodes.

14. The system of claim 13, wherein the operation further comprises outputting instructions via the display for navigating the biopsy device to the location of the subset of the lymph nodes according to the sequence.

15. The system of claim 13, further comprising a sensor configured to generate orientation data of the biopsy device, wherein the operation further comprises outputting a graphical representation of the orientation data along with a graphical representation of the subset of the lymph nodes via the display.

16. A non-transitory computer-readable medium having instructions stored thereon, the instructions causing the computing system to perform operations including the following when executed by one or more processors of a computing system: Receive image data of the patient’s anatomical region, which includes multiple lymph nodes and target lesions; A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion; A subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model; as well as A sequence for navigating a biopsy device to a subset of the lymph nodes during the medical procedure is determined, wherein, in the sequence for navigating the biopsy device, a first lymph node of the subset of the lymph nodes precedes a second lymph node of the subset of the lymph nodes, wherein the location of the first lymph node is further away from the target lesion than the location of the second lymph node, and wherein the determined sequence for navigating the biopsy device is configured to reduce the possibility of cross-contamination between lymph nodes.

17. The non-transitory computer-readable medium of claim 16, wherein the plurality of lymph nodes comprises lymph nodes from a plurality of different lymph node stations.

18. The non-transitory computer-readable medium of claim 16, wherein the operation further includes receiving user input for performing the segmentation of the image data.

19. The non-transitory computer-readable medium of claim 18, wherein the user input includes selection of one or more locations in the image data corresponding to one or more locations of the lymph node.

20. The non-transitory computer-readable medium of claim 18, wherein the input from the operator includes accepting or rejecting one or more of the segmented components.

21. The non-transitory computer-readable medium of claim 16, wherein the segmentation is performed using at least in part a machine learning algorithm.

22. The non-transitory computer-readable medium of claim 16, wherein the subset of lymph nodes comprises one or more mediastinal lymph nodes.

23. The non-transitory computer-readable medium of claim 16, wherein the subset of lymph nodes comprises one or more lymph nodes located downstream of a lymphatic drainage path originating from the target lesion.

24. The non-transitory computer-readable medium of claim 16, wherein the subset of lymph nodes comprises at least one lymph node located on the same side of the anatomical region as the target lesion and at least one lymph node located on the opposite side of the anatomical region as the target lesion.

25. The non-transitory computer-readable medium of claim 16, wherein the subset of lymph nodes is selected based on one or more of the following: lymph node size, lymph node shape, lymph node location, location of the target lesion, the patient's physiology, a predicted metastasis risk score, input from an operator, or clinical guidelines.

26. The non-transitory computer-readable medium of claim 16, wherein the determined sequence for navigating the biopsy device comprises biopsiing a lymph node with a lower probability of malignancy prior to a lymph node with a higher probability of malignancy.

27. The non-transitory computer-readable medium of claim 16, wherein the operation further comprises generating, at least in part, a path for navigating the biopsy device to the target lesion based on the three-dimensional model.

28. The non-transitory computer-readable medium of claim 16, wherein the operation further comprises outputting a graphical representation of the three-dimensional model and the subset of the lymph nodes.

29. The non-transitory computer-readable medium of claim 16, wherein the operation further comprises outputting instructions for navigating the biopsy device to the location of the subset of the lymph nodes according to the sequence.

30. The non-transitory computer-readable medium of claim 16, wherein the operation further comprises: Receive the orientation data of the biopsy device; and Output a graphical representation of the location data along with a graphical representation of the subset of the lymph nodes.

31. The non-transitory computer-readable medium of claim 16, wherein the three-dimensional model further comprises other segmented components corresponding to the patient's airway and lungs.

32. A method for planning medical procedures, comprising: Receive image data of the patient’s anatomical region, which includes multiple lymph nodes and target lesions; A three-dimensional model of the anatomical region is generated by segmenting the image data, wherein the three-dimensional model includes multiple segmented components corresponding to the plurality of lymph nodes and the target lesion; A subset of lymph nodes to be biopsied during the medical procedure is selected, at least in part, based on the location of the target lesion in the three-dimensional model; as well as A sequence for navigating a biopsy device to a subset of the lymph nodes during the medical procedure is determined, wherein, in the sequence for navigating the biopsy device, a first lymph node of the subset of the lymph nodes precedes a second lymph node of the subset of the lymph nodes, wherein the location of the first lymph node is further away from the target lesion than the location of the second lymph node, and wherein the determined sequence for navigating the biopsy device is configured to reduce the possibility of cross-contamination between lymph nodes.

Images