Systems and methods for continuous registration for image-guided surgery

Abstract

The present application relates to systems and methods for continuous registration for image-guided surgery. Methods and systems are provided herein for registering a model of one or more anatomical passageways of a patient to a patient space. An example method can include accessing a set of model points of a model of a passageway, the model points associated with a model space; collecting measured points along a length of a catheter inserted into the passageway of the patient, the measured points determined by a shape of the catheter; and assigning the first set of points to a plurality of subsets. The example method can further include registering each subset with a model point to produce a plurality of registration candidates; comparing the candidates to identify a best subset associated with a best registration of the plurality of candidates, the best registration transforming the set of model points and at least one of the plurality of sets of measured points into a common space.

Description

[0001] This application is a divisional application of Chinese patent application 2017800578210 (PCT / US2017 / 059358) filed on October 31, 2017, entitled "System and method for continuous registration of image-guided surgery".

[0002] Related applications

[0003] This patent application claims priority and benefit on the filing date of U.S. Provisional Patent Application 62 / 416,393, filed November 2, 2016, entitled "Systems and Methods of Continuous Registration For Image-Guided Surgery," the entire contents of which are incorporated herein by reference. Technical Field

[0004] This disclosure relates to systems and methods for performing image-guided procedures, and more specifically to systems and methods for registering a model of a patient’s anatomy to one or more instruments used in a medical procedure. Background Technology

[0005] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during medical procedures, thereby reducing patient recovery time, discomfort, and adverse side effects. These techniques can be performed through natural openings in the patient's anatomical structures or through one or more surgical incisions. Through these natural openings or incisions, clinicians can insert minimally invasive medical instruments (including surgical, diagnostic, therapeutic, or biopsy instruments) to reach the target tissue location. To aid in reaching the target tissue location, the orientation and movement of the medical instrument can be correlated with preoperative or intraoperative images of the patient's anatomy. Using image-guided instruments with image correlation, the instrument can navigate natural or surgically created pathways within anatomical systems such as the lungs, colon, intestines, kidneys, heart, circulatory system, or the like. Systems and methods are needed for performing registration models used during image-guided surgery with minimal clinical interference. Summary of the Invention

[0006] The claims appended to the specification best summarize embodiments of the invention.

[0007] However, an exemplary method for registering a model of one or more anatomical pathways of a patient to a patient space may include: accessing a set of model points in a model space of the model of one or more anatomical pathways of the patient; collecting a first set of measurement points along the length of a catheter inserted into one or more anatomical pathways of the patient, the measurement points in the patient space being determined by the shape of the catheter; and assigning the points of the first set of measurement points to a plurality of first subsets of the measurement points. The exemplary method may further include: registering the first subset of measurement points with the set of model points to generate a first plurality of registration candidates; comparing the registration candidates to identify a best subset associated with the best registration of the first plurality of registration candidates; and displaying a visual representation of the best registration in a user interface provided by a display. The best registration transforms at least one subset of the set of model points and the first set of measurement points into a common space.

[0008] Another exemplary method for registering a model of one or more anatomical pathways of a patient to patient space may include: accessing a set of model points of the model of one or more anatomical pathways of the patient; collecting a set of measurement points along the length of a catheter inserted into the one or more anatomical pathways of the patient, the measurement points being determined by the shape of the catheter in patient space; and registering a subset of the set of measurement points with the set of model points to generate a plurality of registration candidates. This exemplary method may further include: selecting one of the plurality of registration candidates for performing a medical procedure, and applying the selected registration candidate to the set of model points of the model to register the model to patient space.

[0009] An exemplary medical imaging system may include: a motion detector; a flexible catheter proximal to which is an instrument holder movable along an insertion stage; and a point collection device extending along the length of the flexible catheter. The exemplary system may further include a tracking system configured to register a set of measurement points collected by the point collection device to a model of one or more anatomical pathways. The tracking system can access a set of model points in a model space and collect a set of measurement points along the length of the flexible catheter inserted into one or more anatomical pathways in the patient space, where the measurement points are determined by the shape of the catheter. The tracking system can also assign a set of measurement points to a subset based on periodic physiological functions monitored by the motion detector; select a first registration candidate from a plurality of registration candidates based on the subset of measurement points; and apply the selected first registration candidate to a set of model points of the model to register the model to the patient space. It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are intended to provide an understanding of the disclosure rather than to limit its scope. In this regard, additional aspects, features, and advantages of the disclosure will be apparent to those skilled in the art from the following detailed description. Attached Figure Description

[0010] Figure 1This is a simplified schematic diagram of a remotely operated medical system according to an embodiment of the present disclosure.

[0011] Figure 2A This is a simplified schematic diagram of a medical device system utilizing various aspects of this disclosure.

[0012] Figure 2B This is a simplified schematic diagram of a medical device with extended medical tools according to some embodiments.

[0013] Figure 3A and Figure 3B This is a side view of the patient coordinate space of a medical device mounted on an insert assembly, according to some embodiments of the present disclosure.

[0014] Figure 4A , Figure 4B , Figure 4C and 4D The illustration shows the process during insertion into a human lung (Figure 2). Figure 3A , Figure 3B The remote end of the medical device system.

[0015] Figure 5 This is a flowchart illustrating a method for providing guidance in an image-guided surgical procedure according to an embodiment of the present disclosure.

[0016] Figure 6A , Figure 6B and Figure 6C The illustration shows the steps in a segmentation process for generating a human lung model for registration according to an embodiment of the present disclosure.

[0017] Figure 7 The illustration shows a flowchart of a portion of an image-guided surgical procedure according to one embodiment of the present disclosure.

[0018] Figure 8A and Figure 8B The illustration shows a phase-based binning process and an amplitude-based binning process for collecting measurement points within a patient's anatomical structure according to an embodiment of the present disclosure.

[0019] Figure 9 The illustration shows phase-based and amplitude-based binning of measurement points within a patient's anatomy according to one embodiment of the present disclosure.

[0020] Figure 10A , Figure 10B , Figure 10C and Figure 10D The illustration shows a depth-based binning of measurement points within a patient's anatomy according to one embodiment of the present disclosure.

[0021] Figure 11A and Figure 11B The illustration shows the registration between a centerline model of a patient's anatomy and a set of measurement points according to an embodiment of the present disclosure.

[0022] Figure 12A and Figure 12B The illustration shows the registration between a centerline model of a patient's anatomy and a set of measurement points according to another embodiment of the present disclosure.

[0023] Figure 13A and Figure 13B The illustration shows the display stage of a registration technique according to an embodiment of the present disclosure.

[0024] Examples of this disclosure and its advantages will be best understood by referring to the following detailed description. It should be appreciated that the same reference numerals are used to identify the same elements illustrated in one or more of the figures, wherein the identification is for the purpose of illustrating embodiments of this disclosure and is not intended to limit this disclosure. Detailed Implementation

[0025] In the following description, specific details of some embodiments consistent with this disclosure are set forth. Numerous specific details are set forth in order to provide a full understanding of the embodiments. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative and not restrictive. Other elements that are not specifically described herein but are within the scope and spirit of this disclosure can be implemented by those skilled in the art. Furthermore, to avoid unnecessary repetition, one or more features shown and described in connection with one embodiment may be incorporated into other embodiments unless otherwise specifically described or if one or more features would render the embodiment inoperable.

[0026] In some instances, well-known methods, procedures, components, and circuits are not described in detail so as not to unnecessarily obscure multiple aspects of the embodiments.

[0027] This disclosure describes the states of different instruments and instrument parts in three-dimensional space. As used herein, the term "position" refers to the orientation of an object or parts thereof in three-dimensional space (e.g., three translational degrees of freedom along Cartesian X, Y, and Z coordinates). As used herein, the term "orientation" refers to the rotational arrangement of an object or parts thereof (three rotational degrees of freedom, e.g., roll, pitch, and yaw). As used herein, the term "attitude" refers to the position of an object or parts thereof in at least one translational degree of freedom and the orientation of an object or parts thereof in at least one rotational degree of freedom (up to six degrees of freedom in total). As used herein, the term "shape" refers to a set of attitudes, positions, or orientations measured along the object.

[0028] Figure 1 This is a simplified schematic diagram of a remotely operated medical system 100 according to some embodiments. In some embodiments, the remotely operated medical system 100 may be adapted for use in procedures such as surgery, diagnosis, treatment, or biopsy. Figure 1 As shown, the medical system 100 generally includes a remote operating manipulator assembly 102 for operating the medical device 104 to perform various procedures on the patient P. The remote operating manipulator assembly 102 is mounted on or near the operator console T. An operator input system, referred to as the master control assembly 106, allows for... Figure 1 The operator O shown (e.g., surgeon, clinician, or physician) views the intervention site and controls the remote manipulator assembly 102.

[0029] The master control assembly 106 may be located at a physician control console, which is typically situated in the same room as the operating table T, such as on one side of the surgical table on which the patient P is located. However, it should be understood that the operator O may be located in a different room or in a completely different building from the patient P. The master control assembly 106 generally includes one or more control devices for controlling the remote operating manipulator assembly 102. The control devices may include any number of various input devices, such as joysticks, trackballs, data gloves, trigger guns, manual operation controllers, voice recognition devices, body motion or presence sensors and / or similar devices. To provide the operator O with a strong sense of direct control over the instrument 104, the control devices may be provided with the same degrees of freedom as the associated medical instrument 104. In this way, the control devices provide the operator O with a remote presentation or perception of the control devices and the medical instrument 104 as one entity.

[0030] In some embodiments, the control device may have more or fewer degrees of freedom than the associated medical device 104, and still provide remote presentation to the operator O. In some embodiments, the control device may optionally be a manual input device that moves in six degrees of freedom and may also include an actuable handle for actuating instruments (e.g., closing grippers, applying a potential to electrodes, delivering medication and / or similar effects).

[0031] The remote-operated manipulator assembly 102 supports the medical device 104 and may include a kinematic structure of one or more non-servo-controlled links (e.g., one or more links that can be manually positioned and locked in place, often referred to as a set-up structure) and a remote-operated manipulator. The remote-operated manipulator assembly 102 may optionally include multiple actuators or motors that drive inputs on the medical device 104 in response to commands from a control system (e.g., control system 112). The actuators may optionally include a drive system that, when coupled to the medical device 104, can advance the medical device 104 into a natural or surgically created anatomical opening. Other drive systems may move the distal end of the medical device 104 with multiple degrees of freedom, which may include three linear degrees of freedom (e.g., linear motion along the X, Y, Z Cartesian coordinate axes) and three rotational degrees of freedom (e.g., rotation about the X, Y, Z Cartesian coordinate axes). Furthermore, the actuator can be used to actuate the articulated end effector of the medical device 104 to grasp tissue in the jaws of a biopsy device and / or similar equipment. Actuator position sensors (such as solvers, encoders, potentiometers, and other mechanisms) can provide the medical system 100 with sensor data describing the rotation and orientation of the motor shaft. This position sensor data can be used to determine the motion of the object manipulated by the actuator.

[0032] The remote-operated medical system 100 may include a sensor system 108 having one or more subsystems for receiving information about the instruments of the remote-operated manipulator assembly 102. These subsystems may include: a position / orientation sensor system (e.g., an electromagnetic (EM) sensor system); a shape sensor system for determining the position, orientation, velocity, rate, attitude, and / or shape along the distal end and / or one or more segments that may constitute the flexible body of the medical device 104; and / or a visualization system for acquiring images from the distal end of the medical device 104.

[0033] The remote-operated medical system 100 also includes a display system 110 for displaying images or representations of the surgical site and medical device 104 generated by a subsystem of the sensor system 108. The display system 110 and the main control assembly 106 can be oriented such that an operator O can control the medical device 104 and the main control assembly 106 under remotely presented perception.

[0034] In some embodiments, medical device 104 may have a visualization system (discussed in more detail below) that may include an observation endoscope assembly that records concurrent or real-time images of the surgical site and provides the images to an operator or operator O via one or more displays of medical system 100 (such as one or more displays of display system 110). The concurrent images may be, for example, two-dimensional or three-dimensional images acquired by an endoscope positioned within the surgical site. In some embodiments, the visualization system includes an endoscope component that may be integrally or removably coupled to medical device 104. However, in some embodiments, a separate endoscope attached to a separate manipulator assembly may be used in conjunction with medical device 104 to image the surgical site. In some examples, the endoscope may include one or more mechanisms for cleaning one or more lenses of the endoscope when one or more lenses are partially and / or completely blocked by fluids and / or other materials encountered by the endoscope. In some examples, the one or more cleaning mechanisms may optionally include an air and / or other gas delivery system that can be used to fire a stream of air and / or other gas to clean one or more lenses. Examples of one or more cleaning mechanisms are discussed in more detail in International Publication WO / 2016 / 025465 (filed August 11, 2016) (publication “Systems and Methods for Cleaning an Endoscopic Instrument”), which is incorporated herein by reference in its entirety. The visualization system may be implemented as hardware, firmware, software, or a combination thereof that interacts with or is otherwise executed by one or more computer processors (which may include the processor of control system 112). The processor of control system 112 may execute instructions corresponding to the methods and operator described herein.

[0035] Display system 110 can also display images of surgical sites and medical instruments acquired through a visualization system. In some examples, remote-operated medical system 100 can configure the controls of medical device 104 and master assembly 106 such that the relative positions of the medical device are analogous to the relative positions of the operator O's eyes and hands. In this way, operator O can manipulate medical device 104 and hand controls as if observing the workspace with a substantially realistic sense of presence. This realistic sense of presence means that the image presentation is a realistic perspective image simulating the viewpoint of a physician physically manipulating medical device 104.

[0036] In some examples, the display system 110 can present images of surgical sites recorded preoperatively or intraoperatively using image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluorescence microscopy, thermal imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like. Preoperative or intraoperative image data can be presented as two-dimensional, three-dimensional, or four-dimensional images (including, for example, time-based or velocity-based information), and / or as images from models generated from preoperative or intraoperative image datasets.

[0037] In some embodiments, typically for the purpose of image-guided surgical procedures, the display system 110 may display a virtual navigation image in which the actual orientation of the medical device 104 is registered with preoperative or concurrent images / models (i.e., dynamic reference). This can be done to present a virtual image of the internal surgical site to the operator O from the perspective of the medical device 104. In some examples, the perspective may be from the tip of the medical device 104. Images and / or other graphic or alphanumeric indicators of the tip of the medical device 104 may be overlaid on the virtual image to aid the operator O in controlling the medical device 104. In some examples, the medical device 104 may be invisible in the virtual image.

[0038] In some embodiments, display system 110 may display a virtual navigation image in which the actual orientation of medical device 104 is registered with a preoperative or concurrent image to present a virtual image of medical device 104 within the surgical site to operator O from an external perspective. An image of a portion of medical device 104 or other graphic or alphanumeric indicators may be overlaid on the virtual image to assist operator O in controlling medical device 104. As described herein, a visual representation of data points may be rendered to display system 110. For example, measured data points, moving data points, registered data points, and other data points described herein may be displayed visually on display system 110. Data points may be visually represented on the user interface as multiple points or dots on display system 110 or as a rendered model (such as a grid or wire model created based on a set of data points). In some examples, data points may be color-coded according to the data they represent. In some embodiments, the visual representation may be refreshed in display system 110 after each processing operation has been performed to change the data points. In some embodiments, a virtual navigation image may be displayed on a display 110, which depicts a model of the anatomical pathway from the perspective of an instrument inserted along or through the corresponding actual anatomical pathway.

[0039] The remote-operated medical system 100 may also include a control system 112. The control system 112 includes at least one memory and at least one computer processor (not shown) for implementing control between the medical device 104, the main control assembly 106, the sensor system 108, and the display system 110. The control system 112 also includes programmed instructions (e.g., a non-transitory machine-readable medium storing these instructions) to implement some or all of the methods described according to various aspects of the disclosure herein, including instructions for providing information to the display system 110. While the control system 112... Figure 1 The simplified schematic is shown as a single block, but the system may include two or more data processing circuits, with some processing optionally performed on or near the remote operating manipulator assembly 102, and other processing performed at the master control assembly 106 and / or similarly. The processor of the control system 112 may execute instructions, including instructions corresponding to the processing disclosed herein and described in more detail below. Any wide variety of centralized or distributed data processing architectures can be utilized. Similarly, programmable instructions may be implemented as several independent programs or subroutines, or they may be integrated into several other aspects of the remote operating system described herein. In one embodiment, the control system 112 supports wireless communication protocols such as Bluetooth, IrDA (Infrared Data Communication), HomeRF (Home Radio Frequency), IEEE 802.11, DECT (Digital Enhanced Cordless Communication), and wireless telemetry.

[0040] In some embodiments, the control system 112 may receive force and / or torque feedback from the medical device 104. In response to this feedback, the control system 112 may transmit a signal to the master assembly 106. In some examples, the control system 112 may transmit a signal commanding one or more actuators of the remote operating manipulator assembly 102 to move the medical device 104. The medical device 104 may extend to an internal surgical site within the body of the patient P via an opening in the patient P's body. Any suitable conventional and / or specialized actuators may be used. In some examples, one or more actuators may be separate from or integrated with the remote operating manipulator assembly 102. In some embodiments, one or more actuators and the remote operating manipulator assembly 102 are provided as part of a remote operating trolley positioned adjacent to the patient P and the operating table T.

[0041] The control system 112 may optionally further include a virtual visualization system to provide navigational assistance to the operator O when controlling the medical device 104 during an image-guided surgical procedure. Virtual navigation using the virtual visualization system may be based on a reference to a preoperative or intraoperative dataset of acquired anatomical pathways. The virtual visualization system processes images of the surgical site imaged using imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluorescence microscopy, thermal imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like. Software, which may be used in conjunction with manual input, is used to convert the recorded images into segmented two-dimensional or three-dimensional composite representations of local or global anatomical organs or regions. Image datasets are associated with composite representations. Composite representations and image datasets describe various orientations and shapes of pathways and their connectivity. Images used to generate composite representations may be recorded preoperatively or intraoperatively during a clinical procedure. In some embodiments, the virtual visualization system may use a standard representation (i.e., not patient-specific) or a mixture of a standard representation and patient-specific data. The composite representation and any virtual image generated from it can represent the static posture of a deformable anatomical region during one or more motion phases (e.g., during the inspiratory / expiratory cycle of the lungs).

[0042] During the virtual navigation procedure, sensor system 108 can be used to calculate the approximate orientation of medical device 104 relative to the anatomical structure of patient P. This orientation can be used to generate both a macroscopic horizontal (external) tracking image of the anatomical structure of patient P and a virtual internal image of the anatomical structure of patient P. The system can implement one or more electromagnetic (EM) sensors, fiber optic sensors, and / or other sensors to register and display surgical images recorded during medical procedures and preoperative procedures, such as those from virtual visualization systems, which are known. For example, U.S. Patent Application No. 13 / 107,562 (filed May 13, 2011) (publishing “Medical System Providing Dynamic Registration of a Model of an Anatomic Structure for Image-Guided Surgery”) discloses such a system, which is incorporated herein by reference in its entirety. The remotely operated medical system 100 may further include optional operating and support systems (not shown), such as lighting systems, steering control systems, flushing systems, and / or suction systems. In some embodiments, the remotely operated medical system 100 may include one or more remotely operated manipulator assemblies and / or one or more master control assemblies. The exact number of remotely operated manipulator assemblies will depend, among other factors, on the surgical procedure and the space constraints within the operating room. Master control assemblies 106 may be collocated, or they may be positioned in separate locations. Multiple master control assemblies allow more than one operator to control one or more remotely operated manipulator assemblies in various combinations.

[0043] Figure 2A This is a simplified schematic diagram of a medical device system 200 according to some embodiments. In some embodiments, the medical device system 200 may be used as a medical device 104 in an image-guided medical procedure performed using a remotely operated medical system 100. In some examples, the medical device system 200 may be used in non-remotely operated exploratory procedures or in procedures involving conventionally manually operated medical devices (such as endoscopes). Optionally, the medical device system 200 may be used to collect (i.e., measure) a set of data points corresponding to orientation within an anatomical pathway of a patient (such as patient P).

[0044] The medical device system 200 includes an elongated device 202 (such as a flexible catheter) coupled to a drive unit 204. The elongated device 202 includes a flexible body 216 having a proximal end 217 and a distal or tip portion 218. In some embodiments, the flexible body 216 has an outer diameter of approximately 3 mm. Other flexible bodies may have larger or smaller outer diameters.

[0045] The medical device system 200 further includes a tracking system 230 for determining the position, orientation, velocity, rate, attitude, and / or shape of the distal end 218 and / or one or more segments 224 along the flexible body 216 using one or more sensors and / or imaging devices as described in further detail below. The entire length of the flexible body 216 between the distal end 218 and the proximal end 217 can be effectively divided into multiple segments 224. The tracking system 230 is used if the medical device system 200 is aligned with the medical device 104 of the remotely operated medical system 100. The tracking system 230 may optionally be implemented as hardware, firmware, software, or a combination thereof that interacts with or is otherwise executed by one or more computer processors, which may include... Figure 1 The processor of the control system 112.

[0046] Tracking system 230 may optionally use shape sensor 222 to track distal end 218 and / or one or more segments 224. Shape sensor 222 may optionally include an optical fiber aligned with flexible body 216 (e.g., provided within an internal passage (not shown) or externally mounted). In one embodiment, the optical fiber has a diameter of approximately 200 μm. In other embodiments, this size may be larger or smaller. The optical fiber of shape sensor 222 forms an optical fiber bending sensor for determining the shape of flexible body 216. In an alternative, an optical fiber including a fiber Bragg grating (FBG) is used to provide strain measurements in one or more dimensions within the structure. The optical fiber of shape sensor system 222 allows for the simultaneous collection of a set of measurement points that describe the position of various portions of shape sensor 222 along the length of flexible conduit body 216 at a single point in time. Various systems and methods for monitoring the shape and relative positions of optical fibers in three dimensions are described in U.S. Patent Application No. 11 / 180,389 (filed July 13, 2005) (publishing “Fiber optic position and shape sensing device and method relating thereto”); U.S. Patent Application No. 12 / 047,056 (filed July 16, 2004) (publishing “Fiber-optic shape and relative positions sensing”); and U.S. Patent No. 6,389,187 (filed June 17, 1998) (publishing “Optical Fiber BendSensor”), which are incorporated herein by reference in their entirety.

[0047] In some embodiments, the sensor may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. In some embodiments, the shape of the elongated device may be determined using other techniques. For example, the history of the distal attitude of the flexible body 216 can be used to reconstruct the shape of the flexible body 216 during that period. In some embodiments, the tracking system 230 may optionally and / or additionally use a position sensor system 220 to track the distal end 218. The position sensor system 220 may be a component of an EM sensor system having the position sensor system 220, which includes one or more conductive coils capable of withstanding an externally generated electromagnetic field. Each coil of the EM sensor system 220 then generates an induced electrical signal having characteristics that depend on the position and orientation of the coil relative to the externally generated electromagnetic field. In some embodiments, the position sensor system 220 may be configured and positioned to measure six degrees of freedom (e.g., three position coordinates X, Y, Z and three orientation angles for pitch, yaw, and roll of an indicator base point) or five degrees of freedom (e.g., three position coordinates X, Y, Z and two orientation angles for pitch and yaw of an indicator base point). A further description of the position sensor system is provided in U.S. Patent No. 6,380,732 (filed August 11, 1999) (published "Six-Degree of Freedom Tracking System Having a Passive Transponder on the Object Being Tracked"), which is incorporated herein by reference in its entirety. In some embodiments, the shape sensor 222 can also be used as a position sensor because the shape of the sensor 222, along with information about the orientation of the base of the shape sensor 222 (referred to as "patient space" in the patient's fixed coordinate system), allows the calculation of the orientation along various points of the shape sensor, including the distal tip.

[0048] In some embodiments, the tracking system 230 may alternatively and / or additionally rely on historical posture, position, or orientation data stored for known points of the instrument system along cycles of alternating movements (such as breathing). This stored data can be used to develop shape information about the flexible body 216. In some examples, a series of position sensors (not shown) (such as electromagnetic (EM) sensors similar to those of position sensor 220) may be positioned along the flexible body 216 and then used for shape sensing. In some examples, a history of data acquired during a procedure from one or more of these sensors may be used to represent the shape of the elongated device 202, particularly if the anatomical pathway is substantially static. The tracking system 230 may include a position sensor system 220 and a shape sensor system 222 to determine the position, orientation, velocity, posture, and / or shape of the distal end 218 and one or more segments 224 along the instrument 200. The tracking system 230 may be implemented as hardware, firmware, software, or a combination thereof, interacting with or otherwise executed by one or more computer processors, which may include the processor of the control system 116.

[0049] The flexible body 216 includes a passage 221 that is sized and shaped to receive a medical device 226. Figure 2BThis is a simplified schematic diagram of a flexible body 216 with the medical device 226 extended, according to some embodiments. In some embodiments, the medical device 226 can be used for procedures such as surgery, biopsy, ablation, irradiation, perfusion, or aspiration. The medical device 226 can be deployed through the passage 221 of the flexible body 216 and used at a target location within an anatomical structure. The medical device 226 may include, for example, an image acquisition probe, a biopsy instrument, a laser ablation fiber, and / or other surgical, diagnostic, or therapeutic tools. Medical tools may include end effectors with a single working component, such as scalpels, blunt blades, optical fibers, electrodes, and / or the like. Other end effectors may include, for example, forceps, grippers, scissors, clamps, and / or the like. Other end effectors may further include electrically activated end effectors, such as electrosurgical electrodes, transducers, sensors, and / or the like. In various embodiments, the medical device 226 is a biopsy instrument that can be used to remove a sample of tissue or cells from a target anatomical location. Medical device 226 can be used in conjunction with an image acquisition probe also within flexible body 216. In various embodiments, medical device 226 may be an image acquisition probe comprising a distal portion of a stereo camera or single-field-of-view camera having, at or near, the distal end 218 of flexible body 216 for acquiring images (including video images), which are processed by visualization system 231 for display and / or provided to tracking system 230 to support tracking of one or more of distal end 218 and / or segments 224. The image acquisition probe may include a cable coupled to the camera for transmitting the acquired image data. In some examples, the image acquisition device may be a bundle of optical fibers, such as a fiber optic endoscope, coupled to visualization system 231. The image acquisition device may be monospectral or multispectral, for example, acquiring image data in one or more of the visible, infrared, and / or ultraviolet spectra. Alternatively, medical device 226 itself may be an image acquisition probe. Medical device 226 may be advanced from an opening in passage 221 to perform a procedure and then retracted into the passage when the procedure is complete. The medical device 226 can be removed from the proximal end 217 of the flexible body 216 or from another optional device port (not shown) along the flexible body 216.

[0050] Medical device 226 may additionally accommodate a cable, linkage, or other actuation control (not shown) extending between its proximal and distal ends to controllably bend the distal end of medical device 226. Steering devices are described in detail in U.S. Patent No. 7,316,681 (filed October 4, 2005) (published "Articulated Surgical Instrument for Performing Minimally Invasive Surgery with Enhanced Dexterity and Sensitivity") and U.S. Patent Application No. 12 / 286,644 (filed September 30, 2008) (published "Passive Preload and Capstan Drive for Surgical Instruments"), both of which are incorporated herein by reference in their entirety.

[0051] The flexible body 216 may also be accommodated between the drive unit 204 and the distal end 218 to controllably bend the cables, linkages, or other steering controls (not shown) of the distal end 218, as illustrated, for example, by the dashed line depicting the distal end 218. In some examples, at least four cables are used to provide independent “up-down” steering to control the pitch of the distal end 218 and “left-right” steering to control the yaw of the distal end 281. A steerable elongated device is described in detail in U.S. Patent Application No. 13 / 274,208 (filed October 14, 2011) (published “Catheter with Removable Vision Probe”), which is incorporated herein by reference in its entirety. In embodiments where the medical device system 200 is actuated by a remotely operated assembly, the drive unit 204 may include a drive input device that is removably coupled to and receives power from a drive element (such as an actuator) of the remotely operated assembly. In some embodiments, the medical device system 200 may include clamping features, manual actuators, or other components for manually controlling the movement of the medical device system 200. The elongated device 202 may be steerable, or alternatively, the system may be non-steerable without an integrated mechanism for operator control of bending of the distal end 218. In some examples, one or more cavities are defined within the walls of the flexible body 216, through which the medical device can be deployed and used at a target surgical location.

[0052] In some embodiments, the medical device system 200 may include flexible bronchial instruments, such as bronchoscopes or bronchial tubes for use in the examination, diagnosis, biopsy, or treatment of the lungs. The medical device system 200 is also suitable for navigating and treating other tissues via natural or surgically established pathways within any of the various anatomical systems, including the colon, intestine, kidneys and renal calyces, brain, heart, circulatory system including the vascular system, and / or the like.

[0053] Information from tracking system 230 can be sent to navigation system 232, where it is combined with information from visualization system 231 and / or a preoperatively acquired model to provide real-time location information to the physician or other operator. In some examples, the real-time location information can be displayed... Figure 1 The display system 110 is used for control of the medical device system 200. In some examples, Figure 1 The control system 116 can use position information as feedback to position the medical device system 200. Various systems for registering and displaying surgical instruments and surgical images using fiber optic sensors are disclosed in U.S. Patent Application No. 13 / 107,562, filed May 13, 2011, entitled "Medical System Providing Dynamic Registration of a Model of an Anatomic Structure for Image-Guided Surgery," which is incorporated herein by reference in its entirety.

[0054] In some examples, the medical device system 200 can Figure 1 The medical system is operated remotely within 100. In some embodiments, Figure 1 The remote operating manipulator assembly 102 can be replaced by a direct operator control unit. In some examples, the direct operator control unit may include various handles and operator interfaces for handheld operation of the instrument.

[0055] Figure 3A and Figure 3B This is a simplified schematic diagram of a side view of a medical device mounted on an insert assembly, according to some embodiments, in patient coordinate space. Figure 3A and Figure 3B As shown, the surgical environment 300 includes the patient P being located in... Figure 1The patient P is positioned on the operating table T. In the sense that significant patient movement is restricted by sedation, restraint, and / or other means, the patient P may be stationary within the surgical environment. Periodic anatomical movements (including the patient P's breathing and cardiac movements) may continue unless the patient is instructed to hold their breath to temporarily halt respiratory movements. Therefore, in some embodiments, data may be collected at specific phases of respiration and used for tagging and identification. In some embodiments, the phase in which data is collected may be inferred based on physiological information collected from the patient P. Within the surgical environment 300, a point collection device 304 is coupled to an instrument holder 306. In some embodiments, the point collection device 304 may use an EM sensor, shape sensor, and / or other sensor modalities. The instrument holder 306 is mounted to an insertion stage 308 fixed within the surgical environment 300. Alternatively, the insertion stage 308 may be movable but has a known orientation within the surgical environment 300 (e.g., known via a tracking sensor or other tracking device). The instrument holder 306 may be a component of a remotely operated manipulator assembly (e.g., remotely operated manipulator assembly 102) coupled to the point-collecting instrument 304 to control insertion motion (i.e., movement along axis A) and optionally control movement of the distal end 318 of the elongated device 310 in multiple directions (including yaw, pitch, and roll). The instrument holder 306 or insertion stage 308 may include actuators (not shown), such as servo motors, for controlling movement of the instrument holder 306 along the insertion stage 308.

[0056] An elongated device 310 is coupled to an instrument body 312. The instrument body 312 is coupled to and secured relative to an instrument holder 306. In some embodiments, a fiber optic shape sensor 314 is secured at a proximal point 316 on the instrument body 312. In some embodiments, the proximal point 316 of the fiber optic shape sensor 314 may be movable together with the instrument body 312, but the orientation of the proximal point 316 may be known (e.g., known via a tracking sensor or other tracking device). The shape sensor 314 measures the shape from the proximal point 316 to another point (such as the distal end 318 of the elongated device 310). The point collection device 304 may be substantially similar to the medical device system 200.

[0057] As the instrument body 312 moves along the insertion axis A on the insertion stage 308, the position measuring device 320 provides information about the position of the instrument body 312. The position measuring device 320 may include a solver, encoder, potentiometer, and / or other sensors that determine the rotation and / or orientation of the actuator that controls the movement of the instrument carrier 306 and thus the movement of the instrument body 312. In some embodiments, the insertion stage 308 is linear. In some embodiments, the insertion stage 308 may be arcuate, or have a combination of arcuate and linear segments.

[0058] Figure 3A The device body 312 and device holder 306 are shown in their retracted position along the insertion stage 308. In this retracted position, the proximal point 316 is located at position L0 on axis A. In this position along the insertion stage 308, the A component of the orientation of the proximal point 316 can be set to zero and / or another reference value to provide a basic reference describing the position of the device holder 306 and therefore the proximal point 316 on the insertion stage 308. With the device body 312 and device holder 306 in this retracted position, the distal end 318 of the elongated device 310 can be positioned just inside the access port of the patient P. Also in this position, the position measuring device 320 can be set to zero and / or another reference value (e.g., I = 0). Figure 3B In this advanced position, the instrument body 312 and instrument holder 306 have been advanced along a track that may be linear or arcuate on the insertion stage 308, and the distal end 318 of the elongated device 310 has been advanced into the patient P. In this advanced position, the proximal point 316 is located at position L1 on axis A. In some examples, encoder and / or other position data from one or more actuators controlling the movement of the instrument holder 306 along the insertion stage 308 and / or one or more position sensors associated with the instrument holder 306 and / or the insertion stage 308 are used to determine the position L1 of the proximal point 316 relative to position L0. x In some examples, position L x The distal end 318 of the elongated device 310 can be further used as an indicator of the distance or insertion depth within the pathway of the anatomical structure of the patient P. During insertion and while inserted into the anatomical structure of the patient P, the shape sensor 314 can be used to collect measurement data points within the anatomical structure of the patient P.

[0059] Embodiments of the point collection device 304 can use any number of modalities (including EM sensing and shape sensing) to collect measurement points. When measurement points are collected from within the patient's anatomical pathway, these points are stored in a data storage device, such as memory. The set of measurement points can be stored in a database that includes, but may include, at least some, but all, of the measurement points acquired during or immediately prior to the procedure. The set of measurement points can be binned, categorized, or filtered to produce subsets of measurement points that are somehow related, such as during a common phase or part of a phase of periodic physiological movement (e.g., breathing). When stored in memory, each point can be represented by data including the point's coordinates, timestamp, and relative sensor position or a single sensor ID (when multiple sensors distributed along the length of the point collection device 304 are used to simultaneously determine the orientation of several points). The binned sets of measurement points can be stored in separate data structures and / or separate portions of memory. In some embodiments, the data representing each point may also include a respiratory phase marker indicating the respiratory phase of the patient for which the point was collected. The patient motion tracker 322 can be used to monitor and detect the movement of the patient P. This movement can include total movement as well as periodic physiological movements, such as breathing. Therefore, the motion tracker 322 can be used as a physiological phase detector. The motion tracker 322 can be an optical tracking system or any other suitable system. In other embodiments, breathing can be monitored using a ventilator or any other technology.

[0060] Figure 4A , Figure 4B , Figure 4C and Figure 4D The diagram shows Figure 3A and Figure 3B The catheter 310 passes through Figure 1 The advancement of the anatomical pathways 402 of the lung 400 of patient P in Figure 3. These pathways 402 include the trachea and bronchi. As the carrier 306 moves along the insertion stage 308, the catheter 310 is advanced, and the surgeon S can steer the distal end 318 of the catheter 310 to navigate through the anatomical pathways 402. During navigation through the anatomical pathways 402, the catheter 310 presents a shape that can be “read” by a shape sensor 314 extending within the catheter 310. As described herein, measurement points obtained using the flexible catheter 310 can be assigned to one or more bins based on a variety of factors. For example, measurement points can be assigned to bins corresponding to portions of periodic physiological movements, such as breathing. Measurement points can also be assigned to bins corresponding to the insertion depth of the catheter 310, such that points measured within a specific generation of the tracheal or bronchial pathway can be collected or included in a single bin, even as the distal end 318 of the catheter 310 is advanced through the anatomical pathways 402.

[0061] Figure 5This is a flowchart illustrating a general method 500 for image-guided surgical procedures. At procedure 502, preoperative or intraoperative image data is acquired from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermal imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, or nanotube X-ray imaging. The preoperative or intraoperative image data may correspond to two-dimensional, three-dimensional, or four-dimensional images (including, for example, time-based or velocity-based information). For example, the image data may represent... Figures 4A to 4D Human lung 400. At process 504, a computer system, operating alone or in combination with manual input, converts the recorded images into segmented two-dimensional or three-dimensional composite representations or models of parts or entire anatomical organs or regions. For example, Figure 6A The diagram shows Figures 4A to 4D A segmented model 600 of the lung 400. Due to naturally occurring limitations or operator-defined limitations, the segmented model 600 may not include all pathways present in the human lung, but may include some pathways 601. For example, relatively narrow lungs and / or distal pathways may not be fully included in the segmented model 600. The segmented model 600 may be a three-dimensional model, such as a mesh model or other suitable model, which includes the walls defining the lumen of the lung or pathways. Typically, the model provides mechanisms or means for distinguishing points within and outside anatomical regions. Composite representations and image datasets describe the various orientations and shapes of pathways and their connectivity, and unwanted portions of anatomical structures included in preoperative or intraoperative image data may be omitted. In some embodiments, the model 600 may include particularly desired features, such as suspected tumors or other tissue portions of interest.

[0062] During the segmentation process, the image is divided into segments or elements (e.g., pixels or voxels) that share certain features or computational properties such as color, density, intensity, and texture. This segmentation process results in two-dimensional or three-dimensional reconstruction, which forms a model of the target anatomical structure based on the acquired image, such as model 600. To represent the model, the segmentation process may delineate a set of voxels representing the target anatomical structure and then apply a function, such as a marching cube function, to generate a 3D surface surrounding the voxels. The model can be created by generating a mesh, volume, or voxel map. This model can be displayed on display 110 to help surgeons visualize anatomical structures, such as the internal pathways of the lungs.

[0063] Additionally or alternatively, the model may include a centerline model comprising a set of interconnected line segments or points extending through the center of the modeled path. Figure 6BAn exemplary centerline model 602, derived from model 600 or directly from imaging data, is shown. The centerline segmentation model 602 may include a set of three-dimensional straight lines or a set of curved lines corresponding to the approximate center of the pathway contained in the segmentation model 602. The higher the resolution of the model, the more accurately the set of straight lines or curved lines will correspond to the center of the pathway. Representing the lung with the centerline segmentation model 602, which represents the walls of the pathway in model 600, can provide a smaller dataset that can be processed more efficiently by one or more processors or processing cores compared to the dataset of the segmentation model 602. In this way, the functionality of the control system 112 can be improved.

[0064] like Figure 6B As shown, the centerline segmentation model 602 includes several branch points, some of which are located at... Figure 6B The points are highlighted for visibility. Branch points A, B, C, D, and E are shown at each of the several branch points. Branch point A may represent the point in the model where the trachea divides into the left and right main bronchuses. The right main bronchus may be identified in the centerline segmentation model 602 as being located between branch points A and B. Similarly, secondary bronchi are identified by branch points B and C, and between branch points B and E. Another generation may be defined between branch points C and D. Each of these generations may be associated with a representation of the luminal diameter of the corresponding pathway. In some embodiments, model 602 may include an average diameter value for each segmental generation. The average diameter value may be a patient-specific value or a more general value derived from multiple patients.

[0065] In the case where the model includes a centerline model containing a set of interconnected line segments, those line segments can be converted into a point cloud or point set 604, called model points, which are composed of... Figure 6C The dashed lines represent the model. By converting line segments into points, the required number of model points corresponding to the interconnecting line segments can be manually or automatically selected to represent the centerline model 602 (and thus model 600) during the registration process. In the data, each point in this set of model points 604 may include coordinates, such as a set of X... M Y M and Z M The coordinates or other coordinates that identify the orientation of each point in the three-dimensional model space. In some embodiments, each point may include a generation identifier that identifies the path generation associated with the point and / or the diameter or radius value associated with that portion of the centerline segmented model 602. In some embodiments, information describing the radius or diameter associated with a given point may be provided as part of a separate dataset.

[0066] Generate a segmented centerline model 602 and use it as... Figure 6CAfter the point set 604 shown is stored in the data, the model points 604 can be retrieved from the data storage device for use in image-guided surgical procedures. To use the centerline segmentation model 602 and model 600 in an image-guided surgical procedure, the model points 604 can be registered to associate the modeled pathways in model 600 with the actual anatomical structures of the patient present in the surgical environment.

[0067] return Figure 5 At step 506, measurement points can be obtained from the patient's anatomical structures corresponding to the anatomical model, such as... Figures 3A to 3B and Figures 4A to 4D As shown. At process 508, before and / or during the image-guided surgical procedure on the patient, anatomical model data is registered to the patient's anatomical structures. Typically, registration involves matching measurement points to points on the model using rigid and / or non-rigid transformations. Measurement points can be generated using landmarks in the anatomical structures, electromagnetic coils or shape sensor systems that are scanned and tracked during the procedure. Measurement points can be generated for use with iterative nearest point (ICP) techniques, or another point set registration method can be used during the registration process within the scope of this disclosure.

[0068] Following process 508, where the anatomical model is registered to the patient's anatomy so that medical devices positioned relative to the patient's anatomy can be represented relative to the anatomical model, the collection of measurement points from the patient's anatomy can continue. Registration can be updated at process 510 when new measurement points are added. Registration updates can be performed continuously throughout the surgical procedure. In this way, variations due to patient movement (both coarse and periodic physiological movements) can be compensated for. Furthermore, measurement points can be assigned to various subsets of points using one or more criteria, and these criteria can identify some subsets or specific subsets as better than others in registering the patient's anatomy to the anatomical model. Numerical weights can also be assigned to points, reflecting a partial probability that any subset correctly assigns a point to a point. However, a subset that may provide optimal registration in the first time period may become less desirable in later time periods. Updating the registration to use the subset of measurement points that provides optimal registration at that time ensures that the best and most useful relationship between the model and the anatomy is maintained. When a statistically significant change occurs in the registration quality (e.g., a significant change in the error values ​​associated with the registration), a subset of the measurement points associated with the registration can be cleared by deleting all points or can have weighted measurements applied to it.

[0069] Other registration methods for image-guided surgery typically involve the use of techniques based on electromagnetic or impedance sensing. Metallic objects or certain electronic devices used in the surgical environment can introduce interference that impairs the quality of the sensed data. Other registration methods may also hinder clinical workflows. Some embodiments of the systems and methods described herein perform registration based on ICP or another point-set registration algorithm, along with the calibration movement of point-collecting instruments and fiber optic shape sensors, thereby eliminating or minimizing disruptions in the surgical environment. Other registration techniques can be used to register a set of measurement points to a preoperative model or a model obtained using another modality. In the embodiments described below, EM sensors on the patient and instruments, as well as the optical tracking system of the instruments, can be eliminated.

[0070] Figure 7 This is a flowchart illustrating a method 700 for providing guidance to a clinician in an image-guided surgical procedure on a patient P in a surgical setting 300, according to some embodiments of the present disclosure. Method 700 in Figure 7 The diagram illustrates a set of blocks, steps, operations, or procedures. Not all illustrated or enumerated operations can be performed in all embodiments of method 700. Additionally, Figure 7 Additional operations not explicitly stated may be included before, after, between, or as part of the listed operations. Some embodiments of method 700 may include machine-readable instructions corresponding to the operations of method 700 and stored in memory. These instructions may be executed by a processor, such as the processor of control system 112.

[0071] Therefore, some embodiments of method 700 may begin at operation 702, where the set of measurement points is collected along the length of the catheter or other medical device. For example, the catheter may be Figure 3A and Figure 3B The point collection device 304, or the medical device system 200 of Figure 2, may be used. The catheter may include a fiber optic shape sensor, which can be used to generate multiple three-dimensional points representing the shape of the catheter. The three-dimensional points can be correlated with the patient space via the proximal end having the fiber optic shape sensor and a known or detected orientation.

[0072] In practice, the distal end 318 of catheter 310 can pass through the anatomical pathway of patient P (e.g., the airway of the patient's lungs), and orientation data of the distal end of the catheter and other points along the shape of the shape sensor are recorded via data from shape sensor 314. This orientation data can be included or processed to obtain a set of measurement points as described herein. More specifically, movement of the distal tip of catheter 610 can be controlled remotely, manually, or automatically (e.g., via master assembly 106) to measure a portion of the anatomical pathway.

[0073] For example, a remote operation control signal can move the bracket 306 along axis A, causing the distal end 318 of the catheter to advance or retract within the anatomical pathway. Alternatively or additionally, the remote operation control signal can cause actuation of a control member extending within a surgical instrument to move the distal end 318 in a series of movements including deflection, pitch, and roll. As the catheter moves within multiple pathways, shape sensor data (and / or other positional data in other embodiments excluding shape sensors) of multiple orientations of the distal tip are collected. In some embodiments, the catheter may extend to approximately three branches, penetrating deep into each pathway. In some embodiments, the catheter may extend through or into approximately three or more branches on each side of the lung. As the diameter of the flexible catheter 310 decreases and / or the flexibility of the flexible catheter 310 increases, the number of passages accessible to the catheter 310 can increase.

[0074] In some embodiments, a calibration procedure may be performed before collecting measurement points along the relative position and / or orientation of the sensor reference point along the insertion path using a position measurement device such as point collection device 304 or other suitable means. For example, when the bracket 306 moves from the retracted position of point 316 at azimuth L0 to the forward position of point 316 at azimuth L1, Figure 3A and Figure 3B The point collection device 304 can be used to determine the position and orientation of point 316. The calibration procedure can determine the direction of movement of point 316 with each change in the position measuring device 320. For example, the distal end 318 of catheter 310 can be held in a fixed position while the instrument body is routed along the insertion stage 308. As the instrument body is routed along the insertion stage, the position and orientation data collected from the fixed point 316 by the shape sensor are cross-correlated with the position measuring device data, thereby calibrating the movement of point 316 along axis A of the insertion stage 308.

[0075] Optionally, in operation 703, the control system 112 may receive status input. For example, the status input may be a periodic physiological motion signal describing the patient P's breathing or heartbeat. In other embodiments, other periodic physiological motion signals may be received and used. These will be discussed in further detail below. Figure 8A and Figure 8B The illustration shows a single cycle of an exemplary periodic physiological motion signal 800. The periodic physiological motion signal includes a period T shown along the x-axis and an amplitude shown along the y-axis. Figure 8A and Figure 8BThe signals shown are simplified to more clearly convey certain aspects of this disclosure. Other state inputs may include non-physiological inputs such as instrument speed, force, applied strain, or orientation. Other state inputs may include detected camera obstructions (e.g., due to fog or debris). Other state inputs may include marked measurement points designated as belonging to a specific anatomical structure or within a specific anatomical pathway. Such marked points can serve as “ground truth” points during registration.

[0076] At operation 704, measurement points can be assigned to multiple subsets of measurement points. In some embodiments, operations 702 and 704 can be performed substantially simultaneously. In other embodiments, operation 702 may include storing the set of measurement points in a point pool in memory, and operation 704 may include retrieving the set of measurement points from memory and then assigning the retrieved points to multiple subsets. Assigning measurement points to subsets may be referred to herein as “binning.” Measurement points may each be assigned to one of several bins. A bin may be a specific portion of a data structure and / or memory. Figure 8A and Figure 8B The illustrations show some exemplary ways that measurement points can be assigned to specific bins. Assigning each point to an arbitrary bin can be binary (explicitly inside or outside a given bin), or it can involve soft assignments to arbitrary bins represented by some fractional weighting.

[0077] like Figure 8A As shown, the periodic physiological motion signal 800 can be used to divide the collected measurement points into bins. Figure 8A The illustration shows binning based on time or period, where the time at which a specific measurement point is obtained indicates the bin to which that point should be assigned. It should be noted that when using a fiber optic shape sensor to collect measurement points, multiple points are collected at once. In this case, all points collected from the fiber optic shape sensor can be assigned to the same bin or subset of points. For example, if a point is collected from fiber optic sensor 314 between 0 and 1 / 8 of T, the point can be assigned to bin A. If a point is collected between 1 / 8 and 3 / 8 of T, the point can be assigned to bin B. If a point is collected between 3 / 8 and 5 / 8 of T, the point can be assigned to bin C. If a point is collected between 5 / 8 and 7 / 8 of T, the point can be assigned to bin D. If a point is collected between 7 / 8 and T, the point can be assigned to bin A. Other embodiments of operation 704 may assign the collected points to bins differently. For example, measurement points may be assigned to one of more than four bins. These bins may be associated with unequal portions of the period T. The cutoff of the box can occur at the peak and trough of the 800 signal.

[0078] like Figure 8AAs shown, the periodic physiological motion signal 800 can be used to divide the measurement points into bins based on the amplitude of the signal 800. Figure 8B The illustration shows amplitude cutoffs for classifying measurement points into three bins: bin E, bin F, and bin G. For example, points collected when the amplitude of the periodic physiological motion signal 800 is greater than half of its peak value can be assigned to bin E. Points collected when the amplitude of the periodic physiological motion signal 800 is less than half of its minimum value can be assigned to bin G. Points collected between these amplitude values ​​can be assigned to bin F. Other embodiments may include more or fewer bins.

[0079] Now for reference Figure 9 An exemplary illustration of operation 704 of method 700 is shown therein. Figure 9 Depicting Figures 3A to 3B and Figures 4A to 4D The conduit 310. A shape sensor (i.e., shape sensor 314) disposed within the conduit 310 measures points during four discrete time segments (times t1, t2, t3, and t4) within a period T. Time t1, t2, t3, and t4 correspond to different boxes, such as... Figure 9 Boxes A, B, C, and D are shown. (As...) Figure 9 As shown, the shape and position of catheter 310 at time t4 differ from those at times t1, t2, and t3. This difference is not due to catheter 310 being inserted deeper into anatomical pathway 402, but is likely due to the periodic physiological movements of the lungs 400 during respiration in patient P.

[0080] Now for reference Figure 10A , Figure 10B , Figure 10C and Figure 10D This illustrates another embodiment of operation 704. When assigning each measurement point to one of a plurality of boxes, the assignment can be based on the insertion depth of the conduit used for the measurement point. Therefore, a depth value based on the movement of the bracket 306 relative to the stage 308 can be received from an encoder or another sensor. Figure 6B The segmented centerline model 602 preoperatively determines the cutoff between depths. Other depths may be used in other embodiments. For example... Figure 10A As shown, the points collected by catheter 310 at a specific time can be classified or assigned to depth boxes A and B. For example... Figure 10B As shown, when collecting measurement points, the conduit 310 had been inserted more extensively, thus assigning the points to three depth chambers: depth chambers A, B, and C. Figure 10C and Figure 10D As shown, when catheter 310 is inserted deeper into the anatomical pathway of lung 400, measurement points are assigned to depth boxes A, B, C, and D. Figures 10A to 10DAs shown, a deeper depth chamber can include more measurement points because a portion of the conduit 310 has been at proximal depth for a longer period, during which points can be collected periodically. In some embodiments, the number of points in the chamber can be capped such that when a new point is measured, the oldest point is removed from a subset or pool of points associated with that chamber.

[0081] Return again Figure 7 In accordance with method 700, at operation 705, the control system 112 may receive a set of model points from the model. For example, this set of model points may be a set of points representing a model of lung 400. The set of model points received at operation 705 may be similar to... Figure 6C Model points 604 are a set of points derived from centerline model 602, which is based on model 600. This set of model points 604 can be used to register model 600 to this set of measurement points collected using conduit 310.

[0082] Following operation 704, an initial seed transformation can be performed at operation 706 to coarsely position the measurement points relative to the model points, allowing subsequent iterative registration operations to be performed. The registration process can provide a seed using known information about the displacement and orientation relationships between the patient's surgical environment and the anatomical model. For example, landmarks associated with the major carina in lung 400 can be identified in the anatomical model information. The corresponding orientations can be included in the measurement points as part of the workflow. At operation 707, the control system 112 can register each subset of measurement points to that set of model points. Registration can be accomplished using a point set registration algorithm (such as the Iterative Closest Point (ICP) technique described in operations 708 through 714) or by implementing another registration algorithm.

[0083] At operation 708, after the initial coarse seed transformation has been performed to initiate the registration process, a set of measurement data points D collected from the patient P is matched with anatomical model points 604. For example, each measurement data point D can be matched with the nearest point in that set of anatomical model points 604. In this embodiment, the anatomical model points 604 are along a centerline generated from the three-dimensional anatomical model (e.g., from...). Figure 6A and Figure 6B The segmented model 600 generates a set of points for the segmented centerline model 602. A registration algorithm identifies the nearest points in the measurement data points and the set of anatomical model points 604. The results can be... Figure 11A As seen in the image, it shows model points 604 with a set of measurement points 1100 after the initial seeding or registration process has begun. (See image for details.) Figure 11AAs shown, the set of measurement points 1100 includes four subsets of measurement points: subset 1102A (represented by a solid circle), subset 1102B (represented by a plus sign), subset 1102C (represented by a hollow circle), and subset 1102D (represented by an x). Subsets 1102A, 1102B, 1102C, and 1102D can be collectively referred to as subset 1102.

[0084] In various alternatives, matching between model point 604 and each subset 1102 can be accomplished using brute-force techniques, KD-tree techniques, etc. Some matches can be discarded based on maximum distance threshold calculations, maximum angle threshold calculations, or other metrics used to filter out matches not deemed sufficiently reliable for inclusion in the model. Anatomical model point 604 can be represented by any of several different types of points (including centerline points, grid points, and / or volume points).

[0085] Refer again Figure 7 In method 700, at operation 710, the transformations required to map each subset 1102 to the position and orientation of the matching anatomical model point 604 are determined. More specifically, a total calculated offset for position and orientation is determined for each subset 1102. In some embodiments, the calculated correction transformations may be limited such that only a certain number of rotation degrees or a certain number of millimeter displacements can be applied to a single iteration of the process. For example, even if a 20° rotation or reorientation of the anatomical model point 604 is calculated, the medical system may limit the change in orientation to 10°, 5°, or less. Similarly, in some embodiments, even if a 40 mm displacement is calculated, the control system 112 may limit the displacement available in a single iteration to 20 mm, 10 mm, 5 mm, or less. In some embodiments, the limits may vary depending on the number of iterations performed, allowing less movement in later iterations than in earlier iterations.

[0086] At operation 712, each subset 1102 can be transformed using a rigid or non-rigid transformation that applies a calculated offset in displacement and orientation to specifically move each point in a particular subset 1102. In an alternative embodiment, modeling data points can be transformed using a rigid or non-rigid transformation that applies a calculated offset in displacement and orientation to move each point in that set of model points 604 toward subset 1102. Thus, some embodiments of this disclosure may involve registering measurement points to model points and moving the measurement points (including translating measurement points and / or changing their orientation) to better align them with the model points, thereby bringing them into a common space or common frame of reference. These registrations can be understood as registration candidates. To register model 600 with measurement points 1100 and bring model 600 into patient space, optimal registration candidates can be identified and selected.

[0087] At operation 714, the registration error between each subset 1102 and the matched anatomical model point 604 is evaluated. In some embodiments, the error can be calculated as the cumulative error or distance between each measurement point and its nearest transformed model point. Alternatively, it can be expressed as a change in orientation and displacement with respect to previously calculated registrations. This can include the calculation of error values, including the error values ​​for orientation and displacement for each subset 1102. In other words, error factors for orientation and displacement can be determined individually for each matched subset 1102. If the aggregated error factors are greater than a threshold, additional iterations of operations 708 to 714 can be repeated until the total position and orientation error factors fall below the threshold.

[0088] At operation 716, the convergence of each subset 1102 can be compared to determine which registration candidate is most suitable for the subset. Figure 6A The model 600 is optimally used in the procedure. In other words, the total error values ​​associated with registration candidates to model point 604 for each subset 1102A, 1102B, 1102C, and 1102D can be compared. The registration candidate with the smallest total error value can be used to bring model point 604 into the patient space of collection point 1100. The association between model points 604 allows model 600 to be represented in the patient space. In some embodiments, after determining the convergence of each subset, the convergence value can be compared with a threshold, and if the convergence drops below the threshold or if the associated error exceeds the threshold, the collection point up to that point in the failed subset(s) can be discarded. For example, when patient P makes a coarse movement, the registration errors associated with one or more subsets may include significant errors, making the registration unusable for image-guided surgery. Because the subset has been "contaminated" with bad data associated with patient P's movement, the subset can be cleared and point collection can be restarted.

[0089] For example, at operation 718, it can be done by... Figure 1 The display 110 provides a graphical user interface that displays a visual representation of the best registration of model point 604 (and thus model 600). The visual representation can depict the medical device system 200 associated with model 600. Figure 13A An exemplary visual representation is shown in the figure. Figure 13A The illustration shows the display 1300, which is based on Figure 6A The anatomical model 600 displays a rendering of the anatomical pathways of the human lung in the user interface. This is achieved using the method described above. Figure 7The patient-registered model space described herein, the current shape of catheter 310, and the orientation of distal end 318 can be simultaneously located and displayed with the rendering of access 601, which includes accesses 601A and 601B. Data points can be visually represented in the user interface as multiple dots or circles on the display, or as a rendered model (such as a mesh or line model created based on the set of data points). In some embodiments, the visual representation can be refreshed on display 110 after each processing operation to change the data points has been implemented. Figure 13B Another exemplary display 1350 of visual representation is illustrated, which can be provided to the surgeon S to help guide the operation. Figure 13B An internal view of a portion of model 600 is shown from the perspective of medical device 200, and rendered models of access routes 601A and 601B are depicted. Model 600 can be rendered in two or three dimensions to facilitate surgical guidance.

[0090] The principles and embodiments of this disclosure can improve image-guided surgery by improving the registration between a model of the patient's anatomy and a model of a medical device manipulated within the patient's anatomy during a surgical procedure. Multiple candidate registrations can be generated based on a subset of measurement data points, and these registrations can be compared to determine the optimal registration to be applied to the model to bring it into patient space. These multiple candidate registrations can be continuously updated and compared to each other and with a threshold to ensure the highest fidelity registration at any given time, as provided for use in image-guided surgery.

[0091] In some embodiments, when a later registration replaces an earlier registration, or when the control system 112 determines that an earlier registration can be replaced by a later registration, an alert can be provided to the clinician via a user interface to indicate a registration change (e.g., from one subset of measurement points to another) or that a better registration is available. In some embodiments, the control system 112 may require clinician approval via a user interface before implementing the better registration. For example, when a better registration is identified, an alert and a button or other user interface element indicating whether the clinician can approve or disapprove the new registration may be presented to the display 110. The new registration is then implemented or not implemented based on the clinician's decision.

[0092] One or more elements in embodiments of the present invention can be implemented in software to execute on a processor of a computer system (e.g., control system 112). When implemented in software, the elements in embodiments of the present invention are essentially code segments that perform necessary tasks. Programs or code segments can be stored in a non-transitory processor-readable storage medium or device, including any medium capable of storing information, including optical, semiconductor, and magnetic media. Examples of processor-readable storage devices include: electronic circuits, semiconductor devices, semiconductor memory devices, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. Code segments can be downloaded via computer networks such as the Internet or intranets. As described herein, operations such as accessing, collecting, allocating, detecting, initiating, registering, displaying, receiving, generating, determining, moving data points, segmenting, matching, etc., can be performed at least in part by control system 112 or its processor.

[0093] It should be noted that the presented processes and displays may not inherently relate to any particular computer or other device. The necessary structures for various such systems will appear as elements in the claims. Furthermore, embodiments of the invention are described without reference to any particular programming language. It should be understood that the teachings of the invention as described herein can be implemented using various programming languages.

[0094] While certain exemplary embodiments of the invention have been described and illustrated in the accompanying drawings, it should be understood that these embodiments are merely illustrative of the invention and not limiting, and that embodiments of the invention are not limited to the specific constructions and arrangements shown and described, as various other modifications will be apparent to those skilled in the art.

Claims

1. A non-transitory computer-readable medium storing a plurality of instructions, said plurality of instructions causing said one or more processors, when executed by said one or more processors, to: When a flexible catheter is inserted into one or more anatomical pathways in a patient, a set of measurement points are collected along the flexible catheter, the measurement points being based on the shape of the flexible catheter in the patient space; Based on the insertion depth of a portion of the flexible catheter associated with each measurement point, each measurement point in the set of measurement points is assigned to a corresponding subset of a plurality of subsets of measurement points; Multiple registration candidates are generated, each of the multiple registration candidates being based on a corresponding subset of the multiple subsets; The multiple registration candidates are compared to identify a first best subset among the multiple subsets; The model is registered to the patient space based on a set of model points and the first optimal subset; Following the registration, additional measurement points are collected along the flexible catheter; The plurality of subsets are updated by assigning each additional measurement point to a corresponding subset of the plurality of subsets based on the insertion depth of the portion of the flexible catheter associated with each additional measurement point; Following the update, a plurality of second registration candidates are generated, each of the plurality of second registration candidates being based on a corresponding subset of the plurality of subsets; Following the update, the plurality of second registration candidates are compared to identify a second best subset within the plurality of subsets; and The model is registered to the patient space based on the set of model points and the second optimal subset.

2. The non-transitory computer-readable medium of claim 1, wherein the number of subsets of the plurality of subsets of measurement points is based on the insertion depth of the flexible catheter into the one or more anatomical pathways, such that the number of subsets increases as the flexible catheter is advanced distally.

3. The non-transitory computer-readable medium of claim 2, wherein the insertion depth is determined by measuring the movement of the instrument holder to which the flexible catheter is attached relative to the insertion stage.

4. The non-transitory computer-readable medium of claim 1, further storing instructions that, when executed by the one or more processors, cause the one or more processors to: A digital weight is assigned to each of the set of measurement points, wherein the digital weight of a corresponding point reflects the probability that the corresponding point is correctly assigned to its corresponding subset, and wherein the digital weight is used during the comparison of the plurality of registration candidates to identify the first optimal subset.

5. The non-transitory computer-readable medium of claim 1, further storing instructions that, when executed by the one or more processors, cause the one or more processors to: The plurality of subsets are updated by removing at least a portion of the measurement points from at least one subset of the plurality of subsets.

6. The non-transitory computer-readable medium of claim 1, wherein comparing the plurality of registration candidates to identify the first best subset of the plurality of subsets comprises: Determine the convergence value of each subset of the plurality of subsets relative to the set of model points.

7. The non-transitory computer-readable medium of claim 6, further storing instructions that, when executed by the one or more processors, cause the one or more processors to: After determining the convergence value for each subset, all measurement points are removed from at least one subset of the plurality of subsets based on comparing the convergence value associated with at least one subset with a threshold.

8. The non-transitory computer-readable medium of claim 1, further storing instructions that, when executed by the one or more processors, cause the one or more processors to: The at least one subset of the plurality of subsets is updated by discarding the oldest measurement point from the at least one subset based on the new measurement point assigned to the at least one subset.

9. The non-transitory computer-readable medium of claim 1, wherein the first best subset and the second best subset are the same subset of the plurality of subsets.

10. The non-transitory computer-readable medium of claim 1, wherein the first best subset and the second best subset are different subsets of the plurality of subsets.

11. The non-transitory computer-readable medium of claim 10, further storing instructions that, when executed by the one or more processors, cause the one or more processors to: After identifying the second best subset among the plurality of subsets, an alert is generated at the user interface indicating that a new best subset can be used for registration.

12. The non-transitory computer-readable medium of claim 1, wherein registering the model to the patient space based on the set of model points and the first optimal subset comprises converting the set of model points from the model space to the patient space, or converting measurement points in the first optimal subset from the patient space to the model space.

13. A medical device system, comprising: Flexible catheter; A point collection device extends along the length of the flexible conduit and is configured to collect a set of measurement points; and A tracking system configured to continuously register models of one or more anatomical pathways of a patient into patient space, wherein the tracking system is configured to perform operations including: The set of measurement points are collected when the flexible catheter is inserted into one or more anatomical pathways, the measurement points being based on the shape of the flexible catheter in the patient space; Based on the insertion depth of a portion of the flexible catheter associated with each measurement point, each measurement point in the set of measurement points is assigned to a corresponding subset of a plurality of subsets of measurement points; Multiple registration candidates are generated, each of the multiple registration candidates being based on a corresponding subset of the multiple subsets; The multiple registration candidates are compared to identify a first best subset among the multiple subsets; The model is registered to the patient space based on a set of model points and the first optimal subset; Following the registration, additional measurement points are collected along the flexible catheter; The plurality of subsets are updated by assigning each additional measurement point to a corresponding subset of the plurality of subsets based on the insertion depth of the portion of the flexible catheter associated with each additional measurement point; Following the update, a plurality of second registration candidates are generated, each of the plurality of second registration candidates being based on a corresponding subset of the plurality of subsets; Following the update, the plurality of second registration candidates are compared to identify a second best subset within the plurality of subsets; and The model is registered to the patient space based on the set of model points and the second optimal subset.

14. The medical device system of claim 13, further comprising: The instrument holder to which the flexible catheter is attached; An insertion stage, wherein the instrument holder is movable on the insertion stage; and A position measuring device is configured to measure the position of the instrument holder relative to the insertion stage, wherein the number of subsets of the plurality of subsets of the measuring points is based on the insertion depth of the flexible catheter into the one or more anatomical pathways, such that the number of subsets increases as the flexible catheter is advanced distally, wherein the insertion depth is based on the position of the instrument holder measured by the position measuring device.

15. The medical device system of claim 13, wherein the tracking system is further configured to perform: A digital weight is assigned to each of the set of measurement points, wherein the digital weight of a corresponding point reflects the probability that the corresponding point is correctly assigned to its corresponding subset, and wherein the digital weight is used during the comparison of the plurality of registration candidates to identify the first optimal subset.

16. The medical device system of claim 13, wherein the tracking system is further configured to perform: The plurality of subsets are updated by removing at least a portion of the measurement points from at least one subset of the plurality of subsets.

17. The medical device system of claim 13, wherein the tracking system is configured to perform a comparison of the plurality of subsets to identify a first optimal subset of the plurality of subsets by: Determine the convergence value of each subset of the plurality of subsets relative to the set of model points.

18. The medical device system of claim 17, wherein the tracking system is further configured to perform: After determining the convergence value for each subset, all measurement points are removed from at least one subset of the plurality of subsets based on comparing the convergence value associated with at least one subset with a threshold.

Images