Systems and methods related to registration for image-guided surgery

Abstract

A system is configured to perform operations including accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space. Collecting a set of measured points of the anatomical structure of the patient, the measured points being associated with a patient space. Registering the set of model points to the set of measured points using a first initial parameter set to generate a first transformation. Generating one or more perturbed initial parameter sets based on the first initial parameter set. Performing one or more perturbation registration processes to register the set of model points to the set of measured points using the one or more perturbation initial parameter sets, respectively, and generating corresponding perturbation transformations accordingly. Generating a registration quality indicator based on the first transformation and the one or more perturbation transformations.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 670,530, filed May 11, 2018, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to systems and methods for performing image-guided procedures, and more particularly to systems and methods for using registered real-time images and prior anatomical images during image-guided procedures. Background Art

[0004] Minimally invasive medical technology aims to reduce the amount of tissue damaged in medical procedures, thereby reducing the patient's recovery time, discomfort and harmful side effects. Such minimally invasive techniques can be performed through natural orifices in the patient's anatomical structure or through one or more surgical incisions. Through these natural orifices or incisions, the operator can insert a minimally invasive medical device (including a surgical instrument, a diagnostic instrument, a therapeutic instrument or a biopsy instrument) to reach the target tissue position. In order to assist in reaching the target tissue position, the position and movement of the medical device can be associated with the preoperative image or intraoperative image of the patient's anatomical structure. Using an image-related image-guided instrument, the instrument can navigate in a natural or surgically created channel in an anatomical system (such as the lungs, colon, intestines, kidneys, heart, circulatory system, etc.). However, operators usually do not have enough understanding of this related quality (e.g., accuracy, completeness, effectiveness, consistency), which can lead to uncertainty in image-guided procedures.

[0005] Therefore, it would be advantageous to provide improved registration for performing image-guided procedures. Summary of the invention

[0006] Embodiments of the invention are best summarized by the claims that follow the description.

[0007] In one illustrative embodiment, a method performed by a computing system includes: accessing a model point set / set of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measured points of the anatomical structure of the patient, the measured points being associated with a patient space; and registering the model point set with the measured point set using a first initial parameter set to generate a first transformation. The method further includes generating one or more perturbed initial parameter sets based on the first initial parameter set, performing one or more perturbation registration processes to register the model point set to the measured point set using the one or more perturbation initial parameter sets, respectively, to generate one or more perturbation transformations; and generating a registration quality indicator based on the first transformation and the one or more perturbation transformations.

[0008] In another illustrative embodiment, a method performed by a computing system includes: accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measurement points of the model of the anatomical structure of the patient, the measurement points being associated with the patient space; determining a region of interest in the model space; and determining a comparison region in the model space. The method further includes: registering a first set of model points in the region of interest to the set of measurement points to generate a first number of region of interest transformations; registering a second set of the model points in the comparison region to the set of measurement points to generate a first number of comparison region transformations corresponding to the first number of region of interest transformations, respectively; and generating a registration quality indicator for the region of interest based on the first number of region of interest transformations and the first number of comparison region transformations.

[0009] In another illustrative embodiment, a method performed by a computing system includes: accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measured points of the model of the anatomical structure of the patient, the measured points being associated with the patient space; and registering the set of model points with the set of measured points to generate a first transformation. The method further includes: performing a local registration quality analysis for a region of interest to generate a local registration quality indicator; generating a modified model space based on the registration quality indicator; and registering a subset of the model points associated with the modified model space to the set of measured points to generate a second transformation.

[0010] In another illustrative embodiment, a non-transitory machine-readable medium comprising a plurality of machine-readable instructions, which when executed by one or more processors are adapted to cause the one or more processors to perform a method. The method includes: accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measured points of the anatomical structure of the patient, the measured points being associated with a patient space; and registering the set of model points to the set of measured points using a first initial parameter set to generate a first transformation. The method further includes: generating one or more perturbed initial parameter sets based on the first initial parameter set; performing one or more perturbation registration processes to register the set of model points to the set of measured points using the one or more perturbation initial parameter sets, respectively, to generate one or more perturbation transformations; and generating a registration quality indicator based on the first transformation and the one or more perturbation transformations.

[0011] In another illustrative embodiment, a non-transitory machine-readable medium including a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted to cause the one or more processors to perform a method when executed by one or more processors. The method includes: accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with the patient space; determining a region of interest in the model space; and determining a comparison region in the model space. The method further includes: registering a first set of model points in the region of interest to the set of measurement points to generate a first number of region of interest transformations; registering a second set of model points in the comparison region to the set of measurement points to generate a first number of comparison region transformations corresponding to the first number of region of interest transformations, respectively; and generating a registration quality indicator for the region of interest based on the first number of region of interest transformations and the first number of comparison region transformations.

[0012] In another illustrative embodiment, a non-transitory machine-readable medium including a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted to cause the one or more processors to perform a method when executed by one or more processors. The method includes: accessing a set of model points of a model of an anatomical structure of a patient, the model points being associated with a model space; collecting a set of measured points of the anatomical structure of the patient, the measured points being associated with the patient space; registering the set of model points to the set of measured points to generate a first transformation. The method further includes: performing a local registration quality analysis for a region of interest to generate a local registration quality indicator; generating a modified model space based on the registration quality indicator; and registering a subset of model points associated with the modified model space to the set of measured points to generate a second transformation.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory in nature, and are intended to provide an understanding of the present disclosure rather than to limit the scope of the present disclosure. In this regard, other aspects, features and advantages of the present disclosure will be apparent to those skilled in the art based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a simplified diagram of a teleoperated medical system according to some embodiments.

[0015] Figure 2A is a simplified diagram of a medical device system according to some embodiments.

[0016] Figure 2B is a simplified diagram of a medical device with an expanded medical tool according to some embodiments.

[0017] Figure 3A and Figure 3Bis a simplified diagram of a side view of a patient coordinate space including a medical device mounted on an insertion assembly according to some embodiments.

[0018] Figure 4A , Figure 4B , Figure 4C and Figure 4D FIG. 2 shows a schematic diagram of a human lung during insertion according to some embodiments. Figure 3A , Figure 3B The remote end of the medical device system.

[0019] Figure 5 is a flow chart illustrating a method of an image-guided surgery procedure or portion thereof according to some embodiments.

[0020] Fig. 6A , Figure 6B and Figure 6C Shown are steps in a segmentation process for generating a human lung model for registration, according to some embodiments.

[0021] Figure 7 is a flow chart providing a method for performing global registration quality analysis according to some embodiments.

[0022] Figure 8 is a flow chart providing a method for performing local registration quality analysis according to some embodiments.

[0023] Fig. 9 is a flow chart providing a method for performing re-registration based on registration quality analysis according to some embodiments.

[0024] Fig.10 Shown are display stages of a re-registration technique in accordance with some embodiments.

[0025] Fig.11 Shown are display stages of a re-registration technique in accordance with some embodiments.

[0026] By referring to the following detailed description, the embodiments of the present disclosure and its advantages will be best understood.It should be understood that the same reference numerals are used to identify similar elements illustrated in one or more of the accompanying drawings, wherein the present disclosure is for illustrating embodiments of the present disclosure rather than limiting the embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] In the following description, the specific details of some embodiments consistent with the present disclosure are set forth. Many specific details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative rather than restrictive. Those skilled in the art may realize other elements within the scope and spirit of the present disclosure although not specifically described herein. In addition, to avoid unnecessary repetition, unless otherwise specifically stated or if one or more features render an embodiment inoperative, one or more features shown and described in association with an embodiment may be incorporated into other embodiments.

[0028] In some instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0029] The present disclosure describes various apparatuses and parts of apparatuses according to their states in three-dimensional space. As used herein, the term "positioning" refers to the position of an object or a part of an object in three-dimensional space (e.g., along the three translational degrees of freedom of Cartesian x-coordinates, y-coordinates, and z-coordinates). As used herein, the term "orientation" refers to the rotational placement of an object or a part of an object (three rotational degrees of freedom, e.g., roll, pitch, and yaw). As used herein, the term "attitude" refers to the positioning of an object or a part of an object in at least one translational degree of freedom and the orientation of the object or a part of an object in at least one rotational degree of freedom (up to six degrees of freedom). As used herein, the term "shape" refers to the set of attitude, positioning, or orientation measured along an object.

[0030] Figure 1 is a simplified diagram of a teleoperated medical system 100 according to some embodiments. In some embodiments, the teleoperated medical system 100 may be applicable to, for example, surgical procedures, diagnostic procedures, therapeutic procedures, or biopsy procedures. Although some embodiments are provided herein for such procedures, any reference to medical devices or surgical instruments and medical methods or surgical methods is non-limiting. The systems, devices, and methods described herein may be used for animals, human cadavers, animal cadavers, portions of human or animal anatomy, non-surgical diagnostics, and may be used in industrial systems and general purpose robotic or teleoperated systems.

[0031] like Figure 1As shown, medical system 100 generally includes a manipulator assembly 102 for operating medical instruments 104 in performing various procedures on patient P. Manipulator assembly 102 can be a teleoperated assembly with selected degrees of freedom of movement that are powered and / or teleoperated and selected degrees of freedom of movement that are not powered and / or teleoperated, a non-teleoperated assembly, or a hybrid teleoperated and non-teleoperated assembly. Manipulator assembly 102 is mounted on or near operating table T. Main assembly 106 allows an operator (e.g., Figure 1 The surgeon, clinician or physician (see Figure 1) observes the surgical site and controls the manipulator assembly 102.

[0032] The main assembly 106 can be located at an operator console, which is typically located in the same room as the operating table T, for example, on the side of the operating table where the patient P is located. However, it should be understood that the operator O can be in a different room or a completely different building from the patient P. The main assembly 106 typically includes one or more control devices for controlling the manipulator assembly 102. The control device can include any number of various input devices, such as a joystick, trackball, data glove, trigger gun, manual controller, voice recognition device, body movement or presence sensor and / or similar device. In order to give the operator O a strong sense of directly controlling the instrument 104, the control device can be provided with the same degrees of freedom as the associated medical device 104. In this way, the control device provides the operator O with a remote presentation or a perception that the control device is integrated with the medical device 104.

[0033] In some embodiments, the control device may have more or fewer degrees of freedom than the associated medical instrument 104 and still provide telepresence to the operator O. In some embodiments, the control device may optionally be a manual input device that moves with six degrees of freedom and may also include an actuatable handle for actuating the instrument (e.g., for closing grasping jaws, applying electrical potentials to electrodes, delivering medication, and / or the like).

[0034] The manipulator assembly 102 supports the medical device 104 and may include one or more non-servo controlled links (e.g., one or more links that can be manually positioned and locked in place, often referred to as mounting structures) and / or one or more servo controlled links (e.g., one or more links that can be controlled in response to commands from a control system) and the kinematic structure of the manipulator. The manipulator assembly 102 may optionally include a plurality of actuators or motors that drive inputs on the medical device 104 in response to commands from a control system (e.g., control system 112). The actuator may optionally include a drive system that, when coupled to the medical device 104, can advance the medical device 104 into a natural or surgically created anatomical orifice. Other drive systems can move the distal end of the medical device 104 with multiple degrees of freedom, which may include three linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) degrees of freedom and three rotational motion (e.g., rotation around the X, Y, Z Cartesian axes) degrees of freedom. Additionally, the actuator may be used to actuate an articulated end effector of the medical device 104 to grasp tissue in the jaws of a biopsy device and / or similar device. Actuator positioning sensors such as resolvers, encoders, potentiometers, and other mechanisms may provide sensor data describing the rotation and orientation of the motor shaft to the medical system 100. This positioning sensor data may be used to determine the motion of an object manipulated by the actuator.

[0035] The teleoperated medical system 100 may include a sensor system 108 having one or more subsystems for receiving information about the instrument of the manipulator assembly 102. Such subsystems may include a position / location sensor system (e.g., an electromagnetic (EM) sensor system); a shape sensor system for determining the position, orientation, speed, velocity, posture, and / or shape of the distal end and / or along one or more segments of the flexible body that may comprise the medical instrument 104; and / or a visualization system for acquiring images from the distal end of the medical instrument 104.

[0036] The teleoperated medical system 100 also includes a display system 110 for displaying images or representations of the surgical site and medical instrument 104 generated by the subsystems of the sensor system 108. The display system 110 and the main assembly 106 may be oriented so that the operator O can control the medical instrument 104 and the main assembly 106 with a telepresented perception.

[0037] In some embodiments, the medical device 104 can have a visualization system (discussed in more detail below), which can include a scope assembly that records concurrent or real-time images of the surgical site and provides the images to an operator or operator O through one or more displays of the medical system 100 (such as one or more displays of the display system 110). The concurrent images can be, for example, two-dimensional or three-dimensional images acquired by an endoscope positioned within the surgical site. In some embodiments, the visualization system includes an endoscope component that can be integrally or removably coupled to the medical device 104. However, in some embodiments, a separate endoscope attached to a separate manipulator assembly can be used with the medical device 104 to image the surgical site. The visualization system can be implemented as hardware, firmware, software, or a combination thereof that interacts with or is executed by one or more computer processors, which can include a processor of the control system 112.

[0038] The display system 110 can also display images of the surgical site and medical instruments captured by the visualization system. In some examples, the teleoperation medical system 100 can configure the controls of the medical instruments 104 and the main assembly 106 so that the relative positioning of the medical instruments is similar to the relative positioning of the eyes and hands of the operator O. In this way, the operator O can manipulate the medical instruments 104 and hand controls as if viewing a substantially real workspace. Reality refers to the presentation of the image as a real perspective image that simulates the viewpoint of a physician physically manipulating the medical instrument 104.

[0039] In some examples, display system 110 can present images of a surgical site recorded preoperatively or intraoperatively using image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, infrared imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube x-ray imaging, and / or similar imaging techniques. The preoperative or intraoperative image data can be presented as a two-dimensional image, a three-dimensional image, or a four-dimensional (including, for example, time-based or velocity-based information) image and / or as an image of a model created from a preoperative or intraoperative image data set.

[0040] In some embodiments, the display system 110 may display a virtual navigation image in which the actual position of the medical device 104 is registered (i.e., dynamically referenced) to a preoperative or concurrent image / model, often for the purpose of imaging-guided surgical procedures. Such registration may present a virtual image of the internal surgical site to the operator O from the viewpoint of the medical device 104. In some examples, the viewpoint may be from the tip of the medical device 104. An image of the tip of the medical device 104 and / or other graphical or alphanumeric indicators may be superimposed on the virtual image to assist the operator O in controlling the medical device 104. In some examples, the medical device 104 may not be visible in the virtual image.

[0041] In some embodiments, the display system 110 can display a virtual navigation image in which the actual position of the medical device 104 is registered with the preoperative image or concurrent image to provide the operator O with a virtual image of the medical device 104 within the surgical site from an external viewpoint. An image of a portion of the medical device 104 or other graphical or alphanumeric indicator can be superimposed on the virtual image to assist the operator O in controlling the medical device 104. As described herein, a visual representation of the data point can be rendered to the display system 110. For example, the measurement data points, movement data points, registration data points, and other data points described herein can be displayed on the display system 110 in a visual representation. The data points can be visually represented in the user interface by multiple points or dots on the display system 110, or as a rendered model (such as a grid or line model created based on a set of data points). In some examples, the data points can be color-coded according to the data represented by the data points. In some embodiments, the visual representation can be refreshed in the display system 110 after each processing operation has been implemented to change the data point.

[0042] The remotely operated medical system 100 may also include a control system 112. The control system 112 includes at least one memory and at least one computer processor (not shown) for controlling between the medical device 104, the main assembly 106, the sensor system 108, and the display system 110. The control system 112 also includes programmed instructions (e.g., a non-transitory machine-readable medium storing instructions) to implement some or all of the methods described according to the aspects disclosed herein, including instructions for providing information to the display system 110. Although in Figure 1The control system 112 is shown as a single box in the simplified schematic diagram of , however, the system may include two or more data processing circuits, wherein a portion of the processing is optionally performed on or adjacent to the manipulator assembly 102, and another portion of the processing is performed at the main assembly 106, etc. The processor of the control system 112 can execute instructions, which include instructions corresponding to the processes disclosed herein and described in more detail below. Any of a variety of centralized or distributed data processing architectures can be used. Similarly, the programmed instructions can be implemented as multiple separate programs or subroutines, or they can be integrated into many other aspects of the remote operation system described herein. In one embodiment, the control system 112 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry.

[0043] In some embodiments, the control system 112 may receive force and / or torque feedback from the medical device 104. In response to the feedback, the control system 112 may transmit a signal to the main assembly 106. In some examples, the control system 112 may transmit a signal instructing one or more actuators of the manipulator assembly 102 to move the medical device 104. The medical device 104 may extend to an internal surgical site in the patient P via an opening in the patient P. Any suitable conventional and / or dedicated actuator may be used. In some examples, one or more actuators may be separated or integrated with the manipulator assembly 102. In some embodiments, one or more actuators and the manipulator assembly 102 are provided as part of a remotely operated surgical vehicle positioned adjacent to the patient P and the operating table T.

[0044] The control system 112 may optionally further include a virtual visualization system to provide navigation assistance to the operator O when controlling the medical device 104 during the image-guided surgical procedure. Virtual navigation using the virtual visualization system can be based on reference to the acquired preoperative or intraoperative data set of the anatomical channel. The virtual visualization system processes images of the surgical site imaged using imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, infrared imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, etc. Software that can be used in conjunction with manual input can be used to transform the recorded image into a segmented two-dimensional or three-dimensional composite representation of a portion or entire anatomical organ or anatomical region. The image data set is associated with the composite representation. The composite representation and the image data set describe the various positions and shapes of the channel and its connectivity. During the clinical procedure, images used to generate the composite representation can be recorded preoperatively or intraoperatively. In some embodiments, the virtual visualization system can use a standard representation (i.e., non-patient-specific) or a mixture of a standard representation and patient-specific data. The composite representation and any virtual images generated from the composite representation may represent the static pose of the deformable anatomical region during one or more motions (eg, during an inhalation / exhalation cycle of the lungs).

[0045] During the virtual navigation procedure, the sensor system 108 can be used to calculate the approximate position of the medical device 104 relative to the anatomical structure of the patient P. This position can be used to generate both a macroscopic (external) tracking image of the anatomical structure of the patient P and a virtual internal image of the anatomical structure of the patient P. The system can implement one or more electromagnetic (EM) sensors, fiber optic sensors, and / or other sensors to register and display the medical implementation with the surgical images recorded before surgery (such as images from a virtual visualization system). For example, PCT Publication WO2016 / 191298 (published on December 1, 2016) (publishing "Systems and Methods of Registration for Image Guided Surgery") discloses such a system, which is incorporated herein by reference in its entirety. The remotely operated medical system 100 may further include optional operations and support systems (not shown) such as lighting systems, steering control systems, irrigation systems, and / or suction systems. In some embodiments, the remotely operated medical system 100 may include more than one manipulator assembly and / or more than one main assembly. The exact number of remotely operated manipulator assemblies will depend on the surgical procedure and the space limitations in the operating room and other factors. The master assemblies 106 may be collocated, or they may be located in separate locations.Multiple master assemblies allow more than one operator to control one or more teleoperated manipulator assemblies in various combinations.

[0046] Figure 2A 2 is a simplified diagram of a medical device system 200 according to some embodiments. In some embodiments, the medical device system 200 can be used as a medical device 104 in an image-guided medical procedure performed using a teleoperated system medical system 100. In some examples, the medical device system 200 can be used in a non-teleoperated exploratory procedure or in a procedure involving a traditional manually operated medical device (such as an endoscope). Optionally, the medical device system 200 can be used to collect (i.e., measure) a set of data points corresponding to locations within an anatomical passage of a patient such as patient P.

[0047] The medical device system 200 includes an elongated device 202 (such as a flexible catheter) coupled to a drive unit 204. The elongated device 202 includes a flexible body 216 having a proximal end 217 and a distal end or tip portion 218. In some embodiments, the flexible body 216 has an outer diameter of approximately 3 mm. Other flexible bodies may have an outer diameter that is larger or smaller.

[0048] The medical device system 200 further includes a tracking system 230, as described in further detail below, for determining the position, orientation, speed, velocity, posture, and / or shape of the distal end 218 and / or one or more segments 224 along the flexible body 216 using one or more sensors and / or imaging devices. The entire length of the flexible body 216 between the distal end 218 and the proximal end 217 can be effectively divided into the segments 224. The tracking system 230 can optionally be implemented as hardware, firmware, software, or a combination thereof interacting with or executed by one or more computer processors, which can include Figure 1 A processor of the control system 112 in the system.

[0049] Tracking system 230 may optionally track distal end 218 and / or one or more of segments 224 using shape sensor 222. Shape sensor 222 may optionally include an optical fiber aligned with flexible body 216 (e.g., provided within an internal channel (not shown) or mounted externally). In one embodiment, the diameter of the optical fiber is approximately 200 μm. In other embodiments, the size may be larger or smaller. The optical fiber of shape sensor 222 forms an optical fiber bend sensor for determining the shape of flexible body 216. In an alternative, an optical fiber including a fiber Bragg grating (FBG) is used to provide strain measurement in one or more dimensional structures. Various systems and methods for monitoring the shape and relative position of an optical fiber in three dimensions are described in U.S. Patent Application No. 11 / 180,389 (filed July 13, 2005) (disclosing "Fiber optic position and shape sensing device and method relating thereto"); U.S. Patent Application No. 12 / 047,056 (filed July 16, 2004) (disclosing "Fiber-optic shape and relative positions sensing"); and U.S. Patent No. 6,389,187 (filed June 17, 1998) (disclosing "Optical Fiber Bend Sensor"), which are incorporated herein by reference in their entireties. In some embodiments, the sensor may employ other suitable strain sensing techniques such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. In some embodiments, the shape of the elongated device may be determined using other techniques. For example, a history of the posture of the distal end of the flexible body 216 may be used to reconstruct the shape of the flexible body 216 over time intervals. In some embodiments, the tracking system 230 may alternatively and / or additionally track the distal end 218 using the positioning sensor system 220. The positioning sensor system 220 can be a component of an EM sensor system, wherein the positioning sensor system 220 includes one or more conductive coils that can be subjected to an externally generated electromagnetic field. Each coil of the EM sensor system then generates an induced electrical signal, the characteristics of which depend on the positioning and orientation of the coil relative to the externally generated electromagnetic field. In some embodiments, the positioning sensor system 220 can be configured and positioned to measure six degrees of freedom (e.g., three positioning coordinates X, Y, Z and three orientation angles of pitch, yaw and roll indicating a base point) or five degrees of freedom (e.g., three positioning coordinates X, Y, Z and two orientation angles of pitch and yaw indicating a base point).Further description of positioning sensor systems is provided in U.S. Patent No. 6,380,732 (filed August 11, 1999) (disclosing "Six-Degree of Freedom Tracking System Having a Passive Transponder on the Object Being Tracked"), which is incorporated herein by reference in its entirety.

[0050] In some embodiments, tracking system 230 may alternatively and / or additionally rely on historical posture, position, or orientation data stored for known points of the instrument system along a cycle of alternating motion, such as breathing. This stored data may be used to develop shape information about flexible body 216. In some examples, a series of positioning sensors (not shown), such as electromagnetic (EM) sensors similar to those in positioning sensors 220, may be positioned along flexible body 216 and then used for shape sensing. In some examples, a history of data from one or more of these sensors acquired during a procedure may be used to represent the shape of elongated device 202, particularly if the anatomical passage is generally static.

[0051] Flexible body 216 includes a channel 221 that is sized and shaped to receive a medical device 226 . Figure 2B2 is a simplified diagram of a flexible body 216 with an extended medical device 226 according to some embodiments. In some embodiments, the medical device 226 can be used for procedures such as surgery, biopsy, ablation, illumination, irrigation, or suction. The medical device 226 can be deployed through the channel 221 of the flexible body 216, and the medical device 226 can be used at a target location within the anatomical structure. The medical device 226 may include, for example, an image acquisition probe, a biopsy instrument, a laser ablation fiber, and / or other surgical tools, diagnostic tools, or therapeutic tools. The medical tool may include an end effector with a single working member, such as a scalpel, a blunt knife, an optical fiber, an electrode, etc. Other end effectors may include, for example, forceps, graspers, scissors, clip appliers, etc. Other end effectors may further include electrically activated end effectors such as electrosurgical electrodes, transducers, sensors, etc. In various embodiments, the medical device 226 is a biopsy instrument, which can be used to remove sample tissue or cell samples from a target anatomical location. The medical device 226 may also be used with an image acquisition probe within the flexible body 216. In various embodiments, the medical device 226 can be an image acquisition probe that includes a distal portion with a stereoscopic or monoscopic camera at or near the distal end 218 of the flexible body 216 for acquiring images (including video images) processed by the visualization system 231 for display and / or providing to the tracking system 230 to support tracking of the distal end 218 and / or one or more of the segments 224. The image acquisition probe can include a cable coupled to the camera for transmitting the acquired image data. In some examples, the image acquisition instrument can be a fiber optic bundle (such as a fiberscope) coupled to the visualization system 231. The image acquisition instrument can be single-spectrum or multi-spectrum, for example, acquiring image data in one or more of the visible spectrum, infrared spectrum, and / or ultraviolet spectrum. Alternatively, the medical device 226 itself can be an image acquisition probe. The medical device 226 can be advanced from the opening of the channel 221 to perform a procedure and then retracted into the channel when the procedure is completed. The medical device 226 can be removed from the proximal end 217 of the flexible body 216 or from another optional instrument port (not shown) along the flexible body 216.

[0052] The medical device 226 may additionally house a cable, linkage, or other actuation control (not shown) extending between its proximal and distal ends to controllably bend the distal end of the medical device 226. Steerable instruments are described in detail in U.S. Patent No. 7,316,681 (filed on October 4, 2005) (disclosing "Articulated Surgical Instrument for Performing Minimally Invasive Surgery with Enhanced Dexterity and Sensitivity") and U.S. Patent Application No. 12 / 286,644 (filed on September 30, 2008) (disclosing "Passive Preload and Capstan Drive for Surgical Instruments"), which are incorporated herein by reference in their entirety.

[0053] The flexible body 216 may also house cables, linkages, or other steering controls (not shown) extending between the drive unit 204 and the distal end 218 to controllably bend the distal end 218, as shown, for example, by a dashed line depiction 219 of the distal end 218. In some examples, at least four cables are used to provide independent "up-down" steering to control the pitch of the distal end 218, and to provide "left-right" steering to control the yaw of the distal end 281. Steerable elongated devices are described in detail in U.S. Patent Application No. 13 / 274,208 (filed on October 14, 2011) (disclosing "Catheter with Removable Vision Probe"), which is incorporated herein by reference in its entirety. In embodiments where the medical device system 200 is actuated by a teleoperated assembly, the drive unit 204 may include a drive input that is removably coupled to and receives power from a drive element (such as an actuator) of the teleoperated assembly. In some embodiments, the medical device system 200 may include gripping features, manual actuators, or other components for manually controlling the movement of the medical device system 200. The elongated device 202 may be steerable, alternatively, the system may be non-steerable without an integrated mechanism for operator-controlled bending of the distal end 218. In some examples, one or more lumens are defined in the wall of the flexible body 216 through which a medical device may be deployed and used at a target surgical site.

[0054] In some embodiments, the medical device system 200 may include a flexible bronchial instrument (such as a bronchoscope or bronchial catheter) for inspection, diagnosis, biopsy, or treatment of the lungs. The medical device system 200 is also suitable for navigating and treating other tissues via natural or surgically created connecting passages in any of a variety of anatomical systems, including the colon, intestines, kidneys and kidney stones, brain, heart, circulatory system including vascular system, etc.

[0055] Information from tracking system 230 may be sent to navigation system 232 where it is combined with information from visualization system 231 and / or preoperatively acquired models to provide real-time positioning information to a physician or other operator. In some examples, the real-time positioning information may be displayed on a Figure 1 on the display system 110 for controlling the medical device system 200. In some examples, Figure 1 The control system 116 can use the positioning information as feedback for positioning the medical instrument system 200. Various systems using fiber optic sensors to align and display surgical instruments with surgical images are provided in U.S. Patent Application No. 13 / 107,562 (filed on May 13, 2011) (publishing “Medical System Providing Dynamic Registration of a Model of an Anatomic Structure for Image-Guided Surgery”), PCT Publication WO2016 / 1033596 (filed on May 20, 2016) (publishing “Systems and Methods of Registration for Image Guided Surgery”), and PCT Publication WO2016 / 164311 (filed on April 4, 2016) (publishing “Systems and Methods of Registration Compensation in Image Guided Surgery”), which are incorporated herein by reference in their entirety.

[0056] In some examples, the medical device system 200 may be Figure 1 In some embodiments, Figure 1 The manipulator assembly 102 of the apparatus may be replaced by direct operator controls. In some examples, the direct operator controls may include various handle interfaces and operator interfaces for handheld operation of the apparatus.

[0057] Figure 3A and Figure 3Bis a simplified diagram of a side view of a patient coordinate space including a medical device mounted on an insertion assembly according to some embodiments. Figure 3A and Figure 3B As shown, surgical environment 300 includes a Figure 1 Patient P is placed on an operating table T. Patient P can be stationary in the operating environment in the sense that the patient's overall movement is limited by sedation, restraints and / or other means. Unless the patient is asked to hold his breath to temporarily suspend respiratory movement, cyclic anatomical movement of patient P (including respiratory movement and cardiac movement) can continue. Therefore, in some embodiments, data can be collected at a specific stage of breathing, and the data can be marked and identified at that stage. In some embodiments, the stage of collecting data can be inferred based on the physiological information collected from patient P. Within the operating environment 300, a point collection instrument 304 is coupled to an instrument bracket 306. In some embodiments, the point collection instrument 304 can use an EM sensor, a shape sensor and / or other sensor modality. The instrument bracket 306 is mounted to an insertion table 308 fixed within the operating environment 300. Alternatively, the insertion table 308 can be movable, but has a known position within the operating environment 300 (e.g., via a tracking sensor or other tracking device). The instrument bracket 306 can be a component of a manipulator assembly (e.g., the manipulator assembly 102) that is coupled to the point collection instrument 304 to control the insertion motion (i.e., motion along the A axis) and optionally control the motion of the distal end 318 of the elongated device 310 in multiple directions including yaw, pitch, and roll. The instrument bracket 306 or insertion stage 308 can include an actuator (not shown) such as a servo motor that controls the motion of the instrument bracket 306 along the insertion stage 308.

[0058] The elongated device 310 is coupled to an instrument body 312. The instrument body 312 is coupled and fixed relative to the instrument bracket 306. In some embodiments, a fiber optic shape sensor 314 is fixed to the instrument body 312 at a proximal end point 316. In some embodiments, the proximal end point 316 of the fiber optic shape sensor 314 can move with the instrument body 312, but the position of the proximal end point 316 can be known (e.g., via a tracking sensor or other tracking device). The shape sensor 314 measures the shape from the proximal end point 316 to another point (such as a distal end 318 of the elongated device 310). The point collection instrument 304 can be substantially similar to the medical instrument system 200.

[0059] The position measurement device 320 provides information about the position of the instrument body 312 as it moves along the insertion axis A on the insertion stage 308. The position measurement device 320 can include a rotary transformer, an encoder, a potentiometer, and / or other sensor that determines the rotation or orientation of an actuator that controls the movement of the instrument bracket 306 and, therefore, the movement of the instrument body 312. In some embodiments, the insertion stage 308 is linear. In some embodiments, the insertion stage 308 can be curved or have a combination of curved and linear portions.

[0060] Figure 3A The instrument body 312 and instrument bracket 306 are shown in a retracted position along the insertion stage 308. In this retracted position, the proximal end point 316 is at position L0 on the axis A. In this position along the insertion stage 308, the A component of the position of the proximal end point 316 can be set to zero and / or another reference value to provide a baseline reference to describe the positioning of the instrument bracket 306, thereby describing the positioning of the proximal end point 316 on the insertion stage 308. In this retracted position of the instrument body 312 and instrument bracket 306, the distal end 318 of the slender device 310 can be positioned only within the access orifice of the patient P. Also in this position, the positioning measurement device 320 can be set to zero and / or another reference value (e.g., I=0). Figure 3B 308, the instrument body 312 and instrument carrier 306 are advanced along the linear track of the insertion table 308, and the distal end 318 of the elongated device 310 has been advanced into the patient P. In this advanced position, the proximal end point 316 is at position L1 on the axis A. In some examples, encoders and / or other positioning data from one or more actuators that control the movement of the instrument carrier 306 along the insertion table 308 and / or one or more positioning sensors associated with the instrument carrier 306 and / or the insertion table 308 are used to determine the position L0 of the proximal end point 316 relative to the position L1. X In some examples, positioning L X It can further serve as an indicator of the distance or insertion depth of the distal end 318 of the slender device 310 inserted into the passage of the anatomical structure of the patient P.

[0061] Figure 4A , Figure 4B , Figure 4C and Figure 4D Show Figure 3A and Figure 3B The elongated device 310 is provided by Figure 1 , Figure 3A and Figure 3B308. The operator O can steer the elongated device 310 toward the distal end 318 of the elongated device 310 to navigate through the anatomical passages 402 of the lungs 400 of the patient P. These passages 402 include the trachea and bronchi. As the elongated device 310 is advanced as the bracket 306 moves along the insertion table 308, the operator O can turn the distal end 318 of the elongated device 310 to navigate through the anatomical passages 402. While navigating through the anatomical passages 402, the elongated device 310 assumes a shape that can be measured by a shape sensor 314 extending within the elongated device 310.

[0062] Reference Figure 5 , Fig. 6A , Figure 6B , Figure 6C , Figure 7 , Figure 8 , Fig. 9 , Fig.10 and Fig.11 , various embodiments of image-guided surgical procedures using registration quality analysis are described. Figure 5 is a flow chart illustrating a general method 500 used in an image guided surgery procedure. Fig. 6A , Figure 6B and Figure 6C The segmentation process of a general method 500 is shown, which generates a human lung model for registration. Figure 7 and Figure 8 are flow charts showing global registration quality analysis and local registration quality analysis, respectively. Fig. 9 and Fig.10 A method for performing registration based on registration quality analysis is shown. Fig.11 A visual representation of the registration quality indicator is shown. As discussed in detail below, in an example, providing the registration quality indicator to the operator can help the operator better understand the uncertainty in the registration, which can further assist the operator's navigation operations. In another example, various actions can be performed based on the registration quality indicator (e.g., by the operator or the control system), which can improve the quality of the image-guided procedure. In yet another example, the registration quality analysis can be performed for the entire model space or for a specific region of interest, where the registration quality is more important to the operator.

[0063] Figure 5 is a flow chart illustrating a general method 500 used in an image guided surgery procedure. Figure 5 The method 500 is shown as a set of operations or processes 502 to 512. Not all of the illustrated processes 502 to 512 are performed in all embodiments of the method 500. Figure 5One or more processes not explicitly shown in the drawings are included before, after, between, or as part of processes 502 to 512. In some embodiments, one or more processes may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., a processor of the control system 112) may cause the one or more processors to perform one or more processes.

[0064] At process 502, preoperative or intraoperative image data is obtained from an imaging technique such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, infrared imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, or nanotube x-ray imaging. The preoperative or intraoperative image data may correspond to a two-dimensional image, a three-dimensional image, or a four-dimensional (including, for example, time-based or velocity-based information) image. For example, the image data may represent Figure 4A Hezhi Figure 4D 400 human lungs.

[0065] At process 504, the recorded image is transformed into a segmented two-dimensional or three-dimensional composite representation or model of a portion or the entire anatomical organ or anatomical region using a computer system operating alone or in combination with manual input. Fig. 6A Show FIG. 4A to FIG. 4D Segmented model 600 of lung 400. Due to limitations of the data or segmentation algorithms, segmented model 600 may not include all channels of interest present in human lungs, but rather include some channels 601. For example, relatively narrow and / or distal channels of the lungs may not be completely included in segmented model 600. Segmented model 600 may be a three-dimensional model such as a grid model, a linkage model, or another suitable model that defines the internal cavity or channels of the lungs. Typically, the model is used as a spatial template for airway geometry within a preoperative or intraoperative reference frame. The composite representation and image data set describe the various locations and shapes of the channels and their connectivity, and undesirable anatomical portions included in the preoperative or intraoperative image data may be omitted. In some embodiments, model 600 may include particularly desired features such as suspected tumors, lesions, or other tissue portions of interest.

[0066] In the segmentation process, the image is divided into segments or elements (e.g., pixels or voxels) that share certain features or calculated properties (such as color, density, intensity, and texture). The segmentation process results in a two-dimensional or three-dimensional reconstruction that forms a model of the target anatomical structure (such as model 600) based on the acquired image. To represent the model, the segmentation process can depict a set of voxels representing the target anatomical structure and then apply a function (such as a marching cube function) to generate a 3D surface surrounding the voxels. The model can be created by generating a grid, volume, or voxel map. The model can be displayed in a display 110 to help the operator O visualize the anatomical structure (such as the internal passages of the lungs).

[0067] Additionally or alternatively, the model may include a centerline model comprising a set of interconnected line segments or points extending through the center of the modeled channel. Figure 6B An example centerline model 602 derived from the model 600 or directly from the imaging data is shown. The centerline segmented model 602 may include a set of three-dimensional straight lines or a set of curves that correspond to the approximate centers of the passages contained in the segmented model 602. The higher the resolution of the model, the more accurately the set of straight lines or the set of curves will correspond to the center of the passage. Using the centerline segmented model 602 to represent the lungs may provide a smaller data set that can be more efficiently processed by one or more processors or processing cores compared to a data set of the segmented model 602 representing the passage walls of the model 600. In this way, the functionality of the control system 112 may be improved.

[0068] like Figure 6B As shown, the centerline segment model 602 includes several branch points, some of which are located at Figure 6B 602. The trachea is divided into left and right main bronchi at each of the several branch points. Branch point A can represent the point in the model where the trachea is divided into left and right main bronchi. The right main bronchi can be identified as being located between branch points A and B in the centerline segmented model 602. Similarly, the secondary bronchi are identified by branch points B and C and are between branch points B and E. Another generation can be defined between branch points C and D. Each of these generations can be associated with a representation of the lumen diameter of the corresponding channel. In some embodiments, the model 602 can include an average diameter value for each segmented generation. The average diameter value can be a patient-specific value or a more general value from multiple patients.

[0069] Where the model comprises a centreline model, the centreline model comprises a set of interconnected line segments which may be transformed into a cloud or set of points 604 (referred to as model points) consisting of Figure 6CBy transforming the line segments into points, the desired number of model points corresponding to the interconnected line segments can be selected manually or automatically during the registration process to represent the centerline model 602 (and thus the model 600). In the data, each point in the model point set 604 can include a number such as X M , Y M and Z M The coordinates of the coordinate set or other coordinates identifying the location of each point in the three-dimensional model space. In some embodiments, each point may include a generation identifier that identifies which channel generation the points are associated with and / or a diameter or radius value associated with that portion of the centerline segment model 602. In some embodiments, information describing the radius or diameter associated with a given point may be provided as part of a separate data set.

[0070] The centerline segment model is generated 602 and stored in the data as Figure 6C After the point set 604 shown in , the model points 604 can be retrieved from the data store for use in an image guided surgery procedure. In order to use the centerline segmented model 602 and the model 600 in an image guided surgery procedure, the model points 604 can be registered to associate the modeled channels in the model 600 with the actual anatomy of the patient present in the surgical environment.

[0071] return Figure 5 , in process 506, as shown in reference FIG. 3A to FIG. 3B and Figures 4A to 4D As described, measurement points corresponding to the anatomical model can be obtained from the patient anatomy. The measurement points are generated by advancing the elongated device through the anatomy and / or reaching landmarks in the anatomy while measuring the position of the distal end of the elongated device or the posture of the elongated device using a sensor system (e.g., sensor system 108). The measurement points are associated with patient space and may also be referred to as patient space points.

[0072] In process 508, the anatomical model data in the model space is registered to the patient's anatomy in the patient space (or vice versa) before and / or during an image-guided surgical procedure on the patient. Typically, registration involves matching measured points with model points of the model using rigid and / or non-rigid transformations. Within the scope of the present disclosure, point set registration methods (e.g., iterative closest point (ICP) techniques) may also be used in the registration process. Such point set registration methods can generate a transformation that aligns measured points (also referred to as a measurement point set) and model points (also referred to as a model point set).

[0073] In various embodiments, the quality of the registration may depend on various factors including, for example, the number of measurement points and / or model points, the density of the measurement points and / or model points, the distribution of the measurement points and model points relative to the region of interest, measurement errors associated with the measurement points and / or model points, and deformations of the patient's anatomy associated with the measurement points and / or model points.

[0074] At process 510, a registration quality analysis may be performed to generate a registration quality indicator for the registration performed at process 508. Such a registration quality indicator may be provided to an operator (e.g., using display system 110), which may provide the operator with a degree of confidence in the quality of the registration. In examples where the registration quality indicator indicates that the registration quality is high (e.g., based on a high quality threshold), such a registration quality indicator may increase the operator's confidence in the image-guided surgical procedure. In examples where the registration quality indicator indicates that the registration quality is low (e.g., based on a low quality threshold), such a registration quality indicator conveys uncertainty (e.g., a suspect alignment or a misalignment) to the operator.

[0075] At process 512, various actions may be performed based on the registration quality indicator. For example, when the registration quality indicator indicates that the registration quality is low (e.g., based on a low quality threshold), instructions may be provided (e.g., using the display system 110) to the operator to take necessary precautions (such as stopping advancement of the elongated device). In another example, when the registration quality indicator indicates that the registration quality is low, the operator may be instructed to use additional / alternative navigation aids (e.g., fluoroscopy or intraoperative CT). In yet another example, when the registration quality indicator indicates that the registration quality is low, the registration at process 508 may be performed again to improve the registration quality, or the operator may be instructed to perform the registration process again.

[0076] Reference Figure 7 and Figure 8 , a process for registration quality analysis (e.g., Figure 5 Process 510) may include global registration quality analysis, local registration quality analysis and / or a combination thereof. Figure 7 A method 700 for global registration quality analysis is shown, and Figure 8 A method 800 for local registration quality analysis is shown.

[0077] Reference Figure 7 , Figure 7 The method 700 in FIG. 7 is shown as a set of operations or processes 702 to 716. Not all of the illustrated processes 702 to 716 are performed in all embodiments of the method 700. Figure 7One or more processes not explicitly shown in the drawings may be included before, after, between, or as part of processes 702 to 716. In some embodiments, one or more processes may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., a processor of the control system 112) may cause the one or more processors to perform one or more processes.

[0078] Method 700 begins at process 702, where global registration is performed to register the model space to the patient space. Global registration can generate an optimal transformation T opt , for aligning the model point set in the model space with the measurement point set in the patient space. In such a global registration, a registration algorithm (e.g., a point set registration algorithm) can be used to align all points to both the measurement point set and the model point set. In the global registration, the optimal transformation T opt is calculated from points inside and outside such a particular region of interest. Note that in some examples, the optimal transformation T opt is a solution calculated based on an optimal set of input parameters, which may differ from the absolute or true optimal solution.

[0079] In some embodiments, at process 702, a registration algorithm for global registration may use an optimal initial parameter set (eg, best guess initial parameters, initial parameters generated based on parameterization) to generate an optimal transformation T opt The initial parameters may include a transformation seed or a weighting scheme applied to the model or collection points, a point correspondence configuration (e.g., soft correspondence, hard correspondence), and / or other suitable initial parameters. In some embodiments, process 702 is skipped. In such embodiments, Figure 5 The transformation provided by process 508 can be used as the transformation T opt , the initial parameters for the registration performed in step 508 may be used as optimal initial parameters.

[0080] Method 700 continues to process 704, where the perturbation index i is initialized to zero. Then, method 700 may continue to process 706, where a perturbation index i is initialized to zero to generate an optimal transformation T opt The optimal input parameter set of is perturbed to generate the i-th perturbed input parameter set. One or more parameters in the optimal input parameter set (e.g., transformation seed, point weighting scheme, corresponding configuration) may be slightly perturbed to generate the i-th perturbed initial parameter set.

[0081] Then, method 700 may proceed to process 708, where a global alignment is performed using the i-th perturbed initial parameter set to generate a perturbed transform In the example, at process 708, the registration algorithm (eg, point set registration algorithm) is restarted to use the i-th perturbed initial parameter set to generate a perturbed transformation

[0082] Then, the method 700 may proceed to process 710, where it is determined whether the perturbation index i is less than n-1, where n is a predetermined total perturbation number. The total perturbation number n may be an integer having a value equal to or greater than one. In some embodiments, the total perturbation number n and the modified value of each initial parameter may be configured such that the registration transformation generated by the n perturbations is to The distribution of represents the optimal transformation T opt uncertainty in .

[0083] In various embodiments, n perturbation initial parameter sets are generated for n perturbations, respectively. N perturbation initial parameter sets are generated by modifying the optimal initial parameters so that these perturbation initial parameters cover the expected range of initial parameter values. In an example, n perturbation initial parameter sets include random initial parameter values ​​within the expected range of initial parameter values. The registration seed transform can be perturbed. In another example (e.g., airway registration), a perturbation (e.g., a maximum of 2 cm and 30 degrees of perturbation) can be applied to the initial transform. The size of the perturbation can be determined to have a large enough value so that it reflects the uncertainty in the initial guess, and a small enough value to avoid solutions that are too harsh or beyond the realistic range. In another embodiment, the weights of the model or collection points can be adjusted randomly or in relation to the spatial distribution. For example, the point weights can be perturbed to bias points on one side of the lung higher than the other side of the lung, or to bias points on a given lobe higher than another lobe. In another embodiment, the algorithm parameters can be changed so that the correspondence is calculated differently or weighted differently. In an example, by using a particular set of algorithm parameters, the registration can be heavily driven by poorly matched point correspondences (e.g., poorly aligned points). In another example, by using another particular set of algorithm parameters, point correspondences with large errors can be weighted down or ignored entirely.

[0084] In some embodiments, at process 710, it is determined that the perturbation index i is less than n-1. In these embodiments, method 700 proceeds to process 712 to increase the perturbation index i by 1. Then, method 700 may proceed to process 706 and process 708 to calculate the next perturbation transformation using the i-th perturbation initial parameter set.

[0085] In some embodiments, at process 710, it is determined that the perturbation index i is not less than n-1. In these embodiments, method 700 proceeds to process 714, where for each point y in the point set in the model space k, point average registration error E k The calculation is as follows:

[0086]

[0087] Where k is the point index, i is the perturbation index, and n is the total number of perturbations. In various embodiments, the error E k can be calculated in any other way to measure the rigid transformation T opt and T perturbed The back-projection error of the model point y is slightly different from that of the given estimate of . k It can be populated from a variety of point sets including, for example, all points used in the model, points within a region of interest, points within some grid equally spaced throughout the volume, any other suitable point set, and / or combinations thereof.

[0088] In some embodiments, a point average registration error is calculated for each measurement point of the set of measurement points in patient space.In an example, by calculating the point average registration error of the measurement points in patient space, a registration quality indicator can be calculated to indicate uncertainty related to the path of the operator driving the elongated device 202.

[0089] Then, method 700 may proceed to process 716, where the average registration error E 0 To E m-1 The global registration quality index (GRQI) is calculated as follows:

[0090]

[0091] Where k is the point index and m is the total number of points (e.g., some or all points in model space or patient space) used to calculate the point average registration error. Note that although equations (1) and (2) are used as examples to illustrate the calculation of GRQI, any suitable algorithm for calculating the global registration quality indicator can be used.

[0092] In some embodiments, the global registration quality indicator comprises a GRQI. In some embodiments, the global registration quality indicator is determined based on the GRQI. For example, the global registration quality indicator may include a quality level having a value selected from "low", "medium", and "high" determined based on the GRQI. In an example where the GRQI is less than a global registration quality high threshold, the global registration quality indicator comprises a quality level of "high". In another example where the GRQI is greater than a global registration quality low threshold, the global registration quality indicator comprises a quality level of "low". In another example where the GRQI is between a global registration quality high threshold and a global registration quality low threshold, the global registration quality indicator comprises a quality level of "medium".

[0093] For global registration quality analysis Figure 7When method 700 of Calculates the distribution of registration transforms across a range of expected initial parameters, these global registration transforms may be insensitive to misalignment of a subset of points (e.g., in a region of interest). Such a region of interest may be a region where an operator performs a biopsy to extract tissue or a region where an operator applies a treatment. Therefore, the local registration quality of the region of interest may be more important to the operator than the global registration quality. Figure 8 , shows a method 800 for local registration quality analysis, which can provide a registration quality indicator associated with such a smaller subset of points in the region of interest. In some embodiments, the local registration quality indicator is calculated by comparing the consistency of registration transformation results associated with two or more regions in the model space.

[0094] Figure 8 The method 800 in FIG. 8 is shown as a set of operations or processes 802 to 810. Not all of the illustrated processes 802 to 810 are performed in all embodiments of the method 800. Figure 8 One or more processes not explicitly shown in the drawings are included before, after, between, or as part of processes 802 to 810. In some embodiments, one or more processes may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., a processor of the control system 112) may cause the one or more processors to perform one or more processes.

[0095] Method 800 begins at process 802, where a local registration is performed to register a region of interest in model space to patient space and generate a region of interest transformation T1. Such a region of interest transformation T1 may be calculated using model points within the region of interest in model space. In some embodiments, the region of interest is provided by an operator. In some embodiments, at this stage, a global registration has been performed using an optimal initial parameter set (e.g., Figure 5 Process 508) to generate the transformation T opt The optimal initial parameter set or a perturbed initial parameter set generated based on the optimal initial parameter set may be used to perform local registration for registering the region of interest.

[0096] Method 800 can then proceed to process 804, in which a comparison region of the model space is determined. In some embodiments, the comparison region is determined based on the region of interest. For example, the comparison region can be the model space other than the region of interest. In some embodiments, the comparison region is a region provided by an operator. In an example, the region of interest and the comparison region can overlap. In another example, the region of interest and the comparison region do not overlap. In yet another example, the region of interest and the comparison region are not adjacent to each other. In yet another example, the region of interest and the comparison region can be fuzzy so that some points can be partial members of the two regions. In an example for the registration of points within the airway, the region of interest can consist of all points within a given lobe, while the comparison region is all points outside the lobe. In another example, the comparison region can include all points that are not within the lobe but within the main stem bronchus. In yet another example, the comparison region can consist of points within an alternative lobe or multiple lobes.

[0097] Then, method 800 can proceed to process 806, in which local registration of the comparison region in model space to patient space is performed to calculate a comparison region transformation T2 for the comparison region. Such a comparison region transformation T2 can be calculated using only points within the comparison region in model space. In some embodiments, the local registration used to register the comparison region uses an initial parameter set that is the same as the initial parameter set used to register the region of interest at process 802.

[0098] Method 800 may then proceed to process 808, where for each point y within the region of interest k1 , the point registration error E is calculated based on the transformation of the region of interest T1 and the comparison region transformation T2 as follows k1 :

[0099] E k1 =||T1 -1 y k1 -T2 -1 y k1 || (3)

[0100] Where k1 is the point index of the point in the region of interest.

[0101] In some embodiments, the process 802 to 808 is repeated for n1 perturbations, and for the region of interest and the comparison region, the perturbation transforms are generated using n1 perturbation initial parameters, respectively. to and to It can be basically similar to the above Figure 7In some embodiments, in each of the n 1 perturbations, the same set of perturbation initial parameters may be used to align the region of interest and the comparison region to generate the perturbation initial parameters. and where i is the perturbation index between 0 and n1-1. Based on the perturbation transformation to and to The point registration error E k1 It is calculated as the point average registration error as follows:

[0102]

[0103] Where kl is the point index of a point in the region of interest, i is the perturbation index, and nl is the total number of perturbations used for local registration quality analysis.

[0104] Then, method 800 may proceed to process 810 where a local registration quality index (LRQI) is calculated using the average error of all ml points in the region of interest in the model space as follows:

[0105]

[0106] Where k1 is the point index of the region of interest and ml is the total number of points in the region of interest. Note that although equations (3), (4) and (5) are used as examples to illustrate the calculation of LRQI, any suitable algorithm for calculating the local registration quality indicator may be used.

[0107] In some embodiments, the local registration quality indicator comprises an LRQI. In some embodiments, the local registration quality indicator is determined based on the LRQI. For example, the local registration quality indicator may comprise a quality level having a value selected from "low", "medium" and "high" determined based on the LRQI. In an example where the LRQI is less than a local registration quality high threshold, the local registration quality indicator comprises a quality level of "high". In another example where the LRQI is greater than a local registration quality low threshold, the local registration quality indicator comprises a quality level of "low". In another example where the LRQI is between a local registration quality high threshold and a local registration quality low threshold, the local registration quality indicator comprises a quality level of "medium". In some embodiments, such local / global registration quality high / low thresholds may be configured by an operator for a specific region of interest.

[0108] Reference Fig. 9 and Fig.10 In some embodiments, actions performed based on the registration quality indicator (e.g., Figure 5The process 512 of ) may include performing the registration again (re-registration) to improve the registration quality. Such re-registration may be performed in a modified model space by excluding specific areas from the model space based on a registration quality indicator (e.g., a global registration quality indicator and / or one or more local registration quality indicators).

[0109] Fig. 9 The method 900 in FIG. 1 is shown as a set of operations or processes 902 to 910. Not all of the illustrated processes 902 to 910 are performed in all embodiments of the method 900. Fig. 9 One or more processes not explicitly shown in the drawings are included before, after, between, or as part of processes 902 to 910. In some embodiments, one or more processes may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., a processor of the control system 112) may cause the one or more processors to perform the one or more processes.

[0110] Method 900 begins at process 902 where a first set of regions of a model space and corresponding local registration quality thresholds are received. The first set of regions may include one or more regions of interest identified by an operator.

[0111] Reference Fig.10 , which shows an example visual representation for an operator. Fig.10 As shown, the display 110 is based on the user interface Fig. 6A The anatomical model 600 shows a rendering of the anatomical passages of the human lung. Figure 5 described in (for example, Figure 5 ), with the model spatially registered to the patient, the current shape of the elongated device 310 and the position of the distal end 318 can be located and displayed simultaneously with a rendering of the channel 601 including the channels 601A and 601B. The model points can be visually represented in the user interface by a plurality of points or dots on the display, or can be represented as a rendered model (such as a mesh or line model created based on a set of data points).

[0112] like Fig.10 As shown, in some embodiments, at process 902, a region set including region 1002 and region 1004 is received. In an example, region 1002 and region 1004 have the same registration quality threshold. In another example, region 1002 and region 1004 have different registration quality thresholds (e.g., based on their registration quality importance to an operator).

[0113] In some embodiments, at this stage of process 902, a global registration of the entire model space to the patient space has been performed to generate the transformation T opt .exist Fig.10 In the example of , a global registration quality analysis has been performed, and the display 110 includes a global registration quality area 1006 that displays a global registration quality indicator (eg, “High”) for the current global registration.

[0114] Method 900 may proceed to process 904, where a local registration quality analysis (eg, Figure 8 Method 800). A local registration quality indicator may be generated for each region (eg, using a corresponding local registration quality threshold). In some embodiments, the local registration quality analysis receives a transform T for generating a global registration. opt and use the optimal initial parameter set and / or one or more perturbation initial parameter sets (generated based on the optimal initial parameter set), as described above with respect to Figure 8 discussed, for use in registering regions.

[0115] like Fig.10 As shown, in some embodiments, at process 904, display 110 may include a local registration quality area 1008 that displays local registration quality indicators for the first set of regions generated at process 904 (e.g., "medium" for region 1002, "low" for region 1004).

[0116] Method 900 may proceed to process 906, where one or more regions to be excluded are determined based on the global registration quality indicator and / or the local registration quality indicator. In an example, the control system may determine one or more regions to be excluded (e.g., based on a registration quality threshold). In another example, Fig.10 As shown, the operator can provide one or more areas to be excluded. Fig.10 In the example of , display 110 includes re-registration region 1010 , and the operator determines not to exclude region 1002 from re-registration (eg, using selection 1012 ) and to exclude region 1004 from re-registration (eg, using selection 1014 ).

[0117] Method 900 may proceed to process 908 to generate a modified model space based on the one or more regions to be excluded. Fig.10 In the example of , in process 908 , a modified model space is generated by excluding region 1004 from the model space.

[0118] Method 900 may proceed to process 910 to perform a re-registration using the modified model space. In an example, such re-registration is automatically performed by a control system based on the registration quality indicator and a predetermined re-registration criterion. In another example, such as Fig.10 As shown, the operator can initiate re-registration using the modified model space by selecting button 1016. While in some embodiments, the operator can perform re-registration based on a local registration quality indicator associated with the region of interest, in other embodiments, the operator can perform re-registration based on a global registration quality indicator. By performing re-registration using the modified model space, the registration quality (of the entire model space and / or a specific region) can be improved.

[0119] Reference Fig.11 In various embodiments, the registration quality indicator may be displayed to the operator using various visual representations including, for example, a numerical representation; a graphical representation such as a heat map using color, shading, patterns; any suitable visual representation, and / or combinations thereof. Fig.11 As shown in the example of FIG. 1 , the display 110 is based on the user interface Fig. 6A The anatomical model 600 shows a rendering of an anatomical passage of a human lung, wherein the current shape of the elongated device 310 and the position of the distal end 318 are positioned and displayed simultaneously with the rendering of the passage 601. In addition, a heat map based on the registration quality indicator (e.g., using shading, color, pattern, etc.) is displayed in the anatomical model 600. For example, points in region 1102 of the anatomical model 600 have a registration quality indicator value indicating low quality (e.g., by comparing with one or more low registration quality thresholds). Region 1102 can have a first shading (e.g., light) or a first color (e.g., orange) associated with low quality. Points in region 1104 have a registration quality indicator value indicating medium quality (e.g., by comparing with one or more medium registration quality thresholds). Thus, region 1104 can have a second shading (e.g., medium) or a second color (e.g., yellow) associated with medium quality that is different from the first shading or first color. Points in region 1106 have a registration quality indicator value indicating high quality (e.g., by comparing with one or more high registration quality thresholds). Thus, region 1106 may have a third shade (e.g., dark) or a third color (e.g., green) that is different from the first and second shades or the first and second colors. In various embodiments, the operator may accept the registration (e.g., by selecting button 1108 “Accept Registration”) or restart the registration (e.g., by selecting button 1110 “Re-Register”) based on the visual representation of the registration quality indicator in display 110.

[0120] The systems and methods of the present disclosure may be used for connected bronchial passages of the lungs. The systems and methods may also be suitable for navigating and treating other tissues in any anatomical system including the colon, intestines, kidneys, brain, heart, circulatory system, etc. via natural or surgically created connected passages. The systems and methods may also be suitable for navigating around trackable surfaces of organs. The methods and embodiments of the present disclosure are also suitable for non-surgical applications.

[0121] One or more elements in the embodiments of the present invention may be implemented in software to run on a processor of a computer system such as control system 112. When implemented in software, the elements of the embodiments of the present invention are essentially code segments for performing necessary tasks. The program or code segment may be stored in a processor-readable storage medium or device, which may be downloaded via a computer data signal contained in a carrier wave on a transmission medium or communication link. The processor-readable storage device may include any medium (including optical media, semiconductor media, and magnetic media) that may store information. Examples of processor-readable storage devices include electronic circuits, semiconductor devices, semiconductor storage devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs), floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. Code segments may be downloaded via a computer network (such as the Internet, Intranet, etc.).

[0122] Note that the processes and displays presented may not be inherently related to any particular computer or other device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the described operations. The structures required for various of these systems will appear as elements in the claims. In addition, embodiments of the present invention are not described with reference to any particular programming language. It should be understood that the teachings of the present invention as described herein may be implemented using a variety of programming languages.

[0123] Although certain example embodiments of the present invention have been described and shown in the accompanying drawings, it should be understood that such embodiments are merely illustrative of the present invention and not limiting of the present invention, and since various other modifications may be made by those skilled in the art, the embodiments of the present invention are not limited to the structure and arrangement of the present invention shown and described.

Claims

1. A system comprising: non-transitory memory; One or more processors coupled to the non-transitory memory and configured to read instructions to cause the system to perform operations including: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; registering the model point set to the measurement point set using a first initial parameter set to generate a first transformation; generating one or more disturbance initial parameter sets based on the first initial parameter set; performing one or more perturbation registration processes to register the model point set to the measurement point set using the one or more perturbation initial parameter sets, respectively, to generate one or more perturbation transformations; generating a first registration quality indicator based on the first transform and the one or more perturbation transforms; generating a second registration quality indicator by performing a registration quality analysis on a region of interest of the model; and A third registration quality indicator is generated by performing a registration quality analysis on a second region of interest of the model.

2. The system of claim 1, wherein each of the one or more perturbation initial parameter sets is within a predetermined range of initial parameter values, and Wherein the predetermined range is associated with an uncertainty of the first transformation. 3 . The system of claim 2 , wherein each of the one or more perturbation initial parameter sets comprises a random initial parameter value within the predetermined range. 4 . The system of claim 1 , wherein the first perturbation initial parameter set comprises a perturbation transform seed different from a transform seed of the first initial parameter set.

5. The system of claim 1, wherein the first perturbation initial parameter set includes a perturbation point weighting scheme that is different from a point weighting scheme of the first initial parameter set.

6. The system of claim 1, wherein the first perturbation initial parameter set includes a perturbation correspondence configuration different from a point weighting scheme of the first initial parameter set.

7. The system of claim 1, wherein said generating said first registration quality indicator comprises: generating a plurality of point registration errors associated with the anatomical structure based on the first transform and the one or more perturbation transforms; and A first registration quality index of the first registration quality indicator is generated by calculating an average error of the plurality of point registration errors.

8. The system of claim 7, wherein said generating said point registration error comprises: generating one or more disturbance point errors corresponding to the one or more disturbance transforms; and The point registration error is provided by calculating an average error of the one or more perturbation point errors.

9. The system of claim 7, wherein each of the plurality of point registration errors is associated with a model point in the set of model points.

10. The system of claim 7, wherein the plurality of point registration errors are associated with a target region of the anatomical structure.

11. The system of claim 1 , wherein the operations further comprise: A display representation associated with the first registration quality indicator is provided on a display.

12. The system of claim 11, wherein the displayed representation includes one or more numbers associated with the first registration quality indicator.

13. The system of claim 11, wherein the display representation comprises a heat map associated with the first registration quality indicator.

14. The system of claim 11, wherein the operations further comprise: comparing the first registration quality indicator to a registration quality threshold to generate a comparison result; and A registration quality message associated with the comparison result is provided in the display representation.

15. The system of claim 14, wherein the registration quality message includes an option for an operator to re-register the model point set to the measurement point set.

16. A method performed by a computing system, the method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; determining a region of interest in the model space; determining a comparison region in the model space; registering a first set of model points in the region of interest to the set of measured points to generate a first number of region of interest transformations; registering a second model point set in the comparison region to the measurement point set to generate a first number of comparison region transformations corresponding to the first number of region of interest transformations respectively; and A registration quality indicator for the region of interest is generated based on the first number of region of interest transforms and the first number of comparison region transforms. The method of claim 16 , wherein the comparison region includes the model space excluding the region of interest. The method of claim 16 , wherein the region of interest and the comparison region are non-adjacent.

19. The method of claim 16, wherein the first region of interest transform and the corresponding first comparison region transform are generated using the same initial parameter set.

20. The method of claim 16, wherein the first number of region of interest transforms are respectively generated using a first number of perturbed initial parameter sets.

21. The method of claim 16, wherein generating the registration quality indicator for the region of interest comprises: for each model point within the region of interest, generating a point registration error based on the first number of region of interest transformations and the first number of comparison region transformations; and A registration quality index of the registration quality indicator is generated by calculating a first average of the point registration errors associated with the model points within the region of interest.

22. The method according to claim 21, wherein: Generating the point registration error for each model point in the region of interest includes: generating a first number of comparison point errors, wherein each comparison point error is generated using a corresponding region of interest transform and a corresponding comparison region transform; and The point registration error is provided by calculating a second average of the first number of comparison point errors.

23. A method performed by a computing system, the method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; registering a first set of model points in the region of interest to the set of measurement points to generate a first set of region of interest transforms; performing a local registration quality analysis for the region of interest to generate a local registration quality indicator for the region of interest based on the first set of region of interest transforms; generating a modified model space based on the local registration quality indicator; and A subset of model points associated with the modified model space is registered to the set of measurement points to generate a second set of region of interest transforms.

24. The method of claim 23, wherein generating the modified model space comprises: The modified model space is generated by excluding the region of interest from the model space based on a comparison between the local registration quality indicator and a local registration quality threshold associated with the region of interest.

25. The method of claim 23, wherein collecting the set of measurement points comprises: The set of measurement points is collected along the length of an elongated device inserted into the anatomy of the patient, the measurement points being determined by a shape of the elongated device in the patient space.

26. The method of claim 25, wherein the anatomical structure comprises one or more anatomical passages.

27. The method of claim 25, wherein collecting the set of measurement points comprises: A shape sensor of the elongated device is interrogated.

28. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted, when executed by one or more processors, to cause the one or more processors to perform a method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; registering the model point set to the measurement point set using a first initial parameter set to generate a first transformation; generating one or more disturbance initial parameter sets based on the first initial parameter set; performing one or more perturbation registration processes to register the model point set to the measurement point set using the one or more perturbation initial parameter sets, respectively, to generate one or more perturbation transformations; generating a first registration quality indicator based on the first transform and the one or more perturbation transforms; generating a second registration quality indicator by performing a registration quality analysis on a region of interest of the model; and A third registration quality indicator is generated by performing a registration quality analysis on a second region of interest of the model.

29. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted, when executed by one or more processors, to cause the one or more processors to perform a method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; determining a region of interest in the model space; determining a comparison region in the model space; registering a first set of model points in the region of interest to the set of measured points to generate a first number of region of interest transformations; registering a second model point set in the comparison region to the measurement point set to respectively generate a first number of comparison region transformations corresponding to the first number of region of interest transformations; and A registration quality indicator is generated for the region of interest based on the first number of region of interest transforms and the first number of comparison region transforms.

30. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, the plurality of machine-readable instructions being adapted, when executed by one or more processors, to cause the one or more processors to perform a method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; registering a first set of model points in the region of interest to the set of measurement points to generate a first set of region of interest transforms; performing a local registration quality analysis for the region of interest to generate a local registration quality indicator for the region of interest based on the first set of region of interest transforms; generating a modified model space based on the local registration quality indicator; A subset of model points associated with the modified model space is registered to the set of measurement points to generate a second set of region of interest transforms.

31. A method performed by a computing system, the method comprising: accessing a set of model points of a model of the patient's anatomy, the model points being associated with a model space; collecting a set of measurement points of the anatomical structure of the patient, the measurement points being associated with a patient space; registering the model point set to the measurement point set using a first initial parameter set to generate a first transformation; generating one or more perturbation initial parameter sets based on the first initial parameter set; performing one or more perturbation registration processes to register the model point set to the measurement point set using the one or more perturbation initial parameter sets, respectively, to generate one or more perturbation transformations; generating a first registration quality indicator based on the first transform and the one or more perturbation transforms; generating a second registration quality indicator by performing a registration quality analysis on a region of interest of the model; and A third registration quality indicator is generated by performing a registration quality analysis on a second region of interest of the model.

32. The method of claim 31, wherein each of the one or more perturbed initial parameter sets is within a predetermined range of initial parameter values, wherein the predetermined range is associated with an uncertainty of the first transformation.

33. The method according to claim 32, wherein each of the one or more perturbation initial parameter sets comprises a random initial parameter value within the predetermined range.

34. The method of claim 31, wherein the first perturbed initial parameter set comprises a perturbed transform seed different from a transform seed of the first initial parameter set.

35. The method of claim 31, wherein the first perturbation initial parameter set includes a perturbation point weighting scheme that is different from a point weighting scheme of the first initial parameter set.

36. The method of claim 31, wherein the first perturbation initial parameter set includes a perturbation correspondence configuration that is different from a point weighting scheme of the first initial parameter set.

37. The method of claim 31 , wherein said generating said first registration quality indicator comprises: generating a plurality of point registration errors associated with the anatomical structure based on the first transform and the one or more perturbation transforms; and A first registration quality index of the first registration quality indicator is generated by calculating an average error of the plurality of point registration errors.

38. The method of claim 37, wherein said generating said point registration error comprises: generating one or more disturbance point errors corresponding to the one or more disturbance transforms; and The point registration error is provided by calculating an average error of the one or more perturbation point errors.

39. The method of claim 37, wherein each of the plurality of point registration errors is associated with a model point in the set of model points.

40. The method of claim 37, wherein the plurality of point registration errors are associated with a target region of the anatomical structure.

41. The method of claim 31 , further comprising: A display representation associated with the first registration quality indicator is provided on a display.

42. The method of claim 41, wherein the displayed representation comprises one or more numbers associated with the first registration quality indicator.

43. The method of claim 41, wherein the displayed representation comprises a heat map associated with the first registration quality indicator.

44. The method of claim 41, further comprising: comparing the first registration quality indicator to a registration quality threshold to generate a comparison result; and providing a registration quality message associated with the comparison result in the display representation.

45. The method of claim 44, wherein the registration quality message includes an option for an operator to re-register the model point set to the measurement point set.

Images