Systems and methods for device verification and sensor calibration

Abstract

The invention relates to systems and methods for device verification and sensor calibration. Systems and methods for verifying and calibrating robotic instruments are provided. The robotic system may include an instrument carrier for receiving the elongate device, and the instrument carrier may include a set of drive sensors. The system may also include a tracking system configured to receive an indication that the elongate device is mounted on the instrument carriage, operate a set of actuators to articulate a distal portion of the elongate device, and generate a set of drive sensor data from a set of drive sensors. The tracking system may also be configured to generate a set of articulation sensor data from a shape sensor of the elongate device, compare the set of drive sensor data to the set of articulation sensor data to generate a test curve, and determine whether the test curve corresponds to a reference curve. A corrective action may be determined.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980065604.5, titled "Systems and Methods for Device Verification and Sensor Calibration", filed on October 1, 2019.

[0002] Cross - Reference to Related Applications

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 741,242, filed on October 4, 2018, the entire content of which is incorporated herein by reference. Technical Field

[0004] The examples described herein relate to systems and methods for a procedure, such as systems and methods for verifying the operation of a system and for calibrating sensors of a system. Background Art

[0005] Devices can be used to manipulate and perform tasks in a workspace. Such devices can be configured to be supported and operated, in part or in whole, by a manipulator assembly. Such devices and manipulator assemblies can be used to perform non - medical or medical procedures. For example, a medical tool or a medical manipulator can be used to perform minimally invasive medical procedures. As another example, an industrial tool or an industrial manipulator can be used for manufacturing or testing. As other examples, tools or manipulators can be used in procedures for entertainment, exploration, and various other purposes.

[0006] Minimally invasive medical techniques generally can be aimed at reducing the amount of tissue damaged during an invasive medical procedure, thereby reducing the patient's recovery time, discomfort, and harmful side effects. Such minimally invasive techniques can be performed through natural orifices in a patient's anatomy or through one or more incisions. Through these natural orifices or incisions, a clinician can insert a medical tool to reach a target tissue location. Minimally invasive medical tools include devices such as treatment instruments, diagnostic instruments, and surgical instruments. Minimally invasive medical tools can also include imaging devices (such as endoscopic devices) that provide a view inside a patient's anatomy to the user.

[0007] Certain medical and non - medical devices (including manipulation devices, imaging devices, or other sensing devices, etc.) can be remotely operated or otherwise computer - assisted. Before using a system that includes a device to perform a procedure, a safe and reliable mechanism is needed to verify that the system and the device are operating properly and to calibrate the sensors of the system. Summary of the Invention

[0008] A simplified overview of the various examples described herein is presented below and is not intended to identify key or important elements or to delineate the scope of the claims.

[0009] A robotic system can include an instrument carriage configured to receive an elongate device. The instrument carriage can include a set of actuators configured to drive the elongate device along at least one degree of freedom, and a set of drive sensors configured to monitor the set of actuators. The robotic system can also include a tracking system coupled to the instrument carriage and configured to: receive an indication that the elongate device is mounted on the instrument carriage, and operate the set of actuators to articulate a distal portion of the elongate device in at least one degree of freedom. The tracking system can also generate a set of drive sensor data from the set of drive sensors during articulation, generate a set of articulation sensor data from a shape sensor of the elongate device during articulation, and compare the set of drive sensor data with the set of articulation sensor data to generate a test curve. The tracking system can also determine whether the test curve corresponds to a reference curve, and determine whether the elongate device requires a corrective action based on whether the test curve corresponds to the reference curve.

[0010] In another example, a device includes: one or more processors and a non-transitory computer memory storing machine-executable instructions that, when executed by the one or more processors, cause the device to: receive an indication that a flexible elongate device is coupled to a drive unit, and command articulation of a distal portion of the flexible elongate device in at least one degree of freedom. The instructions can also cause the device to receive a set of drive sensor data during articulation, receive a set of articulation sensor data during articulation, and compare the set of drive sensor data with the set of articulation sensor data to generate a test curve. The instructions can also cause the device to determine whether the test curve meets one or more thresholds in a reference curve, and determine whether to perform a corrective action based on whether the test curve meets the one or more thresholds.

[0011] In another example, a method of performing a calibration test sequence for a robotic system can include: receiving an indication that a flexible elongate device is coupled to a drive system of the robotic system, the drive system including one or more drive system sensors; and commanding articulation of the flexible elongate device in at least one degree of freedom. The method can also include: generating drive sensor data from the one or more drive system sensors during the commanded articulation; generating articulation sensor data from an articulation sensor of the flexible elongate device during the commanded articulation; and comparing the drive sensor data with the articulation sensor data to generate a test curve. The method can also include: determining whether the test curve corresponds to a reference curve; and determining whether the flexible elongate device requires a corrective action based on determining whether the test curve corresponds to the reference curve.

[0012] It should be understood that the foregoing summary and the following detailed description are both illustrative and explanatory in nature and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. In this regard, other aspects, features, and advantages of the present disclosure will be apparent to those skilled in the art from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a simplified diagram of a medical system according to some embodiments.

[0014] Figure 2A and Figure 2B is a simplified side view of a patient coordinate space of a medical device mounted on an insertion assembly according to some embodiments.

[0015] Figure 3 is a simplified diagram of a system according to some embodiments.

[0016] Figure 4 is a flowchart showing a method of comparing drive sensor data with articulated sensor data according to some embodiments.

[0017] Figure 5A and Figure 5B is a diagram showing the relationship between groups of sensor data during a method of testing and / or calibrating a system according to some embodiments.

[0018] Figure 6 is a flowchart showing a method of comparing pose data with a commanded pose according to some embodiments.

[0019] Embodiments of the present disclosure and their advantages will be best understood by reference to the following detailed description. It should be understood that the same reference numerals are used to identify similar elements shown in one or more of the figures, wherein the display therein is for the purpose of illustrating embodiments of the present disclosure and not for the purpose of limiting the embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] The techniques described herein provide for testing and calibrating robotic devices or systems, such as during system startup or power-on. The disclosed techniques for testing and calibrating can be implemented on any robotic device or system, including those that implement user-installable or user-replaceable instruments having embedded sensors, actuators, or a combination of sensors and actuators. Examples of suitable robotic devices or systems include remotely operated or otherwise operated medical devices that utilize reusable instruments that are cleaned and reprocessed between uses. As an illustrative and permissive example, with respect to example flexible robotic devices or systems (such as with respect to Figure 1 , Figure 2A and Figure 2BThe described robotically controlled catheter) describes aspects of the disclosed technology. In various embodiments, a robotically controlled catheter can include sensors for identifying the orientation and shape of the catheter, such as fiber optic sensors for shape sensing and localization. Sensors such as fiber optic sensors can provide real-time localization data for navigation and / or for closed-loop control during articulation of the catheter. As described with respect to Figure 4 , Figure 5A , Figure 5B and Figure 6 the disclosed techniques for testing and calibration can be implemented to provide a number of advantages, including ensuring proper alignment of sensors such as fiber optic sensors and stable control loops.

[0021] Figure 1 FIG. is a simplified diagram of a robotic medical system 100 according to some embodiments. In some embodiments, the medical system 100 can be adapted for use in, for example, surgical procedures, diagnostic procedures, therapeutic procedures, or biopsy procedures. Although some embodiments are provided herein with respect to such procedures, any reference to medical or surgical instruments and medical or surgical methods is non-limiting. The systems, instruments, and methods described herein can be used in robotic systems for animals, human cadavers, animal cadavers, parts of human or animal anatomies, non-surgical diagnosis, and industrial systems, and / or other general robotic systems.

[0022] As Figure 1 shown, the medical system 100 can include a manipulator assembly 102 for operating a medical instrument 104 during performance of various procedures on a patient P. The medical instrument 104 can extend through an opening in the patient P into an internal site within the patient P. The manipulator assembly 102 can be a remote operation assembly, a non-remote operation assembly, or a hybrid remote operation and non-remote operation assembly having selectable degrees of freedom of movement that can be motorized and / or remotely operated and selectable degrees of freedom of movement that can be non-motorized and / or non-remotely operated. The manipulator assembly 102 can be mounted to the operating table T and / or located near the operating table T. The master assembly 106 allows an operator O (e.g., a surgeon, clinician, or physician as Figure 1 shown) to observe the intervention site and control the manipulator assembly 102.

[0023] The main component 106 can be located at an operator console, which is typically in the same room as the operating table T, such as on the side of the operating table on which the patient P lies. However, it should be understood that the operator O can be in a different room from the patient P or in a completely different building. The main component 106 generally includes one or more control devices for controlling the manipulator assembly 102. The control devices can include any number of various input devices, such as joysticks, trackballs, rollers, steering wheels, buttons, data gloves, trigger guns, manual controllers, voice recognition devices, human motion or presence sensors, and / or the like.

[0024] The manipulator assembly 102 supports the medical device 104 and can include one or more non-servo-controlled linkages (e.g., one or more linkages that can be manually positioned and locked in place, commonly referred to as setting structures), one or more servo-controlled linkages (e.g., one or more linkages that can be controlled in response to commands from a control system), and / or the kinematic structure of the manipulator. The manipulator assembly 102 can include a plurality of actuators or motors that drive inputs to the medical device 104 in response to commands from a control system (e.g., control system 112). The actuators can include a drive system that, when coupled to the medical device 104, can advance the medical device 104 into a natural or surgically created body orifice. Other drive systems can move the distal portion of the medical device 104 in multiple degrees of freedom, which can include three linear motions (e.g., linear motions along the X, Y, and Z Cartesian axes) and three rotational motions (e.g., rotations about the X, Y, and Z Cartesian axes). Additionally, the actuators can be used to actuate an articulating end effector of the medical device 104 to grasp tissue in the jaws of a biopsy device and / or the like.

[0025] The medical system 100 can include a sensor system 108 having one or more subsystems for receiving information regarding the manipulator assembly 102 and / or the medical device 104. Such subsystems can include an orientation / position sensor system (e.g., an electromagnetic (EM) sensor system); a shape sensor system for determining the orientation, orientation, velocity, rate, pose, and / or shape of the distal portion and / or along one or more segments of a flexible body that can form the medical device 104; a visualization system for capturing images from the distal portion of the medical device 104; and / or an actuator orientation sensor (such as a resolver, encoder, potentiometer, and the like) that describes the rotation and orientation of the motors that control the medical device 104.

[0026] The medical system 100 can include a display system 110 for displaying images or representations of the surgical site and the medical device 104. In some examples, the display system 110 can use imaging modalities to present pre-operative or intra-operative images of the surgical site, such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermography, impedance imaging, laser imaging, nanotube x-ray imaging, and / or the like. In some embodiments, the medical device 104 can include a visualization system that includes an image capture component for recording immediate or real-time images of the surgical site and providing the images to the operator O via one or more displays of the display system 110.

[0027] In some examples, the medical system 100 can configure the controls of the displayed representation, the medical device 104, and the master component 106 such that the relative orientation of the medical device is similar to the relative orientation of the operator O's eyes and / or hands. In this way, the operator O can manipulate the medical device 104 and the hand controls as if viewing the workspace in a substantially true presence.

[0028] In some examples, such as for the purpose of image-guided medical procedures, the display system 110 can display virtual navigation images in which the actual position of the medical device 104 is registered (e.g., dynamically referenced) with the pre-operative or immediate image / model. This can be done from the perspective of the medical device 104 to present a virtual image of the internal surgical site to the operator O.

[0029] The medical system 100 can also include a control system 112. The control system 112 includes at least one memory and at least one computer processor (not shown) for implementing control between the manipulator assembly 102, the medical device 104, the master component 106, the sensor system 108, and / or the display system 110. The control system 112 also includes programmed instructions (e.g., a non-transitory machine-readable medium storing the instructions) to implement some or all of the methods described in accordance with the aspects disclosed herein, including instructions for providing information to the display system 110. Although the control system 112 is shown as a single block in the Figure 1 simplified diagram, the system can include two or more data processing circuits, where a portion of the processing is optionally performed on or adjacent to the manipulator assembly 102, and another portion of the processing is performed at the master component 106 and / or the like. The processor of the control system 112 can execute instructions corresponding to the processes disclosed herein and described in more detail below.

[0030] In some examples, the control system 112 can receive force and / or torque feedback from the medical device 104. In response to the feedback, the control system 112 can transmit a signal to the master component 106. In some examples, the control system 112 can transmit a signal indicating that one or more actuators of the manipulator component 102 move the medical device 104.

[0031] The control system 112 can obtain sensor data from the sensor system 108, and the sensor data is used to calculate the approximate position of the medical device 104 relative to the anatomy of the patient P. The system can implement the sensor system 108 to register and display the medical device together with pre-operative or intra-operative recorded medical images. For example, the PCT publication WO2016 / 191298 (published on December 1, 2016 and titled "Systems and Methods of Registration for Image Guided Surgery") incorporated herein by reference in its entirety discloses exemplary systems.

[0032] The medical system 100 can further include operating and support systems, such as a lighting system, an articulation (such as steering) control system, a flushing system, and / or a suction system (not shown). In some embodiments, the medical system 100 can include more than one manipulator component and / or more than one master component. Among other factors, the exact number of manipulator components can depend on the medical procedure and the space limitations in the operating room. The master components 106 can be juxtaposed, or they can be located in separate positions. Multiple master components can allow more than one operator to control one or more manipulator components in various combinations.

[0033] Figure 2A and Figure 2B is a simplified side view of a patient coordinate space including a medical device mounted on an insertion component according to some embodiments. As Figure 2A and Figure 2B shown, the surgical environment 300 can include a patient P positioned on an operating table T. The patient P can be stationary within the surgical environment 300 in the sense that the overall movement of the patient is restricted by sedation, restraint, and / or other means. Cyclic anatomical movements including the patient P's breathing and cardiac movements can continue. Within the surgical environment 300, the medical device 304 is used to perform a medical procedure, which can include, for example, a surgical, biopsy, ablation, lighting, flushing, suction, or system registration procedure. The medical device 104 can be, for example, the instrument 104. The instrument 304 includes a flexible elongate device 310 (e.g., a catheter) coupled to an instrument body 312. The elongate device 310 includes one or more passageways (not shown) sized and shaped to receive a medical tool (not shown).

[0034] The elongate device 310 may also include one or more sensors (e.g., components of the sensor system 108). In some examples, an articulated sensor 314, such as an optical fiber shape sensor, may be secured at a proximal point 316 on the instrument body 312. The proximal point 316 of the articulated sensor 314 may move with the instrument body 312, and the position of the proximal point 316 (e.g., via a tracking sensor or other tracking device) is known. The articulated sensor 314 may measure the shape from the proximal point 316 to another point, such as the distal portion 318 of the elongate device 310. The articulated sensor 314 may be aligned with the flexible elongate device 310 (e.g., provided within an internal passageway (not shown) or mounted externally). In some examples, the optical fiber may have a diameter of approximately 200 μm. In other examples, the diameter may be larger or smaller. The articulated sensor 314 may be used to determine the shape of the flexible elongate device 310. Optical fibers including fiber Bragg gratings (FBGs) may be used to provide strain measurements in a structure in one or more dimensions. Various systems and methods for three-dimensional monitoring of the shape and relative orientation of optical fibers are described in: U.S. Patent Application No. 11 / 180,389, filed July 13, 2005 and titled "Fiber optic position and shape sensing device and method relating thereto"; U.S. Patent Application No. 12 / 047,056, filed July 16, 2004 and titled "(Optical fiber shape and relative orientation sensing)"; and U.S. Patent No. 6,389,187, filed June 17, 1998 and titled "Optical Fibre Bend Sensor", which are incorporated herein by reference in their entirety. The sensors in some embodiments may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. Various systems for registering and displaying surgical instruments with surgical images using fiber optic sensors are provided in PCT Publication WO 2016 / 191298, published December 1, 2016 and titled "Systems and Methods of Registration for Image Guided Surgery", which PCT publication is incorporated herein by reference in its entirety.

[0035] In some examples, an orientation sensor, such as an electromagnetic (EM) sensor, can be incorporated into the medical device 304. A series of orientation sensors can be positioned along the flexible elongate device 310 and used for shape sensing. In some examples, the orientation sensors can be configured and positioned to measure six degrees of freedom, such as three orientation coordinates X, Y, Z and three orientation angles indicating pitch, yaw, and roll about a base point. In some examples, the orientation sensors can be configured and positioned to measure five degrees of freedom, such as three orientation coordinates X, Y, Z and two orientation angles indicating pitch and yaw about a base point. A further description of the orientation sensor system is provided in U.S. Patent No. 6,380,732 (filed Aug. 11, 1999 and titled “Six-Degree of Freedom Tracking System Having a Passive Transponder on the Object Being Tracked ( Six-Degree of Freedom Tracking System with a Passive Transponder on the Object Being Tracked”), which is incorporated herein by reference in its entirety.

[0036] The elongate device 310 can house a cable, linkage, or other steering control (not shown) that extends between the instrument body 312 and the distal portion 318 to controllably bend the distal portion 318. In some examples, at least four cables are used to provide “up and down” steering that independently controls the pitch of the distal portion 318 and “left and right” steering that controls the yaw of the distal portion 318. The steerable elongate device is described in detail in U.S. Patent Application No. 13 / 274,208 (filed Oct. 14, 2011) (disclosing “Catheter with Removable Vision Probe”), which is incorporated herein by reference in its entirety. The instrument body 312 can include a drive input device that is removably coupled to and receives power from a drive element (such as an actuator) of the assembly.

[0037] The instrument body 312 can be coupled to the instrument carriage 306. The instrument carriage 306 can be mounted to an insertion stage 308 fixed within the surgical environment 300. Alternatively, the insertion stage 308 can be movable, but have a known position within the surgical environment 300 (e.g., via a tracking sensor or other tracking device). The instrument carriage 306 can be part of a manipulator assembly (e.g., manipulator assembly 102) that is coupled to the medical device 304 to control the insertion movement (e.g., movement along axis A) and / or the movement of the distal portion 318 of the elongate device 310 in multiple directions such as yaw, pitch, and / or roll. The instrument carriage 306 or the insertion stage 308 can include an actuator, such as a servo motor (not shown), that controls the movement of the instrument carriage 306 along the insertion stage 308.

[0038] A sensor device 320, which can be part of the sensor system 108, can provide information about the orientation of the instrument body 312 as it moves along the insertion axis A on the insertion stage 308. The sensor device 320 can include one or more resolvers, encoders, potentiometers, and / or other sensors that determine the rotation and / or orientation of the actuator that controls the movement of the instrument carriage 306 and thus the movement of the instrument body 312. In some embodiments, the insertion stage 308 is linear. In some embodiments, the insertion stage 308 can be curved or have a combination of curved and linear segments.

[0039] Figure 2A The instrument body 312 and the instrument carriage 306 are shown in a retracted orientation along the insertion stage 308. In this retracted orientation, the proximal point 316 is at the orientation L0 on the axis A. In Figure 2B , the instrument body 312 and the instrument carriage 306 are advanced along the linear track of the insertion stage 308, and the distal portion 318 of the elongate device 310 has been advanced into the patient P. In this advanced orientation, the proximal point 310 is at the orientation L1 on the axis A. In some examples, encoder and / or other orientation data from one or more actuators that control the movement of the instrument carriage 306 along the insertion stage 308 and / or one or more orientation sensors associated with the instrument carriage 306 and / or the insertion stage 308 can be used to determine the orientation of the proximal point 316 relative to the orientation L0. In some examples, this orientation can further be used as an indicator of the distance or insertion depth at which the distal portion 318 of the elongate device 310 is inserted into a passage in the anatomy of the patient P.

[0040] Figure 3 is a simplified diagram of a robotic system 350 according to some embodiments. The system 350 includes a flexible elongate device 310 and an instrument body 312 releasably coupled to the instrument carriage 306, each of which is substantially similar to Figure 2A and 2BThose described in

[0041] The flexible elongate device 310 may have an articulable portion (e.g., distal portion 326) that can articulate (e.g., steer) with one or more degrees of freedom to guide the elongate device 310 through a branched channel. For this purpose, the flexible elongate device 310 may include a plurality of articulation controls (e.g., cables, linkages, pull wires, tendons, or other articulation controls) that extend from the articulable portion through the flexible elongate device 310 and drive an input device (e.g., input disk) of the instrument body 312.

[0042] The instrument carriage 306 may include a drive unit 322 having a drive output. Coupling the flexible elongate device 310 and the instrument body 312 to the instrument carriage 306 may include coupling the drive output of the drive unit 322 to drive the input device of the instrument body 312. This may establish a drive connection between a set of actuators 324 in the drive unit 322 and the articulable portion of the flexible elongate device 310. The actuators may use the drive connection to articulate the articulable portion (e.g., distal portion 326) of the flexible elongate device 310. In various embodiments, the actuators 324 may include servo motors, rotary actuators, linear actuators, and / or other actuation mechanisms that apply a force to the articulation controls and thereby move the flexible elongate device 310. Suitable drive units 322 and articulation controls are described in detail in U.S. Patent No. 7,316,681 (filed Oct. 4, 2005, titled "Articulated Surgical Instrument for Performing Minimally Invasive Surgery with Enhanced Dexterity and Sensitivity") and U.S. Patent No. 9,259,274 (filed Sep. 30, 2008, titled "Passive Preload and Capstan Drive for Surgical Instruments"), the entire contents of which are incorporated herein by reference.

[0043] The drive unit 322 may include one or more drive sensors 328 coupled to the actuator 324 to measure articulation in a degree of freedom based on the movement, force, and / or other characteristics of the actuator. In some embodiments, the drive sensor 328 may include an orientation resolver, an encoder, a potentiometer, and / or other suitable sensor coupled to the actuator 324 to measure and report the movement of the corresponding actuator 324. In some embodiments, the drive sensor 328 may include a force sensor, a strain sensor, and / or a torque sensor coupled to measure and report the amount of force applied by the corresponding actuator 324. These measurements and other measurements can be used to determine the degree of articulation, which can be characterized as a bend radius, an angle of orientation of the articulated portion relative to the non-articulated portion of the slender device 310 (e.g., a joint angle), or other suitable characteristics.

[0044] Data from the drive sensor 328 and data from sensors associated with the flexible elongated device 310 can be provided to a tracking system 330 of the system 350. In some embodiments, the tracking system 330 can be part of the control system 112 described above. The tracking system 330 can use the received data to determine the position, orientation, speed, velocity, posture, and / or shape of the distal portion 326 and / or one or more segments along the flexible elongated device 310. For example, the tracking system 330 can determine the shape of the flexible elongated device 310 based on data from the drive sensor 328 and / or data from the articulation sensor 314 (e.g., one or more shape sensors or position sensors, such as fiber optic sensors, electromagnetic position sensors, etc.) disposed within the flexible elongated device 310. The tracking system 330 can also take into account historical posture data to reconstruct the shape of the flexible elongated device 310 over a certain time interval.

[0045] Tracking system 330 may optionally be implemented as hardware, firmware, software, or a combination thereof interacting with or otherwise executed by one or more computer processors, which may include Figure 1 The tracking system 330 may include a processor of the control system 112. For example, the tracking system 330 may include one or more processors and non-transitory computer memory storing machine-executable instructions that, when executed by the one or more processors, cause the tracking system 330 to perform the various operations described herein.

[0046] Tracking system 330 and drive unit 322 may be used to perform a set of calibration tests of system 350, some of which will reference Figure 4 , Figure 5A and Figure 5BA description will be given. For example, before performing a procedure using the flexible elongate device 310 or other articulating elements of the system 350, the system 350 may verify the proper articulation of the flexible elongate device 310, verify the feedback loop controlling the articulation of the flexible elongate device 310, calibrate the sensors of the system 350, and / or evaluate other characteristics of the system 350 and / or the flexible elongate device 310 that may affect the ability to reliably manipulate the flexible elongate device 310. Thus, in some embodiments, one or more test procedures may be performed, including articulating the elongate device 310 in one or more degrees of freedom and collecting sensor data for these and other purposes.

[0047] Figure 4 is a flowchart depicting an exemplary method 400 for comparing drive sensor data with articulation sensor data according to some embodiments. For example, method 400 may be used to calibrate the system 350. Method 400 is shown as a set of operations or procedures in Figure 4 as shown. Figure 4 The processes shown in Figure 4 may be performed in an order different from the order shown, and one or more of the shown processes may not be performed in some embodiments of method 400. Additionally, Figure 4 one or more processes not explicitly shown in Figure 5A and Figure 5B may be included before, after, between, or as part of the shown processes. In some embodiments, one or more processes of method 400 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium that, when run by one or more processors (e.g., the processors of the tracking system 330 and / or the control system 112), may cause the one or more processors to perform one or more processes. Figure 5A and Figure 5B are diagrams showing the relationships between sets of sensor data during method 400 for testing and / or calibrating the system 350 according to some embodiments.

[0048] Referring to Figure 4In process 402, the robotic system 350 can provide user instructions for installing the flexible elongate device 310 on the system 350. In some examples, the flexible elongate device 310 and the coupled instrument body 312 can be provided separately from the remainder of the system 350, and process 402 can provide instructions to the operator O to couple the flexible elongate device 310 and the instrument body 312 to the instrument carriage 306 of the system 350. This can include connecting the drive input device of the instrument body 312 - and thus the articulation control of the flexible elongate device 310 - to the actuator 324 of the drive unit 322 of the instrument carriage 306. The instructions can include visual and / or audible instructions and can be presented via the display system 110, via indicators on the instrument carriage 306, and / or other suitable means.

[0049] Reference Figure 4 In process 404, the tracking system 330 of the system 350 may receive an indication that the flexible elongate device 310 has been installed. For example, the indication can represent that the flexible elongate device 310 and the instrument body 312 have been coupled to the instrument carriage 306 of the system 350. Reference Figure 4 In process 406, after the flexible elongate device 310 has been installed, the system 350 can begin one or more tests, such as an assembly test, to verify the installation and proper operation of the flexible elongate device 310.

[0050] Referring to process 408, the tracking system 330 can perform an assembly test by commanding the flexible elongate device 310 to articulate in at least one degree of freedom. For example, the tracking system 330 can command the articulation by activating the actuator 324 associated with the degree of freedom. The actuator 324 can rotate a capstan within the instrument body 312, which rotates to wind a cable of the flexible elongate device 310 around the capstan, and thus effect movement of a portion of the flexible elongate device 310 (e.g., the distal portion 326).

[0051] The tracking system 330 may command the articulation of the flexible elongate device 310 in any suitable mode. For example, the tracking system 330 may first command the flexible elongate device 310 to move from a neutral position to a first point in a first direction along one or more degrees of freedom being utilized. The tracking system 330 may determine that the flexible elongate device 310 has reached the first point based on any suitable sensor data, any suitable sensor data including sensor data obtained during processes 406 and / or 408, which will be described in more detail below. The first point may be a physical limit of the flexible elongate device 310 in the degree of freedom, or the first point may be less than the physical limit of the flexible elongate device 310 (e.g., 50% of the physical limit) to avoid overextension and / or because the sensors may not yet be fully calibrated. After the flexible elongate device 310 has reached the first point, the tracking system 330 may command the flexible elongate device 310 to move from the first point, through the neutral position, to a second point in a second direction opposite the first direction along the degree of freedom. Similar to the first point, the second point may be any point up to the physical limit of the flexible elongate device 310 in the second direction. Any number of cycles may be repeated.

[0052] Referring to process 410, the tracking system 330 may generate a set of drive sensor data in response to the commanded articulation of process 408. The drive sensor data may be obtained from the drive sensors 328 of the drive unit 322 and / or other sensors of the tracking system 330. The drive sensor data may include orientation data of the actuator 324 and / or may include force data, strain data, torque data, and / or other data representative of the relationship between the actuator 324 and the flexible elongate device 310. The drive sensor data may be associated with movement in at least one degree of freedom. In some examples, the drive sensor data may be associated with movement in a single degree of freedom, and the test may be repeated in order to obtain sensor data in other degrees of freedom. In some examples, method 400 may verify operation in more than one degree of freedom simultaneously, and the drive sensor data may be associated with movement in two or more degrees of freedom.

[0053] Referring to process 412, the tracking system 330 can generate a set of articulation sensor data in response to the commanded articulation of process 408. The articulation sensor data can be obtained from sensors of the flexible elongate device 310, which sensors are such as articulation sensors 314 (e.g., fiber optic shape sensors, EM position sensors, etc.) and / or other suitable sensors. Like the drive sensor data, the articulation sensor data can be associated with motion in at least one degree of freedom. In some examples, the articulation sensor data can be associated with motion in a single degree of freedom, and the test can be repeated to obtain sensor data in other degrees of freedom. In some examples, method 400 can verify operation in more than one degree of freedom simultaneously, and the articulation sensor data can be associated with motion in two or more degrees of freedom.

[0054] Referring to process 414, the tracking system 330 can compare a set of drive sensor data obtained in process 410 with a set of articulation sensor data obtained in process 412 to generate a test curve. Since the drive sensor data and the articulation sensor data can represent different characteristics, the tracking system 330 can convert one or both of them to a common measurement. For example, the tracking system 330 can convert the drive sensor data (e.g., actuator orientation data) into a determination of the degree of articulation in a given degree of freedom, which can be characterized as the distal bend radius, the angle of the articulated portion of the elongate device 310 relative to the non-articulated portion of the elongate device 310 (e.g., Figure 3 the joint angle) and / or other suitable determinations. The tracking system 330 can also convert the articulation sensor data into a corresponding determination of the degree of articulation, such as the distal bend radius, the angle of the articulated portion, etc.

[0055] Figure 5A Graph 500 of FIG. 5 shows a comparison between a set of example drive sensor data (e.g., actuator orientation data) that has been converted into a determination of the degree of articulation (e.g., joint angle) plotted along axis 504 and a set of example articulation sensor data that has been converted into a corresponding determination of the degree of articulation (e.g., joint angle) plotted along axis 506. Due to manufacturing tolerances, installation issues, maintenance issues, calibration issues, and / or other reasons, the drive sensor data can be different from the articulation sensor data. Graph 502 of FIG. 5 shows this difference in another way and shows the arithmetic difference or other difference determination between the drive sensor data and the articulation sensor data plotted along axis 508, and the degree of articulation in the drive sensor data or the articulation sensor data plotted along axis 510.

[0056] Based on the test curve, it can be determined whether the system 350 and / or the flexible elongate device 310 is within tolerance, and if not, what corrective action to take. To this end, refer toFigure 4 In process 416, the tracking system 330 can determine whether one or more aspects of the test curve meet one or more metrics in the reference curve, such as thresholds. The reference curve can include a set of predetermined thresholds for one or more metrics, and the thresholds can be determined during manufacturing via a calibration procedure, via analytical considerations based on the design of the device 310, and / or via the characterization of statistically relevant samples over the entire expected life of the elongate device. The reference curve can be stored in the memory of the device 310 (e.g., non-volatile memory) and / or the memory of the system 350.

[0057] In some examples, the aspect can include an offset of the test curve, and the metric can include an offset threshold of the reference curve. Refer Figure 5A and Figure 5B For the markers 512 and 514 in, when the drive sensor data indicates that the actuator is not articulated or there is no force applied by the actuator (e.g., neutral position), the tracking system 330 can measure the neutral position offset based on the amount of articulation measured by the articulation sensor data. The tracking system 330 can then determine whether the articulation in the articulation sensor data is within a predetermined threshold of the reference curve (e.g., 0 ± 5°, 0 ± 10°, etc.).

[0058] In some examples, the aspect can include a slope, and the metric can include a slope threshold. For example, in process 416, the tracking system 330 can determine a slope fit between the amount of articulation measured by the drive sensor data and the amount of articulation measured by the articulation sensor data. The tracking system 330 can determine whether the slope is within a predetermined threshold of the reference curve (e.g., 1.0 ± 10%).

[0059] In some examples, the aspect can include a maximum difference, and the metric can include an appropriate threshold. For example, in process 416, the tracking system 330 can determine whether the maximum difference between the amount of articulation measured by the drive sensor data and the amount of articulation measured by the articulation sensor data is within a predetermined threshold of the reference curve (e.g., ±5°, ±10°, etc.).

[0060] In some examples, the aspect can include sensor hysteresis, and the metric can include a hysteresis threshold. For example, in process 416, the tracking system 330 can determine the hysteresis in the drive sensor data and the articulation sensor data by comparing the amount of articulation measured by the drive sensor data and / or the articulation sensor data when the elongate device 310 travels in a first direction with the amount of articulation measured by the drive sensor data and / or the articulation sensor data when the elongate device 310 travels in a second direction. It can be based on Figure 5A and Figure 5BAny set of data indicated by markers 516 and 518 is used to measure hysteresis. The tracking system 330 can determine whether the hysteresis is within a predetermined threshold of the reference curve (e.g., an increment of 5°, 10°, etc.).

[0061] In some examples, this aspect can include linearity, and the metric can include a linearity threshold. For example, in process 416, the tracking system 330 can determine the determination of the linearity of the amount of articulation measured by the drive sensor data and the amount of articulation measured by the articulation sensor data. The determination can include any suitable linear metric, such as the mean of the residuals, the standard deviation of the residuals, the maximum or minimum value of the residuals, the distribution of the residuals via a chi-square test (e.g., Pearson chi-square test), and / or other suitable metrics. The tracking system 330 can determine whether the linearity determination is within a predetermined threshold of the reference curve.

[0062] In some examples, this aspect can include noise in the drive sensor data and / or the articulation sensor data, and the metric can include a noise threshold. For example, in process 416, the tracking system 330 can determine whether the noise in the amount of articulation measured by the drive sensor data and / or the amount of articulation measured by the articulation sensor data is within a predetermined threshold of the reference curve. In an example, the tracking system 330 can compare the maximum absolute value of the residuals of the drive sensor data and the articulation sensor data with a threshold. When making the comparison, the maximum absolute value can be multiplied by a margin coefficient based on the sampling confidence. Additionally or alternatively, the tracking system 330 can compare the standard deviation of the residuals and / or the variance of the residuals with the corresponding threshold.

[0063] In some examples, in process 416, the flexible elongate device 310 can be mounted in more than one orientation, or elements of the flexible elongate device 310 (such as the articulation sensor 314) can be mounted in more than one orientation during manufacturing. For example, the articulation sensor 314 can include four optical fiber bundles that are arranged such that the sensor can be mounted in the elongate device 310 in one of four rotational orientations (e.g., 0°, 90°, 180°, or -90°) relative to the rest of the device 310. Thus, in such embodiments, the tracking system 330 determines from the test curve how the drive sensor data accommodates the articulation sensor data and / or how the corresponding sensors accommodate the degree(s) of freedom.

[0064] Referring to process 418, the tracking system 330 can determine whether a corrective action is needed based on a comparison of processes 414 and / or 416. For example, the results of processes 414 and / or 416 can be used to adjust the calibration of the respective sensors to adjust the feedback loop used to articulate the flexible elongate device 310 into a particular pose, and / or diagnose and correct a number of problems with the flexible elongate device 310 and / or the system 350. These problems can include installation problems, drive train failures (such as derailment, cable breakage, binding, stretching, excessive compliance or friction), and / or other problems. If the test curve fails to meet any of the above thresholds or other metrics in the reference curve, the operator can be instructed to repair, reinstall, or replace the flexible elongate device 310 and / or another component of the system 350. In contrast, when the test curve meets the metrics of the reference curve, as shown in process 420, method 400 can proceed with further calibration or can use the system 350 for a medical procedure. In some examples, the further calibration can include repeating one or more of processes 406-418 for additional (e.g., untested) degrees of freedom.

[0065] Reference Figure 6 An example of an additional calibration test is described, Figure 6 is a flowchart of a method 600 for comparing pose data with a commanded pose according to some embodiments. For example, method 600 can be used to perform a bend strain test on the system 350. Figure 6 The processes shown in Figure 6 can be performed in an order different from the order shown in Figure 6 and one or more of the shown processes may not be performed in some embodiments of method 600. Additionally,

[0066] Referring to process 602, the tracking system 330 of system 350 may command the flexible elongate device 310 to articulate via a plurality of pose commands. The tracking system 330 may command the articulation by activating the actuators 324 of the drive unit 322. For example, the tracking system 330 may activate more than one of the actuators 324 to achieve a particular pose as compared to activating each actuator 324 individually. The tracking system 330 may utilize a feedback loop to activate the actuator(s) 324 until the tracking system 330 determines that the flexible elongate device 310 has achieved the pose or until another limit (e.g., maximum operating tension) is met. The tracking system 330 may determine that the flexible elongate device 310 has reached the pose based on any suitable sensor data, including the sensor data obtained during processes 604 and / or 606, which will be described in more detail below. In some examples, during process 602 of method 600, system 350 may utilize drive sensor data from moving the flexible elongate device 310 during process 408 of method 400 and, after the articulation sensors (e.g., shape sensors) have been appropriately calibrated in method 400, may utilize the articulation sensor data to determine when the flexible elongate device 310 has reached a particular pose.

[0067] The plurality of poses in process 602 may include any suitable pose. In some examples, system 350 may move the flexible elongate device 310 to an orientation less than the physical limits of the flexible elongate device 310 (such as the physical limits during process 408 of method 400 in some scenarios). After the sensors have been appropriately calibrated (e.g., in method 400), the tracking system 330 may command the articulation of the flexible elongate device 310 to a pose at or near the physical limits (e.g., during process 602 of method 600). Because the sensors used to control the articulation (e.g., the articulation sensor 314) have been appropriately calibrated and verified, the risk of overextension of the elongate device 310 may be reduced.

[0068] As described above, a given pose can include movement in more than one degree of freedom. This can be exploited to detect problems that occur when the system 350 and / or the flexible elongate device 310 are in motion involving more than one degree of freedom. It can also test the full range of motion with fewer poses compared to testing each degree of freedom individually. For example, the tracking system 330 can command the flexible elongate device 310 to articulate from a neutral position to a first pose in a first direction in a first degree of freedom without articulation in a second degree of freedom. The first pose can be at a physical limit in the first direction. The tracking system 330 can then command the flexible elongate device 310 to articulate back to the neutral position. The tracking system 330 can then command the flexible elongate device 310 to articulate from the neutral position to a second pose in a first direction in a first degree of freedom and in a third direction in a second degree of freedom. The second pose can be at a physical limit in the second direction and / or at a physical limit in the third direction. For some flexible elongate devices 310, the physical limit in a given degree of freedom can vary depending on the amount of articulation in another degree of freedom. For example, although the first pose can have joint angles of (-180°, 0), the second pose can have joint angles of (127°, 127°). Continuing with this example, the tracking system 330 can then command the flexible elongate device 310 to articulate from the second pose back to the neutral position. The tracking system 330 can command the flexible elongate device 310 to articulate from the neutral position to a third pose in a second direction in a first degree of freedom and in a fourth direction in a second degree of freedom (e.g., a pose angle of (127°, -127°)). The third pose can be at a physical limit in the second direction and / or at a physical limit in the fourth direction. In this way, the full range of motion in two degrees of freedom can be comprehensively tested with only three poses. Of course, other examples utilize any number of poses.

[0069] Referring to process 604, the tracking system 330 can generate a set of articulation sensor data in response to the commanded articulation of process 602. This can be performed substantially as described above, and the articulation sensor data can be obtained from sensors of the flexible elongate device 310, such as the articulation sensors 314 (e.g., fiber optic sensors, EM orientation sensors, and / or other suitable sensors).

[0070] Referring to process 606, the tracking system 330 can generate a set of drive sensor data in response to the commanded articulation of process 602. The drive sensor data can be obtained from the drive sensors 328 of the drive unit 322 and / or other sensors of the system 350. The drive sensor data can include orientation data of the actuator 324 and / or can include force data, strain data, torque data, and / or other data representative of the relationship between the actuator 324 and the flexible elongate device 310.

[0071] Referring to process 608, the tracking system 330 can compare the pose data from a set of articulation sensor data obtained in process 604 and / or a set of drive sensor data obtained in process 606 with the (one or more) commanded poses. The tracking system 330 can consider any suitable aspects of the pose data, including those described above in the context of process 416. In some examples, the tracking system 330 can determine whether the offset of any measurement of the pose data at the neutral position or any commanded pose exceeds a corresponding threshold. In some examples, the tracking system 330 can determine whether the rate of change (e.g., slope) of the pose data during traveling to a given pose is within the corresponding threshold. In some examples, the tracking system 330 can determine whether the hysteresis of the pose data meets the corresponding threshold. In some examples, the tracking system 330 can determine whether the noise measurement (such as RMS noise) in the pose data meets the corresponding threshold.

[0072] In some examples, the tracking system 330 can consider the continuity of the pose data during traveling to each pose. In this regard, when the flexible elongate device 310 flexes, certain types of sensors (such as the fiber optic shape sensor 314) may be prone to breakage, separation, or other discontinuities. This type of error can be detected from the pose data continuity.

[0073] Method 600 can be used to evaluate the calibration of the sensors of system 350. The results of process 608 can be used to detect many problems with the flexible elongate device 310 and / or system 350, including installation problems, drive system failures (such as derailment, cable breakage, binding, stretching, excessive compliance, or friction), and / or others. If the test curve does not meet certain metrics, the flexible elongate device 310 can be repaired, reinstalled, or replaced. In contrast, if process 608 indicates that system 350 and the flexible elongate device 310 have passed the verification, the results can be used to maintain or adjust the calibration of the corresponding sensors and / or maintain or adjust the feedback loop used to articulate the flexible elongate device 310 into a specific pose. Then, method 600 can use system 350 for further calibration or a medical procedure can be performed, as shown in process 610.

[0074] Various examples of implementations of the present disclosure

[0075] The present invention describes various instruments and parts of instruments based on their states in three-dimensional space. For example, the term orientation refers to the position of an object or a part of an object in three-dimensional space (e.g., three translational degrees of freedom along the Cartesian x, y, and z coordinates). The term orientation refers to the rotational placement of an object or a part of an object (e.g., one or more rotational degrees of freedom, such as roll, pitch, and yaw). The term pose refers to the orientation of an object or a part of an object in at least one translational degree of freedom, or to the orientation of an object or a part of an object in at least one rotational degree of freedom (e.g., up to six total degrees of freedom). The term shape refers to a set of poses, orientations, or orientations measured along an object.

[0076] One or more elements in embodiments of the present disclosure may be implemented in software to be executed on a processor of a computer system such as a control processing system. When implemented in software, the elements of embodiments of the present disclosure may be code segments for performing various tasks. The program segments or code segments may be stored in a processor-readable storage medium or device, which may be downloaded via a computer data signal embodied in a carrier wave over a transmission medium or communication link. The processor-readable storage device may include any medium that can store information, including optical media, semiconductor media, and / or magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor storage devices, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM); floppy disks, CD-ROMs, optical discs, hard disks, or other storage devices. The code segments may be downloaded via a computer network such as the Internet, an intranet, etc. Any of a variety of centralized or distributed data processing architectures may be employed. The programming instructions may be implemented as multiple separate programs or subroutines, or they may be integrated into many other aspects of the systems described herein. In some examples, the control system may support wireless communication protocols such as Bluetooth, Infrared Data Association (IrDA), HomeRF, IEEE 802.11, Digital Enhanced Cordless Telecommunications (DECT), Ultra Wideband (UWB), ZigBee (Internet of Things), and wireless telemetry.

[0077] Medical tools can be delivered through the flexible elongate devices (e.g., catheters) disclosed herein and can include, for example, image capture probes, biopsy instruments, laser ablation fibers, and / or other surgical, diagnostic, or therapeutic tools. The medical tools can include an end effector having a single working member, such as a scalpel, a blunt blade, an optical fiber, an electrode, and / or the like. Other end effectors can include, for example, forceps, graspers, scissors, clip appliers, and / or the like. Other end effectors can further include electrically activated end effectors, such as electrosurgical electrodes, transducers, sensors, and / or the like. The medical tools can include an image capture probe that includes a stereo or monoscopic camera for capturing images (including video images). The medical tools can additionally accommodate a cable, linkage, or other actuation control device that extends between its proximal and distal portions to controllably bend the distal portion of the medical tool. Steerable instruments are described in detail in U.S. Patent No. 7,316,681 (filed Oct. 4, 2005, and titled "Articulated Surgical Instrument for Performing Minimally Invasive Surgery with Enhanced Dexterity and Sensitivity") and U.S. Patent Application No. 12 / 286,644 (filed Sep. 30, 2008, and titled "Passive Preload and Capstan Drive for Surgical Instruments"), the entire contents of which are incorporated herein by reference.

[0078] The systems described herein can be adapted to navigate and treat anatomical tissues via natural or surgically created passages in any of a variety of anatomical systems, including the lungs, colon, intestine, kidneys and renal calyces, brain, heart, circulatory system including the vasculature, and / or the like.

[0079] Note that the processes and displays presented may not be inherently related to any particular computer or other device. Various general purpose systems can be used with the routines according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the described operations. Additionally, it will be appreciated that a variety of programming languages can be used to implement the examples described herein.

[0080] Although certain examples have been described and shown in the drawings, it should be understood that these examples are merely illustrative and not restrictive, and the described examples are not limited to the specific constructions and arrangements shown and described, as various other modifications can be made by those of ordinary skill in the art.

Claims

1. A robotic system, comprising: An instrument carriage configured to receive an elongate device, wherein the instrument carriage includes: a set of actuators configured to drive the elongate device along at least one degree of freedom; and a set of drive sensors configured to monitor the set of actuators; and A tracking system coupled to the instrument carriage and configured to: Receive an indication that the elongate device is mounted on the instrument carriage; Operate the set of actuators to articulate a distal portion of the elongate device in at least one degree of freedom; Generate a set of drive sensor data from the set of drive sensors during the articulation; Generate a set of articulation sensor data from a shape sensor of the elongate device during the articulation; Compare the set of drive sensor data with the set of articulation sensor data to generate a test curve; Determine whether the test curve corresponds to a reference curve; and Based on whether the test curve corresponds to the reference curve, determine whether the elongate device requires a corrective action.

2. The robotic system according to claim 1, wherein the robotic system further includes a display system configured to display an indication for an operator to mount the elongate device.

3. The robotic system according to claim 1, wherein the tracking system is further configured to: Determine a first set of joint angles from the set of drive sensor data; Determine a second set of joint angles from the set of articulation sensor data; and Based on a comparison of the first set of joint angles with the second set of joint angles, compare the set of drive sensor data with the set of articulation sensor data to generate the test curve.

4. The robotic system according to claim 1 or 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a neutral position offset of at least one of the set of drive sensor data or the set of articulation sensor data.

5. The robotic system according to claim 1 or 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a slope of at least one of the set of drive sensor data or the set of articulation sensor data.

6. The robotic system according to claim 1 or 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a difference between the set of drive sensor data and the set of articulation sensor data.

7. The robotic system according to claim 1 or 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a linearity of at least one of the set of drive sensor data or the set of articulation sensor data.

8. The robotic system according to claim 1 or claims 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a noise measurement of at least one sensor data from the set of drive sensor data or the set of articulation sensor data.

9. The robotic system according to claim 1 or claims 2 - 3, wherein the tracking system is configured to determine whether the test curve corresponds to the reference curve based on a hysteresis measurement of at least one sensor data from the set of drive sensor data or the set of articulation sensor data.

10. The robotic system according to claim 1 or claims 2 - 3, wherein the tracking system is further configured to operate the set of actuators to articulate the distal portion of the elongate device in at least two degrees of freedom, and wherein the set of drive sensor data and the set of articulation sensor data are associated with movement in each of the at least two degrees of freedom.

Images