System and method for motion control of a steerable device
By introducing a catch-up curve mechanism and precise actuator control into the control system of minimally invasive medical tools, the problem of tension accumulation during operation is solved, fast and accurate response and intuitive control are achieved, and operating efficiency and accuracy are improved.
Patent Information
- Application Number
- CN201980065605.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-10-04
- Filing Date
- 2019-10-01
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2039-10-01
AI Technical Summary
Existing minimally invasive medical tools are difficult to achieve fast, accurate response and intuitive control during operation, especially when encountering opposite forces, which may lead to tension accumulation, affecting operating efficiency and accuracy.
By introducing a catch-up profile mechanism in the control system, the movement of the steering device is adjusted according to user input to reduce orientation differences and tension, and through precise control of the actuator, the tightness of the user's direction of the direction is achieved. Tracking and reverse force management.
The smooth motion control of the steering device in minimally invasive medical operations is realized, which reduces tension changes, improves the response speed and accuracy to user inputs, and enhances the intuitiveness and safety of the operation.
Smart Images

Figure CN112804931B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 741,354, filed October 4, 2018, the entire contents of which are incorporated herein by reference. Technical Field
[0003] Examples described herein relate to systems and methods for procedures, such as systems and methods for controlling a steerable device of a system. Background Art
[0004] Instruments can be used to manipulate and perform tasks in a workspace. Such instruments can be configured to be partially or completely supported and operated by a manipulator assembly. Such instruments and manipulator assemblies can be used to perform non-medical procedures or medical procedures. For example, a medical tool or medical manipulator can be used to perform minimally invasive medical procedures. As another example, an industrial tool or industrial manipulator can be used for manufacturing or testing. As other examples, a tool or manipulator can be used in programs for entertainment, detection, and various other purposes.
[0005] Minimally invasive medical techniques can generally be intended to reduce the amount of tissue damaged during invasive medical procedures, thereby reducing the patient's recovery time, discomfort, and harmful side effects. Such minimally invasive techniques can be performed through natural orifices in the patient's anatomy or through one or more incisions. Through these natural orifices or incisions, clinicians can insert medical tools to reach the target tissue location. Minimally invasive medical tools include instruments such as therapeutic instruments, diagnostic instruments, and surgical instruments. Minimally invasive medical tools can also include imaging instruments (such as endoscopic instruments) that provide users with a view within the patient's anatomy.
[0006] Certain medical and non-medical instruments, including manipulators, imaging or other sensing instruments, etc., may be remotely operated or otherwise computer-assisted. It would be advantageous to provide intuitive controls for operating these instruments and other steerable devices, and for the controls to respond quickly and accurately to user inputs during a procedure. Summary of the invention
[0007] The following presents a simplified summary of various examples described herein and is not intended to identify key or critical elements or to delineate the scope of the claims.
[0008] In one example, a system may include: a control device configured to receive a user input; a manipulator system including an actuator configured to receive and drive a steerable device; and a control system communicatively coupled to the manipulator system and the control device. The control system may be configured to track a virtual user-indicated orientation based on a first user input, determine an orientation of a portion of the steerable device, and determine an orientation difference in a first direction between the determined orientation of the device and the virtual user-indicated orientation. The control system may also be configured to receive a second user input commanding movement of the steerable device in a second direction. At least a portion of the second direction is opposite to the first direction. In response to the second user input, the control system is configured to reduce the orientation difference based on aspects of the second user input and a catch-up profile.
[0009] In another example, a method may include receiving a first user input from a master controller indicating movement of a steerable elongated device in a first direction of motion, and tracking a virtual user-indicated orientation based on the first user input. The method may also include controlling the motion of the steerable elongated device based on the first user input, and detecting a current device orientation of the steerable elongated device from a sensor of the steerable elongated device. The method may also include determining an orientation difference between the detected current device orientation and the virtual user-indicated orientation, and receiving a second user input from the master controller in a second direction of motion. At least a portion of the orientation of the second direction is opposite to the first direction of motion. In response to the second user input, the orientation difference may be reduced based on a catch-up curve.
[0010] In another example, a device may include one or more processors and a non-transitory computer memory storing machine executable instructions. When executed by one or more processors, the machine executable instructions may cause the device to receive a first user input from a control device commanding the movement of a steerable device in a first direction, and track a virtual user indicated orientation based on the first user input. When executed by one or more processors, the machine executable instructions may also cause the device to command an actuator coupled to the steerable device to move the steerable device in a first direction based on the first user input. The movement of the steerable device in response to the first user input may generate an orientation difference. When executed by one or more processors, the machine executable instructions may also cause the device to detect the device orientation of a part of the steerable device from a sensor of the steerable device and determine the determination of the orientation difference. When executed by one or more processors, the machine executable instructions may also cause the device to receive a second user input from a control device commanding the movement of the steerable device in a second direction. At least a portion of the second direction may be opposite to the first direction. In response to the second user input, the orientation difference may be reduced based on the aspect of the second user input and the catch-up curve.
[0011] It should be understood that the foregoing summary and the following detailed description are illustrative and explanatory in nature, and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. In this regard, other aspects, features and advantages of the present disclosure will be apparent to those skilled in the art based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 is a simplified diagram of a medical system according to some embodiments.
[0013] Figure 2A and Figure 2B is a simplified diagram of a side view of a patient coordinate space including a medical device mounted on an insertion assembly according to some embodiments.
[0014] Figure 3 is a flow chart illustrating a method of controlling a steerable device according to some embodiments.
[0015] Figure 4-Figure 6 is a diagram showing the relationship between user input and the orientation of a steerable device according to some embodiments.
[0016] Figure 7 is a flow chart illustrating a method of controlling a steerable device in response to a fault according to some embodiments.
[0017] By referring to the detailed description below, the embodiments of the present disclosure and its advantages will be best understood. It should be understood that the same reference numerals are used to identify similar elements shown in one or more drawings, wherein the display therein is to illustrate the embodiments of the present disclosure, rather than to limit the embodiments of the present disclosure. DETAILED DESCRIPTION
[0018] The technology described herein provides a technique for controlling a steerable device, such as an instrument of a robotic device or system. Examples of applicable robotic devices or systems include medical devices that are remotely operated or otherwise operated, which drive a flexible elongated device into a passage, such as an airway or a blood vessel. As an illustrative and permissible example, with respect to an example flexible robotic device or system, such as with respect to Figure 1 , Figure 2A and Figure 2B Various aspects of the disclosed technology are described herein. In various embodiments, a user instructs a system to use an actuator of the system to drive a robotically controlled catheter. When the forces applied by the actuators are opposed, tension may build up within the catheter. Figure 3- The disclosed technique for controlling a catheter or other instrument described in FIG. 8 may be implemented to control the manner in which tension is generated and released to provide an intuitive control technique having many advantages, including smooth changes in tension and a high degree of responsiveness to user input.
[0019] Figure 1 is a simplified diagram of a robotic medical system 100 according to some embodiments. In some embodiments, the medical system 100 may be suitable for use in, for example, surgical procedures, diagnostic procedures, therapeutic procedures, or biopsy procedures. Although some embodiments are provided herein with respect to such procedures, any reference to medical or surgical instruments and medical or surgical methods is non-limiting. The systems, instruments, and methods described herein may be used for robotic systems for animals, human cadavers, animal cadavers, portions of human or animal anatomy, non-surgical diagnostic and industrial systems, and / or other general robotic systems.
[0020] like Figure 1 As shown in , medical system 100 may include a manipulator assembly 102 for operating a medical device 104 in performing various procedures on a patient P. The medical device 104 may extend through an opening in the patient P and into an internal location within the patient P. Manipulator assembly 102 may be a remotely operated assembly, a non-remotely operated assembly, or a hybrid remotely operated and non-remotely operated assembly having selected degrees of freedom of movement that may be electrically and / or remotely operated and selected degrees of freedom of movement that may be non-electrically and / or non-remotely operated. Manipulator assembly 102 may be mounted to and / or located near an operating table T. Main assembly 106 allows an operator O (e.g., Figure 1 ) observes the intervention site and controls the manipulator assembly 102.
[0021] The main assembly 106 may be located at an operator console, which is typically located in the same room as the operating table T, such as to the side of the operating table on which the patient P is located. However, it should be understood that the operator O may be located in a different room than the patient P or in an entirely different building. The main assembly 106 typically includes one or more control devices for controlling the manipulator assembly 102. The control devices may include any number of various input devices, such as a joystick, trackball, scroll wheel, steering wheel, button, data glove, trigger gun, hand controller, voice recognition device, human motion or presence sensor, and / or the like.
[0022] The manipulator assembly 102 supports the medical device 104 and can include one or more non-servo-controlled links (e.g., one or more links that can be manually positioned and locked in an appropriate position, often referred to as a setup structure), one or more servo-controlled links (e.g., one or more links that can be controlled in response to commands from a control system), and / or a kinematic structure of the manipulator. The manipulator assembly 102 can include a plurality of actuators or motors that drive inputs on the medical device 104 in response to commands from a control system (e.g., the control system 112). The actuator can include a drive system that, when coupled to the medical device 104, can advance the medical device 104 into a natural or surgically created anatomical orifice. Other drive systems can move the distal portion of the medical device 104 in multiple degrees of freedom, which can include three linear motions (e.g., linear motion along the X, Y, Z Cartesian axes) and three rotational motions (e.g., rotations about the X, Y, Z Cartesian axes). Additionally, the actuator may be used to actuate an articulatable end effector of the medical instrument 104 for grasping tissue in the jaws of a biopsy device and / or the like.
[0023] The medical system 100 may include a sensor system 108 having one or more subsystems for receiving information about the manipulator assembly 102 and / or the medical device 104. Such subsystems may include a position / location sensor system (e.g., an electromagnetic (EM) sensor system); a shape sensor system for determining the position, orientation, speed, velocity, posture, and / or shape of a distal portion and / or along one or more segments of a flexible body that may comprise the medical device 104; a visualization system for capturing images from the distal portion of the medical device 104; and / or an actuator position sensor (such as a resolver, encoder, potentiometer, and the like) that describes the rotation and orientation of a motor that controls the device 104.
[0024] The medical system 100 may include a display system 110 for displaying images or representations of a surgical site and a medical device 104. In some examples, the display system 110 may present preoperative or intraoperative images of the surgical site using an image modality such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like. In some embodiments, the medical device 104 may include a visualization system including an image capture component for recording instant or real-time images of the surgical site and providing the images to the operator O via one or more displays of the display system 110.
[0025] In some examples, the medical system 100 can configure the displayed representation, medical instrument 104, and controls of the primary assembly 106 so that the relative orientation of the medical instrument is similar to the relative orientation of the eyes and / or hands of the operator O. In this manner, the operator O can manipulate the medical instrument 104 and hand controls as if viewing the workspace in substantial telepresence.
[0026] In some examples, such as for the purpose of image-guided medical procedures, the display system 110 can display a virtual navigation image in which the actual position of the medical device 104 is registered (e.g., dynamically referenced) with the preoperative or immediate image / model. This can be done to present the operator O with a virtual image of the internal surgical site from the perspective of the medical device 104.
[0027] The medical system 100 may also include a control system 112. The control system 112 includes at least one memory and at least one computer processor (not shown) for implementing control between the manipulator assembly 102, the medical device 104, the main assembly 106, the sensor system 108 and / or the display system 110. The control system 112 also includes programmed instructions (e.g., a non-transitory machine-readable medium storing instructions) to implement some or all of the methods described according to the various aspects disclosed herein, including instructions for providing information to the display system 110. Although the control system 112 is Figure 1 106, but the system may include two or more data processing circuits, with a portion of the processing optionally being performed on or adjacent to the manipulator assembly 102, another portion of the processing being performed at the master assembly 106, and / or the like. The processor of the control system 112 may execute instructions corresponding to the processes disclosed herein and described in more detail below.
[0028] In some examples, control system 112 can receive force and / or torque feedback from medical device 104. In response to the feedback, control system 112 can transmit a signal to master assembly 106. In some examples, control system 112 can transmit a signal instructing one or more actuators of manipulator assembly 102 to move medical device 104.
[0029] The control system 112 can obtain sensor data from the sensor system 108, which is used to calculate the approximate position of the medical device 104 relative to the anatomy of the patient P. The system can implement the sensor system 108 to register and display the medical device with the medical image recorded before or during surgery. For example, PCT Publication WO2016 / 191298 (published on December 1, 2016 and entitled "Systems and Methods of Registration for Image Guided Surgery"), which is incorporated herein by reference in its entirety, discloses an example system.
[0030] The medical system 100 may further include operating and support systems, such as lighting systems, articulated (such as steering) control systems, flushing systems, and / or suction systems (not shown). In some embodiments, the medical system 100 may include more than one manipulator assembly and / or more than one main assembly. Among other factors, the exact number of manipulator assemblies may depend on the medical procedure and the space limitations in the operating room. The main assemblies 106 may be juxtaposed, or they may be positioned in separate locations. Multiple main assemblies may allow more than one operator to control one or more manipulator assemblies in various combinations.
[0031] Figure 2A and Figure 2B is a simplified diagram of a side view of a patient coordinate space including a medical device mounted on an insertion assembly according to some embodiments. Figure 2A and Figure 2B As shown in , surgical environment 300 may include a patient P positioned on an operating table T. Patient P may be stationary within surgical environment 300 in the sense that the patient's overall movement is limited by sedation, restraints, and / or other means. Cyclic anatomical movement, including respiratory and cardiac movement of patient P, may continue. Within surgical environment 300, a medical instrument 304 is used to perform a medical procedure, which may include, for example, surgery, biopsy, ablation, illumination, irrigation, aspiration, or system registration procedures. Medical instrument 104 may be, for example, instrument 104. Instrument 304 includes a flexible, elongated device 310 (e.g., a catheter) coupled to an instrument body 312. Elongated device 310 includes one or more passages (not shown) sized and shaped to receive a medical tool (not shown).
[0032] The elongated device 310 may also include one or more sensors (e.g., components of the sensor system 108). In some examples, an articulation sensor 314, such as a fiber optic shape sensor, may be fixed at a proximal point 316 on the instrument body 312. The proximal point 316 of the articulation sensor 314 may move with the instrument body 312, and the position of the proximal point 316 is known (e.g., via a tracking sensor or other tracking device). The articulation sensor 314 may measure the shape from the proximal point 316 to another point (such as a distal portion 318 of the elongated device 310). The articulation sensor 314 may be aligned with the flexible elongated device 310 (e.g., provided within an internal passageway (not shown) or mounted externally). In some examples, the optical fiber may have a diameter of approximately 200 μm. In other examples, the diameter may be larger or smaller. The articulation sensor 314 may be used to determine the shape of the flexible elongated device 310. Optical fibers including fiber Bragg gratings (FBGs) may be used to provide strain measurements in a structure in one or more dimensions. Various systems and methods for monitoring the shape and relative orientation of an optical fiber in three dimensions are described in U.S. Patent Application No. 11 / 180,389, filed on July 13, 2005 and entitled "Fiber optic position and shape sensing device and method relating thereto"; U.S. Patent Application No. 12 / 047,056, filed on July 16, 2004 and entitled "(Fiber Optic Shape and Relative Orientation Sensing)"; and U.S. Patent No. 6,389,187, filed on June 17, 1998 and entitled "Optical Fibre Bend Sensor", which are incorporated herein by reference in their entireties. The sensor in some embodiments may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. Various systems for registering and displaying surgical instruments with surgical images using fiber optic sensors are provided in PCT Publication WO 2016 / 191298 (published on December 1, 2016 and entitled “Systems and Methods of Registration for Image Guided Surgery”), which is incorporated herein by reference in its entirety.
[0033] In some examples, a position sensor, such as an electromagnetic (EM) sensor, can be incorporated into the medical device 304. A series of position sensors can be positioned along the flexible elongated device 310 and used for shape sensing. In some examples, the position sensors can be configured and positioned to measure six degrees of freedom, such as three position coordinates X, Y, Z and three orientation angles indicating pitch, yaw and roll of a base point. In some examples, the position sensors can be configured and positioned to measure five degrees of freedom, such as three position coordinates X, Y, Z and two orientation angles indicating pitch and yaw of a base point. Further description of the position sensor system is provided in U.S. Patent No. 6,380,732 (filed on August 11, 1999 and entitled "Six-Degree of Freedom Tracking System Having a Passive Transponder on the Object Being Tracked"), which is incorporated herein by reference in its entirety.
[0034] The elongated device 310 can house cables, linkages, or other steering controls (not shown) extending between the instrument body 312 and the distal portion 318 to controllably bend the distal portion 318. In some examples, at least four cables are used to provide independent "up and down" steering to control the pitch of the distal portion 318 and "side to side" steering to control the yaw of the distal portion 318. Steerable elongated devices are described in detail in U.S. Patent Application No. 13 / 274,208 (filed October 14, 2011) (disclosing "Catheter with Removable Vision Probe"), which is incorporated herein by reference in its entirety. The instrument body 312 can include a drive input device that is removably coupled to a drive element (such as an actuator) of the assembly and receives power from the drive element.
[0035] The instrument body 312 can be coupled to an instrument bracket 306. The instrument bracket 306 can be mounted to an insertion stage 308 that is fixed within the surgical environment 300. Alternatively, the insertion stage 308 can be movable but have a known position within the surgical environment 300 (e.g., via a tracking sensor or other tracking device). The instrument bracket 306 can be a component of a manipulator assembly (e.g., manipulator assembly 102) that is coupled to the medical device 304 to control insertion motion (e.g., motion along axis A) and / or motion of the distal portion 318 of the elongated device 310 in multiple directions such as yaw, pitch, and / or roll. The instrument bracket 306 or insertion stage 308 can include an actuator, such as a servo motor (not shown), that controls the motion of the instrument bracket 306 along the insertion stage 308.
[0036] The sensor device 320, which may be a component of the sensor system 108, can provide information about the orientation of the instrument body 312 as it moves along the insertion axis A on the insertion stage 308. The sensor device 320 may include one or more resolvers, encoders, potentiometers, and / or other sensors that determine the rotation and / or orientation of the actuator that controls the movement of the instrument bracket 306 and, therefore, the movement of the instrument body 312. In some embodiments, the insertion stage 308 is linear. In some embodiments, the insertion stage 308 may be curved or have a combination of curved and linear sections.
[0037] Figure 2A The instrument body 312 and the instrument bracket 306 are shown in a retracted orientation along the insertion station 308. In this retracted orientation, the proximal point 316 is at the orientation L0 on the axis A. Figure 2B 308, the instrument body 312 and the instrument bracket 306 are advanced along the linear track of the insertion table 308, and the distal portion 318 of the elongated device 310 has been advanced into the patient P. In this advanced orientation, the proximal point 310 is at the orientation L1 on the axis A. In some examples, encoders and / or other orientation data from one or more actuators that control the movement of the instrument bracket 306 along the insertion table 308 and / or one or more orientation sensors associated with the instrument bracket 306 and / or the insertion table 308 can be used to determine the orientation of the proximal point 316 relative to the orientation L0. In some examples, the orientation can further be used as an indicator of the distance or insertion depth of the distal portion 318 of the elongated device 310 inserted into the passage of the anatomy of the patient P.
[0038] refer to Figure 3-Figure 7 C, a method for controlling a steerable device (such as the flexible elongated device 310 and / or the instrument 104 described above) is described. Figure 3is a flow chart describing a method 350 for controlling a steerable device according to some embodiments. The method 350 is a set of operations or processes in Figure 3 Shown in. Figure 3 The process shown in can be Figure 3 In some embodiments of method 350, one or more of the processes shown may not be performed. Figure 3 One or more processes not explicitly shown in the method 350 may be included before, after, between, or as part of the processes shown. In some embodiments, one or more processes of the method 350 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., the processors of the control system 112 described above) may cause the one or more processors to perform the one or more processes. Figure 4-Figure 6 is a diagram showing the relationship between user input and the orientation of a steerable device in a control environment according to some embodiments.
[0039] refer to Figure 3 In process 352, a system such as the medical system 100 above can receive a first user input that instructs the system to move a steerable device of the system in a first direction of motion. The first direction of motion can be along any axis (e.g., one or both of the lateral axes, the insertion axis, etc.), or can be in any orientation (e.g., yaw, pitch, roll, etc.). The user input can be provided by the operator O using a control device of the main component 106, and in various examples, the control device can include a joystick, a trackball, a scroll wheel, a steering wheel, a button, a data glove, a trigger gun, a manual controller, a voice recognition device, a human motion or presence sensor, and / or other suitable control devices. The user input can include a desired speed, a desired orientation, and / or other indications for the steerable device.
[0040] refer to Figure 3 and Figure 4 In process 354, the system may track a user-indicated position associated with the steerable device in response to the first user input. Figure 4In the control environment 400 of the present invention, the user-indicated position, which may be referred to as the primary position or the proxy position, is represented by circles 402A and 402B. The user-indicated position may be a virtual position because it may be different from the actual position of the steerable device. At cycle zero where remote operation is started, the user-indicated position (e.g., position 402A) may match the position of the steerable device (e.g., position 404A). In the example shown, the first user input may instruct the system to drive the steerable device so that a portion of the steerable device (e.g., the distal portion of the steerable device) moves from position 402A to position 402B. As shown, the system may track the movement of the user-indicated position from position 402A to position 402B.
[0041] In examples where the first user input includes a speed measure, the amount by which the user indicated position was moved can be determined by applying any applicable speed modifications, such as a position limit or catch-up modification as described in more detail below in the context of the second user input. The modified speed can then be integrated (e.g., multiplied by a cycle time of a constant magnitude speed) to determine the amount by which the user indicated position was moved.
[0042] refer to Figure 3 In process 356, the system can control the steerable device in response to the first user input. In some embodiments, the control system 112 of the medical system 100 can cause an actuator of the manipulator assembly 102 to apply a force to a cable, linkage, pull wire, chord, or other steering control member within the steerable device in response to the first user input. In some embodiments, the control system 112 of the medical system can cause an actuator of an insertion table (e.g., insertion table 308) to apply a force to the steerable device in response to the first user input. The user input can be converted into an actuator position and / or an amount of force applied by the actuator.
[0043] With reference to process 358, the system can determine the current position of one or more parts of the steerable device in response to the movement of the actuator in process 356. For example, the control system 112 can receive data from sensors of the sensor system 108 (e.g., shape sensors, position sensors, actuator position sensors, etc. of the steerable device), and can use the received data to determine the position, orientation, speed, velocity, posture and / or shape of one or more segments along the steerable device. For example, the control system 112 can use the received data to determine the position, orientation, speed, velocity, posture and / or shape of the distal portion of the steerable device or another part of the steerable device. The control system 112 can also consider historical posture data to reconstruct the shape of the steerable device at a certain time interval. In some embodiments, the control system 112 can receive data from sensors of the sensor system 108 associated with the movement of the instrument bracket 306 along the insertion table 308, and can use the received data to determine the position, orientation, speed, velocity, posture and / or shape of one or more segments along the steerable device. The orientation of the steerable device determined in process 358 may be fed back to process 356 in a feedback loop to adjust the actuator so that the steerable device appropriately responds to the user input of process 352. For example, the actuator may be controlled so that the orientation of the steerable device is closely aligned with the user-indicated orientation when the steerable device is unopposed.
[0044] exist Figure 4 An example in which the steerable device is unopposed is shown in . The orientation of a given portion of the steerable device along a first direction of motion, also referred to as a slave orientation, is represented by circles 404A and 404B. Movement of the steerable device in response to user input is represented by a change in orientation from 404A to 404B. As can be seen, the orientation of the steerable device can closely track the orientation of the agent. In some embodiments, when moving in free space or substantially unopposed, such as during process 352, the steerable device can move approximately 1:1 with the received user input.
[0045] However, when used during a procedure, the steerable device may encounter opposing forces. For example, in a medical application where the steerable device is advanced through a passage, the device may encounter resistance from the tissue forming the passage. The resistance may be fed back to the operator O in a variety of ways. In some embodiments, the opposing force may be converted to resistance, vibration or other tactile feedback at the control device, and / or represented visually or by an audio prompt (e.g., at the display system 110).
[0046] exist Figure 5An example in which a steerable device encounters an opposing force is shown in a control environment 500 of FIG. The user-indicated orientation along the first direction of motion is represented by circles 502A and 502B, and the orientation of the steerable device is represented by circles 504A and 504B. In this example, the first user input indicates that the system drives the steerable device so that a portion of the steerable device moves from the user-indicated orientation 502A to the user-indicated orientation 502B. However, due to the opposing force, the additional force applied by the actuator may cause tension in the steering control member of the steerable device without corresponding movement in the steerable device. Therefore, the steerable device may only move from 504A to 504B, which is less than the user-indicated movement from 502A to 502B. An orientation difference may occur between the user-indicated orientation 502B and the corresponding orientation of the steerable device 504B, as indicated by mark 506. The difference may correspond to tension in the actuator and / or steering control member of the steerable device, which may be referred to as wind-up. Certain aspects of winding are beneficial, and the resulting tension can be used to guide the steerable device, anchor the steerable device, and / or manipulate the deformable passageway.
[0047] Even without any input from operator O, the difference between the user-indicated orientation and the actual orientation may change over time. For example, although the user-indicated orientation may be responsive only to user instructions, the actual orientation of the steerable device may change as the surrounding passages move and apply forces to the steerable device. External forces or loads on the steerable device may change the actual orientation of the steerable device without changing the user-indicated orientation. In many subsequent examples, the system may react based on the difference between the user-indicated orientation and the actual orientation rather than solely based on user input. For example, even without instructions from operator O and without changing the operating mode, the system may apply opposing forces during the procedure to maintain the steerable device in a fixed orientation relative to the surrounding anatomy. From the operator's perspective, the system may seamlessly maintain the orientation of the steerable device.
[0048] refer to Figure 3 At process 360, the system may determine an amount of difference in the first direction between the orientation of the steerable device determined in process 358 and the user-indicated orientation of the user input received in process 352. As described above, the difference may change based on changes in the user input and / or changes in the actual orientation of the elongated device due to external forces.
[0049] Subsequent user inputs, such as along the first direction, may further increase the difference between the orientation of the steerable device and the user-indicated orientation. For example, actuator forces and tension in the steerable device may increase in response to the user input without noticeable movement of the steerable device. To avoid damage to surrounding tissue, the system may increase the tension and corresponding difference to a predetermined threshold only in response to the user input. The system may then clip or ignore any additional user input in the first direction that would further move the user-indicated orientation and thereby increase the tension and / or difference beyond the threshold. Inhibiting the response of the steerable device beyond the threshold can protect the surrounding environment. The threshold can vary during the course of the procedure, and in some examples, the threshold can be lowered as the steerable device moves deeper to reduce the risk of damage to increasingly narrow and / or fragile passages.
[0050] When receiving user input to move the steerable device in the opposite direction, the tension can be reduced before driving the steerable device in the opposite direction. The controlled adjustment of the winding can allow the operator O to set the tension suitable for the task without moving the steerable device. Referring to process 362, the system can receive a second user input, which indicates that the system moves the steerable device in the second direction. In some examples, at least a portion of the indicated movement can be opposite to the first direction. For example, the second direction can be directly opposite to the first direction, such as the second direction is left and the first direction is right, or the second direction is up and the first direction is down. As another example, the second direction can be partially opposite to the first direction, such as the second direction is east, and the first direction is northwest or southwest. Like the first user input, the second user input can be provided by the operator O using the control device of the main component 106, and the operator O can use the same control device to provide both the first user input and the second user input to the system. The reception of the system can be performed substantially as described in process 352.
[0051] refer to Figure 3 The process 364 and Figure 6 , the system can move the user indicated position in response to the second user input and based on a catch-up profile. Examples of catch-up profiles are described in more detail below. For example, the catch-up profile can add a velocity to a velocity associated with the second user input. Figure 6 In the control environment 600 of FIG. 600 , the second user input may instruct the system to move the user-indicated orientation 602A to the user-indicated orientation 602B. It can be seen that there is still some difference 606 between the orientation of the steerable device 604 (which may not change in response to the input) and the second user-indicated orientation 602B. However, the subsequent difference 606 may be less than the previous difference between the orientation of the steerable device 604 and the first user-indicated orientation 602A.
[0052] Based on the change in the difference caused by process 364, the system can drive the steerable device to reduce the tension in the steerable device in response to the second user input based on the catch-up curve shown in process 366. In some embodiments, the control system 112 of the medical system 100 can cause the actuator of the manipulator assembly 102 to reduce the force applied to the steering control of the steerable device corresponding to the first direction previously applied in response to the first user input. In some embodiments, the control system 112 can cause the actuator to drive the instrument bracket 306 along the insertion table 308 to reduce the force applied to the steerable device corresponding to the first user input.
[0053] refer to Figure 6 , the amount by which the second user-indicated orientation 602B moves in response to the second user input and the corresponding amount of tension reduction can be determined in part by the catch-up curve. In examples where the second user input includes a velocity determination, the amount by which the user-indicated orientation moves can be determined by applying any applicable velocity modifications (such as orientation limits or catch-up modifications from the catch-up curve). The modified velocity can then be integrated to determine the amount by which the user-indicated orientation is moved. In some embodiments, the catch-up curve can provide a 1:1 relationship between an unwinding input (a user input that eliminates winding) and a winding input (a user input that generates winding). For example, the size or duration of a user input that generates a given amount of difference and / or tension in a steerable device can be approximately the same as the size or duration of a user input that eliminates that difference and / or tension.
[0054] In some examples, the catch-up curve may be more sensitive to the unwind input than the winding input, such that the input for reducing the difference and / or tension may be greater than the input for generating the difference and / or tension. In some examples, the catch-up curve may be less sensitive to the unwind input than the winding input, such that the input for reducing the difference and / or tension may be less than the input for generating the difference and / or tension. The catch-up curve may include any combination of linear, nonlinear, exponential, logarithmic, step, piecewise, hyperbolic, parabolic, periodic / trigonometric, inverse hyperbolic, polynomial, modulus, other monotonic functions, and / or the like that define a relationship between a reduction in the difference and / or tension and an aspect of the user input.
[0055] In some examples, the catch-up curve can make the orientation difference and tension more sensitive to user input that decreases the tension than to user input that increases the tension. For example, the slope of the unwind function can be greater than the slope of the wind function. This increased sensitivity to the unwind input can make the system feel more responsive.
[0056] In some examples, the catch-up curve may include two or more regions of different slopes. In these examples, the slope may be proportional to the difference, and regions with larger differences may have larger slopes and may be more sensitive to unwind user inputs than regions with smaller differences. Reducing the sensitivity when the difference is smaller may reduce the tendency to over-bias the neutral orientation or accidentally start moving the steerable device in the opposite direction.
[0057] In some examples, the catch-up curve for reducing the difference may include two regions. In the first region where the difference is larger, the user input may be more likely to indicate an intention to fine-tune the winding, and the curve may be relatively insensitive to the user input to allow the operator O to make precise adjustments to the tension. In the second region where the difference is smaller, the user input may be more likely to indicate an intention to move the steerable device in the second direction. Therefore, the curve may be relatively more sensitive to the user input in the second region to allow the system to respond better.
[0058] Specific pursuit curves and aspects of the pursuit curves (such as function(s), slope(s), sensitivity, and other aspects) can depend on 1) an amount of difference (e.g., an orientation error); 2) any aspect of the user input, such as the size of the input, the duration of the input, the length of the interval between inputs, and / or other suitable aspects; 3) time; 4) movement of the steerable device due to external loads; 5) the smoothness of response of the actuator or other tensioning mechanism; 6) the shape of the steerable device; 7) the sensitivity of the tissue forming the channel; 8) the curvature of the anatomy; 9) operator preference; and / or 10) other suitable aspects of the system, operator O, or procedure.
[0059] In some examples, the catch-up curve may provide a decaying exponent as the user-indicated orientation asymptotically approaches the orientation of the device when the steerable device has no (or little) movement due to external loads, the user input is at a constant rate, and the catch-up term is a linear function of the instantaneous orientation error. For some systems and actuator configurations, the decaying exponent may provide a smooth change in tension in the steerable device.
[0060] refer to Figure 3 In process 368, the system can receive a third user input, which further instructs the system to move the steerable device in a second direction of motion, such as after the user-indicated orientation and the orientation of the steerable device have converged. The third user input can be provided by operator O using the control device of main assembly 106, and operator O can use the same control device to provide the first user input, the second user input, and the third user input to the system. The system's receiving can be performed substantially as described in process 352.
[0061] Referring to process 370, the system can move the user-indicated orientation in response to the third user input. Since the orientation difference has been eliminated, referring to process 372, the system can control the steerable device to move in the second direction of motion in response to the third user input. This can be performed substantially as described in process 356.
[0062] During a medical procedure, as the steerable device is traversed through the patient's anatomy, method 350 may be repeated. For example, a steerable device traversing a branching anatomical region, such as a human lung, may be inserted into and removed from a series of anatomical branches using method 350 to manage tension associated with a difference between a user-indicated orientation and a corresponding orientation of the steerable device.
[0063] At any time, the system can detect a fault in the system, in the steerable member, and / or in the program. Based on the detection of the fault, the system can eliminate the difference and tension (if any) in the steerable member. Figure 7 is a flow chart describing a method 700 for controlling a steerable device in response to a fault according to some embodiments. The method 700 is a set of operations or processes in Figure 7 In some embodiments of method 700, one or more of the illustrated processes may not be performed. Figure 7 One or more processes not explicitly shown in the method 700 may be included before, after, between, or as part of the processes shown. In some embodiments, one or more processes of the method 700 may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which when executed by one or more processors (e.g., the processor of the control system 112 described above) may cause the one or more processors to perform the one or more processes. The method 700 may be performed at any time including during the method 350 described above.
[0064] refer to Figure 7 In process 702, a system such as the medical system 100 above may detect a fault in the system, in a steerable component, and / or in a procedure, or may receive an emergency indication from an operator O. Figure 7In process 704, if there is an orientation difference, the system can control the steerable device to reduce the orientation difference in response to a fault or emergency indication. The rate and degree of reduction of the orientation difference can be similar to a catch-up curve, because it can include linear, nonlinear, exponential, logarithmic, step, segmented, hyperbolic, parabolic, periodic / trigonometric, inverse hyperbolic, polynomial, modulus, other monotonic functions and / or similar functions to determine the rate and degree of reduction of the orientation difference and tension in the steerable device. Various aspects of this performance (such as (one or more) functions, (one or more) slopes and other aspects) can depend on the shape of the elongated device, the sensitivity of the tissue constituting the channel, the curvature of the anatomy, operator preferences, or other suitable aspects of the system, operator O or program. The rate and degree of reduction of the orientation difference can be selected so that there is no unexpected movement of the steerable device that is noticeable to the operator O.
[0065] In some examples, the orientation difference can be eliminated immediately and the tension in the steerable device can be released immediately. In delicate environments, rapidly reducing tension can be beneficial to reduce the risk of damage or injury to the steerable device. In some examples, the orientation difference can be reduced at a slower rate to avoid elastic rebound, buckling or bundling of the steering control, or other effects of the actuator or steering control.
[0066] Various examples of implementations of the present disclosure
[0067] The present invention describes various instruments and parts of instruments according to their states in three-dimensional space. For example, the term position refers to the position of an object or a part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y, and z coordinates). The term orientation refers to the rotational placement of an object or a part of an object (e.g., one or more rotational degrees of freedom, such as roll, pitch, and yaw). The term posture refers to the position of an object or a part of an object in at least one translational degree of freedom, or refers to the position of an object or a part of an object in at least one rotational degree of freedom (e.g., up to six total degrees of freedom). The term shape refers to a set of postures, positions, or orientations measured along an object.
[0068] One or more elements in the embodiments of the present disclosure may be implemented in software to be executed on a processor of a computer system such as a control processing system. When implemented in software, the elements of the embodiments of the present disclosure may be code segments for performing various tasks. Program segments or code segments may be stored in a processor-readable storage medium or device, which may be downloaded via a computer data signal embodied in a carrier wave form on a transmission medium or a communication link. A processor-readable storage device may include any medium that may store information, including optical media, semiconductor media, and / or magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor storage devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs); floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. Code segments may be downloaded via computer networks such as the Internet, intranets, etc. Any of a variety of centralized or distributed data processing architectures may be used. Programming instructions may be implemented as multiple separate programs or subroutines, or they may be integrated into multiple other aspects of the system described herein. In some examples, the control system can support wireless communication protocols such as Bluetooth, infrared data communication (IrDA), HomeRF, IEEE 802.11, digital enhanced cordless telecommunications (DECT), ultra-wideband (UWB), ZigBee (Internet of Things), and wireless telemetry.
[0069] The medical tool may be delivered by a flexible, elongated device (e.g., a catheter) disclosed herein, and may include, for example, an image capture probe, a biopsy instrument, a laser ablation fiber, and / or other surgical tools, diagnostic tools, or therapeutic tools. The medical tool may include an end actuator having a single working member, such as a scalpel, a blunt knife, an optical fiber, an electrode, and / or the like. Other end actuators may include, for example, forceps, graspers, scissors, clip appliers, and / or the like. Other end actuators may further include electrically activated end actuators, such as electrosurgical electrodes, transducers, sensors, and / or the like. The medical tool may include an image capture probe including a stereo or single-view camera for capturing images (including video images). The medical tool may additionally accommodate a cable, linkage, or other actuation control device extending between its proximal and distal portions to controllably bend the distal portion of the medical tool. Steerable instruments are described in detail in U.S. Patent No. 7,316,681 (filed on October 4, 2005 and entitled "Articulated Surgical Instrument for Performing Minimally Invasive Surgery with Enhanced Dexterity and Sensitivity") and U.S. Patent Application No. 12 / 286,644 (filed on September 30, 2008 and entitled "Passive Preload and Capstan Drive for Surgical Instruments"), the entire contents of which are incorporated herein by reference.
[0070] The systems described herein may be suitable for navigating and treating anatomical tissue via natural or surgically created passageways in any of a variety of anatomical systems, including the lungs, colon, intestines, kidneys and calyces, brain, heart, circulatory system including vasculature, and / or the like.
[0071] Note that the processes and displays presented may not be inherently related to any particular computer or other device. Various general purpose systems may be used with the routines according to the teachings herein, or it may prove convenient to construct more specialized equipment to perform the described operations. In addition, it will be appreciated that a variety of programming languages may be used to implement the examples described herein.
[0072] Although certain examples have been described and shown in the drawings, it should be understood that these examples are merely illustrative and not restrictive, and that the examples described are not limited to the specific constructions and arrangements shown and described, since various other modifications may be made by those skilled in the art.
Claims
1. A system comprising: a control device configured to receive user input; a manipulator system including an actuator configured to receive and drive the steerable device; as well as a control system communicatively coupled to the manipulator system and the control device, the control system being configured to: Tracking a virtual user indicated position based on a first user input; determining a device orientation of a portion of the steerable device; determining a position difference in a first direction between the determined device position and the virtual user-indicated position; receiving a second user input commanding movement of the steerable device in a second direction, wherein at least a portion of the second direction is opposite to the first direction; and In response to the second user input, the orientation difference is reduced based on an aspect of the second user input and a catch-up curve. 2 . The system of claim 1 , wherein the control system is configured to reduce the orientation difference by reducing tension in a steering control of the steerable device. 3 . The system of claim 1 , wherein the aspect of the second user input comprises at least one of a size of the second user input or a duration of the second user input.
4. The system of any one of claims 1-3, wherein the catch-up curve is based on at least one of: a shape of the steerable device, a sensitivity of a channel in which the steerable device is disposed, or an operator preference.
5. The system of any one of claims 1-3, wherein the catch-up curve reduces the position difference by applying a catch-up speed to the second user input. 6 . The system of claim 5 , wherein the catch-up speed is configured such that control of the steerable device is more responsive to inputs that decrease the orientation difference than to inputs that increase the orientation difference.
7. The system of any one of claims 1-3, wherein the catch-up curve includes a first region configured to have a first sensitivity and a second region configured to have a second sensitivity different from the first sensitivity.
8. The system of claim 7, wherein the first region is associated with a greater orientation difference than the second region, and the first sensitivity is greater than the second sensitivity.
9. The system of claim 7, wherein the first region is associated with a greater orientation difference than the second region, and the first sensitivity is less than the second sensitivity.
10. The system according to any one of claims 1 to 3, wherein the control system is configured to: receiving a third user input commanding movement of the steerable device in the first direction; determining that the third user input causes the position difference to exceed a threshold; as well as A response to the third user input is suppressed based on a determination that the third user input causes the orientation difference to exceed the threshold.
11. The system of any one of claims 1-3, wherein the virtual user indicated orientation does not change in response to a load on the steerable device.
12. The system according to any one of claims 1 to 3, wherein the control system is configured to: a fault is detected; and In response to the fault, the orientation difference is reduced without operator-perceptible movement of the steerable device.
13. The system of claim 12, wherein the control system is configured to reduce the orientation difference in response to the fault by removing tension in a steering control of the steerable device.
14. A non-transitory machine-readable medium storing executable code, the executable code, when executed by one or more processors, causing the one or more processors to perform a method for controlling a steerable device, the method comprising: receiving a first user input from a master controller, the first user input indicating movement of the steerable elongated device in a first direction of motion; tracking a virtual user indicated position based on the first user input; controlling movement of the steerable elongated device based on the first user input; detecting a current device orientation of the steerable elongated device from a sensor of the steerable elongated device; determining a position difference between the detected current device position and the virtual user indicated position; receiving a second user input from the master controller in a second direction of motion, wherein at least a portion of the second direction of motion is opposite to the first direction of motion; and In response to the second user input, the orientation difference is reduced based on a catch-up curve.
15. The non-transitory machine-readable medium of claim 14, wherein the method further comprises: The first user input and the second user input are received from a common control device of the master controller.
16. The non-transitory machine-readable medium of claim 14, wherein reducing the orientation difference comprises reducing tension in a steering control of the steerable elongated device.
17. The non-transitory machine-readable medium of any one of claims 14-16, wherein the catch-up curve is based on at least one of a magnitude of the second user input or a duration of the second user input.
18. The non-transitory machine-readable medium of any one of claims 14-16, wherein the catch-up curve is based on at least one of: a shape of the steerable elongated device, a sensitivity of a channel in which the steerable elongated device is disposed, or an operator preference.
19. The non-transitory machine-readable medium of any one of claims 14-16, wherein the catch-up curve reduces the position difference by applying a catch-up speed to the second user input.
20. The non-transitory machine-readable medium of claim 19, wherein the catch-up speed is configured to make control of the steerable elongated device more responsive to inputs that decrease the orientation difference than to inputs that increase the orientation difference.
21. The non-transitory machine-readable medium of any one of claims 14-16, wherein the catch-up curve comprises a first region configured to have a first sensitivity and a second region configured to have a second sensitivity different from the first sensitivity.
22. The non-transitory machine-readable medium of claim 21, wherein the first region is associated with a greater orientation difference than the second region, and the first sensitivity is greater than the second sensitivity.
23. The non-transitory machine-readable medium of claim 21, wherein the first region is associated with a greater orientation difference than the second region, and the first sensitivity is less than the second sensitivity.
24. The non-transitory machine-readable medium of any one of claims 14-16, wherein the method further comprises: detecting a third user input in the first direction of motion; determining that the third user input causes the position difference to exceed a threshold; as well as A response to the third user input is suppressed based on a determination that the third user input causes the orientation difference to exceed the threshold.
25. The non-transitory machine-readable medium of any one of claims 14-16, wherein the method further comprises: After reducing the orientation difference, detecting a third user input in the second direction of motion; moving the virtual user indication position based on the third user input; as well as The steerable elongate device is moved in the second direction of motion based on the third user input.
26. The non-transitory machine-readable medium of any one of claims 14-16, wherein the method further comprises: A fault is detected; as well as In response to the fault, the orientation difference is reduced without operator-perceptible movement of the steerable elongated device.
27. The non-transitory machine-readable medium of claim 26, wherein the method further comprises: In response to the fault, tension in a steering control of the steerable elongated device is removed.
28. A device comprising: one or more processors; as well as a non-transitory computer memory storing machine-executable instructions that, when executed by the one or more processors, cause the apparatus to: receiving a first user input from a control device commanding movement of the steerable device in a first direction; tracking a virtual user indicated position based on the first user input; commanding an actuator coupled to the steerable device to move the steerable device in the first direction based on the first user input, wherein movement of the steerable device in response to the first user input generates a positional difference; detecting a device orientation of a portion of the steerable device from a sensor of the steerable device; determining a measurement of said difference in orientation; receiving a second user input from the control device commanding movement of the steerable device in a second direction, wherein at least a portion of the second direction is opposite to the first direction; and In response to the second user input, the orientation difference is reduced based on an aspect of the second user input and a catch-up curve.
29. An apparatus according to claim 28, wherein the orientation difference is associated with tension in the steerable device, and wherein the non-transitory computer memory stores machine executable instructions that, when executed by the one or more processors, cause the apparatus to command the actuator to reduce the tension in the steerable device based on the catch-up curve.
30. The apparatus of claim 28, wherein the aspect of the second user input comprises at least one of a size of the second user input or a duration of the second user input.
31. The apparatus of any one of claims 28-30, wherein the catch-up curve is based on at least one of: a shape of the steerable device, a sensitivity of a channel in which the steerable device is disposed, or an operator preference.
32. An apparatus according to any of claims 28-30, wherein the catch-up curve applies a catch-up speed to the second user input.
33. The device of any of claims 28-30, wherein the catch-up curve makes the device more sensitive to user input that decreases the orientation difference than to user input that increases the orientation difference.
34. The apparatus of any one of claims 28-30, wherein the catch-up curve has a first region associated with larger orientation differences and a second region associated with smaller orientation differences, and wherein a first sensitivity of the first region is greater than a second sensitivity of the second region.
35. The device of any of claims 28-30, wherein the non-transitory computer memory stores machine-executable instructions that, when executed by the one or more processors, cause the device to suppress a response that causes the orientation difference to exceed a threshold.
36. The apparatus of any one of claims 28-30, wherein the non-transitory computer memory stores machine-executable instructions that, when executed by the one or more processors, cause the apparatus to: After receiving the second user input, receiving a third user input from the control device commanding the steerable device to move in the second direction; and Based on the resolved orientation difference, the actuator is commanded to move the steerable device in the second direction in response to the second user input.
37. The apparatus of any one of claims 28-30, wherein the non-transitory computer memory stores machine-executable instructions that, when executed by the one or more processors, cause the apparatus to: A fault is detected, and In response to the fault, the actuator is commanded to reduce the orientation difference without operator-perceptible movement of the steerable device.
38. The apparatus of claim 37, wherein the non-transitory computer memory stores machine executable instructions that, when executed by the one or more processors, cause the apparatus to command the actuator to remove tension in the steerable device in response to the fault.
39. The apparatus of claim 37, wherein the virtual user indicated orientation does not change based on a load on the steerable device.
