A physician console that generates haptic vibrations for remote operation

Abstract

The medical system may include a first tactile interface device, one or more input sensors, one or more processors, and a memory storing instructions, which when executed by the one or more processors, cause the one or more processors to receive a first input signal from the one or more input sensors, transmit a kinesthetic tactile feedback signal to the first tactile interface device based at least on the first input signal for kinesthetic tactile feedback, receive a second input signal from the one or more input sensors, and transmit a vibrotactile feedback signal to the first tactile interface device based at least on the second input signal for vibrotactile feedback.

Description

[Technical field]

[0001] The present application relates to controllers, and in particular to controllers for robot-enabled teleoperated systems, including medical systems. [Background technology]

[0002] Medical procedures, such as laparoscopy, can involve accessing and visualizing interior regions of a patient. In a laparoscopic procedure, medical instruments can be inserted into the interior regions through laparoscopic access ports.

[0003] In certain procedures, a robotic-enabled medical system can be used to control the insertion and / or manipulation of instruments and their end effectors. The robotic-enabled medical system may include a robotic arm or other instrument positioning device. The robotic-enabled medical system may also include a controller that is used to control the positioning of the instrument during the procedure. Summary of the Invention [Means for solving the problem]

[0004] In a first aspect, a robot-enabled teleoperated medical system is described. The medical system includes a first haptic interface device, one or more input sensors, one or more processors, and a memory storing instructions. When the instructions are executed by the one or more processors, the instructions cause the one or more processors to receive a first input signal from the one or more input sensors, transmit a kinesthetic tactile feedback signal to the first haptic interface device based at least on the first input signal for kinesthetic tactile feedback, receive a second input signal from the one or more input sensors, and transmit a vibrotactile feedback signal to the first haptic interface device based at least on the second input signal for vibrotactile feedback.

[0005] The system may include one or more of the following features in any combination: (a) a first robotic arm; (b) the first input signal is based on a difference between a master command and a movement of the first robotic arm; (c) a second robotic arm separate from the first robotic arm; (d) the second input signal is derived from a collision between the first robotic arm and the second robotic arm; (e) the second input signal is derived from criteria including a distance between the first robotic arm and the second robotic arm being less than a first distance threshold; and (f) the second input signal is derived from a criterion including a distance between the first robotic arm and the second robotic arm being less than a first distance threshold. (g) the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm; (g) the second input signal is derived from a criterion including a distance between the first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold; (h) the second input signal is derived from a criterion including detection of a fault in the medical system; (i) the second input signal is derived from a criterion including detection of a first haptic interface device at a predetermined boundary. (j) a second tactile interface device that is separate from and separated from the first tactile interface device; (k) the second input signal is derived from criteria including a collision between the first tactile interface device and the second tactile interface device; (l) the second input signal is derived from criteria including a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold; (m) the kinesthetic tactile feedback includes a continuous force applied in response to the user input; (n) the second input signal is derived from criteria including a plurality of events, and a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events; (o) the vibrotactile feedback includes a component having a frequency of at least 10 Hz; and / or (p) the first tactile interface device further includes a motor that provides a torque about a roll axis of the first tactile interface device in response to a user input on the first tactile interface device.

[0006] In another aspect, a method is provided that includes receiving a first input signal from one or more input sensors and transmitting a kinesthetic tactile feedback signal to a first tactile interface device based at least on the first input signal for kinesthetic tactile feedback, and receiving a second input signal from the one or more input sensors and transmitting a vibrotactile feedback signal to the first tactile interface device based at least on the second input signal for vibrotactile feedback.

[0007] In another aspect, a robot-enabled teleoperated medical system is described. The medical system includes a first haptic interface device, one or more input sensors, one or more processors, and a memory storing instructions. When the instructions are executed by the one or more processors, the instructions cause the one or more processors to receive a first input signal from the one or more input sensors and transmit a first haptic feedback signal to the first haptic interface device based on the first input signal for at least a first haptic feedback, receive a second input signal from the one or more input sensors, and transmit a second tactile feedback signal to the first haptic interface device based on the second input signal for at least a second tactile feedback. [Brief description of the drawings]

[0008] The disclosed aspects are hereinafter described in conjunction with the accompanying drawings, in which like designations refer to like elements and which illustrate, but do not limit, the disclosed aspects. [Figure 1] 1 illustrates one embodiment of a cart-based robotic system deployed for diagnostic and / or therapeutic bronchoscopy procedures. [Diagram 2] 2 depicts a further embodiment of the robotic system of FIG. 1. [Diagram 3] 2 illustrates an embodiment of the robotic system of FIG. 1 positioned for ureteroscopy. [Figure 4] 2 illustrates one embodiment of the robotic system of FIG. 1 deployed for a vascular procedure. [Diagram 5] 1 illustrates one embodiment of a table-based robotic system positioned for a bronchoscopy procedure. [Figure 6] 6 provides an alternative view of the robotic system of FIG. 5. [Figure 7] 1 illustrates an example system configured to accommodate a robotic arm. [Figure 8] 1 illustrates one embodiment of a table-based robotic system configured for a ureteroscopy procedure. [Figure 9] 1 illustrates one embodiment of a table-based robotic system configured for laparoscopic procedures. [Figure 10] FIG. 10 illustrates one embodiment of the table-based robotic system of FIGS. 5-9 with pitch or tilt adjustment. [Figure 11] FIG. 11 provides a detailed illustration of the interface between the table and column of the table-based robotic system of FIGS. [Figure 12] 1 illustrates an alternative embodiment of a table-based robotic system. [Figure 13] FIG. 13 illustrates an end view of the table-based robotic system of FIG. [Figure 14] FIG. 1 illustrates an end view of a table-based robotic system with a robotic arm attached. [Figure 15] 1 illustrates an exemplary instrument driver. [Figure 16] 1 illustrates an exemplary medical instrument having a pair of instrument drivers. [Figure 17] 13 illustrates an alternative design of the instrument driver and instrument, where the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument. [Figure 18] 1 illustrates an instrument having an instrument-based insertion architecture. [Figure 19] 1 illustrates an exemplary controller. [Figure 20]FIG. 19 is a block diagram illustrating a localization system that estimates the location of one or more elements of the robotic system of FIGS. 1-10, such as the location of the instrument of FIGS. 16-18, in accordance with an exemplary embodiment. [Figure 21A] FIG. 1 is a block diagram illustrating an embodiment of a robot-enabled medical system including a controller for a robot-enabled medical instrument. [Figure 21B] FIG. 21B is a block diagram illustrating an embodiment of the controller of FIG. 21A that can be configured for hybrid impedance and admittance control. [Figure 22] FIG. 2 is an isometric view of one embodiment of a gimbal for the controller. [Figure 23] 2 illustrates a control system 200 according to some embodiments. [Figure 24] 4 illustrates how output signals are combined according to some embodiments. [Figure 25A] 4 illustrates example damping functions having different damping coefficients according to some embodiments. [Figure 25B] 1 illustrates an exemplary control signal to a haptic interface device (HID) or output response from an HID, according to some embodiments. [Figure 25C] 4 illustrates an exemplary control signal to or output response from an HID, according to some embodiments. [Figure 25D] 25B and 25C show example control signals, which are combinations of the control signals shown in FIG. 25B and FIG. 25C, according to some embodiments. [Figure 26] 1 is a flow chart illustrating an embodiment of a method for providing feedback to a user. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1. Overview. Aspects of the present disclosure may be integrated into a robot-enabled medical system capable of performing a variety of medical procedures, including both minimally invasive procedures such as laparoscopy, and non-invasive procedures such as endoscopy, among other endoscopic procedures, such as bronchoscopy, ureteroscopy, gastroscopy, and the like.

[0010] In addition to performing a wide range of procedures, the system may provide additional benefits such as enhanced imaging and guidance to assist the physician. Furthermore, the system may provide the physician with the ability to perform procedures from an ergonomic position without the need for awkward arm movements and positions. Still further, the system may provide the physician with the ability to perform procedures with improved ease of use, such that even a single user may control one or more of the instruments of the system.

[0011] Various embodiments are described below in conjunction with the drawings for illustrative purposes. It should be understood that many other implementations of the concepts of the present disclosure are possible, and that various advantages may be achieved by implementing the concepts of the present disclosure. Headings are included herein for reference and to aid in locating various sections. The headings do not limit the scope of the concepts described therein. Such concepts may be applied throughout the entire specification.

[0012] A. Robotic System - Cart. A robot-enabled medical system may be configured in a variety of ways depending on the particular procedure. FIG. 1 illustrates one embodiment of a cart-based robot-enabled system 10 arranged for diagnostic and / or therapeutic bronchoscopy procedures. During bronchoscopy, the robotic system 10 may include a cart 11 having one or more robotic arms 12 for delivering medical instruments, such as a steerable endoscope 13, which may be a procedure-specific bronchoscope for bronchoscopy, to a natural orifice access point (i.e., the mouth of a patient positioned on a table in this example) for delivering diagnostic and / or therapeutic tools. As shown, the cart 11 may be positioned adjacent to the patient's upper torso to provide access to the access point. Similarly, the robotic arms 12 may be actuated to position a bronchoscope relative to the access point. The arrangement of FIG. 1 may also be utilized when performing gastrointestinal (GI) procedures with a gastroscope, an endoscope specialized for GI procedures. FIG. 2 depicts an example embodiment of the cart in more detail.

[0013] With continued reference to FIG. 1, once the cart 11 is properly positioned, the robotic arm 12 can insert the steerable endoscope 13 into the patient robotically, manually, or a combination thereof. As shown, the steerable endoscope 13 can include at least two nested parts, such as an inner leader section and an outer sheath section, each section coupled to a separate instrument driver from a set of instrument drivers 28, each instrument driver coupled to the distal end of an individual robotic arm. This linear arrangement of the instrument drivers 28, which facilitates coaxial alignment of the leader section with the sheath section, creates a "virtual rail" 29 that can be repositioned in space by manipulating one or more robotic arms 12 to different angles and / or positions. The virtual rails described herein are depicted in the figures using dashed lines, and thus do not depict any physical structure of the system. Translation of the instrument driver 28 along the virtual rail 29 nests the inner leader section relative to the outer sheath section, or advances or retracts the endoscope 13 from the patient. The angle of the virtual rail 29 may be adjusted, translated, or pivoted based on clinical application or physician preference. For example, in bronchoscopy, the angle and position of the virtual rail 29 shown represents a compromise between providing the physician access to the endoscope 13 and minimizing friction resulting from bending the endoscope 13 into the patient's mouth.

[0014] After insertion, the endoscope 13 may be directed downstream of the patient's trachea and lungs using precise commands from the robotic system until the targeted location or surgical site is reached. To enhance navigation through the patient's pulmonary network and / or to reach a desired target, the endoscope 13 may be manipulated to telescope the inner leader portion from the outer sheath portion to enhance articulation and increase bend radius. The use of a separate instrument driver 28 also allows the leader and sheath portions to be driven independently of one another.

[0015] For example, the endoscope 13 may be directed to deliver a biopsy needle to a target, such as a lesion or nodule in the patient's lung. The needle may be deployed through a working channel that runs the length of the endoscope to obtain a tissue sample that is analyzed by a pathologist. Depending on the results of the pathology, additional tools may be deployed through the working channel of the endoscope for additional biopsies. After identifying the nodule as malignant, the endoscope 13 may deliver tools endoscopically to ablate the potential cancerous tissue. In some cases, diagnostic and therapeutic procedures may need to be delivered in separate procedures. In these situations, the endoscope 13 may also be used to deliver fiducials to "mark" the location of the targeted nodule. In other cases, diagnostic and therapeutic procedures may be delivered during the same procedure.

[0016] The robotic system 10 may also include a movable tower 30 that may be connected to the cart 11 via a support cable to provide support for control, electronics, fluidics, optics, sensors, and / or power to the cart 11. Placing such functionality in the tower 30 may reduce the form factor of the cart 11, allowing the surgeon and his / her staff to more easily adjust and / or reposition the cart 11. Additionally, the division of functionality between the cart / table and the support tower 30 reduces clutter in the operating room and promotes improved clinical workflow. The cart 11 may be positioned near the patient, while the tower 30 may be housed in a remote location so as not to get in the way during the procedure.

[0017] To support the robotic system described above, the tower 30 may include computer-based control system components that store computer program instructions in a non-transitory computer-readable storage medium, such as, for example, a persistent magnetic storage drive, a solid-state drive, or the like. Execution of these instructions, whether execution occurs in the tower 30 or in the cart 11, may control the entire system or subsystems thereof. For example, when executed by a processor in a computer system, the instructions may cause the robotic system components to actuate the carriage and arm mounts, operate the robotic arm, and control medical instruments. For example, in response to receiving control signals, motors in the joints of the robotic arm may position the arm in a particular pose.

[0018] The tower 30 may also include pumps, flow meters, valve controls, and / or fluid access to provide controlled irrigation and aspiration capabilities to a system that may be deployed through the endoscope 13. These components may also be controlled using the computer system of the tower 30. In some embodiments, irrigation and aspiration capabilities may be provided directly to the endoscope 13 via separate cables.

[0019] The tower 30 may include voltage and surge protection designed to provide filtered, protected power to the cart 11, thereby avoiding the need to place power transformers and other auxiliary power components within the cart 11 and making the cart 11 smaller and more mobile.

[0020] The tower 30 may also include support equipment for sensors deployed throughout the robotic system 10. For example, the tower 30 may include optoelectronic equipment for detecting, receiving, and processing data received from optical sensors or cameras throughout the robotic system 10. In combination with a control system, such optoelectronic equipment may be used to generate real-time images for display on any number of consoles deployed throughout the system, including in the tower 30. Similarly, the tower 30 may also include electronic subsystems for receiving and processing signals from deployed electromagnetic (EM) sensors. The tower 30 may also be used to house and position EM field generators for detection by EM sensors in or on a medical instrument.

[0021] The tower 30 may also include a console 31 in addition to other consoles available to the rest of the system, e.g., a console mounted on top of a cart. The console 31 may include a user interface and a display screen, such as a touch screen, for the operator, the physician. The console of the robotic system 10 is typically designed to provide both robot control and pre-operative and real-time information of the procedure, such as navigation and localization information for the endoscope 13. If the console 31 is not the only console available to the physician, a second operator, such as a nurse, may use the console 31 to monitor the patient's health or vital signs and the operation of the system, as well as provide procedure-specific data, such as navigation and localization information. In other embodiments, the console 31 is housed in a body separate from the tower 30.

[0022] The tower 30 may be coupled to the cart 11 and endoscope 13 via one or more cables or connections (not shown). In some embodiments, support functions from the tower 30 may be provided to the cart 11 through only one cable, simplifying and decluttering the operating room. In other embodiments, certain functions may be combined in separate wiring and connections. For example, power may be provided to the cart through only one power cable, while support for the controls, optics, fluidics, and / or navigation may be provided through separate cables.

[0023] FIG. 2 provides a detailed illustration of one embodiment of a cart from the cart-based robot-enabled system shown in FIG. 1. The cart 11 generally includes an elongated support structure 14 (often referred to as a "column"), a cart base 15, and a console 16 at the top of the column 14. The column 14 may include one or more carriages, such as a carriage 17 (alternatively an "arm support") for supporting the deployment of one or more robotic arms 12 (three are shown in FIG. 2). The carriage 17 may include individually configurable arm mounts that rotate along orthogonal axes to adjust the base of the robotic arms 12 for better positioning relative to the patient. The carriage 17 also includes a carriage interface 19 that allows the carriage 17 to translate vertically along the column 14.

[0024] The carriage interface 19 is connected to the column 14 through slots, such as slots 20 positioned on either side of the column 14 to guide the vertical translation of the carriage 17. The slots 20 contain the vertical translation interfaces to position and hold the carriage at various vertical heights relative to the cart base 15. The vertical translation of the carriage 17 allows the cart 11 to adjust the reach of the robotic arm 12 to meet various table heights, patient sizes, and physician preferences. Similarly, an individually configurable arm mount on the carriage 17 allows the robotic arm base 21 of the robotic arm 12 to be angled in various configurations.

[0025] In some embodiments, a slot cover may be added to the slot 20 that is flush and parallel with the slot surface to prevent dirt and fluids from entering the internal chamber of the column 14 and the vertical translation interface as the carriage 17 translates vertically. The slot cover may be deployed through a pair of spring spools positioned near the vertical top and bottom of the slot 20. The cover is coiled within the spools until it deploys to extend and retract from the coiled state as the carriage 17 translates vertically up and down. The spring loading of the spools provides a force to retract the cover onto the spool as the carriage 17 translates towards the spool, while also maintaining a seal as the carriage 17 translates away from the spool. The cover may be connected to the carriage 17 using, for example, a bracket at the carriage interface 19 to ensure that the cover extends and retracts properly as the carriage 17 translates.

[0026] Column 14 may contain mechanisms therein, such as gears and motors, designed to use a vertically aligned leadscrew to mechanically translate carriage 17 in response to control signals generated in response to user input, such as input from console 16.

[0027] The robotic arm 12 may generally include a robotic arm base 21 and an end effector 22 separated by a series of links 23 connected by a series of joints 24, each joint including an independent actuator, each actuator including an independently controllable motor. Each separately controllable joint represents an independent degree of freedom available to the robotic arm. Each of the arms 12 has seven joints, thus providing seven degrees of freedom. A large number of joints provides a large number of degrees of freedom, allowing for "redundant" degrees of freedom. The redundant degrees of freedom allow the robotic arm 12 to position its respective end effector 22 at a specific position, orientation, and trajectory in space using different link positions and joint angles. This allows the system to position and orient the medical instrument from a desired point in space, while allowing the physician to move the arm joints to a clinically advantageous position away from the patient to create greater access while avoiding arm collisions.

[0028] The cart base 15 balances the weight of the column 14, carriage 17, and arm 12 on the floor. Thus, the cart base 15 houses the heavier parts, such as electronics, motors, power supplies, as well as components that allow the cart to be both mobile and / or immobilized. For example, the cart base 15 includes casters 25 in the form of rollable wheels that allow the cart to be easily moved around the room prior to a procedure. Once in the proper position, the casters 25 can be immobilized using wheel locks to hold the cart 11 in place during a procedure.

[0029] The console 16 positioned at the vertical end of the column 14 allows for both a user interface and a display screen (or dual-purpose device, e.g., touch screen 26) for receiving user input to provide both pre-operative and intra-operative data to the physician user. Potential pre-operative data on the touch screen 26 may include pre-operative planning, navigation and mapping data derived from a pre-operative computed tomography (CT) scan, and / or notes from a pre-operative patient interview. Intra-operative data on the display may also include vital patient statistics such as respiration, heart rate, and / or pulse, along with optical information provided from tools, sensor information from sensors, and coordinate information. The console 16 may be positioned and tilted to allow the physician to access the console from the side of the column 14 opposite the carriage 17. From this position, the physician can view the console 16, the robotic arm 12, and the patient while operating the console 16 from behind the cart 11. As shown, the console 16 also includes a handle 27 to help manipulate and stabilize the cart 11.

[0030] FIG. 3 illustrates one embodiment of the robot-enabled system 10 positioned for ureteroscopy. In a ureteroscopy procedure, the cart 11 may be positioned to deliver a ureteroscope 32, a procedure-specific endoscope designed to pass through the patient's urethra and ureters, to the patient's lower abdominal region. In a ureteroscopy, it may be desirable for the ureteroscope 32 to be aligned directly with the patient's urethra to reduce friction and forces on sensitive anatomical structures in the area. As shown, the cart 11 may be aligned to a table leg to allow the robotic arm 12 to position the ureteroscope 32 for direct linear access to the patient's urethra. From the table leg, the robotic arm 12 may insert the ureteroscope 32 along a virtual rail 33 directly through the urethra and into the patient's lower abdomen.

[0031] After insertion into the urethra using control techniques similar to those in bronchoscopy, the ureteroscope 32 may be navigated to the bladder, ureters, and / or kidneys for diagnostic and / or therapeutic applications. For example, the ureteroscope 32 may be directed to the ureter and kidney to break up formed kidney stones using a laser or ultrasonic lithotripsy device deployed down the working channel of the ureteroscope 32. After stone fragmentation is complete, the resulting stone fragments can be removed using a basket that deploys the ureteroscope 32.

[0032] FIG. 4 illustrates an embodiment of a robotic-enabled system similarly positioned for a vascular procedure. In a vascular procedure, the robotic system 10 may be configured such that the cart 11 may deliver a medical instrument 34, such as a steerable catheter, to an access point in the femoral artery in the patient's leg. The femoral artery provides both a larger diameter for navigation and a less circuitous and tortuous path to the patient's heart, which simplifies navigation. As in a ureteroscopy procedure, the cart 11 may be oriented and positioned at the patient's leg and lower abdomen to allow the robotic arm 12 to provide direct linear access to the femoral artery access point in the patient's thigh / hip region on the virtual rail 35. After insertion into the artery, the medical instrument 34 may be oriented and inserted by translating the instrument driver 28. Alternatively, the cart may be positioned around the patient's upper abdomen to reach alternative vascular access points, such as the carotid and arm arteries near the shoulder and wrist.

[0033] B. Robot system-table. An embodiment of a robot-enabled medical system may also incorporate a patient table. The incorporation of a table reduces the amount of capital equipment in the operating room by removing the cart and allows better access to the patient. FIG. 5 illustrates one embodiment of such a robot-enabled system deployed for a bronchoscopy procedure. The system 36 includes a support structure or column 37 for supporting a platform 38 (shown as a "table" or "bed") across the floor. Much like a cart-based system, the end effector of the robotic arm 39 of the system 36 includes an instrument driver 42 that is designed to manipulate an elongated medical instrument, such as the bronchoscope 40 of FIG. 5, through or along a virtual rail 41 formed from the linear alignment of the instrument driver 42. In practice, by placing emitters and detectors around the table 38, a C-arm for providing fluoroscopic imaging may be positioned across the patient's upper abdominal region.

[0034] FIG. 6 provides an alternative view of the system 36 without the patient and medical instruments for discussion purposes. As shown, the column 37 may include one or more carriages 43, shown as a ring shape, in the system 36, which may be the base for one or more robotic arms 39. The carriages 43 may translate along a vertical column interface 44 that runs the length of the column 37 to provide different vantage points from which the robotic arms 39 may be positioned to reach the patient. The carriages 43 may rotate about the column 37 using a mechanical motor positioned within the column 37 to allow the robotic arms 39 to access multiple sides of the table 38, such as both sides of the patient. In embodiments having multiple carriages, the carriages may be positioned separately on the column and may translate and / or rotate independently of the other carriages. The carriages 43 do not have to surround the column 37 or even be circular, although the ring shape shown facilitates rotation of the carriages 43 about the column 37 while maintaining structural balance. Rotation and translation of the carriage 43 allows the system to align medical instruments such as endoscopes and laparoscopes to different access points on the patient. In other embodiments (not shown), the system 36 may include a patient table or patient bed having an adjustable arm support in the form of a bar or rail extending alongside it. One or more robotic arms 39 may be attached to the adjustable arm support that can be adjusted vertically (e.g., via a shoulder with an elbow joint). By providing vertical adjustment, the robotic arms 39 may advantageously be stored compactly under the patient table or patient bed and then raised during a procedure.

[0035] The arm 39 may be attached to the carriage via a set of arm mounts 45, which include a series of joints that may individually rotate and / or telescope to provide additional configurability to the robotic arm 39. Additionally, the arm mounts 45 may be positioned on the carriage 43 such that when the carriage 43 is appropriately rotated, the arm mounts 45 may be positioned on either the same side of the table 38 (as shown in FIG. 6), on opposite sides of the table 38 (as shown in FIG. 9), or on adjacent sides of the table 38 (not shown).

[0036] The column 37 structurally provides support for the table 38 and a path for the vertical translation of the carriage. Internally, the column 37 may be equipped with a lead screw for guiding the vertical translation of the carriage and a motor for mechanizing the translation of that carriage based on the lead screw. The column 37 may transmit power and control signals to the carriage 43 and to a robotic arm 39 attached thereto.

[0037] The table base 46 functions similarly to the cart base 15 of the cart 11 shown in FIG. 2, housing the heavier components to counterbalance the table / bed 38, column 37, carriage 43, and robotic arm 39. The table base 46 may also incorporate rigid casters to provide stability during the procedure. The casters extend from the bottom of the table base 46 in opposite directions on either side of the base 46 and may be retracted when the system 36 needs to be moved.

[0038] Continuing with FIG. 6, the system 36 may also include a tower (not shown) that divides the functionality of the system 36 between the table and the tower to reduce the form factor and bulk of the table. As seen in the previously disclosed embodiments, the tower may provide various support functions to the table, such as processing, computing, and control capabilities, power, fluidics, and / or optical and sensor processing. The tower may also be mobile for positioning away from the patient to improve physician access and keep the operating room clutter-free. Furthermore, placing components in the tower allows for more storage space in the table base for possible accommodation of a robotic arm. The tower may also include a master controller or console that provides both a user interface for user input, such as a keyboard and / or pendant, and a display screen (or touch screen) for pre-operative and intra-operative information, such as real-time imaging, navigation, and tracking information. In some embodiments, the tower may also include a holder for a gas tank used for ventilation.

[0039] In some embodiments, the table base may house and store the robot arm when not in use. Figure 7 illustrates a system 47 for housing the robot arm in one embodiment of a table-based system. In the system 47, the carriage 48 may be vertically translated into the base 49 to house the robot arm 50, arm mount 51, and carriage 48 within the base 49. The base cover 52 may be translated and retracted open to deploy the carriage 48, arm mount 51, and arm 50 about a column 53, and closed to store and protect them when not in use. The base cover 52 may be sealed with a membrane 54 along the edge of its opening to prevent dirt and fluid ingress when closed.

[0040] FIG. 8 illustrates one embodiment of a robotic table-based system configured for a ureteroscopy procedure. For ureteroscopy, the table 38 may include a swivel 55 for positioning the patient at an off angle from the column 37 and table base 46. The swivel 55 may rotate or pivot about a pivot point (e.g., located below the patient's head) to position the bottom of the swivel 55 away from the column 37. For example, pivoting the swivel 55 may allow a C-arm (not shown) to be positioned over the patient's lower abdomen without competing for space with the column (not shown) below the table 38. By rotating the carriage (not shown) about the column 37, the robotic arm 39 may insert the ureteroscope 56 directly into the patient's groin area along the virtual rail 57 to reach the urethra. In ureteroscopy, stirrups 58 may also be secured to swivel section 55 of table 38 to support the position of the patient's legs throughout the procedure and allow clear access to the patient's groin area.

[0041] In a laparoscopic procedure, a minimally invasive instrument may be inserted into a patient's anatomy through a small incision in the patient's abdominal wall. In some embodiments, the minimally invasive instrument includes an elongated rigid member, such as a shaft, that is used to access an anatomical structure within the patient. After distension of the patient's abdominal cavity, the instrument may be oriented to perform a surgical or medical task, such as grasping, cutting, ablation, suturing, etc. In some embodiments, the instrument may include a scope, such as a laparoscope. FIG. 9 illustrates one embodiment of a robotic-enabled table-based system configured for a laparoscopic procedure. As illustrated in FIG. 9, the carriage 43 of the system 36 may be rotated and adjusted vertically to position a pair of robotic arms 39 on either side of the table 38 such that the instrument 59 may be positioned using arm mounts 45 to pass through a minimal incision on either side of the patient to reach the patient's abdominal cavity.

[0042] To accommodate laparoscopic procedures, the robotic-enabled table system may also tilt the platform to a desired angle. FIG. 10 illustrates one embodiment of a robotic-enabled medical system with pitch or tilt adjustment. As shown in FIG. 10, the system 36 accommodates the tilt of the table 38 to position one portion of the table higher off the floor than another portion. Additionally, the arm mount 45 may rotate to match the tilt such that the arm 39 maintains the same planar relationship as the table 38. To accommodate steeper angles, the column 37 may also include a telescoping portion 60 that allows for vertical extension of the column 37 to prevent the table 38 from contacting the floor or colliding with the base 46.

[0043] FIG. 11 provides a detailed illustrative view of the interface between the table 38 and the column 37. The pitch mechanism 61 may be configured to vary the pitch angle of the table 38 relative to the column 37 in multiple degrees of freedom. The pitch mechanism 61 may be enabled by positioning orthogonal axes 1, 2 at the column-table interface, each actuated by a separate motor 3, 4 in response to an electrical pitch command. Rotation along one screw 5 would allow tilt adjustment in one axis 1, while rotation along the other screw 6 would allow tilt adjustment along the other axis 2. In some embodiments, the use of ball joints allows the pitch angle of the table 38 relative to the column 37 to be varied in multiple degrees of freedom.

[0044] For example, pitch adjustment is particularly useful when attempting to position the table in the Trendelenburg position, i.e., positioning the patient's lower abdomen higher off the floor than the patient's upper / higher abdomen for lower abdominal surgery. The Trendelenburg position allows gravity to slide the patient's internal organs down to the patient's upper abdomen, emptying the abdominal cavity as minimally invasive tools enter to perform lower abdominal surgical or medical procedures, such as laparoscopic prostatectomy.

[0045] 12 and 13 show isometric and end views of an alternative embodiment of a table-based surgical robotic system 100. The surgical robotic system 100 includes one or more adjustable arm supports 105 that can be configured to support one or more robotic arms relative to a table 101 (see, e.g., FIG. 14). In the illustrated embodiment, only one adjustable arm support 105 is shown, but additional arm supports can be provided on the opposite side of the table 101. The adjustable arm support 105 can be configured to move relative to the table 101 to adjust and / or change the position of the adjustable arm support 105 and / or any robotic arm attached thereto relative to the table 101. For example, the adjustable arm support 105 can be adjusted with one or more degrees of freedom relative to the table 101. The adjustable arm support 105 provides the system 100 with great versatility, including the ability to easily accommodate one or more adjustable arm supports 105 and any robotic arms attached thereto under the table 101. The adjustable arm support 105 can be raised from a stowed position to a position below the top surface of the table 101. In other embodiments, the adjustable arm support 105 can be raised from a stowed position to a position above the top surface of the table 101.

[0046] The adjustable arm support 105 can provide multiple degrees of freedom, including lift, lateral translation, tilt, etc. In the illustrated embodiment of FIGS. 12 and 13, the arm support 105 is configured with four degrees of freedom, which are indicated by arrows in FIG. 12. The first degree of freedom allows adjustment of the adjustable arm support 105 in the z-direction ("Z-lift"). For example, the adjustable arm support 105 can include a carriage 109 configured to move up and down along or relative to the column 102a that supports the table 101. The second degree of freedom can allow the adjustable arm support 105 to tilt. For example, the adjustable arm support 105 can include a rotational joint, which can allow the adjustable arm support 105 to align with the bed in the Trendelenburg position. The third degree of freedom allows the adjustable arm support 105 to "pivot up," which can be used to adjust the distance between the side of the table 101 and the adjustable arm support 105. A fourth degree of freedom may allow translation of the adjustable arm support 105 along the longitudinal length of the table.

[0047] The surgical robotic system 100 of Figures 12 and 13 may include a table supported by a column 102a mounted to a base 103. The base 103 and the column 102a support the table 101 against a support surface. A bed axis 131 and a support axis 133 are shown in Figure 13.

[0048] An adjustable arm support 105 can be mounted to the column 102a. In other embodiments, the arm support 105 can be mounted to the table 101 or the base 103. The adjustable arm support 105 can include a carriage 109, a bar or rail connector 111, and a bar or rail 107. In some embodiments, one or more robot arms mounted to the rail 107 can translate and move relative to each other.

[0049] The carriage 109 can be attached to the column 102a by a first joint 113, which allows the carriage 109 to move relative to the column 102a (e.g., up and down a first or vertical axis 123). The first joint 113 can provide a first degree of freedom ("Z-lift") to the adjustable arm support 105. The adjustable arm support 105 can include a second joint 115 that provides a second degree of freedom (tilt) to the adjustable arm support 105. The adjustable arm support 105 can include a third joint 117a that can provide a third degree of freedom ("pivot up") to the adjustable arm support 105. A further joint 119 (shown in FIG. 13) can be provided that mechanically constrains the third joint 117a to maintain the orientation of the rail 107 as the rail connector 111 rotates about a third axis 127. The adjustable arm support 105 may include a fourth joint 121 that may provide a fourth degree of freedom (translation) to the adjustable arm support 105 along a fourth axis 129 .

[0050] FIG. 14 illustrates an end view of a surgical robotic system 140A with two adjustable arm supports 105A, 105B mounted on either side of a table 101. A first robotic arm 142A is attached to a bar or rail 107A of the first adjustable arm support 105B. The first robotic arm 142A includes a base 144A attached to the rail 107A. The distal end of the first robotic arm 142A includes an instrument drive mechanism 146A that can be attached to one or more robotic medical instruments or tools. Similarly, the second robotic arm 142B includes a base 144B attached to the rail 107B. The distal end of the second robotic arm 142B includes an instrument drive mechanism 146B. The instrument drive mechanism 146B can be configured to attach to one or more robotic medical instruments or tools.

[0051] In some embodiments, one or more of the robotic arms 142A, 142B comprise arms with seven or more degrees of freedom. In some embodiments, one or more of the robotic arms 142A, 142B can include eight degrees of freedom, including an insertion axis (one degree of freedom including insertion), wrist (three degrees of freedom including wrist pitch, yaw and roll), elbow (one degree of freedom including elbow pitch), shoulder (two degrees of freedom including shoulder pitch and yaw), and base 144A, 144B (one degree of freedom including translation). In some embodiments, the insertion degree of freedom can be provided by the robotic arms 142A, 142B, while in other embodiments, the instrument itself provides the insertion via an instrument-based insertion architecture.

[0052] C. Instrument drivers and interfaces. The end effector of the robotic arm of the system includes (i) an instrument driver (alternatively referred to as an "instrument drive mechanism" or "instrument device manipulator") that incorporates electromechanical means for actuating the medical instrument, and (ii) a removable or detachable medical instrument that may lack any electromechanical components such as a motor. This dichotomy may be caused by the need to sterilize medical instruments used in medical procedures and the inability to adequately sterilize expensive capital equipment due to the complex mechanical assembly and sensitive electronics of medical instruments. Thus, medical instruments may be designed to be detached, removed, and replaced from the instrument driver (and thus the system) upon individual sterilization or disposal by a physician or physician's staff. In contrast, the instrument driver does not need to be replaced or sterilized and may be draped for protection.

[0053] FIG. 15 illustrates an exemplary instrument driver. The instrument driver 62, positioned at the distal end of the robotic arm, is comprised of one or more drive units 63 arranged with parallel axes to provide a controlled torque to the medical instrument via a drive shaft 64. Each drive unit 63 includes an individual drive shaft 64 for interacting with the instrument, a gearhead 65 for converting motor shaft rotation to a desired torque, a motor 66 for generating the drive torque, an encoder 67 for measuring the rotational speed of the motor shaft and providing feedback to the control circuit, and a control circuit 68 for receiving a control signal to operate the drive unit. Each drive unit 63 is controlled and motorized independently of the others, and the instrument driver 62 may provide multiple (four shown in FIG. 15) independent drive outputs to the medical instrument. In operation, the control circuit 68 would receive the control signal, send a motor signal to the motor 66, compare the resulting motor rotational speed measured by the encoder 67 to a desired speed, and modulate the motor signal to generate the desired torque.

[0054] For procedures requiring a sterile environment, the robotic system may incorporate a drive interface, such as a sterile adapter connected to a sterile drape, located between the instrument driver and the medical instrument. The primary purpose of the sterile adapter is to transfer angular motion from the drive shaft of the instrument driver to the drive input of the instrument while maintaining physical separation between the drive shaft and the drive input, and thus sterility. Thus, an example sterile adapter may consist of a set of rotational inputs and outputs intended to mate with the drive shaft of the instrument driver and the drive input to the instrument. The sterile drape connected to the sterile adapter is composed of a thin flexible material, such as a transparent or translucent plastic, and is designed to cover the instrument driver, the robot arm, and capital equipment, such as a cart (in a cart-based system) or a table (in a table-based system). The use of the drape allows the capital equipment to be positioned in close proximity to the patient while still being located in an area that does not require sterility (i.e., the non-sterile field). On the other side of the sterile drape, the medical instrument may interface with the patient in an area that requires sterility (i.e., the sterile field).

[0055] D. Medical equipment. 16 illustrates an exemplary medical instrument with a paired instrument driver. Similar to other instruments designed for use with a robotic system, the medical instrument 70 includes an elongated shaft 71 (or elongated body) and an instrument base 72. The instrument base 72, also referred to as an "instrument handle" due to its design intended for manual interaction by a physician, may include a rotary drive input 73, e.g., a receptacle, pulley, or spool, designed to mate with a drive output 74 that passes through a drive interface on an instrument driver 75, typically at the distal end of a robotic arm 76. When physically connected, latched, and / or coupled, the mating drive input 73 of the instrument base 72 may share an axis of rotation with the drive output 74 in the instrument driver 75, allowing for the transfer of torque from the drive output 74 to the drive input 73. In some embodiments, the drive output 74 may include a spline designed to mate with a receptacle on the drive input 73.

[0056] The elongated shaft 71 is designed to be delivered through either an anatomical opening or lumen, such as in an endoscope, or a minimally invasive incision, such as in a laparoscopy. The elongated shaft 71 can be either flexible (e.g., having properties similar to an endoscope) or rigid (e.g., having properties similar to a laparoscope), or can include a customized combination of both flexible and rigid sections. If designed for laparoscopy, the distal end of the rigid elongated shaft can be connected to an end effector extending from an articulating wrist formed from a clevis having at least one degree of freedom, and to a surgical tool or medical instrument, such as a grasper or scissors, that can be actuated based on a force from a tendon as the drive input rotates in response to a torque received from the drive output 74 of the instrument driver 75. If designed for endoscopy, the distal end of the flexible elongated shaft can include a steerable or controllable bend that can be articulated and bent based on a torque received from the drive output 74 of the instrument driver 75.

[0057] Torque from the instrument driver 75 is transmitted downstream of the elongated shaft 71 using tendons along the shaft 71. These individual tendons, such as pull wires, can be individually secured to individual drive inputs 73 in the instrument handle 72. From the handle 72, the tendons pass along the elongated shaft 71 through one or more pull lumens and are secured to a distal portion of the elongated shaft 71 or are secured to a wrist at the distal portion of the elongated shaft. During a surgical procedure, such as a laparoscopic, endoscopic, or hybrid procedure, these tendons can be coupled to a distally mounted end effector, such as a wrist, grasper, or scissors. In such an arrangement, torque exerted on the drive input 73 would transmit tension to the tendons, thereby actuating the end effector in some manner. In some embodiments, the tendons can rotate a joint about an axis to move the end effector in one direction or another during a surgical procedure. Alternatively, the tendon may be connected to one or more jaws of a grasper at the distal end of the elongate shaft 71 such that tension from the tendon causes the grasper to close.

[0058] In endoscopy, the tendons may be coupled to a bend or articulating section positioned along (e.g., at the distal end) of the elongate shaft 71 via adhesive, control rings, or other mechanical fixation. When fixedly attached to the distal end of the bend section, torque exerted on the drive input 73 is transferred to the tendons, causing the softer bend section (sometimes referred to as the articulating section or articulating region) to bend or articulate. Along the non-bend section, it may be convenient to helical or spiral the individual pull lumens that direct the individual tendons along (or inwardly) the wall of the endoscope shaft to counterbalance the radial forces resulting from tension in the pull wires. The angle of the helix and / or spacing between them may be altered or engineered for specific purposes, with narrower helices exhibiting poor shaft compression under load forces, while lesser helices provide superior shaft compression under load forces, but also limited bending. At the other end of the spectrum, orienting the pull lumen parallel to the longitudinal axis of the elongate shaft 71 can allow for controlled articulation at the desired bend or articulation.

[0059] In endoscopy, the elongated shaft 71 houses several components that aid in robotic procedures. The elongated shaft may be configured with a working channel for deploying surgical tools (or medical instruments), irrigation, and / or suction to a surgical site at the distal end of the shaft 71. The elongated shaft 71 may also house wires and / or optical fibers that carry signals at the distal tip to / from an optical assembly that may include an optical camera. The shaft 71 may also house optical fibers for carrying light from a proximally located light source, such as a light emitting diode, to the distal end of the shaft.

[0060] At the distal end of the instrument 70, the distal tip may include a working channel opening for delivering tools to the surgical site for diagnosis and / or treatment, irrigation, and aspiration. The distal tip may also include a port for a camera, such as a fiberscope or digital camera, to capture images of the internal anatomical space. In this regard, the distal tip may also include a port for a light source to illuminate the anatomical space when the camera is in use.

[0061] 16, the drive shaft axis, and therefore the drive input axis, is orthogonal to the axis of the elongated shaft. However, this arrangement complicates the roll ability of the elongated shaft 71. Rolling the elongated shaft 71 along its axis while holding the drive input 73 stationary results in undesirable entanglement of the tendons as they exit the drive input 73 and enter the pull lumen within the elongated shaft 71. Such resulting entanglement of tendons may frustrate any control algorithms aimed at predicting the movement of a flexible elongated shaft during an endoscopic procedure.

[0062] FIG. 17 illustrates an alternative design of the instrument driver and instrument, in which the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument. As shown, a circular instrument driver 80 includes four drive units with their drive outputs 81 aligned in parallel at the end of a robotic arm 82. The drive units and their respective drive outputs 81 are housed in a rotating assembly 83 of the instrument driver 80 that is driven by one of the drive units in that assembly 83. In response to torque provided by the rotating drive units, the rotating assembly 83 rotates along a circular bearing that connects the rotating assembly 83 to a non-rotating portion 84 of the instrument driver. Power and control signals may be conveyed from the non-rotating portion 84 of the instrument driver 80 to the rotating assembly 83 through electrical contacts and may be maintained throughout the rotation by a brushed slip ring connection (not shown). In other embodiments, the rotating assembly 83 may be responsive to a separate drive unit that is integrated into the non-rotatable portion 84 and thus not parallel to the other drive units. The rotation mechanism 83 enables the instrument driver 80 to rotate the drive units and their respective drive outputs 81 as a single unit about the instrument driver axis 85 .

[0063] Similar to the previously disclosed embodiments, the instrument 86 may include an elongated shaft portion 88 and an instrument base 87 (shown with a transparent exterior skin for purposes of discussion) that includes a number of drive inputs 89 (such as receptacles, pulleys, and spools) configured to receive the drive outputs 81 in the instrument driver 80. Unlike the previously disclosed embodiments, the instrument shaft 88 extends from the center of the instrument base 87, with the axis being substantially parallel to the axis of the drive inputs 89, rather than orthogonal as seen in the design of FIG.

[0064] When coupled to the rotating assembly 83 of the instrument driver 80, the medical instrument 86, including the instrument base 87 and the instrument shaft 88, rotates with the rotating assembly 83 about the instrument driver axis 85. Because the instrument shaft 88 is positioned in the center of the instrument base 87, the instrument shaft 88 is coaxial with the instrument driver axis 85 when attached. Rotation of the rotating assembly 83 therefore causes the instrument shaft 88 to rotate about its own longitudinal axis. Also, because the instrument base 87 rotates with the instrument shaft 88, any tendons connected to the drive input 89 at the instrument base 87 do not become entangled during rotation. Thus, the parallelism of the axes of the drive output 81, drive input 89, and instrument shaft 88 allows shaft rotation without entangling any of the control tendons.

[0065] FIG. 18 illustrates an instrument having an instrument-based insertion architecture, according to some embodiments. The instrument 150 can be coupled to any of the instrument drivers discussed above. The instrument 150 includes an elongated shaft 152, an end effector 162 connected to the elongated shaft 152, and a handle 170 coupled to the elongated shaft 152. The elongated shaft 152 includes a tubular member having a proximal portion 154 and a distal portion 156. The elongated shaft 152 includes one or more channels or grooves 158 along its outer surface. The grooves 158 are configured to receive one or more wires or cables 180. Thus, the one or more cables 180 run along the outer surface of the elongated shaft 152. In other embodiments, the cables 180 can run through the elongated shaft 152. Manipulation of one or more of the cables 180 (e.g., via an instrument driver) results in actuation of the end effector 162.

[0066] The instrument handle 170, sometimes referred to as the instrument base, may typically include a mounting interface 172 having one or more mechanical inputs 174, e.g., receptacles, pulleys or spools, designed to intermate with one or more torque couplers on a mounting surface of the instrument driver.

[0067] In some embodiments, instrument 150 includes a series of pulleys or cables that allow elongated shaft 152 to translate relative to handle 170. In other words, instrument 150 itself constitutes an instrument-based insertion architecture that accommodates the insertion of the instrument, thereby minimizing reliance on a robotic arm to effect the insertion of instrument 150. In other embodiments, the robotic arm may be significantly involved in the insertion of the instrument.

[0068] E. Controller. Any of the robotic systems described herein can include an input device or controller for manipulating an instrument attached to the robotic arm. In some embodiments, the controller can be coupled (e.g., communicatively, electrically, wirelessly, and / or mechanically) to the instrument such that manipulation of the controller causes a corresponding manipulation of the instrument, e.g., via master-slave control.

[0069] FIG. 19 is a perspective view of one embodiment of the controller 182. In this embodiment, the controller 182 includes a hybrid controller that can have both impedance control and admittance control. In other embodiments, the controller 182 can only utilize impedance control or passive control. In other embodiments, the controller 182 can only utilize admittance control. Being a hybrid controller advantageously allows the controller 182 to have a lower perceived inertia during use. In the embodiment shown, the controller 182 is configured to allow for the operation of two medical instruments. As shown, the controller 182 can include two handles 184. In some embodiments, the pair of handles 184 operates a single instrument, while in other embodiments, each of the pair of handles 184 operates its own corresponding instrument. Each handle 184 is connected to a gimbal 186. Each gimbal is connected to a positioning platform 188. In some embodiments, the handles 184 are considered distal from the gimbal 186, which is considered distal to the positioning platform 188. The handle 184 and gimbal 186 are shown in greater detail in FIG. 22 and described below.

[0070] As illustrated in FIG. 19 , in the illustrated embodiment, each positioning platform 188 includes a selectively compliant assembly robotic arm 198 (SCARA arm) having multiple links coupled to a column 194 by a prismatic joint 196. The prismatic joint 196 is configured to translate along the column 194 (e.g., along a rail 197) to translate the handle 184 in the z-direction, providing a first degree of freedom. The SCARA arm 198 is configured to allow movement of the handle 184 in the xy plane, providing an additional two degrees of freedom. Thus, each of the positioning platforms 188 shown in FIG. 19 is configured to provide three positional or translational degrees of freedom, allowing an operator to position the handle 184 anywhere in three-dimensional (e.g., x, y, z) space (within the reach of the positioning platform).

[0071] In some embodiments, the column 194 (and rail 197) extends along an axis that is aligned with the vertical direction of the workspace (e.g., the z-direction as shown), which may be aligned with the direction of gravity. An advantage of this positioning platform 188 is that it can provide gravity compensation. In other words, the prismatic joint 196 of the positioning platform 188 can maintain a constant orientation of the gimbal 106 relative to gravity.

[0072] In some embodiments, the positioning platform 188 can have other configurations. For example, the positioning platform 108 need not include a prismatic joint and / or a SCARA arm in all embodiments.

[0073] In some embodiments, a load cell (not shown in FIG. 19 ) may be provided within a portion of the controller 182 (e.g., within the gimbal 186, etc.). The addition of the load cell allows the controller to have admittance control in addition to impedance control. Under admittance control, the perceived inertia of the controller 182 may be reduced. This is because the mass of the gimbal 186 and / or positioning platform may be hidden through the load cell. This may be because the load cell measures the force applied to the controller and may be used to drive motors within the controller 182 to provide outputs that aid in the operation of the controller 182. The amount of hidden mass depends on the location of the load cell. In some embodiments, the mass proximal to the load cell may be partially or substantially hidden, while the mass distal to the load cell is not hidden.

[0074] In some embodiments, by positioning the load cell distally on the controller 182 (e.g., in the gimbal 186 shown in FIG. 19), the mass of the gimbal 186 can be partially or substantially hidden while operating the controller 182. Similarly, the mass of the positioning platform 188 (having a relatively higher mass than the gimbal 186) can also be partially or substantially hidden while operating the controller 182. The hidden mass advantageously results in a lower inertia perceived by the clinician. Without the load cell, to move the handle 184 in the z-direction, the operator would provide sufficient force to the handle 184 to lift the handle 184, the gimbal 186, and the SCARA arm 198 upward. Furthermore, it can be assumed that less force is required to move the handle in the xy plane than to move it in the z-direction. This parallax would likely result in a non-uniform operating experience for the operator making the controller 182 difficult to use. Thus, by including load cells as described herein, the controller 182 can assist the user in translating the handle 184 in the x, y, and z directions, providing a much more uniform and controlled manipulation experience. In some embodiments, the load cells enable the positioning platform 188 to operate substantially or completely under admittance control. In contrast to the positioning platform 188, the moment of inertia of the gimbal 186 can be relatively low. This may be because the gimbal 186 is generally much smaller than the positioning platform 188. For this reason, at least a portion of the gimbal 186 may be suitable for impedance control.

[0075] One advantage of the hybrid impedance / admittance controller 182 as described herein is that the perceived inertia of the system may be relatively lower than a system that relies entirely on impedance control. Furthermore, the mechanical structure of the hybrid controller 182 may be simpler because admittance control can be used to compensate and average the movements of the system. In contrast, the mechanical structure of an impedance-only system is often very complex as it attempts to normalize the forces to move the system in different directions and minimize the perceived inertia.

[0076] In some embodiments, by using a hybrid controller 182 as described herein, it is possible to actually increase the mass and inertia of the gimbal 186 relative to that of an impedance-only controller, since much of the total mass and inertia of the controller 182 can be hidden by admittance control of the positioning platform. Increasing the size of the gimbal can, in some embodiments, enable the use of larger motors, which can enable the controller to provide stronger perceived feedback forces when compared to other systems that must use lighter gimbals and motors to avoid increasing overall mass and inertia.

[0077] As illustrated in FIG. 19, the hybrid controller 182 can be viewed as multiple links and joints in series, for example as a serial-link manipulator. The handle 184, gimbal 186, and positioning platform 188 each comprise one or more operatively coupled links, with the most proximal link adjacent a column 194 of the positioning platform 188 and the most distal link being part of the handle 184 itself. In some embodiments, one or more load cells 112 (not shown in FIG. 19) can be inserted into the controller 182 to provide admittance control of at least a portion of the controller 182. Other portions of the controller 182 can be controlled by impedance control (or in some cases, passive control) by a clinician or operator. In some embodiments, the links and joints proximal to the load cells can be directly or indirectly influenced by the load cells. Thus, manipulation of these proximal links and joints can assist in admittance control. In some embodiments, the links and joints distal to the load cells may not be directly or indirectly influenced by the load cells. Thus, the manipulation of these distal links and joints can be assisted by impedance control. For example, in some embodiments, the distal joints 128, 130, 132 (shown in FIG. 22) are positioned within the gimbal 186 such that they cannot be directly or indirectly influenced by the load cells. In other words, the manipulation of the axes of the gimbal 186 at these joints is not based directly or indirectly on the output of the load cells. These distal links and joints can be moved by impedance control. In contrast, the links and joints proximal to the load cells (such as those within the positioning platform 188) can be directly or indirectly influenced by the load cells. In other words, the manipulation of the axes at these joints is based directly or indirectly on the output of the load cells. These proximal links and joints can be moved by admittance control.

[0078] In some embodiments, one or more load cells are positioned in the controller. For example, in some embodiments, a load cell (not shown) is positioned on the body of each of the gimbals 106. Providing load cells allows a portion of the controller 182 to operate under admittance control, thereby advantageously reducing the perceived inertia of the controller during use. In some embodiments, the positioning platform 108 is configured for admittance control while the gimbal 106 is configured for impedance control. In other embodiments, the positioning platform 108 is configured for impedance control while the gimbal 106 is configured for admittance control. Thus, in some embodiments, the translational or positional degree of freedom of the positioning platform 108 can rely on admittance control, while the rotational degree of freedom of the gimbal 106 is determined by impedance control.

[0079] F. Navigation and Control. Traditional endoscopy may involve the use of fluoroscopy (e.g., as may be delivered through a C-arm) and other forms of radiation-based imaging modalities to provide intraluminal guidance to the operator-physician. In contrast, the robotic system contemplated by the present disclosure may provide non-radiation-based navigation and localization means to reduce the physician's exposure to radiation and reduce the amount of equipment in the operating room. As used herein, the term "localization" may refer to determining and / or monitoring the position of an object in a reference coordinate system. Techniques such as preoperative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to achieve a radiation-free surgical environment. In other cases where radiation-based imaging modalities are still used, preoperative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to improve information available only through radiation-based imaging modalities.

[0080] FIG. 20 is a block diagram illustrating a localization system 90 for estimating the position of one or more elements of a robotic system, such as the position of an instrument, according to an exemplary embodiment. The localization system 90 may be a set of one or more computing devices configured to execute one or more instructions. The computing devices may be embodied by a processor (or processors) and computer readable memory in one or more of the components discussed above. By way of example, and not limitation, the computing devices may be in the tower 30 shown in FIG. 1, the cart shown in FIGS. 1-4, or the bed shown in FIGS. 5-14.

[0081] 20, the localization system 90 may include a localization module 95 that processes the input data 91-94 to generate position data 96 of the distal tip of the medical instrument. The position data 96 may be data or logic that represents the location and / or orientation of the distal tip of the instrument relative to a frame of reference. The frame of reference may be relative to the patient's anatomy or relative to a known object such as an EM field generator (see discussion of EM field generators below).

[0082] The various input data 91-94 will now be described in more detail. Pre-operative mapping can be accomplished through the use of acquisition of low-dose CT scans. The pre-operative CT scans are reconstructed into three-dimensional images that are visualized, for example, as cutaway "slices" of the patient's internal anatomy. When analyzed as a whole, an image-based model may be generated that covers the anatomical cavities, anatomical spaces, and anatomical structures of the patient's anatomy, such as the patient's pulmonary network. Techniques such as centerline geometry can be determined and approximated from the CT images to create a three-dimensional volume of the patient's anatomy, referred to as model data 91 (also referred to as "pre-operative model data" when generated using only pre-operative CT scans). The use of centerline geometry is discussed in U.S. Patent Application Serial No. 14 / 523,760, the contents of which are incorporated herein in their entirety. Network topological models can also be derived from CT images and are particularly suited for bronchoscopy.

[0083] In some embodiments, the instrument may be equipped with a camera to provide visual data 92. A localization module 95 may process the visual data to enable one or more vision-based location tracking. For example, pre-operative model data may be used in conjunction with the visual data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or an instrument advancing through a working channel of an endoscope). For example, using the pre-operative model data 91, the robotic system may generate a model-based predicted endoscopic image library based on the expected path of travel of the endoscope, with each image linked to a location in the model. During surgery, this library may be referenced by the robotic system to compare real-time images captured by a camera (e.g., a camera at the distal end of the endoscope) to those in the image library to aid in localization.

[0084] Other computer vision based tracking techniques use feature tracking to determine the motion of the camera and therefore the endoscope. Some features of the localization module 95 may identify a circular geometry in the pre-operative model data 91 that corresponds to an anatomical lumen and track changes in that geometry to ascertain which anatomical lumen has been selected and the relative rotational and / or translational motion of the camera. The use of a phase map may further enhance vision-based algorithms or techniques.

[0085] Optical flow, another computer vision based technique, may analyze the displacement and translation of image pixels in a video sequence in the visual data 92 to infer camera movement. Examples of optical flow techniques may include motion detection, object segmentation computations, luminance, motion compensated coding, stereo disparity measurements, etc. By comparing multiple frames over multiple iterations, the movement and location of the camera (and therefore the endoscope) may be determined.

[0086] The localization module 95 can use real-time EM tracking to generate a real-time position of the endoscope in a global coordinate system that can be registered to the patient's anatomy represented by the pre-operative model. In EM tracking, an EM sensor (or tracker), consisting of one or more sensor coils embedded in a medical instrument (e.g., an endoscopic instrument) at one or more locations and orientations, measures the variations in the EM field produced by one or more static EM field generators positioned at known locations. The location information detected by the EM sensor is stored as EM data 93. The EM field generator (or transmitter) can be placed near the patient to produce a low-intensity magnetic field that can be detected by the embedded sensor. The magnetic field induces a small current in the sensor coil of the EM sensor, which can be analyzed to determine the spacing and angle between the EM sensor and the EM field generator. This spacing and orientation can be "registered" intraoperatively to the patient's anatomy (e.g., the pre-operative model) to determine a geometric transformation that aligns a location in the pre-operative model of the patient's anatomy with a single location in the coordinate system. Once registered, an EM tracker embedded at one or more locations on the medical instrument (e.g., the distal tip of an endoscope) can provide a real-time indication of the medical instrument's progression through the patient's anatomy.

[0087] The robotic commands and kinematic data 94 may also be used by a localization module 95 to provide localization data 96 for the robotic system. During pre-operative calibration, the device pitch and yaw resulting from the articulation commands may be determined. Intra-operatively, these calibration measurements may be used in combination with known insertion depth information to estimate the position of the instrument. Alternatively, these calculations may be analyzed in combination with EM, vision, and / or phase modeling to estimate the position of the medical instrument within the network.

[0088] As Fig. 20 illustrates, several other input data may be used by the localization module 95. For example, although not shown in Fig. 15, an instrument utilizing shape sensing fibers may provide shape data that the localization module 95 can use to ascertain the position and shape of the instrument.

[0089] The localization module 95 may use a combination of the input data 91-94. In some cases, such a combination may use a probabilistic approach in which the localization module 95 assigns confidence weights to the location determined from each of the input data 91-94. Thus, if the EM data is unreliable (such as in the presence of EM interference), which may reduce the reliability of the location confirmed by the EM data 93, the localization module 95 may rely more heavily on the vision data 92 and / or the robot command and kinematics data 94.

[0090] As discussed above, the robotic systems discussed herein can be designed to incorporate one or a combination of two or more of the above-mentioned technologies. The computer-based control system of the tower, bed, and / or cart-based robotic system may store computer program instructions in a non-transitory computer-readable storage medium, such as, for example, a persistent magnetic storage drive, a solid-state drive, etc., that, when executed, cause the system to receive and analyze sensor data and user commands, generate system-wide control signals, and display navigation and localization data, such as instrument position within a global coordinate system, anatomical maps, etc.

[0091] 2. Controller for robot-enabled remote operation systems. A robot-enabled teleoperation system, such as the system described above, may include an input device or controller configured to enable an operator (e.g., a physician performing a robot-enabled medical procedure) to manipulate and control one or more instruments (or a robotic arm holding one or more instruments). In some embodiments, the robot-enabled teleoperation system comprises a controller for manipulating one or more medical tools. Those skilled in the art will appreciate that the controllers described herein may also be applied in non-medical situations. For example, the controller may be useful for manipulating tools involving hazardous materials. Additionally, in some embodiments, the controllers described herein may be useful for grasping objects in a physical and / or virtual environment. In some embodiments, the controller may be self-sufficient as a service robot that interacts with a human operator. In some embodiments, the controller may be coupled (e.g., communicatively, electronically, electrically, wirelessly, and / or mechanically) with an instrument (e.g., a medical instrument, etc.) such that manipulation of the controller causes a corresponding manipulation of the instrument. In some embodiments, the controller and the instrument are arranged in a master-slave pair. In some embodiments, the controller may be referred to as a manipulator, emulator, master, interface, etc. In some embodiments, the controller may include multiple links assembled in parallel or in series.

[0092] The controller can serve as an input device for an operator to control the operation of a medical instrument, such as an endoscopic, intraluminal, laparoscopic, or open surgical instrument. Movement of the controller by the operator can guide movement of the medical instrument. For example, when the operator translates the controller in three-dimensional space (e.g., up, down, left, right, back, forward), the system can cause a corresponding translational movement of the medical instrument. Similarly, when the operator rotates the controller (e.g., about any of three orthogonal axes), the system can cause a corresponding rotational movement of the medical instrument. The controller can also be configured to receive inputs that enable the operator to actuate the medical instrument. As an example, if the medical instrument includes a grasper, the controller can receive inputs that enable the operator to open and close the grasper.

[0093] The controller may also provide sensory feedback to the operator. For example, in some embodiments, a force or torque applied to the medical instrument may be transmitted back to the operator via the controller. In some embodiments, providing sensory feedback to the operator via the controller provides an improved operation, control, or driving experience to the user. In some embodiments, haptic cues may be provided to facilitate the operator's interaction with the controller and operating the system.

[0094] In some embodiments, the controller is also used to align the operator's hand with the orientation of the medical instrument, for example, when switching between medical instruments. For example, when the medical instrument is positioned within a patient during a medical procedure, it is important that the medical instrument does not move unexpectedly or unintentionally. Thus, if the operator wishes to take control of a medical instrument already positioned within a patient, the controller can first be moved to match the orientation of the medical instrument while the instrument remains in place. With the controller correctly oriented to match the orientation of the medical instrument, the operator can use the controller to operate the medical instrument.

[0095] In some embodiments, the robot-enabled medical system includes a controller with seven degrees of freedom that tracks the movements of the operator's hand, including three positional degrees of freedom (e.g., translational movements in x, y, z space), three rotational degrees of freedom (rotational movements about pitch, roll, and yaw axes), and one (or more) instrument actuation degrees of freedom (e.g., angular degrees of freedom). In some embodiments, the instrument actuation degrees of freedom can control the opening and closing of an end effector of a medical instrument, such as a gripper or grasper for holding an object. In some embodiments, the instrument actuation degrees of freedom may be omitted. In some embodiments, the controller may include a greater or lesser number of degrees of freedom. For example, in some embodiments, the controller may include more than three positional degrees of freedom or more than three rotational degrees of freedom to provide one or more redundant degrees of freedom. In some embodiments, the redundant degrees of freedom can provide the controller with additional mechanical flexibility, for example, to avoid singularities caused by the mechanical structure of the controller.

[0096] 21A shows a block diagram of an embodiment of the robotic enabled medical system 100, including a schematic diagram of an embodiment of a controller 302 and a schematic diagram of an embodiment of a robotic enabled medical instrument 310. As briefly mentioned above, the controller 302 can be coupled (e.g., mechanically, electrically, or functionally) to the robotic enabled medical instrument 310 such that manipulation of the controller 302 causes a substantially corresponding movement of the robotic enabled medical instrument 310, and forces applied to the robotic enabled medical instrument 310 can be transmitted back to the controller and perceptibly communicated to an operator. In some embodiments, the controller 302 and the robotic enabled medical instrument 310 are arranged in a master-slave configuration.

[0097] In the illustrated embodiment of the system 100, the controller 302 includes a handle 104, a gimbal 106, and a positioning platform 108. The handle 104 may be configured to be held by an operator. As shown, in some embodiments, the handle 104 is coupled to the gimbal 106 and the positioning platform 108. As described above, the handle 104 may include one or more degrees of freedom for actuating the instrument. The gimbal 106 may be configured to provide one or more rotational degrees of freedom that allow the operator to rotate the handle 104. In some embodiments, the gimbal 106 is configured to provide at least three rotational degrees of freedom. For example, the gimbal 106 may be configured to allow the operator to rotate the handle 104 about pitch, roll, and yaw axes. An exemplary gimbal 106 is described in more detail with respect to FIGS. 17-19B. The positioning platform 108 may be configured to provide one or more translational (also referred to herein as positional) degrees of freedom to allow an operator to translate the handle 104. In some embodiments, the positioning platform 108 is configured to provide at least three positional degrees of freedom. For example, the positioning platform 108 may be configured to allow an operator to translate the handle 104 in three-dimensional space (e.g., in the x-, y-, and z-directions). An exemplary positioning platform 108 is described in more detail with reference to FIG. 19. Together, the gimbal 106 and the positioning platform 108 may allow a user to manipulate the handle 104.

[0098] In the illustrated embodiment, the robot-enabled medical instrument 310 includes an instrument or tool 312 (which may include an end effector), an instrument driver 314, and a robotic arm 316 (or other instrument positioning device). The medical tool 312 may be, for example, a laparoscopic instrument 59 shown in FIG. 9 above, as well as other types of endoscopic or laparoscopic medical instruments described throughout this application and apparent to one of ordinary skill in the art. The medical tool 312 may include an end effector or multiple end effectors. The end effector may be positioned at a distal end of the medical tool 312. The end effector may be configured for insertion into a patient's body. In some embodiments, the end effector may be a grasper, gripper, cutter, basket device, or scissors, among others. In some embodiments, the medical tool 312 may include a scope or a camera.

[0099] The medical tool 312 may be attached to an instrument driver 314. The instrument driver 314 may be configured to actuate the medical tool 312 as described above. For example, the instrument driver 314 may be configured to pull one or more pull wires of the medical tool 312 to actuate the medical tool 312. In some embodiments, the instrument driver 314 may be an instrument drive mechanism as described above. The instrument driver 314 may be attached to a robotic arm 316, for example, as illustrated in FIG. 14. The robotic arm 316 may be configured to articulate or move to further manipulate and position the medical tool 312. Exemplary medical instruments / tools, instrument drivers, and robotic arms are shown in the systems of FIGS. 1-15 above.

[0100] The controller 302 may be coupled to the robotic enabled medical instrument 310 such that manipulation of the handle 104 causes a substantially corresponding movement of the medical tool 312 and forces applied to the medical tool 312 may be perceptibly transmitted to the operator through the handle 104. Manipulation of the handle 104 may be measured or determined by measuring forces and movements of the gimbal 106 and positioning platform 108. Movement of the medical tool 312 may be caused by articulation and movement of the instrument driver 314 and / or the robotic arm 316. Thus, by manipulating the handle 104, the operator may control the medical tool 312.

[0101] As described below, in some embodiments, the controllers described herein are configured to operate with both admittance and impedance control. These and other features of the controllers are discussed further in the following sections.

[0102] A. Hybrid controller. 21B is a block diagram of an embodiment of a controller 302 configured to operate using both impedance control and admittance control. Such a controller 302 may be referred to as a hybrid controller. However, in some embodiments, a controller configured to operate using only impedance control (without admittance control) or only admittance control (without impedance control) may be used.

[0103] Impedance control and admittance control are two control schemes for controlling robotic systems. Under impedance control, the system measures displacement (e.g., change in position and velocity) and outputs a force. For example, with impedance control, the system can measure how far or fast the operator has moved the controller and, based on that measurement, generate a force on the controller (e.g., by actuating a motor). Under impedance control, the operator's movement of the controller can back drive a portion of the instrument. In many cases, the use of impedance control can result in a large perceived inertia. This may be because, for example, impedance control relies on the operator moving the controller. Under impedance control, the operator overcomes its perceived mass or inertia to move the controller, making the controller feel heavy. In impedance control, the operator physically overcomes most or all of the inertia in the system to move the controller. Some controllers rely solely on impedance control.

[0104] Under admittance control, the system measures the force and / or torque applied by the operator to the controller and outputs the corresponding velocity and / or position of the controller. In some respects, admittance control is the inverse of impedance control. In some embodiments, the use of admittance control can result in a reduction in the perceived inertia or mass of the system. Admittance control can be used to change the dynamics of a controller that is perceived to have high mass or inertia. In some cases, by using admittance control, the operator does not need to overcome all of the inertia in the system to move the controller. For example, under admittance control, when a user applies a force to the controller, the system can measure the force and assist the user in moving the controller by driving one or more motors associated with the controller, thereby resulting in a desired velocity and / or position of the controller. In other words, in admittance control, a force sensor or load cell measures the force the operator is applying to the controller and moves the controller as well as the coupled robotic enabled medical instrument 310 in a light-sensing manner. Admittance control can feel lighter than impedance control because under admittance control, motors in the controller can help accelerate the mass, hiding the perceived inertia of the controller In contrast, with impedance control, the user is responsible for all or substantially all of the mass acceleration.

[0105] As shown in the illustrated embodiment of FIG. 21B, the controller 302 includes a handle 104, a gimbal 106, and a positioning platform 108. As described above, the gimbal 106 can be configured to provide one or more rotational degrees of freedom (e.g., three or four), and the positioning platform 108 can be configured to provide one or more rotational degrees of freedom (e.g., three or four). The gimbal 106 and positioning platform 108 allow a user to move the handle 104 in three-dimensional space and rotate the handle 104 about pitch, roll, and yaw axes. Manipulation of the handle 104 results in a corresponding movement of the medical instrument. Additionally, the handle 104, gimbal 106, and positioning platform 108 can be configured to provide sensory feedback to the operator representative of forces applied to the medical instrument.

[0106] As shown by the dashed box in FIG. 21B, in some embodiments, in the controller 302, the gimbal 106 is configured for impedance control and the positioning platform 108 is configured for admittance control. Thus, in some embodiments, the translational or positional degree of freedom of the positioning platform 108 relies on admittance control and the rotational degree of freedom of the gimbal 106 relies on impedance control. As described further below, this type of hybrid controller 302 can have certain advantages. In other embodiments, the gimbal 106 is configured for admittance control while the positioning platform 108 is configured for impedance control. In some embodiments, the gimbal 106 and the positioning platform may both be configured for admittance control or both be configured for impedance control.

[0107] To utilize admittance control, the controller 302 includes at least one force sensor or load cell 112. The load cell 112 is configured to measure a force applied by an operator to the controller 302 (typically a force applied to the handle 104). The output signal of the load cell 112 (a measure of the force) is used to provide a control signal that controls the movement of the controller 302, such as the positioning platform 108. The robotic enabled medical instrument 310 follows the movement of the handle 104 (e.g., by actuating one or more motors in the instrument driver 314 or the robotic arm 316). In some embodiments, the load cell 112 can be a three degree of freedom load cell that measures force in three directions.

[0108] In the embodiment shown, the load cell 112 is positioned within the gimbal 106. FIG. 22, described below, illustrates different positions in which the load cell 112 may be positioned within the gimbal 106. Other positions of the load cell 112 are possible. In some embodiments, the load cell 112 is positioned within the positioning platform 108. In some embodiments, more than one load cell 112 (e.g., two, three, four, or more load cells) are included and may be positioned within the handle 104, the gimbal 106, and / or the positioning platform 108.

[0109] In some embodiments, the load cell 112 is positioned distally (closer to the handle 104) within the controller 302. In some embodiments, this allows admittance control to be used to hide the perceived mass of portions of the controller 302 proximal to the load cell 112 (e.g., portions of the controller 302 on the opposite side of the handle 104 from the load cell 112).

[0110] As mentioned above, in some embodiments, the load cell 112 (or force sensor) is positioned within the gimbal 106. In some embodiments, the gimbal 106 provides a rotational degree of freedom for the controller 302 with impedance control, while the positioning platform 108 provides a positional degree of freedom for the controller 302 with admittance control (e.g., based on the output of the load cell 112 positioned within the gimbal 106). There are many ways in which the load cell 112 can be positioned within the gimbal 106. The degree to which the perceived inertia of the controller 302 is reduced can be based in part on the position of the load cell 112 within the gimbal 106. Two exemplary embodiments showing the load cell 112 positioned in two different parts of the gimbal 106 are described in this section. Other embodiments are also possible.

[0111] 22 is an isometric view of one embodiment of the gimbal 106. As shown, in some embodiments, the gimbal 106 is positioned at the distal end of the positioning platform 108 (only the last link of the positioning platform 108 is shown in FIG. 22). As used in this application, in the context of the controller 302, the term distal refers to the direction toward the handle 104 (e.g., the handle 104 is the distal-most component of the controller 302) and the term proximal refers in the opposite direction. Thus, the proximal end of the gimbal 106 may be attached to the distal end of the positioning platform 108. Additionally, the handle 104 may be positioned at the distal end of the gimbal 106.

[0112] In some embodiments, the handle 104 is configured to be held by an operator. The handle 104 may be configured to simulate or mimic the medical instrument that the controller 302 is used to control. In some embodiments, the handle includes a grasper handle (e.g., a radially symmetric grasper handle), a stylus, a paddle-type handle, or the like. In the embodiment shown, the handle 104 includes two actuation arms 120 configured to provide the instrument actuation degrees of freedom discussed above. While holding the handle 104, the operator can adjust the angle between the actuation arms 120 to control a corresponding angle associated with the controlled medical instrument. For example, if the medical instrument is a grasper, shear, or the like, the angle between the actuation arms 120 can be used to control the angle between the two jaws of the grasper.

[0113] In the embodiment shown, the gimbal 106 comprises three arms or links connected by joints. Arranged from distal to proximal, and as shown in FIG. 22, the gimbal 106 comprises a first link 122, a second link 124, and a third link 126. Arranged from distal to proximal, and as shown in FIG. 22, the gimbal 106 comprises a first joint 128, a second joint 130, a third joint 132, and a fourth joint 134. The joints allow the various links to rotate, providing the gimbal 106 with the rotational degrees of freedom discussed above.

[0114] The handle 104 is connected to a distal end of the first link 122 by a first joint 128. The first joint 128 may be configured to allow the handle 104 to rotate relative to the first link 122. In the embodiment shown, the first joint 128 allows the handle 104 to rotate about a roll axis 136. In some embodiments, the roll axis 136 is aligned with a longitudinal axis of the handle 104. The first joint 128 may be an external rotation joint.

[0115] The proximal end of the first link 122 is connected to the distal end of the second link 124 by a second joint 130. The second joint 130 may be configured to allow the handle 104 and the first link 122 to rotate relative to the second link 124. In the illustrated embodiment, the second joint 130 allows the handle 104 and the first link 122 to rotate about a yaw axis 138. In some embodiments, the yaw axis 138 extends through the second joint 130 and intersects with the roll axis 136 at a center point of the handle 104. The second joint 130 may be an external rotation joint. As shown, in some embodiments, the first link 122 includes an L-shape. In some embodiments, the first link 122 has a recess formed therein to receive the second link 124 and is configured to allow the second link 124 to rotate relative to the first link 122.

[0116] The proximal end of the second link 124 is connected to the distal end of the third link 126 by a third joint 132. The third joint 132 may be configured to allow the handle 104, the first link 122, and the second link 124 to rotate relative to the third link 126. In the illustrated embodiment, the third joint 132 allows the handle 104, the first link 122, and the second link 124 to rotate about a pitch axis 139. In some embodiments, the pitch axis 139 extends through the third joint 132 and intersects with the roll axis 136 and the yaw axis 138 at a center point of the handle 104. The third joint 132 may be an external rotation joint. As illustrated, in some embodiments, the second link 124 includes an L-shape. In some embodiments, the L-shaped second link 124 is received within a recess of the L-shaped first link 122 (as illustrated in FIG. 17 ). In other embodiments, the L-shaped first link 122 may be received within a recess in the L-shaped second link 124 .

[0117] In the embodiment shown, the first joint 128, the first link 122, the second joint 130, the second link 124, and the third joint 132 provide three rotational degrees of freedom, allowing the rotation of the handle 104 to be adjusted in pitch, roll, and yaw. In the embodiment shown, the gimbal 106 further includes a third link 126 and a fourth joint 134 that provide redundant rotational degrees of freedom. This need not be included in all embodiments, but can provide the gimbal 106 with greater mechanical flexibility.

[0118] As shown, the distal end of the third link 126 is connected to the proximal end of the second link 124 by a third joint 132. The proximal end of the third link 126 is connected to the distal end of the positioning platform 108 by a fourth joint 134. The fourth joint 134 can be configured to allow the handle 104, the first link 122, the second link 124, and the third link 126 to rotate relative to the positioning platform 108. In the embodiment shown, the fourth joint 134 allows the handle 104, the first link 122, the second link 124, and the third link 126 to rotate about an axis 141. In some embodiments, the axis 141 is parallel to the yaw axis 138. In some embodiments, the yaw axis 138 and the axis 141 are coaxial, but as shown, this need not be the case in all embodiments. The axis 141 (and the yaw axis 138) are parallel to the direction of gravity and can maintain the gimbal orientation relative to the direction of gravity as described above. The fourth joint 134 can be an external rotation joint. As shown, in some embodiments, the third link 126 includes an L-shape.

[0119] A haptic interface device (HID) is a device that provides haptic feedback and includes any of the controllers described herein for controlling a robotic system, a robotic arm, and / or an instrument. In some embodiments, the input sensor is separate from the haptic interface device. In some embodiments, the HID includes a haptic feedback device and an input device (e.g., a gripping implement such as the handle 104) configured to be held by an operator. In some embodiments, the HID includes components that are designed or selected to minimize mechanical dissipative effects such as friction and damping. In this manner, the HID is designed to be transparent to the user, such that the user does not experience much resistance or impedance when moving the HID in free space, allowing the user to complete a surgical task with minimal strain and distraction imposed by the HID.

[0120] FIG. 23 illustrates a control system 200 for providing haptic feedback to an operator, according to some embodiments. The control system 200 receives patient-side feedback 202 from the motion and / or state of one or more robotic arms 12 and receives HID feedback 204 from the motion and / or state of one or more input devices (e.g., controller 302). Both patient-side feedback 202 and HID feedback 204 are provided to an impedance control unit 206. In some embodiments, the patient-side feedback 202 includes information indicative of an event, state, or action at least partially associated with the medical instrument 310 (e.g., tool motion mismatch, collision between two medical instruments, such as a "sword fight," collision between robotic arms, collision between a robotic arm and the environment, such as a collision between a robotic arm and a stationary component in the operating room, a fault, etc.). In some embodiments, the HID feedback 204 includes information indicative of an event, state, or action associated with one or more input devices (e.g., controller 302), such as variable damping, gimbal navigation, gimbal collision, ease of holding during re-clutching, a fault, etc. In some embodiments, the patient feedback 202 and / or the HID feedback 204 are generated (or provided or updated) based on input signals from one or more input sensors (e.g., sensors such as encoders coupled to one or more input devices such as the controller 102, 302). In some embodiments, the impedance control unit 206 includes two modules: a roll haptic module 208 and a haptic buzz module 210. In some embodiments, the haptic buzz module 210 further includes two components: an event type classifier 212 and a profile generator 214.

[0121] In some embodiments, the outputs from both the roll haptic module 208 and the haptic buzz module 210 are combined in a signal controller 216 (also referred to as a signal combiner) and delivered to the HID. In some embodiments, the output from the haptic buzz module 210 includes a vibrotactile feedback signal. In some embodiments, the output from the roll haptic module 208 includes a kinesthetic haptic feedback signal. For example, the output control signal from the signal controller 216 is provided to a component of the HID (e.g., an actuator or its controller). In some embodiments, the component is a torque / current control loop 218 of the HID, which includes the gimbal 106. In some embodiments, the torque / current control loop 218 controls a motor of the gimbal 106. In some embodiments, the motor is a gimbal roll motor of the gimbal 106.

[0122] In some embodiments, the patient side feedback 202 is generated (or provided or updated) when an event of a first set of predetermined events (e.g., tool motion mismatch) is detected by (or based on) one or more input sensors (e.g., the patient side feedback 202 includes information indicative of the event of the first set of predetermined events to provide kinesthetic tactile feedback). Tool motion mismatch occurs when a master command signal (e.g., control signal) received by the robotic arm 12 results in an output motion of the robotic arm 12 (or coupled surgical tool) that deviates from a desired motion corresponding to the master command signal (e.g., the robotic arm 12 moves more than specified by the master command / control signal, the robotic arm 12 moves less than specified by the control signal, the robotic arm 12 moves at a speed slower by a predetermined threshold than specified by the control signal, the robotic arm 12 moves in a direction different from that specified by the control signal, etc.). In some embodiments, a tool motion mismatch is detected when the processor of the robotic system 10 determines, based on one or more input sensors in the robotic arm 12 (e.g., encoders at or near one or more motors of the robotic arm 12, one or more inertial sensors, etc.), that the current position of the robotic arm deviates from a predicted position based on control signals sent to the robotic arm 12. In response to the processor detecting or determining an event of a first set of predetermined events (e.g., tool motion mismatch), an output signal is sent to the roll haptic module 208 of the impedance control unit 206 as patient side feedback 202. In some embodiments, the patient side feedback 202 generates kinesthetic haptic feedback.

[0123] In some embodiments, the patient side feedback 202 is generated (or provided or updated) when an event of the second set of predetermined events (e.g., a "sword fight" is about to occur or has occurred) is detected by (or based on) one or more input signals (e.g., the patient side feedback 202 includes information indicative of an event of the second set of predetermined events to provide vibrotactile feedback). In some embodiments, the second set of predetermined events is mutually exclusive with the first set of predetermined events. An instrument "sword fight" occurs when an instrument shaft is within a threshold distance (e.g., less than 10 cm, less than 5 cm, less than 2 cm, or less than 1 cm) to a different instrument shaft or a collision occurs between two instrument shafts. In some embodiments, the instrument shaft is part of a medical instrument that is mechanically grasped directly or indirectly by the robotic arm 12 (e.g., via the instrument driver 28). For example, an instrument shaft of a first medical instrument held and controlled by a first robotic arm may be within or less than a threshold distance to an instrument shaft of a second medical instrument held and controlled by a second robotic arm. In some embodiments, an instrument "fight" is detected when the processor of the robotic system 10 determines, based on one or more sensors in the first and second robotic arms (e.g., encoders at or near one or more motors of the robotic arms, one or more inertial sensors) and the dimensions of the first and second medical instruments, that the current position of the first medical instrument handled (or held) by the first robotic arm is within a threshold distance from the current position of the second medical instrument handled (or held) by the second robotic arm. When the processor detects the instrument "fight", an output signal is sent to the haptic buzz module 210 of the impedance control unit 206 as patient-side feedback 202. The patient-side feedback 202 generates vibrotactile feedback. In some embodiments, haptic technology that relies on tactile stimulation transmits mechanical stimulation to the operator's skin. The mechanical stimulation includes vibration stimulation, surface change stimulation, or friction change stimulation.In some embodiments, the tactile feedback generates sensations such as vibration, friction, or small deformations.

[0124] In some embodiments, a "sword fight" of the robotic arms is considered to have occurred when a first robotic arm is within a threshold distance (e.g., less than 10 cm, less than 5 cm, less than 2 cm, or less than 1 cm) of a different robotic arm or a collision occurs between two robotic arms. In some embodiments, a "sword fight" is considered to have occurred when a medical instrument held by a first robotic arm is within a threshold distance (e.g., less than 10 cm, less than 5 cm, less than 2 cm, or less than 1 cm) of a different robotic arm or a collision occurs between the medical instrument and a robotic arm.

[0125] In some embodiments, the patient side feedback 202 includes information indicative of an event in which the instrument shaft and / or robotic arm of the first medical instrument approaches or collides with a camera in a patient-body workspace (for monitoring a portion of the patient-body workspace). The information may be used to provide vibrotactile feedback. In some embodiments, the patient-body workspace is a space within the patient. A collision or near collision between the instrument arm and the camera occurs when the instrument shaft and / or robotic arm is within a threshold distance (e.g., less than 10 cm, less than 5 cm, less than 2 cm, or less than 1 cm) to the camera or when a collision occurs between the camera and the instrument shaft and / or robotic arm.

[0126] In some embodiments, a collision (or an imminent collision) is detected by a processor of the robotic system 10 based on one or more input sensors in the robotic arm (e.g., encoders at or near one or more motors of the robotic arm, one or more inertial sensors) following a determination that the current position of the robotic arm and / or an instrument handled by the robotic arm is less than a threshold distance from the camera. Following the processor's detection of the collision or near-collision, an output signal is sent to the haptic buzz module 210 of the impedance control unit 206 as patient-side feedback 202.

[0127] In some embodiments, the patient feedback 202 includes information indicative of a fault condition (e.g., at least one of the fault conditions is detected) (e.g., to provide vibrotactile feedback). An example of a fault condition associated with the patient feedback 202 includes detecting excessive force (e.g., force above a force threshold) on the robot arm (e.g., caused by a collision between the robot arm and the environment, such as a patient, nurse, etc.).

[0128] In some embodiments, the HID feedback 204 includes information indicative of a level of variable damping provided by the HID during navigation of the gimbal 106. For example, based in part on how the HID is manipulated by the user, an appropriate level of variable damping resistance is applied to the HID generally, or to one or more joints of the HID (e.g., joints 128, 130, 132, and 134) to modify the force or torque provided by the one or more joints. The variable damping (or a change thereto) can be used to provide kinesthetic haptic feedback. For example, an increase in the level of damping provided as an input signal (e.g., HID feedback 204) to the roll haptics module 208 of the impedance control unit 206 can be indicated. In some embodiments, the impedance control unit 206 changes the level of damping from the current level of variable damping indicated by the HID feedback 204 provided to the impedance control unit 206 when a tool motion mismatch is determined based on information in the patient side feedback 202 to provide or modify the kinesthetic haptic feedback.

[0129] In some embodiments, the HID feedback 204 includes information indicating that a first gimbal approaches or collides with a second gimbal (e.g., the left gimbal in FIG. 19 collides with the right gimbal in FIG. 19 ) (e.g., to provide vibrotactile feedback). A collision or near collision between a first gimbal and a second gimbal occurs when the two gimbals are within or less than a first threshold distance apart (e.g., less than 20 cm, less than 10 cm, less than 5 cm, less than 2 cm, or a collision has occurred). In some embodiments, a collision between two gimbals is detected when a processor of the robotic system 10 determines, based on sensors in the first and second gimbals (e.g., encoders at or near one or more motors of the gimbals, one or more inertial sensors), that the current position of the first gimbal is within a first threshold distance from the current position of the second gimbal.

[0130] In some embodiments, the HID feedback 204 includes information indicative of the navigation (e.g., movement) of the gimbal (e.g., position and / or velocity), which may be compared to the movement of the medical instrument 310 to determine if a tool motion mismatch has occurred or if the tool motion mismatch has exceeded a predefined threshold (e.g., to provide kinesthetic tactile feedback). For example, when the gimbal is near the end of a range of motion for a particular degree of freedom, the HID feedback 204 indicative of the gimbal position or the proximity of the gimbal to the end of the range of motion is provided to the roll haptics module 208 of the impedance control unit 206 to provide kinesthetic tactile feedback (e.g., increased damping). In some embodiments, when the gimbal exceeds a velocity threshold during navigation, the HID feedback 204 including information indicative of the velocity of the gimbal or information indicating that the gimbal has exceeded the velocity threshold is provided to the roll haptics module 208 of the impedance control unit 206 to provide kinesthetic tactile feedback.

[0131] In some embodiments, information indicative of the navigation of the gimbals can be used to provide vibrotactile feedback. For example, when the gimbals reach the end of a range of movement for a particular degree of freedom, HID feedback 204 indicating that the gimbals have reached the end of the range of movement is provided to a haptic buzz module 210 of the impedance control unit 206 to provide vibrotactile feedback.

[0132] In some embodiments, the HID feedback 204 includes information indicating whether the user needs to relax their hold on the HID. For example, when the user over-articulates the gimbal beyond a certain threshold (e.g., after hitting a haptic boundary), the HID feedback 204 may include information indicating whether the user needs to relax their hold on the HID. When the processor detects that the user indicates whether they need to relax their hold on the HID, an output signal is sent from the HID feedback 204 to the haptic buzz module 210 of the impedance control unit 206. The HID feedback 204 generates vibrotactile feedback. The vibrotactile feedback provided in response to the information indicating whether the user needs to relax their hold on the HID informs the user that the gimbal has moved beyond the tool reach. In some cases, when the user's hold on the HID is relaxed, the system can (automatically) reorient the gimbal to match the tool pose. In some other cases, the user may need to reorient their hand position during a procedure, and pressing the clutch button can temporarily decouple the HID from controlling the operation of the instrument, thereby allowing the user to reorient their hand position and re-grip the HID during a procedure. When the user relaxes their grip, such a gesture can be interpreted by the HID or robotic system as returning to a clutch state (e.g., re-engaging the "clutch").

[0133] In some embodiments, the HID feedback 204 is generated when a fault condition is detected. Examples of fault conditions associated with the HID feedback 204 include detecting a collision of a gimbal (e.g., an armrest or other gimbal) or determining that a redundant joint cannot move (due to a collision).

[0134] In some embodiments, when the tactile buzz module 210 receives a signal from the patient feedback 202, a first type of tactile output is generated, and when a signal is received from the HID feedback 204, a second type of tactile output is generated. In some embodiments, the first type of tactile output is separate from the second type of tactile output. In some embodiments, the first type of tactile output has a first vibration frequency component (e.g., a frequency between 10-20 Hz) and the second type of tactile output has a second vibration frequency component (e.g., a frequency between 50-100 Hz, or between 100 Hz and 200 Hz, etc.) that is separate from the first vibration frequency component. In some embodiments, the first type of tactile output has a first vibration pattern (e.g., a sawtooth pattern) and the second type of tactile output has a second vibration pattern (e.g., a square wave pattern) that is separate from the first vibration pattern. In some embodiments, the first type of tactile output has a first duration (e.g., 0.4 seconds, 0.5 seconds, 0.6 seconds, 0.7 seconds, 0.8 seconds, 0.9 seconds, 1 second, or semi-continuous) and the second type of tactile output has a second duration (e.g., 0.1 seconds, 0.2 seconds, 0.3 seconds, etc.) separate from the first duration. In some embodiments, the first type of tactile output has a first number of chirps (3 chirps) and the second type of tactile output has a second number of chirps (e.g., 5 chirps) separate from the first number of chirps. A user may be able to more intuitively resolve ambiguities between the two types of tactile output and more quickly recognize problems occurring on either the patient side or the HID side.

[0135] When the system is operating, various feedbacks received from the patient side or the HID side can generate specific events that trigger either or both traditional haptic and buzz effect feedback.

[0136] In some embodiments, the impedance control unit 206 is implemented as one or more processors (e.g., microprocessors, application specific integrated circuits, etc.). For example, both the roll haptic module 208 and the haptic buzz module 210 may be implemented in a single processor or two separate processors. In some embodiments, the gimbal roll motor torque control 218 is implemented in the same processor as the impedance control unit 206 or in a separate processor. In some embodiments, the robotic system 10 includes one or more processors for processing the patient-side feedback 202 and / or the HID feedback 204. In some embodiments, circuitry for processing the patient-side feedback 202 and / or the HID feedback 204 is included in the one or more processors implementing the impedance control unit 206.

[0137] i.Roll haptic module In some embodiments, the roll haptics module 208 applies a damping function to the robot joints (e.g., the robot joints 24) to modify the resistance to the robot joints' motion (e.g., the damping function causes an increase in the resistance force or torque to the robot joints' motion by applying a damping coefficient). In some embodiments in which joint velocities are received or determined, the damping function applied to the robot joints may also be based on the current velocity. In some embodiments, the current position and / or current velocity of the robot joints is received or determined (e.g., by the control unit, by the impedance control 206) as the medical tool manipulated by the robot joints is moved in three-dimensional space. As mentioned above, each joint 24 may report its velocity and position (e.g., angular velocity and angular position) to the control unit. Additionally or alternatively, the joints 24 may report angular forces or torques imparted by or applied to the joints.

[0138] In some embodiments, the control unit (e.g., impedance control 206) then determines the distance between the current position of the robotic joint 24 (e.g., the rotational position of the robotic joint 24) and the first movement limit of the robotic joint (e.g., the joint's rotation distance). In some embodiments, the roll haptics module 208 implements a kinesthetic haptic technique that includes applying a force to the user that impedes limb movement. Kinesthetic haptic feedback is not typically felt on the skin, but mainly on the muscles and tendons. In some embodiments, the kinesthetic haptic feedback includes a continuous force applied in response to user input. In some embodiments, the kinesthetic haptic feedback includes a tactile sense of body movement and the ability to detect changes in body position and movement without relying on information from the five senses. In some embodiments, the kinesthetic haptic feedback mimics the actual behavior of the robotic arm.

[0139] The roll haptics module 208 provides as its output an output control signal that is used to generate kinesthetic haptic feedback in the HID.

[0140] ii. Haptic Buzz Module The haptic buzz module 210 receives input signals from the patient feedback 202 and / or the HID feedback 204. The input signals are sent to an event type classifier 212 within the haptic buzz 210. In some embodiments, the event type classifier 212 identifies the source of the received signal (e.g., from the patient feedback 202 or from the HID feedback 204) and the condition conveyed by the received signal (e.g., gimbal collision, malfunction, sword fighting, etc.) and provides an output signal to the profile generator 214. In some embodiments, the output signal is unique for each type of event or information. The profile generator 214 provides an output control signal corresponding to the identified type of event indicated in the output signal (e.g., the profile generator 214 may provide a unique output control signal for each identified type of event such that the HID can provide a unique haptic or tactile feedback indicative of the identified type of event). The output control signal is used to generate a tactile signal in the HID. In some embodiments, the user can determine the type or profile of tactile signals to be associated with each event type (e.g., fault from HID feedback 2014, failure from patient side feedback 202, gimbal collision, sword fight, medical instrument collision, camera collision, ease of hold during re-clutch, etc.) In some embodiments, the profile generator 214 automatically generates and assigns tactile signals for one or more of the event types.

[0141] The signal controller 216 generates a composite signal from the output control signal of the roll haptic module 208 and the output control signal of the haptic buzz module 210. In some embodiments, the signal controller 216 sums the output control signal from the roll haptic module 208 and the output control signal from the haptic buzz module 210. In some embodiments, the signal controller 216 sums a first weighting of the output control signal from the roll haptic module 208 (e.g., a fraction of the output control signal, a multiple of the output control signal) and a second weighting of the output control signal from the haptic buzz module 210 to generate a final output control signal used to control a portion of the robotic joint. In some embodiments, the portion of the robotic joint corresponds to a gimbal roll motor, and the final output signal is delivered to a torque control loop 218 of the gimbal roll motor for torque control.

[0142] FIG. 24 illustrates how output signals from the roll haptic module 208 and the haptic buzz module 210 are combined, according to some embodiments. A master gimbal impedance control 209, representing the output from the roll haptic module 208, is provided to the signal controller 216. In some embodiments, the master gimbal impedance control 209 is a non-periodic signal. A selected buzz profile 211, representing the output signal from the buzz haptic module 210, is also provided to the signal controller 216. In some embodiments, the selected buzz profile 211 is a periodic signal. In some embodiments, the selected buzz profile 211 is used to generate a physical vibration (e.g., a periodic physical vibration, such as alternating between high and low frequency vibrations, or alternating between providing vibration and not providing vibration). In some embodiments, the frequency of the physical vibration is at least 10 Hz (e.g., at least 50 Hz, or between 50 Hz and 200 Hz). In some embodiments, the frequency is between 4 and 10 Hz, or about 6 Hz. The frequency profile of the vibration makes the buzz effect easily identifiable by the user and allows information (indicative of the associated event) to be conveyed to the user in a separate and identifiable manner. The frequency of the physical vibration is selected to be far away from typical teleoperated master control (e.g., 5 Hz or less) and roll haptic (e.g., kinesthetic haptic) feedback, so that the vibration only provides vibrotactile haptic effects and does not affect existing operations. The waveform, amplitude, frequency, duration, repetition pattern, etc. may be further selected based on hardware mode analysis (e.g., hardware mode analysis may include determining different modes of vibration on the hardware and preventing or avoiding the use of excitation frequencies associated with such modes to avoid or reduce resonance) and various needs associated with haptic presentation for different applications.

[0143] In some embodiments, the buzz haptic or vibrotactile feedback has a varying amplitude (e.g., alternating high and low amplitude variations for events such as collisions or failures, and monotonically increasing or decreasing amplitude variations for events derived for excessive motion mismatch). In some embodiments, the signal controller 216 determines a weighted sum of the control signal from the master gimbal impedance control 209 and the output control signal from the selected buzz profile 211 (e.g., by adding a first weighted value (e.g., a fraction of the output control signal or a multiple of the output control signal) corresponding to the control signal from the master gimbal impedance control 209 and a second weighted value corresponding to the output control signal from the selected buzz profile 211) to generate a final output control signal used to control a portion of the robot joint. In some embodiments, the portion of the robot joint corresponds to a gimbal roll motor, and the final output signal is delivered to a torque control loop 218 for torque control of the gimbal roll motor.

[0144] Returning to FIG. 23, in some embodiments, the positioning platform 108 of the robotic system 10 includes a gimbal 106 at its distal end. The gimbal 106 further includes several motors. In some embodiments, a motor (e.g., G6) adjacent to the handle 104 (e.g., at the most distal end of the gimbal 106) receives a superimposed vibrotactile control signal in its torque / current control loop in addition to the kinesthetic haptic signal from the master gimbal impedance control to generate a buzz vibration on top of the kinesthetic / roll haptic signal. The G6 motor can provide a torque (e.g., rotational resistance) to the handle 104 (and the first joint 128) about the roll axis 136. In some embodiments, the roll axis 136 is aligned with the longitudinal axis of the handle 104. In some embodiments, providing the vibrotactile signal at the most distal end of the gimbal 106 makes it easier for the user to sense the vibrotactile signal. In some embodiments, the roll haptic signal causes the user to experience resistance (e.g., kinesthetic haptic feedback) when the user applies torque to the handle 104 about the roll axis 136. The gimbal includes other motors, such as motor G5, which provides a torque about the yaw axis 138, and motor G4, which provides a torque about the pitch axis 139.

[0145] Various buzz profiles can be used to alert a user to different events such as an impending physical collision between gimbals, between a gimbal and a positioning platform arm (e.g., the positioning platform 108 of a first gimbal collides with a second gimbal), between medical instruments, or between robotic arms. Vibration buzz profiles can also be used to alert a user to an impending physical collision between a gimbal and a console workspace boundary (including other components of the robotic system 10 such as a display, or any other object adjacent to the gimbal) (e.g., between the console 16 and the gimbal 106).

[0146] In some embodiments, the medical system includes a second haptic interface device that is separate from the first haptic interface device (eg, FIG. 19 shows two haptic interface devices).

[0147] 3. Variable damping for haptic interface control. In some embodiments, the kinesthetic haptic feedback includes the application of a torque (or force) by an actuator (e.g., a motor) to provide resistance to a user input. In some embodiments, roll haptics associated with the HID involves the application of variable damping during navigation and / or operation of the HID. In some embodiments, the difference in position between the master and slave is determined to provide a feedback force to the operator.

[0148] The variable damping method provides an appropriate level of damping resistance to the system and / or user based in part on how the HID is operated by the user. The damping algorithm can employ a damping function having multiple damping regimes. As an example, one damping regime can provide a relatively low amount of resistance to the user (e.g., similar to a hand moving in water) and a different damping regime can provide a relatively high amount of resistance to the user (e.g., assuming a hand moving in molasses). Another damping regime can provide a variable amount of resistance depending on motion information received from the gimbal (e.g., from the joint(s)). In some embodiments, the resistance can be proportional or inversely proportional to the motion information from the medical instrument and / or the gimbal.

[0149] Incorporation of multiple or variable damping regimes can be used to provide kinesthetic haptic feedback. For example, providing a low damping coefficient can indicate to the surgeon that there is no problem with his / her movement of the gimbal, while providing a high damping coefficient can indicate that there may be a problem (e.g., near the end of the range of movement). Increasing the damping coefficient can indicate that a higher level of caution may be required (e.g., moving toward the end of the range of movement) and decreasing the damping coefficient can indicate that a lower level of caution may be required (e.g., moving away from the end of the range of movement).

[0150] 25A shows an example of a damping function used in the master gimbal impedance control 209 in the roll haptic module 208, according to some embodiments. In some embodiments, a low damping coefficient is desired (regime A.1), while a high damping coefficient is desired (regime A.3), for example to provide warning. To avoid or minimize abrupt jumps between the low damping regime (regime A.1) and the high damping regime (regime A.3), a transition damping region (regime A.2) bridges regimes A.1 and A.3.

[0151] The applied damping coefficient(s) can be selected to modify the force or torque of one or more robot joints to provide resistance to a user input. The force or torque may, for example, be modified by a fixed amount when the current speed or velocity of a portion of the robot user interface is within a first range (e.g., corresponding to regime A.1), modified by a variable amount (e.g., increasing in the illustrated embodiment) when the current speed or velocity of a portion of the robot user interface is within a second range (e.g., corresponding to regime A.2), and modified by another fixed amount when in a second range greater than the first range (e.g., corresponding to the illustrated plateau of regime A.3).

[0152] Although the illustrated attenuation regions provide a continuous transition, in some embodiments the transition may not be continuous, for example the transition may include several sub-regions, each with its own attenuation coefficient distinct from the attenuation coefficients of the other sub-regions, eventually progressing to a region of high attenuation.

[0153] FIG. 25B illustrates an example of a portion of an output signal (e.g., voltage or current) provided to the HID (to generate haptic and / or tactile feedback) according to some embodiments. In some embodiments, at least a portion of the output signal from the signal controller 216 derived from the master gimbal impedance control 209 is fed to a torque / current control loop of a component of the HID. In some embodiments, the component of the HID is the gimbal roll motor (e.g., G6) of the HID. FIG. 25B illustrates a control signal (e.g., current) derived from the master gimbal impedance control 209 provided to the HID as a function of time to provide kinesthetic haptic feedback. The figure also illustrates the haptic feedback or output (e.g., force) from the HID as experienced by the user. In some embodiments, the haptic feedback is kinesthetic feedback. In some embodiments, the kinesthetic haptic feedback is provided for a duration of at least 0.5 seconds.

[0154] When the user begins to interact with the HID, a constant control signal may be sent to the HID to provide a fixed amount of resistance as kinesthetic feedback to the user's movement of the HID. At time 220, one or more signals are received as part of the HID feedback 204 and / or patient side feedback 202, and the roll haptics module 208 produces a variable damping signal to increase the resistance provided by the HID. For example, tool motion mismatch causes patient side feedback 202 to be sent to impedance control 206 and / or gimbal navigation causes HID feedback 204 to be sent to impedance control 206. An example of gimbal navigation causing a signal from the HID feedback 204 to be sent is the gimbal being navigated towards the end of the gimbal's range of motion. In some embodiments, at time 222, the gimbal is near its range of motion and a large damping coefficient is applied to make it more difficult to move the HID to the end of its range of motion. The control signal is decreased (to decrease the amount of damping) as the HID moves away from the end of its range of motion.

[0155] FIG. 25C illustrates a control signal (e.g., voltage or current) derived from the haptic buzz module 210 provided to the HID as a function of time to provide vibrotactile tactile feedback to the user. At time 224, one or more signals are received from the HID feedback 204 and / or patient feedback 202, causing the haptic buzz module 210 to generate a physical vibration in the HID. The physical vibration may be at least 50 Hz, or may be between 50 Hz and 200 Hz. The high frequency profile of the vibration makes the buzz effect easily identifiable by the user, allowing information to be conveyed to the user in a separate and identifiable manner (compared to low frequency kinesthetic feedback). For example, one or more of a sword fight, a collision, and / or other malfunction causes the patient feedback 202 to send to the impedance control 206 to provide vibration feedback. Additionally or alternatively, one or more of a gimbal collision, gimbal navigation, a malfunction, or relaxation of holding during re-clutching may cause HID feedback 204 to be sent to impedance control 206 to provide vibration feedback. An example of gimbal navigation that may cause a signal from HID feedback 204 to be sent to provide vibration feedback is a gimbal movement that leads to an imminent collision with a second gimbal.

[0156] Figure 25D shows a control signal (e.g., voltage or current) provided to the HID from signal controller 216 that combines control signals from both the roll haptic module 208 and the haptic buzz module 210. In some embodiments, the control signal provided from signal controller 216 is the sum (or weighted sum) of the signal shown in Figure 25B and the signal shown in Figure 25C. The different characteristics of the kinesthetic haptic feedback and vibrotactile haptic feedback allow the user to identify feedback from the HID at times 220, 224, and 222 that is indicative of various events associated with the medical instrument and / or gimbal.

[0157] FIG. 26 is a flow chart illustrating an example method 250 in which a haptic feedback device provides feedback to a user while the user controls a medical instrument (e.g., a slave) using a controller (e.g., a master). Method 250 can be configured for hybrid control using both impedance control and admittance control. Although shown sequentially, the blocks of method 250 can be performed in other orders, or one or more of the blocks can occur substantially simultaneously. Method 250 begins at block 252, where the system receives a first input signal from one or more input sensors. At block 254, the system transmits a first type of signal (e.g., a kinesthetic tactile feedback signal) to a first haptic interface device based on (e.g., provides or generates) at least the first input signal for a first type of feedback (e.g., kinesthetic tactile feedback).

[0158] At block 256, the system receives a second input signal from one or more input sensors. At block 258, the system transmits a second type of signal (e.g., a vibrotactile feedback signal) to the first haptic interface device based on (e.g., providing or generating) at least the second input signal for the second type of feedback (e.g., vibrotactile feedback).

[0159] 4. Implementation Systems and Terminology. The embodiments disclosed herein provide systems, methods, and apparatus for a robotically controlled medical system. Various embodiments described herein include a controller for a robot-enabled medical system.

[0160] It should be noted that, as used herein, the terms "couple," "coupled," "coupled," or other variations of the word coupled, may indicate either an indirect connection or a direct connection. For example, when a first component is "coupled" to a second component, the first component may be either indirectly connected to the second component through another component, or directly connected to the second component.

[0161] The position estimation functions and robot motion actuation functions described herein may be stored as one or more instructions on a processor-readable medium or computer-readable medium. The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disk read-only memory (CD-ROM), or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that computer-readable media may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code, or data that is executable by a computing device or processor.

[0162] The methods disclosed herein include one or more steps or actions for achieving the described method. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for the proper operation of the described method, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0163] As used herein, the term "plurality" refers to two or more. For example, a plurality of components refers to two or more components. The term "determining" covers a wide variety of acts, and thus "determining" can include calculating, computing, processing, deriving, investigating, looking up (e.g., consulting a table, database, or another data structure), ascertaining, and the like. Also, "determining" can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, "determining" can include resolving, selecting, electing, establishing, and the like.

[0164] The phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" refers to both "based only on" and "based at least on."

[0165] As used herein, the term "approximately" or "about" refers to a range of measurements of a length, thickness, amount, duration, or other measurable value. Such range of measurements encompasses variations of no more than + / -10% of the specified value, preferably no more than + / -5%, more preferably no more than + / -1%, and even more preferably no more than + / -0.1%, so long as such variations are adequate to function in the disclosed devices, systems, and techniques.

[0166] The foregoing description of the disclosed implementations is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these implementations will be readily apparent to those skilled in the art, and the general principles set forth herein may be applied to other implementations without departing from the scope of the present invention. For example, those skilled in the art will recognize that many corresponding alternatives, equivalent structural details, such as similar manners of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for producing specific actuation motions, and equivalent mechanisms for delivering electrical energy, may be employed. Thus, the present invention is not intended to be limited to the implementations shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0167] 5. Examples of subject technology clauses. The following clauses describe several embodiments or implementations. SECTION 1. A health care system comprising: a first haptic interface device; and one or more input sensors; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: receiving a first input signal from one or more input sensors; causing the first haptic interface device to transmit a kinesthetic tactile feedback signal based at least on the first input signal for the kinesthetic tactile feedback; receiving a second input signal from one or more input sensors; The medical system causes the first tactile interface device to transmit a vibrotactile feedback signal based at least on a second input signal for vibrotactile feedback. Item 2. The medical system described in item 1, further comprising a first robotic arm. Clause 3. The medical system of clause 2, wherein the first input signal is based on a difference between a master command and a movement of the first robotic arm. Item 4. The medical system of item 2 or 3, further comprising a second robotic arm separate from the first robotic arm. Clause 5. The medical system of clause 4, wherein the second input signal is derived from a collision between the first robotic arm and the second robotic arm. Clause 6. The medical system of clause 4 or clause 5, wherein the second input signal is derived from a criterion including the distance between the first robotic arm and the second robotic arm being less than a first distance threshold. Clause 7. A medical system described in any one of clauses 4 to 6, wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm. Clause 8. A medical system described in any one of clauses 4 to 7, wherein the second input signal is derived from a criterion including a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold. Clause 9. A medical system as described in any one of clauses 4 to 8, wherein the second input signal is derived from a criterion that includes detection of a fault within the medical system. Clause 10. A medical system according to any one of clauses 4 to 9, wherein the second input signal is derived from a criterion including detection of the first tactile interface device at a predetermined boundary. Clause 11. The medical system of any one of clauses 1 to 10, further comprising a second tactile interface device that is distinct from and separate from the first tactile interface device. Clause 12. The medical system of clause 11, wherein the second input signal is derived from a criterion that includes a collision between the first tactile interface device and the second tactile interface device. Clause 13. The medical system of clause 11 or clause 12, wherein the second input signal is derived from criteria including a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold. Clause 14. A medical system described in any one of clauses 1 to 13, wherein the kinesthetic tactile feedback includes a continuous force applied in response to user input. Section 15. The second input signal is derived from a reference including a plurality of events; 15. The medical system of any one of claims 1 to 14, wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events. Clause 16. The medical system of clause 15, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz. Clause 17. The medical system of any one of clauses 1 to 16, wherein the first tactile interface device further includes a motor that provides a torque about a roll axis of the first tactile interface device in response to user input on the first tactile interface device. Item 18. A method comprising: Receiving a first input signal from one or more input sensors; transmitting a kinesthetic tactile feedback signal to a first haptic interface device based at least on a first input signal for the kinesthetic tactile feedback; receiving a second input signal from one or more input sensors; and transmitting a vibrotactile feedback signal to the first haptic interface device based at least on the second input signal for vibrotactile feedback. Item 19. The method of item 18, wherein the first input signal is for controlling movement of a first robotic arm. Clause 20. The method of clause 19, wherein the first input signal is based on a difference between a master command and a movement of the first robot arm. Clause 21. The method of clause 19 or clause 20, wherein the second input signal is for controlling movement of a second robotic arm separate from the first robotic arm. Clause 22. The method of clause 21, wherein the second input signal is derived from a collision between the first robot arm and the second robot arm. Clause 23. The method of clause 21 or clause 22, wherein the second input signal is derived from a criterion that includes a distance between the first robot arm and the second robot arm being less than a first distance threshold. Clause 24. The method of any one of clauses 21 to 23, wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm. Clause 25. The method of any one of clauses 21 to 24, wherein the second input signal is derived from a criterion that includes a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold. Clause 26. The method of any one of clauses 21 to 25, wherein the second input signal is derived from a criterion that includes detection of a fault in the medical system. Clause 27. The method of any one of clauses 21 to 26, wherein the second input signal is derived from a criterion that includes detection of the first tactile interface device at a predetermined boundary. Clause 28. The method of any one of clauses 18-27, further comprising determining a position of a second tactile interface device that is distinct from and separate from the first tactile interface device. Clause 29. The method of clause 28, wherein the second input signal is derived from a criterion that includes a collision between the first haptic interface device and the second haptic interface device. Clause 30. The method of clause 28 or clause 29, wherein the second input signal is derived from criteria including a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold. Clause 31. The method of any one of clauses 18-30, wherein the kinesthetic tactile feedback comprises a continuous force applied in response to the user input. Section 32. The second input signal is derived from a reference including a plurality of events; 32. The method of any one of clauses 18 to 31, wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events. Clause 33. The method of clause 32, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz. Clause 34. The method of any one of clauses 18 to 33, wherein the first tactile interface device further includes a motor that provides a torque about a roll axis of the first tactile interface device in response to a user input on the first tactile interface device.

[0168] [Embodiment] (1) A health care system, comprising: a first haptic interface device; and one or more input sensors; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: receiving a first input signal from the one or more input sensors; causing the first haptic interface device to transmit a kinesthetic tactile feedback signal based at least on the first input signal for kinesthetic tactile feedback; receiving a second input signal from the one or more input sensors; and causing the first tactile interface device to transmit a vibrotactile feedback signal based at least on the second input signal for vibrotactile feedback. (2) The medical system of embodiment 1, further comprising a first robotic arm. (3) The medical system of embodiment 2, wherein the first input signal is based on a difference between a master command and a movement of the first robotic arm. (4) The medical system of embodiment 2 or 3, further comprising a second robotic arm separate from the first robotic arm. (5) The medical system of embodiment 4, wherein the second input signal is derived from a collision between the first robotic arm and the second robotic arm.

[0169] (6) The medical system of embodiment 4 or 5, wherein the second input signal is derived from a criterion including a distance between the first robotic arm and the second robotic arm being less than a first distance threshold. (7) A medical system described in any of embodiments 4 to 6, wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm. (8) A medical system described in any of embodiments 4 to 7, wherein the second input signal is derived from a criterion including a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold. (9) A medical system according to any one of embodiments 4 to 8, wherein the second input signal is derived from a criterion that includes detection of a fault in the medical system. (10) A medical system described in any one of embodiments 4 to 9, wherein the second input signal is derived from a criterion including detection of the first tactile interface device at a predetermined boundary.

[0170] (11) The medical system described in any one of embodiments 1 to 10, further comprising a second tactile interface device that is separate from and separate from the first tactile interface device. (12) The medical system of embodiment 11, wherein the second input signal is derived from a criterion including a collision between the first tactile interface device and the second tactile interface device. (13) The medical system of embodiment 11 or 12, wherein the second input signal is derived from criteria including a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold. (14) A medical system described in any one of embodiments 1 to 13, wherein the kinesthetic tactile feedback includes a continuous force applied in response to a user input. (15) The second input signal is derived from a reference including a plurality of events; A medical system described in any one of embodiments 1 to 14, wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event among the plurality of events.

[0171] (16) The medical system of embodiment 15, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz. (17) A medical system described in any one of embodiments 1 to 16, wherein the first tactile interface device further includes a motor that provides torque about a roll axis of the first tactile interface device in response to user input on the first tactile interface device. (18) A method comprising the steps of: Receiving a first input signal from one or more input sensors; transmitting a kinesthetic tactile feedback signal to a first haptic interface device based at least on the first input signal for kinesthetic tactile feedback; receiving a second input signal from the one or more input sensors; and transmitting a vibrotactile feedback signal to the first haptic interface device based at least on the second input signal for vibrotactile feedback. (19) The method of embodiment 18, wherein the first input signal is for controlling movement of a first robotic arm. (20) The method of embodiment 19, wherein the first input signal is based on a difference between a master command and a movement of the first robot arm.

[0172] (21) The method of any one of embodiments 19 to 20, wherein the second input signal is for controlling movement of a second robotic arm separate from the first robotic arm. (22) The method of embodiment 21, wherein the second input signal is derived from a collision between the first robotic arm and the second robotic arm. (23) The method of any one of embodiments 21 to 22, wherein the second input signal is derived from a criterion including a distance between the first robot arm and the second robot arm being less than a first distance threshold. (24) A method according to any of embodiments 21 to 23, wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm. (25) A method according to any of embodiments 21 to 24, wherein the second input signal is derived from a criterion including a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold.

[0173] (26) A method according to any one of embodiments 21 to 25, wherein the second input signal is derived from a criterion including detection of a fault in a medical system. (27) A method according to any one of embodiments 21 to 26, wherein the second input signal is derived from a criterion including detection of the first tactile interface device at a predetermined boundary. (28) A method according to any of embodiments 18 to 27, further comprising determining a position of a second tactile interface device that is separate from and separated from the first tactile interface device. (29) The method of embodiment 28, wherein the second input signal is derived from a criterion including a collision between the first haptic interface device and the second haptic interface device. (30) The method of embodiment 28 or 29, wherein the second input signal is derived from criteria including a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold.

[0174] (31) A method according to any one of embodiments 18 to 30, wherein the kinesthetic tactile feedback comprises a continuous force applied in response to a user input. (32) The second input signal is derived from a reference including a plurality of events; A method according to any one of embodiments 18 to 31, wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events. (33) The method of embodiment 32, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz. (34) A method according to any of embodiments 18 to 33, wherein the first tactile interface device further includes a motor that provides torque about a roll axis of the first tactile interface device in response to user input on the first tactile interface device.

Claims

1. 1. A health care system comprising: a first haptic interface device; one or more input sensors; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: receiving a first input signal from the one or more input sensors; causing the first haptic interface device to transmit a kinesthetic tactile feedback signal based at least on the first input signal for kinesthetic tactile feedback; receiving a second input signal from the one or more input sensors; and causing the first tactile interface device to transmit a vibrotactile feedback signal based at least on the second input signal for vibrotactile feedback.

2. The medical system of claim 1 , further comprising a first robotic arm.

3. The medical system of claim 2 , wherein the first input signal is based on a difference between a master command and a movement of the first robotic arm.

4. The medical system of claim 2 or 3, further comprising a second robotic arm separate from the first robotic arm.

5. The medical system of claim 4 , wherein the second input signal is derived from a collision between the first robotic arm and the second robotic arm.

6. 5. The medical system of claim 4, wherein the second input signal is derived from a criterion that includes a distance between the first robotic arm and the second robotic arm being less than a first distance threshold.

7. 5. The medical system of claim 4, wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm.

8. 5. The medical system of claim 4, wherein the second input signal is derived from a criterion including a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold.

9. The medical system of claim 4 , wherein the second input signal is derived from a criterion that includes detection of a fault in the medical system.

10. The medical system of claim 4 , wherein the second input signal is derived from a criterion that includes detection of the first tactile interface device at a predetermined boundary.

11. The medical system of claim 1 , further comprising a second haptic interface device that is separate and distinct from the first haptic interface device.

12. The medical system of claim 11 , wherein the second input signal is derived from a criterion that includes a collision between the first tactile interface device and the second tactile interface device.

13. The medical system of claim 11 or 12, wherein the second input signal is derived from a criterion that includes a distance between the first tactile interface device and the second tactile interface device being less than a third distance threshold.

14. The medical system of claim 1 , wherein the kinesthetic tactile feedback comprises a continuous force applied in response to a user input.

15. the second input signal is derived from a reference including a plurality of events; The medical system of claim 1 , wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events.

16. 16. The medical system of claim 15, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz.

17. 10. The medical system of claim 1, wherein the first haptic interface device further comprises a motor that provides a torque about a roll axis of the first haptic interface device in response to a user input on the first haptic interface device.

18. 1. A method comprising: Receiving a first input signal from one or more input sensors; transmitting a kinesthetic tactile feedback signal to a first haptic interface device based at least on the first input signal for kinesthetic tactile feedback; receiving a second input signal from the one or more input sensors; transmitting a vibrotactile feedback signal to the first haptic interface device based at least on the second input signal for vibrotactile feedback.

19. 20. The method of claim 18, wherein the first input signal is for controlling movement of a first robotic arm.

20. 20. The method of claim 19, wherein the first input signal is based on a difference between a master command and a movement of the first robotic arm.

21. 21. The method of claim 19 or 20, wherein the second input signal is for controlling movement of a second robotic arm separate from the first robotic arm.

22. 22. The method of claim 21, wherein the second input signal is derived from a collision between the first robotic arm and the second robotic arm.

23. 22. The method of claim 21, wherein the second input signal is derived from a criterion that includes a distance between the first robotic arm and the second robotic arm being less than a first distance threshold.

24. 22. The method of claim 21 , wherein the second input signal is derived from a criterion including a collision between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm.

25. 22. The method of claim 21 , wherein the second input signal is derived from a criterion that includes a distance between a first surgical instrument coupled to the first robotic arm and a second surgical instrument coupled to the second robotic arm being less than a second distance threshold.

26. 22. The method of claim 21, wherein the second input signal is derived from a criterion that includes detection of a fault in a medical system.

27. The method of claim 21 , wherein the second input signal is derived from a criterion that includes detection of the first haptic interface device at a predetermined boundary.

28. The method of claim 18 , further comprising determining a position of a second haptic interface device that is distinct and separate from the first haptic interface device.

29. 30. The method of claim 28, wherein the second input signal is derived from a criterion that includes a collision between the first haptic interface device and the second haptic interface device.

30. 30. The method of claim 28 or 29, wherein the second input signal is derived from a criterion that includes a distance between the first haptic interface device and the second haptic interface device being less than a third distance threshold.

31. The method of claim 18 , wherein the kinesthetic tactile feedback comprises a continuous force applied in response to a user input.

32. the second input signal is derived from a reference including a plurality of events; The method of claim 18 , wherein a separate tactile feedback signal is selected as the vibrotactile feedback signal for each event of the plurality of events.

33. 33. The method of claim 32, wherein the vibrotactile feedback includes a component having a frequency of at least 10 Hz.

34. The method of claim 18 , wherein the first haptic interface device further comprises a motor that provides a torque about a roll axis of the first haptic interface device in response to a user input on the first haptic interface device.

Images