Method for actively engaging and disengaging teleoperation of a surgical robotic system

By detecting the combination of user action sequence and interface equipment, the surgical robot system automatically engages or leaves the remote operation mode, solving the problems of complex operation and insufficient safety in the prior art, and achieving safe and simple mode switching.

CN113556989BActive Publication Date: 2024-11-12AURIS HEALTH INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880100540.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-12-28
Filing Date
2018-12-31
Publication Date
2024-11-12
Estimated Expiration
2038-12-31

AI Technical Summary

Technical Problem

Existing surgical robot systems operate complex and difficult to remember when engaged and disengaged from remote operating modes, which can cause damage to unintentional tool movements and lack safe and simple ways of engagement and disengagement.

Method used

By detecting the user's intentional action sequence or collection, using the combination of user interface devices and foot pedals, automatically engage or disengage the remote operation mode, including the pressing and holding of the finger clutch and clutch pedals, ensuring the system is converted in a safe and simple manner. model.

Benefits of technology

Provides a safe and simple way to engage and disengage remote operation modes, reducing the risk of injury caused by unintentional operation and improving user experience to ensure operation safety and reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113556989B_ABST
    Figure CN113556989B_ABST
Patent Text Reader

Abstract

A method for engaging and disengaging a surgical instrument of a surgical robotic system, the method comprising receiving a sequence of user inputs from one or more user interface devices of the surgical robotic system; determining, by one or more processors communicatively coupled to the user interface devices and the surgical instrument, whether the sequence of user inputs indicates an intentional engagement or disengagement of a teleoperation mode, in which the surgical instrument is controlled by the user input received from the user interface device; in response to a determination of engagement, switching the surgical robotic system into the teleoperation mode; and in response to a determination of disengagement, switching the surgical robotic system out of the teleoperation mode so that the user interface device is prevented from controlling the surgical instrument.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-references

[0002] This application claims the benefit of the earlier filing date of U.S. non-provisional application No. 16 / 236,163, filed on December 28, 2018, the disclosure of which is incorporated herein by reference in its entirety. Background Art Technical Field

[0004] Embodiments related to robotic systems are disclosed. More particularly, embodiments related to surgical robotic systems and corresponding methods for engaging and disengaging a teleoperation mode of a surgical robotic system are disclosed. Background Art

[0005] Endoscopic surgery involves looking at the patient's body and using an endoscope and other surgical tools to perform surgery inside the body. For example, laparoscopic surgery can use a laparoscope to access and view the abdominal cavity. Endoscopic surgery can be performed using manual tools and / or surgical robotic systems with robotic-assisted tools.

[0006] The surgical robot system can be remotely operated by a surgeon to command a robotic-assisted tool located at an operating table. Such operation of a robotic-assisted tool remotely by a surgeon may generally be referred to as teleoperation. For example, a surgeon may use a computer console located in an operating room or which may be located in a different city to command a robot to manipulate a surgical tool mounted on an operating table. The robotically controlled surgical tool may be an endoscope mounted on a robotic arm. Thus, an endoscopic surgery may be performed by a remote surgeon using a surgical robot system.

[0007] The surgeon may provide input commands to the surgical robotic system, and one or more processors of the surgical robotic system may control the system components in response to the input commands. For example, the surgeon may hold a user input device (such as a joystick or a computer mouse) in her hand, which she manipulates to generate control signals to cause movement of surgical robotic system components (e.g., actuators, robotic arms, and / or surgical tools of the robotic system). Summary of the invention

[0008] The surgical robot system may be considered to have various system modes. The primary operating mode may be referred to herein as the teleoperation mode. The surgical robot system is considered to be in teleoperation mode when the user actively controls or is able to actively control a surgical robot system component (e.g., an actuator, a robotic arm, a surgical tool, and / or an endoscope), for example, with the aid of a user interface device or a foot pedal. On the other hand, when the user is unable to actively control a surgical robot system component, the system is considered to have exited the teleoperation mode, exited (be out of) the teleoperation mode, is in a non-teleoperation mode, or has left the teleoperation mode. For example, when (1) the surgical robot system does not accept user input, (2) user input commands of the system's graphical user interface (GUI) are accepted, but the associated surgical robot system components cannot be controlled, or (3) the user is still unable to control the surgical robot component, but a sequence of intended actions will cause the user to enter or engage the teleoperation mode, the system may be considered to have exited or left the teleoperation mode or be in a non-teleoperation mode. In a teleoperation surgical robot system, it is important to provide a safe and simple way for the user to actively engage the teleoperation mode to avoid injuries caused by unintentional tool movement. Furthermore, it is desirable to provide the user with a way to safely disengage from teleoperation mode any time the user actively desires to disengage or performs an action that makes teleoperation anything other than the primary focus. While surgical robotic systems may include various safety requirements to prevent inadvertent operation, they may not cover the specific operations of engaging and / or disengaging teleoperation mode and / or may be overly complex and therefore difficult for the user to remember and / or follow.

[0009] Therefore, the present invention relates to a surgical robot system that can detect intentional user actions, sequences of intentional user actions, or sets of intentional user actions, and based on the detection, automatically engage or disengage a teleoperation mode. Representatively, in one aspect, a sequence of intentional user actions or a set of intentional user actions is detected by a user interface device. For example, a sequence of (one or more) intentional user actions can be a set of predetermined gestures detected by a user interface device, such as a user tapping (tap) or squeezing a user interface device on a user interface device. Alternatively, (one or more) intentional actions can be actions that cause the user interface device to be in a specific position, orientation, and / or positioning, such as a user docking the user interface device in a docking station or positioning the user interface very close to the docking station. Further, (one or more) actions can be actions related to the foot pedal of the surgical robot system alone or in combination with the actions previously discussed in conjunction with the user interface device. For example, (one or more) actions or a set of actions can include tapping a clutch pedal or pressing and holding a clutch pedal in combination with tapping and / or squeezing a user interface device.

[0010] Representatively, in one aspect, the user interface device can be a handheld user input or interface device that includes a handle that the surgeon can manipulate with her hand to generate an input command to move an actuator to which the robotic surgical tool and / or end effector is coupled in the surgical robot system. The surgeon can move the handle within a workspace (such as the range of motion of a linkage system connected to the handle) to remotely cause corresponding movement of the actuator when in teleoperation mode. When the limit of the workspace is reached, for example, when the linkage system is fully extended, the surgeon can press a clutch button or finger clutch so that the input from the surgical robot system does not cause movement of the actuator, and then, the robotic surgical tool and / or end effector is paused or otherwise maintained in the current position. That is, when the finger clutch is pressed, the handle can be repositioned within the workspace without causing corresponding movement of the actuator, the robotic surgical tool and / or the end effector, which would otherwise occur if the finger clutch is not pressed. In order to actuate the finger clutch, the surgeon must apply a force large enough to counteract the return spring force of the clutch. For example, the surgeon must press down on the finger clutch. When the surgeon releases the downward pressure on the finger clutch, the robotic surgical tool and / or end effector is no longer suspended in the current position, and the user can continue the movement of the actuator and associated components using the user interface device. However, it should be understood that the single action of pressing / holding the finger clutch to pause the associated robotic surgical component in the current position or releasing the finger clutch to continue the movement of the component should not be understood as an intentional sequence of user actions that can be used to disengage the teleoperation mode. Instead, the clutching or pausing of the associated robotic surgical components is a more temporary operation compared to disengaging the teleoperation mode, which is caused by a relatively simple user action. In particular, during the clutching operation, the signal from the user interface device to the actuator, the robotic surgical tool and / or the end effector to move the actuator, the robotic surgical tool and / or the end effector is temporarily suspended, or is covered by the clutch signal to suspend the operation, but once a single action occurs, such as the user removing their finger from the finger clutch, the desired movement can continue. In contrast, when the teleoperation mode is disengaged, for example in response to the previously discussed intentional action(s) of the user, the user interface device may be considered to be permanently disconnected from the actuator, tool, and / or end effector. For example, when disengaging the teleoperation mode, signals from the user interface device may no longer be transmitted to the actuator, robotic surgical tool, and / or end effector, and / or the user interface device may remain disconnected until a relatively complex and intentional sequence of actions is performed by the user.The sequence of intentional actions is an action indicating the user's intention to engage the teleoperation mode, and may be more complex than a single action for clutching or pausing the system. In other words, when a single action for clutching is detected, the system will not engage (or disengage) the teleoperation mode. Instead, a sequence of intentional actions must be detected for the system to reengage the teleoperation mode, and then reconnect the user interface device to the actuator, tool, and / or end effector. In other words, compared to the clutch operation, simply removing the first action will not automatically reconnect the user interface device to the actuator, tool, and / or end effector. Instead, another intentional action (in some cases, it can be the same as the first intentional action (i.e., a symmetrical action)) is required to reconnect the user interface device to the actuator, tool, and / or end effector, and thus engage the teleoperation mode.

[0011] More specifically, in one aspect, the present invention relates to a method for engaging and disengaging a surgical instrument of a surgical robotic system. The method may include receiving a sequence of user inputs from one or more user interface devices of the surgical robotic system. The method may further include determining, by one or more processors communicatively coupled to the user interface device and the surgical instrument, whether the sequence of user inputs indicates an intentional engagement or disengagement of a teleoperation mode, in which the surgical instrument is controlled by the user input received from the user interface device. In response to the determination of engagement, the surgical robotic system may be switched to the teleoperation mode; and in response to the determination of disengagement, the surgical robotic system may be switched out of the teleoperation mode so that the user interface device is prevented from controlling the surgical instrument. The user interface device may include at least one of a handheld user input device and a foot pedal. The sequence of user inputs indicating engagement or disengagement of the teleoperation mode may be different. In other cases, the sequence of user inputs indicating engagement or disengagement of the teleoperation mode may be the same. In some cases, the sequence of user inputs may include a sequence of first user inputs from a first user interface device and a sequence of second user inputs from a second user interface device. In a further aspect, one or more user interface devices may include a handheld user input device and a clutch pedal, and the sequence of user inputs includes a sequence of first user inputs received from the handheld user input device and a sequence of second user inputs received from the clutch pedal. In some aspects, the sequence of user inputs indicating intentional engagement or disengagement may correspond to pressing and holding the clutch pedal of one or more user interface devices and double-tapping the finger clutch of one or more user interface devices. Further, the sequence of user inputs indicating intentional engagement or disengagement may correspond to pressing and holding the clutch pedal of one or more user interface devices and squeezing another of one or more user interface devices. Further, the sequence of user inputs indicating intentional engagement or disengagement may correspond to double-tapping the clutch pedal of one or more user interface devices. The method may further include determining whether at least one of the following conditions is met before converting the surgical robot system to the teleoperation mode: the chair of the surgical robot system is locked, the user is looking at the screen of the surgical robot system, and the user interface device is in the surgical robot system workspace. The method may further include user feedback for alerting the user that the teleoperation mode is engaged or disengaged.

[0012] In another aspect, the present invention relates to a surgical robot system, comprising: a surgical instrument; a user console including a user interface device and a foot pedal, the user interface device being mechanically ungrounded relative to the user console; and one or more processors communicatively coupled to the surgical instrument and the user console. The processor may be configured to receive a sequence of user actions through the user interface device and / or the foot pedal, determine that the surgical robot system is in a non-teleoperation mode, and that the sequence of user actions indicates an intentional engagement, and convert the surgical robot system to a teleoperation mode, in which the surgical instrument is controlled by user input received from the user interface device and the foot pedal. In one aspect, the user interface device may include a first handheld user input device and a second handheld user input device, and the sequence of user actions indicating an intentional engagement includes tapping or squeezing both the first and second handheld user input devices. In another aspect, the foot pedal may be a clutch pedal, and the sequence of user actions indicating an intentional engagement includes pressing and holding the clutch pedal, and tapping or squeezing the user interface device, or the sequence of user actions indicating an intentional engagement includes tapping the clutch pedal, and tapping or squeezing the user interface device.

[0013] In another aspect, a surgical robot system may include: a surgical instrument; a user console including a user interface device and a foot pedal, the user interface device being mechanically ungrounded relative to the user console; and one or more processors communicatively coupled to the surgical instrument and the user console. In this regard, the processor may be configured to receive a sequence of user actions through the user interface device and / or the foot pedal, determine that the surgical robot system is in a teleoperation mode, and the sequence of user actions indicates an intentional disengagement, wherein in the teleoperation mode, the surgical instrument is controlled by user input received from the user interface device and the foot pedal, and convert the surgical robot system to a non-teleoperation mode, in which the user interface device or the foot pedal is prevented from controlling the surgical instrument. In one aspect, the foot pedal is a clutch pedal, and the sequence of user actions indicating an intentional disengagement includes double tapping the clutch pedal. In another aspect, the sequence of user actions indicating an intentional disengagement may include pressing and holding the clutch pedal, and tapping a finger clutch of the user interface device, or pressing and holding the clutch pedal, and squeezing the user interface device. In a further aspect, the user interface device is a handheld user input device including a sensor operable to detect user inputs, and at least one of the user inputs includes docking of the portable handheld user interface device.

[0014] The above summary does not include an exhaustive list of all aspects of the invention. It is contemplated that the invention includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those systems and methods disclosed in the detailed description below and specifically pointed out in the claims filed with the present application. Such combinations have special advantages not specifically recited in the above summary. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Embodiments of the present invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like references indicate similar elements. It should be noted that references to "one" or "an" embodiment of the present invention in this disclosure do not necessarily refer to the same embodiment, and they mean at least one. In addition, for simplicity and to reduce the total number of figures, a given figure may be used to illustrate features of more than one embodiment of the present invention, and not all elements in the figure are required for a given embodiment.

[0016] Figure 1 is a pictorial view of an example surgical robotic system in an operating arena according to an embodiment.

[0017] Figure 2 is a schematic diagram of a user interface device with a finger clutch according to an embodiment.

[0018] Figure 3 is a perspective view of a user interface device according to an embodiment.

[0019] Figure 4 is a perspective view of a user interface device in a docking station according to an embodiment.

[0020] Figure 5 is a block diagram of a computer portion of a surgical robot system according to an embodiment.

[0021] Figure 6 is a block diagram of an exemplary process for exiting teleoperation mode, according to an embodiment.

[0022] Figure 7 is a block diagram of an exemplary process for engaging teleoperation mode, according to an embodiment.

[0023] Figure 8 is a block diagram of an exemplary process for engaging teleoperation mode, according to an embodiment.

[0024] Fig. 9 is a block diagram of a computer portion of a surgical robot system according to an embodiment. DETAILED DESCRIPTION

[0025] Embodiments describe processes for engaging and disengaging a teleoperation mode based on user actions in conjunction with a user interface device (UID) that may be used by a robotic system, for example, to control an actuator that moves a robotic arm or tool. The robotic system may be a surgical robotic system, the robotic arm may be a surgical robotic arm, and the tool may be a surgical tool. However, the UID may be used by other systems, such as interventional cardiology systems, vision systems, or aircraft systems, to control other output components. These other systems name just a few possible applications.

[0026] In various embodiments, description is made with reference to the accompanying drawings. However, certain embodiments may be implemented without one or more of these specific details, or in combination with other known methods and configurations. In the following description, many specific details, such as specific configurations, dimensions and processes, are described to provide a thorough understanding of the embodiments. In other cases, well-known processes and manufacturing techniques have not been described in particular detail to avoid unnecessarily obscuring the description. Throughout this specification, references to "one embodiment", "embodiment" or the like mean that the specific features, structures, configurations or characteristics described are included in at least one embodiment. Therefore, throughout this specification, the appearance of phrases "one embodiment", "embodiment" or the like in various positions does not necessarily refer to the same embodiment. In addition, in one or more embodiments, specific features, structures, configurations or characteristics may be combined in any suitable manner.

[0027] Relative terms used throughout the description may indicate relative positions or directions. For example, "distal" may indicate a first direction away from a reference point (e.g., away from a user). Similarly, "proximal" may indicate a position in a second direction opposite to the first direction (e.g., toward the user). However, such terms are provided to establish a framework of relative references and are not intended to limit, for example, the use or orientation of a UID to the specific configurations described in the various embodiments below.

[0028] refer to Figure 1 , which is a schematic diagram of an example surgical robotic system 1 in an operating room. The robotic system 1 includes a user console 2, a control tower 3, and one or more surgical robotic arms 4 at a surgical robotic platform 5 (e.g., a table, a bed, etc.). The system 1 can incorporate any number of devices, tools, or accessories for performing surgery on a patient 6. For example, the system 1 can include one or more surgical tools 7 for performing surgery. The surgical tool 7 can be an end effector attached to the distal end of the surgical arm 4 for performing a surgical procedure.

[0029] During surgery, each surgical tool 7 can be operated manually, robotically, or both. For example, the surgical tool 7 can be a tool for entering, viewing, or manipulating the internal anatomical structure of the patient 6. In an embodiment, the surgical tool 7 is a grasper that can grasp the patient's tissue. The surgical tool 7 can be manually controlled by the bedside operator 8; or it can be robotically controlled via actuation movement of the surgical robotic arm 4 to which it is attached. The robotic arm 4 is shown as a system mounted on a table, but in other configurations, the arm 4 can be mounted on a cart, in a ceiling or side wall, or in another suitable structural support.

[0030] Typically, a remote operator 9, such as a surgeon or other operator, can use the user console 2 to remotely manipulate the arm 4 and / or attached surgical tools 7, e.g., teleoperation. Figure 1 As shown in , the user console 2 can be located in the same operating room as the rest of the system 1. However, in other environments, the user console 2 can be located in an adjacent or nearby room, or it can be located at a remote location, such as in a different building, city, or country. The user console 2 may include a chair 10, a foot-operated controller 13, one or more handheld user input or interface devices, a UID 14, and at least one user display 15, the user display 15 being configured to display a view of a surgical site within a patient 6, for example. In the example user console 2, a remote operator 9 sits in the chair 10 and views the user display 15 while manipulating the foot-operated controller 13 and the handheld UID 14 to remotely control the arm 4 and the surgical tool 7 (which is mounted on the distal end of the arm 4).

[0031] In some variations, the bedside operator 8 may also operate the system 1 in an “over the bed” mode, where the side operator 8 (user) is now at the side of the patient 6 and is manipulating the robotically driven tools (such as the end effector attached to the arm 4) at the same time, for example, with the handheld UID 14 and the manual laparoscopic tools held in one hand. For example, the bedside operator's left hand may manipulate the handheld UID to control the robotic components, while the bedside operator's right hand may manipulate the manual laparoscopic tools. Thus, in these variations, the bedside operator 8 may perform both robotic-assisted minimally invasive surgery and manual laparoscopic surgery on the patient 6.

[0032] During the example procedure (surgery), the patient 6 is prepared and covered in a sterile manner to achieve anesthesia. When the arm of the robotic system 1 is in a stowed configuration or a retracted configuration (to facilitate access to the surgical site), the initial entry to the surgical site can be performed manually. Once the entry is completed, the initial positioning or preparation of the robotic system 1 (including its arm 4) can be performed. Next, the surgery is performed by a remote operator 9 at the user console 2 using a foot-operated controller 13 and a UID 14 to manipulate various end effectors and possibly an imaging system to perform the surgery. Manual assistance can also be provided at the operating bed or operating table by a bedside staff wearing sterile work clothes, the bedside staff can, for example, perform tasks such as retracting tissue, performing manual repositioning and tool replacement on one or more of the robotic arms 4. Non-sterile personnel can also be present to assist the remote operator 9 at the user console 2. When a procedure or surgery is complete, the system 1 and user console 2 may be configured or placed in a state to facilitate post-operative procedures, such as cleaning or disinfection and hygiene input or printout via the user console 2 .

[0033] In one embodiment, the teleoperator 9 holds and moves the UID 14 to provide input commands to move the robotic arm actuator 17 in the robotic system 1. The UID 14 can be communicatively coupled to the rest of the robotic system 1, for example, via the console computer system 16. The UID 14 can generate a spatial state signal corresponding to the movement of the UID 14, such as the position and orientation of the handheld housing of the UID, and the spatial state signal can be an input signal for controlling the movement of the robotic arm actuator 17. The robotic system 1 can control the proportional motion of the actuator 17 using a control signal derived from the spatial state signal. In one embodiment, the console processor of the console computer system 16 receives the spatial state signal and generates a corresponding control signal. Based on these control signals (which control how the actuator 17 is stimulated to move the segments or links of the arm 4), the movement of the corresponding surgical tool attached to the arm can mimic the movement of the UID 14. Similarly, the interaction between the teleoperator 9 and the UID 14 may generate, for example, a grasp control signal that causes the jaws of a grasper of the surgical tool 7 to close and grasp tissue of the patient 6 .

[0034] The UID 14 may also be provided to command other operations through the surgical robot system 1. For example, as described below, the UID 14 may include a finger clutch, and pressing on the finger clutch may generate a clutch signal to pause the movement of the actuator 17 corresponding to the surgical robot arm 4 and the surgical tool 7. For example, when the user presses the finger clutch of the UID 14 with a finger, the finger clutch may generate a clutch signal, and the clutch signal may be an input signal for pausing all movements of the (one or more) actuators 7 and correspondingly pausing all movements of the surgical robot arm 4 and the surgical tool 7. When all movements of the surgical robot arm 4 and the surgical tool 7 are paused, there is no movement in any direction, and there is no change in the orientation of the surgical robot arm 4 and the surgical tool 7. When the assertion of the signal pauses the movement of the (one or more) actuators 7, the clutch signal may be referred to as a "clutch activation signal". Similarly, when the touch of the operator 9 is not detected and the movement of the (one or more) actuators 7 is not paused, the input signal may be a "clutch deactivation signal". A clutch signal, such as a clutch activation signal, when asserted, may pause the movement of the robot arm and the surgical tool regardless of the spatial state signal. Therefore, the clutch signal effectively overrides the actuation command derived from the space state signal.

[0035] In addition, the UID 14 can be used to detect an intentional user action, a sequence of intentional user actions, or a collection of intentional user actions that indicate that the user desires to engage or disengage the teleoperation mode. For example, the UID 14 can detect a tap, a double tap, a squeeze, or other gestures associated with the UID 14, and the system can be programmed to recognize one or more of these (one or more) intentional user actions as a command for engaging or disengaging the teleoperation mode, and then send an engagement or disengagement signal to the surgical robot system to engage or disengage the teleoperation mode. In one embodiment, the (one or more) intentional actions required to disengage or engage the teleoperation mode can be the same, or otherwise considered to be symmetrical actions. For example, when not in the teleoperation mode, a double tap on the finger clutch can be used to engage the teleoperation mode, and when in the teleoperation mode, a double tap of the finger clutch can be used to disengage the teleoperation mode. In other cases, the intentional actions for engaging or disengaging the teleoperation mode can be different (e.g., asymmetric). Furthermore, in some cases, the intentional user action(s) may include positioning the UID 14 in (or at least proximate to) the docking station 132, or may be an intentional user action(s) relative to another input device of the surgical robotic system 1. For example, the foot-operated controller(s) 13 may include a clutch pedal, and tapping the clutch pedal, alone or in combination with any of the intentional actions previously discussed, may cause the surgical robotic system to engage and / or disengage from a teleoperation mode. In an embodiment, one or more sensors, such as capacitive sensing pads, may be located on the UID 14, the foot-operated controller(s) 13 (e.g., a clutch pedal), and / or the docking station 132, and an intentional user action or a collection of intentional user actions may be detected by the sensing pads and an engagement and / or disengagement signal may be sent to one or more processors of the surgical robotic system 1 to engage and / or disengage from a teleoperation mode.

[0036] The surgical robotic system 1 may include several UIDs 14, wherein respective control signals are generated for each UID, which control the actuators and surgical tools (end effectors) of the respective arms 4. For example, the teleoperator 9 may move the first UID 14 to control the movement of an actuator 17 in the left robotic arm, wherein the actuator responds by moving a link, gear, etc. in the arm 4. Similarly, movement of the second UID 14 by the teleoperator 9 controls the movement of another actuator 17, which in turn moves other links, gears, etc. of the robotic system 1. The robotic system 1 may include a right arm 4 fixed to a bed or table on the right side of the patient, and a left arm 4 at the left side of the patient. The actuators 17 may include one or more motors that are controlled so that they drive the rotation of the joints of the arm 4, for example to change the orientation of an endoscope or gripper of a surgical tool 7 attached to the arm relative to the patient. The movement of several actuators 17 in the same arm 4 may be controlled by spatial state signals generated by a particular UID 14. The UID 14 can also control the movement of the corresponding surgical tool grasper. For example, each UID 14 can generate a corresponding grasping signal to control the movement of an actuator (e.g., a linear actuator) that opens or closes the jaws of the grasper at the distal end of the surgical tool 7 to grasp tissue within the patient 6.

[0037] In some aspects, the communication between the platform 5 and the user console 2 can be through the control tower 3, which can convert user commands received from the user console 2 (and more particularly, from the console computer system 16) into robot control commands that are transmitted to the arm 4 on the robotic platform 5. The control tower 3 can also transmit status and feedback from the platform 5 back to the user console 2. The communication connection between the robotic platform 5, the user console 2 and the control tower 3 can be via a wired and / or wireless link, using any suitable protocol in a variety of data communication protocols. Any wired connection can be optionally built into the floor and / or wall or ceiling of the operating room. The robotic system 1 can provide video output to one or more displays, including displays within the operating room and remote displays accessible via the Internet or other networks. The video output or feed can also be encrypted to ensure privacy, and all or part of the video output can be saved to a server or electronic health care record system.

[0038] will understand, Figure 1 The operating room scenes in are illustrative and may not accurately represent certain medical practices.

[0039] refer to Figure 2, a schematic diagram of a UID with a finger clutch is shown according to an embodiment. The UID 14 may include a device housing 202 to be held by an operator or user 107. For example, the user 107 may hold the device housing 202 between several fingers and move the UID 14 within a workspace. The workspace may be a range within the reach of the user 107. As described below, the UID 14 may include a tracking sensor to detect the position and / or orientation of the device housing 202 when the user 107 moves the UID 14, and the detected position and / or orientation may be related to another component of the surgical robot system. For example, the tracking sensor may detect translation, rotation, or tilt of the device housing 202 within the workspace. The tracking sensor may include an accelerometer and / or a gyroscope or other inertial sensor. The movement of the UID 14 within the workspace may cause corresponding movement of a surgical robot arm, a surgical tool, or an end effector (e.g., a gripper or jaw) of a surgical tool of the surgical robot system.

[0040] The UID 14 may include a clutch mechanism to decouple the movement of the UID 14 from the movement of the surgical robot arm 112 and / or the surgical tool 104. For example, the UID 14 may include a finger clutch 206 mounted on the device housing 202 to clutch the surgical robot system. The finger clutch 206 may be so called because it can be actuated by a single pressing action from the finger of the user 107. That is, when the user 107 presses the finger clutch 206 with a finger, the touch can be detected as a clutch input. In response to the clutch input, the movement of the UID 14 detected by the tracking sensor may not be used by one or more processors to control the movement of the surgical robot system. When the clutch input is removed (when the touch ends), the movement of the UID 14 can again cause the corresponding movement of the surgical robot system. That is, when the finger clutch 206 is released, for example by removing the finger from the finger clutch 206, the UID 14 movement can be detected by the surgical robot system 1 again and used as a motion control input.

[0041] When the limit of the workspace has been reached, the clutch mechanism of the UID 14 can allow the user 107 to reposition the UID 14 within the workspace. For example, by fully extending the arm from the starting position in one direction while holding the UID 14, the user 107 can reach the limit of the workspace, such as the edge of the workspace. In order to reposition the UID 14 within the workspace and allow additional movement in the direction of the edge of the workspace, the user 107 can press the finger clutch 206 with the index finger to disconnect the robotic system from the movement of the UID 14. The user 107 can then move the UID 14 back to the starting position within the workspace and release the surgical robot system 1 by lifting the index finger from the finger clutch 206. Additional movement in the first direction can then be performed by moving the UID 14 to command movement of the surgical robot arm 112.

[0042] In addition, the UID 14 may also include a sensor 204 or a sensing portion for detecting an intentional user action, a sequence of intentional user actions, or a collection of intentional user actions for engaging and / or disengaging a teleoperation mode. In some embodiments, the sensor 204 may be positioned within, along, on, or otherwise near the finger clutch 206 so that it can be easily reached by the user's finger. For example, the sensor 204 may be a pressure sensor or a conductive sensor that is located within, along, on, or otherwise near the finger clutch 206. Representatively, the sensor 204 may be a portion of the external touch surface of the finger clutch 206 that faces outward toward the surrounding environment. When the finger of the user 107 touches the external touch surface of the finger clutch 206, the finger is separated from the conductive pad within the finger clutch 206 by the wall thickness of the clutch cover. The clutch cover may be formed from a dielectric material (eg, plastic), and thus, when a conductive finger of the user 107 touches the external touch surface, the capacitance across the walls of the finger clutch cover will change.

[0043] refer to Figure 3 , a perspective view of a UID is shown according to an embodiment. From this view, it can be seen that the device housing 202 of the UID 14 can further include a gripping surface 302 to be held between several fingers of the user 107. The device housing 202 can have one or more rounded or bulbous surface profiles. For example, the device housing 202 can be generally oval or egg-shaped, or it can be an ellipsoid. In an embodiment, a portion of the device housing 202 in front of a circumferential ridge 310 of the device housing 202 can be shorter and have a less gradual profile or taper than a portion of the device housing 202 behind the ridge 310.

[0044] In an embodiment, the finger clutch 206 is mounted on the housing end 304. For example, the housing end 304 can be a distal end of the device housing 202. The housing end 304 can be a location or surface located at the end of the housing 202 in the first longitudinal direction. For example, the location can be an edge of the housing 202 farthest from the opposite end of the housing (e.g., the proximal end 306).

[0045] Finger clutch 206 may extend distally from housing end 304. Positioning finger clutch 206 at the front of UID 14 may allow user 107 to easily reach forward and touch finger clutch 206, and in turn allow sensor 204 to detect such touch with an index finger while holding grip surface 302 between the thumb and another finger. Thus, UID 14 may be sized and shaped to be comfortably held in the hand of user 107. Command signals input through UID 14 may be transmitted to computer system 16 via a wired or, more preferably, wireless connection.

[0046] Representatively, in some embodiments, the UID 14 can be a portable handheld user input device or controller that is not grounded relative to another component of the surgical robot system. For example, the UID 14 can be ungrounded when it is connected (tethered) or not connected to the user console. The term "ungrounded" is intended to refer to such an implementation, in which, for example, the two UIDs are neither mechanically constrained nor kinematically constrained relative to the user console. For example, the user can hold the UID 14 in his hand and move freely to any possible position and orientation within a space limited only by, for example, the tracking mechanism of the user console. Signals (e.g., tracking sensor signals, clutch signals, or engagement / disengagement teleoperation mode signals) can be wirelessly communicated between the UID 14 and the computer system 16. In addition, a power source (such as a rechargeable battery) can be stored in the housing of the UID 14 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. Because the UID 14 can be unconnected and / or mechanically ungrounded, the remote operator 9 and / or bedside operator 8 can carry the UID 14 and move around one or more locations within the operating site (e.g., the bedside of the patient 6) during the procedure. In this regard, the UID 14 allows the operator 9 and / or operator 8 to control the arm 4 and / or surgical tool 7 from one or more locations within the operating site (e.g., various locations near the bedside of the patient 6). In addition, when at the bedside, the operator can use the UID 14 and the manual tool to control the arm 4 and / or surgical tool 7 simultaneously. The simultaneous manipulation of the arm 4 and / or surgical tool 7 and the manual tool reduces the need for another operator (e.g., a surgical assistant) to manipulate the manual tool while the remote operator 9 manipulates the arm 4 and / or surgical tool 7.

[0047] In addition, the portable handheld UID 14 will allow the operator 8 or 9 to alternate between sitting at the console 10 and controlling the arm 7 and / or tool 7 and standing at another location within the operating room (e.g., near the bedside) during the course of the procedure. For example, for some surgical procedures, the operator 8 or 9 may find it more comfortable or easier to control his or her movements while sitting at the console 10, while for others, being near the patient's bedside is better. Thus, the operator can move between different locations and / or positions as the procedure dictates.

[0048] In other embodiments, wires may extend from the UID 14 to connect the UID 14 to the computer system 16 and / or the console 10. The wires may provide power to the UID 14 and may transmit sensor signals, such as tracking sensor signals, clutch signals, or engagement / disengagement teleoperation mode signals, to the computer system 16. Thus, the UID 14 may be a peripheral device for inputting commands to the computer system 16. The UID 14 may be used in combination with other peripheral input devices. For example, a clutch pedal (e.g., (one or more) foot-operated controllers 13) may be connected to the computer system 16 to provide a clutch input or an engagement / disengagement input to the surgical robot system 1. Although each UID 14 may be individually clutched to pause teleoperation of the corresponding surgical robot arm or surgical tool, the corresponding surgical robot arm or tool may also be clutched simultaneously by pressing the clutch pedal. Thus, the movement of the actuator 17 may be commanded by the UID 14 and other peripheral input devices of the computer system 16 (such as a clutch pedal operated by the user's foot).

[0049] refer to Figure 4 , Figure 4A perspective view of a UID in a docking station according to an embodiment is illustrated. Representatively, the docking station 132 may form an internal chamber sized to receive the proximal end 306 of the UID 14. In addition, in some embodiments, the docking station 132 may further include a docking sensor 404 that detects when the UID 14 is in the docking station 132 or at the proximal end of the docking station 132. For example, the docking sensor 404 may be a pressure sensor or a capacitive sensor that can detect when the UID 14 contacts the surface of the docking station 132. In other aspects, the sensor 404 may be a sensor that detects when the UID 14 is very close to the docking station 132 but does not necessarily contact the docking station 132. For example, the sensor 404 may be a proximity sensor that detects the proximity of the UID 14 to the docking station 132. In further cases, in addition to or in place of the docking sensor 404, the UID 14 may also include a sensor for detecting when the UID 14 contacts or is very close to the docking station 132. Once the docking sensor 404 detects that the UID 14 is at the docking station 132, the docking sensor 404 sends a signal to the surgical robot system 1 indicating that the UID 14 is docked (or is near and about to be docked), and therefore, for example, the teleoperation mode should be disengaged. In addition, the docking sensor 404 can also detect when the UID 14 is no longer docked or no longer near the docking station 132, and therefore send a signal indicating that the user is ready to enter the teleoperation mode, and therefore the various components (e.g., actuators, tools and / or end effectors) are ready to re-engage.

[0050] Figure 5 is a block diagram of the computer portion of the surgical robot system according to an embodiment. Figure 5As shown in , UID14 may include UID 14L and UID 14R. UID 14L can be operated by the user's left hand, while UID 14R can be operated by the user's right hand. For example, UID 14L can be used to control surgical robot components such as the left side of the patient, while UID 14R can be used to control surgical robot components such as the right side of the patient, as previously discussed. Each of UID 14L and UID 14R can include an internal volume for receiving various electronic devices and / or other components. For example, each of UID 14L and 14R can include a UID processor 506 installed in the device housing 202. The UID processor 506 may include circuits for analog and digital signal processing, including sense amplifier circuits and analog-to-digital conversion circuits for docking with capacitive sensors, and logic circuits including programmable logic or programmable digital processors. The UID processor 506 can be mounted on a printed circuit board with various sensor terminals to connect the UID processor 506 to device sensors, such as a finger clutch sensor 502 or a squeeze sensor 504. A battery (not shown) may be mounted on or otherwise associated with the printed circuit board to power the electronic components of UIDs 14L and 14R.

[0051] Each of the UIDs 14R and 14L may further include a finger clutch sensor 502. The finger clutch sensor 502 may be used in conjunction with, for example, the previously described Figure 4The sensor 204 discussed is substantially the same. Representatively, the finger clutch sensor 502 can be a sensor that can detect a predetermined intentional user action or a set of intentional user actions that indicates that the user desires to engage or disengage the teleoperation mode. For example, the finger clutch sensor 502 can be a capacitive sensor associated with the finger clutch 206, and the finger clutch 206 can be actuated when the user 107 taps a finger on the finger clutch cover. The UID processor 506 can be configured to determine that a predetermined intentional user action (or a sequence of intentional user actions) has been performed by the user 107 in response to a change in the capacitance of the finger clutch sensor 502. For example, in an embodiment, the UID processor 506 can be configured to determine that a predetermined intentional user action, a sequence of intentional user actions, or a set of user actions has occurred when the change in capacitance lasts for a predetermined period of time and / or repeats itself before a predetermined time. For example, the finger clutch sensor 502 can detect a single or double finger tap on the finger clutch 206 by the user 107 and transmit a corresponding signal to the UID processor 506. A single or double tap can be a gesture by the user 107 that includes touching a finger on the finger clutch sensor 502 for a predetermined time period (e.g., 0.5 seconds or less) and at a predetermined interval (e.g., 0.5 seconds or less between taps) in the case of a double tap. When the change in capacitance detected within the predetermined time period (and / or interval) is greater than a predetermined threshold, the UID processor 506 can determine that the user 107 has performed one of the predetermined intentional actions or a set of intentional actions, indicating a desire to engage and / or disengage the teleoperation mode. Therefore, the UID processor 506 can then generate a corresponding signal, which is transmitted to the surgical system processor 912 (e.g., computer system 16). The surgical system processor 912 can then determine whether the signal indicates a desire to engage the teleoperation mode or disengage the teleoperation mode, for example, based on whether the system is already in the teleoperation mode. Once the associated command is determined, an engage signal to engage the teleoperation mode or a disengage signal to disengage the teleoperation mode may be generated or otherwise initiated by the processor to cause the surgical robotic system to engage (e.g., enter) or disengage (e.g., exit) the teleoperation mode.

[0052] It will be appreciated that the finger clutch sensor 502 may be one or more sensor types for detecting a touch by the user 107. More particularly, while the finger clutch sensor 502 has been primarily described as including a capacitive sensor, the finger clutch sensor 502 may be a different type of sensor or combination of sensors to determine that the user 107 has touched the finger clutch. For example, the finger clutch sensor 502 may include a proximity sensor that may detect the proximity of the user's 107 finger to the finger clutch 206, or generally the UID 14L or 14R, before the user's 107 finger actually contacts the surface. Thus, the above-described embodiments are intended to include different types of sensors that detect touch based on the presence or proximity of an object, and in some cases, there is no need to detect a threshold force applied by an object on the finger clutch 206.

[0053] UIDs 14L and 14R may further include a squeeze sensor 504 associated with the device housing 202. In an embodiment, the squeeze sensor 504 generates a squeeze signal when squeezed. More particularly, the squeeze sensor 504 is configured to generate a squeeze signal in response to squeezing on the device housing 202. Thus, the squeeze sensor 504 may detect when the user 107 engages in an intentional user action, a sequence of intentional user actions, or a collection of intentional user actions involving squeezing the device housing 202 as an indication of the user's desire to engage or disengage the teleoperation mode. For example, the squeeze sensor 504 may detect when the user 107 squeezes the device housing 202 once or repeatedly squeezes the device housing 202. The squeeze sensor 504 may then output a corresponding squeeze signal to the UID processor 506, which in turn transmits the signal to the surgical robot system processor 912. The surgical robot system processor 912 may then determine whether the squeeze signal indicates that the user desires to engage and / or disengage the teleoperation mode. For example, when a squeeze is detected within a predetermined time period (e.g., 0.5 seconds or less) (and in the case of a double squeeze, at a predetermined interval (e.g., less than 0.5 seconds between squeezes)), the processor will interpret this as an intentional user action, a sequence of intentional user actions, or a collection of intentional user actions to engage and / or disengage the teleoperation mode. In some cases, the squeeze sensor 504 can be a grip flex circuit or can be associated with a grip flex circuit that will deform when the user 107 squeezes the device housing 202 and the physical deformation can be converted into an electrical signal, such as a capacitive signal. In other cases, the squeeze sensor 504 can be an optical sensor, such as a proximity sensor that can detect the proximity (or change in distance) of the inner wall of the device housing 202. In another example, the squeeze sensor 504 can include an ultrasonic sensor, a magnetic sensor, an inductive sensor, or other suitable type of proximity sensor. Furthermore, it should be understood that in some embodiments, the user may also squeeze the device housing 202 to control a gripping action of the end effector, however, the squeezing associated with a gripping action is different from an intentional single or double squeezing action indicating a desire by the user to engage and / or disengage a teleoperation mode, and the processor 912 is programmed to distinguish between the two to avoid any unintentional operation.

[0054] The UID 14 may include other circuits. For example, the UID 14 may include a drop detection sensor to prevent unintentional instrument movement, for example, when the UID 14 is dropped. For example, the drop detection sensor may generate a drop signal in response to entering a free fall state when dropped. In an embodiment, the drop detection sensor may be a tracking sensor that monitors the movement of the UID 14. When the tracking sensor detects movement corresponding to a drop state, the sensor generates a clutch signal to pause all movements of the surgical robot system 1. In addition, in some embodiments, the UID 14 may include a docking sensor. For example, the previously discussed docking sensor 404 may be coupled to the UID 14 in addition to or in place of the docking station 132. The docking sensor may detect when the UID 14 is docked, or otherwise approaches the docking station 132 in a manner that suggests that the user is about to dock the UID 14.

[0055] Alternatively, each of the docking stations 510L and 510R associated with a respective one of the UIDs 14L and 14R may include a docking sensor 512. The docking sensor 512 may be associated with the previously referenced Figure 4 The docking sensors 404 discussed are substantially the same. For example, docking sensors 512 may be associated with (e.g., mounted to) each docking station 510L and 510R and may detect when a corresponding one of the UIDs 14L and 14R is positioned within or at the proximal end of the docking stations 510L and 510R. For example, docking sensors 512 may be pressure sensors or capacitive sensors that may detect when the UID 14L or 14R contacts the surface of the docking station 510L or 510R. In other aspects, docking sensors 512 may be sensors that detect when the UID 14L or 14R is very close to the docking station 510L or 510R but not necessarily in contact with the docking station 510L or 510R. Once docking sensor 512 detects that the UID 14L or 14R is at the docking station 510L or 510R, docking sensor 512 sends a signal to the surgical system processor 912. The surgical system processor 912 can then use this information to determine whether this is an intentional user action indicating the user's desire to engage and / or disengage the teleoperation mode. In addition, the docking sensor 512 can also detect when the UID 14L or 14R is no longer docked or no longer near the docking station 510L or 510R, and send a corresponding signal to the surgical system processor 912 for use in determining whether to engage and / or disengage the teleoperation mode.

[0056] The surgical robot system may further include a foot-operated controller such as a foot pedal 13. The user 107 may use the foot pedal 13 to control surgical robot components (e.g., actuators, tools, and / or end effectors) or, for example, to pause operation in the case of a clutch pedal, as previously discussed. For example, the user 107 may press on the foot pedal 13 to control the corresponding surgical robot component, or to pause the operation of the surgical robot component (in the case of a clutch pedal). In some cases, the foot pedal 13 may include or may be a clutch pedal having a clutch sensor 508, which is similar to any of the sensors 502, 504, and 512 discussed previously, and may be used to detect a predetermined intentional user action or a set of intentional user actions indicating that the user 107 desires to engage or disengage the teleoperation mode. However, in this case, the action may be performed with the user's foot. For example, the foot pedal clutch sensor 508 may detect an intentional user action, such as a single tap of the user's foot on the clutch pedal, or a double tap of the user's foot on the clutch pedal. For example, when a single foot tap on the foot pedal is detected within a predetermined time period (e.g., 0.5 seconds or less), and in the case of a double tap, at a predetermined interval (e.g., 0.5 seconds or less between taps), the processor will interpret this as an intentional user action, sequence of intentional user actions, or collection of intentional user actions indicating a desire to engage and / or disengage the teleoperation mode. Once detected, the foot pedal clutch sensor 508 can send a corresponding signal to the surgical system processor 912, and the processor 912 can determine whether to engage and / or disengage the teleoperation mode based on the signal and, for example, whether the system is already in the teleoperation mode.

[0057] Now refer to Figure 6-Figure 7 Describing in more detail intentional user actions, sequences of intentional user actions, or sets of intentional user actions, and various combinations thereof, which may be detected by the sensors disclosed herein for use in determining whether to engage and / or disengage a teleoperation mode. Representatively, Figure 6 is a block diagram of an exemplary process for disengaging from teleoperation mode, and Figure 7 and Figure 8An exemplary process for engaging a teleoperation mode is illustrated once an intentional user action or a set of intentional user actions is detected. As previously discussed, an intentional user action, a sequence of intentional user actions, or a set of intentional user actions may include a single action, a combination of repeated actions, and / or actions detected by the surgical robot system, and it is recognized by the surgical robot system as a user command for engaging and / or disengaging the teleoperation mode. In addition, in some embodiments, a sequence of the same intentional user action or a set of intentional user actions for, for example, disengaging from the teleoperation mode may also be used to engage the teleoperation mode. In this regard, an intentional user action, a sequence of intentional user actions, or a set of intentional user actions for engaging and / or disengaging from the teleoperation mode may be considered as a symmetrical action. In other embodiments, an intentional user action, a sequence of intentional user actions, or a set of intentional user actions are different or asymmetric. It should be further recognized that, because both UIDs 14L and 14R can be mechanically ungrounded, uncoupled from, or unrestricted by any other component of the surgical robot system, various combinations of intentional user actions, a sequence of intentional user actions, or a set of intentional user actions and / or actions can be easily and performed at any number of locations within the surgical site.

[0058] Now refer to it in more detail Figure 6 , Figure 6 Illustrated is a process for disengaging a teleoperation mode, process 600 may include an initial operation of detecting intentional user action(s) (block 602). Representatively, in one embodiment, the intentional sequence of user actions(s) detected by the surgical robotic system is a double tap of the UID finger clutches on both UIDs simultaneously. For example, a user may double tap the left UID 14L and the right UID 14R simultaneously, and each double tap may be detected by a corresponding finger clutch sensor 502 associated with each UID 14L and 14R, as previously discussed. It should be appreciated that a double tap or other intentional user action(s) detected by the finger clutch sensor 502 for engaging or disengaging a teleoperation mode is distinct from a user input that would cause a clutch operation (e.g., pressing a finger clutch) to pause the system, as previously discussed.

[0059] In another embodiment, the (one or more) intentional user actions detected at operation 502 can be docking actions, wherein one or more of the UIDs are docked in or positioned near their respective docking stations. Representatively, docking of both UID 14L and UID 14R, docking one of UID 14L or 14R, or positioning one or both of UID 14L and 14R near their respective docking stations 510L and / or 510R can be (one or more) intentional actions detected and indicating a desire by the user to disengage from teleoperation mode. For example, in one aspect, docking of both UID 14L and 14R (or positioning both UID 14L and 14R near a docking station) can result in disengagement from teleoperation mode, and all operations controlled by UID 14L and 14R can be prevented. In other embodiments, docking of one of UID 14L or 14R (or positioning one of UID 14L and 14R near a docking station) can result in disengagement from the teleoperation mode, and all operations controlled by UID 14L and 14R can be prevented. In further embodiments, docking of UID 14L alone (or positioning UID 14L near a docking station) can be used to disengage from the teleoperation mode only with respect to surgical robot components controlled by UID 14L, and not with respect to surgical robot components controlled by UID 14R. In other words, the user can continue to operate the surgical robot components controlled by UID 14R. Similarly, docking of UID 14R (or positioning UID 14R near a docking station) can be used to disengage from the teleoperation mode only with respect to surgical robot components controlled by UID 14R, and not with respect to surgical robot components controlled by UID 14L. Furthermore, it should be appreciated that docking of one or both of UIDs 14L and / or 14R may be used to disengage a teleoperation mode of any number of surgical robotic components (e.g., one surgical robotic component, two surgical robotic components, three surgical robotic components, or more).

[0060] It should be further appreciated that while specific intentional user action(s) have been referenced, it is contemplated that multiple intentional user actions and / or combinations of intentional user actions(s) may be detected with respect to various input devices of the system in operation 602. Table 1 provides a list of predetermined intentional user action(s), sequences of user actions(s), sets of intentional user actions(s), and / or combinations of intentional user actions(s) that may be performed in combination (e.g., in sequence) and detected and recognized by the system as indicating a user desire to engage and / or disengage a teleoperation mode.

[0061] Table 1

[0062] .

[0063] It should be further understood that in some embodiments, in addition to the intentional user action(s) previously discussed, the user may also press and hold the clutch pedal prior to performing any one or more of the action(s). For example, in one embodiment, the user may press and hold the clutch pedal and then (1) single tap the two finger clutch, (2) double tap the two finger clutch, (3) single squeeze the two UIDs, and / or (4) double squeeze the two UIDs. For example, when the UIDS 14L and 14R are in a position / orientation that makes the finger clutch difficult to reach, if the user desires to disengage, the user may press and hold the clutch pedal, orient the UID to better reach the finger clutch, double tap the UID finger clutch (the system disengages at this point), and release the clutch pedal.

[0064] Returning now to process 600, process 600 may further include an operation of determining that the surgical robotic system is in teleoperation mode (block 604). For example, since, as previously discussed, in some embodiments, the intentional user action(s) indicating a desire to disengage the teleoperation mode may be the same as those intentional user actions used to engage the teleoperation mode, the current mode of the surgical robotic system may be determined in order to account for the action(s). In the event that the surgical robotic system is determined to be in teleoperation mode, in operation 606, a sequence of intentional actions(s) is determined to indicate a desire to disengage the teleoperation mode. Then, in operation 608, the teleoperation mode of the surgical robotic system is disengaged.

[0065] In addition, in some embodiments, the process 600 includes an optional user feedback operation (box 610) that provides feedback to the user, alerting them to the mode transition and / or the current operating mode. Representatively, the feedback can be visual (e.g., on-screen or LED lights on the display 15 or console 2), auditory (e.g., tones from the surgeon bridge or control tower 130), or tactile (e.g., UID vibration). For example, when the user is out of teleoperation mode, any one or combination of these feedback systems is possible. For example, a tactile feedback mechanism can be incorporated into one or more of the UIDs 14L and 14R, which will give the user a physical feeling that they have been out of control of the associated surgical robot tool. The visual feedback can be an LED light strip on or around the display 15, and for example, a green light can indicate that the system is out of teleoperation mode, while a red light can indicate that the system is engaged in teleoperation mode. Further, the user can receive visual feedback on the screen from a graphical user interface (which can involve text, icons, or both) to alert them to the current mode. The notification may automatically disappear after the user completes disengagement of teleoperation mode, or after a current time period. Further consideration is given to auditory warnings (eg, beeps, tones, etc.).

[0066] Figure 7 A process for engaging a teleoperation mode is illustrated. Process 700 may include an initial operation of detecting (one or more) intentional user actions (box 702). Representatively, in one embodiment, the (one or more) intentional user actions detected by the surgical robotic system are to double-tap the UID finger clutches on both UIDs simultaneously to engage the teleoperation mode. For example, a user may double-tap the left UID 14L and the right UID 14R simultaneously, and each double-tap may be detected by the corresponding finger clutch sensor 502 associated with each UID 14L and 14R, as previously discussed.

[0067] It should be further appreciated that while specific intentional user actions (one or more) indicating a desire to engage a teleoperation mode have been mentioned, it is contemplated that multiple intentional user actions and / or combinations of intentional user actions (one or more) may be performed in combination (e.g., in sequence) using one or more input devices of the system that may be detected in operation 702. Representatively, in one embodiment, the intentional user action (s) detected by the surgical robotic system may be any one or more of the intentional user actions listed in Table 2.

[0068] Table 2

[0069] .

[0070] It should be further understood that in some embodiments, in addition to the intentional user action(s) previously discussed, the user may also press and hold the clutch pedal prior to performing any one or more of the action(s). For example, in one embodiment, the user may press and hold the clutch pedal and then (1) single tap the two finger clutch, (2) double tap the two finger clutch, (3) single squeeze the two UIDs, and / or (4) double squeeze the two UIDs. For example, if the user needs to orient UIDs 14L and 14R in a particular manner (e.g., to match tools), the user may also press and hold the clutch pedal and then double tap the UID finger clutch (on one or both of UIDs 14L and 14R), orient the UIDs as desired, and then release the clutch pedal. Additionally, in some embodiments, the user may need to perform various unlock gestures prior to engaging the teleoperation mode, such as using a graphical user interface lock icon and then aligning the UIDs, as will be described with reference to FIG. Figure 8 As discussed in more detail.

[0071] Returning now to process 700, process 700 may further include an operation of determining that the surgical robotic system is not in teleoperation mode (block 704). For example, since, as previously discussed, in some embodiments, the intentional user action(s) or sequence of intentional user actions indicating a desire to engage teleoperation mode may be the same as those intentional user actions used to disengage teleoperation mode, the current mode of the surgical robotic system may be determined in order to account for the action(s). In the event that it is determined that the surgical robotic system is not in teleoperation mode, in operation 706, a sequence of intentional actions is determined to indicate a desire to engage teleoperation mode. Then, in operation 708, the teleoperation mode of the surgical robotic system is engaged.

[0072] In addition, in some embodiments, the process 700 includes an optional user feedback operation (box 710) that provides feedback to the user, alerting them to mode conversion and / or current operation mode. Representatively, the feedback can be visual (e.g., on-screen or LED lights on the display 15 or console 2), auditory (e.g., tones from the surgeon's bridge or control tower 130), or tactile (e.g., UID vibration). For example, when the user engages the teleoperation mode, any one or combination of these feedback systems is possible. For example, a tactile feedback mechanism can be incorporated into one or more of the UIDs 14L and 14R, which will give the user a physical sense that they have engaged the control of the associated surgical robot tool. The visual feedback can be an LED light strip on or around the display 15, and for example, a red light can indicate that the system is engaged in the teleoperation mode, while a green light can indicate that the system is disengaged from the teleoperation mode. Further, the user can receive visual feedback on the screen from a graphical user interface (which can involve text, icons, or both), alerting the user that the system is in the teleoperation mode. After the user completes engagement of teleoperation mode, or after a current time period, the notification may automatically disappear. Further consideration is given to audible warnings (eg, beeps, tones, etc.).

[0073] In addition, it should be recognized that, prior to engaging the teleoperation mode, in addition to the (one or more) intentional user actions described herein, there may be additional safety requirements that must be met to ensure that it is safe to enter the teleoperation mode. Representatively, the surgical robotic system may further include additional or optional intentional actions, including alignment requirements that must be met prior to engaging the teleoperation mode. Therefore, it is contemplated that in some embodiments, the detection of (one or more) intentional user actions, such as described with reference to operation 700, may occur, for example, after these additional or optional actions occur and prior to the final alignment operation. The process is Figure 8. Representatively, the process 800 for engaging the teleoperation mode includes determining that the surgical robot system is in a non-teleoperation mode (802), and detecting a first set or sequence of intended user actions (box 804). Representative actions can be, for example, locking a chair, detecting that a user is looking at a display screen, detecting that the UID is not docked, and that the UID is in a workspace (e.g., using a tracking system). Next, a second set or sequence of (one or more) intended user actions is detected, which indicates that it is desired to enter the teleoperation mode as previously discussed (box 806). The process 800 further includes an additional operation of detecting an alignment of at least one of the UIDs with a corresponding surgical robot component (e.g., an actuator, a tool, and / or an end effector) (box 808). For example, during a surgical operation, a surgical robot tool can be positioned within a patient's body and the rotation / orientation of the tool is displayed to the user. Prior to engaging teleoperation, the user must align the rotation / orientation of the UID (where it is held) with the rotation / orientation of the tool within the patient, as viewed on the display. Then, once each of the first intentional user action and the second intentional user action(s) are detected, and alignment operations are satisfied (eg, operations 804 - 808 ), teleoperation mode is engaged (block 810 ).

[0074] Reference now Fig. 9 , Fig. 9 1 is a block diagram of a computer portion of a surgical robotic system for performing the previously discussed operations, shown according to an embodiment. The surgical robotic system 1 may generally include (one or more) UIDs 14, a user console 2 having a computer system 16, and robotic components 104, 112 associated with an actuator 17. The computer system 16 and the UID 14 have circuits suitable for specific functions, and therefore, the illustrated circuits are provided as examples and not limitations.

[0075] One or more processors of the user console 2 may control portions of the surgical robotic system 1, such as the surgical robotic arm 112 and / or the surgical tool 104. The UID 14 may be communicatively coupled to the computer system 16 (of the user console 2) and / or the surgical robotic system 1 to provide input commands processed by the one or more processors of the system 1 to control movement of the arm-mounted surgical robotic arm 112 and / or the surgical tool 104. For example, the UID 14 may transmit electrical command signals 902 to the computer system 16, such as spatial state signals generated by the UID processor 606 in response to signals from the tracking sensor 922, or clutch signals generated by the UID processor 606. The electrical signals may be input commands to cause movement of the surgical robotic system 1 or to pause movement of the surgical robotic system 1. In addition, the input commands may correspond to (one or more) intentional user actions detected by the UID 14 indicating a desire to engage and / or disengage the teleoperation mode, and the processor of the system 1 may use them to engage and / or disengage the teleoperation mode, as described with reference to FIG. Figure 6-Figure 8 Just as described.

[0076] The input electrical signal may be transmitted by the UID processor 606 to the console processor 906 of the computer system 16 via a wired or wireless connection. For example, the UID 14 may transmit the command signal 902 to the console processor 906 via an electrical wire. Alternatively, the UID 14 may transmit the command signal 902 to the console processor 906 via a wireless communication link 904. The wireless communication link may be established by the corresponding RF circuits of the computer system 16 and the UID 14. The wireless communication may be via a radio frequency signal, such as a Wi-Fi or short-range signal and / or a suitable wireless communication protocol, such as Bluetooth.

[0077] The console processor 906 of the computer system 16 can execute instructions to perform the various functions and capabilities described above. The instructions executed by the console processor(s) 906 of the user console 2 can be retrieved from a local memory (not shown), which can include non-transitory machine-readable media. The instructions can be in the form of an operating system program with a device driver to control components of the surgical robotic system 1 (e.g., an actuator 17 operably coupled to the surgical robotic arm(s) 112 or the surgical tool(s) 104), and / or engage / disengage a teleoperation mode.

[0078] In an embodiment, the console processor 906 controls components of the user console 2. For example, one or more seat actuators 909 may receive commands from the console processor 906 to control the movement of the seat 122. The seat actuator (s) 909 may move the seat 122 in one or more degrees of freedom (such as forward / backward, backrest tilt, headrest position, etc.), for example, to align the seat 122 with the display 15 (to engage the teleoperation mode). The console processor 906 may also transmit video data for presentation on the display 15. Thus, the console processor 906 may control the operation of the user console 2. Input commands to the seat actuator (s) 909 or the console processor 906 may be input by the user via the foot pedal (s) 13 or another input device 911 such as a keyboard or joystick.

[0079] The console processor 906 can output control signals 903 to other components of the surgical robot system 1 via a link 910. The control signals 903 can be transmitted to control the movement of the surgical robot system 1. In an embodiment, the computer system 16 is communicatively coupled to downstream components of the surgical robot system 1, such as a control tower 130, via a wired or wireless link. The link can transmit the control signals 903 to one or more surgical system processors 912. For example, at least one processor 912 can be located in the control tower 130 and can be communicatively coupled to system components, such as a surgical robot platform 5 or one or more displays 920. The actuator 17 of the surgical robot system 1 can receive control signals from the surgical system processor 912 to cause movement of the arm 112 and / or tool 104 corresponding to the movement of the UID 14. When the user 107 presses the finger clutch 206 or puts down the UID 14, the control signal can also pause the movement of the robot component by clutching and / or disconnecting the interlock of the surgical robot system 1, or can cause the system to engage and / or disengage the teleoperation mode.

[0080] In the foregoing description, the present invention has been described with reference to specific exemplary embodiments of the present invention. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present invention as set forth in the following claims. Accordingly, the description and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A method for engaging and disengaging a surgical instrument of a surgical robotic system, the method comprising: receiving a user position input, the user position input comprising a user action indicating whether a user is looking at a display screen of the surgical robotic system; receiving a sequence of user inputs from one or more user interface devices of the surgical robotic system, wherein the one or more user interface devices include: at least one handheld user input device having a finger clutch and a squeeze sensor, the squeeze sensor including a flexible circuit operable to deform in response to squeezing of the handheld user input device, and a clutch pedal, and the sequence of user inputs includes tapping of the finger clutch or clutch pedal or squeezing of the handheld user input device for a predetermined period of time; receiving user alignment input, the user alignment input comprising an action indicating whether at least one or more of the user interface devices are aligned with the surgical robot component; determining, by one or more processors communicatively coupled to the one or more user interface devices and the surgical instrument, whether the user position input, the sequence of user inputs, and the user alignment input indicate an intended engagement or disengagement of a teleoperation mode in which the surgical instrument is controlled by the user input received from the user interface device; In response to determining engagement, transitioning the surgical robotic system into a teleoperation mode; and In response to determining the disengagement, the surgical robotic system is transitioned out of the teleoperation mode such that the one or more user interface devices are prevented from controlling the surgical instrument.

2. The method of claim 1, wherein the handheld user input device is a first handheld user input device, and the at least one or more user interface devices further comprises a second handheld user input device, the second handheld user input device comprising a finger clutch; in, The user position input further includes locking a chair of the surgical robotic system, docking the first handheld user input device or the second handheld user input device, or positioning the first handheld user input device or the second handheld user input device in a workspace of the surgical robotic system; wherein the sequence of user inputs comprises a single tap of a finger clutch associated with the first and second handheld user input devices, a double tap of a finger clutch associated with the first or second handheld user input device, a single squeeze of the first and second handheld user input devices, or a double squeeze of the first or second handheld user input device; and Wherein, the user alignment input action includes detecting alignment between the first or second handheld user input device and the surgical robot tool.

3. The method of claim 1, wherein: The sequence of user inputs indicating engagement or disengagement of teleoperation mode is the same.

4. The method of claim 1, wherein: The sequence of user inputs includes a sequence of first user inputs from at least one handheld user input device and a sequence of second user inputs from a clutch pedal.

5. The method of claim 4, wherein: The sequence of first user inputs received from the handheld user input device includes a double tap of a finger clutch, and the sequence of second user inputs received from the clutch pedal includes a single tap of the clutch pedal preceding the sequence of first user inputs.

6. The method of claim 1, wherein: The sequence of user inputs indicating intended engagement or disengagement corresponds to a single tap of the clutch pedal within a predetermined time period or a double tap of the clutch pedal within a predetermined interval and a single tap of the finger clutch within a predetermined time period or a double tap of the finger clutch at a predetermined interval.

7. The method of claim 1, wherein: The sequence of user inputs indicating intended engagement or disengagement corresponds to a single tap of the clutch pedal within a predetermined time period or a double tap of the clutch pedal within a predetermined interval and a single squeeze of the handheld user input device within a predetermined time period or a double squeeze of the handheld user input device at a predetermined interval.

8. The method of claim 1, wherein: The sequence of user inputs indicating an intentional engagement or disengagement corresponds to a single tap of the clutch pedal and a double tap of the finger clutch.

9. The method of claim 1, wherein: The user position input further includes determining whether at least one of the following conditions is met before converting the surgical robotic system into the teleoperation mode: a chair of the surgical robotic system is locked, and a user interface device is within a surgical robotic system workspace.

10. The method of claim 1, further comprising user feedback for alerting a user that the teleoperation mode is engaged or disengaged.

11. A surgical robot system, comprising: Surgical instruments; A user console comprising: first and second handheld user input devices, each handheld user input device having a finger clutch and a squeeze sensor, the squeeze sensor comprising a flexible circuit operable to deform in response to squeezing of the first and second handheld user input devices; and a foot pedal including the clutch sensor, and wherein the first and second handheld user input devices are mechanically ungrounded relative to the user console; and One or more processors communicatively coupled to the surgical instrument and the user console, the processors configured to: receiving a sequence of user actions, wherein the sequence of user actions includes at least a single tap of a foot pedal detected by a clutch sensor and at least a single squeeze of each of a first and a second handheld user input device within a predetermined time period; determining that the surgical robotic system is in a non-teleoperation mode and that the sequence of user actions indicates an intended engagement; and The surgical robotic system is converted into a teleoperation mode in which the surgical instrument is controlled by user input received from the first and second handheld user input devices and the foot pedal.

12. The surgical robot system according to claim 11, wherein: The sequence of user actions comprises a single squeeze of each of the first and second handheld user input devices within the same predetermined time period.

13. The surgical robot system according to claim 11, wherein: The sequence of user actions includes pressing and holding the clutch pedal, and double squeezing each of the first and second handheld user input devices.

14. The surgical robot system according to claim 11, wherein: The sequence of user actions further includes a user position action indicating whether the user is looking at a display screen of the surgical robotic system, or a user alignment action indicating whether the first and second handheld user input devices are aligned with the surgical robotic component.

15. A surgical robot system, comprising: Surgical instruments; a user console comprising: first and second handheld user input devices, each handheld user input device comprising a finger clutch and a squeeze sensor, the squeeze sensor comprising a flexible circuit operable to deform in response to squeezing of the handheld user input device; and a foot pedal, the first and second handheld user input devices being mechanically ungrounded relative to the user console; and One or more processors communicatively coupled to the surgical instrument and the user console, the processors configured to: receiving a sequence of user actions, wherein the sequence of user actions includes a tap of a finger clutch of the first or second handheld user input device or a squeeze of the first or second handheld user input device, and a docking of the first or second handheld user input device; determining that the surgical robotic system is in a teleoperation mode, wherein the surgical instrument is controlled by user inputs received from first and second handheld user input devices and a clutch pedal, and that the sequence of user actions indicates an intentional disengagement; and The surgical robotic system is converted into a non-teleoperation mode in which the first and second handheld user input devices are prevented from controlling the surgical instrument.

16. The system of claim 15, wherein: The sequence of user actions further includes double tapping the clutch pedal.

17. The system of claim 15, wherein: The sequence of user actions further includes pressing and holding a clutch pedal, and double-tapping a finger clutch of each of the first and second handheld user input devices.

18. The system of claim 15, wherein: The user action further includes pressing and holding a clutch pedal, and double squeezing each of the first and second handheld user input devices.