Method and system for controlling movement of a surgical robot within a range of motion

Abstract

A method performed by a surgical system that includes a surgical tool and a human interface device (HID) configured to control the surgical tool. The method receives, via the HID, a user command that includes a target pose of an end effector of the surgical tool, wherein the surgical tool comprises a shaft coupled to the end effector, disposed between the shaft and the end effector includes several joints, determines a range of motion of the end effector along a control point on the surgical tool, determines whether the target pose of the end effector with respect to the control point is to extend beyond the range of motion of the end effector, and responsive to determining that the target pose of the end effector is to extend beyond the range of motion, adjusts the target pose of the end effector to stay within the range of motion.

Description

Atorney Docket No. : AUR6376WOPCT1Electronically FiledMETHOD AND SYSTEM FOR CONTROLLING MOVEMENT OF A SURGICAL ROBOT WITHIN A RANGE OF MOTIONRELATED APPLICATION

[0001] This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 707,187, filed October 14, 2024, which are herein incorporated by reference in their entirety.FIELD

[0002] Various embodiments of the disclosure relate generally to surgical systems, and more specifically to a surgical system for determining a range of motion of a surgical robot and controlling movement of the robot within that range. Other embodiments are also described.BACKGROUND

[0003] Minimally-invasive surgery, MIS, such as laparoscopic surgery, uses techniques that are intended to reduce tissue damage during a surgical procedure. Laparoscopic procedures typically call for creating a number of small incisions in the patient, e.g., in the abdomen, through which several surgical tools such as an endoscope, a blade, a grasper, and a needle, are then inserted into the patient. A gas is injected into the abdomen which insufflates the abdomen thereby providing more space around the tips of the tools, making it easier for the surgeon to see (via the endoscope) and manipulate tissue at the surgical site. MIS can be performed faster and with less surgeon fatigue using a surgical robotic system in which the surgical tools are operatively attached to the distal ends of robotic arms, and a control system actuates the arm and its attached tool. The tip of the tool will mimic the position and orientation movements of an input device as the latter is being manipulated by the surgeon. The surgical robotic system may have multiple surgical arms, one or more of which has an attached endoscope, and others have attached surgical instruments for performing certain surgical actions.Atorney Docket No. : AUR6376WOPCT1Electronically FiledSUMMARY

[0004] According to one embodiment of the disclosure, a method performed by a surgical system that includes a surgical tool and a human interface device (HID) configured to control the surgical tool, the method including: receiving, via the HID, a user command that includes a target pose of an end effector of the surgical tool, where the surgical tool includes a shaft coupled to the end effector, disposed between the shaft and the end effector includes several joints; determining a range of motion of the end effector along a control point on the surgical tool; determining whether the target pose of the end effector with respect to the control point is to extend beyond the range of motion of the end effector; and responsive to determining that the target pose of the end effector is to extend beyond the range of motion, adjusting the target pose of the end effector to stay within the range of motion.

[0005] In one embodiment, the range of motion includes a coned boundary having a vertex at the control point about a vector that extends to a fixed point on the shaft of the surgical tool. In another embodiment, the fixed point includes a remote center of motion (RCM) of the surgical tool. In some embodiments, the end effector includes a grasper. In some embodiments, the joints includes a first joint having a first joint coupled between the shaft and a link and a second joint coupled between the link and the end effector, where the range of motion is determined as a minimum range of motion between: a first range of motion of the end effector when the first joint is at a first joint limit and the second joint is at a first midpoint joint angle within a first joint range; and a second range of motion of the end effector when the second joint is at a second joint limit and the first joint is at a second midpoint joint angle within a second joint range.

[0006] In one embodiment, , responsive to determining that the target pose of the end effector is to stay within the range of motion, providing joint commands to the joints to move the end effector from a current pose to the target pose. In some embodiments, the user command indicates that the end effector is to perform a roll motion about an axis that extends longitudinally through the end effector to achieve the target pose, where the roll motion is to cause the joints to rotate about each of their respective axes while the control point is to remain in place. In another embodiment,Atorney Docket No. : AUR6376WOPCT1Electronically Filed each of the joints of the joints includes a different joint limit. In another embodiment, the range of motion includes a rotational limit around a rotational axis of the control point, where adjusting the target pose includes rotating the target pose with respect to the rotational axis such that an end effector angle is within the rotational limit.

[0007] According to another embodiment of the disclosure, a surgical system including: a surgical tool having a shaft, a first joint arranged to rotate about a first axis, a second joint arranged to rotate about a second axis, different from the first axis, and an end effector, the first joint coupled to the shaft and the second joint is coupled between the first joint and the end effector; a human interface device (HID) arranged to control the surgical tool; at least one processor; and memory having instructions stored therein which when executed by the at least one processor causes the surgical system to: determine a target pose of the end effector based on movement of the HID; determine that the target pose extends beyond a cone limit that is at a control point on the second joint, where a cone axis of the cone limit passes through the control point and a fixed point on the shaft of the surgical tool; and adjust the target pose such that the target pose is in or on the cone limit.

[0008] In one embodiment, the memory includes further instructions to: determining a tool type of the surgical tool; and determining the cone limit based on the tool type. In another embodiment, the tool type includes a needle driver that has a pair of jaws as the end effector. In some embodiments, the cone limit is determined based on a first range of motion of the first joint along the first axis and a second range of motion of the second joint along the second axis, which is different than the first axis. In another embodiment, at least one of the first range of motion or the second range of motion is less than a first joint limit of the first joint or a second joint limit of the second joint, respectively. In one embodiment, the fixed point includes a remote center of motion (RCM) of the surgical tool. In another embodiment, a position of the RCM along the shaft is a predefined position.

[0009] According to another embodiment of the disclosure, a non-transitory machine-readable medium having instructions which when executed by at least one processor of a surgical system causes the surgical system to: receive, via a human interface device (HID), a user command that includes a target pose of an end effector ofAtorney Docket No. : AUR6376WOPCT1Electronically Filed a surgical tool of the surgical system, where the surgical tool includes a shaft coupled to the end effector, disposed between the shaft and the end effector includes several joints; determine a range of motion of the end effector along a control point on the surgical tool; determine whether the target pose of the end effector with respect to the control point is to extend beyond the range of motion of the end effector; and responsive to determining that the target pose of the end effector is to extend beyond the range of motion, adjust the target pose of the end effector to stay within the range of motion.

[0010] In one embodiment, the range of motion includes a coned boundary having a vertex at the control point about a vector that extends to a fixed point on the shaft of the surgical tool. In another embodiment, the fixed point includes a remote center of motion (RCM) of the surgical tool. In some embodiments, the end effector includes a grasper. In another embodiment, the joints includes a first joint having a first joint coupled between the shaft and a link and a second joint coupled between the link and the end effector, where the range of motion is determined as a minimum range of motion between: a first range of motion of the end effector when the first joint is at a first joint limit and the second joint is at a first midpoint joint angle within a first joint range; and a second range of motion of the end effector when the second joint is at a second joint limit and the first joint is at a second midpoint joint angle within a second joint range.

[0011] In one embodiment, responsive to determining that the target pose of the end effector is to stay within the range of motion, providing joint commands to the joints to move the end effector from a current pose to the target pose. In another embodiment, the user command indicates that the end effector is to perform a roll motion about an axis that extends longitudinally through the end effector to achieve the target pose, where the roll motion is to cause the joints to rotate about each of their respective axes while the control point is to remain in place. In some embodiments, each of the joints of the plurality of joints includes a different joint limit. In another embodiment, the range of motion includes a rotational limit around a rotational axis of the control point, where the instructions to adjust the target pose includes instructions to rotate the target pose with respect to the rotational axis such that an end effector angle is within the rotational limit.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0012] According to another embodiment of the disclosure, a system, an apparatus, or an electronic device as shown and as described herein. According to another embodiment of the disclosure, a method substantially as herein described. According to another embodiment of the disclosure, includes a processor configured to perform one or more operations as described herein. According to another embodiment of the disclosure, includes a non-transitory machine-readable medium that includes instructions which when executed by at least one processor causes a system to perform one or more operations as described herein.

[0013] The above summary does not include an exhaustive list of all embodiments of the disclosure. It is contemplated that the disclosure includes all systems and methods that can be practiced from all suitable combinations of the various embodiments summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the claims. Such combinations may have particular advantages not specifically recited in the above summary.Atorney Docket No. : AUR6376WOPCT1Electronically FiledBRIEF DESCRIPTION OF THE DRAWINGS

[0014] The embodiments are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to "an" or “one” embodiment of this disclosure are not necessarily to the same embodiment, and they mean at least one. Also, in the interest of conciseness and reducing the total number of figures, a given figure may be used to illustrate the features of more than one embodiment, and not all elements in the figure may be required for a given embodiment.

[0015] Fig. 1 shows a pictorial view of an example surgical system in an operating arena.

[0016] Fig. 2 shows an example of a user-side human interface device (HID) of the user console with which an operator may use to control a robotic component of the surgical system according to one embodiment of the disclosure.

[0017] Fig. 3 shows a patient-side robotic arm that includes a surgical tool of the surgical system according to one embodiment of the disclosure.

[0018] Figs. 4A and 4B illustrate examples of different joint configurations of the surgical tool due to a pure roll motion according to one embodiment.

[0019] Fig. 5 is a block diagram of the surgical system for determining the cone limit of achievable range of motion of the surgical tool and controlling movement of the surgical tool within the range according to one embodiment.

[0020] Fig. 6 is a flowchart of one embodiment of a process for determining the cone limit of the surgical tool.

[0021] Figs. 7A and 7B illustrate examples of configurations of the surgical tool for computing the cone angle of the cone limit according to one embodiment.

[0022] Fig. 8 is a flowchart of another embodiment of a process for determining the cone limit of the surgical tool.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0023] Fig. 9 illustrates an example of adjusting the pose of the surgical tool so that the end effector is within the cone limit according to one embodiment.Atorney Docket No. : AUR6376WOPCT1Electronically FiledDETAILED DESCRIPTION

[0024] Several embodiments of the disclosure with reference to the appended drawings are now explained. Whenever the shapes, relative positions and other embodiments of the parts described in a given embodiment are not explicitly defined, the scope of the disclosure here is not limited only to the parts shown, which are meant merely for the purpose of illustration. Also, while numerous details are set forth, it is understood that some embodiments may be practiced without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description. Furthermore, unless the meaning is clearly to the contrary, all ranges set forth herein are deemed to be inclusive of each range’s endpoints.

[0025] Fig. 1 shows a pictorial view of an example (e.g., laparoscopic) surgical robotic system (which hereafter may be referred to as “surgical system” or “system”) 100 in an operating arena. The system 100 includes a user console 120, a control tower 131, and one or more surgical robotic arms 110 at a surgical robotic table (surgical table or surgical platform) 151. In one embodiment, the arms 110 may be mounted to a table or bed on which the patient rests as shown in the example of Fig. 1. In one embodiment, at least some of the arms 110 may be configured differently. For example, at least some of the arms may be mounted on a ceiling, sidewall, or in another suitable structural support, such as a cart separate from the table. The system 100 can incorporate any number of devices, tools, or accessories used to perform surgery on a patient 161. For example, the system 100 may include one or more surgical tools (instruments) 171 used to perform surgery (surgical procedure). A surgical tool 171 may be an end effector that is attached to a distal end of a surgical arm 110, for executing a surgical procedure.

[0026] Each surgical tool 171 may be manipulated manually, robotically, or both, during the surgery. For example, the surgical tool 171 may be a tool used to enter, view, or manipulate an internal anatomy of the patient 161. In an embodiment, the surgical tool 171 may include a grasper that can grasp tissue of the patient. In another embodiment, the surgical tool may include one or more cameras (e.g., an endoscopic camera), which may be configured to capture images of a surgical site inAtorney Docket No. : AUR6376WOPCT1Electronically Filed which one or more other surgical tools 171 may be used to perform one or more surgical tasks. For instance, the camera may be arranged to have a field of view that includes a surgical site with one or more other surgical tools, which may be manipulated by an operator. The surgical tool 171 may be controlled manually by a bedside operator 180; or it may be controlled robotically, via actuated movement of the surgical robotic arm 110 to which it is attached. For example, when manually controlled an operator may (e.g., physically) hold a portion of the tool (e.g., a handle), and may manually control the tool by moving the handle and / or pressing one or more input controls (e.g., buttons) on the (e.g., handle of the) tool. In another embodiment, when controlled robotically, the surgical system may manipulate the surgical tool-based user input (e.g., received via the user console 120, as described herein).

[0027] Generally, a remote operator 190, such as a surgeon or other operator, may use the user console 120 to remotely manipulate the arms 110 and / or the attached surgical tools 171, e.g., during a teleoperation. The user console 120 may be located in the same operating room as the rest of the system 100, as shown in Fig. 1. In other environments however, the user console 120 may be located in an adjacent or nearby room, or it may be at a remote location, e.g., in a different building, city, or country. The user console 120 may include one or more components, such as a seat 119, one or more foot-operated controls (or foot pedals) 130, one or more human interface devices (HIDs) 140, and at least one display 150. In one embodiment, the user console may include less components. For example, the seat 119 may be separate from the user console. This may allow the user to sit on the seat and to move into a position that allows the user to use the console. In particular, the seat may be on casters that allow the seat to move on the floor.

[0028] The display 150 is configured to display, for example, a view of the surgical site inside the patient 161. The display may be configured to display image data (e.g., still images and / or video) that may be captured by a camera that may be used during a surgical procedure, as described herein. In one embodiment, the display may be any type of display, such as a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic LED (OLED) display, etc. In some embodiments, the display may be a three-dimensional (3D) immersive display that is for displaying 3D (surgical) presentations. For instance, during a surgical procedure one or moreAtorney Docket No. : AUR6376WOPCT1Electronically Filed endoscopic cameras may be capturing image data of a surgical site, which the display presents to the user in 3D. The display may include a viewer that includes one or two screens that may be configured to display stereoscopic images. In this case, the viewer of the display may include a contoured portion that may be arranged to come into contact with and contour to at least a portion of the operator’s face when the remote operator 190 moves towards and / or comes into contact with the display. Coming into contact and contouring with the operator’s face may prevent ambient light from interfering with the displayed surgical presentation, while the screen(s) of the display 150 may give the remote operator 190 a perception of a 3D space captured by one or more cameras. In one embodiment, the 3D display may be an autostereoscopic display that provides 3D perception to the user without the need for special glasses. As another example, the 3D display may be a stereoscopic display that provides 3D perception with the use of glasses (e.g., via active shutter or polarized).

[0029] In another embodiment, the display 150 may be configured to display at least one graphical user interface (GUI) that may provide informative and / or interactive content, to thereby assist a user in performing a surgical procedure with one or more instruments in the surgical system 100. For example, some of the content displayed may include image data captured by one or more endoscopic cameras, as described herein. In another embodiment, the GUI may include selectable UI items, which when manipulated by the user may cause the system to perform one or more operations. For instance, the GUI may include a UI item as interactive content to switch control between robotic arms. In one embodiment, to interact with the GUI, the system may include input devices, such as a keyboard, a mouse, etc. In another embodiment, the user may interact with the GUI using the HID 140. For instance, the user may manipulate the HID to navigate through the GUI, (e.g., with a cursor), and to make a selection may hover the cursor over a UI item and manipulate the HID (e.g., selecting a control or button). In some embodiments, the display may be a touch-sensitive display screen. In this case, the user may perform a selection by navigating and selecting through touching the display. In some embodiments, any method may be used to navigate and / or select a UI item.

[0030] As shown, the remote operator 190 is sitting in the seat 119 and viewing the user display 150 while manipulating a foot-operated control 130 and a handheldAtorney Docket No. : AUR6376WOPCT1Electronically FiledHID 140 in order to remotely control one or more of the arms 110 and the surgical tools 171 (that are mounted on the distal ends of the arms 110.)

[0031] In some variations, the bedside operator 180 may also operate the system 100 in an “over the bed” mode, in which the bedside operator 180 (user) is now at a side of the patient 161 and is simultaneously manipulating a robotically-driven tool (end effector as attached to the arm 110), e.g., with a handheld HID 140 held in one hand, and a manual laparoscopic tool. For example, the bedside operator’s left hand may be manipulating the handheld HID to control a robotic component, while the bedside operator’s right hand may be manipulating a manual laparoscopic tool. Thus, in these variations, the bedside operator 180 may perform both robotic-assisted minimally invasive surgery and manual laparoscopic surgery on the patient 161.

[0032] During an example procedure (surgery), the patient 161 is prepped and draped in a sterile fashion to achieve anesthesia. Initial access to the surgical site may be performed manually while the arms of the system 100 are in a stowed configuration or withdrawn configuration (to facilitate access to the surgical site.) Once access is completed, initial positioning or preparation of the system 100 including its arms 110 may be performed. Next, the surgery proceeds with the remote operator 190 at the user console 120 utilizing the foot-operated controls 130 and the HIDs 140 to manipulate the various end effectors and perhaps an imaging system, to perform the surgery. Manual assistance may also be provided at the procedure bed or table, by sterile- gowned bedside personnel, e.g., the bedside operator 180 who may perform tasks such as retracting tissues, performing manual repositioning, and tool exchange upon one or more of the robotic arms 110. Non-sterile personnel may also be present to assist the remote operator 190 at the user console 120. When the procedure or surgery is completed, the system 100 and the user console 120 may be configured or set in a state to facilitate post-operative procedures such as cleaning or sterilization and healthcare record entry or printout via the user console 120.

[0033] In one embodiment, the remote operator 190 holds and moves the HID 140 to provide an input command to drive (move) one or more robotic arm actuators 170 (or driving mechanism) in the system 100 for teleoperation. The HID 140 may be communicatively coupled to the rest of the system 100, e.g., via a console computerAtorney Docket No. : AUR6376WOPCT1Electronically Filed system 160 (or host). The HID 140 can generate spatial state signals corresponding to movement of the HID 140, e.g., position and orientation of the handheld housing of the HID, and the spatial state signals may be input signals to control motions of the robotic arm actuators 170. The system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuators 170. In one embodiment, a console processor of the console computer system 160 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuators 170 are energized to drive a segment or link of the arm 110, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the HID 140. Similarly, interaction between the remote operator 190 and the HID 140 can generate, for example, a grip control signal that causes a jaw of a grasper of the surgical tool 171 to close and grip the tissue of patient 161.

[0034] The system 100 may include one or more user-side (or surgeon-side) HIDs 140, where respective control signals are generated for each HID that control the actuators and the surgical tool (end effector) of a respective arm 110. For example, the user console 120 may include two HIDs 140, a first (or left) HID arranged to be held and controlled by the operator’s left hand and a second (or right) HID arranged to be held and controlled by the operator’s right hand. In which case, the remote operator 190 may move the left HID 140 to control the motion of an actuator 170 that is in a one robotic arm, where the actuator responds by moving linkages, gears, etc., in that arm 110. Similarly, movement of the right HID 140 by the remote operator 190 controls the motion of another actuator 170, which in turn drives other linkages, gears, etc., of (e.g., another robotic component, such as a robotic arm of) the system 100. The system 100 may include a right arm 110 that is secured to the bed or table to the right side of the patient, and a left arm 110 that is at the left side of the patient. An actuator 170 may include one or more motors that are controlled so that they drive the rotation of a joint of the arm 110, to for example change, relative to the patient, an orientation of an endoscope or a grasper of the surgical tool 171 that is attached to that arm. Motion of several actuators 170 in the same arm 110 can be controlled by the spatial state signals generated from a particular HID 140. The HIDs 140 can also control motion of respective surgical tool graspers. For example, each HID 140 can generate a respectiveAtorney Docket No. : AUR6376WOPCT1Electronically Filed grip signal to control motion of an actuator, e.g., a linear actuator that opens or closes jaws of the grasper at a distal end of surgical tool 171 to grip tissue within patient 161.

[0035] In some embodiments, the communication between the surgical robotic table 151 and the user console 120 may be through a control tower 131, which may translate user commands that are received from the user console 120 (and more particularly from the console computer system 160) into robotic control commands that transmitted to the arms 110 on the surgical table 151. The control tower 131 may also transmit status and feedback from the surgical table 151 back to the user console 120. The communication connections between the surgical table 151, the user console 120, and the control tower 131 may be via wired (e.g., optical fiber) and / or wireless links, using any suitable one of a variety of wireless data communication protocols, such as BLUETOOTH protocol. Any wired connections may be optionally built into the floor and / or walls or ceiling of the operating room. The system 100 may provide video output to one or more displays, including displays within the operating room as well as remote displays that are accessible via the Internet or other networks. The video output or feed may also be encrypted to ensure privacy and all or portions of the video output may be saved to a server or electronic healthcare record system.

[0036] As described herein, this shows an example of the operator 190 using the user console 120 to control one or more robotic components, such as a robotic arm 110 and / or a surgical tool 171 coupled to a distal end of the robotic arm, of the surgical system 100. Specifically, this figure shows the operator 190 seated on the seat 119 in front of the user console 120, while controlling (or moving) one or more robotic components by manipulating control inputs, such as one or both of the HIDs 140 and / or the foot pedal(s) 130.

[0037] The user console 120 includes a base 201 to which the foot pedals 130 may be attached and a support structure 202 that may be mounted on the base 201 and extends vertically upward from the base. Coupled to the support structure 202 includes the display 150 and the HIDs 140. As shown, the HIDs may be coupled (mounted) on a bottom side of a structure (e.g., an arm rest) that is coupled to the support structure 202. In which case, the HID may be a wired or “grounded” input device that may be connected to the user console via a wired connection (e.g., to exchange data), which isAtorney Docket No. : AUR6376WOPCT1Electronically Filed in contrast to an “ungrounded’ input device that may be wirelessly connected to the user console (e.g., exchanging data via a wireless connection). As described herein, the HID 140 may include one or more joints that couple one or more links, which allow the HID to have one or more rotational degrees of freedom. In one embodiment, the HID may be grounded such that the joints of the HID may be arranged to hold their position and / or orientation absent to an external applied force upon the HID. More about the configuration of the HID is described herein.

[0038] In another embodiment, the HIDs may be coupled to the user console in other configurations. For example, the HIDs may be separate from the user console, but may be communicatively coupled (e.g., through wired-connection). In which case, the HIDs may be mounted on another device (e.g., a surgical cart), which may be coupled to the user console.

[0039] As described herein, the operator 190 may be using the user console to control a robotic arm during a teleoperation. As shown, the user may be viewing a surgical workspace through the display 150, while operating one or more components of the surgical system by manipulating one or both of the HIDs 140 and / or the foot pedals 130.

[0040] Fig. 2 shows an example of the HID 140 of the user console 120 with which the operator 190 may control a robotic arm of the surgical system 100 during a surgical procedure. In particular, this figure shows the HID 140 that includes a support arm 200 and a gimbal 205, which may include one or more links coupled together by one or more joints. As described herein, the HID 140 may be a low -inertia such that as the user manipulates the HID, one or more joints may move (or be adjusted) such that the HID may be moved between positions (or poses) within a workspace surrounding the user console 120. The support arm 200 may be configured to couple the gimbal 205 to the user console 120, where the gimbal may be arranged to rotate about one or more axes and / or the support arm may be arranged to rotate about one or more axes to allow the user to move an end user control 220 of the gimbal within three-dimensional (3D) space. Movement of the end user control may allow the user to control (e.g., movement of) a robotic component, such as a robotic arm, of the surgical system. In oneAtorney Docket No. : AUR6376WOPCT1Electronically Filed embodiment, each of the joints may be capable of rotating about and / or translating along one or more axes.

[0041] As shown, the HID 140 includes seven joints 215a-215g, which may provide the HID 140 with seven (rotational) degrees of freedom (DoF). In particular, the support arm 200 includes three links 210a-210c and three joints 215a-215c. In particular, link 210a may be coupled to link 210b, via joint 215b, and link 210b may be coupled to link 210c, via joint 215c. In one embodiment, the HID 140 may be arranged to couple to the support structure 202 at the joint 215a. The gimbal 205 may be coupled to the support arm 200 via joint 215d. The gimbal includes three links 210d-210f and four joints 215d-215g. Link 210d may be coupled to joint 215d and joint 215e. Link 210d may be coupled to link 210e, via joint 215e, and link 210e may be coupled to link 210f, via joint 215f The gimbal 205 may also include an end user control 220 that may be coupled to link 21 Of via joint 215g.

[0042] As described herein, the HID may include seven DoF. In one embodiment, one or more DoF may be redundant with respect to other DoF. For example joint 215d may be redundant with respect to joint 215f, since both joints allow the end user control 220 to rotate about a same axis (e.g., a Z-axis). In which case, the system 100 may be configured to provide null space motion in which one or more joints of the HID may be moved, while a pose of an end effector (e.g., the end user control 220) may remain unaffected. As described herein, this redundancy may allow the system to perform one or more secondary tasks by manipulating one or more joints, without affecting the end user control 220. For instance, as described herein, the gimbal 205 may include four joints 215d, 215e, 215f, and 215g, where the last three may provide a 3D range of motion, and the first, joint 215d, may be redundant by having a redundant range of motion as a range of motion of joint 215f, whereby motion of joint 215d may occur without affecting (or causing motion) to the other three joints. In one embodiment, a secondary task may include an adjustment of one or more joints, which may include redundant joint 215d in order to provide the user with a more ergonomic position of the end user control 220, without affecting the pose of the end user control.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0043] In one embodiment, the HID 140 may include more or fewer links and / or joints, which may provide more or fewer DoF for the HID. For example joint 215a may be arranged to rotate the entire HID about an axis (e.g., the Z-axis) with respect to the link 210a. Conversely, joint 215b may be arranged to rotate link 210b (and the links / joints to which the link 210b may be coupled) about an orthogonal axis (e.g., Y-axis) to the axis that runs through link 210a and joint 215b.

[0044] In one embodiment, at least some of the joints may include one or more motors (or actuators) that may allow the joint to rotate around and / or translate along one or more axes. In this way, the HID may provide a user with assisted movement in response to user input, such as a user applying an external force upon the end user control 220. In another embodiment, the HID 140 may be arranged to provide haptic feedback to the user. As described herein, the HID may be configured to control movement of a robotic component, such as a surgical tool coupled to a distal end of a robotic arm. As a result of this movement, external forces may be applied onto the surgical tool when the tool presses up against an object, such as pressing up against an abdominal wall of a patient during a surgical procedure. This force applied by the object may be haptically applied by the HID such that the operator may perceive the force. In one embodiment, the system 100 may be configured to determine whether an external force is being applied to the robotic arm, such as through one or more sensors (e.g., force sensors). In another embodiment, the system may determine that the robotic arm is pressing onto an object, based on a comparison between control commands from the HID and encoder data from one or more encoders of the robotic arm. Upon determining this external force, the system may be configured to provide haptic feedback to the user by controlling one or more motors of the HID to relay an applied force through the HID onto the user. In one embodiment, this applied force may be an opposite (and / or proportional) external force applied onto the robotic arm. More about providing haptic feedback force is described herein.

[0045] Fig. 3 shows a patient-side robotic arm 110 that includes the surgical tool 171 of the surgical system 100 according to one embodiment of the disclosure. The surgical arm includes ten links 303a-303j that are coupled together through nine joints 302a-302i, where each joint may be arranged to move a correspondingly coupled link by at least one of rotating the link about at least one axis and / or translating the linkAtorney Docket No. : AUR6376WOPCT1Electronically Filed along at least one axis. For instance, each joint may include an actuator, which may move its corresponding joint based on user input (e.g., user commands) received through movement of the HID 140. In one embodiment, the robotic arm may include more or less joints and / or links.

[0046] The robotic arm may be coupled to a structure (such as the surgical table 151 or a surgical cart) at the most proximal joint 302a. Coupled to the most distal link 303j includes a tool drive 305 configured to control a surgical tool 171. In another embodiment, any type of end effector may be coupled to the tool drive, such as an endoscopic camera. Coupled to the tool drive 305 is a cannula 304 that may be inserted into a patient’s cavity (e.g., abdominal region), where the (e.g., end effector of the) surgical tool may be received through the cannula.

[0047] As described herein, the system 100 may be configured to control movement of the robotic arm based on user input through the HID 140. In particular, the system may be configured to cause one or more joints of the robotic arm to move in order to match motion of the HID. To match motion, the system may attempt to align (or match) a pose of the (e.g., end user control 220 of the) HID 140 with that of the surgical tool 171 of the robotic arm with respect to at least one reference frame. For instance, the system 100 may be configured to receive user input through the HID, where the user input may indicate one or more spatial state signals corresponding to movement of the HID, as described herein. From this input, the system may determine a pose of the HID, or more specifically a pose of the end user control of the HID, and may be configured to generate one or more control (or joint) commands to cause one or more joints of the robotic arm to actuate in order for a pose of the surgical tool coupled to the robotic arm to match the pose of the end user control, as perceived by the user through the display 150 of the user console 120. The system may perform one or more conversions of the HID pose with respect to one or more reference frames into the pose of the tool 171 with respect to the reference frames. One reference frame may be with respect to the display. As described herein, the user may control the surgical instrument by manipulating the HID, while viewing the surgical instrument through the display 150 that may receive video images from one or more cameras. In which case, the system may define tool motion with respect to the camera to follow the HID motion with respect to the display. Thus, the system may convert a user command (e.g., HIDAtorney Docket No. : AUR6376WOPCT1Electronically Filed pose based on position data from sensors, such as encoders of the HID) that may with respect to the display 150 to a target (or desired) tool pose with respect to a camera of the system 100. As a result, the system may ensure that the pose of the HID matches the pose of the surgical tool, where both may share a global reference frame. In one embodiment, the system may be configured to adjust a pose of the surgical tool according to movement (or changing poses of the HID) in real-time, whereby motion of the tool may track motion of the HID in real-time and during a surgical procedure.

[0048] Figs. 4A and 4B illustrate examples of different joint configurations of the surgical tool due to a pure roll motion according to one embodiment. Each of these figures shows a surgical tool 171 that may be arranged to be received through the cannula 304 and coupled to the tool drive 305 of the robotic arm 110. The tool 171 may be a needle driver tool, which may include a grasper (e.g., having two jointed jaws) as the end effector 430. Such a tool may be used during a surgical procedure to perform a surgical task, such as suturing a portion of tissue. In another embodiment, the tool 171 may include any type of surgical instrument, such as a surgical vessel sealer arranged to cauterize (or seal) tissue.

[0049] The surgical tool 171 includes a tool shaft 410 coupled to the end effector 430 via one or more joints. In particular, the surgical tool 171 includes a yaw joint 425 coupled to the end effector 430 and a pitch link 420, and a pitch joint 415 coupled to and disposed between the pitch link 420 and the tool shaft 410. In one embodiment, the yaw and pitch joints may be joints that are arranged to rotate about orthogonal axes with respect to each other. For instance, in Fig. 4A, the pitch joint 415 may be arranged to rotate about an X-axis, while the yaw joint 425 may be arranged to rotate about the Z-axis. In another embodiment, the joints may not be orthogonal to each other. In another embodiment, the surgical tool 171 may include additional joints (not shown), such as joint (e.g., rotary joint) that may allow the surgical tool 171 to rotate (e.g., about a roll axis). In this case, the roll axis may be along the shaft axis that may extend longitudinally along the tool shaft 410, in the Y-direction. As a result of the joints, the surgical tool may be able to adjust the 6 DoF pose of the end effector 430 due to motion of its one or more joints in space.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0050] The tool shaft 410 includes a remote center of motion (RCM) 405 that refers to a point in space about which the surgical tool 171 may pivot about one or more axes due to or in response to user commands. In one embodiment, the RCM may be a fixed point on the tool shaft, which may be predefined. In another embodiment, the RCM may be defined as a pivot point located at an incision port of a patient. For instance, a trocar may be inserted into an incision port of the patient to create an access point for the surgical tool. Once the robotic arm 110 is docked to the trocar, the surgical tool (and cannula) may be inserted into the trocar, where the position of the RCM may be aligned and remain constant during at least a portion of the surgical procedure. In one embodiment, the position of the RCM along the shaft 410 may differ between surgical tools.

[0051] Some surgical tools include one or more joints that provide an end effector of the tool several DoF, as shown in Figs. 4A and 4B. This may enable a surgeon to bend or pivot the end effector into a desired (or target) pose (e.g., the end effector’s position and / or orientation with respect to one or more reference frames) in order to perform a surgical task. An end effector’s overall reachable workspace and position / orientation capabilities within 3D space may depend upon the range of motions (or joint limits) of the joints that articulate the movement of the surgical tool. Some of these joints may have different joint limits, which may be due to varying physical constraints or due to design specifications. The different joint limits may affect an end effector’s range of motion. For instance, as joints are rotated in space, the end effector’s position and / or orientation capabilities may change about a point in space due to varying rotational axes of the joints change in space. As an example, Figs. 4A and 4B illustrate a pure roll motion by the surgical tool. For instance, Fig. 4A shows the pitch joint 415 having a range of motion in the X-direction and the yaw joint 425 having a range of motion in the Z-direction. Fig. 4B shows the result of a pure roll motion by the tool shaft 410, which changes the rotational axis of the pitch joint 415 to in the Z-direction and changes the rotational axis of the yaw joint 425 to be (approximately) in the Y-direction. In some cases, the pitch joint 415 may have a lesser range of motion than the yaw joint 425. Due to some joints having different joint limits, there may be some desired commands from the input device that are not possible to actualize by the tool 171. An example of this may be a pure roll motion (or CartesianAtorney Docket No. : AUR6376WOPCT1Electronically Filed roll motion), where if the last two joints of the tool have different ranges of motion, the tool would eventually get stuck if the pure roll motion is started at a configuration where the joint with the larger range of motion is at its joint limit due to the end effector’s changing range of motion as the tool rolls. Therefore, there is a need for defining a range of motion for the end effector that ensures smooth motion during various pose changes, such as during a roll motion.

[0052] To solve this problem, the present disclosure provides a method and system for a teleoperated surgical system that maintains a surgical tool’s pose within a cone limit to ensure smooth tool motion. The system receives, via the HID, a user command that includes a target pose of the end effector 430 of the surgical tool 171, where the surgical tool includes the shaft 410 coupled to the end effector, disposed between both are several joints. The system 100 determines a range of motion of the end effector along a control point of the surgical tool. For instance, this range of motion may include a cone limit (or boundary) having a vertex at the control point about a vector that extends to a fixed point on the shaft of the surgical tool. The system may determine, responsive to the target pose of the end effector extending beyond the range of motion, adjusting the target pose of the end effector to stay within the range of motion. As a result, for any motion, such as a pure roll motion of the surgical tool 171, the control point position of the cone may not move the center of the cone, thereby keeping it aligned with the control point. Moreover, the cone may be defined such that joint angles of joints of the tool 171 may remain within their joint limits throughout the motion of the end effector 430. Therefore, this range of motion may effectively define a cone boundary or cone limit of the end effector with respect to a control point (e.g., a center of a last (or most distal) wrist joint of a series of joints before the end effector), where the control point may be aligned with a fixed point on a shaft of the surgical tool along a vector that passes through a center of the cone. This may allow the end effector to perform smooth motions, such as pure roll motions without exceeding its cone limit, thereby preventing the tool from getting stuck or causing jerking motions.

[0053] Fig. 5 is a block diagram of the surgical system 100 for determining the cone limit of achievable range of motion of the surgical tool and maintaining the pose of the surgical tool within the cone limit according to one embodiment. The system includes the HID 140, the robotic arm 110, and a controller 500. In one embodiment,Atorney Docket No. : AUR6376WOPCT1Electronically Filed the system may include more or less components, such as having two or more robotic arms, each with one or more tools 171 that may be arranged to be manipulated by one or more HIDs.

[0054] The robotic arm 110 includes one or more sensors 505, one or more actuators 170, and one or more surgical tools 171. As described herein, the actuators may be a part of and / or cause one or more joints of the arm to move based on joint commands. This movement may cause corresponding joints to rotate and / or translate about one or more axes. The sensors may be configured to produce position data of the robotic arm 110. For instance, the sensors may include encoders, each of which may be configured to measure a joint position (e.g., translational and / or rotational values) as encoder (or position) data of a respective joint. In which case, the position data from the sensors 505 may indicate the actual position of a respective joint in space (with respect to a reference point or frame). As described herein, this position data may be used to determine an actual tool pose of the tool 171 coupled to the robotic arm 110.

[0055] The HID 140 includes one or more sensors 510 and one or more actuators 515. The sensors 510 and / or the actuators 515 may perform similar operations as the sensors 505 and / or the actuators 170, respectively, of the robotic arm 110. For instance, the sensors 510 may produce joint position data, which may indicate joint positions of one or more joints of the HID 140, which may be used to determine the pose of the HID (e.g., the pose of the end user control 220 of the HID 140).

[0056] In one embodiment, the controller 500 may be a special-purpose processor such as an application-specific integrated circuit (ASIC), a field- programmable gate array (FPGA), a general-purpose microprocessor, a digital signal controller, or a set of hardware logic structures (e.g., arithmetic logic units, filters, and dedicated state machines). In one embodiment, the controller 500 may be a part an electronic device, such as the console computer system 160, the control tower 131, and / or the user console 120. Although illustrated as being a single component, the controller may include one or more electronic components (e.g., processors, memory, etc.) that may be communicatively coupled on a single electronic device (such as the console computer system 160), or across multiple devices (e.g., communicating over a wireless computer network). In some embodiments, the controller 500 may be a part ofAtorney Docket No. : AUR6376WOPCT1Electronically Filed a separate device, such as a part of a remote server that may be in communication with one or more electronic devices of the surgical system 100. In which case, the remote sever may be configured to communicate between the user console for receiving user commands and the control tower 131 for transmitting control signals (e.g., joint commands) for moving the arms 110.

[0057] The controller 500 may be configured to determine a cone limit of an achievable range of motion of the surgical tool 171 and maintain the pose of the surgical tool within the determined cone limit. The controller 500 includes a HID / Tool pose estimator 550, a range (or cone) limiter 560, and an inverse kinematics (IK) 470. The controller 500 may be configured to receive a user command form the HID 140, responsive to user input, such as the user moving and / or manipulating the end user control 220 of the HID 140. The user command may include position data (e.g., spatial state signals) from the HID 140, responsive to the user input. The position data may be produced by the sensors 510 of the HID, such as encoders that may be arranged to measure joint positions. The HID / Tool pose estimator 550 may be configured to estimate a pose of the HID based on the user command from the HID. The HID pose may be a six degrees of freedom (6 DoF) pose of the end user control 220 (e.g., in three-dimensional (3D) Cartesian space) that is being held by the user while moving the HID. The estimator 550 may be configured to convert the user command into the HID pose. For instance, when the user command includes position data of one or more joints of the HID, the estimator 550 may use a forward kinematics algorithm to convert the data into the HID pose.

[0058] The HID / tool pose estimator 550 may be configured to determine a target pose of the (e.g., surgical tool 171, which may be coupled to the) robotic arm 110, which may be referred to hereafter as “target tool pose”. The estimator 550 may be configured to convert the HID pose into the target tool pose. The conversion may allow for the pose of the tool and the HID pose to be the same with respect to one or more reference frames. As described herein, the user may control the surgical instrument by manipulating the HID, while viewing the surgical instrument through the display 150 that may receive video images from one or more cameras. In which case, the system may convert the HID pose into the target tool pose such that tool motion with respect to the camera follows the HID motion with respect to the display. Thus, the system mayAtorney Docket No. : AUR6376WOPCT1Electronically Filed convert a user command (or HID pose based on position data from the HID) that may be with respect to the display 150 to a target tool pose with respect to a camera of the system 100. The HID / tool pose estimator 550 may be configured to determine the actual (current) tool pose of the (tool 171 of the) robotic arm 110 based on sensor data from the sensors 505. For instance, the estimator 550 may perform a forward kinematics process to convert the sensor data into a 6 DoF tool pose of the tool 171.

[0059] In one embodiment, the operations of the estimator 550 (and / or the limiter 560 or the IK 570) may be performed on a different device. For instance, when the controller 500 is a part of an electronic device within the surgical operating room, such as the control tower 131, the desired tool pose may be received (as a user command) from a separate electronic device, such as the console computer system 160. In which case, the console computer system 160 may perform at least some of the operations of the estimator 550 to produce a target tool pose responsive to user input through the HID, and may provide the target tool pose as a user command (e.g., through a computer network) to the controller 500.

[0060] The range limiter 560 may be configured to receive the target tool pose from the HID / Tool pose estimator 550 and may be configured to produce a limited target tool pose by adjusting the target tool pose based on whether the pose exceeds beyond a range of motion (or cone limit) of the tool 171. For instance, upon determining that the target tool pose exceeds the cone limit, the range limiter 560 may adjust the pose by adjusting its orientation within the 3D space. More about the range limiter 560 adjusting the pose of the surgical tool is described herein.

[0061] The range limiter 560 may be configured to determine the range of motion of the tool. As described herein, the cone limit may be at a control point (e.g., on the yaw joint 425) on the surgical tool 171, where the cone limit may be aligned along a vector that passes through the control point and a fixed point on the surgical tool, such as the RCM 405 of the surgical tool. This position of the RCM may differ between different surgical tools. As a result, to determine the cone limit, the range limiter may be configured to determine the position of the RCM on the surgical tool. In which case, the range limiter may be configured to determine a tool type of the surgical tool and may be configured to determine the cone limit based on the tool type. ForAtorney Docket No. : AUR6376WOPCT1Electronically Filed example, the (e.g., memory of the) system 100 may include a data structure that associates one or more parameters of surgical tools with a position of its RCM (in 3D space), where the parameters may include a tool type, a model number of the tool, a serial number of the tool, etc. In which case, to determine the range of motion, the range limiter may be configured to perform a table lookup into the data structure to select a position of the RCM associated with one or more parameters of associated with the surgical tool. For example, the tool type may include a needle driver that has a pair of jaws as the end effector 430. Upon determining the RCM, the range limiter may be configured to determine the range of motion of the tool. More about this determination is described herein. In another embodiment, the RCM position may be determined by user input (e.g., through the HID 140).

[0062] In one embodiment, the range of motion of the tool may be determined by the range limiter 560 performing a calibration process, which may be performed when the surgical tool 171 is installed (coupled to) the robotic arm 110. In another embodiment, the calibration process may be performed each time the system is activated (e.g., powered up). In another embodiment, the calibration process may be performed intermittently or periodically during system operation. The range of motion may be stored in memory of the system, as described herein. In another embodiment, the system 100 may receive the range of motion from a remote electronic device (e.g., a server). The range of motions may be predefined. In which case, the range limiter may retrieve the data structure that includes the range of motions, as described herein, and may determine the range of motion for the surgical tool 171 by performing the table lookup.

[0063] The IK 570 may be configured to perform an iterative process for generating joint motion for one or more joints (e.g., joints 415 and / or 425) to cause the (e.g., end effector 430 of the) tool 171 to move into the target tool pose. For instance, the IK may receive the limited target tool pose from the range limiter 560 and may be configured to generate joint commands for one or more joints of the robotic arm 110 to move the end effector of the tool 171 from its current position to the limited target tool pose.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0064] Figs. 6 and 8 are flowcharts of processes 600 and 800, respectively, for performing cone limit determination and maintaining the target tool pose within the cone limit. At least some of the operations of these processes may be performed by the (e.g., estimator 550, limiter 560, and / or IK 570 of the) controller 500 of the system 100. In another embodiment, at least some of the operations may be performed by another electronic device, which may communicate with the controller 500. Some of these processes may be performed in “real-time”, during a surgical procedure in which an operator is controlling a robotic arm to perform one or more surgical tasks.

[0065] Fig. 6 is a flowchart of one embodiment of a process 600 for determining the cone limit of the surgical tool and maintaining the tool pose within the cone limit. The process 600 begins with the system receiving, via the HID 140, a user command that includes a target pose of the end effector 430 of the surgical tool 171 that includes a shaft 410 coupled to the end effector, disposed between both are several joints (at block 605). For instance, the surgical tool may include a needle driver with a grasper as the end effector and several joints for allowing an operator to articulate the end effector to perform surgical tasks. As described herein, at least one joint of the several joints may have a different joint limit with respect to another joint. In one embodiment, the user command may indicate that the end effector is to perform a (e.g., pure) roll motion. For instance, this roll motion may be about an axis that extends longitudinally through the end effector to achieve a target pose. The roll motion may cause several joints to rotate about their own respective axes and with respect to a global reference frame, while a control point is to remain in place. The system 100 determines a range of motion of the end effector along a control point of the surgical tool (at block 610). As described herein, this determination may be based on the tool type of the surgical tool 171 that is being manipulated by the operator. In which case, the range limiter 560 may be configured to determine the position of the RCM (with respect to a global reference frame), and may be configured to determine the cone limit based on the RCM position. To determine the cone limit, the range limiter 560 may be configured to determine a minimum cone angle between one or more joint configurations of the surgical tool, where in each configuration at least one joint is positioned at its joint limit, and the other joint is extended along an angular direction of the joint. For instance, this angular direction may be a midpoint, such as 0° along aAtorney Docket No. : AUR6376WOPCT1Electronically Filed joint’s range of motion. Figs. 7A and 7B illustrate examples of configurations of the surgical tool for computing the cone angle of the cone according to one embodiment. In one embodiment, the cone angle may be the angle between the cone’s central axis and its side. For each figure, the surgical tool 171 may be at a lowest insertion limit in which the surgical tool 171 is inserted into an incision port of a patient, where this insertion limit may be a minimum insertion or retraction limit of the surgical tool. In one embodiment, this lowest insertion limit may be a predefined minimum position (e.g., based on a predefined specification of the surgical tool) along the tool shaft 410 at which the surgical tool (and the cannula) may extend into a surgical site from an incision port.

[0066] Turning to Fig. 7A, the range limiter 560 may be configured to determine a first range of motion (or cone angle), zi, of the end effector 430, where the pitch joint 415 is at a joint limit (e.g., a maximum or minimum along the joint’s range of motion) and the yaw joint 425 is at a joint angle of 0° within its range of motion. In one embodiment, a joint angle of 0° may be a midpoint joint angle within the joint’s range (or range of motion). To determine this first cone angle, the Law of Cosines may be applied such that:

[0067] where drcm.cpi may be a vector (or distance) between the RCM 405 and a control point, cpi, while the surgical tool is in this configuration. This control point is shown as being on the end effector 430. The other distances include a distance between the pitch joint 415 and the yaw joint 425 (e.g., along the pitch link 420), dpy, the insertion limit, / / , which is at a minimum, and a distance between the yaw joint 425 and cpi, dycp. In one embodiment, It may be the distance from the RCM to the pitch joint 415. Also included is the joint limit, >P. of the pitch joint 415 that may be predefined. This equation may be simplified by:

[0068] This equation may be rearranged to:Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0069] From this equation pi may be determined by taking the inverse cosine. Turning to Fig. 7B, the range limiter 560 may determine a second range of motion, p2, of the end effector, where the yaw joint 425 is at a joint limit and the joint angle of the pitch joint 415 is at (approximately) 0° (e.g., a midpoint joint angle of the pitch joint’s range of motion). Again, to determine this second range of motion, the Law of Cosines may be applied such that:

[0070] where drcm,cP2 may be a distance between the RCM 405 and cp2, while the surgical tool 171 is in this configuration. In one embodiment, cp2 may be at a different position or a same position as cpi in Fig. 7A. The joint limit, ),. of the yaw joint 425 that may be predefined. Again, this equation may be rearranged to: cosCfe)

[0071] With both range of motions, the range limiter 560 may be configured to determine the actual cone angle, / , as the minimum range of motion between the first and second determined ranges of motion, such that p = minOi, ^)

[0072] Thus, the actual cone angle may define a cone at a control point on the surgical tool, where a vector from the fixed point on the surgical tool (e.g., the RCM 405) passes through the control point and the center of the cone, thereby creating a 3D range of motion around the control point. The vector being (and passing through) a central axis of the cone that is at the control point. By tying the cone to the vector from the RCM to the end effector, wherever the end effector moves, the cone may be defined with respect to the end effector and the vector. This may allow the system to determine whether an end effector exceeds the cone and adjust the end effector’s pose during use of the surgical system, as described herein.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0073] In one embodiment, the actual cone angle may be determined during a calibration process (e.g., in response to the system determining that the surgical tool has been installed to the robotic arm 110), or may be determined periodically (e.g., one or more times) during use of the surgical system 100, such as during a surgical procedure.

[0074] Thus, the actual cone angle may be determined as the minimum range of motion between both joint configurations, where the cone angle is determined based on the range of motion of the pitch joint 415 and the range of motion of the yaw joint 425, which are arranged to articulate around different axes with respect to a global reference frame and / or may have different ranges of motion. Since the actual cone angle is within joint ranges of the two joints, the range of motions of the surgical tool joints required to keep the end effector within the actual cone angle may be smaller (or equal) to their own joint limits. As a result, joint motion of the surgical tool’s joints will stay within their joint limits as the pose of the end effector is manipulated, such as during a pure roll motion, thereby ensuring that the surgical system does not reach a singularity.

[0075] The system 100 determines whether the target pose extends beyond the range of motion (at decision block 615). As described here, the target pose of the end effector may be a 6 DoF pose, which may include three translational components and three rotational components (roll, yaw, and pitch angles), as a rotational matrix. In which case, the range limiter 560 may determine whether at least one of the rotational angles exceeds the cone angle, with respect to the center point. More about this determination is described herein. If not, the system 100 may provide joint commands to one or more joints of the robotic arm based on the target pose (at block 620). In which case, responsive to determining that the target pose is to stay within the range of motion (e.g., an angle of the end effector is equal to or less than / ), the system may provide joint commands to move the end effector from a current pose to the target pose. If, however, the target pose does extend beyond / , the system 100 may limit the target pose of the end effector to stay within the range of motion (at block 625). For instance, the system 100 may adjust one or more rotational components of the end effector’s rotational matrix based on how much it exceeds / . The system 100 may provide joint commands to the joints of the robotic arm based on the limited target pose (at block 620).Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0076] Fig. 8 is a flowchart of another embodiment of a process 800 for determining the cone limit of the surgical tool and maintaining the surgical tool angle within the cone limit. In particular, at least some of the operations of process 800 may be performed within operational blocks 615, and / or 625 of process 600 in Fig. 6. A description of this process 800 will be described with reference to Fig. 9, which shows an example of adjusting the pose of the surgical tool so that the end effector is within a cone limit according to one embodiment. In particular, this figure shows the cone limit (or cone) 435 for limiting the range of motion of the end effector 430 about a cone axis that passes through the RCM 405 and the control point 900. In which case, prior to performing this process, the system may determine a cone angle (e.g., / ) of a cone limit of an end effector at a control point on the surgical tool. The process 800 begins by determining a vector form the control point to a fixed point (e.g., RCM) of the surgical tool (at block 805). In one embodiment, this vector may be computed from the rotation part of the input frame of the end effector pose, Rin, and the RCM frame, expressed in world (or global) coordinates. For instance, the range limiter may determine the vector prcm,cp between the RCM 405 and the control point 900, cp. In one embodiment, this vector may be determined based on position data associated with the RCM 405 and the control point 900, which is located at the yaw joint 425. Since the control point 900 is on the yaw joint, the position data received from one or more joints of the surgical tool 171 may be used to identify the location of the yaw joint (and therefore the control point). In another embodiment, when the control point is positioned off of a joint, its position may be determined based on position data and / or geometric knowledge of one or more portions (e.g., link) of the surgical tool 171. In one embodiment, the vector prcm,cp may correspond to the cone axis of the cone limit, which passes through the control point and the RCM.

[0077] The system 100 determines an end effector angle between a longitudinal axis of the end effector and an axis of the cone limit (at block 810). In this case, the axis of the cone limit may pass through a center of the cone limit and the control point 900. The longitudinal axis may be based on the rotation part of the input frame, Rin, that may describe the target orientation of the end effector based on the target tool pose. In this case, this axis may be Rtn,z, which is the Z-axis of the input rotation of the pose of the end effector. In one embodiment, Rin may be represented as a rotation matrix thatAtorney Docket No. : AUR6376WOPCT1Electronically Filed includes rotational components of the target tool pose, where R ,z may be the third column of the rotation matrix. The end effector angle, 3, may be derived by„9 = arccos

[0078] where 3 may be equal to the arccosine of the Euclidian scalar product of Rin,z and unit vector of Prcm,cP. The system 100 determines whether the end effector angle is greater than the cone angle (at decision block 815). In which case, the system 100 determines whether the target pose extends beyond the cone limit that is at the control point 900 on the yaw joint 425. As shown in Fig. 9, 3 extends from inside the cone 435 to outside the cone. Thus, as a result of 3 > / , the system determines an axis of rotation at the control point (at block 820). In particular, the system 100 determines an axis of rotation, rCorr between Rin and a rotation part of an output frame, Rout, which may represent the rotational matrix of a limited target tool pose, as described herein. The system determines rcorr as

[0079] where this axis of rotation is computed with a cross product of the unit vector formed by Prcm.cp and Rtn,z. Thus, rcorr is a perpendicular vector along which the input frame may be rotated. Specifically, the system 100 rotates the target pose along the axis of rotation by at least a difference between the cone angle and the end effector angle (at block 825). For instance, the system may define a correctional rotation frame asRcorr=AngleAxis(ji — 9, rcorr)

[0080] which indicate the rotation as the difference between the cone angle and the end effector angle along rCOrr. The limited target tool pose may be derived based on an adjusted output frame as out RcorrRinAtorney Docket No. : AUR6376WOPCT1Electronically Filed

[0081] As a result, the system 100 may adjust the target pose by rotating the target pose with respect to the rotational axis (e.g., about one or more axes at the center point) such that the end effector angle is within the rotational limit. Returning to decision block 815, if, however, the end effector angle is not greater than the cone angle, the system may not adjust the end effector angle, thereby causing the system to not limit the target tool pose based on the cone limit, since the end effector already lies within the cone angle.

[0082] Some aspects may perform variations to the processes 600 and / or 800 described herein. For example, the specific operations of at least some of the processes might not be performed in the exact order shown and described. The specific operations might not be performed in one continuous series of operations, and different specific operations might be performed in different embodiments.

[0083] As previously explained, an embodiment of the disclosure may be a non-transitory machine-readable medium (such as microelectronic memory) having stored thereon instructions, which program one or more data processing components (generically referred to here as a “processor”) to (automatically) determine a cone limit of a surgical tool and ensure that the target tool pose satisfies the cone limit, as described herein. In other embodiments, some of these operations might be performed by specific hardware components that contain hardwired logic. Those operations might alternatively be performed by any combination of programmed data processing components and fixed hardwired circuit components.

[0084] To aid the Patent Office and any readers of any patent issued on this application in interpreting the claims appended hereto, applicants wish to note that they do not intend any of the appended claims or claim elements to invoke 35 U.S.C. 112(f) unless the words “means for” or “step for” are explicitly used in the particular claim.

[0085] While certain embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad disclosure, and that the disclosure is not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those of ordinary skill in the art. The description is thus to be regarded as illustrative instead of limiting.Atorney Docket No. : AUR6376WOPCT1Electronically Filed

[0086] In some embodiments, this disclosure may include the language, for example, “at least one of [element A] and [element B].” This language may refer to one or more of the elements. For example, “at least one of A and B” may refer to “A,” “B,” or “A and B.” Specifically, “at least one of A and B” may refer to “at least one of A and at least one of B,” or “at least of either A or B.” In some embodiments, this disclosure may include the language, for example, “[element A], [element B], and / or [element C] ” This language may refer to either of the elements or any combination thereof. For instance, “A, B, and / or C” may refer to “A,” “B,” “C,” “A and B,” “A and C,” “B and C,” or “A, B, and C.”

Claims

Atorney Docket No. : AUR6376WOPCT1Electronically FiledCLAIMSWhat is claimed is:

1. A method performed by a surgical system that comprises a surgical tool and a human interface device (HID) configured to control the surgical tool, the method comprising: receiving, via the HID, a user command that includes a target pose of an end effector of the surgical tool, wherein the surgical tool comprises a shaft coupled to the end effector, disposed between the shaft and the end effector comprises a plurality of joints; determining a range of motion of the end effector along a control point on the surgical tool; determining whether the target pose of the end effector with respect to the control point is to extend beyond the range of motion of the end effector; and responsive to determining that the target pose of the end effector is to extend beyond the range of motion, adjusting the target pose of the end effector to stay within the range of motion.

2. The method of claim 1, wherein the range of motion comprises a coned boundary having a vertex at the control point about a vector that extends to a fixed point on the shaft of the surgical tool.

3. The method of claim 2, wherein the fixed point comprises a remote center of motion (RCM) of the surgical tool.

4. The method of claim 1, wherein the end effector comprises a grasper.

5. The method of claim 1, wherein the plurality of joints comprises a first joint having a first joint coupled between the shaft and a link and a second joint coupled between the link and the end effector, wherein the range of motion is determined as a minimum range of motion between:Atorney Docket No. : AUR6376WOPCT1Electronically Filed a first range of motion of the end effector when the first joint is at a first joint limit and the second joint is at a first midpoint joint angle within a first joint range; and a second range of motion of the end effector when the second joint is at a second joint limit and the first joint is at a second midpoint joint angle within a second joint range.

6. The method of claim 1, responsive to determining that the target pose of the end effector is to stay within the range of motion, providing joint commands to the plurality of joints to move the end effector from a current pose to the target pose.

7. The method of claim 1, wherein the user command indicates that the end effector is to perform a roll motion about an axis that extends longitudinally through the end effector to achieve the target pose, wherein the roll motion is to cause the plurality of joints to rotate about each of their respective axes while the control point is to remain in place.

8. The method of claim 1, wherein each joint of the plurality of joints comprises a different joint limit.

9. The method of claim 1, wherein the range of motion comprises a rotational limit around a rotational axis of the control point, wherein adjusting the target pose comprises rotating the target pose with respect to the rotational axis such that an end effector angle is within the rotational limit.

10. A surgical system comprising: a surgical tool having a shaft, a first joint arranged to rotate about a first axis, a second joint arranged to rotate about a second axis, different from the first axis, and an end effector, the first joint coupled to the shaft and the second joint is coupled between the first joint and the end effector; a human interface device (HID) arranged to control the surgical tool; at least one processor; andAtorney Docket No. : AUR6376WOPCT1Electronically Filed memory having instructions stored therein which when executed by the at least one processor causes the surgical system to: determine a target pose of the end effector based on movement of the HID; determine that the target pose extends beyond a cone limit that is at a control point on the second joint, wherein a cone axis of the cone limit passes through the control point and a fixed point on the shaft of the surgical tool; and adjust the target pose such that the target pose is in or on the cone limit.

11. The surgical system of claim 10, wherein the memory comprises further instructions to: determining a tool type of the surgical tool; and determining the cone limit based on the tool type.

12. The surgical system of claim 11, wherein the tool type comprises a needle driver that has a pair of jaws as the end effector.

13. The surgical system of claim 11, wherein the cone limit is determined based on a first range of motion of the first joint along the first axis and a second range of motion of the second joint along the second axis, which is different than the first axis.

14. The surgical system of claim 13, wherein at least one of the first range of motion or the second range of motion is less than a first joint limit of the first joint or a second joint limit of the second joint, respectively.

15. The surgical system of claim 11, wherein the fixed point comprises a remote center of motion (RCM) of the surgical tool.

16. The surgical system of claim 15, wherein a position of the RCM along the shaft is a predefined position.

17. A non-transitory machine-readable medium having instructions which when executed by at least one processor of a surgical system causes the surgical system to:Atorney Docket No. : AUR6376WOPCT1Electronically Filed receive, via a human interface device (HID), a user command that includes a target pose of an end effector of a surgical tool of the surgical system, wherein the surgical tool comprises a shaft coupled to the end effector, disposed between the shaft and the end effector comprises a plurality of joints; determine a range of motion of the end effector along a control point on the surgical tool; determine whether the target pose of the end effector with respect to the control point is to extend beyond the range of motion of the end effector; and responsive to determining that the target pose of the end effector is to extend beyond the range of motion, adjust the target pose of the end effector to stay within the range of motion.

18. The non-transitory machine-readable medium of claim 17, wherein the range of motion comprises a coned boundary having a vertex at the control point about a vector that extends to a fixed point on the shaft of the surgical tool.

19. The non-transitory machine-readable medium of claim 18, wherein the fixed point comprises a remote center of motion (RCM) of the surgical tool.

20. The non-transitory machine-readable medium of claim 17, wherein the end effector comprises a grasper.

21. The non-transitory machine-readable medium of claim 17, wherein the plurality of joints comprises a first joint having a first joint coupled between the shaft and a link and a second joint coupled between the link and the end effector, wherein the range of motion is determined as a minimum range of motion between: a first range of motion of the end effector when the first joint is at a first joint limit and the second joint is at a first midpoint joint angle within a first joint range; andAtorney Docket No. : AUR6376WOPCT1Electronically Filed a second range of motion of the end effector when the second joint is at a second joint limit and the first joint is at a second midpoint joint angle within a second joint range.

22. The non-transitory machine-readable medium of claim 17, responsive to determining that the target pose of the end effector is to stay within the range of motion, providing joint commands to the plurality of joints to move the end effector from a current pose to the target pose.

23. The non-transitory machine-readable medium of claim 17, wherein the user command indicates that the end effector is to perform a roll motion about an axis that extends longitudinally through the end effector to achieve the target pose, wherein the roll motion is to cause the plurality of joints to rotate about each of their respective axes while the control point is to remain in place.

24. The non-transitory machine-readable medium of claim 17, wherein each joint of the plurality of joints comprises a different joint limit.

25. The non-transitory machine-readable medium of claim 17, wherein the range of motion comprises a rotational limit around a rotational axis of the control point, wherein the instructions to adjust the target pose comprises instructions to rotate the target pose with respect to the rotational axis such that an end effector angle is within the rotational limit.

Images