Force feedback of robotic surgical instruments

Abstract

A console for use in a robotic surgical system includes one or more input control devices, a viewer, and a user interface associated with the viewer. Each input control device is configured to control manipulation of a respective surgical instrument. The viewer is configured to generate an endoscopic view showing a distal portion of the surgical instrument relative to tissue at a surgical site. The user interface includes a force feedback indicator. The force feedback indicator is configured to provide an indication of a force being applied to the tissue via the surgical instrument. The indication of the force corresponding to an actual force applied to a tissue by a portion of the surgical instrument.

Description

FORCE FEEDBACK OF ROBOTIC SURGICAL INSTRUMENTSPRIORITY

[0001] This application claims priority to U.S. Provisional Application Serial No. 63 / 718,020, entitled “Force Feedback of Robotic Surgical Instruments,” fded on November 8, 2024, the disclosure of which is incorporated by reference herein.BACKGROUND

[0002] A variety of surgical instruments include an end effector for use in conventional medical treatments and procedures conducted by a medical professional operator, as well as applications in robotically assisted surgeries. Such surgical instruments may be directly gripped and manipulated by a surgeon or incorporated into robotically assisted surgical systems. In the case of robotically assisted surgery, the surgeon may operate a master controller to remotely control the motion of such surgical instruments at a surgical site. The controller may be separated from the patient by a significant distance (e.g., across the operating room, in a different room, or in a completely different building than the patient). Alternatively, a controller may be positioned quite near the patient in the operating room. Regardless, the controller may include one or more hand input devices (such as joysticks, exoskeletal gloves, master manipulators, or the like), which are coupled by a servo mechanism to the surgical instrument. In one example, a servo motor moves a manipulator supporting the surgical instrument based on the surgeon's manipulation of the hand input devices. During the surgery, the surgeon may employ, via a robotic surgical system, a variety of surgical instruments including an ultrasonic blade, a surgical stapler, a tissue grasper, a needle driver, an electrosurgical cautery probe, endoscope, and / or etc. Each of these structures performs functions for the surgeon, for example, cutting tissue, coagulating tissue, holding or driving a needle, grasping a blood vessel, dissecting tissue, cauterizing tissue, or visualizing the surgical site and associated areas.

[0003] In some circumstances, robotically assisted surgical systems may include one or more operator perception features. Such operator perception features may be desirable toassist the surgeon in perceiving the surgical environment despite the surgeon not necessarily having physical presence at or near the surgical environment. Thus, such operator perception features may provide information not the surgeon to inform decision making during a surgical procedure.

[0004] While several surgical instruments and systems have been made and used, it is believed that no one prior to the inventors has made or used the invention described in the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] While the specification concludes with claims which particularly point out and distinctly claim this technology, it is believed this technology will be better understood from the following description of certain examples taken in conjunction with the accompanying drawings, in which like reference numerals identify the same elements and in which:

[0006] FIG. 1 depicts a top plan view of a robotic surgical system being used to perform a surgical procedure;

[0007] FIG. 2 depicts a perspective view of a surgeon's control console of the robotic surgical system of FIG. 1;

[0008] FIG. 3 depicts a front elevation view of a patient side cart of the robotic surgical system of FIG. 1 ;

[0009] FIG. 4 depicts a perspective view of a surgical instrument for use with the robotic surgical system of FIG. 1;

[0010] FIG. 5 depicts a perspective schematic view of potential degrees of freedom in which the wrist of the surgical instrument of FIG. 4 may be able to articulate (pivot) and translate;

[0011] FIG. 6 depicts a detailed perspective view of an end effector of the surgical instrument of FIG. 4;

[0012] FIG. 7 depicts a perspective view of a bottom portion of an instrument base of the surgical instrument of FIG. 4;

[0013] FIG. 8 depicts an example of a surgeon interface with an instrument overlay and one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0014] FIG. 9 depicts the surgeon interface of FIG. 8 with another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0015] FIG. 10 depicts the surgeon interface of FIG. 8 with yet another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0016] FIG. 11 depicts the surgeon interface of FIG. 8 with still another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0017] FIG. 12 depicts the surgeon interface of FIG. 8 with still another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0018] FIG. 13 depicts the surgeon interface of FIG. 8 with still another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0019] FIG. 14 depicts the surgeon interface of FIG. 8 with still another example of one or more force feedback indicators, which can be associated with the surgical instrument of FIG. 4;

[0020] FIG. 15 depicts detailed perspective view of the input control device of the surgeon’s control console of FIG. 2, the input control device having one or more force feedback mechanisms associated with a first portion or feature of the input control device;

[0021] FIG. 16 depicts another detailed perspective view of the input control device of FIG. 15, the input control device having one or more force feedback mechanisms associated with a second portion or feature of the input control device; and

[0022] FIG. 17 depicts yet another detailed perspective view of the input control device of FIG. 15, the input control device having one or more force feedback mechanisms associated with a third portion or feature of the input control device.

[0023] The drawings are not intended to be limiting in any way, and it is contemplated that various embodiments of the technology may be carried out in a variety of other ways, including those not necessarily depicted in the drawings. The accompanying drawings incorporated in and forming a part of the specification illustrate several aspects of the present technology, and together with the description serve to explain the principles of the technology; it being understood, however, that this technology is not limited to the precise arrangements shown.DETAILED DESCRIPTION

[0024] The following description of certain examples of the technology should not be used to limit its scope. Other examples, features, aspects, embodiments, and advantages of the technology will become apparent to those skilled in the art from the following description, which is by way of illustration, one of the best modes contemplated for carrying out the technology. As will be realized, the technology described herein is capable of other different and obvious aspects, all without departing from the technology. Accordingly, the drawings and descriptions should be regarded as illustrative in nature and not restrictive.

[0025] It is further understood that any one or more of the teachings, expressions, embodiments, examples, etc. described herein may be combined with any one or more of the other teachings, expressions, embodiments, examples, etc. that are described herein. The following-described teachings, expressions, embodiments, examples, etc. should therefore not be viewed in isolation relative to each other. Various suitable ways in which the teachings herein may be combined will be readily apparent to those of ordinary skill in the art in view of the teachings herein. Such modifications and variations are intended to be included within the scope of the claims.

[0026] For clarity of disclosure, the terms “proximal” and “distal” are defined herein relative to a human or robotic operator of the surgical instrument. The term “proximal” refers the position of an element closer to the human or robotic operator of the surgic al instrument and further away from the surgical end effector of the surgical instrument. The term “distal” refers to the position of an element closer to the surgical end effector of the surgical instrument and further away from the human or robotic operator of the surgical instrument. It will be further appreciated that, for convenience and clarity, spatial terms such as “clockwise,” “counterclockwise,” “inner,” “outer,” “upper,” “lower,” “lateral” and the like also are used herein for reference to relative positions and directions. Such terms are used below with reference to views as illustrated for clarity and are not intended to limit the invention described herein.

[0027] Aspects of the present examples described herein may be integrated into a robotically-enabled medical system, including as a robotic surgical system, capable of performing a variety of medical procedures, including both minimally invasive, such as laparoscopy, and non-invasive, such as endoscopy, procedures. Among endoscopy procedures, the robotically-enabled medical system may be capable of performing bronchoscopy, ureteroscopy, gastroscopy, etc.

[0028] I. Illustrative Robotic Surgical System with Endoscope

[0029] FIG. 1 shows a top plan view of an illustrative robotic surgical system (10) that may be used for performing a diagnostic or surgical procedure on a patient (12) who is lying down on an operating table (14). Robotic surgical system (10) may be constructed and operable in accordance with at least some of the teachings of U.S. Pat. No. 9,839,487, entitled “Backup Latch Release for Surgical Instrument,” issued December 12, 2017; U.S. Pat. No. 10,485,621, entitled “Sterile Barrier Between Surgical Instrument and Teleoperated Actuator,” issued November 26, 2019; U.S. Pat. No. 10,806,530, entitled “System and Method for Patient-Side Instrument Control,” issued October 20, 2020; U.S. Pat. No. 10,537,400, entitled “Detection Pins to Determine Presence of Surgical Instrument and Adapter on Manipulator,” issued January 21, 2020; U.S. Pat. No. 10,863,988, entitled “Surgical Instrument with Lockout Mechanism,” published December 15, 2020; U.S. Pat.No. 10,610,313, entitled “Surgical Instrument with Shiftable Transmission,” issued April 7, 2020; U.S. Pat. No. 11,020,138, entitled “Push-Pull Surgical Instrument End Effector Actuation Using Flexible Tension Member,” issued June 1, 2021; U.S. Pat. No. 11,026,755, entitled “Systems and Methods for Operating an End Effector,” issued June 8, 2021; U.S. Pat. No. 11,076,926, entitled “Manual Release for Medical Device Drive System,” issued August 3, 2021; U.S. Pub. No. 11,147,552, entitled “Stapler Cartridge With an Integral Knife,” issued October 19, 2021 ; U.S. Pat. No. 11,234,700, entitled “Wrist Architecture,” issued February 1, 2022; U.S. Pat. No. 11,259,884, entitled “Robotic Surgical Stapler Assembly Configured to Use Stapler Reload,” issued March 1, 2022; U.S. Pat. No. 11,364,029, entitled “Stapler Reload Detection and Identification,” issued June 21, 2022; U.S. Pat. No. 11,439,390, entitled “Surgical Instrument With Lockout Mechanism,” issued September 13, 2022; and / or U.S. Pat. No. 1 1,633,239, entitled “Locking System for Medical Device Drive System,” issued April 25, 2023. The disclosure of each of the above-cited U.S. Patents and U.S. Patent Publications is incorporated by reference herein in its entirety.

[0030] Robotic surgical system (10) may include a surgeon’s console (16) for use by a surgeon (18) during a surgical procedure. One or more assistants (20) may also participate in the procedure. Robotic surgical system (10) may include a patient side cart (22) (i.e., a surgical robot) and an electronics cart (24). Patient side cart (22) may manipulate at least one surgical instrument (26) (also referred to as a “tool assembly” or “tool”) through an incision in the body of patient (12) while surgeon (18) views the surgical site through surgeon’s console (16). As will be described in greater detail below, surgical instrument(s) (26) and an imaging device (shown as an endoscope (28)) may be removably coupled with patient side cart (22). Electronics cart (24) may be used to process the images of the surgical site for subsequent display to the surgeon (18) through surgeon’s console (16). Electronics cart (24) may be coupled with endoscope (28) and may include a processor (38) (shown schematically) to process captured images for subsequent display, such as to surgeon (18) on the surgeon's console (16), on a viewer (40) of electronics cart (24), or another suitable display located locally and / or remotely. The images may also be processed by a combination of electronics cart (24) and processor (38), which may becoupled together to process the captured images jointly, sequentially, and / or combinations thereof. Electronics cart (24) may overlay the captured images with a virtual control interface prior to displaying combined images to the surgeon (18) via surgeon’s console (16).

[0031] FIG. 2 shows aperspective view of surgeon’s console (16). Surgeon’s console (16) includes a left eye viewer (32) and a right eye viewer (34) for presenting surgeon (18) with a coordinated stereo view of the surgical site that enables depth perception. Surgeon’s console (16) includes one or more input control devices (36) causing patient side cart (22) (shown in FIG. 1) to manipulate one or more surgical instruments (26). Input control devices (36) may provide the same degrees of freedom as their associated surgical instruments (26) (shown in FIG. 1) to provide surgeon (18) with telepresence, or the perception that the input control devices (36) are integral with surgical instruments (26). To this end, position, force, and tactile feedback sensors (not shown) may be employed to transmit position, force, and tactile sensations from surgical instruments (26) back to the surgeon's hands through input control devices (36). In some instances, surgeon’s console (16) may be located in the same room as the patient so that surgeon (18) may directly monitor the procedure, be physically present if necessary, and speak to an assistant directly rather than over the telephone or other communication medium. Alternatively, surgeon (18) may be located in a different room, a completely different building, or other remote location from the patient allowing for remote surgical procedures.

[0032] In the present example, surgeon’s console (16) includes two input control devices (36) - a left hand input device (HID) (36L) and a right HID (36R), configured to be manipulated by the physician’s left and right hands, respectively (an HID is also sometimes referred to herein as a “human interface device”). Each of the HIDs can include a handle and / or finger inputs (also referred to as “external inputs”) that are manipulated by a user’s hands to control a corresponding instrument and / or corresponding robotic manipulator. For example, left HID (36L) may be controlled by a user’s left hand to control a left-hand instrument manipulated by a first robotic arm of the surgical robot, and the right HID (36R) may be controlled by a user’s right hand to control a right-hand instrument manipulated bya second robotic arm of the surgical robot. In the present example, each of the HIDs is physically supported by an armrest or other support feature of surgeon’s console (16) and / or a pillar by a respective positioning arm. Such positioning arms can include a series of links and series of joints, including a gimbal -based support, that supports the respective HID in space while permitting the respective HID to be manipulated in six degrees of freedom to control a corresponding position (e.g., location and / or orientation) of the respective instrument. Alternatively, or in combination, each of the left HID (36L) or right HID (36R) can include graspers and / or buttons that may be actuated by the user’s respective hands to actuate the instrument (e.g., to open or close instrument jaws, cut tissue with an instrument, seal tissue with an instrument) or control other functions of the surgical system. The illustrated configuration depicts grounded HIDs that are physically grounded to the console via positioning arms. In some variations, the surgeon console can employ ungrounded HIDs, such as free-floating and / or wireless input devices.

[0033] FIG. 3 shows patient side cart (22) that manipulates surgical instruments (26). An image of the surgical site may be obtained by endoscope (28), which may include a stereoscopic endoscope. Manipulation is provided by robotic mechanisms, shown as robotic arms (42) that include at least one robotic joint (44) and an output coupler (not shown) that is configured to removable secure surgical instrument (26) with robotic arm (42). Endoscope (28) and surgical tools (26) may be positioned and manipulated through incisions in the patient so that a kinematic remote center is maintained at the incision to minimize the size of the incision. Images of the surgical site may include images of the distal ends of the surgical instruments (26) when they are positioned within the field-of- view of the endoscope (28). Patient side cart (22) may output the captured images for processing outside electronics cart (24). The number of surgical instruments (26) used at one time will generally depend on the diagnostic or surgical procedure and the space constraints within the operating room, among other factors. To change one or more of surgical instruments (26) being used during a procedure, assistant (s) (20) may remove surgical instrument (26) from patient side cart (22) and replace surgical instrument (26) with another surgical instrument (26) from a tray (30) (shown in FIG. 1) in the operating room.

[0034] FIGS. 4-7 show an illustrative surgical instrument (110) (also referred to as a surgical tool or tool) that may be mounted on and used with patient side cart (22) shown in FIG. 3 similar to surgical instrument (26) described above. Surgical instrument (110) can have any of a variety of configurations capable of performing one or more surgical functions. As shown, surgical instrument (110) includes an instrument base (112) and a shaft assembly (114) (also referred to as a shaft) extending distally from instrument base (112). Instrument base (112) includes an attachment interface (118) that includes input couplers (130) that are configured to interface with and be driven by corresponding output couplers (not shown) of robotic arm (42) of patient side cart (22).

[0035] Surgical instrument (110) further includes an end effector (140) having a wrist (160) (also referred to as a wrist joint or an articulable wrist joint) disposed on a distal end of shaft assembly (114). During use of the surgical instrument (110), end effector (140) is configured to move (e.g., pivot, rotate, etc,) relative to the shaft assembly (114) at wrist (160) to position the end effector (140) at desired orientations and locations relative to a surgical site. Thus, although not shown, instrument base (112) includes (contains) various drive inputs and mechanisms (e.g., gears, actuators, etc.) designed to control operation of various features associated with end effector (140) (e.g., clamping, firing, cutting, rotation, articulation, etc.). In at least some examples, shaft assembly (114), and hence the end effector (140) coupled thereto, is configured to rotate about a longitudinal axis (Ai) of shaft assembly (114). In such embodiments, at least one of input couplers (130) (e.g., the drive inputs) included in instrument base (112) is configured to control rotational movement of shaft assembly (114) about the longitudinal axis (Ai).

[0036] FIG. 5 illustrates the potential degrees of freedom in which wrist (160) may be able to articulate (pivot) and thereby move end effector (140). Wrist (160) can have any of a variety of configurations. In general, wrist (160) includes a joint configured to allow pivoting movement of end effector (140) relative to shaft assembly (114), thereby pivoting the longitudinal axis (A2) of end effector (140) relative to the longitudinal axis (Ai) of shaft assembly (114). The degrees of freedom of wrist (160) are represented by three translational variables (i.e., surge, heave, and sway), and by three rotational variables (i.e.,Euler angles or roll, pitch, and yaw). The translational and rotational variables describe the position and orientation of end effector (140) with respect to a given reference Cartesian frame. As depicted in FIG. 5, “surge” refers to forward and backward translational movement, “heave” refers to translational movement up and down, and “sway” refers to translational movement left and right. With regard to the rotational terms, “roll” refers to tilting side to side, “pitch” refers to tilting forward and backward, and “yaw” refers to turning left and right.

[0037] The pivoting motion can include pitch movement about a first axis of wrist (160) (e.g., X-axis), yaw movement about a second axis of wrist (160) (e.g., Y-axis), and combinations thereof to allow for 360° rotational movement of end effector (140) about wrist (160). In other applications, the pivoting motion can be limited to movement in a single plane, e.g., only pitch movement about the first axis of wrist (160) or only yaw movement about the second axis of wrist (160), such that end effector (140) moves only in a single plane.

[0038] As best seen in FIG. 6, end effector (140) can include a variety of sizes, shapes, and configurations. In the present example, end effector (140) includes a combination tissue grasper and vessel sealer that include opposing first (upper) and second (lower) jaws (142, 144) configured to move (articulate) between open and closed positions. As will be appreciated, however, opposing jaws (142, 144) may alternatively form part of other types of end effectors such as, but not limited to, a surgical scissors, a clip applier, a needle driver, a babcock including a pair of opposed grasping jaws, bipolar jaws (e.g., bipolar Maryland grasper, forceps, a fenestrated grasper, etc.), etc. One or both of jaws (142, 144) may be configured to pivot to articulate end effector (140) between the open and closed positions.

[0039] In some examples, surgical instrument (110) may be supplied with electrical power (current) via a power cable (113) coupled to instrument base (112). In other examples, power cable (113) may be omitted and electrical power may be supplied to the surgical instrument (110) via an internal power source, such as one or more batteries, capacitors, or fuel cells. Optionally, power cable (113) may place surgical instrument (110) in electrical communication with a generator (not shown) or other component configured to supplyenergy, such as electrical energy (e.g., radio frequency energy), ultrasonic energy, microwave energy, heat energy, or any combination thereof, to surgical instrument (110) and, more particularly, to end effector (140). Accordingly, the generator may comprise a radio frequency (RF) source, an ultrasonic source, a direct current source, and / or any other suitable type of electrical energy source that may be activated independently or simultaneously.

[0040] As best seen in FIG. 6, one or more portions of end effector (140), wrist (160), and / or shaft assembly (114) may include one or more force sensors (116, 146) to provide force related feedback during a procedure. In the present example, a wrist force sensor (116) is incorporated into a distal portion of shaft assembly (114) proximate wrist (160). In this configuration, wrist force sensor (116) is configured to provide force feedback with respect to forces applied to tissue with respect to wrist (160). In some examples, multiple forces may act on wrist (160) via end effector (140). Thus, such multiple forces may be vectored into a single force vector to indicate a direction and magnitude of the force in a cartesian coordinate system corresponding to the degrees of freedom shown in FIG. 5. Although wrist force sensor (116) is shown in the present example schematically as one sensor, it should be understood that in some examples multiple wrist force sensors (116).

[0041] Additionally, end effector (140) may include one or more jaw force sensors (146) configured to detect a magnitude of force applied to tissue between jaws (142, 144). This jaw force sensors (146) may be incorporated into first jaw (142), second jaw (144), or both. Alternatively, in some examples, one or more jaw force sensors (146) may be incorporated into one or more components used to drive jaws (142, 144). For instance, in some examples, jaws (142, 144) may be driven by one or more pull cables. In such examples, the tension applied to such pull cables via one or more motors may be used to extrapolate the force applied to tissue via jaws (142, 144) using known kinematics of robotic surgical system (10). Regardless, such jaw force sensors (146) may provide information related to the force applied to tissue via jaws (142, 144).

[0042] FIG. 7 shows attachment interface (118) of instrument base (112) in greater detail. Attachment interface (118) is configured to operatively couple instrument base (112) to acorresponding attachment interface of a respective robotic arm (42). Attachment interface (118) may releasably couple instrument base (112) to a tool driver in a variety of ways, such as by clamping thereto, dipping thereto, or slidably mating therewith. In some examples, attachment interface (118) may include an array of electrical connecting pins, which may be coupled to an electrical connection on the mounting surface of the respective robotic arm (42). While attachment interface (118) is described herein with reference to mechanical, electrical, and magnetic coupling elements, it should be understood that a wide variety of telemetry modalities might be used, including infrared, inductive coupling, or the like.

[0043] Input couplers (130) may be configured to control a variety of functions of surgical instrument (110). For instance, in some examples, actuation of one input coupler (130) a may be configured to control rotation of shaft assembly (114) about its longitudinal axis (Ai). In such examples, shaft assembly (114) may be rotated clockwise or counterclockwise depending on the rotational actuation of a given input coupler (130). Similarly, actuation of other input coupler (130) may be configured to operate movement of wrist (160) and / or articulation (operation) of the end effector (140) including opening and closing of jaws (142, 144). Additionally, actuation of another input coupler (130) may be configured to advance and / or retract other components of end effector (140) such as a knife (not shown) incorporated within a portion of jaws (142, 144). Each of input couplers (130) may be actuated based on user inputs communicated to the attachment interface of a given robotic arm (42) coupled to attachment interface (118), and the user inputs may be received via processor (38) incorporated into electronics cart (24) of robotic surgical system (10).

[0044] Attachment interface (118) defines an interface configured to mechanically, magnetically, and / or electrically couple instrument base (112) a respective robotic arm (42). In the present example, the interface of attachment interface (118) includes and supports a plurality of drive inputs referred to herein as input couplers (130). Each input coupler (130) includes a rotatable disc configured to align with and couple to a corresponding actuator or “drive output” of an attachment interface of a respective robotic arm (42), such that rotation (actuation) of a given drive output drives (rotates) acorresponding one of input couplers (130). Each input coupler (130) may provide or define one or more surface features configured to align with mating surface features provided on the corresponding drive output. Suitable surface features can include, for example, various protrusions and / or indentations that facilitate a mating engagement. In some examples, some or all of input couplers (130) may include one surface feature that is positioned closer to an axis of rotation of the associated input coupler (130) than the other surface feature(s). This may help to ensure positive angular alignment of each input coupler (130).

[0045] Although various aspects of robotic surgical system (10) are described herein in the context of surgical instrument (110), it should be understood that the such teachings may be readily applied to other instruments such as surgical instruments (26) described above. Indeed, surgical instrument (110) may be readily used in robotic surgical system (10) interchangeably with other suitable instruments such that one or more functions of robotic surgical system (10) may be applied to other instruments when used with robotic surgical system (10). Reference to surgical instrument (110) herein is merely for illustrative purposes to show how various features of robotic surgical system (10) may be applied in practice to one or more instruments generally.

[0046] FIG. 8 shows an illustrative endoscopic view (200), which may be presented to a user to guide robotic surgical system (10) in real-time using one or more input control devices (36). In the present example, endoscopic view (200) is presented to a user via surgeon’s console (16) using a combination of left eye viewer (32) and right eye viewer (34). In other examples, endoscopic view (200) may be optionally presented to a user using viewer (40) of electronics cart (24) or other suitable viewers or screens used in connection with robotic surgical system (10). In such examples, endoscopic view (200) may be presented to a user in combination with both left and right eye viewers (32, 34) and viewer (40).

[0047] Endoscopic view (200) of the present example includes a real-time or near realtime video feed of a view of the surgical site. The source of endoscopic view (200) may originate from a variety of sources such as endoscope (28) described above, a combination of a plurality of endoscopes (28), or a variety of other image sources. Regardless of source,endoscopic view (200) may show images of an anatomical site and a least a portion of the active surgical instruments (26, 110) being used during a surgical procedure at the anatomical site. Thus, endoscopic view (200) may be used for the purpose of guiding surgical instruments (26, 110) during the course of a surgical procedure.

[0048] In the present example, endoscopic view (200) may be associated with two instruments including a first surgical instrument (210a) (e.g., a tissue grasper) and a second surgical instrument (210b) (e.g., a tissue cutter). Optionally, in some examples, a third instrument may be visible and under control of robotic surgical system (10) such as another version of surgical instrument (26, 110) or a endoscope (28). Of course, various other suitable combinations of surgical instruments (26, 110) viewed by endoscopic view (200) may be used as will be apparent to those of ordinary skill in the art in view of the teachings herein. Although certain specific surgical instruments are shown herein with respect to endoscopic view (200), it should be understood that in other examples, other suitable instruments may be used, which may correspond to instruments (26, 110) described above.

[0049] Additional information may be displayed as one or more overlays displayed concurrently with the endoscopic view (200). The overlays may be displayed in varying positions and be of varying shapes and may also feature characteristics such as pulsing. In the present example, endoscopic view (200) is associated with an instrument overlay (220) that indicates additional information associated with the surgical instruments introduced to the anatomical site. Here, instrument overlay (220) is presented at a bottom of the display, but it will be appreciated that instrument overlay (220) can be presented in other arrangements.

[0050] Instrument overlay (220) is configured to provide information such as an identification of a robotic arm controlling each instrument (e.g., the number 1, 2, or 3), an identification of a type of each instrument (e.g., monopolar scissors, scope, sealer, forceps), an identification of a status of each instrument, and / or an identification of operations that can be performed by one or more of the instruments (e.g., cut, seal, jaw compression assessment, jaw aperture assessment). For instance, in FIG. 8, instrument overlay (220) includes a first section corresponding to the first surgical instrument (210a) and the firstrobotic arm and a second section corresponding to second surgical instrument (210b) and the second robotic arm. Optionally, in versions using a third surgical instrument, instrument overlay (220) may include a third section corresponding to the third surgical instrument and the third robotic arm. Further, a section may include one or more icons that provide information associated with the corresponding instrument; for example, the first section may identify first surgical instrument (210a) as a tissue gasper controlled by robotic arm 1 with a first operation icon (222a) corresponding to a first operation associated with first surgical instrument (210a) and a second operation icon 202b) corresponding to a second operation associated with first surgical instrument (210a). Similarly, second section may identify second surgical instrument (210b) as a tissue cutter controlled by robotic arm 2 with a first operation icon (224a) corresponding to a first operation associated with second surgical instrument (210b) and a second operation icon (224b) corresponding to a second operation associated with second surgical instrument (210b). In this example, icons (222a- b, 224a-b) may also be representative of the actuation means for the corresponding operations. In other words, icons (222a-b, 224a-b) may be assigned to any of the input control devices (36), whereby actuating the assigned input device executes the operation.

[0051] In some variations, the surgeon interface can provide additional information associated with the procedure or instrumentation. For instance, instrument overlay (210) may be configured to provide an alert overlay or other status indicator related to one or more of surgical instruments (210a, 210b). Such alert overlays may be configured to convey one or more messages, such as a status of one or more surgical instruments (210a, 210b) or other operational information. Where such alert overlays are used, such alert overlays may be presented in a persistent and unobtrusive manner to avoid obscuring the endoscopic view (200) of the surgical site and convey the overlay’s message effectively, such as by positioning proximate to one or more distal ends of surgical instruments (210a, 210b) or proximate to another overlay (e.g., instrument overlay (220)). Optionally, such alert overlays may also be repositioned in real-time to maintain proximate relationships with other user interface features.

[0052] II. Illustrative Force Feedback Indicators and Mechanisms for Robotic Surgical System

[0053] In some examples, it may be desirable to incorporate one or more force feedback indicators into one or more components of robotic surgical system (10). Real-time feedback is generally desirable in robotically assisted surgery settings to provide perception more aligned with conventional non -robotically assisted surgery settings, or even enhance perception beyond such settings. Force feedback may be provided in a variety of ways or through a variety of modes. For example, force feedback may be in tactile, vi sual, and / or auditory forms or modes. Regardless of the form or mode of delivery, it is desirable to provide force feedback in a way that is easily perceptible to a user without undermining other aspects of a surgical procedure. In particular, force feedback that may be impede input to input control devices (36), or obscure or distract from aspects of endoscopic view (200) may be less desirable than force feedback without such impediments. In other words, force feedback may be more desirable in forms that reduce or eliminate any impedance of delivery to a user.

[0054] In some examples, force feedback indicators may be used in combination with other force feedback mechanisms to complement or supplement such force feedback mechanisms. For instance, force may be discernible from various visual cues within endoscopic view (200). This form of force feedback may be enhanced or supplemented by other visual force feedback indicators included along with endoscopic view (200). Such visual force feedback indicators may additionally be desirable in training settings to build associations between visual cues and applied force. Similarly, one or more input control devices (36) may provide force feedback by increasing resistance on those inputs in response to detected forces. Such resistance based feedback mechanisms may be supplemented through the addition of other tactile force feedback mechanisms, such as vibrations, bumps, and / or etch. Although certain specific force feedback indicators and mechanisms are described below, it should be understood that such force feedback indicators and mechanisms may be implemented in various combinations as will be appreciated by those of ordinary skill in the art in view of the teachings herein.

[0055] A. Illustrative Force Feedback Indicators

[0056] In some examples, it may be desirable to include one or more force feedback indicators (310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422) in combination with one or more aspects of viewers (32, 34, 40) described above. For instance, as best seen in FIG. 9, endoscopic view (200) displayed via left eye viewer (32) and right eye viewer (34) or viewer (40) may be associated with a force feedback indicator (310). Force feedback indicator (310) in the present example is superimposed onto a portion of endoscopic view (200) corresponding to tissue (T). In particular, force feedback indicator (310) is configured as a tissue highlighter, highlighting the tissue being manipulated by one or more of surgical instruments (210a, 210b). Thus, force feedback indicator (310) is configured to provide direct visualization of the force applied to tissue (T) in real time. In the present example, force feedback indicator (310) has a form corresponding to the particular structure of the tissue (T) being manipulated and therefore may change dynamically over time. In other examples, the form of force feedback indicator (310) may have a predetermined shape, not necessary corresponding to the particular structure of the tissue (T) being manipulated.

[0057] One or more characteristics of force feedback indicator (310) may change as a function of force applied either by first surgical instrument (210a), second surgical instrument (210b), or both. For instance, in the present example, force feedback indicator (310) may have a variable opacity corresponding to the amount of force applied. In one example, the opacity of force feedback indicator (310) may be proportional to the force applied with a 0% percent opacity (e.g., completely transparent) corresponding to no force applied and a 100% opacity (e.g., completely opaque, solid) corresponding to a maximum or upper limit force applied. In other examples, a color of force feedback indicator (310) may change at one or more predetermined thresholds. For instance, force feedback indicator (310) may be in the color green in the presence of relatively low forces, the color yellow in the presence of moderate forces, and the color red in the presence of relatively high forces. In some examples, variable opacity and variable color may be used with force feedback indicator (310) in combination.

[0058] The force indicated by force feedback indicator (310) may be generated using one or more sensors associated with a given surgical instrument illustrated by first surgical instrument (210a) and / or second surgical instrument (210b). For instance, as described above, surgical instrument (110) or other similar surgical instruments may include one or more force sensors (116, 146). One or more of such force sensors (116, 146) or similar sensors may be used to provide input for force feedback indicator (310) or other force feedback indicators (320, 322, 330, 332, 340, 342, 410, 412, 420, 422) or force feedback mechanisms (510, 520, 530) described herein. In one example, wrist force sensor (116) or sensors may be configured to compile multiple forces detected proximate wrist (160) into a single force vector expressing a magnitude and direction (e.g., sway, heave, pitch, roll, etc.) of the net force applied to wrist (160) or proximate to wrist (160). Such a force vector may then be expressed via force feedback indicator (310) to indicate the magnitude of the force and / or the direction of the force. In addition, or in the alternative, jaw force sensor (146) may provide a compressive force vector indicative of the compressive force applied to tissue (T) via jaws (142, 144). Such a compressive force vector may then be similarly expressed via force feedback indicator (310) to indicate the magnitude of the compressive force applied to tissue (T).

[0059] In other examples, one or more force feedback indicators may be used in connection with other aspects of endoscopic view (200). For instance, as best seen in FIG. 10, endoscopic view (200) displayed via left eye viewer (32) and right eye viewer (34) or viewer (40) may be associated with one or more force feedback indicators (320, 322) associated with one or more of surgical instruments (210a, 210b). Force feedback indicators (320, 322) in the present example are superimposed onto a portion of endoscopic view (200) corresponding to first surgical instrument (210a) and second surgical instrument (210b). In particular, a left force feedback indicator (320) corresponds to first surgical instrument (210a) and a right force feedback indicator (322) corresponds to second surgical instrument (210b). Although the present example includes two force feedback indicators (320, 322), it should be understood that in other examples other combinations of force feedback indicators (320, 322) may be used. For instance, in examples where endoscopicview (200) includes a third surgical instrument (not shown), another force feedback indicator similar to force feedback indicators (320, 322) may be used.

[0060] Similar to force feedback indicator (310) described above, force feedback indicators (320, 322) of the present example are configured with a force-related highlighting function. However, rather than highlighting the tissue being manipulated, at least a portion of surgical instruments (210a, 210b) is highlighted instead. Such a configuration may be desirable in circumstances where enhanced visibility of tissue is desired at all times during a procedure, while still having one or more force feedback indicators with at least a peripheral field of view.

[0061] In the present example, each force feedback indicator (320, 322) has a form corresponding to the particular structure of a corresponding surgical instrument (210a, 210b). For instance, each force feedback indicator (320, 322) may correspond to the shape of a shaft assembly or other element of a given surgical instrument (210a, 210b). Optionally, the shape and / or size of each force feedback indicator (320, 322) may dynamically change over time as one or more of surgical instruments (320, 322) move within endoscopic view (200), thereby change the visible shape and / or size of the structures of each surgical instrument (320, 322). In other examples, the form of force feedback indicator (310) may have a predetermined shape, not necessary corresponding to the particular structure of a given surgical instrument (210a, 210b) as it moves within endoscopic view (200).

[0062] As similarly described above with respect to force feedback indicator (310), one or more characteristics of each of force feedback indicators (320, 322) may change as a function of force applied either by a corresponding surgical instrument (210a, 210b). For instance, in the present example, force feedback indicators (320, 322) may have a variable opacity corresponding to the amount of force applied to tissue by a given surgical instrument (210a, 210b). In one example, the opacity of force feedback indicators (320, 322) may be proportional to the force applied to tissue with a 0% percent opacity (e.g., completely transparent) corresponding to no force applied and a 100% opacity (e.g., completely opaque, solid) corresponding to a maximum or upper limit force applied. Inother examples, a color of force feedback indicators (320, 322) may change at one or more predetermined thresholds. For instance, one or more of force feedback indicators (320, 322) may be in the color green in the presence of relatively low forces, the color yellow in the presence of moderate forces, and the color red in the presence of relatively high forces. In some examples, variable opacity and variable color may be used with one or more of force feedback indicators (320, 322) in combination.

[0063] FIG. 11 shows use of force feedback indicators (330, 332) in combination with endoscopic view (200). Force feedback indicators (330, 332) are substantially similar to force feedback indicators (320, 322) described above. For instance, as similarly described above, force feedback indicators (330, 332) are superimposed on endoscopic view (200) in a region corresponding to a respective surgical instrument (210a, 210b). Force feedback indicators (330, 332) are also responsive to a force applied by a respective surgical instrument (210a, 210b) to change in opacity and / or color based on the amount of force applied. However, unlike force feedback indicators (320, 322), force feedback indicators (330, 332) are isolated to a limited region of endoscopic view (200). Specifically, each force feedback indicator (330, 332) is isolated to a small rounded or semi -circular protrusion at each edge of endoscopic view (200). Any suitable protrusion may be used. By way of example only, in some examples, each force feedback indicator (330, 332) may protrude for about the first l / 5thto 178thof the visible length of a respective surgical instrument (210a, 210b). Such an isolated characteristic may be desirable in some circumstances to maximize instrument visibility, while still providing at least some force feedback indicators within at least a peripheral field of view of a user.

[0064] FIG. 12 shows use of force feedback indicators (340, 342) in combination with endoscopic view (200). Force feedback indicators (340, 342) are substantially similar to force feedback indicators (320, 322) described above. For instance, as similarly described above, force feedback indicators (340, 342) are superimposed on endoscopic view (200). However, rather than being in a region corresponding to a respective surgical instrument (210a, 210b), force feedback indicators (340, 342) are in a region proximate a respective surgical instrument (210a, 210b), but positioned a predetermined distance away from therespective surgical instrument (210a, 210b). Generally, force feedback indicators (340, 342) may be positioned in a region of endoscopic view (200) of relatively low visual utility such as near edges and a distance from surgical instruments (210a, 210b). Such positioning may be desirable to maximize unity of endoscopic view (200) for guidance of surgical instruments (210a, 210b), while still providing force feedback at least within a peripheral field of view of a user. Such positioning may also be desirable in circumstances where one or more of surgical instruments (210a, 210b) are positioned outside of endoscopic view (200) and thus in a “blind spot.” In such circumstances, force feedback indicators (340, 342) may remain visible despite one or more of surgical instruments (210a, 210b) being out of frame.

[0065] As with force feedback indicators (320, 322) described above, force feedback indicators (340, 342) of the present example are also responsive to a force applied by a respective surgical instrument (210a, 210b) to change in opacity and / or color based on the amount of force applied. However, unlike force feedback indicators (320, 322), force feedback indicators (340, 342) are of a predetermined shape not necessarily associated with the shape of surgical instruments (210a, 210b). For instance, in the present example, each force feedback indicator (340, 342) has a rounded or button shaped configuration. Of course, other suitable shapes may be used such as square, rectangular, or triangular shapes.

[0066] In some examples, it may be desirable to incorporate one or more force feedback indicators into viewers (32, 34. 40) separately from endoscopic view (200) rather than as a superimposed indicator as described above with respect to force feedback indicators (310, 320, 322, 330, 332, 340, 342). For instance, as best seen in FIG. 13, in some examples, one or more force feedback indicators (410, 412) may be incorporated into views (32, 34, 40) proximate one or more sides of endoscopic view (200). In the present example, force feedback indicators (410, 412) include a left side force feedback indicator (410) and a right side force feedback indicator (412) with left side force feedback indicator (410) being positioned on a left side of endoscopic view (200) and right side force feedback indicator (412) being positioned on a right side of endoscopic view (200). Such positioning may be desirable in some examples to avoid obscuring any portion of endoscopic view (200), whilemaximizing the visual appearance of each force feedback indicator (410, 412) within a peripheral field of view. Such positioning may also be desirable where force feedback indicators (410, 412) are used in combination with other force feedback indicators (310, 320, 322, 330, 332, 340, 342) to provide feedback with respect to multiple forces (e.g., tissue compression and tissue lift) simultaneously.

[0067] Each force feedback indicator (410, 412) of the present example is in a bar -shaped configuration. In particular, each force feedback indicator (410, 412) is defined as an elongate rectangular bar extending the height of endoscopic view (200). Within the bar of each force feedback indicator (410, 412), a movable indicator line (411, 413) or other graphical element is used to indicate the force applied by a respective surgical instrument (210a, 210b). In other words, indicator line (411, 413) may move up and down to indicate the magnitude of force applied by a respective surgical instrument (210a, 210b). In the present example, indicator line (411, 413) is indicative of tissue compression via sensors such as jaw force sensor (146) (e.g., tissue grip). In other examples, indicator line (411, 413) may be indicative of other forces such as tissue lift via sensors such as wrist force sensor (116).

[0068] Optionally, the bar of each force feedback indicator (410, 412) may include a color pattern or gradient, which may be indicative of the level of force applied (e.g., green or yellow to indicate lower or moderate forces and orange or red to indicate higher forces). Where such a color pattern or gradient is used, the particular appearance may be configurable by a user to match the user’s preferences in terms of force sensitivity.

[0069] In other examples, force feedback indicators such as force feedback indicators (410, 412) described above may be presented in various different configurations. For instance, FIG. 14 shows force feedback indicators (420, 422) that are substantially similar to force feedback indicators (410, 412), but in a different configuration. As can be seen, force feedback indicators (420, 422) are configured as dial indicators rather than bar indicators. Like force feedback indicators (410, 412) described above, force feedback indicators (420, 422) include a left side force feedback indicator (420) corresponding to tissue compression of first surgical instrument (210a) and a right side force feedback indicator (412)corresponding to tissue compression of second surgical instrument (210b). As similarly described above, left side force feedback indicator (420) is positioned on a left side of endoscopic view (200) and right side force feedback indicator (422) is positioned on a right side of endoscopic view (200).

[0070] Each force feedback indicator (420, 422) is also similarly responsive to force (e.g., tissue compression, or tissue lift). In particular, each force feedback indicator (420, 422) includes a graphical arm (421, 423) or arrow indicator that is movable within a generally circular or arc-shaped outline to indicate the force applied by a respective surgical instrument (210a, 210b). In other words, graphical arm (421, 423) may pivot up and down to indicate the magnitude of force applied by a respective surgical instrument (210a, 210b). Optionally, the circular or arc -shaped outline of each force feedback indicator (420, 422) may be fdled with a color pattern or gradient, which may be indicative of the level of force applied (e.g., green or yellow to indicate lower or moderate forces and orange or red to indicate higher forces). In examples with such a color pattern or gradient, the appearance of such color pattern or gradient may be customizable by a user to match the user’s preference in terms of force sensitivity.

[0071] B. Illustrative Force Feedback Mechanisms

[0072] In some examples, it may be desirable to include one or more force feedback mechanisms in robotic surgical system (10) in addition to, or in lieu of, force feedback indicators (310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422) described above. One or more force feedback mechanisms may be desirable in some circumstances to provide force feedback to a user via other modes of perception beyond the visual modes of perception described above with respect to force feedback indicators (310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422). The addition of other modes of perception for force feedback may be desirable, for example, to accommodate user preferences, to enhance existing force feedback mechanisms, and / or to complement other modes of force feedback perception. Although examples of certain specific forms of force feedback mechanisms are shown and described herein, it should be understood that in other examples, various combinations ofsuch force feedback mechanisms may be used as will be appreciated by those of ordinary skill in the art in view of the teachings herein.

[0073] FIGS. 15 through 17 show ways in which various force feedback mechanisms may be incorporated into one or more of input control devices (36) of Surgeon’s console (16). For instance, FIG. 15 shows the implementation of a force feedback mechanism (510) associated with input control device (36) that is responsive to force applied to tissue (e.g., lift force) by a surgical instrument such as surgical instrument (110) described above. In the present example, input control device (36) is generally configured to vibrate with one or more predetermined vibration patterns when a force applied to tissue is detected as exceeding one or more predetermined force thresholds. By way of example only, the force applied to tissue may be detected by one or more sensors such as sensor (116) described above with respect to surgical instrument (110).

[0074] The predetermined force thresholds associated with force feedback mechanism (510) may correspond to different thresholds of interest for an operator. For instance, in one example, the predetermined force thresholds associated with force feedback mechanism (510) may include one predetermined force threshold corresponding to a high level of force. Such a single predetermined force threshold may be desirable in some examples to alert a user to the presence of high forces. In other examples, predetermined force thresholds associated with force feedback mechanism (510) may correspond to a plurality of different levels of force such as low, medium, and / or high. Such multiple predetermined force thresholds may be desirable in some examples to provide force perception closer to real-time rather than more episodic based perceptive modes.

[0075] Force feedback mechanism (510) may include a variety of vibration patterns. In some examples, the vibration patterns used may be related to the different predetermined force thresholds used with force feedback mechanism (510). For instance, in examples with the single predetermined force threshold, only a slight bump of vibration may be used by force feedback mechanism (510) to provide a user with a short indicator of the detected force exceeding a high value. When the multiple predetermined force threshold is used, multiple different vibration patterns may be used to indicate different levels of force. Forinstance, short bursts or bumps of vibrations may be used in connection with force thresholds at lower levels, while continuous or partially continuous vibrations may be used in connection with force thresholds at higher levels. Alternatively, in other examples, the intensity of vibrations may be varied based on the predetermined force threshold, with lower intensity vibrations associated with lower force levels and higher intensity vibrations associated with higher force thresholds. In addition, or in the alternative, vibration patterns and / or intensities may be preselected or user adjustable.

[0076] FIG. 16 shows another force feedback mechanism (520) that may be incorporated into one or more of input control devices (36) of Surgeon’s console (16) either alone or in combination with other force feedback mechanisms (510, 530) described herein. Force feedback mechanism (520) of the present example is generally configured to provide one or more vibrations in response to tissue compression detected by a sensor such as jaw force sensor (146) described above with respect to surgical instrument (110). In other words , force feedback mechanism (520) is configured to provide tactile feedback with respect to the grip of tissue by one or more elements of a surgical instrument such as jaws (142, 144) of surgical instrument (110).

[0077] Force feedback mechanism (520) may optionally be associated with a different element of input control device (36) relative to force feedback mechanism (510) described above. For instance, vibrations associated with force feedback mechanism (510) may originate from a larger portion of input control device (36) such as a base or other component, while vibrations associated with force feedback mechanism (520) may originate from a more particular portion of input control device (36) such as a grip. Such variation in the score of vibrations may be desirable in circumstances where one or more of force feedback mechanisms (510, 520) are used in combination to permit a user to differentiate between force feedback mechanisms (510, 520).

[0078] As with force feedback mechanism (510) described above, force feedback mechanism (520) of the present example may be operable in accordance with certain predetermined force thresholds corresponding to different thresholds of interest for an operator. As similarly described above, in one example, the predetermined force thresholdsassociated with force feedback mechanism (520) may include one predetermined force threshold corresponding to a high compressive force, which may be indicative of grip on tissue. In other examples, predetermined force thresholds associated with force feedback mechanism (520) may correspond to a plurality of different levels of force such as low, medium, and / or high. Such multiple predetermined force thresholds may be desirable in some examples to provide perception of different tissue thicknesses such as thin, medium and thick.

[0079] Force feedback mechanism (520) may similarly include a variety of vibration patterns. In some examples, the vibration patterns used may be related to the different predetermined force thresholds used with force feedback mechanism (520). For instance, in examples with the single predetermined force threshold, only a slight bump of vibration may be used by force feedback mechanism (520) to provide a user with a short indicator of tissue grasping. When the multiple predetermined force threshold is used, multiple different vibration patterns may be used to indicate different tissue compression attributes. For instance, short bursts or bumps of vibrations may be used in connection with force thresholds indicative of thinner tissues, while continuous or partially continuous vibrations may be used in connection with force thresholds indicative of thicker tissues. Alternatively, in other examples, the intensity of vibrations may be varied based on the predetermined force threshold, with lower intensity vibrations associated with thinner tissue thicknesses and higher intensity vibrations associated with higher tissue thicknesses. In addition, or in the alternative, vibration patterns and / or intensities may be preselected or user adjustable.

[0080] FIG. 17 shows yet another force feedback mechanism (530) that may be incorporated into one or more of input control devices (36) of Surgeon’s console (16) either alone or in combination with other force feedback mechanisms (510, 520) described herein. Force feedback mechanism (520) of the present example is generally configured to alter the resistance pattern across one or more control dimensions in input control device (36) in response to tissue compression detected by a sensor such as jaw force sensor (146) described above with respect to surgical instrument (110). In other words, force feedback mechanism (530) is configured to provide tactile feedback with respect to the grip of tissueby one or more elements of a surgical instrument such as jaws (142, 144) of surgical instrument (110).

[0081] The particular resistance provided by force feedback mechanism (530) may be varied in accordance with one or more predetermined force thresholds related to detected tissue compression. For instance, in one example, the particular amount of resistance may depend on one or more force thresholds corresponding to certain tissue thicknesses. In such examples, lower resistance may be provided by force feedback mechanism (530) at force thresholds corresponding to thinner tissue thicknesses. Meanwhile, moderate resistance may be provided by force feedback mechanism (530) at force thresholds corresponding to moderately thick tissue. Finally, high resistance may be provided by force feedback mechanism (530) at force thresholds corresponding to thicker tissue thicknesses. In addition, or in the alternative, resistance patterns and / or intensities may be preselected or user adjustable.

[0082] C. Illustrative Auditory Force Feedback Mechanisms

[0083] Although not shown, it should be understood that in some examples, one or more auditory force feedback mechanisms may be used. Such auditory force feedback mechanisms may include the emission of one or more tones at variable intensity corresponding to the detection of force or compression applied to tissue by elements of surgical instrument. Like force feedback indicators (310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422) and force feedback mechanisms (510, 520, 530) described above, such forces may be detected by one or more sensors such as wrist force sensor (116) and / or jaw force sensor (146). Similarly, auditory force feedback mechanisms may be responsive to one or more predetermined force or compression thresholds to indicate different levels of force, compression, and / or tissue thickness. Optionally, such auditory force feedback mechanisms may be used in combination with any one or more of force feedback indicators (310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422) or force feedback mechanisms (510, 520, 530) described above. In such examples, the tone or tones of auditory force feedback mechanisms may correspond to one or more aspects of force feedback indicators(310, 320, 322, 330, 332, 340, 342, 410, 412, 420, 422) and / or force feedback mechanisms (510, 520, 530).

[0084] III. Examples of Combinations

[0085] The following examples relate to various non -exhaustive ways in which the teachings herein may be combined or applied. It should be understood that the following examples are not intended to restrict the coverage of any claims that may be presented at any time in this application or in subsequent filings of this application. No disclaimer is intended. The following examples are being provided for nothing more than merely illustrative purposes. It is contemplated that the various teachings herein may be arranged and applied in numerous other ways. It is also contemplated that some variations may omit certain features referred to in the below examples. Therefore, none of the aspects or features referred to below should be deemed critical unless otherwise explicitly indicated as such at a later date by the inventors or by a successor in interest to the inventors. If any claims are presented in this application or in subsequent fdings related to this application that include additional features beyond those referred to below, those additional features shall not be presumed to have been added for any reason relating to patentability.

[0086] Example 1

[0087] A console for use in a robotic surgical system, comprising: one or more input control devices, each input control device being configured to control manipulation of a respective surgical instrument; a viewer, the viewer being configured to generate an endoscopic view showing a distal portion of the surgical instrument relative to tissue at a surgical site; and a user interface associated with the viewer, the user interface including a force feedback indicator, the force feedback indicator being configured to provide an indication of a force being applied to the tissue via the surgical instrument, the indication of the force corresponding to an actual force applied to a tissue by a portion of the surgical instrument.

[0088] Example 2

[0089] The console of Example 1, the force feedback indicator of the user interface being superimposed on the endoscopic view of the viewer.

[0090] Example 3

[0091] The console of Example 2, the force feedback indicator of the user interface being superimposed on a portion of the tissue.

[0092] Example 4

[0093] The console of Example 2, the force feedback indicator of the user interface being superimposed on a portion of the surgical instrument.

[0094] Example 5

[0095] The console of any of Examples 1 through 4, a portion of the force feedback indicator being configured to change opacity in proportion to the actual force applied to the tissue.

[0096] Example 6

[0097] The console of any of Examples 1 through 5, a portion of the force feedback indicator being configured to change color between a plurality of predetermined colors, each color of the plurality of predetermined colors corresponding to a respective threshold force value associated with the actional force applied to tissue.

[0098] Example 7

[0099] The console of Example 6, the plurality of predetermined colors including a red color, a portion of the force feedback indicator being configured to change color being configured to emit the red color, the red color corresponding to the actual force applied to the tissue exceeding a predetermined high force value.[000100] Example 8[000101] The console of any of Examples 1 through 7, the force feedback indicator of the user interface being adjacent to the endoscopic view of the viewer.[000102] Example 9[000103] The console of Example 8, the force feedback indicator including a movable bar or arm, the movable bar and arm being configured to move relative to another portion of the force feedback indicator in proportion to the actual force applied to the tissue.[000104] Example 10[000105] The console of any of Examples 1 through 9, a portion of the one or more input control devices being configured to vibrate to generate a force feedback mechanism, the force feedback mechanism including one or more predetermined vibration patterns.[000106] Example 11[000107] The console of Example 10, each predetermined vibration pattern corresponding to the actual force applied to the tissue exceeding a respective predetermined value.[000108] Example 12[000109] The console of any of Examples 1 through 9, each input control device including a first portion and a second portion, the first portion being configured to vibrate to generate a first force feedback mechanism, the second portion being configured to vibrate to generate a second force feedback mechanism, the first force feedback mechanism being different from the second force feedback mechanism.[000110] Example 13[000111] The console of Example 12, the first force feedback mechanism corresponding to a lift force applied to the tissue, the second force feedback mechanism corresponding to a compressive force applied to the tissue.[000112] Example 14[000113] The console of any of Example 1 through 13, the viewer including a left eye viewer and a right eye viewer, the force feedback indicator being superimposed on at least a portion of both the left eye viewer and the right eye viewer.[000114] Example 15[000115] The console of any of Examples 1 through 14, the indication of force corresponding to a compressive force or a lift force associated with the tissue.[000116] Example 16[000117] A surgical system for performing a surgical procedure by a user comprising: a processor; a surgical instrument in communication with the processor and configured to execute a set of operations on a tissue based on instructions from the processor to apply a lift force or a compressive force to the tissue; a surgeon’s console in communication with the processor, comprising: a viewer configured to present an endoscopic view of a surgical site that includes a portion of the surgical instrument, and an input control device, a first portion or a second portion of the input control device being configured to vibrate to provide force feedback mechanism corresponding to the lift force or the compressive force applied to the tissue by the surgical instrument.[000118] Example 17[000119] The surgical system of Example 16, the first portion or the second portion of the input control device being configured to vibrate in a plurality of predetermined vibration patterns, each predetermined vibration pattern of the plurality of predetermined vibration patterns corresponding to a respective threshold value associated with the lift force or the compressive force applied to the tissue by the surgical instrument.[000120] Example 18[000121] The surgical system of Example 16, the first portion of the input control device being configured to vibrate to provide a first force feedback mechanism, the second portion of the input control device being configured to provide a second force feedback mechanism, the first force feedback mechanism being indicative of an attribute of the left force applied to the tissue by the surgical instrument, the second force feedback mechanism being indicative of an attribute of the compressive fore applied to the tissue by the surgical instrument.[000122] Example 19[000123] A method for force feedback during a robotic surgical procedure, the method comprising: manipulating a surgical instrument to apply a first force to a tissue; viewing the manipulation of the surgical instrument via an endoscopic view provided by a viewer; indicating the first force applied to the tissue using a force feedback indicator superimposed on a portion of the viewer in association with the endoscopic view; and further manipulating the surgical instrument to apply a second force to the tissue, the second force being different from the first force; indicating the second force applied to the tissue by changing one or more characteristics the force feedback indicator.[000124] Example 20[000125] The method of Example 19, the step of indicating the second force applied to the tissue including changing an opacity or a color associated with the force feedback indicator.[000126] IV. Miscellaneous[000127] It should be appreciated that any patent, publication, or other disclosure material, in whole or in part, that is said to be incorporated by reference herein is incorporated herein only to the extent that the incorporated material does not conflict with existing definitions, statements, or other disclosure material set forth in this disclosure. As such, and to the extent necessary, the disclosure as explicitly set forth herein supersedes any conflicting material incorporated herein by reference. Any material, or portion thereof, that is said to be incorporated by reference herein, but which conflicts with existing definitions, statements, or other disclosure material set forth herein will only be incorporated to the extent that no conflict arises between that incorporated material and the existing disclosure material.[000128] Versions described above may be designed to be disposed of after a single use, or they can be designed to be used multiple times. Versions may, in either or both cases, be reconditioned for reuse after at least one use. Reconditioning may include any combination of the steps of disassembly of the systems, instruments, and / or portions thereof, followed by cleaning or replacement of particular pieces, and subsequent reassembly. In particular, some versions of the systems, instruments, and / or portions thereof may be disassembled, and any number of the particular pieces or parts of the systems, instruments, and / or portionsthereof may be selectively replaced or removed in any combination. Upon cleaning and / or replacement of particular parts, some versions of the systems, instruments, and / or portions thereof may be reassembled for subsequent use either at a reconditioning facility, or by an operator immediately prior to a procedure. Those skilled in the art will appreciate that reconditioning of systems, instruments, and / or portions thereof may utilize a variety of techniques for disassembly, cleaning / replacement, and reassembly. Use of such techniques, and the resulting reconditioned systems, instruments, and / or portions thereof, are all within the scope of the present application.[000129] By way of example only, versions described herein may be sterilized before and / or after a procedure. In one sterilization technique, the systems, instruments, and / or portions thereof is placed in a closed and sealed container, such as a plastic or TYVEK bag. The container and system, instrument, and / or portion thereof may then be placed in a field of radiation that can penetrate the container, such as gamma radiation, x-rays, or high-energy electrons. The radiation may kill bacteria on the system, instrument, and / or portion thereof and in the container. The sterilized systems, instruments, and / or portions thereof may then be stored in the sterile container for later use. Systems, instruments, and / or portions thereof may also be sterilized using any other technique known in the art, including but not limited to beta or gamma radiation, ethylene oxide, or steam.[000130] Having shown and described various embodiments of the present invention, further adaptations of the methods and systems described herein may be accomplished by appropriate modifications by one of ordinary skill in the art without departing from the scope of the present invention. Several of such potential modifications have been mentioned, and others will be apparent to those skilled in the art. For instance, the examples, embodiments, geometries, materials, dimensions, ratios, steps, and the like discussed above are illustrative and are not required. Accordingly, the scope of the present invention should be considered in terms of the following claims and is understood not to be limited to the details of structure and operation shown and described in the specification and drawings.

Claims

I / We claim:

1. A console for use in a robotic surgical system, comprising:(a) one or more input control devices, each input control device being configured to control manipulation of a respective surgical instrument;(b) a viewer, the viewer being configured to generate an endoscopic view showing a distal portion of the surgical instrument relative to tissue at a surgical site; and(c) a user interface associated with the viewer, the user interface including a force feedback indicator, the force feedback indicator being configured to provide an indication of a force being applied to the tissue via the surgical instrument, the indication of the force corresponding to an actual force applied to a tissue by a portion of the surgical instrument.

2. The console of claim 1, the force feedback indicator of the user interface being superimposed on the endoscopic view of the viewer.

3. The console of claim 1 or claim 2, the force feedback indicator of the user interface being superimposed on a portion of the tissue.

4. The console of any one of claims 1 to 3, the force feedback indicator of the user interface being superimposed on a portion of the surgical instrument.

5. The console of any preceding claim, a portion of the force feedback indicator being configured to change opacity in proportion to the actual force applied to the tissue.

6. The console of any preceding claim, a portion of the force feedback indicator being configured to change color between a plurality of predetermined colors, each color of the plurality of predetermined colors corresponding to a respective threshold force value associated with the actional force applied to tissue.

7. The console of claim 6, the plurality of predetermined colors including a red color, a portion of the force feedback indicator being configured to change color being configured to emit the red color, the red color corresponding to the actual force applied to the tissue exceeding a predetermined high force value.

8. The console of any preceding claim, the force feedback indicator of the user interface being adjacent to the endoscopic view of the viewer.

9. The console of any preceding claim, the force feedback indicator including a movable bar or arm, the movable bar and arm being configured to move relative to another portion of the force feedback indicator in proportion to the actual force applied to the tissue.

10. The console of any of any preceding claim, a portion of the one or more input control devices being configured to vibrate to generate a force feedback mechanism, the force feedback mechanism including one or more predetermined vibration patterns.

11. The console of claim 10, each predetermined vibration pattern corresponding to the actual force applied to the tissue exceeding a respective predetermined value.

12. The console of any preceding claim, each input control device including a first portion and a second portion, the first portion being configured to vibrate to generate a first force feedback mechanism, the second portion being configured to vibrate to generate a second force feedback mechanism, the first force feedback mechanism being different from the second force feedback mechanism.

13. The console of claim 12, the first force feedback mechanism corresponding to a lift force applied to the tissue, the second force feedback mechanism corresponding to a compressive force applied to the tissue.

14. The console of any preceding claim, the viewer including a left eye viewer and a right eye viewer, the force feedback indicator being superimposed on at least a portion of both the left eye viewer and the right eye viewer.

15. The console of any preceding claim, the indication of force corresponding to a compressive force or a lift force associated with the tissue.

16. A surgical system for performing a surgical procedure by a user comprising:(a) a processor;(b) a surgical instrument in communication with the processor and configured to execute a set of operations on a tissue based on instructions from the processor to apply a lift force or a compressive force to the tissue;(c) a surgeon’s console in communication with the processor, comprising:(i) a viewer configured to present an endoscopic view of a surgical site that includes a portion of the surgical instrument, and(ii) an input control device, a first portion or a second portion of the input control device being configured to vibrate to provide force feedback mechanism corresponding to the lift force or the compressive force applied to the tissue by the surgical instrument.

17. The surgical system of claim 16, the first portion or the second portion of the input control device being configured to vibrate in a plurality of predetermined vibration patterns, each predetermined vibration pattern of the plurality of predetermined vibration patterns corresponding to a respective threshold value associated with the lift force or the compressive force applied to the tissue by the surgical instrument.

18. The surgical system of claim 16 or claim 17, the first portion of the input control device being configured to vibrate to provide a first force feedback mechanism, the second portion of the input control device being configured to provide a second force feedback mechanism, the first force feedback mechanism being indicative of an attribute of the left force applied to the tissueby the surgical instrument, the second force feedback mechanism being indicative of an attribute of the compressive fore applied to the tissue by the surgical instrument.

19. A method for force feedback during a robotic surgical procedure, the method comprising:(a) manipulating a surgical instrument to apply a first force to a tissue;(b) viewing the manipulation of the surgical instrument via an endoscopic view provided by a viewer;(c) indicating the first force applied to the tissue using a force feedback indicator superimposed on a portion of the viewer in association with the endoscopic view; and(d) further manipulating the surgical instrument to apply a second force to the tissue, the second force being different from the first force;(e) indicating the second force applied to the tissue by changing one or more characteristics the force feedback indicator.

20. The method of claim 19, the step of indicating the second force applied to the tissue including changing an opacity or a color associated with the force feedback indicator.

Images