Systems and methods for control of a surgical system

Abstract

Systems and methods are provided for control of computer‑assisted system. Accordingly, an output signal is received from a force sensor unit. The output signal corresponds to a sensed force exerted on a distal end portion of a medical instrument of the surgical system. A force vector corresponding to a magnitude and direction of the sensed force is determined. A first haptic output based on the force vector and a second haptic output indicative of a limiting operating condition are concurrently provided.

Description

SYSTEMS AND METHODS FOR CONTROL OF A SURGICAL SYSTEMCross-Reference to Related Applications

[0001] This application claims benefit of priority to U.S. Provisional Application Serial No. 63 / 535,167, entitled “Systems and Methods for Control of a Surgical System,” filed August 29, 2023, which is incorporated herein by reference in its entirety.Background

[0002] The embodiments described herein relate to surgical systems, and more specifically to teleoperated surgical systems. More particularly, the embodiments described herein relate to systems and methods for conveying information about the surgical system via a haptic feedback system.

[0003] Known techniques for Minimally Invasive Surgery (MIS) employ instruments to manipulate tissue that can be either manually controlled or controlled via hand-held or mechanically grounded teleoperated medical systems that operate with at least partial computer-assistance (“telesurgical systems”). In a mechanically grounded telesurgical system, an instrument used to carry out a surgical action is supported by a kinematic chain. This kinematic chain typically includes a mechanically grounded base (e.g., resting on the floor, or mounted to the ceiling, a wall, or a portion of the operating table), a manipulator support structure that supports a teleoperated manipulator, and the manipulator to which the instrument is coupled.

[0004] Many known MIS instruments include a therapeutic or diagnostic end effector (e.g., forceps, a cutting tool, or a cauterizing tool) mounted on an optional wrist mechanism at the distal end of a shaft. During an MIS procedure, the end effector, wrist mechanism, and the distal end of the shaft are typically inserted into a small incision or a natural orifice of a patient via a cannula to position the end effector at a work site within the patient’s body. The clinician operating the telesurgical system controls one or more input devices so that motion of the input devices results in corresponding motion of the instrument as a whole, the end effector, or a portion of the end effector. The optional wrist mechanism can be used to change the end effector’s position and orientation with reference to the shaft to perform a desired procedure at the work site. In known instruments, motion of the instrument as a whole provides mechanical degrees of freedom (DOFs)for movement of the end effector, and the wrist mechanisms generally provide the desired DOFs for movement of the end effector with reference to the shaft of the instrument. For example, for forceps or other grasping tools, known wrist mechanisms are able to change the pitch and yaw of the end effector with reference to the shaft. A wrist may optionally provide a roll DOF for the end effector, or the roll DOF may be implemented by rolling the shaft. An end effector may optionally have additional mechanical DOFs, such as grip or knife blade motion. In some instances, wrist and end effector mechanical DOFs may be combined. For example, U.S. Patent No. 5,792,135 (filed May 16, 1997) discloses a mechanism in which wrist and end effector grip DOFs are combined.

[0005] Force sensing surgical instruments are known and together with associated telesurgical systems may be used to provide haptic feedback to a surgeon operating the telesurgical system to perform a surgical procedure. The haptic feedback increases the immersion, realism, and intuitiveness of the surgeon’s experience while performing the telesurgical procedure. For effective haptics rendering and accuracy, force sensors to sense forces between a teleoperated surgical instrument and tissue may be placed on the instrument at various locations, such as close to the anatomical tissue interaction (i.e., near the instrument’s distal end, where the end effector is located). One representative design approach is to include a force sensor unit having electrical sensor elements (e.g., strain sensors or strain gauges) at the distal end of a medical instrument shaft to measure strain imparted to the medical instrument by the end effector’s tissue interactions. The measured strain is used to determine the force imparted to the medical instrument and so as input upon which the desired haptic feedback to the operator is generated.

[0006] The use of telesurgical systems provides both advantages and challenges to the personnel operating them. Many of these medical devices may be capable of autonomous or semiautonomous motion of one or more repositionable arms and / or end effectors. It is also common to operate the medical devices via teleoperation using one or more input controls on an operator workstation to control the motion and / or operation of the repositionable arms and / or the end effectors. When the medical device is operated remotely from the operator workstation and / or the end effectors are being used in an area not directly visible to the operator, such as during computer-assisted surgery when the end effectors are hidden by patient anatomy, surgical drapes,and / or the like, it may complicate the operator's ability to detect and / or avoid collisions between the one or more repositionable arms that could result in damage to the medical device, injury to a patient or other personnel, and / or failures in a sterile field.

[0007] In view of the aforementioned, the art is continuously seeking new and improved systems and methods for control of a surgical system.Summary

[0008] This summary introduces certain aspects of the embodiments described herein to provide a basic understanding. This summary is not an extensive overview of the inventive subject matter, and it is not intended to identify key or critical elements or to delineate the scope of the inventive subject matter.

[0009] The systems and methods described herein facilitate the control and operation of a computer-assisted system, such as an MIS system. In particular, the systems and methods described herein utilize a haptic feedback system to provide both (i) information about the mechanical load affecting a instrument, and (ii) an operating condition of the instrument or the computer-assisted system. For example, a first haptic output to the surgeon can be based on a force sensor unit output that corresponds to a sensed force exerted on a distal end portion of the instrument. The first haptic output simulates the load that the surgeon would feel if the surgeon were holding the instrument in their hand. In other words, the first haptic output can be considered to be a haptic feedback configured to deliver a representation of the forces affecting the instrument. Thus, the first haptic output increases the immersion, realism, and intuitiveness of the surgeon’s experience while performing a telesurgical procedure. In contrast and in addition to the first haptic output, a second haptic output to the surgeon is indicative of a system limit or other condition that may affect an operation of the surgical system, such as the presence of a limiting operating condition (e.g., a mechanical range-of-motion (ROM) limit, a collision between an instrument and a non-anatomical object, or a system operating fault condition). In other words, the second haptic output can be considered to be a haptic feedback configured to deliver system operating information to the surgeon rather than a representation or simulation of the mechanical load sensed by the force sensor unit. Both haptic outputs can optionally be generated concurrently via the samehaptic feedback system, and so the individual haptic outputs are formed to provide different sensations to the surgeon so that the surgeon can distinguish the information conveyed by each of the first and the second haptic outputs. The haptic outputs can be distinguished, for example, based on the characteristics of the signal and the actuators used to generate the second haptic output versus the actuators used to generate the first haptic output.

[0010] In one aspect, the present disclosure is directed to a computer-assisted system that includes a manipulator configured to support an instrument with a distal end portion and a force sensor unit operably coupled thereto. The computer-assisted system also includes an operator input device operably coupled to the manipulator and the instrument. Additionally, the computer-assisted system includes a controller operably coupled to the manipulator, the operator input device, and the force sensor unit. The controller includes at least one memory device having stored instructions and at least one processor operably coupled to execute the instructions. Executing the instructions includes receiving an output signal from the force sensor unit. The output signal corresponds to a sensed force exerted on the distal end portion of the instrument. The actions also include determining a force vector corresponding to a magnitude and direction of the sensed force. Additionally, the actions include concurrently providing a first haptic output and a second haptic output at the operator input device, the first haptic output being based on the force vector, the second haptic output being indicative of a limiting operating condition of the manipulator, the instrument, or both the manipulator and the instrument, and the second haptic output being different from the first haptic output.

[0011] In some embodiments, the second haptic output is based on a modeled parameter associated with the limiting operating condition. The modeled parameter is different than the sensed force exerted on the distal end portion of the instrument as indicated by the output signal of the force sensor unit.

[0012] In some embodiments, the first haptic output conveys a simulated sensation of the sensed force to a handle portion of the operator input device. The second haptic output conveys information about the limiting operating condition to the operator holding the handle portion of the operator input device without simulating a sensation of the limiting operating condition.

[0013] In some embodiments, the first haptic output is indicative of a force at the distal end portion of the instrument at a surgical site, and the second haptic output is indicative of an approach of the manipulator, the instrument, or both the manipulator and the instrument to the limiting operating condition.

[0014] In some embodiments, the second haptic output has a directional component indicative of a direction associated with the limiting operating condition relative to a position of the manipulator, the instrument, or both the manipulator and the instrument.

[0015] In some embodiments, the operator input device includes at least a first actuator and a second actuator. The first actuator is configured to produce the first haptic output. The second actuator is configured to produce the second haptic output.

[0016] In some embodiments, the operator input device includes a gimbal assembly movably coupled to a support-arm assembly. The support-arm assembly includes a first support segment and a second support segment movably coupled between a base portion of the support-arm assembly and the gimbal assembly. The first actuator is a gimbal actuator operably coupled to a handle portion of the gimbal assembly and configured to exert a torque on the handle portion to produce the first haptic output. The second actuator is a support actuator operably coupled to the support-arm assembly and configured to exert a torque on the support arm assembly to produce the second haptic output.

[0017] In some embodiments, the operator input device includes a first actuator and a second actuator. The first actuator is associated with a motion control relationship between the operator input device and the manipulator, the instrument, or both the manipulator and the instrument. The second actuator is dedicated to providing the second haptic output.

[0018] In some embodiments, an available range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together, and the limiting operating condition is defined at least in part by a non-anatomical object affecting the available range-of-motion volume.

[0019] In some embodiments, the computer-assisted system includes a sensor positioned to sense a position, an orientation, or both a position and orientation of the object. The sensor generates kinematic information indicative of the position, the orientation, or both the position and orientation of the object. The second haptic output is based at least in part on the kinematic information.

[0020] In some embodiments, the limiting operating condition is a design range-of-motion limit of the manipulator, the instrument, or both the manipulator and the instrument.

[0021] In some embodiments, the manipulator includes at least two segments coupled by a joint. The computer-assisted system includes a joint encoder positioned to output a sensor signal indicative of a rotational orientation of the joint. Determining the limiting operating condition includes determining a pose of the manipulator and the instrument based on the sensor signal from the joint encoder.

[0022] In some embodiments, the limiting operating condition is a design range-of-motion limit of the operator input device.

[0023] In some embodiments, a range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together. The limiting operating condition is defined at least in part by an anatomical object affecting the range-of-motion volume.

[0024] In some embodiments, the limiting operating condition is a dynamic range-of-motion limit. The dynamic range-of-motion limit corresponds to an available range of motion at a given instant. The manipulator includes at least two segments coupled by a joint. The instructions include determining the available range of motion based on poses of each of the segments and the instrument at the given instant.

[0025] In some embodiments, executing the instructions includes determining that a collision is occurring between an object and the manipulator, the instrument, or both the manipulator and the instrument. The limiting operating condition is the collision.

[0026] In some embodiments, the limiting operating condition is at least in part a distance being less than a defined distance between an object and the manipulator, the instrument, or both the manipulator and the instrument.

[0027] In some embodiments, the limiting operating condition is at least in part a velocity being larger than a defined velocity between an object and the manipulator, the instrument, or both the manipulator and the instrument.

[0028] In some embodiments, the first haptic output has a first output profile, the second haptic output has a second output profile, and the first output profile is different than the second output profile.

[0029] In some embodiments, the second output profile replicates a profile associated with a collision. The second output profile includes a spike portion followed by a vibratory portion. The spike portion has a duration and a maximum magnitude. The vibratory portion also has a duration and a maximum magnitude. In an example, the duration of the spike portion is ten percent or less of the duration of the vibratory portion. In another example, the maximum magnitude of the spike portion is at least two times the maximum magnitude of the vibratory portion. The relative durations and magnitudes of the spike and vibratory portions are optionally combined in the second output profile.

[0030] In some embodiments, the second output profile is a vibratory signal profile, and the vibratory signal profile oscillates at a constant frequency between a first fixed magnitude and a second fixed magnitude.

[0031] In some embodiments, the second output profile is a vibratory signal profile. The vibratory signal profile oscillates at a constant frequency. The vibratory signal profile has a variable magnitude. The variable magnitude increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

[0032] In some embodiments, the second output profile is a vibratory signal profile. The vibratory signal profile has a constant magnitude. The vibratory signal profile has a variablefrequency. The variable frequency increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

[0033] In some embodiments, the second output profde includes a square wave form.

[0034] In some embodiments, on the condition in which the limiting operating condition exists, a gain magnitude of the first haptic output is reduced.

[0035] In some embodiments, the limiting operating condition is a modeled position of the manipulator, the instrument, or both the manipulator and the instrument.

[0036] In some embodiments, the manipulator includes a motor. The limiting operating condition is a load on the motor.

[0037] In some embodiments, the limiting operating condition is a position of the manipulator, the instrument, or both the manipulator and the instrument based on a signal from an image-based motion tracking system.

[0038] In some embodiments, the computer-assisted system includes a second manipulator, a second instrument supported by the second manipulator, and a second operator input device operatively coupled to the second manipulator and the second instrument. Additionally, executing the instructions on the condition in which the limiting operating condition exists includes providing a third haptic output at the second operator input device. The third haptic output is the same as the second haptic output.

[0039] In one aspect, the present disclosure is directed to a method of control for a computer-assisted system. The computer-assisted system includes a manipulator, a instrument supported by the manipulator and including a distal end portion, a force sensor unit operably coupled to the instrument, an operator input device operably coupled to the manipulator and the instrument. The computer-assisted system also includes a controller operably coupled to the manipulator, the operator input device, and the force sensor unit. The method includes receiving, via the controller, an output signal from the force sensor unit, the output signal corresponding to a sensed force exerted on the distal end portion of the instrument. The controller also determines aforce vector corresponding to a magnitude and direction of the sensed force. Additionally, the method includes determining, via the controller, a limiting operating condition of the manipulator, the instrument, or both the manipulator and the instrument. On a condition in which the limiting operating condition exists, the method includes concurrently providing a first haptic output and a second haptic output at the operator input device, the first haptic output being based on the force vector, and the second haptic output being indicative of the limiting operating condition and different from the first haptic output. Further, the method can include any of the instructions, features, procedures, and / or operations described herein.

[0040] These and other features, aspects and advantages of the present invention will become better understood with reference to the following description and drawings.Brief Description of the Drawings

[0041] FIG. l is a plan view of a computer-assisted system configured as a minimally invasive teleoperated medical system according to an embodiment being used to perform a medical procedure such as surgery.

[0042] FIG. 2 is a perspective view of a user control console of the minimally invasive teleoperated surgery system shown in FIG. 1.

[0043] FIG. 3 is a side view of an operator input device of the user control console shown in FIG. 2.

[0044] FIG. 4 is a perspective view of a gimbal assembly of the operator input device shown in FIG. 3.

[0045] FIG. 5 is a perspective view of an optional auxiliary unit of the minimally invasive teleoperated surgery system shown in FIG. 1.

[0046] FIG. 6 is a front view of a manipulator, including a plurality of instruments, of the minimally invasive teleoperated surgery system shown in FIG. 1.

[0047] FIG. 7 is a perspective view of an arm assembly of the manipulator shown in FIG. 6.

[0048] FIG. 8 is a perspective view of an instrument according to an embodiment.

[0049] FIG. 9 is a side view of a portion of the instrument of FIG. 8 with an outer shaft removed.

[0050] FIG. 10 is a flow chart of a set of instructions for control of a surgical system.

[0051] FIG. 11 is a perspective view of a portion of the of the minimally invasive teleoperated surgery system shown in FIG. 1 positioned in a range-of-motion volume according to an embodiment.

[0052] FIG. 12 is a perspective view of a portion of the of the minimally invasive teleoperated surgery system shown in FIG. 1 positioned in a range-of-motion volume according to an embodiment.

[0053] FIGS. 13 A and 13B are graphical depictions of output profiles of the haptic outputs, with FIG. 13A corresponding to a first output profile of the first haptic output and FIG. 13B corresponding to a concurrently delivered second output profile of the second haptic output according to an embodiment.

[0054] FIG. 14 is a graphical depiction of a second output profile relative to a position of a portion of the minimally invasive teleoperated surgery system shown in FIG. 1 according to an embodiment.

[0055] FIG. 15 is a graphical depiction of a second output profile relative to a position of a portion of the minimally invasive teleoperated surgery system shown in FIG. 1 according to an embodiment.

[0056] FIG. 16 is a graphical depiction of a second output profile corresponding to a collision event according to an embodiment.

[0057] FIG. 17 is a graphical depiction of a second output profile of the second haptic output according to an embodiment.

[0058] FIG. 18 is a graphical depiction of a second output profile of the second haptic output according to an embodiment.

[0059] FIG. 19 is a graphical depiction of a second output profile of the second haptic output according to an embodiment.

[0060] FIG. 20 is a graphical depiction of a second output profile of the second haptic output according to an embodiment.

[0061] FIG. 21 is a schematic illustration of a controller for use with a computer-assisted system according to an embodiment.

[0062] FIG. 22 is a flow chart of a method of control for a computer-assisted system according to an embodiment.Detailed Description

[0063] Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.

[0064] The embodiments described herein can advantageously be used in a wide variety of grasping, cutting, and manipulating operations associated with minimally invasive surgery. The medical instruments or devices of the present application enable motion in three or more degrees of freedom (DOFs). For example, in some embodiments, an end effector of the medical instrument can move with reference to the main body of the instrument in three mechanical DOFs, e.g., pitch, yaw, and roll (shaft roll). There may also be one or more mechanical DOFs in the end effector itself, e.g., two jaws, each rotating with reference to a clevis (2 DOFs) and a distal clevis that may rotate with reference to a proximal clevis (one DOF). Thus, in some embodiments, the medicalinstruments or devices of the present application may enable motion in six DOFs. The embodiments described herein further may be used to deliver haptic feedback to a system operator based on a load indication from the force sensor unit.

[0065] Generally, the present disclosure is directed to systems and methods for controlling a surgical system such as a minimally invasive teleoperated surgery system. In particular, the present disclosure includes a system and methods that use various channels of kinesthetic force feedback to concurrently provide the user with indications of sensed forces affecting the instrument and with indications that correspond to various operating conditions and / or limits of the surgical system. Accordingly, the systems and methods described herein deliver to the operator a first haptic output that is based on the sensed forces affecting the instrument and a second haptic output that is indicative of a limiting operating condition, assistive feedback, instructional feedback, and / or guiding feedback provided by the system. More specifically, the systems and methods described herein facilitate the differentiation between the concurrently provided first and second haptic outputs. By differentiating the channels of the kinesthetic force feedback provided to the user, the user can differentiate the feedback channels and avoid confusion as to the source and / or cause of the feedback.

[0066] By way of a nonlimiting illustration, the surgical system can utilize multiple channels of kinesthetic force feedback to provide the user with haptic feedback based on sensed forces, indications of an approaching range-of-motion limitation, an error condition, a collision event, assistive feedback, instructional feedback, and / or guiding feedback. Should the channels of kinesthetic force feedback be rendered similarly to the user, the user can be confused regarding the source of the feedback. This confusion is potentially problematic as the appropriate response of the user to the force feedback can differ depending on the source of the feedback. Therefore, it is desirable to structure the feedback in such a manner that the distinction between sources is intuitive to the operator. In other words, the systems and methods described herein provide different sensations to the operator depending on the source of the feedback. The channels of the kinesthetic force feedback (e.g., the haptic outputs) can be distinguished based on the characteristics of the signal and / or by the actuators used to generate the different channels.

[0067] By way of an additional nonlimiting illustration, a representation of the sensed load on the medical instrument (e.g., haptic feedback) can be conveyed to an operator by applying forces and torques on the user’s hands via a control unit of the surgical system. The representation of the sensed load increases the immersion, realism, and intuitiveness of the operators experience while performing the procedure. Concurrently, a second haptic output (e.g., a haptic signal) can be used to indicate the presence of a range-of-motion limit. To differentiate the range-of-motion limit from the representation of the sensed load a vibrotactile feedback can be overlaid onto the representation of the sensed load (e.g., an existing force feedback). A vibrotactile signature of the second haptic output can be rendered using different actuators than those employed to render the representation of the sensed load. Additionally, a carrier frequency can be chosen and amplitude modulated to provide a sensation of pulses. The number of pulses, timing between the pulses, and / or the relative amplitude of pulses are configurable to generate a set of parameters that are intuitively distinguishable from the representation of the sensed load. The overall second haptic output can also be scaled up or down based on a distance until the range-of-motion is reached, with some saturation on the scaling. The pulsating vibrotactile feedback of the second haptic output is meant to inform the operator of the presence of approach to the range-of-motion limit.

[0068] As used herein, the term “about” when used in connection with a referenced numeric indication means the referenced numeric indication plus or minus up to 10 percent of that referenced numeric indication. For example, the language “about 50” covers the range of 45 to 55. Similarly, the language “about 5” covers the range of 4.5 to 5.5.

[0069] As used in this specification and the appended claims, the word “distal” refers to direction towards a work site, and the word “proximal” refers to a direction away from the work site. Thus, for example, the end of a tool that is closest to the target tissue would be the distal end of the tool, and the end opposite the distal end (i.e., the end manipulated by the user or coupled to the actuation shaft) would be the proximal end of the tool.

[0070] Further, specific words chosen to describe one or more embodiments and optional elements or features are not intended to limit the invention. For example, spatially relative terms — such as “beneath”, “below”, “lower”, “above”, “upper”, “proximal”, “distal”, and the like — may be used to describe the relationship of one element or feature to another element or feature asillustrated in the figures. These spatially relative terms are intended to encompass different positions (i.e., translational placements) and orientations (i.e., rotational placements) of a device in use or operation in addition to the position and orientation shown in the figures. For example, if a device in the figures is turned over, elements described as “below” or “beneath” other elements or features would then be “above” or “over” the other elements or features. Thus, the term “below” can encompass both positions and orientations of above and below. A device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. Likewise, descriptions of movement along (translation) and around (rotation) various axes includes various spatial device positions and orientations. The combination of a body’s position and orientation define the body’s pose (e.g., a kinematic pose).

[0071] Similarly, geometric terms, such as “parallel”, “perpendicular”, “round”, or “square”, are not intended to require absolute mathematical precision, unless the context indicates otherwise. Instead, such geometric terms allow for variations due to manufacturing or equivalent functions. For example, if an element is described as “round” or “generally round,” a component that is not precisely circular (e.g., one that is slightly oblong or is a many-sided polygon) is still encompassed by this description.

[0072] In addition, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context indicates otherwise. The terms “comprises”, “includes”, “has”, and the like specify the presence of stated features, steps, operations, elements, components, etc. but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, or groups.

[0073] Unless indicated otherwise, the terms apparatus, medical device, instrument, and variants thereof, can be interchangeably used.

[0074] Inventive aspects are described with reference to a computer-assisted system (e.g., a teleoperated surgical system). An example architecture of such a teleoperated surgical system is the da Vinci® surgical system commercialized by Intuitive Surgical, Inc., Sunnyvale, California. Knowledgeable persons will understand, however, that inventive aspects disclosed herein may be embodied and implemented in various ways, including computer-assisted, non-computer-assisted, and hybrid combinations of manual and computer-assisted embodiments and implementations.Implementations are merely presented as examples, and they are not to be considered as limiting the scope of the inventive aspects disclosed herein. As applicable, inventive aspects may be embodied and implemented in both relatively smaller, hand-held, hand-operated devices and relatively larger systems that have additional mechanical support.

[0075] FIG. 1 is a plan view illustration of a computer-assisted system (“system”)1000 that operates with at least partial computer assistance. The system 1000 can, for example, be configured as a teleoperated surgical system (e.g., a “telesurgical system”). When configured as a telesurgical system, the system 1000 and its components can be considered medical devices. In some embodiments, the system 1000 is a Minimally Invasive Robotic Surgical (MIRS) system used for performing a minimally invasive diagnostic or surgical procedure on a patient P who is lying on an operating table 1010. The system can have any number of components, such as a user control unit 1100 for use by an operator of the system, such as a surgeon or other skilled clinician S (e.g., an operator), during the procedure. The system 1000 can further include a manipulator unit 1200 (popularly referred to as a surgical robot) and an optional auxiliary equipment unit 1150. The manipulator unit 1200 can include an arm assembly 1300 and a surgical instrument tool assembly removably coupled to the arm assembly. The manipulator unit 1200 can manipulate at least one removably coupled medical instrument (instrument) 1400 through a minimally invasive incision in the body or natural orifice of the patient P while the surgeon S views the surgical site and controls movement of the instrument 1400 through control unit 1100. An image of the surgical site is obtained by an endoscope (not shown), such as a stereoscopic endoscope, which can be manipulated by the manipulator unit 1200 to orient the endoscope. The auxiliary equipment unit 1150 can be used to process the images of the surgical site for subsequent display to the surgeon S through the user control unit 1100. The number of instruments 1400 used at one time will generally depend on the diagnostic or surgical procedure and the space constraints within the operating room, among other factors. If it is necessary to change one or more of the instruments 1400 being used during a procedure, an assistant removes the instrument 1400 from the manipulator unit 1200 and replaces it with another instrument 1400 from a tray 1020 in the operating room. Although shown as being used with the instruments 1400, any of the instruments described herein can be used with the system 1000.

[0076] FIG. 2 is a perspective view of the control unit 1100. The user control unit 1100 includes a left eye display 11 12 and a right eye display 11 14 for presenting the surgeon S with a coordinated stereoscopic view of the surgical site that enables depth perception. The user control unit 1100 further includes one or more operator input devices 1116 (input device), which in turn cause the manipulator 1200 (shown in FIG. 6) to manipulate one or more tools. The input devices1116 provide at least the same degrees of freedom as instruments 1400 with which they are associated to provide the surgeon S with telepresence, or the perception that the input devices 1116 are integral with (or are directly connected to) the instruments 1400. In this manner, the user control unit 1100 provides the surgeon S with a strong sense of directly controlling the instruments 1400. To this end, position, force, strain, or tactile feedback (not shown) or any combination of such feedback from the instruments 1400 are provided back to the surgeon's hand or hands through the one or more input devices 1116.

[0077] Referring now to FIGS. 2-4, in some embodiments the operator input device 1116 includes a gimbal assembly 1117 movably coupled to a support-arm assembly 1118, which is, in turn, supported by the control unit 1100. The gimbal assembly 1117 is configured for engagement via a portion of at least one hand of the surgeon S. In such a configuration, the gimbal assembly1117 of the operator input device 1116 can include a first link 1119 (which functions as a first gimbal link), a second link 1120 (which functions as a second gimbal link), a third link 1122 (which functions as a third gimbal link), and an input handle 1124. As depicted, the input handle 1124 includes a handle portion 1128, a first handle input 1130, a second handle input 1132, and a handle input shaft 1134. The handle input shaft 1134 can define a first rotational axis Ai (which may function as a roll axis; the term roll is arbitrary) and may be rotatably coupled to the first link 1119. The handle portion 1128 is supported on the handle input shaft 1134 and is configured to be rotated relative to the first link 1119 about the first rotational axis Ai. The input shaft 1134 extends at least partially within the first link 1119. The first handle input 1130 and the second handle input 1132 can be manipulated to produce a desired action at the end effector (e.g., end effector 1460 as depicted in FIG. 8). For example, in some embodiments, the first handle input 1130 and the second handle input 1132 can be squeezed together to produce a gripping movement at the end effector. The first and second handle inputs 1130, 1132 can be similar to the grip members shown and described in in U.S. Patent Application Pub. No. US 2020 / 0015917 Al (filedJune 14, 2019), entitled “Actuated Grips for Controller,” which is incorporated herein by reference in its entirety for all purposes. In other embodiments, however, the input handle 1 124 need not include the handle inputs.

[0078] The first link 1119 is pivotably coupled to the second link 1120. The coupling between the first link 1119 and the second link 1120 defines a second rotational axis A2 (which may function as a yaw axis; the term yaw is arbitrary). The second rotational axis A2 is perpendicular to the first rotational axis Ai. The second link 1120 is pivotably coupled to the third link 1122. The coupling between the second link 1120 and the third link 1122 defines a third rotational axis A3 (which may function as a pitch axis; the term pitch is arbitrary). The third rotational axis A3 is perpendicular to the first rotational axis Ai and the second rotational axis A2.

[0079] In some embodiments, the gimbal assembly 1117 of the operator input device 1116 includes a first gimbal actuator 1136 operably coupled to the handle input shaft 1134. The first gimbal actuator 1136 is configured to exert a torque on or receive torque from the handle input shaft 1134. Thus, the first gimbal actuator 1136 is configured to exert a torque on or receive a torque from the hand of an operator gripping the handle portion 1128. Said another way, the first gimbal actuator 1136 is positioned to develop or receive a rotation of the handle input shaft 1134 about the first rotation axis Ai. The gimbal assembly 1117 also includes a second gimbal actuator 1138. The second gimbal actuator 1138 is positioned to affect a rotational orientation of the first link 1 119 relative to the second link 1120. In other words, the second gimbal actuator 1 138 is positioned to develop or receive a rotation of the first link 1119 about the second rotational axis A2. As further depicted, the gimbal assembly 1117 also includes a third gimbal actuator 1140. The third gimbal actuator 11 forties position to affect a rotational orientation of the second link 1120 relative to the third link 1122. In other words, the third gimbal actuator 1140 is positioned to develop or receive a rotation of the second link 1120 about the third rotational axis A3. In addition to the gimbal actuators, the gimbal assembly 1117 can also include a suitable number of passive control elements, such as springs and or counterweights, that are positioned to be employed with the gimbal actuators to affect the positioning of the elements of the gimbal assembly 1117 and / or the sensations delivered to the hands of the operator via the handle portion 1128 of the gimbal assembly 1117.

[0080] In some embodiments, the support-arm assembly 1118 of the operator input device 1116 includes a base portion 1141 that is pivotably coupled to a fixed portion of the control unit 1100. A first support actuator 1144 can be positioned to affect a rotational orientation (e.g., a yaw orientation) of the base portion 1141 relative to the fixed portion of the control unit 1100. A change in the rotational orientation of the base portion 1141 can be reflected by a corresponding change in the rotational orientation of the arm assembly 1300 relative to the manipulator 1200. As depicted, the support-arm assembly 1118 also includes a first support segment 1142 pivotably coupled to the base portion 1141. The position (e.g., pitch) of the first support segment 1142 can be established by a second support actuator 1145. A change in the position of the first support segment 1142 relative to the base portion 1141 can be reflected by a corresponding change in the position of the arm assembly 1300 relative to the manipulator 1200. A second support segment 1143 is pivotably coupled to the first support segment 1142. The angle between the first support segment 1142 and the second support segment 1143 (e.g., an elbow position) can be established by a third support actuator 1146. A change in the angle between the first support segment 1142 and the second support segment 1143 can be reflected by a corresponding change in the angle between two pivotably coupled segments of the arm assembly 1300. The position of the arm assembly 1300 relative to the manipulator 1200 and / or the position of the segments of the arm assembly 1300 can affect a position of the end effector 1460 of the medical instrument 1400. Any of the actuators described herein can be an electric motor, a servo, a stepper motor, or other similar electro-mechanical device.

[0081] The user control unit 1100 is shown in FIG. 1 as being in the same room as the patient so that the surgeon S can directly monitor the procedure, be physically present if necessary, and speak to an assistant directly rather than over the telephone or other communication medium. In other embodiments, however, the user control unit 1100 and the surgeon S can be in a different room, a completely different building, or other location remote from the patient, allowing for remote surgical procedures.

[0082] FIG. 5 is a perspective view of the auxiliary equipment unit 1150. The auxiliary equipment unit 1150 can be coupled with the endoscope (not shown) and can include one or more processors to process captured images for subsequent display, such as via the user control unit1100, or on another suitable display located locally (e.g., on the unit 1150 itself as shown, on a wall-mounted display) and / or remotely. For example, where a stereoscopic endoscope is used, the auxiliary equipment unit 1150 can process the captured images to present the surgeon S with coordinated stereo images of the surgical site via the left eye display 1112 and the right eye display 1114. Such coordination can include alignment between the opposing images and can include adjusting the stereo working distance of the stereoscopic endoscope. As another example, image processing can include the use of previously determined camera calibration parameters to compensate for imaging errors of the image capture device, such as optical aberrations.

[0083] FIG. 6 shows a front perspective view of the manipulator 1200. The manipulator 1200 includes the components (e.g., arms, linkages, motors, sensors, and the like) to provide for the manipulation of the instruments 1400 and / or an imaging device (not shown), such as a stereoscopic endoscope, used for the capture of images of the site of the procedure. Specifically, the instruments 1400 and / or the imaging device can be manipulated by teleoperated mechanisms having one or more mechanical joints. Moreover, the instruments 1400 and / or the imaging device are positioned and manipulated through incisions or natural orifices in the patient P in a manner such that a center of motion remote from the manipulator and typically located at a position along the instrument shaft is maintained at the incision or orifice by either kinematic mechanical or software constraints. In this manner, the incision size can be minimized.

[0084] FIG. 7 is a perspective view of the arm assembly 1300 of the manipulator 1200 supporting an instrument 1400. As depicted, the arm assembly 1300 includes a plurality of segments coupled together via a plurality of joints. For example, the arm assembly 1300 includes at least a first segment 1310 and a second segment 1312. The first segment 1310 and the second segment 1312 are movably coupled together via a joint 1320. In some embodiments, the system 1000 includes a joint encoder 1330. The joint encoder 1330 is positioned to output a position signal that is indicative of a rotational orientation of the joint 1320. The rotation of the joint 1320 can, for example, be in response to an operator input provided to the operator input device 1116 (e.g., a movement of the second support segment 1143 relative to the first support segment 1142). In some embodiments, the arm assembly 1300 can include at least one motor 1340 positioned to affect the rotation of the joint 1320.

[0085] Referring now to FIGS. 8 and 9, a perspective view of the instrument 1400 is depicted in FIG. 8, and a side view of a portion of the instrument 1400 with an outer shaft portion removed is depicted in FIG. 9. In some embodiments, the instrument 1400 or any of the components therein are optionally parts of a surgical system that performs surgical procedures, and which can include a manipulator unit, a series of kinematic linkages, a set of cannulas, or the like. The instrument 1400 (and any of the instruments described herein) can be used in any suitable surgical system, such as the system 1000 shown and described above. As shown in FIG. 8, the instrument 1400 includes a proximal mechanical structure 1470, a shaft 1410, a distal end portion 1402, and a set of cables (not shown). The cables function as tension elements that couple the proximal mechanical structure 1470 to the distal end portion 1402. In some embodiments, the distal end portion 1402 includes a distal wrist assembly 1500 and a distal end effector 1460. The instrument 1400 is configured such that movement of one or more of the cables produces movement of the end effector 1460 (e g., pitch, yaw, or grip) about axes of a beam coordinate system BCS.

[0086] Moreover, although the proximal mechanical structure 1470 is shown as including capstans 1472, in other embodiments, a mechanical structure can include one or more linear actuators that produce translation (linear motion) of a portion of the cables. Such proximal mechanical structures can include, for example, a gimbal, a lever, or any other suitable mechanism to directly pull (or release) an end portion of any of the cables. For example, in some embodiments, the proximal mechanical structure 1470 can include any of the proximal mechanical structures or components described in U.S. Patent Application Pub. No. US 2015 / 0047454 Al (filed Aug. 15, 2014), entitled “Lever Actuated Gimbal Plate,” or U.S. Patent No. US 6,817,974 B2 (filed lun. 28, 2001), entitled “Surgical Tool Having Positively Positionable Tendon- Actuated Multi-Disc Wrist Joint,” or U.S. Patent No. US 6,312,435 Bl (filed Oct. 8, 1999), entitled “Surgical Instrument with Extended Reach for Use in Minimally Invasive Surgery,” each of which is incorporated herein by reference in its entirety.

[0087] The shaft 1410 can be any suitable elongated shaft that is coupled to the wrist assembly 1500 and to the proximal mechanical structure 1470. Specifically, the shaft 1410 includes a proximal end 1411 that is coupled to the proximal mechanical structure 1470, and a distal end portion 1412 that is coupled to the wrist assembly 1500 (e.g., a proximal link of the wrist assembly 1500). The shaft 1410 defines a passageway or series of passageways through which the cablesand other components (e.g., electrical wires, ground wires, or the like) can be routed from the proximal mechanical structure 1470 to the wrist assembly 1500. In some embodiments, the shaft 1410 can be formed, at least in part with, for example, an electrically conductive material such as stainless steel. In such embodiments, the shaft may include any of an inner insulative cover or an outer insulative cover. Thus, the shaft 1410 can be a shaft assembly that includes multiple different components. For example, the shaft 1410 can include (or be coupled to) a spacer that provides the desired fluid seals, electrical isolation features, and any other desired components for coupling the wrist assembly 1500 to the shaft 1410. Similarly stated, although the wrist assembly 1500 (and other wrist assemblies or links described herein) are described as being coupled to the shaft 1410, it is understood that any of the wrist assemblies or links described herein can be coupled to the shaft via any suitable intermediate structure, such as a spacer and a cable guide, or the like.

[0088] As depicted in FIG. 9, the instrument 1400 (e.g., the surgical or medical instrument) includes a force sensor unit 1800 including a beam 1810, with one or more strain sensors 1830. The strain sensor 1830 can include a set of strain gauges (e.g., tension strain gauge resistor(s) or compression strain gauge resistor(s)) arranged as at least one bridge circuit (e.g., Wheatstone bridges) mounted on a surface along the beam 1810. In some embodiments, the end effector 1460 can be coupled at a distal end portion 1815 of the beam 1810 (e.g., at a distal end portion 1402 of the instrument 1400) via the wrist assembly 1500. The shaft 1410 includes a distal end portion 1412 that is coupled to a proximal end portion 1813 of the beam 1810. In some embodiments, the distal end portion 1412 of the shaft 1410 is coupled to the proximal end portion 1813 of the beam 1810 via another coupling component (such as an anchor or coupler, not shown). In some embodiments, the force sensor unit 1800 can include any of the structures or components described in U.S. Patent Application Pub. No. US 2020 / 0278265 Al (filed May. 13, 2020), entitled “Split Bridge Circuit Force Sensor,” which is incorporated herein by reference in its entirety.

[0089] In some embodiments, the end effector 1460 can include at least one tool member 1462 having a contact portion configured to engage or manipulate a target tissue during a surgical procedure. For example, in some embodiments, the contact portion can include an engagement surface that functions as a gripper, cutter, tissue manipulator, or the like. In other embodiments, the contact portion can be an energized tool member that is used for cauterization or electrosurgical procedures. The end effector 1460 may be operatively coupled to the proximal mechanicalstructure 1470 such that the tool member 1462 rotates relative to shaft 1410. In this manner, the contact portion of the tool member 1462 can be actuated to engage or manipulate a target tissue during a surgical procedure. The tool member 1462 (or any of the tool members described herein) can be any suitable medical tool member. Moreover, although only one tool member 1462 is identified, as shown, the instrument 1400 can include two tool members that cooperatively perform gripping or shearing functions. In other embodiments, an end effector can include more than two tool members.

[0090] Referring now to FIGS. 10-22, in some embodiments, the system 1000, as described above, is configured to deliver a first haptic output (e.g., haptic feedback or tactile feedback) indicative of a sensed force affecting the instrument 1400 and a second haptic output (e.g., a haptic signal or haptic communication), which conveys information concerning an operating condition of the system 1000 to the surgeon S. Accordingly, the system 1000 includes the medical instrument 1400 supported by the manipulator 1200. The medical instrument 1400 includes the distal end portion 1402, and the force sensor unit 1800 is operably coupled thereto. As previously described, the operator input device 1116 is operably coupled to the manipulator 1200 and the instrument 1400. The system also includes a controller 1180 (see FIG. 21) operably coupled to the manipulator 1200, the operator input device 1116, and the force sensor unit 1800. As further described below, the controller 1180 includes at least one memory device 1184 that has stored instructions 1700 and at least one processor 1182 operably coupled to execute the stored instructions 1700. The stored instructions 1700 described a plurality of operations that can be employed to concurrently deliver the first haptic output and the second haptic output to the operator (e.g., surgeon S) of the system 1000. Although the set of instructions 1700 is described with respect to the instrument 1400 and the controller 1180, in other embodiments, the set of instructions 1700 and any of the methods described herein can be performed on any suitable instrument or any suitable controller.

[0091] In some embodiments, the set of instructions 1700 includes receiving, at 1702, an output signal 1802 from the force sensor unit 1800. The output signal 1802 corresponds to a sensed force (e.g., the magnitude of the forces within a Cartesian coordinate system located at the force sensor unit) exerted on the instrument 1400 (e.g., on the distal end portion 1402). The output signal 1802 can, for example, be an output of the strain sensor 1830 that is indicative of a strainmagnitude developed by the beam 1810 in response to a load applied to the instrument 1400. Accordingly, the output signal 1802 can be a voltage differential measured across a Wheatstone bridge circuit.

[0092] As depicted at 1704, a force vector 1706 is determined for the sensed force. The force vector 1706 corresponds to the magnitude and the direction of the sensed force as indicated by the output signal 1802. A first haptic output 1720 (e.g., the haptic feedback) is then generated, at 1718, based on the force vector 1706. Therefore, the first haptic output 1720 is configured to deliver a force in the same direction as the force vector 1706 to the hand of the surgeon S via the handle portion 1128. In other words, the first haptic output 1720 is a simulation (e.g., representation) of the actual forces affecting the instrument 1400 as sensed by the force sensor unit 1800 at any given instant. Insofar as the first haptic output 1720 is representative of the actual forces sensed by the force sensor unit 1800, the first haptic output 1720 increases the immersion, realism, and intuitiveness of the surgeon’s experience while using the system 1000 to perform a procedure. In some embodiments, the first haptic output 1720 is generated by a haptic feedback system of the control unit 1100 in response to a signal from the controller 1180 based on the execution of the instruction 1700 by the processor 1182.

[0093] In some embodiments, executing the instruction 1700 includes determining, at 1708, a limiting operating condition 1710 of a portion of the system 1000. Determining the limiting operating condition 1710 can include identifying at least one limiting operating condition that is relevant at a given instant based at least in part on an operational state and / or position of a portion of the system 1000. Said another way, the presence of a relevant limiting operating condition 1710, the occurrence of a limiting operating condition 1710, or the interaction of a portion of the system 1000 with a limiting operating condition 1710 can be identified, detected, or determined.

[0094] With reference to the limiting operating condition 1710, the portion of the system 1000 includes the manipulator 1200 (including the arm assembly 1300), the instrument 1400, or both the manipulator 1200 and the medical instrument 1400. The limiting operating condition 1710 can, for example, be a range-of-motion limitation, a positional limitation or requirement (e.g., a positional limitation of the end effector 1460), a collision event, a procedural limitation or requirement, a surgical system error condition, a movement velocity limitation, a temperaturelimitation or requirement, an end effector orientation limitation or requirement, and / or other similar physical or control limitation or requirement. In some embodiments, the limiting operating condition 1710 can be a predetermined limiting operating condition, such as a design range-of-motion limitation, that is relevant / applicable when the portion of the system 1000 is in specified kinematic poses. In some embodiments, the limiting operating conditional 1710 can correspond to the unplanned or unintended occurrence of an event during a procedure, such as a collision between portions of the system 1000 and is, thus, relevant only on occurrence. In some embodiments, the limiting operating condition 1710 can correspond to an unplanned but deliberate limit generated by the controller 1800 based on an inspection of the environment and the determination of “no-go zones,” which is thus relevant for the inspected environment.

[0095] Following the determination / identification of the limiting operating condition 1710, or the sensing of an interaction therewith, the processor 1180 is configured to determine, at 1712, whether the limiting operating condition 1710 is present during an operation of the system 1000. On a condition in which the limiting operating condition 1710 is relevant based at least in part on an operational state and / or position of a portion of the system 1000 (e.g., is encountered during or otherwise potentially impacting a procedure utilizing the system 1000), a second haptic output 1716 (e.g., the haptic signal or haptic communication) is generated, at 1714. The second haptic output 1716 is configured to convey information to the surgeon S in a tactile form (e.g., a vibration, rumbling, a skin stretch, a pressure, and / or a pinching) via the control unit 1100 (e.g., the operator input device 1116). The conveyed tactile information is indicative of an operating condition of at least a portion of the system 1000. In contrast to the first haptic output 1720, in some embodiments, the second haptic output 1716 is not a simulation or representation of the actual forces affecting the instrument 1400. For example, the second haptic output 1716 can provide the surgeon S with an indication that a portion of the system 1000 is approaching a range-of-motion limit prior to the portion of the system 1000 physically reaching the range-of-motion limit. However, in an embodiment in which the limiting operating condition 1710 corresponds to a collision event, the second haptic output 1716 can simulate the collision event, although the second haptic output 1716 need not replicate the forces encountered during the actual collision event.

[0096] By way of illustration, in some embodiments, the second haptic output 1716 is based on a modeled parameter 1724. The modeled parameter 1724 is associated with the limitingoperating condition 1710. For example, the modeled parameter can correspond to a distance, direction, and / or rate of approach to a range of motion limit, a boundary region, a target tissue and / or other relevant limiting operating condition 1710. Therefore, the modeled parameter 1724 is different than the sensed force exerted on the distal end portion 1402 as indicated by the output signal 1802 of the force sensor unit 1800. Said another way, in some embodiments the first haptic output 1720 conveys a simulated sensation of the sensed force (e.g., provides a force feedback) to the handle portion 1128 of the operator input device 1116, while the second haptic output 1716 conveys information about the relationship between the limiting operating condition 1710 and the portion of the system 1000. The information about the relationship (e.g., the presence of, approach to, and / or contact with the limiting operating condition 1710) can be conveyed to the operator holding the handle portion 1128 of the operator input device 1116 without simulating a sensation of the limiting operating condition 1710. In other words, while the first haptic output 1720 is configured to increase realism and immersion, the second haptic output 1716 is configured to communicate information to the operator. Said yet another way, in some embodiments the first haptic output 1720 is indicative of a force at the distal end portion 1402 of the instrument 1400 at a surgical site, while the second haptic output 1716 is indicative of an approach of the portion of the system 1000 to the limiting operating condition 1710. By way of illustration, the first haptic output 1720 can provide the operator with a sense of how much pressure they are applying to a target in a manner that is substantially similar to that which would be encountered if the instrument were manually operated. In the same illustration, concurrent with the sense provided by the first haptic output 1720, the second haptic output 1716 can indicate an approach to a range of motion limit via a series of pulses that increase in frequency as the separation distance diminishes.

[0097] As depicted at 1722, on a condition in which the limiting operating condition 1710 is encountered (i.e., at 1712 determined to exist during the operation), the first haptic output 1720 and the second haptic output 1716 are concurrently (e.g., having at least a partial temporal overlap) provided at the operator input device 1116. The second haptic output 1716 is different than the first haptic output 1720. Accordingly, although delivered concurrently (e.g., simultaneously), the second haptic output 1716 is configured to be discernible by the surgeon S even in the presence of the first haptic output 1720 so that the surgeon unambiguously perceives both the first and second haptic outputs and therefore unambiguously perceives the information conveyed by each of thefirst and second haptic outputs. In other words, as further described below, the second haptic output 1716 has an output profile and / or is generated in such a manner so as to be distinguished from the first haptic output 1720. In some embodiments, however, the second haptic output 1716 can be provided at the operator input device 1116 in the absence of the first haptic output 1720 to convey information about the operating condition of the portion of the system 1000 while the distal end portion 1402 of the instrument 1400 supported by the manipulator 1200 is in an unloaded position (e.g., not in contact with another object such as the tissue of a patient). In some embodiments, the first haptic output 1720 and the second haptic output 1716 are provided at the operator input device 1116 at different times (i.e., non-simultaneous). For example, in some embodiments, the first haptic output 1720 is provided starting at a first time and the second haptic output 1716 is provided starting at a second time that is different from (e.g., that occurs after) the first time. In such embodiments, the first haptic output 1720 and the second haptic output 1716 can be provided for a duration such that that the delivery of each ends at the same time. In this manner, the first haptic output 1720 and the second haptic output 1716 are provided in an overlapping manner at the operator input device 1116.

[0098] While the first haptic output 1720 is configured to provide the operator with an impression of the forces affecting the instrument 1400 as if the operator were directly holding the medical instrument, the second haptic output 1716 can be configured to provide information about the operating environment of the system 1000, including the surgical site. Accordingly, in some embodiments, the second haptic output 1716 has a directional component that is indicative of a direction associated with the limiting operating condition 1710 the directional component can be relative to a position of the portion of the system 1000. For example, the directional component of the second haptic output 1716 can manifest as a tactile sensation perceived by the operator to be offset (e.g., laterally with respect to a midline) from the distal end portion 1402 of the instrument 1400 in a specified direction based on the location of an obstruction or other limitation. Similarly, the directional component of the second haptic output 1716 can indicate that the portion of the system 1000 is moving toward or away from the limiting operating condition 1710 (e.g., an obstruction).

[0099] In some embodiments, the first haptic output 1720 has a first output profile Pi (the magnitude of force as a function of time) as depicted in FIG. 13A. FIG. 13A depicts changes inthe magnitude of the force delivered to the handle portion 1128 of the operator input device 1116 corresponding to changes in the sensed force affecting the distal end portion 1402 of the instrument 1400 over a specified interval. As depicted in FIG. 13B, the second haptic output 1716 has a second output profile P2 over the same interval as depicted in FIG. 13A. Therefore, FIGS. 13A and 13B illustrate that over the same time period, the first output profile Pi is different than the second output profile P2.

[0100] As depicted in FIG. 13B, the second output profile P2 (e.g., the vibrotactile signature) of the second haptic output 1716 can be rendered to be distinguishable from the first output profile Pi. For example, a frequency OP2F (e.g., a carrier frequency) can be chosen for the second haptic output 1716, and the amplitude OP2A of the frequency OP2F can be amplitude modulated to provide a sensation of pulses. The number of pulses over a given interval, the duration of each pulse OP2D, the intervals OP21 between the initiation of each pulse, and / or changes in the relative amplitudes OP2A of the pulses are configurable to generate a set of parameters that are intuitively distinguishable from the first output profile Pi of the first haptic output 1720.

[0101] As depicted in FIGS. 13B-20, in some embodiments, the second output profile P2 of the second haptic output has a vibratory signal profile. In some embodiments, such as depicted in FIGS. 14, 17, and 18 the vibratory signal profile oscillates at a constant frequency. As depicted in FIGS. 17 and 18, for example, the second output profile P2 can oscillate at the constant frequency between a first fixed magnitude and a second fixed magnitude (e.g., a first fixed amplitude and a second fixed amplitude).

[0102] In some embodiments, the second haptic output 1716 has a second output profile P2 with a magnitude that varies based on the position of the portion of the system 1000. For example, as depicted in FIG. 14, the second haptic output 1716 can have a vibratory signal profile that can exhibit a square wave form. The vibratory signal profile can oscillate at a constant frequency. However, the vibratory signal profile can have a variable magnitude. The variable magnitude can increase as the portion of the system 1000 (depicted as the instrument 1400 in FIG. 14) approaches the limiting operating condition 1710 (e.g., as a separation distance D decreases). Similarly, the magnitude of the second output profile P2 can decrease as the distance D between the portion of the system 1000 and the limiting operating condition 1710 (e.g., a range-of-motion limit, anobstruction, etc.) increases. In other words, in some embodiments, the operator can perceive the second haptic output 1716 as a series of constant pulsations that increase in intensity in conjunction with an increase in the proximity of the portion of the system 1000 to the limiting operating condition 1710.

[0103] In some embodiments, the second haptic output 1716 has a second output profile P2 with a magnitude that is constant but a frequency that varies based on the position of the portion of the system 1000. For example, as depicted in FIG. 15, the vibratory signal profile can oscillate at a constant magnitude. However, the vibratory signal profile can have a variable frequency. The variable frequency can increase as the portion of the system 1000 (depicted as the instrument 1400 in FIG. 15) approaches the limiting operating condition 1710 (e.g., as a separation distance D decreases). Similarly, the magnitude of the second output profile P2 can decrease as the distance D between the portion of the system 1000 and the limiting operating condition 1710 (e.g., a range-of-motion limit, an obstruction, etc.) increases. In embodiments such as depicted in FIG. 15, the operator can perceive the second haptic output 1716 as a series of pulsations that come with increased frequency (e.g., a decreasing duration between the initiation of each pulse) in conjunction with an increase in the proximity of the portion of the system 1000 to the limiting operating condition 1710. In such embodiments, each of the pulses can have the same magnitude (e.g., perceived intensity).

[0104] In some embodiments, the second haptic output 1716 can have a second output profile P2 that has a regular periodic signature that is manifest by the vibrotactile sensations that are delivered to the operator input device 1116. For example, FIG. 17 depicts embodiments in which the second output profile P2 has a triangular wave form. FIG. 18 depicts exemplary embodiments in which the second output profile P2 has a sinusoidal wave form. FIG. 19 depicts exemplary embodiments in which the second output profile P2 has a complex wave form. FIG. 20 depicts exemplary embodiments in which the second output profile P2 has a complex wave form of repeating patterns. In the graphical depictions of FIGS. 17-20, the y-axis is indicative of the magnitude of the signal, while the x-axis is indicative of either a time interval or a position of the portion of the system 1000 (e.g., the position of the end effector 1460 in the surgical field). It should be appreciated that any second output profile P2 described herein can be selected based on the information to be communicated to the operator and / or the discemability of the second hapticoutput 1716 in the presence of the first haptic output 1720. For example, in some embodiments, a sinusoidal wave form can be used to provide an early indication of the limiting operating condition 1710. In contrast, a triangular wave form can be used to convey a more urgent indication of the limiting operating condition 1710. Further, the abrupt onset and precise pulse duration of a square wave form can be selected to communicate a critical message, such as an imminent collision event.

[0105] Referring now to FIGS. 10 and 16, in some embodiments, the set of instructions 1700 includes determining, at 1730, that a collision (i.e., undesired contact) is occurring between an object and the portion of the system 1000. In such an embodiment, the limiting operating condition 1710 corresponds to the occurrence of the collision. In such embodiments, the second output profile P2 replicates the collision identified at 1730. The replication of the collision refers to the replication of the vibratory signature of a generic collision involving at least one metallic component rather than the specific forces and / or frequencies generated by the actual collision of the portion of the system 1000. To replicate the collision, the second output profile P2 includes a spike portion SP followed by a vibratory portion VP. The spike portion has a duration that is ten percent or less of the duration of the vibratory portion. Additionally, the spike portion has a maximal magnitude that is at least two times the maximum magnitude of the vibratory portion.

[0106] In some embodiments, the second haptic output 1716 is generated in such a manner so as to be distinguished from the first haptic output 1720. Said another way, the channels of the kinesthetic force feedback (e g., the haptic outputs) can be distinguished based on the characteristics of the signal and / or by the actuators used to generate the different channels to avoid confusion as to the source and / or cause of the feedback. In some embodiments, the operator input device 1116 can include at least a single actuator that is associated with a motion control relationship between the operator input device 1116 and the portion of the system 1000, is configured to produce at least a portion of the first haptic output 1720, and is configured to produce at least a portion of the second haptic output 1716. In some embodiments, however, the operator input device 1116 includes at least a first actuator and a second actuator, such as the first gimbal actuator 1136 and the first support actuator 1144 depicted in FIG. 3. The first actuator is associated with a motion control relationship between the operator input device 1116 and the portion of the system 1000 and is configured to produce the first haptic output 1720, while the second, different actuator is at least configured to produce the second haptic output 1716.

[0107] In some embodiments, the first actuator is the first gimbal actuator 1136, which is operably coupled to the handle portion 1128 (FIG. 4), via the handle input shaft 1 134 (FIG. 4). Therefore, the first gimbal actuator 1136 is configured to exert a torque on the handle portion 1128 to produce the first haptic output 1720. Of all the actuators described herein, the first gimbal actuator 1136 is positioned in closer proximity to the hand of an operator on a condition that the operator is gripping the handle portion. The proximal positioning of the first gimbal actuator 1136 can facilitate the rendering and resultant perception of accurate force feedback (i.e., the first haptic output 1720) generated in response to sense forces acting on the distal end portion 1402 of the instrument 1400. For example, the first gimbal actuator 1136 can be configured to provide feedback for a high frequency portion of the first haptic output 1720 to increase realism and immersion. In some embodiments, however, the first haptic output 1720 can be rendered by additional actuators of the system 1000 configured to affect the pitch, yaw and elbow of the operator input device 1116, as such actuators can affect cartesian forces affecting the hand of the operator.

[0108] In some embodiments, the second actuator is one of the second gimbal actuator 1138, the third gimbal actuator 1140, the first support actuator 1144, the second support actuator 1145, or the third support actuator 1146 depicted in FIGS. 3 and 4. In embodiments wherein the second actuator is one of the support actuators 1144, 1145, or 1146, the second actuator is configured to exert a torque on the support arm assembly 1118 to produce a second haptic output 1716. For example, in some embodiments, the first support actuator 1144 is configured to produce the second haptic output 1716. The first support actuator 1144 is positioned within the base portion 1141 (FIG. 3). In this position, the first support actuator 1144 is configured to affect a rotational orientation of the base portion 1141 relative to a fixed portion of the control unit 1100. In other words, the first support actuator 1144 is positioned to react off (e.g., based from) of a fixed component (e.g., the control unit 1100, which is positioned in contact with the support surface) rather than a movable component (e.g., a support segment and / or a gimbal link). Therefore, the first support actuator 1144 is particularly well-suited to produce the second output profiles P2 of the second haptic output 1716 as described herein. Of all the actuators described herein in reference to the operator input device 1116, the first support actuator 1144 is positioned at a distal most position of the operator input device 1116 and thus the farthest from the hand of an operatoron a condition that the operator is gripping the handle portion 1128. The distal positioning of the first support actuator 1144 facilitates the differentiation of the second haptic output 1716 from the first haptic output 1720, which is generated from a more proximal actuator (e.g., the first gimbal actuator 1136).

[0109] In some embodiments, the second actuator is an auxiliary actuator 1148 (FIG. 2). The auxiliary actuator 1148 can be operably coupled to a portion of the control unit 1100. The auxiliary actuator 1148 can be dedicated to the production of the second haptic output 1716. In other words, the auxiliary actuator 1148 may be activated only on a condition that the limiting operating condition 1710 is encountered during an operation of the system 1000. The intermittent haptic feedback provided by the auxiliary actuator 1148 can be readily distinguished from the first haptic output 1720 by the operator. Additionally, the auxiliary actuator 1148 can be positioned such that the tactile feedback produced by auxiliary actuator 1148 is received by a portion of the operator’s body other than the hands gripping the handle portion 1128.

[0110] Referring to FIGS. 11 and 12, in some embodiments, an available range-of-motion volume RMv is defined for the portion of the system 1000. The range-of-motion volume RMv can be bounded by a range-of-motion limit RML. Therefore, the limiting operating condition 1710 can correspond to a condition where a system component moves (or is commanded to move) beyond the range-of-motion limit RML and / or approaches the range-of-motion limit. For example, FIGS. 11 and 12 are perspective views of the distal end portion 1402 of the instrument 1400 positioned in a range-of-motion volume RMv defined in a surgical site and bound by. In FIGS. 11 and 12, the partially opaque border represents portions of the surgical site that fall outside of the range-of-motion volume RMv.

[0111] In some embodiments, the limiting operating condition 1710 is a design range-of-motion limit RML of the manipulator 1200, the arm assembly 1300, the instrument 1400, or a combination thereof. In additional embodiments, the limiting operating condition 1710 can be a design range-of-motion limit RML of the operator input device 1116.

[0112] In some embodiments, the range-of-motion volume RMv, and thus the limiting operating condition 1710, is defined at least in part by a non-anatomical object that affects the travel of the portion of the system 1000. For example, the available travel of the distal end portion1402 of the instrument 1400 in the surgical site can be limited by the position of the manipulator 1200, other arm assemblies 1300 supported by the manipulator 1200, the operating table 1010 (FIG. 1), and / or other instruments within the surgical site). In some embodiments, the system 1000 includes a sensor 1170 (FIG. 6) positioned to sense an orientation, and / or a position of the non-anatomical object and / or the portion of the system 1000. The sensor 1170 generates kinematic information 1726. The kinematic information 1726 is indicative of the orientation and / or position of the object and / or the portion of the system 1000. In some embodiments, the second haptic output 1716 is based at least in part on the kinematic information 1726. For example, as depicted in FIG. 6, the sensor 1170 can be positioned to generate kinematic information indicative of the orientation of each arm assembly 1300 supported by the manipulator 1200. The second haptic output 1716 can then be configured to convey information to the operator about the orientation of any arm assemblies 1300 affecting the available range of travel of the portion of the system 1000.

[0113] In some embodiments, the kinematic information 1726 can be used, at least in part, by the controller 1180 to model the position of the portion of the system 1000. In some embodiments, the controller 1180 can utilize alternative sensor data to model the position of the portion of the system 1000. Regardless of the data source underpinning the model position, in some embodiments, the limiting operating condition 1710 corresponds to a modeled position of the portion of the system 1000.

[0114] In some embodiments, the range-of-motion volume RMv, and thus the limiting operating condition 1710, is defined at least in part by an anatomical object. For example, contact with certain anatomical objects within the surgical site can be undesirable. Therefore, the desirable avoidance of the anatomical object can affect the range-of-motion volume RMv. The effect of the anatomical object on the range-of-motion volume RMv can result in an irregularly shaped range-of-motion volume RMv, as depicted in FIG. 12.

[0115] As depicted in FIG. 7, in some embodiments, the arm assembly 1300 includes at least the first segment 1310 and the second segment 1312, which are movably coupled together via a joint 1320. The joint encoder 1330 is positioned to output a signal 1728 that is indicative of a rotational orientation of the joint 1320. In such embodiments, determining the available range of travel for the portion of the system 1000, and thus the limiting operating condition 1710, includesdetermining a pose of the manipulator, the arm assembly 1300, and the supported instrument 1400 based on the position signal 1728 from the joint encoder 1330. In other words, the signal 1728 can be indicative of degrees of flexion, extension, and / or rotation that are available at each joint 1320 of the arm assembly 1300 at a given instant. The degrees of flexion, extension, and / or rotation that are available can determine the range of available positions for the distal end portion 1402 of the instrument 1400.

[0116] In some embodiments, the limiting operating condition 1710 is a dynamic range-of-motion limit RML(D). The dynamic range-of-motion limit RML(D) corresponds to an available range of motion at a given instant. For example, the dynamic range-of-motion limit RML(D) can be determined based on the relative positions of each of the segments (e.g., the first segment 1310 and the second segment 1312) of the arm assembly 1300 at the given instant. Therefore, the dynamic range-of-motion limit RML(D) can define a range-of-motion volume RMv that is different than the design range-of-motion volume RMv. The resultant range-of-motion volume RMv can have an irregular shape, such as depicted in FIG. 12.

[0117] As depicted in FIGS. 6 and 11, in some embodiments the limiting operating condition 1710 is based at least in part on a distance D between an object and the portion of the system 1000 that is less than a defined distance. In other words, a desired clearance distance relative to an anatomical object or non-anatomical object can be defined by the controller 1180. On a condition that a portion of the system 1000, such as the distal end portion 1402 of the instrument 1400, has a separation from the object that is less than the clearance distance, a warning can be communicated to the operator in the form of the second haptic output 1716. Generating the second haptic output 1716 in response to the distance D being less than the defined distance can facilitate collision avoidance.

[0118] As depicted in FIG. 15, in some embodiments, the second haptic output 1716 can be initiated as the portion of the system 1000 crosses an initial point IP. Prior to crossing initial point IP, the second haptic output 1716 may not be generated, but upon crossing the initial point IP, the portion of the system 1000 can be within a specified distance D of the limiting operating condition 1710 and the second haptic output 1716 is generated. In some embodiments, on the condition which the limiting operating condition 1710 exists (e.g., upon crossing the initial point IP) a gainmagnitude of the first haptic output 1720 can be reduced. The reduction in the gain magnitude of the first haptic output 1720 can increase the operator’s perception of the second haptic output 1716.

[0119] In some embodiments, the limiting operating condition 1710 is based at least in part on a velocity between an object and the portion of the system 1000 that is greater than a defined velocity limit. The velocity can, for example, be a closure velocity indicating that the rate of closure between the portion of the system 1000 (e.g., the distal end portion 1402 of the instrument 1400) and another object is greater than an operational limit. On a condition that the velocity is greater than the defined velocity limit, a warning can be communicated to the operator in the form of the second haptic output 1716. Generating the second haptic output 1716 in response to the velocity being greater than the defined velocity limit can facilitate collision avoidance and compliance with additional operating procedures and / or parameters of the system 1000.

[0120] As depicted in FIG. 7, in some embodiments, the arm assembly 1300 can include at least one motor 1340 positioned to affect the rotation of the joint 1320. In such embodiments, the limiting operating condition 1710 can correspond to a load 1732 on the motor. The load on the motor 1340 can, for example, be indicative of a lack of additional travel of the joint in the commanded direction, an error condition, and / or an excessive load applied to a portion of the system 1000.

[0121] In some embodiments, the system 1000 includes at least a second instrument supported by a second manipulator, as depicted in FIG. 6. The system 1000 also includes a second operator input device, as depicted in FIG. 2, operably coupled to the second manipulator and the second instrument. In such embodiments, on a condition which the limiting operating condition 1710 is present, a third haptic output can be provided at the second operator input device. In some embodiments, the third haptic output is the same as the second haptic output. In other words, the third haptic output can have an output profile that coincides with the second output profile OP2.

[0122] As shown particularly in FIG. 21, a schematic diagram of one embodiment of suitable components that may be included within the controller 1180 is illustrated. In some embodiments, the controller 1180 is positioned within a component of the system 1000, such as the user control unit 1100 and / or the optional auxiliary equipment unit 1150. However, the controller 1180 may also include distributed computing systems wherein at least one aspect of the controller 1180 is ata location which differs from the remaining components of the system 1000 for example, at least a portion of the controller 1 180 may be an online controller.

[0123] As depicted, the controller 1180 includes one or more processor(s) 1182 and associated memory device(s) 1184 configured to perform a variety of computer implemented functions (e.g., performing the methods, steps, calculations and the like and storing relevant data as disclosed herein). Additionally, in some embodiments, the controller 1180 includes a communication module 1186 to facilitate communications between the controller 1180 and the various components of the system 1000.

[0124] As used herein, the term “processor” refers not only to integrated circuits referred to in the art as being included in a computer, but also refers to a controller, a microcontroller, a microcomputer, a programmable logic controller (PLC), an application specific integrated circuit, and other programmable circuits. Additionally, the memory device(s) 1184 can generally comprise memory element(s) including, but not limited to, computer readable medium (e.g., random access memory (RAM)), computer readable nonvolatile medium (e.g., a flash memory), a floppy disc, a compact disc read only memory (CD ROM), a magneto optical disc (MOD), a digital versatile disc (DVD) and / or other suitable memory elements. Such memory device(s) 1184 can generally be configured to store suitable computer readable instructions that, when implemented by the processor(s) 1182, configure the controller 1180 to perform various functions.

[0125] In some embodiments, the controller 1180 includes a haptic feedback module 1820. The haptic feedback module 1820 can be configured to deliver a haptic feedback to the operator based on inputs received from a force sensor unit 1800 of the instrument 1400. In some embodiments, the haptic feedback module 1820 can also be configured to deliver the second haptic output indicative of limiting operating condition. In some embodiments, haptic feedback module 1820 can be an independent module of the controller 1180. However, in some embodiments the haptic feedback module 1820 can be included within the memory device(s) 1184.

[0126] The communication module 1186 can include a control input module 1188 configured to receive control inputs from the operator / surgeon S, such as via the operator input device 1116 of the user control unit 1100. The communication module can also include an indicator module 1192 configured to generate various indications in order to alert the operator.

[0127] The communication module 1186 can also include a sensor interface 1190 (e.g., one or more analog to digital converters) to permit signals transmitted from one or more sensors (e.g., strain sensors of the force sensor unit 1800) to be converted into signals that can be understood and processed by the processors 1182. The sensors can be communicatively coupled to the communication module 1186 using any suitable means. For example, the sensors can be coupled to the communication module 1186 via a wired connection and / or via a wireless connection, such as by using any suitable wireless communications protocol known in the art. Additionally, in some embodiments, the communication module 1186 includes a device control module 1194 configured to modify an operating state of the instrument 1400 (and / or any of the instruments described herein. Accordingly, the communication module is communicatively coupled to the manipulator 1200 and / or the instrument 1400. For example, the communications module 1186 can communicate to the manipulator 1200 and / or the instrument 1400 an excitation voltage for the strain sensor(s), a handshake and / or excitation voltage for a positional sensor (e.g., for detecting the position of the designated portion relative to the cannula), cautery controls, positional setpoints, and / or an end effector operational setpoint (e.g., gripping, cutting, and / or other similar operation performed by the end effector).

[0128] FIG. 22 is a flow chart of a method 60 of control for a surgical system according to an embodiment. The method 60 may, in an embodiment, be performed via a teleoperated system, such as system 1000 as described with reference to FIGS. 1-21. However, it should be appreciated that in various embodiments, aspects of the method 60 may be accomplished via additional embodiments of the system 1000 or components thereof as described herein. Accordingly, the method 60 may be implemented on any suitable device as described herein. Thus, the method 60 is described below with reference to instrument 1400 and the controller 1180 of the system 1000 as previously described, but it should be understood that the method 60 can be employed using any of the medical devices / instruments and controllers described herein.

[0129] As depicted at 61, the method 60 includes receiving, via the controller, an output signal from the force sensor unit. The output signal corresponds to a sensed force exerted on the distal end portion of the medical instrument. As depicted at 62, the method 60 includes determining, via the controller, a force vector corresponding to a magnitude and direction of the sensed force. As depicted at 63, the method 60 concurrently providing a first haptic output and a second hapticoutput at the operator input device. The first haptic output is based on the force vector. The second haptic output is indicative of a limiting operating condition of the manipulator, the instrument, or both the manipulator and the instrument. The second haptic output being different from the first haptic output.

[0130] While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. Where methods and / or schematics described above indicate certain events and / or flow patterns occurring in certain order, the ordering of certain events and / or operations may be modified. While the embodiments have been particularly shown and described, it will be understood that various changes in form and details may be made.

[0131] For example, any of the instruments described herein (and the components therein) are optionally parts of a surgical assembly that performs minimally invasive surgical procedures, and which can include a manipulator unit, a series of kinematic linkages, a set of cannulas, or the like. Thus, any of the instruments described herein can be used in any suitable surgical system, such as the system 1000 shown and described above. Moreover, any of the instruments shown and described herein can be used to manipulate target tissue during a surgical procedure. Such target tissue can be cancer cells, tumor cells, lesions, vascular occlusions, thrombosis, calculi, uterine fibroids, bone metastases, adenomyosis, or any other bodily tissue. The presented examples of target tissue are not an exhaustive list. Moreover, a target structure can also include an artificial substance (or non-tissue) within or associated with a body, such as for example, a stent, a portion of an artificial tube, a fastener within the body or the like.

[0132] Although various embodiments have been described as having particular features and / or combinations of components, other embodiments are possible having a combination of any features and / or components from any of embodiments as discussed above. Aspects have been described in the general context of medical devices, and more specifically surgical instruments, but inventive aspects are not necessarily limited to use in medical devices.

Claims

What is claimed is:

1. A computer-assisted system, comprising: a manipulator configured to support an instrument, the instrument having a distal end portion and a force sensor unit operably coupled to the instrument; an operator input device operably coupled to the manipulator and the instrument; and a controller operably coupled to the manipulator, the operator input device, and the force sensor unit, the controller including at least one memory device having stored instructions and at least one processor operably coupled to execute the instructions, executing the instructions includes: receiving an output signal from the force sensor unit, the output signal corresponding to a sensed force exerted on the distal end portion of the instrument, determining a force vector corresponding to a magnitude and direction of the sensed force, concurrently providing a first haptic output and a second haptic output at the operator input device, the first haptic output being based on the force vector, the second haptic output being indicative of a limiting operating condition of the manipulator, the instrument, or both the manipulator and the instrument, and the second haptic output being different from the first haptic output.

2. The computer-assisted system of claim 1, wherein: the second haptic output is based on a modeled parameter associated with the limiting operating condition; and the modeled parameter is different than the sensed force exerted on the distal end portion of the instrument as indicated by the output signal of the force sensor unit.

3. The computer-assisted system of claim 1, wherein: the first haptic output conveys a simulated sensation of the sensed force to a handle portion of the operator input device; andthe second haptic output conveys information about the limiting operating condition to a handle portion of the operator input device without simulating a sensation of the limiting operating condition.

4. The computer-assisted system of claim 1, wherein: the first haptic output is indicative of a force at the distal end portion of the instrument at a surgical site; and the second haptic output is indicative of an approach of the manipulator, the instrument, or both the manipulator and the instrument to the limiting operating condition.

5. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a directional component indicative of a direction associated with the limiting operating condition relative to a position of the manipulator, the instrument, or both the manipulator and the instrument.

6. The computer-assisted system of any of claims 1 to 4, wherein: the operator input device includes at least one actuator configured to produce at least a portion of the first haptic output and at least a portion of the second haptic output.

7. The computer-assisted system of any of claims 1 to 4, wherein: the operator input device includes at least a first actuator and a second actuator; the first actuator is configured to produce the first haptic output; and the second actuator is configured to produce the second haptic output.

8. The computer-assisted system of claim 7, wherein: the operator input device includes a gimbal assembly movably coupled to a support-arm assembly; the support-arm assembly includes a first support segment and a second support segment movably coupled between a base portion of the support-arm assembly and the gimbal assembly;the first actuator is a gimbal actuator operably coupled to a handle portion of the gimbal assembly and configured to exert a torque on the handle portion to produce the first haptic output; and the second actuator is a support actuator operably coupled to the support-arm assembly and configured to exert a torque on the support arm assembly to produce the second haptic output.

9. The computer-assisted system of claim 8, wherein: the gimbal actuator is proximal relative to the handle portion of the gimbal assembly; and the support actuator is distal relative to the handle portion of the gimbal assembly.

10. The computer-assisted system of any of claims 1 to 4, wherein: the operator input device includes a first actuator and a second actuator; the first actuator is associated with a motion control relationship between the operator input device and the manipulator, the instrument, or both the manipulator and the instrument; and the second actuator is dedicated to providing the second haptic output.

11. The computer-assisted system of any of claims 1 to 4, wherein: an available range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together; and the limiting operating condition is defined at least in part by an object affecting the available range-of-motion volume.

12. The computer-assisted system of claim 11, wherein: the computer-assisted system includes a sensor positioned to sense a position, an orientation, or both a position and orientation of the object; the sensor generates kinematic information indicative of the position, the orientation, or both the position and orientation of the object; and the second haptic output is based at least in part on the kinematic information.

13. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is a design range-of-motion limit of the manipulator, the instrument, or both the manipulator and the instrument.

14. The computer-assisted system of claim 13, wherein: the manipulator includes at least two segments coupled by a joint; the computer-assisted system includes a joint encoder positioned to output a position signal indicative of a rotational orientation of the joint; and determining the limiting operating condition includes determining a position of the manipulator and the instrument based on the position signal from the joint encoder.

15. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is a design range-of-motion limit of the operator input device.

16. The computer-assisted system of any of claims 1 to 4, wherein: a range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together; and the limiting operating condition is defined at least in part by an anatomical object affecting the range-of-motion volume.

17. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is a dynamic range-of-motion limit; the dynamic range-of-motion limit corresponds to an available range of motion at a given instant; the manipulator includes at least two segments coupled by a joint; and the instructions include determining the available range of motion based on positions of each of the segments and the instrument at the given instant.

18. The computer-assisted system of any of claims 1 to 4, wherein:executing the instructions includes determining that a collision is occurring between an object and the manipulator, the instrument, or both the manipulator and the instrument; and the limiting operating condition is the collision.

19. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is based at least in part a distance between an object and the manipulator, the instrument, or both the manipulator and the instrument being less than a defined distance.

20. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is based at least in part a velocity between an object and the manipulator, the instrument, or both the manipulator and the instrument relative to an object; and the velocity being larger than a defined velocity.

21. The computer-assisted system of any of claims 1 to 4, wherein: the first haptic output has a first output profile; the second haptic output has a second output profile; and the first output profile is different than the second output profile.

22. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a second output profile the second output profile replicates a collision; the second output profile includes a spike portion followed by a vibratory portion; the spike portion has a duration and a maximum magnitude; the vibratory portion has a duration and a maximum magnitude; the duration of the spike portion is ten percent or less of the duration of the vibratory portion; and the maximum magnitude of the spike portion is at least two times the maximum magnitude of the vibratory portion.

23. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a vibratory signal profile; and the vibratory signal profile oscillates at a constant frequency between a first fixed magnitude and a second fixed magnitude.

24. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a vibratory signal profile; the vibratory signal profile oscillates at a constant frequency; the vibratory signal profile has a variable magnitude; and the variable magnitude increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

25. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a vibratory signal profile; the vibratory signal profile has a constant magnitude; the vibratory signal profile has a variable frequency; and the variable frequency increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

26. The computer-assisted system of any of claims 1 to 4, wherein: the second haptic output has a square wave form.

27. The computer-assisted system of any of claims 1 to 4, wherein: on the condition in which the limiting operating condition exists, a gain magnitude of the first haptic output is reduced.

28. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is a modeled position of the manipulator, the instrument, or both the manipulator and the instrument.

29. The computer-assisted system of any of claims 1 to 4, wherein: the manipulator includes a motor; and the limiting operating condition is a load on the motor.

30. The computer-assisted system of any of claims 1 to 4, wherein: the limiting operating condition is a position of the manipulator, the instrument, or both the manipulator and the instrument based on a signal from an image-based motion tracking system.

31. The computer-assisted system of any of claims 1 to 4, wherein: the computer-assisted system includes: a second manipulator, a second instrument supported by the second manipulator, and a second operator input device operatively coupled to the second manipulator and the second instrument; executing the instructions on the condition in which the limiting operating condition exists includes providing a third haptic output at the second operator input device; and the third haptic output is the same as the second haptic output.

32. A method of control for a computer-assisted system, the computer-assisted system including a manipulator configured to support an instrument, the instrument having a distal end portion and a force sensor unit operably coupled to the instrument, an operator input device operably coupled to the manipulator and the instrument, and a controller operably coupled to the manipulator, the operator input device, and the force sensor unit, the method comprising: receiving, via the controller, an output signal from the force sensor unit, the output signal corresponding to a sensed force exerted on the distal end portion of the instrument; determining, via the controller, a force vector corresponding to a magnitude and direction of the sensed force; and concurrently providing a first haptic output and a second haptic output at the operator input device, the first haptic output being based on the force vector, the second haptic output being indicative of a limiting operating condition of the manipulator, the instrument, orboth the manipulator and the instrument, and the second haptic output being different from the first haptic output.

33. The method of claim 32, wherein: the second haptic output is based on a modeled parameter associated with the limiting operating condition; and the modeled parameter is different than the sensed force exerted on the distal end portion of the instrument as indicated by the output signal of the force sensor unit.

34. The method of claim 32, wherein: the first haptic output conveys a simulated sensation of the sensed force to a handle portion of the operator input device; and the second haptic output conveys information about the limiting operating condition to an operator holding the handle portion of the operator input device without simulating a sensation of the limiting operating condition.

35. The method of claim 32, wherein: the first haptic output is indicative of a force at the distal end portion of the instrument at a surgical site; and the second haptic output is indicative of an approach of the manipulator, the instrument, or both the manipulator and the instrument to the limiting operating condition.

36. The method of any of claims 32 to 35, wherein: the second haptic output has a directional component indicative of a direction associated with the limiting operating condition relative to a position of the manipulator, the instrument, or both the manipulator and the instrument.

37. The method of any of claims 32 to 35, wherein: the operator input device includes at least a first actuator and a second actuator; the first actuator is configured to produce the first haptic output; and the second actuator is configured to produce the second haptic output.

38. The method of claim 37, wherein: the operator input device includes a gimbal assembly movably coupled to a support-arm assembly; the support-arm assembly includes a first support segment and a second support segment movably coupled between a base portion of the support-arm assembly and the gimbal assembly; the first actuator is a gimbal actuator operably coupled to a handle portion of the gimbal assembly and configured to exert a torque on the handle portion to produce the first haptic output; and the second actuator is a support actuator operably coupled to the support-arm assembly and configured to exert a torque on the support arm assembly to produce the second haptic output.

39. The method of any of claims 32 to 35, wherein: the operator input device includes a first actuator and a second actuator; the first actuator is associated with a motion control relationship between the operator input device and the manipulator, the instrument, or both the manipulator and the instrument; and the second actuator is dedicated to providing the second haptic output.

40. The method of any of claims 32 to 35, wherein: an available range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together; and the limiting operating condition is defined at least in part by an object affecting the available range-of-motion volume.

41. The method of claim 40, wherein: the computer-assisted system includes a sensor positioned to sense a position, an orientation, or both a position and orientation of the object;the sensor generates kinematic information indicative of the position, the orientation, or both the position and orientation of the object; and the second haptic output is based at least in part on the kinematic information.

42. The method of any of claims 32 to 35, wherein: the limiting operating condition is a design range-of-motion limit of the manipulator, the instrument, or both the manipulator and the instrument.

43. The method of claim 42, wherein: the manipulator includes at least two segments coupled by a joint; the computer-assisted system includes a joint encoder positioned to output a sensor signal indicative of a rotational orientation of the joint; and determining the limiting operating condition includes determining a position of the manipulator and the instrument based on the sensor signal from the joint encoder.

44. The method of any of claims 32 to 35, wherein: the limiting operating condition is a design range-of-motion limit of the operator input device.

45. The method of any of claims 32 to 35, wherein: a range-of-motion volume is defined for the manipulator, the instrument, or both the manipulator and the instrument together; and the limiting operating condition is defined at least in part by an anatomical object affecting the range-of-motion volume.

46. The method of any of claims 32 to 35, wherein: the limiting operating condition is a dynamic range-of-motion limit; the dynamic range-of-motion limit corresponds to an available range of motion at a given instant; the manipulator includes at least two segments coupled by a joint; andthe method includes determining the available range of motion based on positions of each of the segments and the instrument at the given instant.

47. The method of any of claims 32 to 35, wherein: the method includes determining that a collision is occurring between an object and the manipulator, the instrument, or both the manipulator and the instrument; and the limiting operating condition is the collision.

48. The method of any of claims 32 to 35, wherein: the limiting operating condition is at least in part a distance between an object and the manipulator, the instrument, or both the manipulator and the instrument being less than a defined distance.

49. The method of any of claims 32 to 35, wherein: the limiting operating condition is at least in part a velocity between an object and the manipulator, the instrument, or both the manipulator and the instrument being larger than a defined velocity.

50. The method of any of claims 32 to 35, wherein: the first haptic output has a first output profile; the second haptic output has a second output profile; and the first output profile is different than the second output profile.

51. The method of any of claims 32 to 35, wherein: the second haptic output has a second output profile the second output profile replicates a collision; the second output profile includes a spike portion followed by a vibratory portion; the spike portion has a duration and a maximum magnitude; the vibratory portion has a duration and a maximum magnitude; the duration of the spike portion is ten percent or less of the duration of the vibratory portion; andthe maximum magnitude of the spike portion is at least two times the maximum magnitude of the vibratory portion.

52. The method of any of claims 32 to 35, wherein: the second haptic output has a vibratory signal profde; and the vibratory signal profde oscillates at a constant frequency between a first fixed magnitude and a second fixed magnitude.

53. The method of any of claims 32 to 35, wherein: the second haptic output has a vibratory signal profile; the vibratory signal profile oscillates at a constant frequency; the vibratory signal profile has a variable magnitude; and the variable magnitude increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

54. The method of any of claims 32 to 35, wherein: the second haptic output has a vibratory signal profile; the vibratory signal profile has a constant magnitude; the vibratory signal profile has a variable frequency; and the variable frequency increases as the manipulator, the instrument, or both the manipulator and the instrument approach the limiting operating condition.

55. The method of any of claims 32 to 35, wherein: the second haptic output has a square wave form.

56. The method of any of claims 32 to 35, wherein: on the condition in which the limiting operating condition exists, a gain magnitude of the first haptic output is reduced.

57. The method of any of claims 32 to 35, wherein:the limiting operating condition is a modeled position of the manipulator, the instrument, or both the manipulator and the instrument.

58. The method of any of claims 32 to 35, wherein: the manipulator includes a motor; and the limiting operating condition is a load on the motor.

59. The method of any of claims 32 to 35, wherein: the limiting operating condition is a position of the manipulator, the instrument, or both the manipulator and the instrument based on a signal from an image-based motion tracking system.

60. The method of any of claims 32 to 35, wherein: the computer-assisted system includes a second manipulator, a second instrument supported by the second manipulator, and a second operator input device operatively coupled to the second manipulator and the second instrument; on the condition in which the limiting operating condition exists, the method includes providing a third haptic output at the second operator input device; and the third haptic output is the same as the second haptic output.

61. A non-transitory computer-readable medium having stored thereon a plurality of machine-readable instructions which when executed by one or more processors associated with a computer-assisted device are adapted to cause the one or more processors to perform a method comprising: receiving an output signal from a force sensor unit, the output signal corresponding to a sensed force exerted on a distal end portion of an instrument; determining a force vector corresponding to a magnitude and direction of the sensed force; and concurrently providing a first haptic output and a second haptic output at an operator input device, the first haptic output being based on the force vector, the second haptic output being indicative of a limiting operating condition of a manipulator, the instrument, orboth the manipulator and the instrument, and the second haptic output being different from the first haptic output.

Images