System and method for remote operation based on an input device and control of the position of the input device

Abstract

A computer-assisted system includes a manipulator system and an input device configured to be manipulated by a user. The computer-assisted system also includes one or more actuators to drive the input device in one or more degrees of freedom. The manipulator system is configured to support one or more instruments. The computer-assisted system further includes a control system communicatively coupled with the input device and the manipulator system. The control system is configured to determine an instrument type of an instrument supported by the manipulator system. The control system is also configured to determine a first configuration of the input device based at least in part on the instrument type of the instrument and cause the one or more actuators to drive the input device to its first configuration.

Description

[0001] Cross-references to related applications This application claims priority and benefit from the filing date of U.S. Provisional Patent Application 63 / 566,543, filed March 18, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention generally provides improved computer-aided devices, systems, and methods. Background Technology

[0003] Computer-aided systems can be used to perform tasks at a work site. For example, a computer-aided system may include a handheld power tool assembly. As another example, a computer-aided system may include a manipulator system that may include one or more manipulator arms to manipulate machinery for performing a task.

[0004] Example computer-aided systems include industrial and recreational manipulator systems. Example computer-aided systems also include medical manipulator systems used in procedures such as diagnostics, non-surgical treatments, and surgical treatments.

[0005] Some computer-aided systems include user input systems, each of which includes one or more input devices. These input devices allow users to command and control the movement of various aspects of the computer-aided system, such as manipulator systems. Typically, user operations on hand-based input devices are performed by the user touching and manipulating the input devices with their hands.

[0006] Input devices can control the functions of various types of mechanisms and instruments. Examples include instruments supported by computer-aided systems that are articulated to perform various procedures. For instance, a computer-aided system can use various types of medical instruments to perform minimally invasive surgical procedures, where the medical instruments are remotely operated by a user manipulating and controlling the input device.

[0007] Therefore, there is a desire to improve the remote operation of computer-aided systems. Summary of the Invention

[0008] It should be understood that the foregoing concepts and the additional concepts discussed below can be arranged in any suitable combination, as this disclosure is not limited in this respect. Furthermore, other advantages and novel features of this disclosure will become apparent from the following detailed description of various non-limiting embodiments when considered in conjunction with the accompanying drawings.

[0009] Generally, the embodiments disclosed herein relate to a computer-aided system comprising an input device configured for user manipulation, one or more actuators driving the input device, and a manipulator system. The input device is actuable in one or more degrees of freedom using one or more actuators. The manipulator system is configured to support one or more instruments. The system also includes a control system communicatively coupled to the input device and the manipulator system. The control system is configured to: determine the instrument type of the instrument supported by the manipulator system, determine a first configuration of the input device based at least in part on the instrument type of the instrument, and cause one or more actuators to drive the input device to its first configuration.

[0010] Generally, the embodiments disclosed herein relate to a method for controlling a computer-aided system. The method is performed by a control system of the computer-aided system. The method includes: determining an instrument type of an apparatus supported by a manipulator system of the computer-aided system; determining a first configuration of an input device for the computer-aided system based at least in part on the instrument type of the apparatus, wherein the input device is configured to be manipulated by a user, and the computer-aided system further includes one or more actuators to drive the input device in one or more degrees of freedom; and causing the one or more actuators to drive the input device to its first configuration.

[0011] Generally, the embodiments disclosed herein relate to a non-transitory machine-readable medium having a plurality of machine-readable instructions executable by one or more processors associated with a computer-aided system. The plurality of machine-readable instructions cause one or more processors to perform a method comprising: determining an instrument type of an apparatus supported by a manipulator system of the computer-aided system; determining a first configuration of an input device of the computer-aided system based at least in part on the instrument type of the apparatus, wherein the input device is configured to be manipulated by a user, and the computer-aided system further includes one or more actuators to drive the input device in one or more degrees of freedom; and causing the one or more actuators to drive the input device to its first configuration.

[0012] Other aspects of the invention will become apparent from the following description and the appended claims. Attached Figure Description

[0013] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be represented by similar numbers. For clarity, not every component may be labeled in every figure. In the accompanying drawings: Figure 1 An example computer-aided system according to one or more embodiments is shown.

[0014] Figure 2 A block diagram of an example computer-aided system according to one or more embodiments is depicted.

[0015] Figure 3 A perspective view of an apparatus according to one or more embodiments is shown.

[0016] Figure 4A , Figure 4B , Figure 4C and Figure 4D A perspective view of an exemplary wrist component and an end effector according to one or more embodiments is shown.

[0017] Figure 5 A controller portion of an example input device according to one or more embodiments is depicted.

[0018] Figure 6 An example mapping between input device degrees of freedom and instrument degrees of freedom according to one or more embodiments is depicted.

[0019] Figure 7 An exemplary mapping between the degrees of freedom of gripping an input device and the degrees of freedom of gripping an instrument, according to one or more embodiments, is depicted.

[0020] Figure 8 An exemplary mapping between the gripping degrees of freedom of an input device and the function of an instrument is depicted according to one or more embodiments.

[0021] Figure 9 An exemplary zero-mapping is depicted in relation to the degrees of freedom of the input device.

[0022] Figure 10 A method according to one or more embodiments is shown.

[0023] Figure 11 A flowchart according to one or more embodiments is shown. Detailed Implementation

[0024] Specific embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. For consistency, the same elements are denoted by the same reference numerals in the various figures.

[0025] In the following detailed description of embodiments of the present disclosure, numerous specific details are set forth in order to provide a more complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0026] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as adjectives for elements (i.e., any noun in this application). Unless explicitly disclosed, for example by using the terms “before,” “after,” “single,” and other such terms, the use of ordinal numbers does not imply or create any particular order of elements, nor does it limit any element to being a single element. Rather, ordinal numbers are used to distinguish elements. As an example, the first element is different from the second element, and the first element may contain more than one element and be after (or before) the second element in the element order.

[0027] This disclosure describes various devices, components, and computer-aided systems and component parts in their state in three-dimensional space. As used herein, the term "position" refers to the orientation of an element or part of an element (e.g., three translational degrees of freedom in three-dimensional space, such as along Cartesian x, y, and z coordinates). As used herein, the term "orientation" refers to the rotational placement of an element or part of an element (e.g., three rotational degrees of freedom in three-dimensional space, such as about roll, pitch, and yaw axes, represented by angular axes, rotation matrices, quaternions, and / or the like). As used herein, and for devices having a kinematic sequence, such as a repositionable structure having multiple links coupled by one or more joints, the term "proximal" refers to a direction toward the base of the kinematic sequence, and "distal" refers to a direction away from the base along the kinematic sequence.

[0028] As used herein, the term "pose" refers to the multi-degree-of-freedom (DOF) spatial position and orientation of a rigid body in a coordinate system of interest. Generally, a pose includes pose variables for each DOF within that pose. For example, a full 6-DOF pose of a rigid body in 3D space would include 6 pose variables corresponding to 3 position degrees of freedom (e.g., x, y, and z) and 3 orientation degrees of freedom (e.g., roll, pitch, and yaw). A position pose with only 3 degrees of freedom would only include pose variables for 3 position degrees of freedom. Similarly, an orientation pose with only 3 degrees of freedom would only include pose variables for 3 rotational degrees of freedom. Furthermore, the velocity of the pose captures the change of pose over time (e.g., the first derivative of the pose). For a full 6-DOF pose of a rigid body in 3D space, the velocity would include 3 translational velocities and 3 rotational velocities. Pose with other numbers of degrees of freedom would have corresponding numbers of translational and / or rotational velocities.

[0029] The aspects of this disclosure are described with reference to computer-aided systems capable of including devices that are remotely operated, externally manipulated, autonomous, semi-autonomous, and / or similarly functional. Furthermore, aspects of the invention are described in the context of implementations using remotely operated surgical systems, such as the da Vinci® Surgical System, commercially available from IntuitiveSurgical, Inc., Sunnyvale, California. However, those skilled in the art will understand that the inventive aspects disclosed herein can be embodied and implemented in various ways, including remotely and non-remotely operated, as well as medical and non-medical embodiments and implementations. Implementations on the da Vinci® Surgical System are merely exemplary and should not be construed as limiting the scope of the inventive aspects disclosed herein. For example, the techniques described with reference to surgical instruments and methods can be used in other contexts. Therefore, the instruments, systems, and methods described herein can be used for humans, animals, parts of human or animal anatomy, industrial systems, general-purpose robots, or remote operating systems. As further examples, the instruments, systems, and methods described herein can be used for non-medical purposes, including industrial applications, general robotic applications, sensing or manipulating non-tissue artifacts, cosmetic enhancements, imaging of human or animal anatomy, collecting data from human or animal anatomy, establishing or dismantling systems, training medical or non-medical personnel, and / or similar applications. Additional example applications include procedures for handling tissue removed from human or animal anatomy (with or without return to the human or animal anatomy) and procedures for handling human or animal cadavers. Furthermore, these techniques can also be used in medical treatments or diagnostic procedures, with or without surgical involvement.

[0030] Referring now to the accompanying drawings, where the same reference numerals denote the same portions throughout multiple views, Figure 1 A top view of an example computer-aided system (100) as a medical system according to one or more embodiments is shown.

[0031] Although Figure 1 The computer-aided system (100) is shown as a medical system, but the following description applies to other scenarios and systems, such as medical scenarios or non-surgical, non-medical scenarios or computer-aided systems, etc.

[0032] In this example, a diagnostic or therapeutic medical procedure is performed on a patient (190) on an operating table (110). The computer-aided system (100) may include a manipulator system (130) (e.g., a patient-side robotic device in the medical example). The manipulator system (130) may include at least one manipulator arm (150A, 150B, 150C, 150D), each manipulator arm potentially supporting a detachably coupled instrument (160) (also referred to as a tool (160)). The manipulator assembly may be said to include the manipulator arms (150A, 150B, 150C, 150D) and the instrument (160). Figure 1 In the illustration, an instrument (160) not attached to a manipulator arm is depicted on an auxiliary table or trolley. In the illustrated procedure, the instrument (160) can enter the workspace through an entry point (e.g., through a natural opening such as the throat or anus or through an incision into the patient's (190) body), while the operator (192) (e.g., a clinician such as a surgeon) views the work site (e.g., the surgical site in a surgical setting) through a user input system (“input system”) (120).

[0033] In other examples, the manipulator system may include a common proximal repositionable structure and one or more distal repositionable structures attached to the proximal repositionable structure. Each of the one or more distal repositionable structures may be configured to support one or more instruments. For example, each distal repositionable structure may include a prism joint that, when actuated, linearly moves an instrument carriage and any one or more instruments coupled to the carriage along an insertion axis. In further examples, the repositionable structure may include a single instrument manipulator without cascade coupling of manipulators. In additional examples, the repositionable structure may include a single instrument manipulator coupled to a single base manipulator. In yet another additional example, the computer-aided system may include a movable base mounted to a trolley or an operating table, and one or more manipulators mounted to the movable base.

[0034] Images of the work site can be acquired using an imaging instrument (160) that includes imaging devices such as an endoscope, optical camera, ultrasound probe, etc., in a medical example. The imaging instrument (160) can be used to image the work site and can be manipulated by one of the manipulator arms (150A, 150B, 150C, 150D) of the manipulator system (130) to position and orient the imaging instrument (160). An auxiliary system (140) can process the captured images in various ways prior to any subsequent display. For example, the auxiliary system (140) can overlay the captured images with a virtual control interface before displaying the combined images to the operator via an input system (120) or another display system located locally or remotely within the program. One or more separate displays (not shown) can also be coupled to the control system and / or auxiliary system (140) for local and / or remote display of images, such as images of the procedure site or other relevant images.

[0035] The number of instruments (160) used at one time typically depends on the task and space constraints, among other factors. If it is desired to change, clean, inspect, or reload one or more of the instruments (160) in use during the procedure, an assistant (194A, 194B, 194C) can remove the instrument (160) from the manipulator arm (150A, 150B, 150C, 150D) and replace it with the same instrument (160) or another instrument (160).

[0036] Go to Figure 2 , Figure 2 It describes computer-aided systems (100) (such as...) Figure 1 A block diagram (200) showing the selected components of a medical system and the relationships and interactions between them. Figure 2 The illustration, division, organization, and interaction of the components and / or modules of the computer-aided system (100) in the block diagram (200) are intended to facilitate clear discussion and should not be considered fixed or restrictive. For example, Figure 2 The display (244) is described as a separate entity; however, it should be understood that in practice the display (244) can be implemented as part of, for example, an auxiliary system (140).

[0037] Typically, a computer-aided system (100) may include a control system (242). The control system (242) may be used to process inputs provided by a user input system (120) from an operator (192), such as to control the computer-aided system (100) (e.g., direct movement of manipulator arms (150A, 150B, 150C, 150D)). The control system (242) may also be used to process signals from other devices, from sensors, from any network connected to the control system (242), etc. Example sensors include those associated with actuators or joints of the computer-aided system, such as motor encoders, rotary or linear joint encoders, torque sensors, current sensors, accelerometers, force sensors, inertial measurement units, optical or ultrasonic sensors or imagers, RF sensors, etc. It may be said that a computer-aided system includes one or more actuators to drive input devices of the input system. The control system (242) may further be used to provide outputs, such as video images, for display on a monitor (244). The control system (242) may further be used to control a manipulator system (130).

[0038] The control system (242) may include one or more computer processors, non-permanent memory (e.g., volatile memory such as random access memory (RAM), cache memory), permanent memory (e.g., hard disk, optical drive such as optical disc (CD) drive or digital multifunction disc (DVD) drive, flash memory, etc.), communication interfaces (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.) and many other elements and functions.

[0039] The computer processor of the control system (242) may be part or all of an integrated circuit used to process instructions. For example, the computer processor may be one or more cores or microcores of a processor. The control system (242) may also communicate with one or more auxiliary input devices (which may be independent of the user input system (120)) and one or more input devices of the user input system (120) such as touch screens, keyboards, mice, microphones, touchpads, electronic pens, or any other type of input devices.

[0040] The communication interface of the control system (242) may include an integrated circuit for connecting the control system (242) to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or to another device, such as another control system (242).

[0041] Furthermore, the control system (242) can communicate with one or more output devices, such as display devices (244) (e.g., liquid crystal displays (LCDs), plasma displays, touch screens, organic LED displays (OLEDs), projectors, or other display devices), printers, speakers, external memory, or any other output device. One or more output devices may be the same as or different from one or more auxiliary input devices. Many different types of control systems exist, and the aforementioned input and output devices may take other forms.

[0042] Software instructions in the form of computer-readable program code for performing embodiments of the present disclosure may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer-readable medium, such as a CD, DVD, storage device, disk, magnetic tape, flash memory, physical memory, or any other computer-readable storage medium. Specifically, the software instructions may correspond to computer-readable program code that, when executed by one or more processors, is configured to perform one or more embodiments of the present invention.

[0043] The control system (242) can be connected to a network or be part of a network. The network may include multiple nodes. Each node may correspond to a computing system or a group of nodes. As an example, embodiments of this disclosure can be implemented on nodes of a distributed system connected to other nodes. By another example, embodiments of the invention can be implemented on a distributed computing system with multiple nodes, wherein each part of this disclosure may reside on a different node within the distributed computing system. Furthermore, one or more elements of the control system (242) described above may be located at remote locations and connected to other elements via a network.

[0044] As described above, the computer-aided system (100) may include a manipulator system (130) (e.g., a patient-side robotic device in a medical example). The manipulator system (130) may include at least one manipulator arm (150A, 150B, 150C, 150D), each manipulator arm supporting a detachably coupled instrument (160) (also referred to as a tool (160)). The manipulator assembly may be said to include manipulator arms (e.g., 150A, 150B, 150C, 150D) and an instrument (160), wherein, as described above, the manipulator arm typically supports the instrument (160) extending distally from the manipulator arm and enables movement of the instrument (160).

[0045] In the least invasive scenario, the device (160) can be positioned and manipulated through the entry point such that the kinematic remote center is maintained at the entry point. When the device (160) is positioned within the field of view of an imaging device, an image of the working part taken by an imaging device such as an optical camera may include an image of the distal end of the device (160). In some embodiments, a distal device retainer facilitates the removal and replacement of the installed device.

[0046] For reference Figure 1 Understood, the manipulator arms (e.g., 150A, 150B, 150C, 150D) are mounted proximally to the manipulator system (130) or its base. Alternatively, the manipulator arm or manipulator assembly may be mounted to a separate base that can be moved independently, such as by being mounted to a single manipulator arm cart, or by a mounting clamp provided with a mounting fixture that allows the manipulator arm to be mounted directly or indirectly to the operating table in different locations. An exemplary manipulator arm includes multiple links and joints extending between a proximal base and a distal instrument holder.

[0047] In one example, the manipulator arm includes multiple joints (e.g., rotary joints, prismatic joints, etc.) and links. The joints of the manipulator arm may or may not have redundant degrees of freedom.

[0048] As described above, various instruments (160) can be used with or may be used with a computer-aided system (100) (e.g., manipulated by a manipulator arm). Furthermore, at least some of the instruments (160) can be removed and replaced during procedures implemented by or executed by the computer-aided system (100). In some cases, instruments are designated based on their distal portion (referred to as end effectors). In surgical settings, end effectors may include, but are not limited to, applicators, needle drivers, aspiration and / or irrigation tools, grippers, scissors, staplers, scalpels, endoscopes, and energy emission devices (e.g., electrocautery or ablation devices). Some of these end effectors may have a single end effector element, while others may include multiple end effector elements, such as first and second end effector elements, which may pivot relative to each other to define a pair of end effector clamps. In some embodiments, instruments (160) are classified according to their instrument type. In one or more embodiments, and as will be described later in this disclosure, instruments are classified into a given type based on how the so-called degrees of freedom (DOF) of the input device (i.e., the input device grip DOF) is mapped to or otherwise used to control aspects of the instrument. For example, an instrument may have an instrument grip DOF, in which case manipulation of the input device grip DOF may correspond to, be mapped to, or otherwise cause movement of the instrument along the instrument grip DOF. As another example, an instrument may provide functions such as suction, irrigation, application of surgical clips, etc. In these cases, the degrees of freedom of the input device (e.g., the input device grip DOF) can be used to select the state or activation of the function provided by the instrument. As another example, the degrees of freedom of the input device (e.g., the input device grip DOF) may not be mapped to or otherwise used to control any aspect of the instrument. In other words, for such an instrument, manipulation of one or more input device DOFs (e.g., the input device grip DOF) does not affect the instrument controlled by manipulation of the input device. Various instrument types depending on the mapping of the input device grip DOF to the articulation or functionality of the instrument will be described later in this disclosure. In one or more embodiments, the device type of the device is known by the specifications of the end effector associated with the device (160). Examples of devices and different end effectors are provided in... Figure 3 and Figures 4A to 4D Described in the text.

[0049] In some embodiments, the instrument (160) is releasably mounted on an instrument retainer. The instrument retainer can translate along a linear guide formed, for example, by a prism joint and a distal link. Thus, the instrument retainer can provide in / out movement of the instrument (160) along the insertion axis. The distal link can further support a sleeve through which the instrument axis of the instrument (160) extends. Alternatively, the sleeve can be mechanically supported by another component of the manipulator arm, or it can be unsupported at all.

[0050] In surgical situations, the slender shaft of the instrument (160) allows the end effector of the instrument to be inserted into the surgical site through a micro-incision (e.g., the abdominal wall). The surgical site can be insufflated.

[0051] Actuation of the instrument (160) in one or more degrees of freedom can be provided by actuators of the manipulator assembly. These actuators may be integrated into the instrument retainer or into the drive train within the instrument retainer, and the instrument (160) may be actuated via a transmission assembly. Thus, movement of the end effector within the working part is typically achieved, at least in part, by pivoting the instrument (160) about a predetermined position or a kinematically remote center (e.g., where the instrument (160) enters the orifice). For example, manipulator arms (e.g., 150A, 150B, 150C, 150D) may move the proximal housing of the instrument (160) outside the working part, such that an elongated shaft of the instrument (160) extends through the orifice to provide the desired movement of the associated end effector.

[0052] Figure 3 An example of the instrument (160) is shown. In one or more embodiments, the instrument (160) can be used for surgical procedures. Figure 3 In one example, the device (160) includes an elongated shaft (310) having a distal end (302) and a proximal end (304). The device (160) may include a housing portion (330) located at the proximal end (304) of the shaft (310). The housing portion (330) can be arranged to couple the device (160) to components of a computer-aided system (100), such as a manipulator arm (e.g., 150A, 150B, 150C, 150D). In some embodiments, the shaft (310) is rotatably coupled to the housing portion (330) such that the shaft (310) is rotatable relative to the housing portion (330) about a longitudinal axis (312) of the shaft (310). Figure 3 Arrows (392) are shown to indicate the rotational degrees of freedom (DOF) of the instrument (160). In one or more embodiments, the rotational DOF defined by the rotation of the axis (310) and / or the wrist assembly (320) about the longitudinal axis (312) is referred to as the “rolling” DOF of the instrument (160).

[0053] In some embodiments, the device (160) includes a wrist assembly (320) located at the distal end of the axis (310). The wrist assembly (320) is capable of providing additional degrees of freedom. For example, the wrist assembly (320) is capable of providing pitch and yaw degrees of freedom. In some embodiments, the wrist assembly (320) provides a wrist with a single degree of freedom, such as pitch or yaw. In some embodiments, the wrist assembly (320) does not provide any degrees of freedom or is not included as part of the device (160).

[0054] An end effector (340) may be located at the distal end (302) of the shaft (310). In some embodiments, the end effector (340) is considered part of the wrist assembly (320). Some end effectors (340) are capable of providing one or more additional degrees of freedom, such as instrument grip degrees of freedom (DOF), where “instrument grip DOF” is used here to distinguish that degree of freedom from the corresponding input device grip DOF, as will be described later in this disclosure. The housing portion (330) may include mechanisms such as a rotatable winch to control a tension member extending through the shaft to actuate the wrist assembly (320) and / or the end effector (340). For example, the instrument holder (not shown) of the manipulator arm (e.g., 150A, 150B, 150C, 150D) coupled to the instrument (160) may include an actuable disc (e.g., a motorized disc) coupled to a winch of the housing portion (330) to manipulate the wrist assembly (320) and / or the end effector (340) by “introducing” or “out”ing the tension member.

[0055] Figure 4A , Figure 4B , Figure 4C and Figure 4D The distal end (302) of the instrument (160), including a wrist assembly (320) and an end effector (340), is depicted. Various types of end effectors (340) can be used, each with different geometries, degrees of freedom, and / or functions. Figure 4A A first example end effector (440A) coupled to a shaft (310) via a wrist assembly (320) is shown. In the illustrated example, the wrist assembly (320) includes a proximal clevis (420) mounted on the distal end of the instrument shaft (310) and a distal clevis (430) rotatably mounted on the proximal clevis (420). Furthermore, the first example end effector (440A) includes a pair of movable clamps (442, 444) mounted to the distal clevis (430).

[0056] exist Figure 4AAs shown, a first pin in the proximal U-clamp (420) attaches the distal U-clamp (430) to the proximal U-clamp (420) and allows the distal U-clamp (430) to rotate about a first pivot axis (422). Similar to the rolling DOF discussed above, the first pivot axis (422) may define a first pivotal degree of freedom (DOF) (425) of the wrist assembly (320). Similarly, a second pin in the distal U-clamp (430) attaches clamps (442, 444) to the distal U-clamp (430) and allows the end effector to rotate as a whole (340A) about a second pivot axis (432). In some embodiments, the clamps (442, 444) may also pivot about the second pivot axis (432) in coordination with each other or as a whole (340A) with or separately from the end effector. The second pivot axis (432) may define a second pivot DOF (435) of the wrist assembly (320). In one or more embodiments, and as in Figure 4A As depicted, the second pivot axis (432) is perpendicular to the first pivot axis (422). In one or more embodiments, one of the first DOF (425) and the second DOF (435) of the wrist assembly (320) is referred to as the "pitch" DOF and the other as the "yaw" DOF. That is, the first pivot DOF (425) and the second pivot DOF (435) are capable of providing pitch and yaw degrees of freedom for manipulating the end effector (340).

[0057] exist Figure 4A In the illustrated example, a pair of clamps (442, 444) includes a first clamp (442) and a second clamp (444). Furthermore, as shown, the first clamp (442) and the second clamp (444) have surfaces for gripping, making these clamps suitable for applications such as tweezers or cauterization.

[0058] So far, roll, pitch, and yaw degrees of freedom have been described for instruments (160) with wrist components (320). As seen, degrees of freedom (DOF) are typically defined by geometric axes or planes. Furthermore, a DOF can be associated with one or more DOF variables, which can take values ​​relative to a given datum. Each DOF variable can have its own datum, and a datum can provide a fixed or relative frame of reference. In some cases, a DOF also has a range of motion (ROM), where the ROM defines the range of values ​​that the DOF variables can take. That is, for a given DOF with one DOF variable, the ROM of that DOF defines the limits of that DOF variable. For a kinematic sequence with more than one degree of freedom, the “pose” of the kinematic sequence can be specified by the values ​​of its DOF ​​variables containing degrees of freedom. Thus, the “pose” of a kinematic sequence such as a manipulator arm assembly (i.e., the manipulator arm and the instrument) can be defined by knowledge of the position and orientation of each of its components or joints and the geometry and arrangement of any interconnecting links between the joints. The pose of a kinematic sequence (or information about the position and orientation of each of its parts and / or joints (e.g., the value of each DOF variable)) can be stored in various mathematical or computational data structures such as tensors.

[0059] More specifically, consider the case where the device (160), including the wrist assembly (320), provides roll, pitch, and yaw degrees of freedom. In this case, the DOF variables for roll, pitch, and yaw degrees of freedom can be represented as φ, ψ, and ξ, respectively. Furthermore, the range of motion for the DOF variables of roll, pitch, and yaw degrees of freedom can be given as follows: , and The ROM of a DOF may be a result of mechanical constraints or limitations on components affected by or impacted by the DOF. Furthermore, the ROM of a given DOF can depend on the values ​​of DOF variables of another DOF. That is, the ROM of a given degree of freedom can depend on the pose of the kinematic sequence containing the given DOF. For example, the ROM of a machine's yaw DOF. It can depend on the value of the instrument's pitch DOF variable (i.e. and / or (It can be a function of ψ). The interdependence of two or more degrees of freedom on the ROM can be the result of mechanical interactions between the components of the kinematic sequence, whose pose is determined by the degree-of-freedom variables.

[0060] Typically, the end effector (340) is capable of providing zero or more additional degrees of freedom to the instrument (160). For example, Figure 4AThe first example end effector (440A) is described as having an "instrument grip" DOF (450). In this example, the instrument grip DOF (450) is defined by the "closed" and "open" relative to each other of the first clamp (442) and / or the second clamp (444). The relative closure and opening of the clamps (442, 444) can be implemented in a variety of ways. For example, in one or more embodiments, the instrument grip DOF has only one associated DOF variable, namely the angle θ between the first clamp (442) and the second clamp (444). In some embodiments, the instrument grip DOF variable is varied by the angular displacement of the first clamp (442) and / or the second clamp (444). For example, consider a notation in which the angular positions of the first clamp (442) and the second clamp (444) are given by θ1 and θ2, respectively. Furthermore, according to this marking method, a reference for the angular positions of the first clamp (442) and the second clamp (444) is established, such that the angular position change of the first clamp (442) that causes the first clamp (442) to "close" is determined by the following method. and the angular position change of the second clamp (444) that causes the second clamp (444) to "close". They have the same labeling. Then, the change in the instrument grip DOF variable. The sum of the angular position changes of the first clamp (442) and the second clamp (444) is given. In some implementations, the change in the device grip DOF variable... This is achieved through the equal angular displacement of the first clamp (442) and the second clamp (444), such that the changes in the angular positions of the first clamp (442) and the second clamp (444) are respectively... (Right now, and In some embodiments, one of the first clamp (442) or the second clamp (444) is stationary or fixed relative to the wrist assembly (320), the instrument axis (310), or other parts of the instrument (160) to which the clamp is attached. In these embodiments, the instrument grip DOF may define the movement of only one clamp (i.e., a non-fixed clamp). In other embodiments, the instrument grip DOF is associated with two instrument grip DOF variables, one for the first clamp (442) and the other for the second clamp (444). For example, the first grip DOF variable θ1 can be associated with the first clamp (442), and the second grip DOF variable θ2 can be associated with the second clamp (444). In one or more of these embodiments, the first grip DOF variable and the second grip DOF variable can be adjusted independently. In some embodiments, the ROM of at least one of the instrument grip DOF variables depends on the value of the other instrument grip DOF variable.

[0061] Depending on the type of device, an end effector may provide one or more functions, such as the firing of an applicator or activation of suction or flushing. In some examples, the functions provided by the end effector can be indicated as states. For example, the provided functions can be represented by a categorical variable that can take values ​​such as "on" and "off". Such functions can be associated with, for example, end effectors capable of supplying and emitting energy (e.g., energy-emitting end effectors). In some embodiments, the functions provided by the end effector are controlled by or mapped to the corresponding degree of freedom of an input device; this is described in more detail later in this disclosure. However, as an introductory example, where the function of the end effector can be represented by a binary categorical variable (e.g., capable of taking values ​​of "on" or "off"), the state of the binary categorical variable can be selected using the input device, or more specifically, using the associated or mapped DOF of the input device.

[0062] Figure 4B A second example end effector (440B) is shown, coupled to the instrument axis (410) via a wrist assembly (320). The second example end effector (440B) includes a single component forming a cauterizing hook. In some embodiments, the cauterizing hook is fixed relative to the distal portion of the wrist assembly (320).

[0063] Figure 4C A third example end effector (440C) is shown, coupled to an instrument axis (310) via a wrist assembly (320). The third example end effector (440C) includes a single component forming a nozzle of a flushing or suction device. In one or more embodiments, activation (i.e., functionality) of the flushing or suction device is selectively enabled using an input device.

[0064] Figure 4D A fourth example end effector (440D) is depicted, coupled to an instrument axis (310) via a wrist assembly (320). The fourth example end effector (440D) comprises two members forming a clamp-like arrangement of an applicator. In one or more embodiments, the fourth example end effector (440D) provides a DOF (e.g., an "instrument grip" DOF) that defines the angular spacing between the two members. In other embodiments, the fourth example end effector (440D) provides functions such as activation or "triggering" of the applicator. In one or more embodiments, the function of "firing" or quickly engaging the clamp is indicated by an input device.

[0065] Clamping end effectors, such as the first example end effector (440A), can take the form of any desired tool, such as having two components or fingers or clamps that pivot relative to each other, such as scissors, two-finger blunt-tipped dissecting tools, forceps, pliers used as needle actuators, or the like. In another example (not shown), the tips (442, 444) of the clamps are blades, such that a DOF-closed clamp relative to the instrument provides a cutting action, wherein the blades cooperate to cut like scissors. As will be described in more detail below, the end effector (340) is capable of performing different functions depending on whether it is attached to or otherwise used on or as part of the instrument (160).

[0066] Non-clamping end effectors, such as the second example end effector (440B) and the third example end effector (440C), may include a single working component, such as a scalpel, a cauterization electrode, a flushing device, etc. While end effectors including clamps, such as the fourth example end effector (440D), may have two clamp-like components, in one or more embodiments, end effectors including clamps are classified separately from clamping end effectors.

[0067] Other end effector configurations may exist without departing from this disclosure. The imaging device may also be considered an end effector (340) (i.e., for acquiring images) and, like other end effectors, may be coupled to the axis (310) of the instrument (160) with or without a wrist assembly (320).

[0068] As discussed, in some embodiments, the instruments (160) are classified according to their instrument type, wherein the instrument type is known by the specifications of the end effector (340) associated with or included in the instrument (160). Furthermore, the instrument type depends on the control or mapping of the input device holding DOF on the associated instrument, as described below.

[0069] The degrees of freedom (DOF) of the instrument (e.g., roll, pitch, yaw, and grip) can be controlled by appropriately positioned actuators (e.g., electric motors) in response to input from an associated input device. That is, in one or more embodiments, the input device is used to command or otherwise signal the value of the DOF variable of the instrument (160). In some embodiments, the DOF variable value is commanded by movement of the input device, where the user manipulates the input device. Furthermore, appropriately positioned sensors, such as encoders, potentiometers, etc., can be provided to enable measurement of the DOF variable of the instrument (160). That is, the sensors of the instrument (160) can be used to determine the pose of the instrument. The actuators and sensors can be disposed in the housing portion (330) and / or the instrument holder.

[0070] As mentioned above, the value of a DOF variable can be specified or controlled by manipulating the input device. (See later...) Figure 5 As described, the input device itself can have multiple degrees of freedom. In some embodiments, the degrees of freedom of the input device are mapped to other degrees of freedom of the computer-aided system (100), such as degrees of freedom associated with the instrument (160) (e.g., roll, pitch, yaw, grip, etc.). The degrees of freedom of the input device can have names similar to those of the degrees of freedom they control (i.e., the degrees of freedom of the manipulator arm and / or the instrument). Therefore, prior to these descriptions, the degrees of freedom of the instrument will be referred to as instrument degrees of freedom (e.g., instrument roll DOF, instrument grip DOF, etc.) and the degrees of freedom of the input device will be referred to as input device degrees of freedom (e.g., input device roll DOF, input device grip DOF) to avoid ambiguity.

[0071] The user input system (120) of the computer-aided system (100) may include one or more input devices. As described above, the computer-aided system (100) may include a control system (242) that processes input received from an operator (192) via the input system (120) to control aspects of the computer-aided system (100) (e.g., directly or remotely controlling the movement of one or more manipulator arms (150A, 150B, 150C, 150D) and / or one or more instruments supported by the manipulator arms). For example, the input devices of the input system (120) may be used to control the manipulator assembly (300) to position, orient, and / or manipulate the instrument (160). Typically, the control system (242) is capable of providing and / or facilitating mapping between input devices (e.g., movement of input devices and / or other input signals such as button presses) and components of the computer-aided system (100) such as the manipulator system (130). Furthermore, the behavior of the computer-aided system (100) can be influenced by enabling or disabling one or more modes and / or remote operation controls, wherein these modes and / or remote operation control conditions at least configure the mapping between the input devices of the input system and other components of the computer-aided system (100) such as the manipulator system (130).

[0072] In one or more embodiments, the user input system (120) of the computer-aided system (100) includes two input devices provided for user manipulation. In some embodiments, the input devices can each be configured to control the movement and function of an associated part of the manipulation system (130), such as a manipulator arm, a device supported by the manipulator arm, etc. For example, the input devices can move in multiple degrees of freedom to move the corresponding device of the manipulator system (130) in the corresponding degree of freedom (or, in some cases, specifically, the end effector of the device). In one or more embodiments, the input device grip DOF is mapped or otherwise associated with the device grip DOF or the function of the device. For example, in the case where the device has a device grip DOF, the value of the input device grip DOF variable can be mapped to the value of the device grip DOF variable. Thus, changing the value of the input device grip DOF variable, for example by the operator's manipulation of the input device, can change the value of the corresponding device grip DOF variable according to the provided mapping. As another example, when an instrument provides a function (e.g., energy emission, flushing, aspiration, etc.), the activation or state of the function (e.g., "on" or "off") can be indicated using the input device hold DOF (or, more specifically, the value of the input device hold DOF variable). For example, a threshold can be specified for the input device hold DOF variable such that determining whether the value of the input device hold DOF variable is less than or greater than a predefined threshold indicates the state of the function provided by the associated instrument or otherwise activates / deactivates the function provided by the associated instrument. In some embodiments, the input device is a manual input device that can be moved in six or more degrees of freedom. In some embodiments, the two input devices of the user input system (120) are positioned such that each is held or manipulated by the user's hand, specifically one input device per hand.

[0073] Figure 5 This is a perspective view of a controller portion (500) of an example input device that can be used as an input device according to one or more embodiments. In some embodiments, the controller portion (500) includes one or more universal joint mechanisms. In some embodiments, the input device consists of a kinematic sequence, such as a repositionable structure having multiple links coupled through one or more joints (e.g., Figure 5The controller section (500) typically includes sensors (e.g., encoders) coupled to components or joints of the input device to detect the position (and / or velocity, acceleration) of the components or joints in their respective degrees of freedom (i.e., input device degrees of freedom). Therefore, the pose of the input device can be defined by the position and / or orientation of each of its components or joints, along with knowledge of the geometry and arrangement of any interconnecting links between the joints of the input device. The pose of the input device (or information about the position and orientation of each of its components and / or joints (e.g., the value of each input device DOF variable)) can be stored in various mathematical or computational data structures such as tensors.

[0074] exist Figure 5 In this example, the controller portion (500) includes a handle (502) that is touched by a user to manipulate an input device. Furthermore, in this example, the handle (502) includes two grips, a first grip member (506A) and a second grip member (506B), each including a finger ring (504). The two grip members (506A, 506B) are positioned on opposite sides of the central portion (503) of the handle (502), and the grip members (506A, 506B) can be grasped, held, or otherwise contacted by the user's fingers. Each finger ring (504) is attached to the corresponding grip member (506A or 506B) and can be used to secure the user's finger to the associated grip member (506A or 506B). In this example, a finger contact (505) can be attached to or formed at the unattached end of the grip member (506A, 506B) to provide a surface for contacting the user's finger. Users can also touch other parts of the handle (502) while gripping the gripping components (506A, 506B).

[0075] exist Figure 5In the example, the first grip member (506A) is movable with a first input device grip degree of freedom (DOF) defined as a pivotal motion about a first grip axis (507A), which indicates rotational coupling between the first grip member (506A) and the center portion (503). Similarly, the second grip member (506B) is movable with a second input device grip degree of freedom (DOF) defined as a pivotal motion about a second grip axis (507B), which indicates rotational coupling between the second grip member (506B) and the center portion (503). In various embodiments, a single grip member (e.g., 506A) and a ring (504) may be provided, or only one grip member (e.g., 506A) may be movable with a corresponding input device grip degree of freedom (e.g., 508A), while the other grip member (e.g., 506B) may be fixed relative to the handle (502). For example, the position of the gripping components (506A, 506B) in their input device degrees of freedom can control the corresponding rotational position of the instrument or its components.

[0076] One or more grip sensors (not shown) are coupled to the handle (502) and / or other components of the controller section (500) and are capable of detecting the position of the gripping members (506A, 506B) in their respective degrees of freedom (508A, 508B). The grip sensors are capable of sending signals describing the sensed position and / or motion to the control system (242) of the computer-aided system (100). In some modes (e.g., the remote operation mode described later in this disclosure) or embodiments, the control system (242) is capable of providing control signals to the device manipulated by the computer-aided system (100). For example, the position of the gripping members (506A, 506B) in their respective degrees of freedom (508A, 508B) can be used to control any of the various degrees of freedom of an instrument controlled by the manipulator system (130) (or, in some cases, the end effector of the instrument).

[0077] Various implementations of the input device, such as in Figure 5The input device described in the controller section (500) is capable of providing one or more active actuators (e.g., motors, voice coils, etc.) to output active force on the gripping members (506A, 506B) in their degrees of freedom (508A, 508B). For example, sensors and / or actuators can be housed in a central portion (503) or a housing (509) and coupled to the gripping members (506A, 506B) via a transmission. Some embodiments may provide one or more passive actuators (e.g., brakes) or springs between the gripping members (506A, 506B) and the central portion (503) of the handle (502) to provide resistance in a particular direction of the gripping members (e.g., movement in directions toward each other in their degrees of freedom (508A, 508B)). The actuators of the input device can be considered as one or more actuators of a computer-aided system.

[0078] The handle (502) can be further provided with a rotational degree of freedom, or input device roll DOF (510), about a rolling axis of an input device (512) defined between a first end and a second end of the handle (502). That is, the input device roll axis (512) in this example is a longitudinal axis extending approximately along the center of the central portion (503) of the handle (502). The handle (502) is rotatable about the input device roll axis (512) relative to a base member of the controller portion (500), such as the base member of the housing (509) including the input device. For example, a user can rotate the gripping members (506A, 506B) and the central portion (503) as a single unit about the input device roll axis (512) relative to the housing (509) to provide control of a manipulator assembly such as an instrument roll DOF.

[0079] Additionally, one or more input sensors (not shown) can be coupled to the handle (502) to detect the orientation of the handle (502) in the input device roll DOF (510). For example, the sensors can send signals describing the orientation to the control system (242), which can provide control signals to the manipulator system (130) as described above. For example, rotation of the handle (502) in the input device roll DOF (510) can control a specific degree of freedom of the instrument (160) of the manipulator system (130), which is different from another degree of freedom controlled by the input device grip degree of freedom (508A, 508B) of the gripping members (506A, 506B) (e.g., the instrument roll DOF).

[0080] Some implementations of the controller section (500) are capable of providing one or more actuators to output force on a handle (502) (including grip members (506A, 506B) and a finger ring (504)) in the input device rolling DOF (510). For example, a sensor and / or actuator may be housed in a housing (509) and coupled to the handle (502) via a shaft extending through a central portion (503) of the handle (502).

[0081] In various embodiments, the handle (502) can be provided with additional degrees of freedom. For example, a rotational degree of freedom, namely an input device yaw DOF (520) about the input device yaw axis (522), can be provided to the handle (502) at a rotational coupling between the elbow link (524) and the link (526), ​​where the elbow link (524) (e.g., at the housing (509)) is coupled to the handle (502). In this example, the input device yaw axis (522) intersects and is orthogonal to the input device roll axis (512). Similarly, additional degrees of freedom can be provided. For example, the link (526) can be elbow-shaped and a rotational coupling can be provided between the other end of the link (526) and another link (not shown). Another rotational degree of freedom (528) about axis (530) can be provided to the handle (502) at the rotational coupling.

[0082] In some examples, the controller section (500) may allow the handle (502) to move within the workspace of the user input system (120) in multiple degrees of freedom (e.g., six degrees of freedom including three rotational degrees of freedom and three translational degrees of freedom). One or more additional degrees of freedom can be sensed by associated input sensors and / or actuated by actuators (motors, etc.), similar to those described above for degrees of freedom (508A, 508B, 510, etc.), with the sensors and actuators coupled to the controller section (500). In various embodiments, the sensors are capable of sensing the position of the handle in a degree of freedom, or sensing the orientation of the handle in a degree of freedom, or sensing the position and orientation of the handle in multiple degrees of freedom. For example, the position in the translational degree of freedom and the orientation in the rotational degree of freedom can be sensed by one or more associated input sensors. In some examples, the position in the translational degree of freedom and / or the orientation in the rotational degree of freedom can be derived from the rotation of a component (e.g., a linkage) coupled to the handle (502) sensed by a rotational sensor. Some implementations may include a linear sensor capable of directly sensing translational motion of one or more components coupled to the handle (502). In some implementations, each additional degree of freedom of the handle (502) is capable of controlling a different degree of freedom (or other motion) in the manipulator system (130).

[0083] In an exemplary embodiment, the handle (502) is mechanically grounded, i.e., supported in space by a kinematic chain, one end of which is fixed to a mechanical foundation such as a floor, wall, or ceiling. For example, the housing (509) can be coupled to a mechanical linkage that is coupled to or connected to the ground, thereby providing a stable platform for the use of the controller section (500). For example, the grounded mechanical linkage can be connected to a base structure, such as via one or more rotary couplers, ball-and-socket joints, or other couplers, including linear joints. The mechanical linkage can provide six or more degrees of freedom for the handle (502).

[0084] exist Figure 5 In the example, the handle (502) includes one or more control switches (550). As shown, one or more control switches (550) can be coupled to a mechanism within the central portion (503) or the central portion (503). For example, two control switches (550) can be positioned on opposite sides of the scroll axis (512), and / or additional control switches can be provided. In some examples, the control switch (550) has a portion that can slide parallel to the scroll axis (512), for example guided by a user's finger, or the control switch portion can be pressed down. In some embodiments, the control switch (550) can be moved to various positions to provide specific command signals, such as selecting the function, option, or mode of the input system (120) and / or input device. In some embodiments, one or more of the control switches (550) can be implemented as buttons (e.g., pressed in a direction such as perpendicular to the scroll axis (512) of the input device or other directions), rotary dials, switches that move perpendicular to the scroll axis (512) of the input device, or other types of input controls. The control switch (550) can use an electromagnetic sensor, mechanical switch, magnetic sensor or other type of sensor to detect the position of the switch.

[0085] As mentioned above, Figure 5 The described example input device includes a first grip member (506A) and a second grip member (506B). One or more grip sensors (not shown) are capable of being coupled to the handle (502) and / or other components of the controller section (500) and are capable of detecting the position of the grip members (506A, 506B) in their respective input device grip degrees of freedom (508A, 508B). The grip sensors are capable of sending signals describing the sensed position and / or motion to the control system (242) of the computer-aided system (100). In some cases, the control system (242) is capable of providing control signals to a device manipulated by the computer-aided system (100), such as a manipulator assembly. For example, consider Figure 4AIn some embodiments of the first example end effector (440A) depicted, the position of the first input device gripping member (506A) in the first input device grip DOF can be used to control the position of the first clamp (442) in the associated first instrument grip DOF. Similarly, in these embodiments, the position of the second input device gripping member (506B) in the second input device grip DOF can be used to control the position of the second clamp (444) in the associated second instrument grip DOF. Thus, in this example, the clamp of the end effector can be independently controlled via its relationship to two independent input device degrees of freedom (e.g., using the first gripping member and the second gripping member (506A, 506B)). In another example, the first input device grip DOF (508A) and the second input device grip DOF (508B) are combined into a single input device grip DOF, which can be mapped to (i) a single device grip DOF, (ii) the execution of a function or a change in device state, condition, or parameters, or (iii) no device DOF and no function, based on the operating mode and / or the type of the associated instrument. For example, the relative values ​​of the first and second input device grip DOF variables can be used to determine the separation angle between the first gripper (442) and the second gripper (444) of a clamp-type end effector (e.g., the first example end effector (440A)). In this way, two input device degrees of freedom can be used collaboratively as a single input device degree of freedom to, for example, control a single instrument degree of freedom. For example, the separation angle between the first gripping member (506A) and the second gripping member (506B) can be used as an input device grip DOF. In some implementations, the input device has only one input device grip degree of freedom (DOF) (e.g., an input device with only one gripping member, an input device with two gripping members whose movement is mechanically linked (e.g., moving together but in opposite directions), etc.). In some implementations, one or more input device grip degrees of freedom can be configured to control the non-grip functions of the instrument and / or change the instrument state, condition, or parameters based on the operating mode and / or the associated instrument type. For example, one or more input device grip degrees of freedom can be used to determine the state of an end effector having more than one state, such as "on" or "off" associated with functions such as flushing and energy emission. As another example, one or more input device grip degrees of freedom can be used to actuate the function or operation of the end effector, such as "firing" the applicator.

[0086] As discussed, in some embodiments, the instruments (160) are classified according to their instrument type, wherein the instrument type is known by the specifications of the end effector (340) associated with or included in the instrument (160). Furthermore, the instrument type depends on the control or mapping of the associated instrument by the input device grip DOF. In one or more embodiments, the instrument type is one of a first instrument type, a second instrument type, and a third instrument type. Instruments classified as the first instrument type are those that have an instrument grip DOF, wherein the value of the instrument grip DOF variable is set based on the value of the corresponding input device grip DOF variable. That is, an input device with an input device grip DOF has (e.g., using the control system (242)) an input device grip DOF variable mapped to the instrument grip DOF variable. Instruments classified as the second instrument type are those that do not necessarily have an instrument grip DOF (in terms of one or more components that can be hinged in a clamping manner) but have functionality. Example functions include energy application (e.g., energy-emitting end effector), flushing, suction, and firing clips, as well as other functions. For devices of type 2, the device hold DOF can be used to control or otherwise indicate the state, condition, or parameters of the device. For example, when the value of the associated input device hold DOF variable is below a predefined threshold, the device function can be enabled or activated (e.g., "on"), while when the value of the input device hold DOF variable is equal to or higher than the predefined threshold, the device function can be disabled or deactivated (e.g., "off"). In this way, the input device hold DOF can be said to be mapped to the device function. Devices classified as type 3 are those devices for which the value of the input device hold DOF variable corresponding to the device's input device has no effect on the device. That is, for devices of type 3, the input device hold DOF does not map to the device hold DOF (because there is none) or the device function.

[0087] To summarize the preceding discussion, in a simplified manner, instruments can be classified based on how the input device hold DOF (Designated Point of View) of the input device used to control the instrument is mapped to the control or function of the instrument. Instruments whose input device hold DOF is mapped to the instrument hold DOF are classified as Type 1 instruments. Instruments whose input device hold DOF is mapped to the function of the instrument (e.g., setting a state, condition, or parameter) are classified as Type 2 instruments. Instruments whose input device hold DOF is not mapped to any aspect of the instrument are classified as Type 3 instruments. Examples of instruments for each of the Type 1, Type 2, and Type 3 instrument types are given below.

[0088] For example, if the first device has a gripper end effector, it can be classified as a first device type. This is because gripper end effectors typically have a device grip DOF, allowing the value of the device grip DOF variable to be set or commanded to a value based on the value of the corresponding input device grip DOF variable. That is, the input device grip DOF variable can be mapped to the device grip DOF variable. Therefore, input in the input device grip DOF can cause movement or actuation of the device grip DOF. As another example, if the second device has a suction flushing end effector, it can be said that the device type of the second device is the second device type if the input device grip DOF is used to activate the suction flushing device. Some devices that provide a certain function, such as a suction flushing device, can be classified as a third device type if the function is not controlled by mapping to the input device grip DOF. That is, in some examples, the state, condition, or parameters of the device can be indicated by other means such as foot pedals, buttons, or switches of the user input system (120). As another example, if a third device has an applicator end effector and the applicator is triggered or fired based on the value of the associated input device holding DOF variable, then the third device is a second device type. In some embodiments, the listed device types form a completely exhaustive list, such that any device can be classified according to at least one device type. Additionally, in some embodiments, the list of device types is both completely exhaustive and mutually exclusive (i.e., each device has exactly one device type).

[0089] The embodiments disclosed herein relate to one or more input devices configured to be controlled or manipulated by a user, the input devices having one or more actuable degrees of freedom. Furthermore, embodiments of this disclosure are described below with respect to a single input device. However, it should be noted that this choice was made for clarity and brevity and does not impose any limitation on this disclosure. It is to be emphasized that the embodiments disclosed herein are readily applicable to input systems (120) that include more than one input device.

[0090] As described above, the input device has one or more actuable degrees of freedom (DOFs). The input device may also have non-actuable degrees of freedom (DOFs). An actuated DOF is coupled to or otherwise associated with one or more active actuators (e.g., a motor, voice coil, etc.) such that the actuated DOF can be commanded to a specified value relative to its DOF ​​variable. An actuated DOF may also include, or be associated with, one or more passive actuators (e.g., a brake) or springs. A non-actuated DOF may include, or be associated with, one or more passive actuators (e.g., a brake) or springs. Actuators coupled to or associated with the input device, such as the active actuators described above, can be considered as one or more actuators of a computer-aided system.

[0091] Typically, the pose of an input device is described by the DOF value of each DOF variable associated with the input device. Therefore, the pose of an input device can include DOF values ​​for both operable and non-operable degrees of freedom. One or more sensors (e.g., encoders) are used to determine the DOF values, said sensors being coupled to parts or joints of the input device to detect the position (and / or velocity, acceleration) of those parts or joints in their respective degrees of freedom. Thus, in some embodiments, the pose of the input device represents the state of the input device, which is read from sensors of the input device or otherwise determined. The distinction is that the “configuration” of the input device is referred to as a set of DOF values ​​for the operable degrees of freedom of the input device. The set of DOF values ​​for the operable degrees of freedom of the input device does not need to be the same as the determined or observed DOF values ​​for the same operable degrees of freedom. For example, consider a case where the input device has a range of motion (ROM). Input device holding DOF variables The input device hold DOF is represented by the superscript "id" in parentheses, indicating that the DOF variable is associated with the input device. The pose of the input device can then indicate the value that the input device hold DOF variable has. ,in Similarly, the configuration of the input device can specify the values ​​used by the input device to hold DOF variables. ,in and It does not need to be equal to .Right now, This indicates that the input device holds the observations of the DOF variable, and This represents a specified value (or setting) for the input device's grip DOF variable. An input device grip DOF that has been defined as actuable can be commanded to a given value. Therefore, in one or more embodiments, the actuable degrees of freedom of the input device are commanded from its current pose to a predetermined configuration. More than one configuration can be defined for the input device. For example, the input device can be associated with a first configuration, a second configuration, and so on, each configuration "containing" a set of specified values ​​for the input device's actuable degrees of freedom.

[0092] In summary, the input device has N actuable degrees of freedom, where N is an integer greater than or equal to 1 (i.e., the input device has at least one actuable degree of freedom). For simplicity, each degree of freedom of the input device is said to be associated with a DOF variable. To represent the degrees of freedom (DOF) of an input device, for It can list actuable DOF variables as Similarly, an input device can have M non-actuable degrees of freedom, where M is an integer greater than or equal to 0. For simplicity, each degree of freedom of the input device is said to be associated with a DOF variable. Let β represent the DOF variable of the non-actuable degrees of freedom of the input device. When M=0, the non-actuable DOF variables can be listed as an empty set [], or when M≥1, the non-actuable DOF variables can be listed as []. Using this notation, the pose of the input device can be represented as a joint list of observed actuated degrees of freedom and observed non-actuated degrees of freedom, for example... Similarly, a given configuration of the input device (e.g., a first configuration, a second configuration, etc.) can be represented as a list of specified actuable degrees of freedom values. As can be seen, observed DOF values ​​(i.e., poses) are distinguished by a "hat symbol," and specified degrees of freedom (i.e., configurations) are distinguished by a "wavy line symbol." In some embodiments, one or more actuable degrees of freedom may not have specified DOF values, but may be allowed to be manipulated to any DOF value or may be maintained at a DOF value depending on the current pose of the input device. In other words, the configuration does not need to include specified DOF values ​​for each actuable degree of freedom of the input device. Furthermore, the specified DOF values ​​of the actuable degrees of freedom can be determined based on the pose of the input device. Therefore, the action of commanding the input device to a given configuration includes, where applicable, actuating one or more actuable degrees of freedom to their specified DOF values. In one or more embodiments, the action of commanding the input device to a given configuration includes actuating the actuable degrees of freedom to such that In summary, the input device is actuable in one or more degrees of freedom, including at least the first degree of freedom (DOF).

[0093] As described above, the computer-aided system (100) can include a control system (242) that processes inputs received from an operator (192) via an input system (120) to control aspects of the computer-aided system (100) (e.g., direct movement of manipulator arms (150A, 150B, 150C, 150D)). For example, the input devices of the input system (120) can be used to control the manipulator assembly (300) to position, orient, and use the instrument (160). Typically, in order to control aspects of the computer-aided system (100), the control system (242) can provide and / or facilitate the mapping of movement and other input signals (e.g., button presses) between the input devices and components of the computer-aided system (100), such as the manipulator system (130). Similar to the degrees of freedom of the input devices, the controllable degrees of freedom of the computer-aided system (e.g., instrument roll DOF, instrument pitch DOF, etc.) can be listed as Where K is an integer and represents the number of controllable degrees of freedom (usually K≥1), and express Controllable DOF variables.

[0094] In one or more embodiments, the degrees of freedom (DOF) of the input device are mapped to the controllable DOFs of the computer-aided system (100). In other words, the values ​​of controllable DOF variables can be commanded or set to specific values ​​based on the values ​​of the input device DOF variables, regardless of whether the input device's DOFs are considered actuable or non-actuable. In some embodiments, function notation can be used to represent the mapping of one or more input device DOFs to one or more controllable DOFs. For example, in some examples, controllable DOFs (e.g., ...) Controllable DOF variables) and input device DOF (e.g.) There can be a one-to-one relationship between input device (DOF) variables, making... Where f is a function. As another example, in some cases, two or more input device degrees of freedom are mapped to a single controllable DOF. Using function notation, this relationship can be expressed as... Where g represents a function, and i, j, and k index specific DOF variables and For example, input devices (such as...) Figure 5 The depicted input device has both a first input device degree of freedom (508A) and a second input device degree of freedom (508B). In abbreviated form, these input device grip degrees of freedom (508A, 508B) can be written as... and In some cases, input devices can be used to control instruments with a single degree of freedom in grasping the instrument, succinctly written as... Therefore, in some cases , where h represents a function. As another example, a controllable DOF can be associated with one or more input device degrees of freedom, while being parameterized by one or more other input device degrees of freedom. For example, Here, u represents a function, and the indices i, j, and k specify DOF variables. In another example, one or more controllable degrees of freedom can be associated with or parameterized by one or more other controllable degrees of freedom. For example, Where v represents a function, and the indices i, j, k, w, and x specify DOF variables, and Functions relating to one or more input device degrees of freedom and controllable degrees of freedom can take many forms, including linear, nonlinear, and piecewise. In some examples, the relationship between the input device degrees of freedom and one or more controllable degrees of freedom cannot be performed using functions. In some implementations, lookup tables are used to perform the mapping between one or more controllable degrees of freedom and one or more input device degrees of freedom. In one or more implementations, the mapping between input device degrees of freedom and controllable degrees of freedom is only valid when remote operation control between the input device and the controlled component of the computer-aided system (100) is enabled. Further details regarding enabling and disabling remote operation control are provided later in this disclosure.

[0095] Figure 6 An exemplary mapping (600) is depicted between input device degrees of freedom (602) (e.g., input device grip DOF) and corresponding controllable degrees of freedom (i.e., instrument DOF) (604), wherein the input device degrees of freedom are defined by having a range of motion (ROM). The degree of freedom variable θ represents the controllable degree of freedom, while the corresponding controllable degree of freedom is represented by the ROM. The degrees of freedom variable Ω is represented. To demonstrate how the mapping between the input device DOF (602) and the instrument DOF (604) occurs, the four values ​​for the input device DOF variable are depicted as points. Specifically, these points are the first input device point (612), the second input device point (614), the third input device point (616), and the fourth input device point (618). Similarly, the three values ​​for the instrument DOF variable are described as points. These points are: the first instrument point (622), the second instrument point (626), and the third instrument point (628). Figure 6 The dashed lines in the diagram illustrate the correspondence between the four input device points (612, 614, 616, 618) and the three instrument points (622, 626, 628). As an example, after this mapping (600), when remote operation control is enabled and the input device DOF variable has a value at the third input device point (616), the instrument DOF variable should have a value at the second instrument point (626). Therefore, when remote operation control is enabled, the movement of the input device, captured by the change in the input device DOF variable θ, is propagated to the instrument DOF variable Ω.

[0096] Figure 6The example mapping (600) can be further divided into three regimes: an irreversible regime (606), a linear regime (608), and a nonlinear regime (610). In the depicted example, the irreversible regime (606) behaves such that more than one value of the input device DOF variable is mapped to the same value of the instrument DOF variable. As can be seen, both the first input device point (612) and the second input device point (614) are mapped to or correspond to the first instrument point (622). Furthermore, in the depicted example, the first instrument point (622) is located at the lower bound of the ROM for the instrument DOF variable. In some implementations, a variable force can be applied to an input device component (e.g., a gripping member (506A, 506B)) associated with the input device DOF to distinguish between a first input device point (612) and a second input device point (614). For example, consider the case where the input device DOF is an input device grip DOF implemented using a gripping member (e.g., a first gripping member (506A)) and the instrument DOF is an instrument grip DOF implemented using a clamp-type instrument (or a clamp-type end effector). Furthermore, consider the case where the lower limits of the input device DOF and instrument DOF variables correspond to the mechanically closed position. Then, in this example case, the first instrument point (622) indicates that the clamp of the instrument is fully closed; however, the gripping member controlling the input device (e.g., the first gripping member (506A)) can still move in the "closed" direction. Therefore, in some examples, the instrument DOF variable can be maintained at its ROM limit without needing to maintain the input device DOF variable at its ROM limit. In addition, it can provide the user with tactile feedback force (possibly having a variable force through an irreversible boundary (606)) or other audio or visual cues to indicate that the device's DOF ​​variable is at its ROM limit.

[0097] Continuing with the example, in the linear bound (608), the change in the instrument DOF variable corresponding to the change in the input device DOF variable is constant with respect to the value of the input device DOF variable. To illustrate this behavior, Figure 6 Two line segments are depicted near the third input device point (616), where the magnitude of the change in the input device DOF variable is... The superscript indicates a change in the "input device" DOF variable. As can be seen, these displacements in the input device DOF variable are equal in magnitude, both corresponding to changes in the instrument DOF variable with equal magnitudes. In particular, the magnitude changes in the input device DOF variable... The change in magnitude is shown as a change in the instrument's DOF ​​variable. Correspondingly, the superscript "inst." indicates that the displacement occurs in the instrument's DOF ​​variable. It is worth noting that... It does not need to be equal to Conversely, in the nonlinear bound (610), the change in the instrument DOF variable corresponding to the change in the input device DOF variable depends on the value of the input device DOF variable. To illustrate this behavior, Figure 6 Two line segments are depicted near the fourth input device point (618), where the magnitude of the change in the input device DOF variable is... and ,in As can be seen, these displacements in the input device DOF variables, although not equal in magnitude, all correspond to changes in the device DOF variables with equal magnitudes. Specifically, based on the values ​​of the input device DOF variables, there are magnitudes... and Changes in the input device DOF variable are shown as changes in the magnitude of the instrument DOF variable. Correspondingly, the nonlinear boundary (610) can be beneficial for tuning the sensitivity of the instrument's DOF ​​in response to movement of the control input device (or more specifically, in response to changes in the value of the corresponding input device's DOF ​​variable).

[0098] In some implementations, “remote operation control” indicates the control of one or more components (e.g., manipulator components) of a computer-aided system (100) through user interaction with the input devices of the input system (120). In some implementations, remote operation control is enabled by the user satisfying one or more remote operation control enable conditions. For example, remote operation control enable conditions may include the user selecting a remote operation mode (e.g., using a graphical user interface, pressing a button, etc.). In some implementations, remote operation control can be enabled without a specifically specified mode (e.g., remote operation mode). In some examples, the process of enabling remote operation control of the computer-aided system (100) may require the user to satisfy additional conditions, such as moving the input device in a specified direction or movement (e.g., via a specific input device DOF), and ensuring that the input device and its associated controllable components (e.g., manipulator components (300)) are synchronized and / or properly aligned according to a mapping relating to their movement (e.g., see [link to relevant documentation]). Figure 6Example mapping (600)). Therefore, in some implementations, it can be said that in order to enable remote operation control of the computer-aided system (100), one or more remote operation control enable conditions must be met, wherein remote operation control enable conditions can include a number of related (and possibly sequential) processes. For example, remote operation control enable conditions may require the user to first indicate (e.g., via a selection mechanism) the expectation of initiating remote operation (e.g., selection of a remote operation mode), followed by user-guided movement of the input device by specifying movement (e.g., moving the grip or roll axis through its degrees of freedom), and then, according to some mapping, to move the input device to be aligned with the associated manipulator component controlled by the input device. In this example, where the selection of a remote operation mode is a condition for remote operation control, the disabling (or exiting) of remote operation control (i.e., no longer enabling remote operation control) does not need to be associated with or necessarily result in the exit from the remote operation mode. That is, in some implementations, the remote operation mode persists even when remote operation control is disabled. In some implementations, remote operation control can be re-enabled by satisfying remote operation control enable conditions, wherein in some cases, the selection of a remote operation mode may have already been satisfied. In other embodiments, disabling (or exiting) remote operation control triggers an exit or switch from remote operation mode to non-remote operation mode. In some embodiments, when remote operation control is disabled (or not enabled), accessible input device commands (e.g., signals corresponding to button presses) associated with movement of the input device by the input system (120) are not transmitted by the control system (242) to other components of the computer-aided system (100). For example, an input device DOF variable may undergo a change in value (i.e., through movement of the input device) without being mapped to a corresponding instrument DOF variable command or propagating the change in value according to a predefined mapping.

[0099] When remote operation control is enabled and disabled (or not enabled) (where disabling remote operation control may still keep the computer-aided system (100) or control system (242) in remote operation mode), the operation of the control system (242) can vary depending on the device type of the instrument (160) controlled by the computer-aided system (100). For example, in the context of a computer-aided system (100) as a medical system, various medical instruments (160) can be attached to and manipulated by a manipulator system (130). Examples of such medical instruments (160) can include, but are not limited to, clamps, needle actuators, aspiration and / or flushing tools, grippers, scissors, staplers, endoscopes, and energy emission devices (e.g., electrocautery or ablation devices). In many cases, the instrument is identified by the end effector it employs. Each instrument can have one device type. As previously described, example device types can include: a first device type; a second device type; and a third device type. For example, in a state where remote operation control and / or non-remote operation mode are disabled, the actuation degrees of freedom of the input device can be commanded to values ​​that depend on the determined device type and / or the device's pose (i.e., the values ​​of the device's DOF ​​variables) intended to be controlled by the input device (160). That is, the input device can be commanded to a given configuration (e.g., a first configuration, a second configuration, etc.), wherein the given configuration includes specified values ​​for one or more of the actuation degrees of freedom of the input device, which depends at least on the device type of the device to be controlled by the input device.

[0100] Generally, enabled remote operation control (or, without ambiguity, enabled remote operation control can simply be referred to as "remote operation control") allows control of the manipulator system (130) or a portion thereof (e.g., the manipulator arm and / or the apparatus supported by the manipulator arm) via user interaction with an input device. Similarly, generally, disabled remote operation control disconnects control of the manipulator system (130) from the input device. As mentioned above, one or more remote operation and non-remote operation modes may be associated with the remote operation control of the computer-aided system (100), but this is not required. As will be described below, the behavioral differences of the computer-aided system (100) in the state of enabled remote operation control can depend on (or dynamically change based on) factors such as the type and / or state (e.g., apparatus type, apparatus pose, etc.) of the apparatus (160) employed in the computer-aided system (100).

[0101] Generally, the embodiments disclosed herein relate to methods for determining one or more configurations of an input device (i.e., specified values ​​of one or more actuable degrees of freedom of the input device), remote operation control enable conditions, and remote operation control behavior, wherein these items depend at least on the device type of the device controlled (or designated as being controlled) by the input device.

[0102] Specifically, the embodiments disclosed herein include an input device having at least one actuable degree of freedom (DOF). This allows the at least one DOF to be commanded to a given configuration, wherein the given configuration includes a specified value for at least one DOF variable of the at least one DOF of the input device. One or more configurations of the input device can depend on the instrument type of the apparatus controllable by the input device. In some embodiments, the three instrument types are considered to be: a first instrument type, a second instrument type, and a third instrument type. In some embodiments, the instrument types are mutually exclusive and form an exhaustive list.

[0103] According to one or more embodiments, given an input device configured to control an instrument, the instrument type of the instrument is determined. Furthermore, a first configuration for the input device is determined, at least based on the instrument type of the instrument controlled (or designated as being controlled) by the input device. The first configuration specifies a value (i.e., a predefined value) of at least one of one or more actuable DOF variables associated with the input device. In one or more embodiments, the input device is commanded or caused to be actuated to the first configuration. That is, the input device is actuated in one or more actuable degrees of freedom to one or more predefined values ​​included in the first configuration (e.g., such that one or more DOF variables of one or more actuable degrees of freedom match one or more predefined values). Actuation of the input device to the first configuration can occur in various scenarios, including but not limited to: selection of a remote operation mode for the control system (242) and / or the computer-aided system (100); enabling remote operation control of the instrument (and / or at least a portion of the manipulator assembly) via the input device; changing the designation of the input device to control a different instrument; detecting the presence of a user at the input system when the presence of a user was not previously detected; determining that the instrument designated to be controlled by the input device has been replaced with another instrument. For example, in some embodiments, the input device includes or is monitored by a hand presence sensor. The hand presence sensor can indicate the presence of a user's hand near the input device, which is configured to be manipulated by the user's hand. Thus, in some embodiments, in response to the hand presence sensor detecting the presence of a user's hand after it was previously not detected, the input device is actuated to a first configuration, which depends at least on the type of instrument specified by the input device for control. That is, a hand presence sensor whose output changes from an indication that the hand is not present at the input device to an indication that the hand is present at the input device can cause the input device to be actuated to the first configuration. As another example, the input device is specified to control a manipulator assembly including an instrument. During a program implemented by or executed using a computer-aided system (100), the instrument controlled by (or specified for control by) the manipulator assembly by the input device is replaced with another input device. This may occur, for example, when a different instrument (160) is desired during the program. Therefore, the instrument (160) is removed from its associated manipulator arm (e.g., 150A, 150B, 150C, 150D) and replaced with another instrument (160) having the desired end effector. The instrument (160) can also be removed and reinserted, for example, to clean the instrument, inspect the instrument, reload the instrument (e.g., by loading the applicator with a clamp, loading the stapler with a stapler cartridge), etc.In this case, it is possible to determine the instrument type of the newly coupled instrument to the manipulator arm and cause the input device (or command the input device) to be actuated to the newly determined first configuration, wherein the newly determined first configuration is based at least on the instrument type of the newly coupled instrument.

[0104] As described above, the input device includes at least a first input device DOF. In one or more embodiments, the first input device DOF is an input device holding DOF. For example, Figure 5 The input device depicted has a first gripping member (506A) and a second gripping member (506B). As previously described, the relative movement of the first gripping member (506A) and the second gripping member (506B) can be used to define and determine an input device gripping DOF with an associated gripping DOF variable having a ROM. Such an input device gripping DOF can be described as "open" and "closed". Or more specifically, the input device gripping DOF variable can have a ROM, wherein the ROM limits (i.e., the lower and upper limits of the input device gripping DOF variable) are specified as "open" or "closed". For example, the gripping members (506A, 506B) can be "pressed" or biased toward the central portion (503) until their respective mechanical limits are reached; in this case, the associated input device gripping DOF variable is referred to as "closed". Similarly, the gripping members (506A, 506B) can be "pulled out" or biased away from the central portion (503) until their respective mechanical limits are reached; in this case, the associated input device grip DOF variable is referred to as "open". An input device grip DOF variable having a ROM operating between the "open" and "closed" states can be equipped with only one gripping member. Furthermore, an input device having two or more gripping members (e.g., 506A, 506B) can have two or more input device grip DOF variables, each having a ROM operating between the "open" and "closed" states.

[0105] In cases where the device type for an instrument controlled by an input device (or designated for control by an input device) is determined to be a first device type, a predetermined value for the input device grip DOF variable included in the first configuration is determined. In one or more embodiments, when the device type is determined to be the first device type, the predetermined value is offset relative to the alignment position. The offset is defined using an offset distance. In some embodiments, the predetermined value is offset relative to the alignment position toward the open position or state of the input device grip DOF variable by an offset distance. That is, in these embodiments, the offset is offset toward the open position of the input device grip DOF. In other embodiments, the offset causes the input device grip DOF variable to be offset toward the closed position of the input device grip DOF. In one or more embodiments, the alignment position is determined based on the value of the corresponding device DOF variable and a mapping that associates the input device grip DOF variable with the device grip DOF variable. Specifically, the alignment position represents the value of the input device grip DOF variable, which (using a given mapping or mapping diagram) is "aligned" or mapped to the observed value of the corresponding device grip DOF variable. Therefore, when the instrument type specified by the input device for control is a first instrument type, the alignment position of the input device holding DOF variable can be determined. Then, a first configuration can be determined, wherein the first configuration includes a predetermined value for the input device holding DOF variable, which is offset from the alignment position by a predefined offset distance. In one or more embodiments, the offset causes the input device holding DOF variable to be biased toward an open position or state of the input device holding DOF relative to the alignment position. Furthermore, according to one or more embodiments, the input device holding DOF can be actuated to the first configuration in response to one or more triggers (e.g., enabling remote operation control of the instrument via the input device, the control system (242) entering remote operation mode, instrument replacement, etc.).

[0106] Figure 7 An exemplary mapping (700) between an input device grip DOF variable (702) and an instrument grip DOF variable (704) is shown, wherein an input device is specified to control an instrument of an instrument type having a first instrument type. As can be seen, the input device grip DOF variable θ has a ROM defined by "closed" and "open" states. Similarly, the instrument grip DOF variable Ω has a ROM defined by "closed" and "open" states. The example mapping (700) associates the value of the input device grip DOF variable (702) with the value of the instrument grip DOF variable (704). Figure 7 The observed input device grip DOF value (740) is depicted. In other words, the observed input device hold DOF value (740) indicates the actual value (or position / or orientation) of the input device hold DOF variable. The observed input device hold DOF value (740) is included in the input device's "pose". Similarly, Figure 7 The observed instrument grip DOF value (750) is shown. In other words, the observed instrument grip DOF value (750) indicates the actual value (or position / or orientation) of the instrument grip DOF variable. The observed instrument grip DOF value (750) is included in the “pose” of the instrument (or manipulator arm assembly and / or system).

[0107] In addition, such as Figure 7 The observed input device grip DOF value (740) described herein does not need to be consistent with the instrument grip DOF value (750) according to a given mapping (700). Generally, when the observed value of the input device DOF variable matches the value of the corresponding instrument DOF variable (or other controllable DOF variable of the computer-aided system (100)), the input device DOF and instrument DOF (or other controllable DOF) are referred to as “matched” or “aligned”. Figure 7 Alignment position (760) is shown. Alignment position (760) represents the value of an input device DOF variable (e.g., input device grip DOF variable (702)) that is mapped to an observed instrument DOF value (e.g., instrument grip DOF variable (704)) according to a given mapping. In some embodiments, the input device DOF variable and the corresponding instrument DOF (or other controllable DOF) are said to be aligned if the observed input device DOF value is within a predefined alignment threshold from the alignment position. The alignment threshold may depend on the alignment position, the observed values ​​of the input device and / or the instrument DOF variable, or any combination thereof. The alignment threshold may further vary based on the orientation of the input device DOF variable (towards the closed ROM limit or the open ROM limit).

[0108] Figure 7 The specified value (770) of the input device holding the DOF variable is also shown. The specified value (770) is included in the input device's "configuration" (e.g., first configuration). Figure 7In the example, where the instrument type specified for the instrument controlled by the input device is the first instrument type, the specified value (770) is depicted as an offset distance (780) toward the "open" ROM limit offset of the input device grip DOF variable (702). In other cases, the specified value may deviate from the alignment position so that the position of the input device grip DOF variable is toward its "closed" ROM limit offset. When the input device is actuated to the first configuration, the input device grip DOF is actuated from the observed input device grip DOF value (740) to the specified input device grip DOF value (770) contained in the first configuration.

[0109] If the device type for the device controlled by the input device (or specified for control by the input device) is determined to be a second device type (e.g., an applicator), a predetermined value for the input device grip DOF variable included in the first configuration is determined. As previously mentioned, for a device of device type two, there is no corresponding device DOF variable; however, a “mapping” can still be provided between the input device grip DOF variable (702) and the device function. This function can specify the state, condition, or parameter of the device. Furthermore, this function can specify an action (which may be an irreversible action), such as “firing” the applicator. That is, in some embodiments, the input device grip DOF variable (702) is mapped to the triggering or action of the device. Figure 8 An exemplary mapping (800) between an input device holding DOF variable (702) and an instrument function (804) is shown, wherein the input device is specified to control an instrument of instrument type two. Similar to... Figure 7 The input device holds a DOF variable θ with a ROM defined by "closed" and "open" states. However, with Figure 7 In contrast, there is no corresponding instrument grip DOF variable. Instead, the example mapping (800) associates the value of the input device grip DOF variable (702) with the value of the instrument function (804). Figure 8 In the example, the function is represented as a state, where the state can have a value of "on" (806) or "off" (808). Additional state categories can be specified without restriction. Furthermore, although... Figure 8 The example mapping (800) represents the instrument function as a binary state, but the instrument function can also be specified as a condition (such as “firing”) or a parameter (such as the amount of energy emitted).

[0110] Figure 8 The observed input device grip DOF value (740) is depicted. In other words, the observed input device hold DOF value (740) indicates the actual value (or position / or orientation) of the input device hold DOF variable. The observed input device hold DOF value (740) is included in the input device's "pose". Figure 8 It also describes the specified value (770) of the DOF variable held by the input device. The specified value (770) is included in the "configuration" (e.g., first configuration) of the input device. According to one or more embodiments, when the device type specified by the input device for controlling the instrument is a second device type, the specified value (770) is set at a limit (e.g., "open" or "closed") of the ROM of the input device's hold DOF variable (702) or within a limit threshold (801) from that limit. Figure 8 In the example, where the device type specified by the input device for control is the second device type, the predefined value (770) is described as being set near the “open” ROM limit of the input device holding the DOF variable (702) (i.e., within the limit threshold (801)). That is, in some embodiments, the predefined value is set at the “open” ROM limit, while in other embodiments, the predefined value is set at a value within the limit threshold (801) of the open ROM limit of the input device holding the DOF variable. Figure 8 A threshold (802) is also described relative to the input device grip DOF variable (702). In the given example, the threshold (802) defines the listed states of the instrument function (“on” (806) or “off” (808)). That is, in the given example, the value of the input device grip DOF variable (702) that is close to the “closed” ROM limit of the input device grip DOF relative to the threshold (802) is mapped to the “on” state (806) of the instrument function. Similarly, in the given example, the value of the input device grip DOF variable (702) that is close to the “open” ROM limit of the input device grip DOF relative to the threshold (802) is mapped to the “off” state (808) of the instrument function. Therefore, in the given example, the specified value (770) is set to maintain the instrument function in its “off” state (808). In addition, it is possible to specify a threshold to distinguish three or more states. As another example, consider the case where the instrument with a second instrument type is an applicator. In this case, the specified value of the input device grip DOF variable can be set to the “open” limit of the ROM. Furthermore, the threshold (802) can represent a trigger, wherein an instrument action such as firing the applicator is performed when the input device holds the DOF variable across the threshold (802).

[0111] In an example where the device type of the device controlled (or designated for control) by the input device is determined to be a third device type, a predetermined value for the input device holding DOF variable included in the first configuration is determined. In one or more embodiments, when the device type is determined to be a third device type, the predetermined value is set to a predetermined value, wherein the predetermined value can be any value in the ROM of the input device holding DOF variable as expected or defined (or assigned) by the user. In one or more embodiments, the predetermined value is the middle of the ROM of the input device holding DOF variable. As previously stated, for a device of device type third device type, there is no corresponding device DOF variable, and there is no mapping between the input device holding DOF variable (702) and the device function. Figure 9 An example “zero” mapping (900) for an input device hold DOF variable (702) is shown, where the input device is specified to control a device of device type 3. As can be seen, the zero mapping (900) does not associate or map the input device hold DOF variable to a device DOF variable or function. Therefore, changes in the value of the input device hold DOF variable do not affect the device controlled by (or specified to be controlled by) the input device, or result in “no-op” (i.e., “no-op”) for the device. In some cases, devices of device type 3 may be said to have no mapping.

[0112] Figure 9 The observed input device grip DOF value (740) is depicted. In other words, the observed input device hold DOF value (740) indicates the actual value (or position / or orientation) of the input device hold DOF variable. The observed input device hold DOF value (740) is included in the input device's "pose". Figure 9 It also describes the specified value (770) of the DOF variable held by the input device. The specified value (770) is included in the input device's "configuration" (e.g., first configuration). Figure 9 In the case where the device type specified by the input device for control is a third device type, the predefined value (770) is described as being set in the middle or near the middle of the ROM of the input device holding the DOF variable (702). Typically, for a device type determined to be a third device type, the predefined value is set to a predetermined value, wherein the predetermined value can be any value within the ROM of the input device holding the DOF variable as desired or defined (or assigned) by the user. In one or more embodiments, the predetermined value is set to provide ergonomic comfort to the user of the input device and improve user endurance (e.g., reduce user hand fatigue).

[0113] Typically, the computer-aided system (100) can enter or enable a state of remote operation control. In some embodiments, remote operation control can be enabled or disabled independently across multiple input devices. For example, consider an input system (120) including a first input device and a second input device, and the first and second input devices being configured to manipulate a first manipulator component and a second manipulator component, respectively. In this case, remote operation control can be enabled or disabled on the first or second input device without affecting the remote operation control state (i.e., enabled or disabled) on the other input device. In one or more embodiments, the process of enabling remote operation control of the computer-aided system (100) (or the input devices of the computer-aided system) includes the user satisfying one or more remote operation control enabling conditions. In other words, in order to enable remote operation control of the input device to enable manipulation control of the input device or command the movement of associated controllable components (e.g., instruments) of the computer-aided system (100), one or more remote operation control enabling conditions must be satisfied. Remote operation control enabling conditions can include a number of related (and possibly sequential) processes. According to one or more embodiments, the remote operation control enabling conditions of the input device depend on the type of instrument designated for control by the input device.

[0114] In one or more embodiments, the remote operation control enable condition is the selection of a remote operation mode of the computer-aided system (100). In some embodiments, the remote operation mode is selected by a user, for example, using a graphical user interface or by pressing a button or switch.

[0115] Another possible remote operation control enabling condition involves manipulating the input device in a specified direction or through a predetermined movement. This type of remote operation control enabling condition may be referred to as a "deliberate movement" condition. For example, the input device may have a "roll" degree of freedom. In this case, the remote operation control enabling condition (e.g., a deliberate movement condition) may require the input device to be manipulated in a manner that the value of the input device's roll DOF variable undergoes an absolute change greater than a roll deliberate movement threshold (e.g., a rotation with a value of 15 degrees or more). The roll deliberate movement threshold may also be directional (e.g., requiring a "clockwise" rotation relative to a given reference). A deliberate movement condition applied to or acting on the input device's roll DOF variable can be referred to as a roll deliberate movement condition. In other cases, the remote operation control enabling condition may require manipulating the input device such that the input device's roll DOF variable achieves a predefined roll value. As another example, the input device may have a "grip" degree of freedom. In this case, the remote operation control enabling condition (e.g., a deliberate movement condition) may require the input device to be manipulated in a manner that the value of the input device's grip DOF variable undergoes a change greater than a grip deliberate movement threshold. A deliberate movement condition applied to or acting on an input device grip DOF variable can be termed a grip deliberate movement condition. Furthermore, in some embodiments, the grip deliberate movement threshold is directional, specifying not only the amount of change that must be observed in the value of the input device grip DOF variable, but also the direction in which the change in the value of the input device grip DOF variable must occur, such as "open" or "closed". For example, the grip deliberate movement threshold can specify the amount of change in the value of the input device grip DOF variable that must change towards the "closed" ROM limit to satisfy the grip deliberate movement threshold. As another example, the grip deliberate movement threshold can specify the amount of change in the value of the input device grip DOF variable that must change towards the "open" ROM limit to satisfy the grip deliberate movement threshold.

[0116] Continuing with possible remote operation control enabling conditions, these conditions may require the input device to change the value of a scroll DOF variable by at least a scroll intentional movement threshold, or require the input device to change the value of a hold DOF variable by at least a hold intentional movement threshold. As described above, the intentional movement threshold (e.g., the hold intentional movement threshold) can be directional, meaning that in order to satisfy the intentional movement condition, the input device must be manipulated in a given direction and at a given amount. For example, a hold intentional movement condition might require the input device to change the hold DOF variable toward the “closed” ROM limit according to a given hold intentional movement threshold. That is, a hold intentional movement condition might require one or more holding members (e.g., 506A, 506B) to be “squeezed” or “pressed” by a certain amount.

[0117] Another possible condition for enabling remote operation control is that one or more degrees of freedom of the input device (e.g., input device grip DOF) and their corresponding controllable degrees of freedom (e.g., instrument grip DOF) are synchronized and / or properly aligned according to a mapping associated with their movement. For example, as Figure 7 As depicted, according to the mapping, the observed input device DOF value (e.g., the observed input device grip DOF value (740)) does not need to be aligned with the observed instrument DOF value (or other controllable DOF). Alignment positions (e.g., alignment positions (760)) can indicate the value of the input device DOF variable corresponding to the observed value of the corresponding instrument DOF variable. In some embodiments, the input device DOF variable and the instrument DOF variable (or other controllable DOF) are considered "aligned" when the observed input device DOF value is within a predefined alignment threshold from a determined alignment position. Therefore, in some embodiments, the remote operation control enable condition includes an "alignment condition" satisfied by manipulating the input device to align or match its pose with the pose of the controlled component (e.g., the instrument). Specifically, the input device can be manipulated such that the values ​​of one or more of its degree-of-freedom variables are within one or more alignment thresholds from a corresponding alignment position, the alignment position being determined based on the observed values ​​of one or more degree-of-freedom variables of the component controlled by the input device (e.g., the instrument) and the mapping that associates the input device DOF variable with the controllable DOF variable.

[0118] According to one or more embodiments, when the device type of the device controlled by the input device (or specified for control by the input device) is determined to be a first device type, the remote operation control enable conditions are as follows. First, the input device must be manipulated to satisfy a grip intentional movement condition or a roll intentional movement condition (wherein the roll intentional movement condition applies only when the input device has a roll DOF). That is, a grip intentional movement threshold can be specified. As mentioned above, the grip intentional movement threshold can be directional (e.g., movement toward the "closed" ROM limit). Furthermore, a roll intentional movement threshold can be specified, where the roll intentional movement threshold can also be directional (e.g., "clockwise" rotation). Then, if the value of the input device grip DOF variable undergoes a change satisfying the grip intentional movement threshold through manipulation of the input device, the grip intentional movement condition is considered satisfied. Similarly, if the value of the input device roll DOF variable undergoes a change satisfying the roll intentional movement threshold through manipulation of the input device, the roll intentional movement condition is considered satisfied. Second, the value of the input device grip DOF variable must be within an alignment threshold relative to the alignment position determined by the input device grip DOF variable. The alignment position is determined based on the observed instrument DOF value designated for use with the instrument controlled by the input device and a given mapping between the input device grip DOF variable and the instrument grip DOF variable. In some examples, the input device must be manipulated to align or match the input device with its corresponding instrument. In summary, in one or more embodiments, when the instrument type determined to be controlled by the input device (or designated for use with the instrument controlled by the input device) is a first instrument type, the remote operation control enable condition includes a deliberate movement condition satisfied by satisfying a roll deliberate movement condition or a grip deliberate movement condition, and an alignment condition in which the input device grip DOF variable is aligned with the instrument grip DOF variable (at least within an alignment threshold) (determined using a mapped alignment position).

[0119] As previously stated, when the device type is a first device type, the input device can be commanded to a first configuration. The first configuration includes a predetermined value for at least the input device grip DOF variable. According to one or more embodiments, the predetermined value of the input device grip DOF variable in the first configuration is set to a value offset from the "open" ROM limit of the input device grip DOF variable by a predefined offset distance from a defined alignment position. In one or more embodiments, the predefined offset distance is substantially equal in magnitude to a grip intentional movement threshold and opposite in direction (if applicable). For example, consider the case where the grip intentional movement threshold is oriented, and it is specified that to satisfy this threshold, the input device grip DOF variable must change a "distance" D toward the "closed" ROM limit. Therefore, in this example, the offset distance can be set to -D, or an offset distance toward the "open" ROM limit. In these implementations, it is highly likely that the alignment condition will also be met when the intentional grip movement condition is satisfied. This is because the offset distance being substantially equal in magnitude and opposite in direction to the intentional grip movement threshold ensures that the observed input device grip DOF value when the intentional grip movement condition is satisfied will be close to the alignment position (e.g., within the alignment threshold). In fact, the predefined definition of an offset distance substantially equal to the intentional grip movement threshold is that, when the intentional grip movement condition is satisfied, the observed input device grip DOF value is within the alignment threshold relative to the alignment position. Therefore, the alignment condition is inherently satisfied for the first device type and, for example, an example where the remote operation control enable condition includes a first remote operation control enable condition that is actually satisfied by satisfying the intentional grip movement condition.

[0120] According to one or more embodiments, when the device type of the device controlled by the input device (or designated for control by the input device) is determined to be a second device type, the remote operation control enable conditions are as follows. First, the input device must be manipulated to satisfy either a grip intentional movement condition or a roll intentional movement condition (wherein the roll intentional movement condition applies only when the input device has a roll DOF). Second, the value of the input device grip DOF variable must be within the opening threshold of its "open" ROM limit. In one or more embodiments, the input device's grip DOF is actuated to its open ROM limit when the intentional movement condition is satisfied—whether by satisfying the roll intentional movement condition or the grip intentional movement condition. Therefore, in some embodiments, the second remote operation control enable condition when the device type is a second device type is automatically satisfied after the first remote operation control enable condition is satisfied.

[0121] According to one or more embodiments, when the device type of the device controlled by the input device (or specified for control by the input device) is determined to be a third device type, the remote operation control enable conditions are as follows: The input device must be manipulated to satisfy a grip intentional movement condition or a roll intentional movement condition (if the input device has a roll DOF).

[0122] According to one or more embodiments, the behavior of an input device with enabled remote operation control depends on the type of instrument controlled by the input device.

[0123] Specifically, when the instrument type of the instrument controlled by the input device is determined to be a first instrument type, the first configuration is not maintained while remote operation control using the input device is enabled. Furthermore, the control system (242) provides and / or facilitates the mapping of movement and other input signals (e.g., button presses) between the input device with remote operation control enabled and components of a computer-aided system (100), such as the manipulator system (130). More specifically, changes in the observed values ​​of the input device's DOF ​​variables are mapped and correspondingly implemented as changes in the values ​​of the corresponding controllable DOF variables (e.g., instrument DOF variables). Therefore, manipulation of the input device causes actuation of controlled components, such as manipulator assemblies including the instrument.

[0124] Similarly, when the instrument type of the device controlled by the input device is determined to be a second instrument type, the first configuration is not maintained while remote operation control using the input device is enabled. Additionally, the control system (242) provides and / or facilitates the mapping of movement and other input signals (e.g., button presses) between the input device when remote operation control is enabled and components of a computer-aided system (100), such as a manipulator system (130). Taking a clamping device as an example, manipulating the input device, particularly by holding the DOF, can cause the clamping device to "fire" or otherwise engage the loaded clamp. More specifically, the "fire" value (e.g., thus defining a threshold (802) for the input device holding the DOF variable), and manipulation of the input device such that the input device holding the DOF variable reaches (or crosses or traverses) the fire value, causes the clamping device to fire the loaded clamp. According to one or more embodiments, the fire value is defined near the "closed" ROM limit of the input device holding the DOF variable. Recall that the remote operation control activation condition for the second type of instrument is that the input device holds the DOF variable within the opening threshold of the "open" ROM limit. When remote operation control is enabled, the clamping instrument will not fire the loaded clamp and must be manipulated toward the "closed" ROM limit to fire the loaded clamp.

[0125] Finally, when it is determined that the device type of the instrument controlled by the input device is a third device type, the input device is actively actuated to a second configuration while enabling remote operation control using the input device. The second configuration includes at least a predetermined value for the input device grip DOF variable. In one or more embodiments, the predetermined value of the input device grip DOF variable in the second configuration is the midpoint of the ROM from which the input device grips the DOF variable. That is, in one or more embodiments, the second configuration is the same as the first configuration; however, this is not necessary. Typically, in the case of "active actuation" to the second configuration, the user can relatively easily resist any actuated movement of the input device. For example, when the input device grip DOF is actively actuated to the midpoint of its corresponding ROM, the user can interact with the input device to open and / or close the input device grip DOF relative to the midpoint of its ROM. And, if the user stops applying force to the input device, the input device grip DOF value will return to its midpoint ROM value. Furthermore, by enabling remote operation control of the input device, the control system (242) provides and / or facilitates the mapping of movement and other input signals (e.g., button presses) between the input device and components of a computer-aided system (100), such as a manipulator system (130). More specifically, changes in observed values ​​of input device DOF variables are mapped and correspondingly implemented as changes in the values ​​of corresponding controllable DOF variables (e.g., instrument DOF variables). Thus, manipulation of the input device causes actuation of controlled components such as manipulator assemblies including instruments.

[0126] Figure 10 A method is described for determining one or more configurations of an input device (i.e., specified values ​​for one or more actuable degrees of freedom of the input device) based at least on the type of device controlled (or designated as controlled) by the input device. According to one or more embodiments, Figure 10 The method depicted in the flowchart is applied to, or used with, a computer-aided system including an input device configured for user manipulation. The computer-aided system includes one or more actuators to drive the input device in one or more degrees of freedom. The computer-aided system also includes a manipulator system configured to support one or more instruments and a control system communicatively coupled to the input device and the manipulator system. Figure 10 One or more steps of the method depicted in the flowchart can be executed using the control system of a computer-aided system.

[0127] In block 1002, the device type of the instrument supported by the manipulator system is determined. The device type can be determined, for example, by reading a device identifier associated with the device type when coupled to the computer-aided system (100) (e.g., connected to a manipulator arm). In some embodiments, the device type is specified by the user and entered into the computer-aided system (e.g., using an input device of an input system). The device type can be one of a group consisting of a first device type, a second device type, and a third device type. The device type can be directly related to the type of the end effector included on or as part of the instrument.

[0128] In block 1004, a first configuration of the input device is determined at least based on the device type. The first configuration includes, or is a set of predetermined values ​​for one or more degrees of freedom (DOF) variables of one or more actuated degrees of freedom of the input device, wherein these actuated degrees of freedom are driven using one or more actuators of a computer-aided system. In one or more embodiments, one of the actuated degrees of freedom of the input device is a grip degree of freedom (i.e., the input device grip DOF). Furthermore, in one or more embodiments, the first configuration includes predetermined values ​​for the input device grip DOF variables.

[0129] When the device type is determined to be a first device type, the device has a grip DOF (i.e., device grip DOF) defined by a device grip DOF variable having a specified range of motion (ROM). In these cases, observations of the device grip DOF variable are determined or obtained (e.g., using the device's sensors, such as encoders). Additionally, an alignment position in the input device grip DOF is determined based on the device grip DOF variable and a mapping that associates the input device grip DOF variable with the device grip DOF variable. In these cases, the specified value of the input device grip DOF variable under the first configuration is set to a value offset by an offset distance from the alignment position. In some embodiments, the offset causes the specified value of the input device grip DOF variable to be biased relative to the alignment position toward the "open" ROM limit of the input device grip DOF variable.

[0130] When the device type is determined to be a second device type, the predetermined value of the input device grip DOF variable is set to a predetermined value. In some embodiments, the predetermined value is equal to or close to the "open" ROM limit of the input device grip DOF variable (i.e., within the open threshold range).

[0131] When the device type is determined to be a third device type, the predetermined value of the input device holding DOF variable is set to a predetermined value. In some embodiments, the predetermined value is located in the middle of the ROM of the input device holding DOF variable.

[0132] In block 1006, one or more actuators are used to drive the input device to a first configuration. That is, one or more actuators are used to actuate the input device's operable degrees of freedom to predetermined values ​​included in the first configuration, where applicable (i.e., therefore, not all operable degrees of freedom need to have predetermined values). In one or more embodiments, the input device hold DOF variable is commanded to a predetermined value included in the first configuration.

[0133] As described herein, the embodiments disclosed herein generally relate to methods and systems for selectively implementing remote operational control of an instrument (160) supported by a manipulator system (130) of a computer-aided system (100) by at least partially utilizing an actuated input device grip degree of freedom (DOF) designated for controlling at least one instrument. Specifically, selectively enabling remote operational control requires satisfying one or more remote operational control enabling conditions, wherein these conditions can depend on the instrument type of the instrument. Additionally, an initial position of the actuated input device grip DOF (or more specifically, the value of the input device grip DOF variable) is determined based on the instrument type of the instrument. The actuated input device grip DOF is actuated to the initial position using one or more actuators. Furthermore, the subsequent behavior of the input device grip DOF also depends on the instrument type of the instrument when one or more remote operational control enabling conditions are satisfied, or when otherwise entering a state of enabled remote operational control. The instrument type can be one of a group consisting of a first instrument type, a second instrument type, and a third instrument type. Generally, device type is based on a mapping (or lack thereof) from input device hold DOF to device hold DOF or device function (if applicable). Figure 11 A flowchart illustrating the initial position, remote operation control activation conditions, and input device behavior (relative to the input device hold-on-field) during a device-based device type demonstration using an input device for remote operation control activation is provided. According to one or more embodiments, Figure 11 The boxes depicted in the flowchart can be applied to or implemented through a computer-aided system that includes an input device configured to be operated by a user.

[0134] In block 1102, the device type of the instrument supported by the manipulator system is determined. The device type can be determined, for example, by reading a device identifier associated with the device type when coupled to the computer-aided system (100) (e.g., connected to a manipulator arm). In some embodiments, the device type is specified by the user and entered into the computer-aided system (e.g., using an input device of an input system). The device type can be directly related to the type of end effector included on or part of the instrument.

[0135] Boxes 1104, 1106 and 1108 indicate the device type as one of the first device type, the second device type and the third device type.

[0136] Box 1104 indicates that the device type is a first device type. In this case, and as shown in box 1110, the initial position in the input device grip degree of freedom included in the first configuration is offset from the alignment position. The alignment position is based on the observed position in the corresponding device grip degree of freedom. For example, given a mapping between the input device grip DOF variable and the device grip DOF variable (e.g., mapping (700)), the alignment position in the input device grip DOF is the value of the input device grip DOF variable, which is mapped to the observed value of the device grip DOF variable according to the given mapping. In one or more embodiments, the offset causes the initial position to be biased relative to the alignment position toward the "open" limit of the input device grip DOF variable ROM. Continuing with the determination that the device is a first device type, box 1116 specifies a condition included in one or more remote operation control enable conditions. Specifically, box 1116 indicates that the remote operation control enable condition for the case where the device type is a first device type is that the user of the input device manipulates the input device grip DOF from the initial position to within an alignment threshold of the alignment position. Box 1120 indicates the behavior of the input device degrees of freedom actuated during or in the context of remote operation control of the device enabled using the input device. Specifically, Box 1120 indicates that, in the case of enabled remote operation control and a device of a first device type, changes in observed values ​​of the input device grip degrees of freedom are mapped to the device grip degrees of freedom according to a given mapping that associates the input device grip DOF variable and the device grip DOF variable. That is, the initial position in the input device grip DOF is not maintained, and movement of the input device in the input device grip DOF (e.g., by user manipulation) results in actuation or movement in the corresponding device grip DOF. In other words, "normal" remote operation behavior.

[0137] Box 1106 indicates that the device type is a second device type. In this case, and as shown in box 1112, the initial position in the input device grip DOF included by the first configuration is within a limit threshold from the limit of the input device grip DOF. For example, the input device grip DOF variable can have an associated range of motion (ROM). The limits on the ROM of the input device grip DOF variable are the limits of the input device grip DOF. In one or more embodiments, the input device grip DOF variable is limited by an "open" limit and a "closed" (or "closed") limit. Therefore, the input device grip DOF variable is set to a value within a limit threshold from the specified limit. In one or more embodiments, the initial position is within a predefined limit threshold from the "open" limit. Continuing with the device type determined to be a second device type, box 1118 specifies a condition included in one or more remote operation control enable conditions. Specifically, box 1118 indicates that the remote operation control enable condition for the case where the device type is a second device type is that the input device grip DOF is within a limit threshold from the specified limit of the input device grip DOF (e.g., "open"). It is worth noting that, in one or more embodiments, this condition is satisfied by actuating the input device to the initial position in block 1112 during the DOF (Device of Freedom) hold. Block 1122 indicates the behavior of the input device degrees of freedom actuated when or during the use of the input device to enable remote operation control of the device. Specifically, block 1122 indicates that, in the case of enabled remote operation control and a device having a second device type, the initial position in the DOF hold is not maintained. In other words, at least during the DOF hold, the user can manipulate the input device relatively freely.

[0138] Box 1108 indicates that the device type is a third device type. In this case, and as shown in box 1114, the initial position in the input device grip degrees of freedom included by the first configuration is at a predefined position (or a predefined input device grip DOF variable value). In one or more embodiments, the predetermined position is at the midpoint of the ROM of the input device grip DOF variable. Continuing with the determination that the device is a third device type, box 1124 indicates the behavior of the input device degrees of freedom actuated when or during the use of the input device to enable remote operation control of the device. Specifically, box 1124 indicates that when remote operation control is enabled and the device has a third device type, the input device grip degrees of freedom are actively actuated to another position included by the second configuration. In one or more embodiments, the other position is the same as the initial position. Therefore, during the use of the input device to enable remote operation control of the device, the input device grip DOF is actuated to another position, which is predefined by the second configuration.

[0139] Figure 10 and Figure 11 One or more of the boxes in the diagram can be executed by various components of the system, as referenced above. Figures 1-2 As described. Furthermore... Figure 10 and Figure 11 The flowchart frames can be executed on one or more processors of the control system (242) of, for example, a computer-aided system (100).

[0140] In summary, the embodiments disclosed herein generally relate to methods and systems for selectively implementing remote operational control of an instrument (160) supported by a manipulator system (130) of a computer-aided system (100) using an actuated grip degree of freedom (DOF) of an input device designated as at least a control instrument. The input device may have at least a first actuable degree of freedom, namely, the input device grip degree of freedom. The input device can include additional actuable and non-actuable degrees of freedom. Specifically, selectively enabling remote operational control requires the satisfaction of one or more remote operational control enabling conditions. Furthermore, a "first configuration" is determined, which defines a set of values ​​for one or more actuable degrees of freedom of the input device. The one or more remote operational control enabling conditions, the first configuration including the specified values ​​of the grip DOF of the input device, and subsequent behavior of the user input system (or, at least the grip DOF of the input device) depend on the configuration of the manipulator system. In particular, the one or more remote operational control enabling conditions, the first configuration, and subsequent behavior of the manipulator system depend on the type of instrument designated for control by the input device.

[0141] Note that for the sake of brevity and clarity, the preceding discussion has referred to a single input device. However, in practice, an input system (120) (or more generally, a computer-aided system (100)) can have more than one input device. In cases where a computer-aided system (100) has more than one input device, one or more input devices can be part of the user input system (120) or distributed across multiple components of the computer-aided system (100). The embodiments disclosed herein, which involve at least selectively activating an instrument using an input device, can be readily applied independently to one or more input devices of a computer-aided system. For example, consider a computer-aided system (100) having two input devices (i.e., a first input device and a second input device). The computer-aided system can independently perform remote operation control on the first and second input devices. For example, remote operation control of the computer-aided system (100) associated with the first input device can be referred to as first remote operation control. Similarly, remote operation control of the computer-aided system (100) associated with the second input device can be referred to as second remote operation control. Continuing this example, a first input device with enabled first remote operation control can be configured to manipulate a first manipulator arm (e.g., 150A) and / or a first instrument supported by the first manipulator arm, while a second input device with enabled second remote operation control can be configured to manipulate a second manipulator arm (e.g., 150B) and / or a second instrument supported by the second manipulator arm; the first and second manipulator arms (150A, 150B) are part of a manipulator system (130). Furthermore, the first and second input devices can each have their own remote operation control enabling conditions, a "first" configuration, and subsequent behavior based on the instrument type designated as being controlled by the input device. In summary, embodiments of this disclosure are not limited to a single input device or a computer-aided system (100) with only one input device. Those skilled in the art will readily understand that the above description can be readily applied to multiple input devices, including computer-aided systems (100) with more than one input device.

[0142] While only a few exemplary embodiments have been described in detail above, those skilled in the art will readily understand that many modifications are possible in the exemplary embodiments without substantially departing from the invention. Therefore, all such modifications are intended to be included within the scope of this disclosure as defined in the appended claims.

Claims

1. A computer-aided system, comprising: An input device configured for user manipulation; One or more actuators that drive the input device with one or more degrees of freedom; A control system configured to support one or more instruments; as well as The control system, communicatively coupled to the input device and the actuator system, is configured to: Determine the type of device supported by the control system; The first configuration of the input device is determined at least in part based on the device type of the device, and The one or more actuators drive the input device to its first configuration.

2. The computer-aided system according to claim 1, further comprising: Another input device configured to be operated by a user, wherein the one or more actuators are configured to drive the other input device with one or more degrees of freedom; The control system is communicatively coupled to the other input device and is further configured to: Determine the type of device for another device supported by the control system; The first configuration for the other input device is determined at least in part based on the device type of the other device; and The one or more actuators drive the other input device to its first configuration.

3. The computer-aided system according to claim 1, in, Determining the instrument type of the instrument, determining the first configuration for the input device, and causing the one or more actuators to drive the input device to its first configuration are performed in response to any one of the following: It was detected that the device had been replaced with another device. The input device has been designated to control another instrument supported by the manipulator system. The device was detected to have been manually adjusted or manipulated, and The system detects that the user is near the input system, which includes the input device, and that the user was previously determined not to be near the input system.

4. The computer-aided system according to claim 1, The first configuration includes the initial position in the degrees of freedom of the input device grip; and Driving the input device to the first configuration using the one or more actuators includes driving the input device to the initial position in the input device grip degrees of freedom using the one or more actuators.

5. The computer-aided system of claim 4, wherein the initial position in the gripping degree of freedom of the input device is determined based on the type of instrument.

6. The computer-aided system according to claim 4, The device type mentioned above is selected from the group consisting of: The first type of instrument, characterized in that The first mapping between the input device's grip degree of freedom and the instrument's grip degree of freedom. The second type of device is characterized by a second mapping between the gripping degree of freedom of the input device and the function of the device, and The third type of instrument is characterized by the absence of a mapping between the gripping freedom of the input device and the instrument.

7. The computer-aided system according to claim 6, Wherein the device type of the first device is the first device type. The initial position of the input device is determined based on the observed position of the instrument in the instrument grip degree of freedom.

8. The computer-aided system according to claim 7, Determining the first configuration includes: Determine the observed position of the instrument in the instrument grip degree of freedom, and The alignment position of the input device in the input device's grip degree of freedom is determined based on the observed position of the instrument in the instrument's grip degree of freedom. The process of causing the one or more actuators to drive the input device to the first configuration includes: The input device is driven in the input device grip degree of freedom to the initial position offset by an amount from the alignment position using the one or more actuators.

9. The computer-aided system of claim 8, wherein the alignment position of the input device remotely corresponds to the observation position of the instrument.

10. The computer-aided system according to claim 8, The control system is further configured to: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from input devices to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, remote operation control of the instrument is enabled using the input device. The one or more remote operation control enabling conditions include conditions under which the user manipulates the input device's grip degree of freedom from the initial position to a position within an alignment threshold of the alignment position.

11. The computer-aided system according to claim 6, The device type of the device is the second device type. The initial position is at the limit of the degrees of freedom of the input device.

12. The computer-aided system according to claim 11, The control system is further configured to: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from the input device to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, remote operation control of the instrument is enabled using the input device. The remote operation control enabling condition includes the condition that the input device's grip freedom is within a limit threshold from the limit.

13. The computer-aided system according to claim 11, wherein When remote operation control is enabled using the input device, the control system is also configured to cause the one or more actuators to drive the input device to the limit.

14. The computer-aided system according to claim 6, The device type of the device is the third device type. The initial position is located at the midpoint of the range of motion of the input device in the degree of freedom of gripping the input device.

15. The computer-aided system according to claim 14, The control system is further configured to: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from the input device to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, remote operation control of the instrument is enabled using the input device. When remote operation control is enabled using the input device, the control system is further configured to cause one or more actuators to drive the input device to a second configuration, the second configuration including another position of the input device's grip degree of freedom.

16. The computer-aided system according to claim 15, The other position is the same as the initial position.

17. The computer-aided system according to any one of claims 4 to 16, The input device is actuated to the first configuration in response to entering the remote operation mode of the computer-aided system.

18. A method for controlling a computer-aided system, the method being performed by a control system of the computer-aided system and comprising: Determine the type of device supported by the manipulator system of the computer-aided system; The first configuration of the input device of the computer-aided system is determined at least in part based on the type of the instrument, wherein the input device is configured to be operated by a user and driven in one or more degrees of freedom using one or more actuators of the computer-aided system; as well as The one or more actuators drive the input device to its first configuration.

19. The method of claim 18, further comprising: Determine the type of device for another device supported by the control system; The first configuration of another input device for the computer-aided system is determined at least in part based on the device type of the other device, wherein the other input device is configured to be manipulated by the user and driven by the one or more actuators in one or more degrees of freedom; as well as The one or more actuators drive the other input device to its first configuration.

20. The method according to claim 18, The determination of the device type, the determination of the first configuration, and the driving of the input device to the first configuration by the one or more actuators are performed in response to any one of the following: It was detected that the device had been replaced with another device. The input device has been designated to control another instrument supported by the manipulator system. The device was detected to have been manually adjusted or manipulated, and The system detects that the user is near the input system, which includes the input device, and that the user was previously determined not to be near the input system.

21. The method according to claim 18, The first configuration includes the initial position in the degrees of freedom of the input device grip; and Driving the input device to the first configuration using the one or more actuators includes driving the input device to the initial position in the input device grip degrees of freedom using the one or more actuators.

22. The method of claim 21, wherein the initial position in the gripping degrees of freedom of the input device is determined based on the type of the device.

23. The method of claim 21, The device type mentioned above is selected from the group consisting of: The first type of instrument, characterized in that The first mapping between the input device's grip degree of freedom and the instrument's grip degree of freedom. The second type of device is characterized by a second mapping between the gripping degree of freedom of the input device and the function of the device, and The third type of instrument is characterized by the absence of a mapping between the gripping freedom of the input device and the instrument.

24. The method according to claim 23, Wherein the device type of the first device is the first device type, The initial position of the input device is determined based on the observed position of the instrument in the instrument grip degree of freedom.

25. The method according to claim 24, Determining the first configuration includes: Determine the observed position of the instrument in the instrument grip degree of freedom, and The alignment position of the input device in the input device's grip degree of freedom is determined based on the observed position of the instrument in the instrument's grip degree of freedom. The process of causing the one or more actuators to drive the input device to the first configuration includes: The input device is driven in the input device grip degree of freedom to the initial position offset by an amount from the alignment position using the one or more actuators.

26. The method of claim 25, wherein the alignment position of the input device remotely corresponds to the observation position of the instrument.

27. The method of claim 25, further comprising: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from the input device to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, the input device is used to enable remote operation control of the instrument. The one or more remote operation control enabling conditions include the condition that the user manipulates the input device's grip freedom from the initial position to a position within an alignment threshold of the alignment position.

28. The method according to claim 23, The device type of the device is the second device type. The initial position is at the limit of the degrees of freedom of the input device.

29. The method of claim 28, further comprising: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from the input device to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, the input device is used to enable remote operation control of the instrument. The remote operation control enabling condition includes the condition that the input device's grip freedom is within a limit threshold from the limit.

30. The method of claim 28, wherein When remote operation control of the input device is enabled, the control system is also configured to cause the one or more actuators to drive the input device to the limit.

31. The method according to claim 23, The device type of the device is the third device type. The initial position is located at the midpoint of the range of motion of the input device in the degree of freedom of gripping the input device.

32. The method of claim 31, further comprising: Based on the device type of the device, determine one or more remote operation control enabling conditions for enabling remote operation control of the device using the input device; Monitor input from the input device to determine whether the one or more remote operation control enable conditions are met; as well as Based on the determination that one or more remote operation control activation conditions are met, the input device is used to enable remote operation control of the instrument. When remote operation control is enabled using the input device, the control system is also configured to cause the one or more actuators to drive the input device to a second configuration, the second configuration including another position of the input device's grip degree of freedom.

33. The method of claim 32, The other position is the same as the initial position.

34. The method according to any one of claims 21 to 33, The input device is actuated to the first configuration in response to entering the remote operation mode of the computer-aided system.

35. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions executable by one or more processors associated with a computer-aided system, the plurality of machine-readable instructions causing the one or more processors to perform the method according to any one of claims 18 to 34.

Images