Facilitating operation of a manipulator through reconfiguration of a proximal segment of the manipulator

Abstract

A computer-assisted system includes a manipulator arm and a control system. The manipulator arm has a distal manipulator with distal joints and a proximal manipulator with proximal joints. The proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument with an end effector. The control system includes one or more processors and is configured to, when in a task execution mode, positionally lock the proximal joints, and drive the distal joints according to a task execution input. The control system is further configured to, when in a mode different from the task execution mode, drive at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

Description

FACILITATING OPERATION OF A MANIPULATOR THROUGH RECONFIGURATION OF A PROXIMAL SEGMENT OF THE MANIPULATORCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of the filing date of U.S. Provisional Patent Application 63 / 676,517, filed on July 29, 2024, which is hereby incorporated by reference herein in its entirety.BACKGROUNDField of Invention

[0002] The present invention generally provides improved computer-assisted devices, systems, and methods.Overview

[0003] Computer-assisted systems can be used to perform a task at a worksite. For example, a computer-assisted system may comprise a handheld, motorized tool assembly. As another example, a computer-assisted system may comprise a robotic system and may include one or more robotic manipulators to manipulate instruments for performing the task.

[0004] Example computer-assisted systems include industrial and recreational robotic systems. Example computer-assisted systems also include medical robotic systems used in procedures for diagnosis, non-surgical treatment, surgical treatment, etc.

[0005] Some computer-assisted systems include one or more instruments that are articulated in order to perform various procedures. The computer-assisted system can be automated, semi- automated, teleoperated, etc. In a teleoperated example, a human operator manipulates one or more leader input controls to command motion of one or more follower instruments located in a workspace.

[0006] While robotic systems and devices have proven highly effective and advantageous, further improvements would be desirable. For example, it may be desirable to increase the range of motion available to an end effector or manipulator arm.SUMMARY

[0007] In general, in one aspect, one or more embodiments relate to a computer- assisted system comprising : a manipulator arm comprising a distal manipulator with distaljoints and a proximal manipulator with proximal joints, wherein the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument with an end effector; a control system comprising one or more processors, the control system configured to: when in a task execution mode: positionally lock the proximal joints; and drive the distal joints according to a task execution input; when in a mode different from the task execution mode: drive at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

[0008] In general, in one aspect, one or more embodiments relate to a method for facilitating operation of a computer-assisted system comprising: a manipulator arm comprising a distal manipulator with distal joints and a proximal manipulator with proximal joints, wherein the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument with an end effector; wherein the method comprises: when in a task execution mode: positionally locking the proximal joints; and driving the distal joints according to a task execution input; when in a mode different from the task execution mode: driving at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

[0009] Other aspects of the invention will be apparent from the following description and the appended claims.BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. In the drawings:

[0011] FIGs. 1A and IB depict computer-assisted systems in accordance with one or more embodiments.

[0012] FIG. 2 depicts a manipulator arm in accordance with one or more embodiments.

[0013] FIG. 3A schematically illustrates a scenario with a manipulator in a first configuration in accordance with one or more embodiments.

[0014] FIG. 3B schematically illustrates a scenario with a manipulator in a second configuration in accordance with one or more embodiments.

[0015] FIG. 3C schematically illustrates scenario with a manipulator in a third configuration in accordance with one or more embodiments.

[0016] FIG. 3D schematically illustrates scenario with a manipulator in a fourth configuration in accordance with one or more embodiments.

[0017] FIG. 3E schematically illustrates scenario with a manipulator in a fourth configuration in accordance with one or more embodiments.

[0018] FIG. 4 depicts a flowchart in accordance with one or more embodiments.DETAILED DESCRIPTION

[0019] In the following description, specific details are set forth describing some embodiments consistent with the present disclosure. Numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent, however, to one skilled in the art that some embodiments can be practiced without some or all of these specific details. The specific embodiments disclosed herein are meant to be illustrative but not limiting. One skilled in the art may realize other elements that, although not specifically described here, are within the scope and the spirit of this disclosure. In addition, to avoid unnecessary repetition, one or more features shown and described in association with one embodiment may be incorporated into other embodiments unless specifically described otherwise or if the one or more features would make an embodiment non-functional.

[0020] In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. For example, devices and substitutes that provide or enable linear actuation, such as pneumatic cylinders and lead screws are well understood without detailed descriptions of aspects of such devices like gaskets, thread pitch, etc. Thus, specific descriptions regarding such devices, procedures, components, and how circuits can be integrated with, or used within, embodiments of the instant disclosure are omitted herein for concision where applicable without causing undue ambiguity or uncertainty.

[0021] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as an adjective for an element (i.e., any noun in the application). The use of ordinal numbers is not to imply or create any particular ordering of the elements nor to limit any element to being only a single element unless expressly disclosed, such as using the terms “before,” “after,” “single,” and other such terminology. Rather, the use of ordinalnumbers is to distinguish between the elements. By way of an example, a first element is distinct from a second element, and the first element can encompass more than one element and succeed (or precede) the second element in an ordering of elements.

[0022] It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.

[0023] Terms such as “approximately,” “substantially,” etc., mean that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations and other factors known to those of skill in the art, may occur in amounts that do not preclude the effect the characteristic was intended to provide.

[0024] In the following description of FIGs. 1-4, any component described with regard to a figure, in various embodiments disclosed herein, may be equivalent to one or more like-named components described with regard to any other figure. For brevity, descriptions of these components are not necessarily repeated with regard to each figure. Thus, each and every embodiment of the components of each figure is incorporated by reference and assumed to be optionally present within every other figure having one or more like-named components. Additionally, in accordance with various embodiments disclosed herein, any description of the components of a figure is to be interpreted as an optional embodiment which may be implemented in addition to, in conjunction with, or in place of the embodiments described with regard to a corresponding like-named component in any other figure.

[0025] Considering the figures, specific non-limiting embodiments are described in further detail. It should be understood that the various systems, components, features, and methods described relative to these embodiments can be used either individually and / or in any desired combination as the disclosure is not limited to only the specific embodiments described herein.

[0026] This disclosure describes various devices, elements, and portions of computer- assisted systems and elements in terms of their state in three-dimensional space. As used herein, the term “position” refers to the location of an element or a portion of an element (e.g., three degrees of translational freedom in a three-dimensional space, such as along Cartesian x-, y-, and z-coordinates). As used herein, the term “orientation” refers to the rotational placement of an element or a portion of an element (e.g., three degrees of rotational freedom in three-dimensional space, such as about roll, pitch, and yaw axes, represented inangle-axis, rotation matrix, quaternion representation, and / or the like). As used herein, and for a device with a kinematic series, such as with a repositionable structure with a plurality of links coupled by one or more joints, the term “proximal” refers to a direction toward a base of the kinematic series, and “distal” refers to a direction away from the base along the kinematic series.

[0027] As used herein, the term “pose” refers to the multi-degree of freedom (DOF) spatial position and orientation of a coordinate system of interest attached to a rigid body. In general, a pose includes a pose variable for each of the DOFs in the pose. For example, a full 6-DOF pose for a rigid body in three-dimensional space would include 6 pose variables corresponding to the 3 positional DOFs (e.g., x, y, and z) and the 3 orientational DOFs (e.g., roll, pitch, and yaw). A 3-DOF position only pose would include only pose variables for the 3 positional DOFs. Similarly, a 3-DOF orientation only pose would include only pose variables for the 3 rotational DOFs. Further, a velocity of the pose captures the change in pose over time (e.g., a first derivative of the pose). For a full 6-DOF pose of a rigid body in three-dimensional space, the velocity would include 3 translational velocities and 3 rotational velocities. Poses with other numbers of DOFs would have a corresponding number of velocities translational and / or rotational velocities.

[0028] Aspects of this disclosure are described in reference to computer-assisted systems, which can include devices that are teleoperated, externally manipulated, autonomous, semi-autonomous, and / or the like. Further, aspects of this disclosure are described in terms of an implementation using a teleoperated surgical system, such as the da Vinci® Surgical System commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. Knowledgeable persons will understand, however, that inventive aspects disclosed herein may be embodied and implemented in various ways, including teleoperated and nonteleoperated, and medical and non-medical embodiments and implementations. Implementations on da Vinci® Surgical Systems are merely exemplary and are not to be considered as limiting the scope of the inventive aspects disclosed herein. For example, techniques described with reference to surgical instruments and surgical methods may be used in other contexts. Thus, the instruments, systems, and methods described herein may be used for humans, animals, portions of human or animal anatomy, industrial systems, general robotic, or teleoperated systems. As further examples, the instruments, systems, and methods described herein may be used for non-medical purposes including industrial uses, general robotic uses, sensing or manipulating non-tissue work pieces, cosmetic improvements,imaging of human or animal anatomy, gathering data from human or animal anatomy, setting up or taking down systems, training medical or non-medical personnel, and / or the like. Additional example applications include use for procedures on tissue removed from human or animal anatomies (with or without return to a human or animal anatomy) and for procedures on human or animal cadavers. Further, these techniques can also be used for medical treatment or diagnosis procedures that include, or do not include, surgical aspects.

[0029] Specific embodiments of the disclosure will now be described in detail with reference to the accompanying figures. Like elements in the various figures are denoted by like reference numerals for consistency.

[0030] FIG. 1A depicts an example computer-assisted system that can be used for medical procedures (e.g., surgical procedures). As such, the example computer-assisted system 100 of FIG. 1A is herein also referred to as a computer-assisted medical system 100 (or, more simply, “medical system 100”). The computer-assisted system 100 includes a robotic manipulation system 110 that, in the example, may be used to perform a computer- assisted procedure such as a diagnostic or therapeutic procedure. For example, the computer assisted system 100 may be used to perform minimally invasive surgery on a patient 198 positioned on an operating table 190. An instrument (not shown in FIG. 1A) supported by a manipulator arm 150A, 150B may enter a workspace, such as the body of the patient 198, through an entry location (e.g., a natural orifice such as the throat or anus, or through an incision), while an operator views the worksite with the instrument (e.g., a surgical site in the surgical scenario) and / or controls the instrument using the user control system 120. The number of instruments used at one time generally depends on the task and space constraints, among other factors. The robotic manipulation system 110, in the example of FIG. 1 A, includes two manipulator arms 150A, 150B, each of which may support one or more instruments which can be removably or permanently mounted thereon. A robotic manipulation system 110 in accordance with embodiments of the disclosure may include any number of manipulator arms, without departing from the disclosure. A more detailed description of the manipulator arms 150A, 150B is provided below in reference to FIG. 2.

[0031] The operating table 190 includes a tabletop 192 on which the patient 198 is placed and a support column 194 vertically extending along a column axis 196 that supports and elevates the tabletop 192 above a floor surface. The operating table may further include at least one mechanical interface 199 for attaching a device (such as a manipulator arm) to the operating table 190. In the example of FIG. 1A, the at least one mechanical interface 199 is arail system. Although FIG. 1A depicts the at least one mechanical interface 199 as a rail system, other configurations and arrangements are considered, such as clevis fasteners, interference fits, etc. Further, although FIG. 1A illustrates a single, centrally located support column 194 that is statically grounded to the floor surface, other suitable operating tables 190 may have a horizontally extending pillar, a wheeled base, a base designed to slide along a track or rail, and / or a multi-legged support system. The operating table 190 may be provided in the form of any structure suitable for supporting a patient during a medical procedure.

[0032] The computer-assisted system 100 may include a control system 142. The control system 142 may be used to process input provided by the user control system 120 from an operator, such as to control the computer-assisted system 100. The control system 142 may also be used to process signals from other devices, from sensors, from any networks to which the control system 142 connects, etc. Example sensors include those associated with actuators or joints of the computer-assisted system, such as motor encoders, rotary or linear joint encoders, torque sensors, current sensors, accelerometers, force sensors, inertial measurement units, optical or ultrasonic sensors or imagers, RF sensors, etc. An auxiliary system 140 may receive images from an imaging instrument (e.g., an endoscope, an optical camera, an ultrasonic probe, etc. in a medical example) that may be disposed on one of the manipulator arms and may process the captured images in a variety of ways prior to any subsequent display on a display monitor. For example, the auxiliary system 140 may overlay the captured images with a virtual control interface prior to displaying the combined images to the operator via the user control system 120 or other display systems located locally or remotely from the procedure.

[0033] The control system may further be used to provide an output, e.g., a video image for display by the display 144. The control system 142 may further be used to control the robotic manipulating system 110, including but not limited to movement of the manipulator arms 150A, 150B, and / or instruments between kinematic configurations, releasing or engaging brakes, etc. A more detailed description is provided below. In some instances, the control system 142 may be disposed within, or otherwise considered part of, the robotic manipulation system 110, the user control system 120, and / or the auxiliary system 140.

[0034] The control system 142 may include one or more computer processors, non- persistent storage (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive such as a compact disk (CD)drive or digital versatile disk (DVD) drive, a flash memory, etc.), a communication interface (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.), and numerous other elements and functionalities.

[0035] A computer processor of the control system 142 may be part or all of an integrated circuit for processing instructions. For example, the computer processor may be one or more cores or micro-cores of a processor. The control system 142 may also communicate with one or more input devices, such as a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device.

[0036] A communication interface of the control system 142 may include an integrated circuit for connecting the control system 142 to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, mobile network, or any other type of network) and / or to another device, such as another control system 142.

[0037] Further, the control system 142 may communicate with one or more output devices, such as a display device (e.g., a liquid crystal display (LCD), a plasma display, touchscreen, organic LED display (OLED), projector, or other display device), a printer, a speaker, external storage, or any other output device. One or more of the output devices may be the same or different from the input device(s). Many different types of control systems exist, and the aforementioned input and output device(s) may take other forms.

[0038] Software instructions in the form of computer readable program code to perform embodiments of the disclosure may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer readable medium such as a CD, DVD, storage device, a diskette, a tape, flash memory, physical memory, or any other computer readable storage medium. Specifically, the software instructions may correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments of the invention.

[0039] FIG. IB shows another example computer-assisted system 100, in accordance with one or more embodiments. Analogous to the computer-assisted system 100 shown in FIG. 1A, in the example, a diagnostic or therapeutic medical procedure is performed on a patient 198 on an operating table 190. The computer-assisted system 100 may include a robotic manipulating system 110 (e.g., a patient-side robotic device in a medical example). The robotic manipulating system 110 may include at least one manipulator arm 150A, 150B, 150C, 150D, each of which may support a removably coupled instrument 160 (also called tool 160).

[0040] The number of instruments 160 used at one time generally depends on the task and space constraints, among other factors. Various classes and types of instruments 160 can be used, or exchanged during a process or procedure, according to the needs of a procedure or process undertaken by one or more manipulator arms (e.g., 150A, 150B, 150C, 150D) of a computer-assisted system 100. In both the computer-assisted system 100 of FIGs. 1A and IB, instruments 160 may include, but are not limited to: monopolar and bipolar instruments for dissection, coagulation, sealing, transection, and / or cutting of tissue; suction and irrigation instruments; clip appliers; and instruments for visualization of the procedure site (e.g., visual camera, infrared camera. Some instruments 160, such a monopolar and bipolar instruments, can be “energized” or apply electrical cortical stimulation in a tunable manner (e.g., “on” or “off’).

[0041] Analogous to the computer-assisted system 100 of FIG. 1 A, the computer- assisted system 100 of FIG. IB may include a control system 142, a display 144, a user control system 120, and / or an auxiliary system 140, as previously described.

[0042] While FIGs. 1A and IB depict particular computer-assisted systems with particular robotic manipulation systems, embodiments of the disclosure are applicable to computer-assisted systems with any type of robotic manipulation systems.

[0043] FIG. 2 depicts a manipulator arm in accordance with one or more embodiments. Generally, a manipulator arm may be configured to support and instrument at a distal end of the manipulator arm, and consists of a sequence of joints and links that move in coordination forming a kinematic configuration that determines the position of the instrument. The combination of the manipulator arm with and instrument may form a manipulator assembly or manipulator. The manipulator arm 150 of FIG. 2 has eleven labelled joints, namely, a first joint Ji, a second joint J2, and so on and so forth until the eleventh joint Ju. In some instances, joints are connected by links. The manipulator arm 150 of FIG. 2 has nine links annotated as the first link Li, second link L2, and so on and so forth until the ninth link L9. Generally, links provide a spatial separation between joints and introduce a greater range of motion to the manipulator arm 150. Joints can be rotary or prismatic. Various aspects of manipulator arms in accordance with embodiments of the disclosure are described in International Patent Application PCT / US2023 / 020441, which is incorporated herein by reference.

[0044] At a proximal end, the manipulator arm 150 may be supported by a base (not shown), and at a distal end, the manipulator arm may support one or more instruments.

[0045] More specifically, at the proximal end, the manipulator arm 150, in the example, includes a carriage which forms the first link Li of the manipulator arm. The carriage may move in a translational degree of freedom A along a rail or other structure (not shown). The rail may be disposed on the operating table 190, shown in FIG. 1, which may form the base. Accordingly, movement in degree of freedom A provided by a prismatic joint formed between the rail and the carriage enables the first link Li, which supports the entire manipulator arm 150, to translate along the operating table.

[0046] Additional degrees of freedom of the manipulator arm are subsequently described in a direction from proximal to distal.

[0047] A rotational degree of freedom B is provided by the rotational joint J2, between Li and L2; a rotational degree of freedom C is provided by a rotational joint J3 between Li and L2; a rotational degree of freedom D is provided by a rotational joint J4 between L3 and L4; a translational degree of freedom E is provided by a prismatic joint J5 between L4 and L5; a rotational degree of freedom F is provided by a rotational joint F> also between L4 and L5 (or alternatively joints J5 and F> may be combined in a single joint capable of translation and rotation); the rotational degree of freedom G is provided by a rotational joint J7 between L5 and Le; the rotational degree of freedom H is provided by a rotational joint Js between Lz, and L7; the rotational degree of freedom I is provided by a rotational joint J9 between L7 and Ls, the rotational degree of freedom J is provided by a rotational joint J10 also between L7 and Ls, and the translational degree of freedom K is provided by a prismatic joint Ju between Ls and L9.

[0048] L9 forms the distal end of the manipulator arm 150 and may be configured as an instrument holder 252 for the attachment of an instrument or tool (not shown). Herein, the term “tool” encompasses both general or industrial robotic tools and specialized robotic surgical instruments. Often, specialized robotic surgical instruments include a so-called “end effector” that is suitable for manipulation of tissue, treatment of tissue, imaging of tissue, or the like. Lor example, the end effector may be an actuatable gripping device, and endoscopic camera, etc. Attachment of an instrument can be provided on the distal end L9 of the manipulator arm 150 through a quick-disconnect coupling of the instrument holder 252. Actuation of the end effector can be controlled by the manipulator arm 150 upon attachment of the instrument. In a medical scenario, the positioning of a surgical instrument relative to the patient may be controlled by adjustment of the joints Ji-Ju of the manipulator arm.

[0049] The instrument holder 252 may include an interface to provide driving forces or other inputs to the mounted instrument (not shown) to control various degrees of freedom movement and / or other functionality of the instrument. An accessory mount portion 254, disposed on Ls may be configured to receive an accessory mounted thereon, such as a cannula 256 (a cannula 256 is illustrated mounted to the accessory mount portion 254 in FIG. 2 as an example). The cannula 256 mounted to the accessory mount portion 254 can be positioned to receive an instrument shaft of an instrument mounted to the instrument holder 252. The instrument shaft and a passage through the cannula can define an insertion axis along with the instrument can translate in response to translation of the instrument holder 244 along Ls in degree of freedom K. A remote center of motion, further discussed below, can be located on the insertion axis in, at, or near the cannula. The remote center, in some embodiments, is software-configurable and may be placed such that it is located at a point where the cannula 256 enters the patient (e.g., through an incision in the skin). To avoid mechanically straining the surrounding tissue, translational movement of the cannula at the remote center (aside from insertion and retraction of the cannula) may be prevented, whereas pivoting movements of the cannula about the remote center, which can be performed without mechanically straining the surrounding tissue, may be allowed.

[0050] While FIG. 2 shows a particular configuration of a manipulator arm with a particular arrangement of links, connected by joints, embodiments of the disclosure are not limited to this configuration. Any configuration with any type and number of joints and links may be used.

[0051] In one or more embodiments, all or most joints of the manipulator arm (e.g., joints Ji-Jn of manipulator arm 150) are driven joints, i.e., joints equipped with an actuator that enables a commanded reconfiguration of the joints. Further, joint of an instrument disposed on the manipulator arm may be driven joints as well. Accordingly, most of all joints of the manipulator assembly or manipulator (consisting of the combination of the manipulator arm and the instrument) are driven joints. A driven joint may include a joint actuator (e.g., a servo motor, a stepper motor, etc.) and all necessary components for operating and stopping the joint. For example, a joint actuator can include a motor, one or more joint sensors, gears, and / or a brake. A joint may be driven locally (with the joint actuator included in a joint assembly) or remotely (with the joint actuator(s) remotely driving the joint, e.g., using pull wires). The motor may change the state of the joint (e.g., position, velocity, acceleration), the joint sensor(s) may sense the state of the joint, and the brake (if present) may maintain astationary position of the joint relative to a link proximal to the joint. The term “state” of a joint or the like will herein refer to the control variables associated with the joint. For example, the state of a rotary joint can include an angle (0) of the joint relative to an angular datum of the joint. Thus, the orientation of a rotary joint is known given its state or current angle. The state of a joint can further include the velocity (e.g., angular velocity (0)) at which the joint is moving. Similarly, the state of an axial or prismatic joint may refer to the joint’s axial position (x), and / or to its axial velocity (x).

[0052] In some embodiments, when performing a procedure using the manipulator arm 150, i.e., when in a task execution mode, at least some joints (e.g., some of joints Ji-Ju and / or joints of the instrument disposed on the manipulator arm) are controlled either teleoperatively by a user, or by an autonomously or semi-autonomously performing algorithm. For example, a human operator (e.g., a surgeon) may manipulate one or more leader input controls to command motion of the instrument or an end effector of the instrument, which is enabled by a reconfiguration of the manipulator by one or more of the joints of the instrument and / or the manipulator arm 150.

[0053] The kinematic configuration of the manipulator arm 150 is completely specified by the state of the joints of the manipulator arm 150 in view of the relative positions and given lengths / sizes of the intervening links, if any. Operation of the manipulator arm 150, which primarily consists of controlling the manipulator arm 150 through a continuous space of joint states, is governed by a control system, e.g., control system 142.

[0054] The control system 142 may perform at least some of the calculations to determine the kinematic configuration of the manipulator arm 150 and the sequence and timing of joint states that should be undertaken to achieve a desired kinematic configuration. Often, there will be many potential sequences of joint states that can achieve a desired kinematic configuration such that the control system 142 may also apply a constrained optimization routine to select a sequence of joint states (or select a path of traversal through a space of joint states) that is safe (i.e., avoids collisions between joints and links of the manipulator arm 150) while optimizing some predefined criterion / criteria (e.g., a minimum length path through the space of joint states, maximizing the range of motion of all available joints upon achieving the desired kinematic configuration, etc.).

[0055] In some embodiments, when in a task execution mode, only a subset of the joints of the manipulator arm (e.g., a subset of joints Ji-Ju) are driven to change their joint states when the manipulator arm is teleoperatively controlled by a user or by anautonomously or semi-autonomously performing algorithm, e.g. to perform a procedure. The remaining joints may be locked, e.g., by engaging their brakes, thereby preventing a change of their joint states while in the task execution mode. For example, some of the more distal joints of the manipulator arm (e.g., joints J5-J11) may be driven while in the task execution mode, whereas the more proximal joints (e.g., joints J1-J4) may be prevented from being driven while in the task execution mode. In some embodiments, whether joints are driven / prevented from being driven while in the task execution mode is specific to the mechanical and / or control design of the manipulator arm. In the example of FIG. 3A, J3 and J4 may be so-called setup joints (discussed in detail below) that would be prevented from being driven while in task execution mode, whereas the other remaining joints may be driven while in task execution mode. In an alternative configuration, J3, J4, J5, and J9 may be considered setup joints. In some embodiments, there is not strict separation between setup joints and other (non-setup) joints. In this case, whether or not a joint is considered setup joint may depend on the task at hand, spatial constraints, and / or any other factors that may affect task execution. Accordingly, the same joint may be operated as a setup joint under certain conditions, and as a non-setup joint under other conditions. Based on this distinction, a manipulator arm 150 in accordance with embodiments of the disclosure may be defined to include a distal manipulator with distal joints and a proximal manipulator with proximal joints. In this configuration, the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support the instrument. While both proximal joints and distal joints are all driven joints, in some embodiments they have different characteristics.

[0056] The distal joints may be specifically designed to accommodate movements as they are typical for teleoperative control, semi-autonomous, or autonomous control. As a result, they have characteristics such as a frequency response, an accuracy, etc., that may be desirable for these operating modes. For example, a frequency response that accommodates fast and yet accurate movements may be desirable in a medical scenario that involves a surgeon teleoperating an instrument disposed on the manipulator arm. This ensures that the end effector of the instrument follows the surgeon’s input with ideally no perceivable delay.

[0057] In contrast, the proximal joints (or setup joints) may be designed to enable or facilitate setting up (e.g., positioning and / or orienting) of the distal manipulator prior to performing a procedure under teleoperative, semi-autonomous, or autonomous control, i.e., when in the task execution mode. This may be motivated by the distal manipulator incombination with the instrument having a limited range of motion. While that range of motion may be sufficient to perform a procedure at a work site (as further illustrated below in reference to FIG. 3A), there may be circumstances that require or would benefit from a different range of motion. Examples include, but are not limited to, navigating around an obstacle, targeting the work site from a different angle, using an instrument with different kinematic characteristics, etc. A more detailed description is provided below. In such a situation, it may be desirable to change the kinematic configuration of the manipulator arm, including the kinematic configuration of the proximal manipulator to improve alignment of the end effector with the work site. By changing the joint configuration of one or more proximal joints, the distal manipulator may be repositioned and / or reoriented such that the required range of motion is available to continue or complete the procedure. An entire procedure may even include repeated kinematic reconfiguration of the proximal manipulators section, depending on the task to be performed.

[0058] Accordingly, in some embodiments, the proximal joints have characteristics optimized for the proximal manipulator to provide support for the distal manipulator, while enabling reconfiguration when necessary or desirable.

[0059] In some embodiments, to provide support for the distal manipulator, the proximal manipulator is designed to have a high stiffness when in a locked state, thereby ensuring good support of the distal manipulator with the instrument while a procedure is performed either teleoperationally, semi-autonomously, or autonomously. The implication is that the kinematic operations performed to control the distal joints and the instrument can be performed under the assumption that a base of the distal manipulator (where the distal manipulator is disposed on the proximal manipulator) is stationary in space, and remains mostly stationary even in presence of interaction forces from the work site or elsewhere.

[0060] The desired stiffness may be provided by links of the proximal manipulator that are mechanically stiff, even in presence of the interaction forces and / or torques, and potentially high static loads imposed on the proximal manipulator by the distal manipulator extending from the proximal manipulator. The desired stiffness may further be provided by proximal joints that are positionally locked while a procedure is performed either teleoperationally, semi-autonomously, or autonomously. The proximal joints may be equipped with brakes that are engaged to positionally lock the proximal joints. Further, the proximal joints may be designed to be relatively larger and heavier, equipped with more powerful motors and brakes.

[0061] All of these measures increase stiffness but at the same time limit the frequency response. This tradeoff may make the proximal manipulator less suitable for teleoperation (or semi-autonomous, or autonomous operation). In contrast, the distal joints being smaller, lighter, potentially equipped with less powerful motors, may not necessarily having brakes, have the capability to move faster, thereby qualifying them for teleoperation, semi-autonomous, or autonomous operation rather than setup operations.

[0062] Turning to FIGs. 3A-3E, in one or more embodiments, a setup operation is performed to position and / or orient the distal manipulator with the instrument by reconfiguring the proximal manipulator of the manipulator arm. Various reasons may trigger such a setup operation, as further discussed below in reference to the flowchart of FIG. 4.

[0063] FIGs. 3A-3E illustrate various such setup operations that are intended to achieve different goals. The setup operations as discussed have in common that they involve driving at least one proximal joint to change its kinematic configuration.

[0064] FIG. 3A introduces a first configuration 300A from which other configurations may be reached using a setup operation. These other configurations are illustrated in FIGs. 3B-3E. The schematically illustrated scenario shown in FIG. 3A may correspond to the scenario shown in FIG. 1. A patient 398 is placed on an operating table 390. Two manipulator arms 350, 380 are mechanically attached to the operating table 390. For the purpose of simplifying the subsequent discussion, the manipulator arms 350, 380 are shown as having a limited number of joints. All joints are assumed to operate in the D- plane as shown. More generally, the manipulator arms 350, 380 may have any number of joints, e.g., the joints as illustrated in FIGs. 1 and 2. In the configuration as shown, the manipulator arm 350 supports an instrument 360. The instrument 360 has an instrument shaft 362 that enters the patient 398 (e.g., through an incision). An end effector 364 is distally disposed on the instrument shaft 362. The end effector may be designed to engage in a task at a work site 386 when teleoperated, or autonomously / semi-autonomously operated. In a medical scenario, the task may be a surgical task such as cutting, clamping, irrigating, ablating, suturing, etc., of an anatomical structure, e.g., a blood vessel. The second manipulator arm 380, in the illustration, supports an imaging instrument 382. The imaging instrument 382 may be, for example, an endoscopic camera that allows an operator (e.g., a surgeon performing the procedure at the work site 386) to monitor the ongoing procedure.Accordingly, the imaging instrument 382 may be positioned / oriented such that a field of view 384 of the imaging instrument 382 captures the work site 386 and the end effector 364interacting with the work site 386. The position and / or orientation of the imaging instrument may be adjusted as needed, analogous to how instrument 360 with the end effector 364 may be controlled.

[0065] In one or more embodiments, the manipulator arm 350 includes a proximal manipulator 352 and a distal manipulator 354. The proximal manipulator 352 includes proximal joints connected by links, and the distal manipulator includes distal joints connected by links. In the scenario as shown, J3 and J4 are considered proximal joints, and J7, J9, Ji 1 are considered distal joints. The proximal joints and the distal joints may have characteristics as previously discussed. The distinction between proximal vs distal joints is not limited to the example illustrated in the scenario as shown. For example, in another scenario, J3, J4, and J7 may be considered proximal joints, and J9 and Ju may be considered distal joints.

[0066] In some embodiments, the teleoperation, autonomous or semi-autonomous operation may be limited by a range of motion. The range of motion may be defined in different manners, and under each definition the range of motion, when reached, may disrupt the teleoperation, autonomous or semi-autonomous operation. For example, the range of motion may be defined as a task range of motion or a range of motion of a distal joint. The task range of motion may relate to a range of motion in general terms, needed to execute a task at work site 386. For example, the task range of motion may need be such that the end effector 364 can reach the work site 386. The task range of motion may also need to be such that the end effector 364 can reach the work site 386 from a desired or required angle. In addition, the task range of motion may need to be such that the end effector 364 can perform a manipulation operation at the work site 386 as needed to complete the task. Whether or not the required task range of motion is available may depend on various factors, including a range of motion of one or more of the distal joints, as subsequently discussed.

[0067] In one or more embodiments, each of the distal joints may have a range of motion. In other words, a prismatic joint may be limited to a translational movement within a fixed linear range, and a rotary joint may be limited to a rotational movement within a fixed angular range. Once a distal joint reaches its range of motion limit, the teleoperation, autonomous or semi-autonomous control of the manipulator arm and / or instrument may be affected, if it requires movement of the distal joint beyond the range of motion limit. In this case, it may not be possible to continue the teleoperation, autonomous or semi-autonomous operation as intended. Accordingly, operation of a distal joint near the center of its range of motion may be desirable, because it reduces the likeliness of reaching and eventuallyviolating the range of motion limit of the distal joint. While, generally speaking, all distal joints (and instrument joints) may be involved in the teleoperation, autonomous or semi- autonomous operation, some distal joints may require larger amplitude joint movements than other distal joints, depending on the task to be performed, the current configuration of the manipulator arm 350, etc. In such a case, it may be particularly desirable to have distal joints that are known to require larger amplitude joint movements centered within their respective range of motion, while allowing distal joints that are known to require relatively limited amplitude joint movements to operate closer to their respective ranges of motion.

[0068] The task range of motion may further be affected by the possibility of a collision. A collision of an element of the manipulator arm 350 or an element of the instrument 360 with an obstacle may prevent completion of a movement associated with teleoperation, autonomous or semi-autonomous operation. In the scenario shown in FIG. 3A, such a collision could happen, for example, between manipulator arm 350 and manipulator arm 380. The manipulator arm 350 (and / or manipulator arm 380) may, thus, be reconfigured to avoid or move away from a potential collision site, e.g., by ensuring that a minimum distance is maintained.

[0069] The task range of motion may further be affected by additional constraints. For example, in some embodiments, the manipulator arm 350 is configured to maintain a remote center 388. As previously described, the remote center is typically placed at the incision, or wherever else the instrument shaft 362 traverses a body wall of the patient 398, thereby ensuring that no translational movement occurs at that point to prevent mechanical tension on the incision. When considering the control problem of moving the manipulator arm 350 with the instrument 360 during teleoperation, autonomous or semi-autonomous control as a problem to be solved in an n-dimensional joint space (n being the number of distal joints and instrument joints), the presence of the remote center 388 makes some areas of this n-dimcnsional joint space inaccessible. Depending on the inaccessible area, this may result in undesirable limitations of the task range of motion. Accordingly, it may be desirable to reconfigure the manipulator arm with the inaccessible joint space further away from where it would affect the task range of motion.

[0070] FIG. 3 A shows an example of range of motion limits 391. The range of motion limits 391 are for a pivoting about the remote center 388. In the example as shown, the instrument shaft 362 is centered within the range of motion, i.e., the instrument shaft 362 may pivot by an equal amount in either direction before reaching the range of motion limits 391.The range of motion limits may be software-imposed, or they may be a result of the current kinematic configuration of the manipulator arm 350, given range of motion limits of the distal joints.

[0071] In one or more embodiments, any of the above conditions may be addressed by a setup operation involving the proximal joints of the proximal manipulator. Various examples are subsequently discussed in reference to FIGs. 3B-3E. While these examples are based on the manipulator arm 350, the methods, e.g., as described in reference to FIG. 4 are applicable to any type of manipulator arm with any kinematics, and any type and number of joints, without departing from the disclosure. For example, the methods may also be applicable to the manipulator arm 380.

[0072] FIG. 3A identifies three different reference frames including a reference frame of the manipulator arm 392, a reference frame of operating table 394, and a reference frame of a patient body on the operating table 396. These reference frames are discussed following the introduction of the configurations illustrated in FIGs. 3B-3E.

[0073] FIG. 3B illustrates a second configuration 300B, in accordance with embodiments of the disclosure. In the example as illustrated, the transition from the first configuration 300A of FIG. 3A to the second configuration 300B of FIG. 3B in a setup operation is accomplished by driving the proximal joint J3 in a clockwise direction as illustrated. The setup operation may be performed as discussed below in reference to FIG. 4. None of the other joints of the manipulator arm 350 are driven during the setup operation. As a result of driving proximal joint J3, the entire manipulator arm 350 (including the manipulator 352 and the distal manipulator 354) and the instrument 360 pivot as shown. Because the setup operation is limited to proximal joint J3, it results in a translation of the remote center 388. Further, it results in a translation / rotation of the end effector 364 away from the work site 386. Other setup operations that maintain the remote center and / or the end effector position / orientation are discussed below in reference to FIGs. 3C-3E.

[0074] FIG. 3C illustrates a third configuration 300C, in accordance with embodiments of the disclosure. The third configuration 300C, in comparison to the first configuration 300A, changes the orientation of the instrument shaft 362 of instrument 360, as it enters the patient body 398, while maintaining the position of the remote center 388. A transition from the first configuration 300A to the third configuration may be useful, for example, to aim for the work site 386 from a different angle. In the example as illustrated, the setup operation that results in the transition from the first configuration 300A of FIG. 3Ato the third configuration 300C of FIG. 3C is accomplished by driving the proximal joint J3 in a counterclockwise direction, the proximal joint J4 in a clockwise direction, and the distal joint J9 in a counterclockwise direction. The setup operation may be performed as discussed below in reference to FIG. 4. None of the other joints of the manipulator arm 350 are driven during the setup operation. As a result of driving proximal joints J3, J4, and the distal joint J9, the kinematics of the manipulator arm 350 (including the proximal manipulator 352 and the distal manipulator 354) and the instrument 360 change as shown. To facilitate a direct comparison to the first configuration 300A, the first configuration 300A is shown using dotted lines. The setup operation maintains the position of the remote center 388, while not enforcing a particular position / orientation of the end effector 364. More specifically, the end effector 364 pivots along with the instrument shaft 362 (while remaining stationary relative to the instrument shaft 362).

[0075] FIG. 3D illustrates a fourth configuration 300D, in accordance with embodiments of the disclosure. The fourth configuration 300D, in comparison to the first configuration 300A, maintains the orientation of the instrument shaft 362 of instrument 360, as it enters the patient body 398, and the position of the remote center 388, while causing a change in the kinematic configuration of the distal manipulator 354 such that the range of motion limits 391 change as illustrated. Assume, for example, that the range of motion limit 391, in the example, is caused by a range of motion limit of J9. Accordingly, changing the kinematic configuration of the manipulator arm 350 such that J9 is reconfigured can shift the range motion limits 391 as desired. A transition from the first configuration 300A to the fourth configuration may be useful, for example, to accommodate an upcoming operation that requires the instrument to be oriented differently (and outside the original range of motion limits 391 shown in FIG. 3 A. In the example as illustrated, the setup operation that results in the transition from the first configuration 300A of FIG. 3A to the fourth configuration 300D of FIG. 3D is accomplished by driving the proximal joint J3 in a counterclockwise direction, the proximal joint J4 in a clockwise direction, the distal joint J7 in a counterclockwise direction and the distal joint J9 in a clockwise direction. The setup operation may be performed as discussed below in reference to FIG. 4. To facilitate a direct comparison to the first configuration 300A, the first configuration 300A is shown using dotted lines.

[0076] FIG. 3E illustrates a fifth configuration 300D, in accordance with embodiments of the disclosure. The fifth configuration 300E, in comparison to the first configuration 300A, involves a pivoting of the operating table 390. The pivoting results in achange in the mass distribution of the patient body 398. More specifically, some volume of the patient body shifts to the right, caused by gravity. Without a reconfiguration of the manipulator arm 350, this may trigger various concerns. For example, a collision between the patient body 398 and the manipulator arm 350 may be increasingly likely. In the illustration of FIG. 3E, the manipulator arm 350 in its original configuration (as introduced in FIG. 3A, but pivoted along with the operating table 390) is shown using dotted lines, and it can be seen that J9, in absence of the reconfiguration, is close to being in contact with the patient body 398. Furthermore, both the location of the incision and / or the work site may have shifted, thereby requiring an updating of the remote center and / or the end effector.

[0077] In the example as illustrated, it is assumed that the incision has shifted, requiring an updating of the remote center 388. It is further assumed that the work site has not shifted, therefore keeping the end effector stationary. Accordingly, the fifth configuration 300E changes the orientation of the instrument shaft 362 of instrument 360, as it enters the patient body 398, and the position of the remote center 388, while keeping the end effector stationary. The setup operation may be performed as discussed below in reference to FIG. 4.

[0078] While FIGs. 3A-3E illustrate embodiments of the disclosure in a medical context, similar scenarios may occur in non-medical scenarios, where embodiments of the disclosure are equally applicable. Further, while for the sake of simplicity, embodiments of the disclosure are illustrated in a 2D plane, embodiments of the disclosure generalize to a 3D space and any number of manipulator arms of any type operating therein. Finally, while range of motion limits are illustrated for a pivoting about a remote center, range of motion limits may be present elsewhere and may be the determining factor triggering a kinematic reconfiguration.

[0079] In the discussion of FIGs. 3A-3E, it is pointed out that various elements, e.g., the end effector and the remote center are either being moved or being kept stationary during a kinematic reconfiguration of the manipulator arm. Different reference frames may be used to describe position and / or orientation of the remote center and end effector. For example, it may be said that the remote center and / or the end effector are kept stationary / are moved in a reference frame of the manipulator arm. This may be a meaningful characterization if one assumes that the patient remains stationary relative to the reference frame of the manipulator arm. The characterization may be less meaningful if the patient shifts relative to the reference frame, e.g., because the patient is moved on the operating table, or because theoperating table with the patient on the operating table moves relative to the reference frame of the manipulator arm.

[0080] A reference frame of the operating table may thus be used for scenarios that involve movement of the operating table relative to the manipulator arm. A reference frame of the patient may be used for scenarios that involve movement of the patient relative to the manipulator arm. The most appropriate reference frame may be used when executing the method described in FIG. 4. While three reference frames have been introduced, other reference frames may be used without departing from the disclosure. For example, a world frame (which may be anchored to the base of the operating table) or a local frame attached to a specific location of the patient body may be used.

[0081] Turning to FIG. 4, a flowchart in accordance with one or more embodiments is shown. The flowchart of FIG. 4 depicts a method 400 for computer-assisted systems. On a high level, the described steps may be performed for a computer assisted system equipped with a manipulator arm that comprises a distal manipulator with distal joints and a proximal manipulator with proximal joints, where the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument. The method 400 may be used to facilitate the operation of the manipulator arm through reconfiguration of the proximal segment of the manipulator.

[0082] One or more of the steps in FIG. 4 may be performed by various components of systems, previously described with reference to FIGs. 1 and 2. While these figures illustrate particular configurations of computer assisted systems, the method is equally applicable to other configurations.

[0083] The method may be executed on one or more processors, e.g., of the control system(s) 142 of the computer-assisted system.

[0084] While the various steps in the flowchart are presented and described sequentially, one of ordinary skill will appreciate that some or all of the steps may be executed in different orders, may be combined or omitted, and some or all of the steps may be executed in parallel. Additional steps may further be performed. Furthermore, the steps may be performed actively or passively. For example, some steps may be performed using polling or be interrupt driven in accordance with one or more embodiments of the disclosure.

[0085] While not shown, the operations of the method 400 may be executed in a loop, thereby enabling repeated reconfigurations of the manipulator arm, e.g., during an ongoingprocedure. Additional operations that are not shown in the flowchart may be performed without departing from the disclosure.

[0086] Turning to the method 400, in Step 402 it is determined whether the computer- assisted system is currently operating in task execution mode (i.e., in teleoperation mode, autonomous, or semi-autonomous mode). Broadly speaking, Step 402 tests whether the manipulator arm is involved in actively performing a task, e.g., (in a medical scenario) interacting with tissue at a work site, which typically occurs with the manipulator arm and the instrument being either teleoperated or operated by an autonomous or semi-autonomous algorithm.

[0087] Step 402, in some embodiments, is used to ensure that a kinematic reconfiguration is allowed only when the when the manipulator arm is not currently involved in actively performing a task as previously mentioned. The reconfiguration in accordance with embodiments of the disclosure may, however, be performed in a mode different from the teleoperation mode (or semi-autonomous, or autonomous mode).

[0088] Whether or not the computer-assisted system is currently in a task execution mode may be information available system-internally as, for example, state information. Accordingly, Step 402 may be performed by internally querying the computer-assisted system.

[0089] If the computer assisted system is determined to be in the task execution mode, the execution of the method 400 may proceed with Step 404. Alternatively, if the computer assisted system is determined not to be in the task execution mode, the execution of the method 400 may proceed with Step 408.

[0090] In Step 404, the proximal joints of the manipulator arm are positionally locked. The positional locking of the proximal joints may occur prior to or at the time when entering the task execution mode. The locking may be performed using the brakes of the proximal joints. Step 404 may ensure that the brakes of all the proximal joints are engaged.

[0091] In Step 406, the distal joints may be driven according to a teleoperation input, or alternatively, according to an input generated by an autonomous or semi-autonomous algorithm (i.e., according to a task execution input).

[0092] In a teleoperated example, a human operator manipulates one or more leader input controls to command motion of one or more follower instruments. Alternatively, the one or more follower instruments may be driven by an autonomous or semi-autonomous algorithm.

[0093] The architecture of the control methods used for controlling the manipulator arm and the instrument can be of any appropriate form. As a specific example, the control architecture can be hierarchical, and could include a high-level controller and multiple joint controllers. A commanded movement is received by the high-level controller in, for example, a Cartesian-coordinate space (referred to herein as Cartesian- space). The commanded movement could be, for example, based on a movement command (e.g., in the form of a position and / or velocity) received from a user input system, or any other system that provides a movement command (e.g., an autonomous or semi-autonomous algorithm). The commanded movement is converted into commanded joint positions or joint velocities (e.g., linear or angular joint positions, linear or angular joint velocities). In some embodiments, the conversion is performed using an inverse kinematics algorithm. Subsequently, the joint controllers convert the received commanded joint positions or velocities into commanded currents to drive the actuators producing joint movements of the distal joints. The joint movements together produce a movement of the manipulator arm and the instrument that reflects the commanded movement. In some embodiments, controlling the distal joints is performed under consideration of potentially one or more constraints. Such constraints may be imposed by range of motion limitations on a cartesian and / or joint level, velocity limits, spatial constraints such as the maintaining of a remote center (as previously described), collision avoidance, and other constraints that may be satisfiable when controlling a manipulator arm and the instrument. Various aspects of control systems that enable such operation are described in U.S. Pat. No. 6,699,177, which is incorporated herein by reference.

[0094] The computer-assisted system may remain in the task execution mode for a prolonged time. Departure from this mode may be detected in Step 402 as the method 400 is executed in a loop.

[0095] In Step 408, it is determined whether a change of the kinematic configuration of the manipulator arm is desirable or necessary. Various conditions, e.g., as previously discussed in reference to FIGs. 3A-3E may make a kinematic reconfiguration desirable or necessary. A few examples are subsequently provided.

[0096] Regarding the general motivation for a kinematic reconfiguration, as previously noted, the initial setup of the proximal manipulator to position / orient the distal manipulator with the instrument may be performed based on workspace considerations. Workspace requirements may, however, change over the duration of a procedure, e.g., whenmoving to a different work site, targeting the work site from a different angle, etc. As a result, the limits of the task range of motion of the manipulator arm may be reached. The task range of motion may be affected for example, by the range of motion of one or more distal joints, but also by possible collisions with other objects, as previously discussed.

[0097] Accordingly, it may be determined that a change of the kinematic configuration is desirable or necessary, based on a detection that one of the distal joints is in a joint configuration close to a range of motion limit. The reaching of a range of motion limit may be detected using a joint sensor such as a joint encoder.

[0098] Similarly, it may be determined that a change of the kinematic configuration is desirable or necessary, based on a detection that a collision condition is present for the manipulator arm, e.g., based on a proximity of a nearby object, the current movement trajectory, etc. Collision conditions may be determined for collisions with the patient body, another manipulator, and other generic objects in the environment of the manipulator arm. Collision conditions between manipulator arms may be detected using the known kinematics of the potentially colliding manipulator arms, and more generally collision conditions may be detected using sensors including cameras, LIDAR sensors, etc. In a scenario that involves the pivoting of the operating table, the determination that a collision condition may be present can be made based on a detection of the tilting of the operating table.

[0099] The above detections may be made by the computer assisted system without requiring user input. Alternatively, the user may provide a control input to indicate that a kinematic configuration is desired or necessary.

[0100] If it is determined that a change of the kinematic configuration of the manipulator arm is necessary or desirable, the execution of the method may proceed with Step 410.

[0101] In some embodiments, an absence from the task execution mode for at least a specified duration (e.g., a second or a few seconds) may be required for the manipulator arm prior to proceeding to Step 410. Requiring the absence from the task execution mode avoids initiating the kinematic reconfiguration when the user is merely taking a brief break from teleoperating the manipulator arm. This may help avoid unintentional interruption of an ongoing procedure.

[0102] In Step 410, at least one of the proximal joints is driven to change the kinematic configuration of the manipulator arm. As previously discussed, the proximal joints are equipped with actuators. Accordingly, the kinematic reconfiguration is performed byissuing a command to the actuator(s) to drive the at least one proximal joint. The command may be determined in different manners.

[0103] In one scenario, the command is determined based on a command by a user (e.g., a surgeon) manually controlling the at least one proximal joint. In another scenario, the command is determined by the computer-assisted system, or with the assistance of the computer-assisted system. For example, when the computer-assisted system determines that one or more of the distal joints are operating near a range of motion limit, it may determine a command suitable to reconfigure the at least one proximal joint such that the one or more distal joints reach a more favorable joint configuration. Similarly, the computer-assisted system may determine a command suitable to reconfigure the at least one proximal joint such that the distal manipulator avoids or moves away from a collision condition.

[0104] In one or more embodiments, the command is determined such that it satisfies additional constraints. In some embodiments, an additional constraint is that the end effector is to be kept stationary. By keeping the end effector stationary, the likelihood of any interaction of the end effector with the work site during the kinematic reconfiguration is reduced. In some embodiments, an additional constraint is that the remote center is kept stationary. By keeping the remote center stationary, the location at which the instrument enters the patient body is kept stationary, thereby reducing the likelihood of putting significant stress on the incision during the kinematic reconfiguration is reduced. In some embodiments, an additional constraint is that the remote center and the end effector are kept stationary, with the combined benefits of keeping the remote center and the end effector stationary.

[0105] These additional constraints may be addressed by driving not only the at least one proximal joint, but also other joints. These other joints may be additional proximal joints, distal joints, and / or instrument joints. The combination of all the joints are driven such that the kinematic reconfiguration occurs, while the additional constraints remain satisfied. This behavior is possible based on the kinematic redundancies in the manipulator arm and the instrument, which provide the required null-space. To keep the remote center stationary, the kinematic reconfiguration may be performed in a null space of the remote center. To keep the end effector stationary, the kinematic reconfiguration may be performed in a null space of the end effector. To keep the remote center and the end effector stationary, the kinematic reconfiguration may be performed in a null space of the remote center and the end effector. The driving of the joints under consideration of a null space as described may be performedby a control method as used to perform Step 406. However, in contrast to the execution of Step 406 which is limited to the driving of distal joints and / or instrument joints, the execution of Step 410 involves the driving of one or more proximal joints, and potentially distal joints and / or instrument joints. Depending on the scenario to be addressed, stationarity of the remote center and / or end effector in different reference frames may be required, as previously discussed in reference to FIGs. 3A-3E.

[0106] In one or more embodiments, when the kinematic reconfiguration can be performed in multiple different manners due to the kinematic redundancies in the manipulator arm, the reconfiguration is performed in one manner that is selected based on a priority criterion. The priority criterion may prioritize a reconfiguration goal that is relatively more important than other reconfiguration goals. For example, when multiple distal joints are to be moved within their respective ranges of motion, it may be desirable to have one particular joint centered within its range of motion, whereas centering the other joints within their respective ranges of motion may be considered a lower priority.

[0107] In one or more embodiments, prior to beginning the driving of the at least one proximal joint the brake of the at least one proximal joint is disengaged. The brake may be reengaged once the kinematic reconfiguration is completed.

[0108] In Step 412, the end effector is prevented from engaging in a task during the driving of the at least one of the proximal joints. This may prevent interactions between the end effector and, for example, tissue at the work site. The end effector may be prevented from engaging in a task, for example, by blocking execution of teleoperation, semi- autonomous / autonomous operation of the manipulator arm.

[0109] In some embodiments, in a computer-assisted system that is equipped with multiple manipulator arms, teleoperation, autonomous / semi-autonomous operation of one or more of the other manipulator arms may be allowed even during the reconfiguration performed at Step 410. This may be beneficial, for example, to monitor the end effector of the instrument using an imaging instrument (e.g., an endoscopic camera) disposed on one of the other manipulator arms. The imaging instrument, along with the manipulator arm supporting the imaging instrument may, thus, be teleoperated (or autonomously / semi- autonomously operated) to keep track of the end effector, even during the reconfiguration of step 410. Teleoperation (or autonomous / semi-autonomous operation) of the other manipulator arm, may, however affect the ongoing reconfiguration of Step 410. This may be the case, for example, when the teleoperated (or autonomously / semi-autonomously operated)other manipulator arm gets sufficiently close to the manipulator arm being reconfigured to trigger a collision condition. In this case, the ongoing reconfiguration may be altered to address the collision condition.

[0110] In Step 414, the user is notified of the driving of the ongoing kinematic reconfiguration. A status message, an audio signal, or any other form of notification may be provided. The notification may be provided prior to and / or during the kinematic reconfiguration.

[0111] Alternatively, in one or more embodiments, the kinematic reconfiguration is transparent to the user. In such a scenario, the user may not be notified as the reconfiguration is performed. In addition, the reconfiguration may be performed without having been commanded or requested by the user. The reconfiguration, thus, results in an optimally kinematically configured manipulator arm throughout a procedure which would have otherwise been interrupted by one or more reconfigurations to address range of motion or other constraints.

[0112] After completion of the reconfiguration, teleoperation or autonomous / semi- autonomous operation of the manipulator arm may be allowed to resume.

[0113] While the method has been described for a single manipulator arm of a computer-assisted system, it may be performed for any number of manipulator arms, without departing from the disclosure.

[0114] Embodiments of the disclosure provide various benefits. Embodiments of the disclosure eliminate the need to pause an ongoing procedure, manually reconfigure (reposition and / or reorient) the manipulator arm to address a range of motion issue or other concern, and then return to the procedure. Such manual reconfiguration can be tedious and disruptive and may be be undesirable not only because it has the potential to disrupt the workflow, but also because clinical staff tasked with the manual reconfiguration may not necessarily know how to reconfigure the manipulator arm to meet the needs of the surgeon. Embodiments of the disclosure provide improved devices, systems, and methods for surgery, robotic surgery, and other robotic applications that address these concerns. Embodiments of the disclosure provide an enhanced task range of motion without requiring a manual reconfiguration. Embodiments of the disclosure provide the enhanced tasked range of motion by performing a reconfiguration that provides a better or the best kinematic configuration to perform a procedure or part of the procedure based on at least one of the following: (i) a reconfiguration that facilitates the task to be performed by the instrument; (ii) a 1reconfiguration that avoids a collision condition involving the patient; (iii) a reconfiguration that avoids a collision condition between multiple manipulator arms (including consideration of the movement other manipulator arms under teleoperation, autonomous or semi- autonomous control).

[0115] Although only a few example embodiments have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the example embodiments without materially departing from this invention. Accordingly, all such modifications are intended to be included within the scope of this disclosure as defined in the following claims.

Claims

CLAIMSWhat is claimed is:

1. A computer-assisted system comprising: a manipulator arm comprising a distal manipulator with distal joints and a proximal manipulator with proximal joints, wherein the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument with an end effector; a control system comprising one or more processors, the control system configured to: when in a task execution mode: positionally lock the proximal joints; and drive the distal joints according to a task execution input; when in a mode different from the task execution mode: drive at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

2. The computer-assisted system of claim 1, wherein each of the proximal joints is equipped with a brake, and wherein when in the task execution mode, the brakes are engaged to positionally lock the proximal joints.

3. The computer-assisted system of claim 1, wherein each of the proximal joints is equipped with a brake, and wherein when in the mode different from the task execution mode, the brakes are disengaged during the driving the at least one of the proximal joints.

4. The computer-assisted system of claim 1, wherein the control system is further configured to, when in the mode different from the task execution mode: prevent the end effector from engaging in a task during the driving of the at least one of the proximal joints.

5. The computer-assisted system of claim 1, wherein the control system is further configured to, when in the mode different from the task execution mode: drive at least one of the distal joints while driving the at least one of the proximal joints to change the kinematic configuration of the manipulator arm while keeping the end effector stationary.

6. The computer-assisted system of claim 1, wherein the control system is further configured to, when in the mode different from the task execution mode: require an absence from the task execution mode for the manipulator arm for at least a specified duration prior to the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

7. The computer-assisted system of claim 1 further comprising: a second manipulator arm, wherein the control system is further configured to, when in the mode different from the task execution mode: allow task execution using a second manipulator arm.

8. The computer-assisted system of claim 7, wherein the second manipulator arm is configured to support an imaging instrument.

9. The computer-assisted system of claim 8, wherein the imaging instrument has a field of view with the end effector in the field of view, during the driving of the at least one of the proximal joints.

10. The computer-assisted system of any of claims 1-9, wherein to drive the at least one of the proximal joints, the control system is configured to:kinematically reconfigure the manipulator arm such that at least one of the distal joints is centered within a range of motion of the at least one distal joint.

11. The computer-assisted system of any of claims 1-9, wherein to drive the at least one of the proximal joints, the control system is configured to: kinematically reconfigure the manipulator arm such that a task range of motion is increased.

12. The computer-assisted system of claim 11, wherein the task range of motion is increased by kinematically reconfiguring the manipulator arm to move away from a potential collision site.

13. The computer-assisted system of claim 11, wherein the task range of motion is increased by kinematically reconfiguring the manipulator arm to improve alignment of the end effector with a work site.

14. The computer-assisted system of any of claims 1-9, wherein to drive the at least one of the proximal joints, the control system is configured to: keep the end effector stationary in one selected from a group consisting of a reference frame of the manipulator arm, a reference frame of operating table, and a reference frame of a patient body on the operating table.

15. The computer-assisted system of any of claims 1-9, wherein to drive the at least one of the proximal joints, the control system is configured to: further keep a remote center stationary in one selected from a group consisting of a base frame of the manipulator arm, a base frame of an operating table, and a base frame of a patient body on the operating table.

16. The computer-assisted system of any of claims 1-9, wherein the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm is performed without being commanded by a user of the computer-assisted system, and without notifying the user of the driving.

17. The computer-assisted system of any of claims 1-9, wherein the control system is further configured to, when in the mode different from the task execution mode: notify a user of the computer-assisted system of the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm.

18. The computer-assisted system of any of claims 1-9, wherein the control system is further configured to, when in the mode different from the task execution mode: receive a user input to initiate the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm.

19. The computer-assisted system of any of claims 1-9, wherein the control system is further configured to, when in the mode different from the task execution mode: make a determination that one of the distal joints is in a joint configuration close to a range of motion limit, and perform the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary, based on the determination.

20. The computer-assisted system of any of claims 1-9, wherein the control system is further configured to, when in the mode different from the task execution mode: make a determination that a collision condition is present for the manipulator arm, andperform the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary, based on the determination.

21. The computer-assisted system of claim 20, wherein the collision condition involves one selected from a group consisting of a patient, another manipulator, and a generic object.

22. The computer-assisted system of claim 21, wherein making the determination that the collision condition is present is made based on detecting a tilting of an operating table supporting the patient.

23. The computer-assisted system of any of claims 1-9, wherein an aggregate plurality of joints comprising the distal joints and the proximal joints provides a kinematic redundancy in the kinematic configuration of the manipulator arm.

24. The computer-assisted system of claim 23, wherein the kinematic redundancy enables the change of the kinematic configuration to be performed in a plurality of different manners, and wherein the change of the kinematic configuration is performed in one particular manner of the plurality of different manners, based on a priority criterion.

25. A method for facilitating operation of a computer-assisted system comprising: a manipulator arm comprising a distal manipulator with distal joints and a proximal manipulator with proximal joints, wherein the proximal manipulator supports the distal manipulator, and the distal manipulator is configured to support an instrument with an end effector; wherein the method comprises: when in a task execution mode: positionally locking the proximal joints; and driving the distal joints according to a task execution input; when in a mode different from the task execution mode: driving at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

26. The method of claim 25, wherein each of the proximal joints is equipped with a brake, and wherein when in the task execution mode, the brakes are engaged to positionally lock the proximal joints.

27. The method of claim 25, wherein each of the proximal joints is equipped with a brake, and wherein when in the mode different from the task execution mode, the brakes are disengaged during the driving the at least one of the proximal joints.

28. The method of claim 25, further comprising, when in the mode different from the task execution mode: preventing the end effector from engaging in a task during the driving of the at least one of the proximal joints.

29. The method of claim 25, further comprising, when in the mode different from the task execution mode: driving at least one of the distal joints while driving the at least one of the proximal joints to change the kinematic configuration of the manipulator arm while keeping the end effector stationary.

30. The method of claim 25, further comprising, when in the mode different from the task execution mode: requiring an absence from the task execution mode for the manipulator arm for at least a specified duration prior to the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary.

31. The method of claim 25, wherein the computer-assisted system further comprises: a second manipulator arm; and wherein the method further comprises, when in the mode different from the task execution mode: allowing task execution using a second manipulator arm.

32. The method of claim 31, wherein the second manipulator arm is configured to support an imaging instrument.

33. The method of claim 32, wherein the imaging instrument has a field of view with the end effector in the field of view, during the driving of the at least one of the proximal joints.

34. The method of any of claims 25-33, wherein the driving of the at least one of the proximal joints comprises: kinematically reconfiguring the manipulator arm such that at least one of the distal joints is centered within a range of motion of the at least one distal joint.

35. The method of any of claims 25-33, wherein the driving of the at least one of the proximal joints comprises: kinematically reconfiguring the manipulator arm such that a task range of motion is increased.

36. The method of claim 35, wherein the task range of motion is increased by kinematically reconfiguring the manipulator arm to move away from a potential collision site.

37. The method of claim 35, wherein the task range of motion is increased by kinematically reconfiguring the manipulator arm to improve alignment of the end effector with a work site.

38. The method of any of claims 25-33, wherein the driving of the at least one of the proximal joints comprises: keeping the end effector stationary in one selected from a group consisting of a reference frame of the manipulator arm, a reference frame of operating table, and a reference frame of a patient body on the operating table.

39. The method of any of claims 25-33, wherein the driving of the at least one of the proximal joints comprises: further keeping a remote center stationary in one selected from a group consisting of a base frame of the manipulator arm, a base frame of an operating table, and a base frame of a patient body on the operating table.

40. The method of any of claims 25-33, wherein the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm is performed without being commanded by a user of the computer-assisted system, and without notifying the user of the driving.

41. The method of any of claims 25-33, further comprising, when in the mode different from the task execution mode: notifying a user of the computer-assisted system of the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm.

42. The method of any of claims 25-33, further comprising, when in the mode different from the task execution mode: receiving a user input to initiate the driving of the at least one of the proximal joints to change the kinematic configuration of the manipulator arm.

43. The method of any of claims 25-33, further comprising, when in the mode different from the task execution mode: making a determination that one of the distal joints is in a joint configuration close to a range of motion limit; and performing the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary, based on the determination.

44. The method of any of claims 25-33, further comprising, when in the mode different from the task execution mode: making a determination that a collision condition is present for the manipulator arm, and performing the driving of the at least one of the proximal joints to change a kinematic configuration of the manipulator arm while keeping the end effector stationary, based on the determination.

45. The method of claim 44, wherein the collision condition involves one selected from a group consisting of a patient, another manipulator, and a generic object.

46. The method of claim 45, wherein making the determination that the collision condition is present is made based on detecting a tilting of an operating table supporting the patient.

47. The method of any of claims 25-33, wherein an aggregate plurality of joints comprising the distal joints and the proximal joints provides a kinematic redundancy in the kinematic configuration of the manipulator arm.

48. The method of claim 47, wherein the kinematic redundancy enables the change of the kinematic configuration to be performed in a plurality of different manners, and wherein the change of the kinematic configuration is performed in one particular manner of the plurality of different manners, based on a priority criterion.

9. A non-transitory machine-readable medium comprising a plurality of machine- readable instructions executed by one or more processors associated with a computer- assisted system, the plurality of machine-readable instructions causing the one or more processors to perform the method of any of claims 25 to 48.

Images