System and method for managing multiple null-space targets and saturated SLI behavior

Abstract

The invention relates to a system and method for managing multiple null-space targets and saturated SLI behavior. Devices, systems, and methods for providing commanded movement of an end effector of a manipulator while accompanying desired movement of one or more joints of the manipulator according to one or more merged null-space targets. The null-space target may include joint state combinations, relative joint states, ranges of joint states, joint state profiles, kinetic energy, bite movement, collision avoidance movement, singular point avoidance movement, attitude or pitch preferences, desired manipulator configuration, commanded reconfiguration of the manipulator, and anisotropic enhancement of the joints. The method includes calculating a plurality of null-space movements from different null-space targets, determining attributes for each null-space target, and merging the null-space movements by a null-space manager using various approaches. These approaches may include applying weighting, scaling, saturation level, priority, master speed limit, saturation limit integration, and various combinations thereof.

Description

[0001] This application is a divisional application of Chinese patent application 202110307020.3, filed on March 18, 2014, entitled “System and method for managing multiple zero space targets and saturated SLI behavior”, and Chinese patent application 202110307020.3 is a divisional application of Chinese patent application 201811194748.4, filed on March 18, 2014, entitled “System and method for managing multiple zero space targets and saturated SLI behavior”, and Chinese patent application 201811194748.4 is further a divisional application of Chinese patent application 201480014809.8, filed on March 18, 2014, entitled “System and method for managing multiple zero space targets and saturated SLI behavior”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 61 / 800,810, filed on March 15, 2013, entitled “Systems and Methods for Managing Multiple Null-Space Objectives and SLI Behaviors” (Agent Docket No. ISRG03790PROV / US), the entire disclosure of which is incorporated herein by reference.

[0004] This application is generally related to the following commonly owned applications: U.S. Application No. 12 / 494,695, filed on June 30, 2009, entitled “Control of Medical Robotic System Manipulator About Kinematic Singularities”; U.S. Application No. 12 / 406,004, filed on March 17, 2009, entitled “Master Controller Having Redundant Degrees of Freedom and Added Forces to Create Internal Motion”; U.S. Application No. 11 / 133,423, filed on May 19, 2005 (U.S. Patent No. 8,004,229), entitled “Software Center and Highly Configurable Robotic Systems for Surgery and Other Uses”; U.S. Application No. 12 / 133,423, filed on May 19, 2005 (U.S. Patent No. 8,004,229), entitled “Offset Remote Center Manipulator For Robotic and U.S. Application No. 09 / 398,507, filed on September 17, 1999, entitled “Master Having Redundant Degrees of Freedom” (U.S. Patent No. 6,714,839); U.S. Application No. 12 / 494,695, filed on June 30, 2012, entitled “Control of Medical Robotic System Manipulators About Kinematic Singularities”; U.S. Provisional Application No. 61 / 654,755, filed on June 1, 2012, entitled “Manipulator Arm-to-Patient Collision Avoidance Using a Null-Space”; and U.S. Provisional Application No. 61 / 654,755, filed on June 1, 2012, entitled “System and Methods for Avoiding Collisions Between Manipulator Arms Using a Null-Space”; and the following U.S. Provisional Application No.61 / 800,381 (Attorney Docket No. ISRG03800), 61 / 800,924 (Attorney Docket No. ISRG03870), and 61 / 799,920 (Attorney Docket No. ISRG03900), the entire disclosures of which are incorporated herein by reference. Technical Field

[0005] The present invention generally provides improved surgical and / or robotic devices, systems and methods. Background Art

[0006] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during a diagnostic or surgical procedure, thereby reducing patient recovery time, discomfort, and harmful side effects. Millions of "open" or traditional surgical procedures are performed in the United States each year; many of these procedures could potentially be performed in a minimally invasive manner. However, only a relatively small number of surgical procedures currently use minimally invasive techniques because of the limitations of surgical instruments and techniques and the additional surgical training required to master these techniques.

[0007] Minimally invasive telesurgery systems used in surgical procedures are being developed, increasing the dexterity of surgeons and allowing surgeons to operate on patients from remote locations. Telesurgery is a general term for surgical systems in which surgeons use some form of remote control (e.g., servomechanisms, etc.) to manipulate the movement of surgical instruments rather than directly holding and moving instruments by hand. In such telesurgery systems, images of the surgical site are provided to the surgeon at a remote location. When the surgeon typically views a three-dimensional image of the surgical site on a suitable viewer or display, the surgeon controls the movement of the robotic instrument by manipulating the main control input device, thereby performing a surgical procedure on the patient. Robotic surgical instruments can be inserted through small minimally invasive surgical orifices to treat tissue at the surgical site in the patient's body, generally avoiding trauma associated with open surgical access methods. These robotic systems can move the working end of the surgical instrument dexterously enough to perform fairly complex surgical tasks, such as by pivoting the shaft of the instrument at the minimally invasive orifice, sliding the shaft axially through the orifice, rotating the shaft in the orifice, and / or other similar actions.

[0008] The servomechanism used for telesurgery will typically receive input from two master controllers (one for each surgeon's hand) and may include two or more robotic arms or manipulators. Mapping the movements of the hands onto the image of the robotic instrument displayed by the image acquisition device can help provide the surgeon with accurate control of the instrument associated with each hand. In many surgical robotic systems, one or more additional robotic manipulator arms are included for moving an endoscope or other image acquisition device, additional surgical instruments, or the like.

[0009] Various structural arrangements can be used to support surgical instruments at the surgical site during robotic surgery. A driven linkage or "slave" device is generally referred to as a robotic surgical manipulator, and is described in U.S. Patents 6,758,843, 6,246,200, and 5,800,423 as a robotic surgical manipulator during minimally invasive robotic surgery, the entire contents of which are incorporated herein by reference. These linkages are generally arranged in a parallelogram to hold an instrument with an axis. This manipulator structure can constrain the movement of the instrument so that the instrument axis pivots around a remote center of spherical rotation, which is positioned in a space along the length of the rigid axis. By aligning the center of rotation with the incision point of the internal surgical site (e.g., with a trocar or cannula at the abdominal wall during laparoscopic surgery), the end effector of the surgical instrument can be safely positioned by using the manipulator linkage to move the proximal end of the axis without applying potentially dangerous forces to the abdominal wall. For example, alternative manipulator structures are described in US Pat. Nos. 6,702,805, 6,676,669, 5,855,583, 5,808,665, 5,445,166, and 5,184,601, the entire contents of which are incorporated herein by reference.

[0010] Although new robotic surgical systems and devices have proven to be efficient and advantageous, further improvements are still needed. For example, the manipulator arm can include additional redundant joints to provide increased movement or configuration in some cases. However, when the surgical instrument is moved within the minimally invasive surgical site, these joints may show large movements outside the patient's body (usually more than required or expected), especially when the instrument pivots around the minimally invasive orifice over a large angle range. Alternative manipulator structures have been proposed, which use software control of the joints of highly configurable kinematic manipulators to limit the pivoting motion at the insertion site while preventing unintentional manipulator / manipulator contact (or similar situations) outside the patient's body. These highly configurable "software-centric" surgical manipulator systems can provide significant advantages, but they also pose challenges. Specifically, mechanically constrained remote center linkages can have safety advantages in some cases. In addition, many joints of a wide range of configurations typically included in these manipulators can cause the manipulator to be difficult to manually set in a configuration desired by a specific procedure. As the scope of surgical operations performed using remote surgical systems continues to expand, the requirements for expanding available configurations and expanding the range of motion of instruments in the patient's body increase. Unfortunately, both of these variations increase the challenges associated with motion of the manipulator outside the body and further increase the importance of avoiding unnecessary movement of the manipulator arm and undesirable configurations and movements.

[0011] For these and other reasons, it would be advantageous to provide improved apparatus, systems and methods for surgical procedures, robotic surgery, and other robotic applications. It would be particularly beneficial if these improved techniques provided the ability to limit the amount of movement of the manipulator arm and / or provide one or more beneficial movements during certain tasks. Additionally, it would be desirable to provide such improvements while increasing the range of motion of the instrument for at least some tasks without significantly increasing the size, mechanical complexity or cost of these systems, and while maintaining or improving their dexterity. Summary of the invention

[0012] The present invention generally provides improved robotic and / or surgical devices, systems and methods. In many embodiments, the present invention will employ highly configurable surgical robotic manipulators. For example, these manipulators may have more degrees of freedom of motion than associated surgical end effectors within a patient's surgical workspace. A robotic surgical system according to the present invention typically includes a manipulator arm supporting a robotic surgical instrument and a processor that calculates coordinated joint movements to manipulate the end effector of the instrument. For a given end effector orientation and / or a given pivot point position, the joints of the robotic manipulator supporting the end effector allow the manipulator to move within different configuration ranges. The manipulator may include additional redundant joints to allow various types of auxiliary movements (such as reconfiguration movements in response to user commands) or another type of movement (such as collision avoidance movements). Although various aspects of the present invention are described for use in a manipulator system with redundant manipulators, it should be appreciated that many aspects can be used in systems with non-redundant manipulators.

[0013] In one aspect, various movements of the manipulator arm may be expected to achieve one or more goals within the null space while maintaining a given state of the distal end effector; these null space goals may include, but are not limited to, joint states, combinations of joint states, relative joint states, ranges of joint states, profiles of joint states, velocities of one or more of a plurality of joints, sums of velocities of one or more of a plurality of joints, kinetic energy of one or more of a plurality of joints, snap moves, motion cancellation moves, locking of one or more of a plurality of joints, collision avoidance moves, singularity avoidance moves, manipulator pose preferences, desired manipulator configurations, commanded reconfigurations of manipulators, and anisotropic enhancements of joints. The manipulator system may be configured to perform multiple different null space goals, and typically a user may expect the manipulator to perform null space movements that simultaneously achieve multiple null space goals. For example, in a manipulator configured to achieve four different goals through null space movements, a user may expect two or more goals to be achieved simultaneously through commanded movements of the distal end effector. One difficulty encountered in achieving multiple goals is that a movement associated with one goal may interfere with a movement associated with the performance of another goal. In other cases, combining null space moves associated with different targets may reduce the benefits associated with one or two targets, or may eliminate targets so that certain combinations of targets are completely undesirable. Another difficulty associated with combining targets is that combining certain targets will limit manipulator movement, because the null space of the manipulator may have fewer dimensions than the number of targets competing for null space resources. In view of these challenges, there is a need for a manipulator system with a management system, which is used to manage multiple null space targets when executing multiple null space targets simultaneously to avoid the above-mentioned disadvantages, so as to provide improved manipulator movement that retains the benefits and advantages of various null space moves. This can be achieved by managing multiple null space targets so as to perform null space moves according to the desired relationship and / or behavior between null space targets. These null space targets may include any null space operation described in any application incorporated by reference herein or various other null space operations. Various different methods or algorithms can be used based on their corresponding attributes when combining multiple null space targets, and the methods or algorithms may include weighting, scaling, saturation level, priority between targets, main speed limit, and saturation limit integration algorithm, or various other aspects associated with manipulator movement.

[0014] In some aspects, the desired movement of one or more joints provided by the null space target can include a joint state, a combination of joint states, a relative joint state, a range of joint states, a profile of joint states, or any combination thereof. For example, the desired movement can include maintaining relatively consistent joint velocities between joints of the first set of joints, limiting joint velocities within a desired range or at a joint limit, and / or maintaining a combination of joint states that corresponds to a desired pose or collision prevention configuration of the manipulator.

[0015] In one aspect of the present invention, a redundant degree of freedom (RDOF) surgical robot system with a manipulation input is provided. The RDOF surgical robot system includes a manipulator assembly, one or more user input devices, and a processor with a controller. The manipulator arm of the assembly has multiple joints that provide sufficient degrees of freedom, and the degrees of freedom allow a series of joint states for a given end effector state. Typically, in response to receiving a manipulation command to move the end effector with a desired movement, the system calculates the end effector displacement movement of the joint by calculating the joint velocity in the zero vertical space orthogonal to the null space of the Jacobian matrix, and drives the joint according to the calculated movement, thereby achieving the desired end effector movement. In order to provide an increased range of motion of one or more joints or to allow various auxiliary tasks, the system may include a proximal swivel joint and / or a distal swivel joint that couples the instrument to the proximal portion of the manipulator arm.

[0016] In another aspect of the invention, the manipulator is configured to move so that the middle portion of the instrument shaft pivots about the remote center. There are multiple driven joints between the manipulator and the instrument that provide sufficient degrees of freedom to allow a series of joint states for end effector orientation when the middle portion of the instrument shaft passes through the entry site. A processor with a controller couples an input device to the manipulator. In response to receiving a manipulation command for achieving a desired end effector movement, the system calculates an end effector displacement movement of the joint, including calculating the joint velocity in the zero perpendicular space of the Jacobian matrix orthogonal to the zero space.

[0017] In one aspect, the proximal portion of the manipulator arm is attached to the base so that when the joint is driven, the movement of the proximal portion relative to the base is blocked. On the other hand, the proximal portion is coupled to the base through a joint so that when the joint is driven, the proximal portion of the manipulator arm can move relative to the base. In an exemplary embodiment, the joint that couples the proximal portion of the manipulator to the base is a swivel joint that supports the manipulator arm, so that the joint movement of the swivel joint pivots one or more joints of the manipulator arm around the pivot axis of the swivel joint. In some embodiments, the pivot axis of the swivel joint extends from the joint through the remote center around which the instrument shaft of the end effector pivots. In one aspect, the movement of the swivel joint pivots one or more joints of the manipulator arm around a cone that is distally reduced and oriented toward the distal end effector (usually the remote center). In this regard, the cone around which the manipulator arm pivots corresponds to a conical gap within the range of motion of the tool tip, and the movement of the tool within this range of motion is impossible or weakened, which will be discussed in further detail below.

[0018] In another aspect, a joint coupling a proximal portion of the manipulator to the base is movable relative to the base along a generally arcuate or generally circular path, such that movement of the joint along the path pivots one or more joints of the manipulator arm about an axis extending through a distal portion of the manipulator arm near the instrument (preferably through a remote center about which the instrument axis pivots). In some embodiments, the manipulator includes a swivel joint coupling a proximal portion of the manipulator to the base, the swivel joint being movable relative to the base along a path, which may be linear, arcuate, or generally circular.

[0019] In yet another aspect of the invention, a surgical robotic manipulator is provided having a proximal swivel joint and a distal parallelogram linkage, the pivot axis of the swivel joint substantially intersecting the axis of the instrument shaft of the end effector, preferably at a remote center if appropriate.

[0020] A further understanding of the nature and advantages of the present invention will become apparent by reference to the remaining portions of the specification and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A is a top view of a robotic surgical system according to an embodiment of the present invention, the robotic surgical system having a surgical table with multiple robotic manipulators, which are used to automatically move surgical instruments with surgical end effectors, which are located at an internal surgical site within a patient's body.

[0022] Figure 1B Shown schematically Figure 1A Robotic surgical system.

[0023] Figure 2 is a diagram showing a method for inputting surgical procedure commands into Figure 1A A perspective view of a primary surgeon's console or workstation within a surgical system including a processor that generates manipulator command signals in response to input commands.

[0024] Figure 3 yes Figure 1A Perspective view of an electronics cart.

[0025] Figure 4 Perspective view of a patient side cart with four manipulator arms.

[0026] Figures 5A-5D An example manipulator arm is shown.

[0027] Figure 6A-6B An example manipulator arm is shown in a pitch-forward configuration and a pitch-back configuration, respectively.

[0028] Figure 6C A graphical representation of the range of motion of a surgical instrument tool tip of an example manipulator arm is shown, including a cone of silence or tapered tool entry restriction area in each of the pitch-forward and pitch-back configurations.

[0029] Fig. 7A An example manipulator arm is shown having a proximal swivel joint that swivels the manipulator arm about an axis of the proximal swivel joint.

[0030] Figure 7B An exemplary manipulator arm and associated range of motion and cone of silence are shown, the exemplary manipulator arm having a proximal swivel joint that enables the manipulator arm to swivel about its axis, movement of the manipulator arm about its axis can be used to reduce the described cone of silence.

[0031] Figure 8 An example manipulator arm is shown having a swivel joint proximate a distal instrument holder.

[0032] Fig. 9 An example manipulator arm is shown having a swivel joint proximate a distal instrument holder that swivels or twists the instrument holder about a joint axis.

[0033] Figures 10A-10C Sequential diagram showing an exemplary manipulator arm having a swivel joint proximate a distal instrument holder as the joint moves through its joint range of motion.

[0034] Figures 11A-11B The swivel profiles of an exemplary manipulator arm having a distal swivel joint are shown when the angular displacement of the joint is 0° and 90°, respectively.

[0035] Figures 12A-12C An exemplary manipulator arm is shown having a proximal joint that translates the proximal joint of the manipulator arm about a path of the joint.

[0036] Figures 13A-13B The relationship between the null space and the null perpendicular space of the Jacobian matrix of an exemplary manipulator assembly is graphically represented.

[0037] Figures 14A-14D Graphically depicted are different methods of merging multiple null space targets using a manager in accordance with certain embodiments.

[0038] Fig.15 A block diagram is shown for executing a general algorithm in an example manipulator assembly.

[0039] Figure 16-17 A block diagram of an exemplary method according to the present invention is shown. DETAILED DESCRIPTION

[0040] The present invention generally provides improved surgical and robotic apparatus, systems and methods. The present invention is particularly advantageous for use with surgical robotic systems wherein during a surgical procedure, a plurality of surgical tools or instruments will be mounted on and moved by associated plurality of robotic manipulators. The robotic system will typically include a telerobotic system, a telesurgical system and / or a telepresence system including a processor configured as a master-slave controller. By providing a robotic system employing a processor appropriately configured to move a manipulator assembly having an articulated linkage, wherein the linkage has a relatively large number of degrees of freedom, the motion of the linkage can be adjusted to work through a minimally invasive access site.

[0041] The robotic manipulator assembly described herein will typically include a robotic manipulator and a tool mounted thereon (the tool typically includes a surgical instrument in a surgical version), although the term "robotic assembly" will also include a manipulator without a tool mounted thereon. The term "tool" includes both general or industrial robotic tools and special robotic surgical instruments, wherein these special robotic surgical instrument structures typically include end effectors suitable for tissue manipulation, tissue processing, tissue imaging, etc. The tool / manipulator interface will typically be a quick-disconnect tool holder or coupler, allowing the tool to be quickly removed and replaced with an alternative tool. The manipulator assembly will typically have a base that is fixed in space during at least a portion of the robotic procedure, and the manipulator assembly may include a large number of degrees of freedom between the base and the end effector of the tool. The actuation of the end effector (such as opening or closing the jaws of the gripping device, energizing the electrosurgical defibrillation electrodes, etc.) will typically be separated from these manipulator assembly degrees of freedom and supplement the degrees of freedom of these manipulator assemblies.

[0042] The end effector will typically move in the workspace with between two and six degrees of freedom. As used herein, the term "orientation" includes both position and orientation. Therefore, the change in the orientation of the end effector (for example) can involve the translation of the end effector from a first position to a second position, the rotation of the end effector from a first orientation to a second orientation, or a combination of the two. When used in minimally invasive robotic surgery, the movement of the manipulator assembly can be controlled by the processor of the system so that the shaft or intermediate portion of the tool or instrument is constrained to be safe movement through the minimally invasive surgical access site or other orifices. This movement can include, for example, the axial insertion of the shaft into the surgical workspace through the orifice site, the rotation of the shaft around its axis, and the pivotal movement of the shaft around the pivot point adjacent to the access site.

[0043] Many of the example manipulator assemblies described herein have more degrees of freedom than those required to position and move the end effector within the surgical site. For example, in some embodiments, a surgical end effector that can be positioned with six degrees of freedom at an internal surgical site through a minimally invasive orifice can have nine degrees of freedom (six degrees of freedom of the end effector - three for positioning, three for orientation - plus three degrees of freedom to comply with entry site constraints), but typically has ten or more degrees of freedom. A highly configurable manipulator assembly with more degrees of freedom than those required for a given end effector orientation can be described as having or providing sufficient degrees of freedom to allow a series of joint states for the end effector orientation within the workspace. For example, for a given end effector orientation, the manipulator assembly can occupy any position in a series of alternative manipulator linkage device orientations (and be driven between them). Similarly, for a given end effector velocity vector, the manipulator assembly can have different joint movement velocity ranges for various joints of the manipulator assembly within the null space of the Jacobian matrix.

[0044] The present invention provides a robotic linkage structure that is particularly well suited for surgical applications (among other applications) where a wide range of motion is desired and a limited dedicated volume is available due to the presence of other robotic linkages, surgical personnel and equipment, etc. The large range of motion and reduced volume required for each robotic linkage can also provide greater flexibility between the location of the robotic support structure and the surgical or other workspace, thereby facilitating and speeding up setup.

[0045] The term "state" of a joint, etc., will generally refer to a control variable associated with the joint in this document. For example, the state of an angular joint may refer to the angle defined by the joint within its range of motion and / or the angular velocity of the joint. Similarly, the state of an axial or prismatic joint may refer to the axial orientation of the joint and / or its axial velocity. Although many of the controllers described herein include velocity controllers, they typically also have some orientation control aspects. Alternative embodiments may rely primarily or entirely on orientation controllers, acceleration controllers, etc. Many aspects of a control system that can be used for such a device are more fully described in U.S. Pat. No. 6,699,177, the entire disclosure of which is incorporated herein by reference. Therefore, as long as the movement is based on associated calculations, the calculations of the movement of the joint and the movement of the end effector described herein may be performed using an orientation control algorithm, a velocity control algorithm, a combination of the two, etc.

[0046] In some embodiments, the tool of the exemplary manipulator arm pivots around the pivot point adjacent to the minimally invasive orifice. In some embodiments, the system can utilize a hardware remote center, such as the remote center kinematics described in U.S. Patent No. 6,786,896, the entire content of which is incorporated herein by reference. Such a system can utilize a dual parallelogram linkage, which constrains the movement of the linkage, so that the axis of the instrument supported by the manipulator pivots around the remote center point. Alternative mechanically constrained remote center linkage systems are known and / or can be developed in the future. Surprisingly, the work combined with the present invention indicates that the remote center linkage system can benefit from a highly configurable kinematic architecture. Specifically, when the surgical robot system has a linkage that allows pivotal motion of two axes intersecting at or near the minimally invasive surgical access site, the spherical pivoting motion can include the desired range of motion of the full extent in the patient's body, but still encounter avoidable defects (such as not being fully adjusted, prone to arm-to-arm or arm-to-patient contact and / or other situations outside the patient's body). First, adding one or more additional degrees of freedom that are also mechanically constrained to pivot motion at or near the entry site may appear to provide little or no improvement in the range of motion. Surprisingly, such a joint can provide significant advantages by allowing the entire system to be configured to be in a collision-inhibiting posture or driven toward the posture, by further expanding the range of motion of other surgical procedures, etc. In other embodiments, the system can utilize software to implement a remote center, such as described in U.S. Patent Application 8,004,229, the entire content of which is incorporated herein by reference. In a system with a software remote center, a processor calculates the movement of the joint so as to pivot the middle portion of the instrument shaft around a determined pivot point (which is different from a mechanical constraint). By having the ability to calculate a software pivot point, different modes characterized by the flexibility or stiffness of the system can be selectively implemented. More specifically, different system modes can be implemented on a series of pivot points / centers (e.g., movable pivot points, passive pivot points, fixed / rigid pivot points, soft pivot points) as needed.

[0047] Although robotic surgical systems with multiple highly configurable manipulators have many advantages, because the manipulators include a relatively large number of joints and linkages between a base and an instrument with redundant degrees of freedom, the commanded motion of multiple joints to achieve the desired movement of a distal end effector and / or a remote center may produce undesirable joint velocities, excessive kinetic energy associated with one or more joints, or may produce motion that does not meet the desired motion preferences. Examples of undesirable joint velocities may include undesirable combinations of joint states, excessive joint velocities for one or more joints, or incommensurate joint states. The present invention provides desired movement for one or more joints during commanded end effector movement, such as the joint states described herein or other such combinations of movement.

[0048] In one aspect, calculated null space movements associated with various other objectives (such as avoidance movements, commanded reconfigurations, desired manipulator poses, or joint behaviors) can override calculated joint velocities to achieve commanded end effector movements while achieving the various other objectives. This can be accomplished using a null space management system that incorporates null space movements associated with multiple objectives so that when these objectives conflict, null space movements are provided in accordance with the multiple objectives or at least in accordance with desired relationships and / or behaviors between the multiple objectives.

[0049] In certain aspects, the manipulator system is configured to determine each null space objective function as a separate software entity that assumes a unique objective exists for each null space objective function, assigns some useful attributes to each objective function, and transmits the output of each objective function to a null space manager. The null space manager typically includes a processor of the system having programmable instructions recorded thereon for routing the objectives by executing the methods described herein. The null space manager may include one or more modes for managing null space movements associated with different objectives according to any of the methods and approaches described herein. The attributes of the null space objectives are then used by the manager when combining multiple inputs into a merged null space command, which can then be combined with a zero vertical space command and sent to the joint controller of the manipulator. The manager can use a variety of different methods or algorithms based on their respective attributes when combining multiple null space objectives, which methods or algorithms may include weighting, scaling, saturation levels, priorities between objectives, primary velocity limits, and saturation limit integration algorithms or various other features. In one aspect, the null space coefficients can be multipliers for a set of null space basis vectors.

[0050] Examples of such avoidance moves are described in U.S. Provisional Patent Application No. 61 / 654,755, filed on June 1, 2012, entitled “Manipulator Arm-to-Patient Collision Avoidance Using a Null-Space,” and U.S. Provisional Patent Application No. 61 / 654,773, filed on June 1, 2012, entitled “System and Methods for Avoiding Collisions Between Manipulator Arms Using a Null-Space,” the entire disclosures of which are incorporated herein by reference. Examples of null-space targets using commanded reconfiguration are described in U.S. Provisional Patent Application No. 61 / 654,764, filed on June 1, 2012, entitled “Commanded Reconfiguration of a Surgical Manipulator Using the Null-Space,” the entire disclosure of which is incorporated herein by reference.

[0051] In the following description, various embodiments of the present invention will be described. For illustrative purposes, specific configurations and details are set forth to provide a thorough understanding of the embodiments. However, it will be appreciated by those skilled in the art that the present invention may be practiced without the various specific details. In addition, in order not to confuse the embodiments being described, well-known features may be omitted or simplified.

[0052] Referring now to the drawings, wherein like reference numerals refer to like parts throughout the several views, Figure 1A1 is a top view of a minimally invasive robotic surgery (MIRS) system 10 for performing minimally invasive diagnostic or surgical procedures on a patient 12 lying flat on an operating table 14, according to many embodiments. The system can include a surgeon's console 16 for use by a surgeon 18 during the procedure. One or more assistants 20 can also participate in the procedure. The MIRS system 10 can further include a patient-side cart 22 (surgical robot) and an electronics cart 24. The patient-side cart 22 can manipulate at least one removably coupled tool assembly 26 (hereinafter referred to as a "tool") through a minimally invasive incision in the body of the patient 12 while the surgeon 18 views the surgical site through the console 16. An image of the surgical site can be obtained through an endoscope 28 (such as a stereo endoscope), which can be manipulated by the patient-side cart 22 to orient the endoscope 28. The electronics cart 24 can be used to process the image of the surgical site for subsequent display to the surgeon 18 through the surgeon's console 16. The number of surgical tools 26 used simultaneously will generally depend on the diagnostic or surgical procedure and the space constraints in the operating room, as well as other factors. If one or more of the tools 26 being used must be replaced during a procedure, the assistant 20 may remove the tool 26 from the patient side cart 22 and replace it with another tool 26 from the tray 30 within the operating room.

[0053] Figure 1B A robotic surgical system 50 (such as Figure 1A As described above, the surgeon console 52 (such as Figure 1A The surgeon console 16 in FIG. 1 can be used by the surgeon to control the patient cart (surgical robot) 54 (such as a Figure 1A The patient side cart 54 can use an imaging device (such as a stereo endoscope) to capture images of the procedure site and output the captured images to an electronics cart 56 (such as a Figure 1AThe electronic cart 24 in the patient side cart 56). As described above, the electronic cart 56 can process the acquired images in various ways prior to any subsequent display. For example, the electronic cart 56 can overlay the acquired images with a virtual control interface before displaying the combined images to the surgeon via the surgeon console 52. The patient side cart 54 can output the acquired images for processing outside the electronic cart 56. For example, the patient side cart 54 can output the acquired images to the processor 58 and can be used to process the acquired images. The images can also be processed by a combination of the electronic cart 56 and the processor 58, and can be coupled together so as to process the acquired images together, sequentially and / or in combination. One or more separate displays 60 can also be coupled to the processor 58 and / or the electronic cart 56 for local and / or remote display of images (such as images of the procedural site or other related images).

[0054] Figure 2 3 is a perspective view of the surgeon's console 16. The surgeon's console 16 includes a left-eye display 32 and a right-eye display 34 for presenting the surgeon 18 with a coordinated stereoscopic view of the surgical site with depth perception. The console 16 further includes one or more input controls 36, which in turn enable the patient side cart 22 ( Figure 1A The input control device 36 can provide the tool 26 (as shown) associated therewith. Figure 1A The surgeon may also use the same degrees of freedom as the tool 26 (shown) to provide the surgeon with a sense of integration of the telepresence or input control device 36 with the tool 26, thereby giving the surgeon a strong sense of directly controlling the tool 26. For this purpose, position, force, and tactile feedback sensors (not shown) may be employed to transmit position, force, and tactile sensations from the tool 26 back to the surgeon's hand through the input control device 36.

[0055] The surgeon's console 16 is typically located in the same room as the patient so that the surgeon can directly monitor the surgical procedure, be physically present if necessary, and speak directly to an assistant rather than over a telephone or other communication medium. However, the surgeon can be located in a different room, an entirely different building, or other remote location away from the patient to allow for telesurgical procedures.

[0056] Figure 32 is a perspective view of the electronics cart 24. The electronics cart 24 can be coupled to the endoscope 28 and can include a processor for processing the acquired images for subsequent display, such as to the surgeon at a surgical console or another suitable display located locally and / or remotely. For example, when a stereoscopic endoscope is used, the electronics cart 24 can process the acquired images to present the surgeon with a coordinated stereoscopic image of the surgical site. Such coordination can include alignment between opposing images and can include adjusting a stereoscopic working distance of the stereoscopic endoscope. As another example, the image processing can include using previously determined camera calibration parameters to compensate for imaging errors of the image acquisition device, such as optical aberrations.

[0057] Figure 4 A patient side cart 22 is shown with multiple manipulator arms, each of which supports a surgical instrument or tool 26 at the distal end of the manipulator arm. The patient side cart 22 shown includes four manipulator arms 100 that can be used to support surgical tools 26 or imaging devices 28, such as a stereo endoscope for acquiring images of the procedure site. Manipulation is provided by a robotic manipulator arm 100 having several robotic joints. The imaging device 28 and the surgical tool 26 can be positioned and manipulated through an incision in the patient's body so that the kinematic distal center is maintained at the incision to minimize the size of the incision. When the distal end of the surgical instrument or tool 26 is positioned within the field of view of the imaging device 28, the image of the surgical site can include an image of the distal end of the surgical instrument or tool 26.

[0058] With respect to the surgical tool 26, alternative robotic surgical tools or instruments of various different types and different end effectors may be used, wherein the instruments of at least some manipulators are removed and replaced during a surgical procedure. Some of these end effectors, including DeBakey forceps, micro forceps, Potts scissors, and clip appliers, include a first end effector element and a second end effector element that pivot relative to each other to define a pair of end effector jaws. Other end effectors, including scalpels and electrocautery probes, have a single end effector element. For instruments having end effector jaws, the jaws are typically actuated by squeezing a gripping member of the handle. Single end effector-type instruments may also be actuated by grasping of a gripping member, for example, to energize an electrocautery probe.

[0059] The elongated shaft of the instrument 26 allows the end effector and the distal end of the shaft to be inserted distally through a minimally invasive orifice, typically through the abdominal wall, etc. to a surgical working site. The surgical working site can be irrigated, and movement of the end effector within the patient's body is typically achieved at least in part by pivoting the instrument 26 about the shaft through the location of the minimally invasive orifice. In other words, the manipulator 100 will move the proximal housing of the instrument outside the patient's body so that the shaft extends through the minimally invasive orifice location to help provide the desired movement of the end effector. Therefore, during the surgical procedure, the manipulator 100 will typically undergo significant movement outside the patient's body P.

[0060] An exemplary manipulator arm according to many embodiments of the present invention can be referenced Figure 5A-12C As described above, the manipulator arm generally supports the distal instrument or surgical tool and enables movement of the instrument relative to the base. Since a large number of different instruments with different end effectors may be successively mounted on each manipulator during a surgical procedure (usually with the help of a surgical assistant), the distal instrument holder will preferably allow for rapid removal and replacement of the mounted instrument or tool. As described in reference Figure 4 As can be appreciated, the manipulator is proximally mounted to a base of the patient side cart. Typically, the manipulator arm includes a plurality of linkages and associated joints extending between the base and the distal instrument holder. In one aspect, the exemplary manipulator includes a plurality of joints with redundant degrees of freedom so that for a given end effector orientation, the joints of the manipulator arm can be driven into a range of different configurations. This may be the case for any of the embodiments of the manipulator arm disclosed herein.

[0061] In many embodiments, such as Figure 5A In the example shown, an exemplary manipulator arm includes a proximal swivel joint J1 that rotates about a first joint axis to cause the manipulator arm distal to the joint to swivel about the joint axis. In some embodiments, the swivel joint J1 is mounted directly to the base, while in other embodiments, the joint J1 can be mounted to one or more movable linkages or joints. The joints of the manipulator have redundant degrees of freedom in their combined form, so that for a given orientation of the end effector, the joints of the manipulator arm can be driven into a series of different configurations. For example, Figures 5A-5D The manipulator arm of the can be manipulated into different configurations, while the distal member 511 supported in the instrument holder 510 maintains a specific state and can include a given orientation or velocity of the end effector. The distal member 511 is typically a cannula through which the tool shaft 512 extends, and the instrument holder 510 is typically a bracket (shown as a block-like structure that translates on a beam) to which the instrument is attached before extending through the cannula 511 through a minimally invasive orifice into the patient's body.

[0062] Now describe Figures 5A-5DThe various links of the manipulator arm 500 and the Figures 5A-5D The rotation axis of the joint connecting the links is shown, where the first link 504 extends distally from the pivot joint J2, which pivots about its joint axis and is coupled to the swivel joint J1 that rotates about the joint axis of the swivel joint J1. Many of the remaining parts of the joint can be identified by the rotation axis associated with the joint, such as Figure 5A For example, as shown, the distal end of the first link 504 is coupled to the proximal end of the second link 506 at a pivot joint J3, wherein the pivot joint J3 pivots about its pivot axis, and the proximal end of the third link 508 is coupled to the distal end of the second link 506 at a pivot joint J4, wherein the pivot joint J4 pivots about its axis. The distal end of the third link 508 is coupled to the instrument holder 510 at a pivot joint J5. Typically, the pivot axes of each joint J2, J3, J4 and J5 are substantially parallel when positioned close to each other and the linkage appears "stacked", such as Figure 5D As shown, in order to provide a reduced width w of the manipulator arm and improve patient clearance during manipulation of the manipulator assembly. In many embodiments, the instrument holder also includes an additional joint, such as a prismatic joint J6, which facilitates axial movement of the instrument 306 through the minimally invasive orifice and facilitates attachment of the instrument holder to a cannula through which the instrument is slidably inserted.

[0063] The sleeve 511 through which the distal member or tool 512 extends can include additional degrees of freedom distal to the instrument holder 510. The actuation of the degrees of freedom of the instrument will typically be driven by the motor of the manipulator, and alternative embodiments can separate the instrument from the supporting manipulator structure at the quick-detachable instrument holder / instrument interface, so that one or more joints shown here on the instrument are actually on the interface, or vice versa. In some embodiments, the sleeve 511 includes a rotary joint J7 (not shown) near or proximal to the insertion point or pivot point PP of the tool tip, wherein the insertion point or pivot point PP of the tool tip is typically set at the location of the minimally invasive orifice. The distal wrist of the instrument allows the end effector of the surgical tool 512 to pivot around the instrument joint axis of one or more joints at the instrument wrist. The angle between the end effector jaw elements can be controlled independently of the position and orientation of the end effector.

[0064] By reference Figures 6A-6C The range of motion of the exemplary manipulator assembly can be appreciated. During a surgical procedure, the exemplary manipulator arm can be manipulated into a pitch-forward configuration (e.g., Fig. 6A as shown) or in a pitch-back configuration (as Figure 6B) to access patient-specific tissue within a surgical workspace. A typical manipulator assembly includes an end effector that is capable of pitching forward and backward about an axis by at least ±60 degrees, preferably about ±75 degrees, and is also capable of yaw by ±80 degrees about an axis. While this aspect allows for increased maneuverability of the end effector and assembly, there may still be configurations where movement of the end effector may be limited, particularly when the manipulator arm is in a Fig. 6A and 6B When configured in full pitch forward or full pitch aft. In one embodiment, the manipulator arm has a (+ / - 75 degree) range of motion (ROM) for the outer pitch and a (+ / - 300 degree) range of motion (ROM) for the outer deflection joints, respectively. In some embodiments, the ROM can be increased for the outer pitch to provide a ROM greater than (+ / - 90 degrees), in which case the spatial cone in which joint movement is restricted or impossible can disappear entirely, but the inner sphere typically associated with the insertion restriction will continue to exist. It should be appreciated that various embodiments can be configured to have increased or decreased ROM, the ROMs mentioned above are provided for illustrative purposes, and it should be further appreciated that the present invention is not limited to the ROMs described herein.

[0065] Figure 6C Graphical representation Figure 5A-5B 1 shows the overall range of motion and workspace of the tool tip of an exemplary manipulator. Although the workspace is shown as a hemisphere, it can also be represented as a sphere depending on the range of motion and configuration of one or more revolute joints (such as joint J1) of the manipulator. As shown in the figure, Figure 6C The hemisphere in includes a central, small spherical void and two conical voids. The void represents an area where movement of the tool tip is not possible due to mechanical constraints or is impractical due to extremely high joint speeds that make movement of the end effector difficult or sluggish. For these reasons, the conical void is called the "cone of silence". In some embodiments, the manipulator arm can reach a singularity at a point within the cone. Because the movement of the manipulator within or near the cone of silence may be impaired, it is difficult to move the manipulator arm out of the cone of silence without manually moving one or more links of the manipulator to reconfigure the linkages and joints of the manipulator, which may require alternative operating modes and delay the surgical procedure.

[0066] When the angle between the distal linkages in the manipulator is relatively small, movement of the instrument axis within or near these tapered portions typically occurs. Therefore, this configuration can be avoided by various null space goals, such as anisotropically enhancing the movement of the manipulator so as to increase the angle between the linkages (so that the linkages are moved to a more orthogonal orientation relative to each other). For example, in the configurations shown in 6A and 6B, when the angle between the distalmost link and the instrument holder (angle a) becomes relatively small, movement of the manipulator becomes more difficult. In various embodiments, depending on the range of joint movement of the remaining joints, when the angle between certain linkages is reduced, movement of the manipulator may be blocked, and in some cases, the manipulator arm may no longer be redundant. Manipulator configurations in which the instrument axis is close to these tapered portions or the angle between the linkages is relatively small are considered to be "poorly adjusted", and therefore the manipulability and dexterity of the manipulator arm are limited. It is desirable that the manipulator is "better adjusted" in order to maintain dexterity and range of movement. In one aspect, the present invention allows the user to avoid movement of the instrument shaft near the tapered portion as needed by simply inputting a command to reconfigure the manipulator, even during movement of the end effector during a surgical procedure. This aspect is particularly useful if the manipulator becomes "poorly adjusted" for whatever reason.

[0067] Although the above-described embodiments of manipulators can be used in the present invention, some embodiments can include additional joints, which can also be used to improve the dexterity and adjustment of the manipulator arm. For example, an exemplary manipulator can include a swivel joint and / or linkage proximal to joint J1 that can be used to swivel about the axis of the swivel joint. Figure 5A The manipulator arm and its associated cone of silence may be configured to reduce or eliminate the cone of silence. In another embodiment, the exemplary manipulator may further include a distal pivot joint that pivots the instrument holder about an axis substantially perpendicular to joint J5, thereby offsetting the tool tip to further reduce the cone of silence and improve the range of motion of the surgical tool. In yet another embodiment, the proximal joint of the manipulator arm (such as J1) may be movably mounted on a base to move or shift the cone of silence as desired and improve the range of motion of the manipulator tool tip. By reference to Figure 7A-12C Being able to understand the use and advantages of this additional connector, Figure 7A-12C Examples of such joints are shown, and such joints may be used independently of one another or in combination with any of the example manipulator arms described herein.

[0068] Figure 7A-7BAn additional redundant joint for use with an exemplary manipulator arm is shown, namely, a first joint coupling the proximal portion of the manipulator arm to the base. The first joint is a proximal swivel joint J1 that allows the manipulator arm to swivel about the joint axis of the joint J1. The proximal swivel joint J1 includes a link 501 that offsets the joint J1' from the proximal swivel joint J1 by a predetermined distance or angle. The link 501 can be an arcuate linkage (e.g., Fig. 7A as shown) or linear or angled linkages (as shown Figure 7B Typically, the joint axis of the joint J1 is aligned with the remote center RC or the insertion point of the tool tip, both at Fig. 7A Shown. In an exemplary embodiment, the joint axis of joint J1 passes through the remote center, as does each other swivel joint axis in the manipulator arm, to prevent movement at the body wall, and can be moved during surgery. The axis of joint J1 is coupled to the proximal portion of the arm, so it can be used to change the position and orientation of the rear portion of the arm. In general, redundant axes (such as this axis) allow the instrument tip to comply with the surgeon's orders while avoiding collisions with other arms or patient anatomical structures. In one aspect, the proximal swivel joint J1 is only used to change the mounting angle of the manipulator relative to the ground. In order to 1) avoid collisions with external patient anatomical structures and 2) reach the anatomical structures in the body, this angle is important. Typically, the angle a between the manipulator proximal link and the axis of the proximal swivel joint attached to the proximal swivel joint J1 is about 15 degrees.

[0069] Figure 7B The proximal swivel joint J1 and its associated joint axis in an exemplary manipulator arm are shown in relation to the cone of silence. The joint axis of the proximal swivel joint J1' may pass through the cone of silence, or may be completely outside the cone of silence. By rotating the manipulator arm about the axis of the proximal swivel joint J1, the cone of silence can be reduced (in embodiments where the joint J1 axis passes through the cone of silence), or can be effectively eliminated (in embodiments where the proximal swivel joint axis extends completely outside the cone of silence). The distance and angle of the link 501 determine the orientation of the joint J1' axis relative to the cone of silence.

[0070] Figure 8Another type of redundant joint is shown for use with an exemplary manipulator arm, namely, a distal swivel joint J7 that couples the instrument holder 510 to the distal link of the manipulator arm 508. The distal swivel joint J7 allows the system to twist the instrument holder 510 about a joint axis that typically passes through the remote center or insertion point. Ideally, the swivel joint is located distally on the arm and is therefore particularly well suited for moving the orientation of the insertion axis. The addition of this redundant axis allows the manipulator to assume multiple orientations for any single instrument tip orientation. In general, redundant axes (such as this axis) allow the instrument tip to comply with the surgeon's commands while avoiding collisions with other arms or patient anatomy. Because the distal swivel joint J7 has the ability to move the insertion axis closer to the yaw axis, the arm pitch back range of motion can be increased. Fig. 9 The relationship between the axis of the distal swivel joint J7, the deflection axis of J1 and the insertion axis of the tool tip is shown in FIG. Figures 10A-10C Shown is the sequential movement of J7 and how it shifts the insertion axis of the tool tip from side to side.

[0071] Another advantage of the distal swivel joint J7 is that it can reduce the patient clearance cone, which is the swept volume of the distal portion of the manipulator arm proximal to the insertion point that the manipulator arm must avoid to avoid collision between the patient and the instrument holder or distal linkage of the manipulator arm. Fig.11A The patient clearance cone of the proximal portion of the manipulator arm is shown with the angular displacement of the distal swivel joint held at 0°. Fig. 11B The reduced patient clearance cone of the proximal portion of the manipulator arm is shown when the distal swivel joint is shown having an angular displacement of 90° about its axis. Thus, in procedures with minimal patient clearance close to the insertion point, the use of joint J7 according to the present invention can provide additional clearance while maintaining the remote center position or orientation of the end effector as desired.

[0072] Figures 12A-12C Another type of redundant joint for use with an exemplary manipulator arm translates or rotates a proximal joint of the manipulator arm about an axis. In many embodiments, the proximal translatable joint translates a proximal joint of the manipulator (such as joint J1 or J1') along a path to reduce or eliminate the cone of silence by shifting or rotating the range of motion of the manipulator arm, thereby providing better adjustment and improved maneuverability of the manipulator arm. The translatable joint may include a circular path (such as Figures 12A-12C511 ) or may include a semicircular or arcuate path. Generally, the joint rotates the manipulator arm about the axis of the translatable joint, the axis of the translatable joint intersects the remote center RC, and the axis of the tool 512 extending through the cannula 511 pivots about the remote center RC. In the embodiment shown, the axis of Jl" is a vertical axis, but in various other embodiments, the axis can be at a certain angle or horizontal.

[0073] In some embodiments, the manipulator arm 500 may include any or all of proximal and distal swivel joints, proximal translatable joints, and a parallelogram configuration of distal linkages. In accordance with the present invention, the use of any or all of these features provides additional redundant degrees of freedom and facilitates reconfiguration to provide a better "tuned" manipulator assembly by increasing the angles between linkages, thereby improving the manipulator's dexterity and motion. The increased flexibility of the exemplary manipulator can also be used to optimize the kinematics of the manipulator linkages to avoid joint limitations, singularities, etc.

[0074] In an exemplary embodiment, the joint movement of the manipulator is controlled by a controller using one or more joints driven by the motor of the system, wherein the joint is driven according to the coordination and joint movement calculated by the processor of the controller. Mathematically, the controller can use vectors and / or matrices to perform at least some calculations of joint commands, wherein some of the vectors and / or matrices can have elements corresponding to joint configurations or speeds. The range of alternative joint configurations available to the processor can be conceptualized as joint space. For example, this joint space can have as many dimensions as the degrees of freedom that the manipulator has, and the specific configuration of the manipulator can be represented at a specific point in the joint space, wherein each coordinate corresponds to the joint state of the associated joint of the manipulator.

[0075] In an exemplary embodiment, the system includes a controller, wherein the command position and velocity of a feature in a workspace (here indicated as Cartesian space) are inputs. The feature can be any feature on or off the manipulator that can be used as a control frame that is articulated using a control input. An example of a feature on the manipulator used in many examples described herein can be a tool tip. Another example of a feature on the manipulator can be a physical feature that is not on the tool tip, but a part of the manipulator such as a pin or a coloring pattern. An example of a feature that is off the manipulator can be a reference point in empty space that is just a certain distance and angle away from the tool tip. Another example of a feature that is off the manipulator can be a target tissue, the orientation of which relative to the manipulator can be established. In all these cases, the end effector is associated with an imaginary control frame that is articulated using a control input. However, in the following, the "end effector" and "tool tip" used are used as synonyms. Although in general, there is no closed form relationship that maps the desired Cartesian space end effector orientation to an equivalent joint space orientation, there is generally a closed form relationship between the Cartesian space end effector velocity and the joint space velocity. The kinematic Jacobian is the partial derivative matrix of the Cartesian space position elements of the end effector with respect to the joint space position elements. In this way, the kinematic Jacobian captures the kinematic relationship between the end effector and the joint. In other words, the kinematic Jacobian captures the effect of the joint motion on the end effector. The kinematic Jacobian (J) can be used to map the joint space velocity (dq / dt) to the Cartesian space end effector velocity (dx / dt) using the following relationship:

[0076] dx / dt=J dq / dt

[0077] Thus, even when there is no closed-form mapping between input positions and output positions, a mapping of velocities can be used iteratively in a controller such as a Jacobian-based controller to achieve movement of the manipulator according to commanded user input, although various implementations can be used. While many embodiments include a Jacobian-based controller, some implementations can use various controllers that can be configured to access the Jacobian matrix of the manipulator arm to provide any of the features described herein.

[0078] One such implementation is described below in simplified terms. The commanded joint positions are used to compute the Jacobian matrix (J). During each time step (Δt), the Cartesian space velocity (dx / dt) is calculated to perform the desired movement (dx des / dt) and correct the accumulated deviations (Δx) from the desired Cartesian space orientation. Then, the pseudo-inverse of the Jacobian matrix (J #) converts the Cartesian space velocity into the joint space velocity (dq / dt). The resulting joint space command velocity is then integrated to produce the joint space command position (q). These relationships are listed below:

[0079] dx / dt=dx des / dt+k△x (1)

[0080] dq / dt=J#dx / dt (2)

[0081] q i =q i-1 +dq / dt △t (3)

[0082] The pseudo-inverse of the Jacobian matrix (J) directly maps the desired tool tip motion (and in some cases, the remote center of the pivot tool motion) into the joint velocity space. If the manipulator being used has more useful joint axes than the tool tip degrees of freedom (up to six), (and when the remote center of the tool motion is in use, the manipulator should have an additional 3 joint axes for the 3 degrees of freedom associated with the position of the remote center), the manipulator is called redundant. The Jacobian matrix of the redundant manipulator includes a "null space" with at least one dimension. In this context, the "null space" (N(J)) of the Jacobian matrix is ​​the space of joint velocities that instantaneously achieve no tool tip motion (and when the remote center is used, no movement of the pivot point position); and "zero motion" is a combination, trajectory or path of joint orientations that also produces non-instantaneous movement of the position of the tool tip and / or remote center. The calculated zero space velocity is incorporated into or injected into the control system of the manipulator to achieve the desired reconfiguration of the manipulator (including any reconfiguration described herein), and the above equation (2) is changed as follows:

[0083] dq / dt=dq perp / dt+dq null / dt (4)

[0084] dq perp / dt=J # dx / dt (5)

[0085] dq null / dt=(1-J # J)z=V n V n T z=V n α (6)

[0086] Alternatively, in certain aspects, an augmented Jacobian matrix including the potential function gradient and applied to the Cartesian space end effector velocity may be used. The augmentation of the Jacobian matrix is ​​used to calculate the joint velocity as needed. It should be understood that with reference to calculating joint motion using the Jacobian matrix, such calculations may include an augmented Jacobian matrix method. According to the augmented Jacobian matrix method, the following formula may be used, but it should be appreciated that a column vector may be used:

[0087] dx / dt=J*dq / dt

[0088] Y=h(q)

[0089]

[0090]

[0091] d(x;y) / dt=[J;h']*dq / dt

[0092] dq / dt=[J;h'] # d(x;y) / dt

[0093] In one example, a complex network of potential field functions y=h(q) is set, dy / dt and and dy / dt can be specified as needed based on the potential field function, and the augmented formula will produce the desired result of both driving the end effector and tracking the combination of paths in joint space.

[0094] The joint velocity according to equation (4) has two components: the first component is the zero vertical spatial component dq perp / dt, the “single most” joint velocity (shortest vector length) which produces the desired tool tip motion (and the desired remote center motion when a remote center is used); and the second component is the null space component dq null / dt. Equations (2), (4) and (5) show that the same equations are achieved without the zero-space component. Equation (6) begins on the left with the traditional form of the zero-space component and on the far right shows the form used in the exemplary system, where V n is a set of orthogonal basis vectors of the null space, and α is a coefficient that mixes those basis vectors. In some embodiments, α is determined by a control parameter, variable, or setting, such as by using a knob or other control device to adjust or control motion within the null space as desired.

[0095] Figures 13A-13B The relationship between the null space of the Jacobian matrix and the null perpendicular space of the Jacobian matrix of an exemplary manipulator arm is graphically illustrated. Fig.13AA two-dimensional schematic diagram is shown, which shows the null space along the horizontal axis and the null vertical space along the vertical axis, which are orthogonal to each other. The diagonal vector represents the sum of the velocity vector in the null space and the velocity vector in the null vertical space, which represents the above equation (4).

[0096] Fig. 13B The relationship between the null space and the null-motion-manifold in four-dimensional joint space is graphically shown, shown as "null-motion-manifold". Each arrow (q1, q2, q3 and q4) represents a major joint axis. The closed curve represents the null-motion manifold, which is a set of joint space orientations that instantaneously achieve the same end effector orientation. For a given point A on the curve, since the null space is the space that instantaneously produces the joint velocity of the end effector without movement, the null space is parallel to the tangent of the null-motion manifold at point A.

[0097] Figures 14A-14D Various methods of merging multiple targets using a null space manager according to embodiments of the present invention are graphically illustrated. Fig.14A Involving weighted summation of multiple objectives; Fig. 14B Saturation limits involving multiple targets; Fig. 14C involves the prioritization of multiple targets in the null space; and Fig.14D Involving management of multiple objectives related to walls or boundaries of a joint space. In certain aspects, a null space manager is used to manage multiple null space objectives by determining a null space coefficient (α) for each null space objective function and combining the null space coefficients using any of the following properties (or combinations thereof) for each objective:

[0098] (a) Weighting : This property is used in the weighted sum paradigm, which allows for proportional blending of multiple features or goals. For example, if the user wishes to give twice as much importance to the null space usage for the extended pitch backwards goal as to the null space usage for the arm-to-arm collision avoidance goal, then the weight of the former would be set to twice that of the latter. Such an example is in Fig.14A It is shown in Fig.14A The null space coefficient vector α for the first objective is shown 1 and the null space coefficient vector α for the second objective 2 To determine the nullspace shift of the second target, which is weighted twice as much as the first target, the nullspace coefficient vector for the second target is α 1 is doubled before addition, so that the resulting sum α ∑= 0.0015 0.002 ...

[0099] (b) Saturation Level : This property allows for the management of multiple nullspace objectives that may conflict or cancel each other out. As mentioned above, the number of objective functions can be greater than the dimensionality of the nullspace. In these cases (and sometimes also in cases where the dimensionality is less strict), multiple objective functions can produce nullspace outputs that are directly opposite to each other. Examples of this are in Fig. 14B It is shown in Fig. 14B Show the directly opposite null space coefficient vector α 1u and α 2u . If these objective functions are added without intervention, their result is zero, which results in no beneficial action being taken for either objective function. By using saturation limits, the objective function with the highest saturation limit can override the objective function with the lower limit. For example, for clinical reasons, a user may desire that arm-to-patient collision avoidance must always take precedence over the arm-to-arm collision avoidance function. In this case, the null space manager can allow arm-to-patient avoidance to win, and in response, when in direct conflict, the manipulator can drive itself into the adjacent manipulator before penetrating the patient's body surface.

[0100] Fig. 14B is a schematic diagram of the null space coefficients in the null space, showing the above saturation properties. Although the original unsaturated vector α 1u and α 2u may be equal and opposite initially, but their saturation limits may be different so that after saturation, α 1 Shorter than α 2 In this example, when multiple coefficients are added together to obtain a combined output, their saturation level allows α 2 Overwhelm α 1 . Thus, if it is desired that a secondary objective (such as manipulator-to-manipulator collision avoidance) should not offset a primary objective (such as patient-to-manipulator collision avoidance), then an appropriate saturation level may be applied when combining the objectives so that the primary objective overwhelms any secondary objectives that conflict with or offset the primary objective.

[0101] (c) Priority:This method uses continuous space mathematical concepts rather than logical or discrete branching concepts. For example, assume that a requirement is to allow a certain objective function to have unrestricted use of the null space, and other objective functions also use the null space but do not interfere with the use of the previous objective. The concept of priority can be applied in such an example to allow the previous objective to enter the entire null space, but only allow the subsequent objective to enter the remaining unused orthogonal part of the null space. If the previous objective is linear in nature and orthogonal interference in the null space has no effect on it, then this method is advantageous because the subsequent objective does not hinder the use of the former. This method can be as follows Fig. 14C 's example. In this example, the objective function operating under the highest priority can combine its output such as by using one or both of the weighted and saturated techniques of (a) and (b) above. The output vector thus generated is then mapped into the null space. If the null space has a dimension greater than 1, then the subspace of the null space orthogonal to the output vector under the highest priority can be used for the next highest priority. The saturated weighted sum of all objective functions under the next highest priority is then projected onto the remaining null space subspace. This process can continue until all objective function priority levels have been resolved, or until all null space dimensions have been consumed. On the other hand, the method can also be used to realize joint locking or joint motion cancellation.

[0102] Fig. 14C A schematic diagram showing the coefficient vectors in the null space, illustrating the priority property described above. Here, the unchanged α p1o has the highest priority and therefore has unimpeded access to the entire null space. In this example, the modified form α p1 With α p1o are exactly the same, while α p2o is at a lower priority. Therefore, α p2o Only the unused higher priority α to the null space pl The part used (e.g., orthogonal to α p1 This is done by p2o Project onto this part to obtain α p2 to allow a lower priority target to be executed only to the extent that it does not interfere with the highest priority.

[0103] (d) Signs for major speed limits: Various null space objective functions can be configured to be partially autonomous. For efficiency and usability reasons, it is beneficial in various applications to allow some autonomous functionality. However, for safety reasons, it would be desirable to allow the surgeon the ability to oversee and override autonomous motion. One way to allow this ability is to limit the output values ​​of certain null space objective functions to be proportional to the master velocity. In this way, if autonomous motion is not desired, the surgeon can simply stop moving the master device and the autonomous motion stops. However, at least some objective functions may not function correctly if the master velocity limit is applied to the objective function. Therefore, this property tells the null space manager whether the master velocity limit can be applied to a given objective function. Therefore, certain objectives can be marked in the null space manager so that the master velocity limit is only applied to those objectives for which the limit is appropriate.

[0104] (e) Flags for the Saturation Limited Integration (SLI) algorithm : The saturation limited integration algorithm integrates the commanded joint velocity into the commanded joint orientation without interfering with joint velocity constraints or joint orientation constraints (e.g., hard stops). Some changes (such as those described below) may be made to certain embodiments to allow for two or more specific null space behaviors. This property allows the manager to classify the merged null space outputs from multiple objective functions into two or more buckets, each bucket being associated with each of two or more behaviors such as in the following example.

[0105] Behavior 1 : The SLI algorithm has the authority to change the input joint velocity commands in order to conform to the motion profile within the joint velocity and joint orientation constraints. However, in order for the zero-space velocity commands to remain in the null space, the SLI algorithm must constrain only the velocity value and not the direction. For example, to avoid disturbing the tool tip pose, the velocity vector may not include directional changes. This behavior may be more useful for some null-space target outputs than for others, and may be selectively used depending on the null-space target that is desired to be incorporated. Aspects of Behavior 1 are depicted in Fig.14D middle.

[0106] Behavior 2 : One disadvantage associated with behavior 1 is that it can lead to a phenomenon commonly referred to as a "sticky wall" (see Fig.14D), for example, this phenomenon occurs when the manipulator is against a joint constraint wall and is commanded to slide along the wall. For such a move, applying behavior 1 will not change the direction and will simply shorten the length to zero for any sliding command (e.g., any move with a component toward the wall). Therefore, sliding will not be allowed and the joint constraint will behave like a sticky wall. Certain algorithms can avoid this problem by performing velocity limiting on a DOF-by-DOF basis. In this case, only the DOF associated with the joint constraint will be set to zero, while the remaining DOFs will not be affected. The result will be a sliding command along the joint constraint, which is the desired behavior when commanding the tool tip to move. In some embodiments, the joint velocity limit can be applied in a vectorized manner, while the joint orientation limit is applied in a DOF-by-DOF manner.

[0107] Fig.14D Schematic diagram showing a joint space and its limits (e.g., hard stops drawn by boundaries), illustrating the SLI concept described above. The dashed arrow on the upper right shows an unsaturated arrow that penetrates the joint limit and therefore must be saturated. The shorter arrow above it shows behavior 1, (e.g., only a change in value without a change in direction). In this case, sliding does not occur. The triangle below the dashed arrow shows behavior 2. In this case, only the (horizontal) component of the unsaturated (dashed) arrow pointing to the wall is saturated, while the vertical component remains intact. This result allows sliding along the joint limit walls. Since this corresponds to a change in direction, and therefore cannot be applied to various null space moves.

[0108] (f) Other attributes: Other attributes based on these general concepts but not currently used or that may be contemplated by those skilled in the art can be used to differently handle multiple objective functions in specific ways by the null space manager. In addition, it should be understood that various combinations of the above attributes can be used and applied in various ways based on the desired objectives and / or associated null space moves.

[0109] Fig.15 A block diagram showing the requirements needed to implement the general algorithm for controlling the patient side cart connector state associated with the equations discussed above. Fig.15 The method of the system: calculates the forward kinematics of the manipulator arm; then, calculates dx / dt using equation (1) and dq using equation (5) perp / dt; then use equation (6) according to the dq perp / dt and Jacobian matrix z calculation dq null / dt. According to the calculated dq perp / dt and dq null / dt, the system calculates dq / dt and q using equations (4) and (3), respectively, thereby providing a calculated movement, wherein the controller is able to achieve the desired reconfiguration of the manipulator while maintaining the desired state of the end effector (and / or the position of the remote center).

[0110] Figure 16-17 A flow chart showing an exemplary method according to various aspects of the present invention is shown. Fig.16 As shown, an exemplary method performed by a manipulator may include: receiving a manipulation command to achieve a desired end effector movement; calculating joint movements of a first set of joints for a first goal; calculating joint movements of a second set of joints in a null space according to a second goal; using properties of each of the calculated null space joint movements of the first and second sets of joints to determine a null space command that combines the first and second goals; and driving the joints according to the null space command to achieve the desired movement, thereby achieving a combination of the first and second goals. Fig.17 As shown, an exemplary method of a manipulator system may include: receiving a manipulation command to move an end effector with a desired end effector movement; determining a plurality of null space coefficients, each corresponding to a different null space target; determining an attribute of each of the plurality of null space coefficients according to its corresponding target; merging the plurality of null space coefficients into a single null space coefficient using their associated attributes; and driving a joint based on the single null space coefficient according to a calculated joint movement to achieve a desired combination of null space targets.

[0111] Although exemplary embodiments have been described in detail and by way of example for purposes of clarity of understanding, various modifications, improvements and variations will be apparent to those skilled in the art. Therefore, the scope of the invention is intended to be limited solely by the appended claims.

Claims

1. A remote operation method, comprising: providing a manipulator arm comprising a movable distal surgical end effector, a proximal portion coupled to a base, and a plurality of joints between the distal portion and the base, the plurality of joints having sufficient degrees of freedom to allow a range of different joint states for a given end effector state; computing a first movement of a first set of joints in the plurality of joints within a null space of a Jacobian matrix according to a first objective, and computing a second movement of a second set of joints in the plurality of joints within the null space of the Jacobian matrix according to a second objective, the second objective being different from the first objective; determining a first attribute associated with a first calculated movement based on the first target, and determining a second attribute associated with a second calculated movement based on the second target; merging the first and second calculated movements into a combined movement of the plurality of joints based in part on respective properties of the first and second calculated movements according to a desired relationship and / or behavior between the first and second objects; as well as The plurality of joints are driven to achieve movement of the combination of the plurality of joints. 2 . The remote operation method of claim 1 , wherein each of the first set of joints and the second set of joints includes one or more joints from the plurality of joints. 3 . The remote operation method according to claim 2 , wherein the first group of joints and the second group of joints include one or more joints common to the plurality of joints and / or one or more different joints of the plurality of joints.

4. The teleoperation method of claim 2, wherein the first target or the second target comprises any one of: a joint state, a combination of joint states, a relative joint state, a range of joint states, a profile of joint states, a velocity of one or more of the multiple joints, a sum of the velocities of one or more of the multiple joints, a kinetic energy of one or more of the multiple joints, an engagement move, a motion cancellation move, a lock of one or more of the multiple joints, a collision avoidance move, a singularity avoidance move, a manipulator attitude preference, a desired manipulator configuration, a commanded reconfiguration of the manipulator, and an anisotropic enhancement of the joint. 5 . The teleoperation method of claim 2 , wherein each of the first and second attributes includes a weight corresponding to a weighting between the respective objectives.

6. The remote operation method of claim 2, wherein each of the first and second properties comprises a saturation level. 7 . The teleoperation method of claim 6 , wherein merging comprises applying the saturation level when the first and second moves conflict.

8. The remote operation method of claim 2, wherein each of the first and second attributes further comprises a priority between the first and second targets.

9. The teleoperation method of claim 8, wherein merging according to priority comprises applying the priorities of the attributes such that when the first target has a priority that exceeds the second target, merging comprises applying the first calculated movement at the expense of the second calculated movement. 10 . The teleoperation method of claim 8 , wherein merging according to priority includes applying only portions of the second calculated movement that do not conflict with the first calculated movement.

Images