Medical robotic system with linked control mode

Abstract

This application is titled "Medical Robot System with Linked Control Mode". In the linked control mode, the surgeon directly controls the movement of associated slave manipulators using input devices, and indirectly controls the movement of one or more non-associated slave manipulators in response to the direct control commands of the slave manipulators to achieve secondary objectives. By automating secondary tasks via the linked control mode, the system's usability is improved because it reduces the need for the surgeon to switch to another direct mode to manually achieve the desired secondary objectives. Therefore, the linked control mode allows the surgeon to better focus on performing medical procedures and pay less attention to manipulating the system.

Description

[0001] This application is a divisional application of Chinese patent application 201910402911.X, filed on May 11, 2011, entitled "Medical Robot System with Linked Control Mode". Patent application 201910402911.X is a divisional application of Chinese patent application 2016100890290, filed on May 11, 2011, entitled "Medical Robot System with Linked Control Mode". Patent application 2016100890290 is a divisional application of Chinese patent application 201180024113.X, filed on May 11, 2011, entitled "Medical Robot System with Linked Control Mode".

[0002] Cross-reference to related applications

[0003] This application is a continuation-in-part of U.S. Patent Application No. 11 / 762,200, entitled “Minimally Invasive Surgical System,” filed June 13, 2007, which is incorporated herein by reference.

[0004] This application is also a continuation-in-part of U.S. Patent Application No. 12 / 489,566, filed June 23, 2009, entitled "Medical robotic system providing an auxiliary view including range of motion limitations for articulated instruments extending out of a distal end of an entry guide," and U.S. Patent Application No. 12 / 613,328, filed November 5, 2009, entitled "Controller assisted reconfiguration of an articulated instrument during movement into and out of an entry guide," the latter being a continuation-in-part of U.S. Patent Application No. 12 / 613,328, filed August 15, 2009, entitled "Smooth control of an articulated instrument across areas with different work space." This continues in part with U.S. Patent Application No. 12 / 541,913, entitled "Smooth Control of Pivoted Devices in Areas with Different Workspace Conditions," all of which are incorporated herein by reference.

[0005] In addition, this application relates to the following U.S. patent applications, all of which are incorporated herein by reference:

[0006] Mohr’s U.S. Patent Application No. 11 / 762,217 entitled “Retraction of tissue for single port entry, robotically assisted medical procedures”;

[0007] Mohr et al.’s U.S. Patent Application No. 11 / 762,222 entitled “Bracing of bundled medical devices for single portentry, robotically assisted medical procedures”.

[0008] Schena’s U.S. Patent Application No. 11 / 762,231, entitled “Extendable suction surface for bracing medical devices during robotically assisted medical procedures”;

[0009] Diolaiti et al.’s U.S. Patent Application No. 11 / 762,236, entitled “Control system configured to compensate for non-ideal actuator-to-joint linkage characteristics in a medical robotic system”;

[0010] Cooper et al.’s U.S. Patent Application No. 11 / 762,185 entitled “Surgical instrument actuation system”;

[0011] Cooper et al.’s U.S. Patent Application No. 11 / 762,172 entitled “Surgical instrument actuator”;

[0012] Larkin et al.’s U.S. patent application number 11 / 762,165 entitled “Minimally invasive surgical system”;

[0013] Larkin et al.’s U.S. patent application number 11 / 762,161 entitled “Minimally invasive surgical instrument advancement”;

[0014] Cooper et al.’s U.S. Patent Application No. 11 / 762,158 entitled “Surgical instrument control and actuation”;

[0015] Cooper's U.S. Patent Application No. 11 / 762,154, entitled "Surgical instrument with parallel motion mechanism".

[0016] Larkin's U.S. patent application number 11 / 762,149, entitled "Minimally invasive surgical apparatus with side exit instruments";

[0017] Larkin's U.S. Patent Application No. 11 / 762,170, entitled "Minimally invasive surgical apparatus with side exit instruments";

[0018] Larkin's U.S. patent application number 11 / 762,143, entitled "Minimally invasive surgical instrument system";

[0019] Cooper et al.’s U.S. patent application number 11 / 762,135, entitled “Side looking minimally invasive surgery instrument assembly”;

[0020] Cooper et al.’s U.S. patent application number 11 / 762,132, entitled “Side looking minimally invasive surgery instrument assembly”;

[0021] Larkin et al.’s U.S. patent application number 11 / 762,127, entitled “Guide tube control of minimally invasive surgical instruments”;

[0022] Larkin et al.’s U.S. patent application number 11 / 762,123 entitled “Minimally invasive surgery guide tube”;

[0023] Larkin et al.’s U.S. patent application number 11 / 762,120, entitled “Minimally invasive surgery guide tube”;

[0024] Larkin's U.S. Patent Application No. 11 / 762,118, entitled "Minimally invasive surgical retractor system";

[0025] Schena et al.’s U.S. patent application number 11 / 762,114 entitled “Minimally invasive surgical illumination”;

[0026] Duval et al.’s U.S. Patent Application No. 11 / 762,110 entitled “Retrograde instrument”;

[0027] Duval et al.’s U.S. Patent Application No. 11 / 762,204 entitled “Retrograde instrument”;

[0028] Larkin's U.S. Patent Application No. 11 / 762,202, entitled "Preventing instrument / tissue collisions";

[0029] Larkin et al.’s U.S. patent application number 11 / 762,189, entitled “Minimally invasive surgery instrument assembly with reduced cross section”;

[0030] Larkin et al.'s U.S. Patent Application No. 11 / 762,191, entitled "Minimally invasive surgical system"; and

[0031] Duval et al.’s U.S. Patent Application No. 11 / 762,196 entitled “Minimally invasive surgical system”. Background Technology 1. Technical Field

[0033] This invention generally relates to medical robot systems, and more particularly to medical robot systems that provide a linkage control mode. 2. Background Technology

[0035] Minimally invasive surgery is known by various names (e.g., endoscopy, laparoscopy, arthroscopy, intravascular, keyhole, etc.) and is typically specific to the anatomical area where the work is performed. This type of surgery involves the use of handheld and remotely operated / manipulated / presented devices (robot-assisted / remote-controlled robotics), such as the da... (The sentence is incomplete and requires further context to translate accurately). Surgical systems. Used for both diagnostic (e.g., biopsy) and therapeutic procedures (“medical procedures”). Instruments can be inserted percutaneously into the patient via a surgical incision or through natural orifices. A novel experimental minimally invasive surgical variant is Natural Orifice Transluminal Endoscopic Surgery (NOTES), in which instruments are inserted through natural orifices (e.g., mouth, nostrils, ear canal, anus, vagina, urethra) and continued through a transluminal incision (e.g., in the wall of the stomach or colon) to the surgical site. Despite the use of da Remote-controlled surgery offers many benefits beyond those of many handheld procedures, but for some patients and certain anatomical areas, da Surgical systems may not be able to effectively access the surgical site. Furthermore, reducing the size and number of incisions often aids patient recovery and helps minimize patient injury and discomfort.

[0036] This medical robotic system provides multiple slave manipulators to perform useful functions such as manipulating instruments to perform medical procedures on patients, positioning and orienting imaging systems such as endoscopic imaging devices to capture images of the working ends of instruments, and transferring the working ends of instruments and the image-capturing ends of imaging systems to their working positions within the patient's body. The transfer of the working ends and image-capturing ends of instruments and imaging systems (“medical devices”) utilizes one or more guide tubes and a structure for gripping and manipulating the guide tubes (one or more). Additionally, master manipulators are used as input devices to track the movements of their operator's hand and provide the operator with appropriate tactile feedback indicative of their association with the states of the slave manipulators. Depending on their respective functions, slave and master manipulators (“robotic manipulators”) may be designed with different working areas and sensitivities.

[0037] Typically, the reachable working area of ​​a medical device manipulated by a manipulator is a set of points and orientations in space that can be reached by its distal end (e.g., the working end or image-capturing end). On the other hand, the sensitive working area of ​​the distal end of the medical device typically identifies a set of points in space that can be reached by primarily changing its orientation (e.g., changing the position of the wrist joint towards the distal end). For explanation, sensitivity is a measure of the ability of a robotic manipulator to control the position (in a limited way) and orientation of the working end of its associated medical device. Further, it involves joint degrees of freedom (i.e., the number of independently initiated joints in the kinematic chain of the robotic manipulator / medical device) and Cartesian / output degrees of freedom, the latter describing the independent rigid body position and orientation of the distal end. While the number of output (from the manipulator) degrees of freedom (DOFs) is typically at most six, the number of input (from the master manipulator) joint DOFs varies considerably depending on the master manipulator design.

[0038] As readily understood, the sensitive working area is typically a subset of the reachable working area. To enable surgeons to achieve precise control of the instrument's working end, instruments are often designed from the manipulator to optimize their sensitivity, even at the expense of their overall reachable working area. To compensate for this limitation, base manipulators (such as patient-side trolleys) with large reachable working areas can be used to transfer instruments and imaging systems from the manipulator to access openings within the patient's body (e.g., minimally invasive incisions or natural orifices). Furthermore, when instruments and imaging systems are placed within a common guide tube, the guide tube acts as a secondary base, as movement of the guide tube effectively moves all instruments and imaging systems placed within it. Subsequently, the instruments and imaging systems from the manipulator can ultimately transfer their respective medical device's working end and image-capturing end to their working position within the patient's body (e.g., the target anatomical structure).

[0039] The overall capability of a medical robotic system is achieved through a balance between the working areas and sensitivities of all the robotic manipulators that constitute it. However, the individual capabilities of each manipulator must be clearly understood and well-informed by the user in order to effectively utilize the system. It is often difficult for users to select which manipulator to control from the console and how to move it to achieve the desired "working configuration" of their respective medical devices within the patient's body, with the instrument's working end possessing the best possible sensitivity and extension, while simultaneously positioning the imaging system's capture end in a manner that provides good visualization of the medical procedure being performed in the working position without interfering with the movement of the instrument. Therefore, it is desirable to provide a system capable of secondary or coordinated controlled movement, for example, of camera manipulators and base manipulators (guide tube manipulators and / or manipulators for moving the assembly arm and / or supports for the patient-side support system), so as not to distract the user's attention from performing medical procedures while using surgical instruments.

[0040] The number of degrees of freedom (DOF) is the number of independent variables that uniquely identify the posture / configuration of a system. Since a robot manipulator is a kinematic chain that plots (input) joint space into (output) Cartesian space, the concept of DOF can be expressed in either of these two spaces. Specifically, a joint DOF set is a group of joint variables for all independently controlled joints. Typically, a joint is a mechanism that provides a single translational (prism joint) or rotational (rotary joint) DOF. According to the kinematic simulation perspective, any mechanism that provides more than one DOF motion is considered as two or more separate joints. A Cartesian DOF set is typically represented by three translational (position) variables (e.g., surge, heave, sway) and three rotational (or orientation) variables (e.g., Euler angles or roll / pitch / offset angles) describing the position and orientation of the end effector (or end effector) frame relative to a given Cartesian reference frame.

[0041] For example, a planar mechanism with end actuators mounted on two independent and vertical tracks has the ability to control the x / y position (prism DOF) within the area spanned by the two tracks. If the end actuators are rotatable about an axis perpendicular to the track plane, there are three input DOFs (two track positions and offset angles) corresponding to three output DOFs (x / y position and orientation angle of the end actuator).

[0042] Although the number of Cartesian DOFs is at most six—the case where all translational and orientation variables are independently controlled—the number of joint DOFs is usually a result of design choices that take into account mechanism complexity and task details. Therefore, the number of joint DOFs can be more, equal to, or less than six. For non-redundant kinematic chains, the number of independently controlled joints equals the degree of mobility of the end-actuator frame. For some prismatic and rotary joint DOFs, the end-actuator frame has an equal number of DOFs in Cartesian space corresponding to combinations of translational (x / y / z position) and rotational (roll / pitch / off-orientation angle) motions (except in odd-numbered configurations).

[0043] The difference between input and output degrees of freedom (DOF) is extremely important in the case of redundant or “defective” kinematic chains (e.g., mechanical manipulators). Specifically, a “defective” manipulator has fewer than six independently controlled joints and therefore cannot fully control the position and orientation of the end actuator. Instead, a defective manipulator is limited to controlling only a subset of position and orientation variables. On the other hand, a redundant manipulator has more than six joint DOFs. Therefore, a redundant manipulator can use more than one joint configuration to establish the desired 6-DOF end actuator posture. In other words, the additional degrees of freedom can be used to control not only the position and orientation of the end actuator but also the “shape” of the manipulator itself. In addition to kinematic degrees of freedom, mechanisms can have other DOFs, such as the pivoting lever motion of a gripper or scissor blade.

[0044] It is equally important to consider a reference frame in the space of the specified DOF. For example, a single DOF change in the joint space (e.g., joint rotation between two links) can result in a motion combining changes in the Cartesian translation and orientation variables of the frame attached to the distal end of one of the links (rotation and translation of the frame at the distal end through space). Kinematics describes the process of transforming from one measurement space to another. For example, using joint space measurements at the end of a kinematic chain to determine the Cartesian spatial position and orientation of a reference frame is “forward” kinematics. Using the Cartesian spatial position and orientation of a reference frame at the end of a kinematic chain to determine the required joint position is “reverse” kinematics. If any swivel joint is present, the kinematics involves nonlinear functions (trigonometric functions). Summary of the Invention

[0045] One object of this invention is to provide a linkage control mode in which one or more devices can be directly controlled to achieve a primary objective, and one or more other devices can be indirectly controlled to achieve a secondary objective.

[0046] These and other objectives are accomplished through various aspects of the invention, wherein, in brief, one aspect is a robotic system comprising: a first instrument; a base connected to the first instrument such that the first instrument moves when the base moves; a base controller means for moving the base to optimize the working area of ​​the first instrument when the first instrument moves; and a first instrument controller for moving the first instrument according to instructions while compensating for the movement of the base.

[0047] Another aspect is a method for controlling the movement of one or more instruments connected to a base so that the instruments move when the base moves, the method comprising: instructing the manipulation of the base to optimize the operable working area of ​​a first instrument; and instructing the manipulation of the first instrument according to the instruction to move, while compensating for the movement of the base.

[0048] Another aspect is a robotic system comprising: multiple devices, each manipulated by a corresponding manipulator from a plurality of slave manipulators; a master input device; and means for instructing the slave manipulators to move their respective devices in a common degree-of-freedom direction in response to the movement of the master input device.

[0049] Another aspect is a method for retracting multiple devices into a guide tube, the method comprising: receiving a retraction command from an input device; in response to the retraction command, retracting together multiple devices extending beyond the distal end of the guide tube toward the guide tube; and driving the multiple devices into a retraction configuration so that each of the multiple devices can freely enter the guide tube.

[0050] Another aspect is a robotic system comprising: a plurality of instruments, each manipulated by a corresponding manipulator of a plurality of slave manipulators; an imaging system manipulated by an imaging system manipulator; a master input device; and means for instructing the plurality of slave manipulators to move the plurality of instruments in response to the movement of the imaging system when the imaging system manipulator moves in response to a movement of the master input device.

[0051] Another aspect is a method for linking the control of an imaging system with other devices in a robotic system, which includes moving multiple instruments so that they follow the image capture end of the imaging system when the imaging system moves in response to the movement of the main input device. Attached Figure Description

[0052] Figure 1 A schematic diagram of a robot-assisted minimally invasive remote-controlled surgery system.

[0053] Figure 2 and 3 This is a schematic diagram of the patient-side support system in a remote surgical system.

[0054] Figure 4 A simplified front view of the surgeon's console in a remote surgical system.

[0055] Figure 5 This is a schematic diagram illustrating various aspects of the components of a minimally invasive surgical instrument.

[0056] Figure 6 This is a schematic diagram illustrating various aspects of the components of a minimally invasive surgical instrument.

[0057] Figure 7 for Figure 6 A detailed schematic side view.

[0058] Figure 8 A perspective view of surgical instrument components.

[0059] Figure 9This is a schematic diagram of the interface between the surgical instrument assembly and the actuator assembly.

[0060] Figure 10 This is a perspective view of the proximal segment of a minimally invasive surgical instrument.

[0061] Figure 11 To coordinate and start Figure 10 A perspective view of a segment of the actuator assembly of the device shown.

[0062] Figure 12 A perspective view illustrating the mounting of minimally invasive surgical instruments and actuator assemblies on the end of an assembly arm.

[0063] Figure 13 Another illustrative perspective view to illustrate the mounting of minimally invasive surgical instruments and actuator assemblies on the end of the assembly arm.

[0064] Figure 14 A schematic view of the transmission mechanism associated with a flexible coaxial guide tube and instrument.

[0065] Figure 15 A diagrammatic view of a multi-port surgery.

[0066] Figure 16 Another illustrated view of multi-port surgery.

[0067] Figure 17-19 A schematic plan view illustrating further aspects of preventing unwanted collisions between instruments and tissues.

[0068] Figure 20 A graphical view of an image mosaic output display for use by surgeons.

[0069] Figure 21 An illustrative perspective view of a minimally invasive surgical instrument assembly, including a multi-joint instrument designed for traction.

[0070] Figure 22 This is a block diagram of components used in a remote surgical system for controlling and selectively associating devices on a patient-side support system with an input device.

[0071] Figure 23 A block diagram of a master / slave control system included in the manipulator controller of a remote surgical system.

[0072] Figure 24-25 A block diagram of the direct "tool tracking" mode system executed in the manipulator controller of a remote surgical system.

[0073] Figure 26-27 A block diagram of the direct "imaging system" mode architecture executed in the manipulator controller of a remote surgical system.

[0074] Figures 28-29 A block diagram of the direct "guide tube" mode system executed in the manipulator controller of a remote surgical system.

[0075] Figure 30 This is a diagram of the centralized motion control system of a minimally invasive remote-controlled surgery system.

[0076] Figure 31 A diagrammatic view of the distributed motion control system of a minimally invasive remote-controlled surgery system.

[0077] Figure 32 A block diagram of the linkage "tool tracking" mode system executed in the manipulator controller of a remote surgical system.

[0078] Figure 33 A block diagram of the linkage "imaging system" mode system executed in the manipulator controller of a remote surgical system.

[0079] Figure 34 A block diagram of the linkage "guide tube" mode system executed in the manipulator controller of a remote surgical system.

[0080] Figures 35-37 This is a flowchart illustrating an example of a device linkage control mode.

[0081] Figure 38 This is a flowchart illustrating an example of a guide tube linkage control mode.

[0082] Figure 39 This is a flowchart for retracting a tool into a perforated guide tube, which is used for tool replacement or accessory provisioning operations in a linked "tool tracking" mode instance.

[0083] Figure 40 This is a flowchart illustrating an example of a linkage control mode for an imaging system. Invention Details

[0084] This specification and accompanying drawings, which illustrate aspects and embodiments of the invention, should not be considered limiting—the claims define the protected invention. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the spirit and scope of this specification and the claims. In some cases, well-known circuits, structures, and techniques have not been shown in detail to avoid obscuring the invention. Identical numbers in two or more figures represent the same or similar elements.

[0085] Furthermore, the terminology used in this specification is not intended to limit the invention. For example, spatially relative terms—such as “below,” “under,” “lower,” “above,” “upper,” “proximal,” “farthest,” etc.—are used to describe the relationship of one element or feature to another element or feature as illustrated. In addition to the positions and orientations shown in the figures, these spatially relative terms are intended to encompass different positions and orientations of the device in use or operation. For example, if the device in the figures is flipped, an element described as “below other elements or features” or “below other elements or features” would become “above other elements or features” or “on top of other elements or features.” Thus, the exemplary term “below” can encompass both above and below positions and orientations. The device can be in other orientations (rotated 90 degrees or in other orientations), and the spatially relative descriptive terms used herein are interpreted accordingly. Similarly, descriptions of movement along and about different axes encompass different specific device positions and orientations. Additionally, the singular forms “a,” “an,” and “the” are also intended to encompass the plural forms unless the context otherwise requires. Meanwhile, the terms "comprising," "including," and "include" specify the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups. Components described as connected may be electrically or mechanically directly connected, or they may be indirectly connected via one or more intermediate components.

[0086] Remote control and similar terms generally refer to the manipulation of a master device (e.g., an input kinematic chain) by an operator in a relatively natural manner (e.g., natural hand or finger movements), on which the movement of the master device is made into instructions that are processed in real time and transmitted to a slave device (e.g., an output kinematic chain) that responds almost instantly to the instructions and environmental forces. Remote control is disclosed in U.S. Patent No. 6,574,355 (Green), which is incorporated herein by reference.

[0087] To avoid repetition in the figures and descriptions of the following multiple aspects and illustrative embodiments, it should be understood that many features are common to many aspects and embodiments. The omission of an aspect in a description or figure does not mean that the aspect disappears from the embodiments incorporated into that aspect. Rather, the aspect may have been omitted for clarity and to avoid lengthy descriptions.

[0088] Therefore, several general aspects apply to the following descriptions. For example, at least one surgical end actuator is shown or described in different figures. An end actuator is part of a minimally invasive surgical instrument or component that performs a specific surgical function (e.g., surgical forceps / grippers, needle actuators, scissors, electrocautery hooks, staplers, clip applicators / removers, etc.). Many end actuators have a single DOF (e.g., an open and close gripper). End actuators can be connected to the surgical instrument body via mechanisms such as “wrist” type mechanisms that provide one or more additional DOFs. Examples of such mechanisms are shown in U.S. Patent No. 6,371,952 (Madhani et al.) and U.S. Patent No. 6,817,974 (Cooper et al.), both of which are incorporated herein by reference and are known to be various Intuitive Surgical, Inc. Institutions, such as those used for da Both 8mm and 5mm instruments of the surgical system are described herein. Although the surgical instruments described herein generally include end actuators, it should be understood that end actuators may be omitted in some respects. For example, the distal end of the instrument body shaft may be used for tissue traction. As another example, aspiration or irrigation opening may be present at the distal end of the body shaft or wrist joint mechanism. In these respects, it should be understood that the description of positioning and orienting end actuators includes positioning and orienting the end of the surgical instrument without end actuators. For example, the description of a reference frame for the end of an end actuator should also be understood to include a reference frame for the end of the surgical instrument without end actuators.

[0089] In this specification, it should be understood that a single-imaging or stereoscopic imaging system / image capture component / camera device may be placed at the distal end of the instrument, wherever the end actuator (which may be considered a "camera device") is shown or described, or it may be placed near or at the distal end of any guide tube or other instrument component element. Therefore, the term "imaging system" and similar terms as used herein should be broadly interpreted to include both the image capture component and the combination of the image capture component with associated circuitry and hardware, within the context of the described aspects and embodiments. Such endoscopic imaging systems (e.g., optical, infrared, ultrasonic, etc.) include systems with a distally placed image sensing chip and associated circuitry that transmit captured image data externally via a wired or wireless connection. Such endoscopic imaging systems also include systems for externally transmitting captured images (e.g., using cylindrical lenses or optical fibers). In some instruments or instrument components, a direct-viewing optical system (where the endoscopic image is seen directly in the eyepiece) may be used. Examples of distally placed semiconductor stereoscopic imaging systems are described in U.S. Patent Application No. 11 / 614,661, “Stereoscopic Endoscope” (Shafer et al.), which is incorporated herein by reference. For clarity, well-known endoscopic imaging system components such as electrical illumination connections and fiber optic illumination connections are omitted or symbolically represented. Illumination for endoscopic imaging is typically represented in the figures by a single illumination port. It should be understood that these descriptions are exemplary. The size, location, and number of illumination ports can vary. Illumination ports are typically arranged on multiple sides of the imaging aperture or completely surround the imaging aperture to minimize deep shadows.

[0090] In this specification, the cannula is generally used to prevent surgical instruments or guide tubes from rubbing against patient tissue. Cannulas can be used for both incisions and natural openings. Cannulas are not necessary for cases where the instrument or guide tube does not frequently translate or rotate relative to its insertion (longitudinal) axis. For cases requiring air inhalation, the cannula may contain a seal to prevent excessive air leakage through the instrument or guide tube. For example, in thoracic surgeries where air inhalation is not required, the cannula seal may be omitted, and the cannula itself may be omitted if the movement of the instrument or guide tube insertion axis is minimal. In some configurations of instruments inserted relative to the guide tube, a rigid guide tube can function as a cannula. Cannulas and guide tubes can be, for example, steel or extruded plastic. Plastics, which are less expensive than steel, are suitable for single-use applications.

[0091] Various examples and components of flexible surgical instruments and guide tubes are considered suitable for the present invention. In this specification, this flexibility is achieved in different ways. For example, a segment of an instrument or guide tube may be a continuously curved flexible structure, such as a structure based on a spirally wound coil or a tube with multiple segments removed (e.g., a suture-type incision). Alternatively, the flexible portion may be made of a series of short, pivotally connected segments (“vertebral bodies”) that provide a nearly serpentine, continuously curved structure. Instrument and guide tube structures may include those in U.S. Patent Application Publication No. US2004 / 0138700 (Cooper et al.), which are incorporated herein by reference. For clarity, figures and associated descriptions generally show only two segments of the instrument and guide tube, referred to as the proximal end (closer to the drive mechanism; further from the surgical site) and the distal end (further from the drive mechanism; closer to the surgical site). It should be understood that the instrument and guide tube may be divided into three or more segments, each of which may be rigid, passively flexible, or actively flexible. The flexure and bending described for the distal segment, proximal segment, or the entire mechanism also apply to intermediate segments that have been omitted for clarity. For example, the intermediate segment between the proximal and distal segments can be bent into a simple or complex curve. Flexible segments can be of various lengths. When bent, a segment with a smaller outer diameter can have a smaller minimum radius of curvature compared to a segment with a larger outer diameter. For cable-controlled systems, unacceptably high cable friction or adhesion limits the minimum radius of curvature and total bending angle when bent. The minimum bending radius of a guide tube (or any joint) is such that it does not twist or otherwise impede the smooth movement of the mechanism of internal surgical instruments. Flexible components can be, for example, up to approximately four feet in length and approximately 0.6 inches in diameter. Other lengths and diameters (e.g., shorter, smaller) and degrees of flexibility for a particular mechanism can be determined by the target anatomy for which the mechanism is designed.

[0092] In some cases, only the distal segment of the device or guide tube is flexible, while the proximal segment is rigid. In other cases, the entire device or guide tube within the patient is flexible. In still other cases, the distal segment may be rigid, and one or more other proximal segments may be flexible. The flexible segments may be passive or they may be actively controllable (“manipulated”). Such active control may be implemented using, for example, multiple sets of opposing cables (e.g., one set controls “pitch”, and an orthogonal set controls “deviation”; three cables may be used to perform similar actions). Other control elements may be used, such as small electric or magnetic actuators, shape memory alloys, electroactive polymers (“artificial muscles”), pneumatic or hydraulic bellows or pistons, and the like. In cases where one device or guide tube is entirely or partially located within another guide tube, various combinations of passive and active flexibility may exist. For example, an active flexible device within a passively flexible guide tube may apply sufficient lateral force to flex the surrounding guide tube. Similarly, an active flexible guide tube may flex a passive flexible device located within it. The active flexible sections of the guide tube and the instrument can work together. For both flexible and rigid instruments and guide tubes, control cables placed further away from the central longitudinal axis can provide mechanical advantages over cables placed closer to the central longitudinal axis, depending on the flexibility considerations in different designs.

[0093] The flexibility (stiffness) of a flexible segment can vary from almost complete relaxation (with minor internal friction) to substantially rigidity. In some respects, the flexibility is controllable. For example, a section or all of the flexible segment of an instrument or guide tube can be made substantially (i.e., effectively but not infinitely) rigid (the segment is "rigidifiable" or "lockable"). A lockable segment can be locked in a straight line, a simple curve, or in the shape of a compound curve. Locking can be accomplished by applying tension to one or more cables that travel longitudinally along the instrument or guide tube along a path sufficient to induce friction to prevent movement of adjacent vertebral bodies. The cables or cables can travel through a large central hole in each vertebral body or through a smaller hole near the periphery of the vertebral body. Optionally, the drive element of one or more motors that move one or more control cables can be soft-locked in place (e.g., by servo control) to keep the cables in place and thereby prevent movement of the instrument or guide tube, thus locking the vertebral bodies in the correct position. Maintaining the motor drive element in the correct position can be performed to effectively keep other movable instrument and guide tube components in the correct position as well. It should be understood that, while effective, the stiffness under servo control is generally less than the stiffness achievable by applying a brake directly to the connector, such as a brake used to keep a passively assembled connector in the correct position. Cable stiffness is usually dominant because it is typically less than the stiffness of the servo system or the brake connector.

[0094] In some cases, the flexibility of the flexible segment can be continuously varied between relaxed and rigid states. For example, the tension of the locking cable can be increased to increase stiffness without locking the flexible segment into a rigid state. This intermediate flexibility allows for remote operation while reducing tissue damage that can occur due to movement from the surgical site caused by reaction forces. A suitable bending sensor incorporated into the flexible segment allows the remote surgical system to determine the position of instruments and / or guide tubes when bending. U.S. Patent Application Publication No. US2006 / 0013523 (Childers et al.), which is incorporated by reference, discloses a fiber optic position and shape sensing device and method. U.S. Patent Application No. 11 / 491,384 (Larkin et al.), which is incorporated by reference, discloses a fiber optic bending sensor (e.g., a fiber Bragg grating) for controlling such segments and flexible devices.

[0095] Surgeons typically input information to control aspects of minimally invasive surgical instrument components, instruments, and end effectors, as described herein, using intuitive, camera-referenced control interfaces. For example, da The surgical system includes a surgeon's console with such a control interface, which can be modified to control the aspects described herein. The surgeon manipulates one or more master manual input mechanisms, having, for example, six DOFs, to control the instrument components and instruments. The input mechanisms include finger-operated grippers to control one or more end-actuator DOFs (e.g., closing the gripper clamp). Intuitive control is provided by orienting the relative positions of the end-actuators and the endoscopic imaging system using the position of the surgeon's input mechanisms and the image output display. This orientation allows the surgeon to manipulate the input mechanisms and end-actuator controls as if seeing a substantially real surgical work site. This remote operation realism refers to the surgeon seeing an image from a perspective that appears to be an image that the operator directly sees and works on at the surgical site. U.S. Patent No. 6,671,581 (Niemeyer et al.), incorporated herein by reference, contains further information regarding the control of camera references in minimally invasive surgical instruments.

[0096] Figure 1 This is a schematic diagram illustrating aspects of a robot-assisted (remotely controlled) minimally invasive surgical system 2100, in which instruments are inserted into the patient via a single access port through a guide tube. The general architecture of this system is similar to other such systems, such as those from Intuitive Surgical, Inc. Surgical systems and Surgical system. The three main components are the surgeon's console 2102, the patient-side support system 2104, and the video system 2106, all of which are interconnected as shown via wired or wireless connections 2108.

[0097] like Figure 4 As shown, the surgeon's console 2102 includes, for example, hand-operable, multi-DOF mechanical input ("master") devices 203, 204 and foot pedals 215, 217, which allow the surgeon to manipulate surgical instruments, guide tubes, and imaging systems ("slave") as described herein. These input devices can provide tactile feedback to the surgeon in some respects from instruments and instrument component parts. Buttons 205, 207 are provided on the hand-operable input devices 203, 204 for switching functions as described herein or for other operational purposes. The console 2102 also includes a stereoscopic video output display 201, configured to focus the image on the display approximately at a distance corresponding to the surgeon's hand working behind / below the display screen. A processor 220, communicating with other components of the console via bus 210, implements various functions within system 2100. One important function it implements is to execute various controllers described herein to translate and transfer the mechanical movement of the input devices via control signals, enabling the surgeon to effectively manipulate and otherwise move devices such as surgical instruments, imaging systems, and one or more guide tubes selectively associated with the input devices at that time. Although described as a processor, it should be understood that processor 220 can be executed in practice by any combination of hardware, software, and firmware. Similarly, its functionality as described herein can be implemented by a single unit or distributed among different components, each of which can be executed sequentially by any combination of hardware, software, and firmware. Furthermore, although shown as part of or physically adjacent to console 2102, processor 220 may also include a number of subunits distributed throughout the system. These aspects are discussed more fully in U.S. Patent No. 6,671,581, which is incorporated herein by reference.

[0098] Turn back Figure 1 The patient-side support system 2104 includes a floor-mounted structure 2110, or optionally a ceiling-mounted structure 2112, as shown by the dotted line. Structure 2110 can be movable or fixed (e.g., relative to a floor, ceiling, or other equipment such as an operating table). In one embodiment, the mounting arm assembly 2114 is an improved da Surgical system arm assembly. Arm assembly 2114 includes two illustrative passive rotary assembly joints 2114a, 2114b, which allow manual positioning of the connected links when their brakes are released. A passive prismatic assembly joint (not shown) between the arm assembly and structure 2110 is available to allow for large vertical adjustments. Additionally, guide tube manipulator 2116 includes illustrative active roll joint 2116a and active deflection joint 2116b. Joints 2116c and 2116d act as parallel mechanisms to allow the guide tube (of the surgical instrument assembly), held by platform 2118, to move around a remote center 2120 at an access point such as the umbilicus of patient 1222. In one embodiment, active prismatic joint 2124 is used for inserting and withdrawing the guide tube. One or more surgical instruments and an endoscopic imaging system are independently mounted to platform 2118. As patient 2122 is placed in different positions on movable stage 2126, the various assembly and active joints allow the manipulator to move the guide tube, instruments, and imaging system.

[0099] Figure 2 and 3 Schematic side and front views of another illustrative embodiment of the patient-side support system. Support 2150 is fixed (e.g., mounted on the floor or ceiling). Link 2152 is connected to support 2150 at a passive rotary fitting joint 2154. As shown, the axis of rotation of fitting 2154 is aligned with a remote center point 2156, which is typically the location where a guide tube (not shown) (of a surgical instrument assembly) enters the patient (e.g., at the umbilicus for abdominal surgery). Link 2158 is connected to link 2152 at a rotary fitting joint 2160. Link 2162 is connected to link 2158 at a rotary fitting joint 2164. Link 2166 is connected to link 2162 at a rotary fitting joint 2168. A guide tube is mounted to slide through end 2166a of link 2166. Platform 2170 is supported by prism fitting 2172 and rotary fitting 2174 and connected to link 2166. As it slides along link 2166, prism connector 2172 inserts into and retracts from the guide tube. Connector 2174 includes a bearing assembly that holds the cantilever of the "C"-shaped ring. As the "C"-ring slides through the bearing, it rotates about a central point within the "C," thereby rolling the guide tube. The opening in the "C" allows for the installation or replacement of the guide tube without moving the manipulator above it. Platform 2170 supports multiple manipulators 2176 for surgical instruments and an imaging system, as described below.

[0100] These illustrative robotic arm assemblies are used, for example, in instrument assemblies that include rigid guide tubes and are manipulated to move relative to a remote center. If movement about a remote center is not required, some assemblies and active joints in the manipulator arm can be omitted. It should be understood that the manipulator arm can contain different combinations of links, passive and active joints (providing redundant DOF) to achieve the necessary range of surgical postures.

[0101] Refer again Figure 1 The video system 2106 performs image processing functions for, for example, endoscopic imaging data captured at the surgical site and / or preoperative or real-time image data from other imaging systems outside the patient. The video system 2106 outputs the processed image data (e.g., an image of the surgical site, along with associated controls and patient information) to the surgeon at the surgeon's console 2102. In some aspects, the processed image data is output to optional external monitors visible to other operating room personnel or to one or more locations remote from the operating room (e.g., the surgeon at another location can monitor the video; live-transmitted video can be used for training, etc.).

[0102] As an example of a medical device component, Figure 5 This is a schematic diagram illustrating various aspects of the minimally invasive surgical instrument assembly 1600. Two surgical instruments 1602a and 1602b extend through channels 1604a and 1604b that extend longitudinally through a rigid guide tube 1606. In some aspects, the guide tube 1606 is straight; in others, it is bent to accommodate a specific insertion port (the instruments are similarly bent to facilitate insertion). The guide tube 1606 can have various cross-sectional shapes (e.g., circular, elliptical, rounded polygons) and can use different numbers of surgical instruments and channels. Some optional working channels can be used to support surgical functions such as irrigation and aspiration. In some aspects, an endoscopic imaging system (e.g., single-image or stereoscopic image capture or direct viewing) is located at the distal end 1610 of the guide tube 1606. In one aspect, the guide tube 1606 is inserted into the patient through an incision (e.g., approximately 2.0 cm at the umbilicus) or natural orifice, with or without a cannula 1612 or similar guiding structure. In some respects, the guide tube 1606 can rotate within the sleeve 1612.

[0103] Surgical instruments 1602a and 1602b function in a similar manner, and many of their functions (body rotation, wrist manipulation, end-actuator manipulation, etc.) are similar to those used in da Surgical instruments for the surgical system (8mm and 5mm instrument body diameter). In other aspects, the instruments may serve different functions and / or have features not present in the da... The capabilities embodied in surgical system instruments (e.g., an instrument can be straight, an instrument can be joined, an instrument can be flexible, etc.). In this example, instrument 1602a includes a transmission portion (not shown) at its proximal end, an elongated instrument body 1614, one of a variety of surgical end actuators 1616, and a serpentine two-degree-of-freedom wrist joint mechanism 1618 connecting the end actuator 1616 to the instrument body 1614. As in da In some aspects of the surgical system, the drive mechanism includes a disc connected to an electric actuator (e.g., a servo motor) permanently mounted on a support arm for easy instrument replacement. Other connection systems, such as mating universal joint plates and levers, can be used to transfer actuation forces on the mechanical interface. Mechanical mechanisms (e.g., gears, levers, universal joints) in the drive mechanism transfer actuation forces from the disc to cables, wires, and / or combinations of cables, wires, and hypotubes traveling through one or more channels in the instrument body 1614 (which may include one or more pivot sections) to control movement of the wrist joint 1618 and the end actuator 1616. In some aspects, one or more discs and associated mechanisms transfer actuation forces to roll the instrument body 1614 about a longitudinal axis 1619, as shown. In some aspects, the actuator of a specific instrument is itself mounted on a single linear actuator that moves the instrument body 1614 longitudinally within a channel 1604a, as shown. The main section of the device body 1614 is a substantially rigid single tube, although in some respects it may be slightly flexible. This slight flexibility allows the proximal body section 1620 (i.e., the patient's exterior) near the guide tube 1606 to be slightly bent so that several device bodies can be spaced closer together within the guide tube 1606 than their separate drive section housings would otherwise allow, as if several flower vases of equal length were placed in a small-necked vase. This bending is minimal (e.g., in one embodiment, a bend angle of less than or equal to approximately 5 degrees) and does not cause significant friction because of the small bend angles of the control cables and hysteresis tubes within the device body.

[0104] Devices 1602a and 1602b each include a proximal body segment extending through a guide tube and at least one distal body segment positioned beyond the guide tube. For example, device 1602a includes a proximal body segment 1620 extending through a guide tube 1606, a distal body segment 1622 connected to the proximal body segment 1620 at a connector 1624, a wrist joint mechanism 1626 (which may include another short distal body segment) connected to the distal body segment 1622 at another connector 1628, and an end actuator 1630. In some aspects, the distal body segment 1622 and connectors 1624 and 1628 function as a parallel motion mechanism 1632, wherein the position of a reference frame at the distal end of the mechanism can be changed relative to a reference frame at the proximal end of the mechanism without changing the orientation of the distal reference frame.

[0105] Figure 6 The diagram illustrates various aspects of another minimally invasive surgical instrument assembly 1700. Surgical instrument assembly 1700 is similar to instrument assembly 1600, wherein surgical instruments 1702a and 1702b function similarly to instruments 1602a and 1602b as described above, but assembly 1700 has an independently operating endoscopic imaging system 1704, rather than a fixed endoscopic imaging system at the end of the guide tube.

[0106] In one respect, the imaging system 1704 is mechanically similar to the surgical instrument 1602 as described above. In summary... Figure 6 In these aspects, the optical system 1704 includes a substantially rigid, elongated proximal body segment 1706 extending through a guide tube 1708, and at the distal end of the proximal body segment 1706, a parallel motion mechanism 1712, similar to parallel motion mechanism 1622, is connected to one or two DOFs. Parallel motion mechanism 1712 includes a first connector 1714, an intermediate distal body segment 1716, and a second connector 1718. A wrist joint mechanism or other active connector (e.g., one DOF to allow changes in pitch angle; two DOFs to allow changes in pitch and drift angles) 1720 connects the image capture component 1722 to the second connector 1718. Optionally, connector 1714 is an independently controllable one or two DOF connector (pitch / drift), connector 1718 is another independently controllable one or two DOF connector (e.g., pitch / drift), and the image capture component 1722 is directly connected to the distal end of the connector 1718 mechanism. Examples of suitable stereoscopic image capturing components are shown in U.S. Patent Application No. 11 / 614,661, which is incorporated herein by reference. In some aspects, the imaging system 1704 moves longitudinally (back and forth) within the guide tube 1708. Control of the imaging system 1704 is further described in concurrently filed U.S. Patent Application No. 11 / 762,236, which is incorporated herein by reference. In some aspects, roll may be undesirable because it is necessary to maintain a specific field of view orientation. Having vertical translation (up / down), horizontal translation (side to side), back and forth translation (retraction / insertion), offset, and pitch (DOF) allows the image capturing component to move to different positions while maintaining a specific camera reference for component 1700 and seeing the surgeon's calibration.

[0107] Figure 7 This is for illustrative purposes only. Figure 6 A side view of the schematic plan. Figure 7The parallel motion mechanism 1712 is shown moving the image capture unit 1722 away from the longitudinal centerline of the surgical instrument assembly 1700. This shift provides an improved view of the surgical site 1724 because some or all of the distal ends of the instrument body are not present in the image output to the surgeon, as occurs, for example, in the instrument assembly 1600. Figure 5 The pitch of the parallel motion mechanism 1712 and the image capture unit 1722 is controllable, as illustrated by the arrows.

[0108] Figure 8 This is a schematic perspective view illustrating an embodiment of the surgical instrument assembly 1700. As shown, two independent, remotely operated surgical instruments 1740a and 1740b (each instrument associated with a separate master manipulator—e.g., a left-hand master manipulator for the left instrument and a right-hand master manipulator for the right instrument) travel through and appear at the distal end of the rigid guide tube 1742. Each instrument 1740a and 1740b is one of the six DOF instruments described above and includes parallel motion mechanisms 1744a and 1744b as described above, with wrist joints 1746a and 1746b attached and end actuators 1748a and 1748b. Additionally, an independently remotely operated endoscopic imaging system 1750 travels through and appears at the distal end of the guide tube 1742. In some aspects, the imaging system 1750 also includes a parallel motion mechanism 1752, a tilt-only wrist joint mechanism 1754 distal to the parallel motion mechanism 1752 (which may have one or two DOFs in the joint space), and a stereoscopic endoscopic image capturing unit 1756 connected to the wrist joint mechanism 1754. In other aspects, the wrist joint mechanism 1754 may include an offset DOF. In yet another aspect, the proximal and distal joints in the imaging system 1750 are independently controlled. In an illustrative use, the parallel motion mechanism 1752 translates the image capturing unit 1756 vertically and horizontally, and the wrist joint mechanism 1754 is oriented toward the image capturing unit 1756 to center the field of view between the instrument ends if the instrument is operating laterally toward the extended centerline of the guide tube. In another illustrative use, the distal body segment of the imaging system is independently tilted upward (and in some aspects, independently offset), and the image capturing unit 1756 is independently tilted downward (and in some aspects, independently offset). As discussed above and below, the imaging system 1750 can be moved to multiple locations to traction tissue.

[0109] Also shown is the auxiliary channel 1760, through which procedures such as irrigation, aspiration, or other surgical operations can be introduced or withdrawn. In some aspects, one or more small, easily manipulated devices can be inserted through the auxiliary channel 1760 to clean the windows of the imaging system by spraying cleaning fluid (e.g., pressurized water, gas) and / or desiccant (e.g., pressurized air or blown gas). In another aspect, such a cleaning rod can be a passive device attached to the camera before insertion. In yet another aspect, the end of the rod is automatically hooked onto the image capture component as it emerges from the distal end of the guide tube. A spring gently pulls the cleaning rod so that it tends to retract into the guide tube when the imaging system is withdrawn from it.

[0110] Figure 7 Further illustrating that when the moving image-capturing component 1722 is moved away from the centerline of component 1700, it can be pressed against and moved against the tissue structure surface 1726 thereon, thereby pulling the tissue structure from the surgical site as shown. Using the imaging system 1704 to pull tissue is an example of using other surgical instruments or devices specifically designed for this task to pull tissue. This “tent-post” type of traction can be achieved by any of the various movable components described herein, such as parallel motion mechanisms on distal or lateral exit flexible devices and rigid body component devices, as well as other devices discussed below (e.g., see references). Figure 21 ).

[0111] Figure 9 This schematic diagram illustrates aspects of the interface between the surgical instrument assembly 2302 and the illustrative actuator assembly 2304. The surgical instrument assembly 2302 represents flexible and rigid mechanisms as described herein in various ways. For the purposes of this example, the instrument assembly 2302 includes a surgical instrument 2306, a main guide tube 2308 surrounding the instrument 2306, and a secondary guide tube 2310 surrounding the main guide tube 2308.

[0112] like Figure 9 As shown, transmission mechanisms are disposed on the proximal end of each instrument or guide tube: transmission mechanism 2306a for instrument 2306, transmission mechanism 2308a for the main guide tube 2308, and transmission mechanism 2310a for the secondary guide tube 2310. Each transmission mechanism is mechanically and movably connected to an associated actuator mechanism: transmission mechanism 2306a is connected to actuator mechanism 2312, transmission mechanism 2308a is connected to actuator mechanism 2314, and transmission mechanism 2310a is connected to actuator mechanism 2316. In one aspect, a mating disc is used, as in da In the surgical system instrument interface, as shown in more detail below, a mating universal joint plate and lever are used in another aspect. Multiple mechanical components in the drive mechanism (e.g., gears, levers, cables, pulley systems, cable guides, universal joints, etc.) are used to transmit mechanical force from the interface to the controlled element. Each actuator mechanism includes at least one actuator (e.g., a servo motor (brushed or brushless)) that controls movement on the distal end of the associated instrument or guide tube. For example, actuator 2312a is an electric servo motor that controls the end actuator 2306b clamp DOF of surgical instrument 2306. An instrument (containing a guide detector as described herein) or guide tube (or collectively, instrument assembly) can be detached from and slid out of the associated actuator mechanism (one or more) as shown. It can then be replaced by another instrument or guide tube. In addition to the mechanical interface, an electronic interface exists between each drive mechanism and actuator mechanism. This electronic interface allows the transmission of data (e.g., instrument / guide tube type).

[0113] In some cases, one or more DOFs can be manually activated. For example, surgical instrument 2306 may be a passive flexible laparoscopic instrument with a hand-activated actuator gripper DOF, and guide tube 2308 may be actively maneuverable to provide wrist joint movement as described above. In this example, the surgeon servo-controls the guide tube DOF and the assistant controls the instrument gripper DOF.

[0114] In addition to the actuators controlling the instrument and / or guide tube elements, each actuator assembly may also include actuator components (e.g., motor-driven cables, lead screws, pinions, etc.; linear motors; and the like) providing movement along the longitudinal axis (forward and backward translation) of the instrument assembly 2302. Figure 9 As shown in the example, actuator mechanism 2312 includes a linear actuator 2312b, actuator mechanism 2314 includes a linear actuator 2314b, and actuator mechanism 2316 includes a linear actuator 2316b, so that each of the instrument 2306, the primary guide tube 2308, and the secondary guide tube 2310 can be independently and coaxially moved. Figure 9 As further shown, actuator assembly 2316 is passively or actively mounted to assembly arm 2318 as described above. In an active mounting system, active mounting can be used to control the DOF of one or more components (e.g., insertion of a rigid guide tube).

[0115] Control signals from control system 2320 control multiple servo motor actuators in actuator assembly 2304. These control signals are associated, for example, with a surgeon's primary input to input / output system 2322 to move the mechanically driven parts of instrument assembly 2302. Furthermore, various feedback signals from sensors in actuator assembly 2304, and / or instrument assembly 2302, and / or other components are transmitted to control system 2320. These feedback signals can generate information such as servo motor position or other position, orientation, and force information, which can be obtained, for example, by using fiber Bragg grating-based sensors. The feedback signals may also include force-sensing information, such as tissue reaction forces on input / output system 2322 that are visual or tactile outputs to the surgeon.

[0116] Image data from the endoscopic imaging system associated with instrument assembly 2302 is transmitted to image processing system 2324. This image data may include, for example, stereoscopic image data processed by input / output system 2322 and output to the surgeon, as shown. Image processing can also be used to determine instrument position, which is input to the control system as a remote position feedback sensor. Additionally, an optional sensing system 2326, placed outside and near the patient, can sense position or other data associated with instrument assembly 2302. Sensing system 2326 may be stationary or controllable by control system 2320 (actuators not shown and may be similar to those described or similar to known mechanical servo components), and it may include one or more physical sensors placed near the patient. Position information from sensing system 2326 (e.g., from one or more wireless transmitters, RFID chips, etc.) and other data may be sent to control system 2320. If such position information or other data is visually output to the surgeon, control system 2320 transmits it, either unprocessed or processed, to image processing system 2324 integrated with the surgeon's output display on input / output system 2322. Furthermore, any image data from sensing system 2326, such as fluorescence microscopy imaging or other real-time imaging (ultrasound, X-ray, MRI, and similar data), is fed to image processing system 2324 integrated with the surgeon's display. The real-time images from sensing system 2326 can be integrated with preoperative images obtained via image processing system 2324 integrated with the surgeon's display. In this way, for example, preoperative images of some tissues (e.g., brain structures) received from data storage location 2328 can be enhanced for better visibility, the preoperative images can be recorded with other tissue markers from the real-time images, and the combined preoperative and real-time images can be used with positional information from instrument and actuator assemblies 2302, 2304 and / or sensing system 2326 to present an output display that assists the surgeon in maneuvering instrument assembly 2302 toward the surgical site without damaging intermediate tissue structures.

[0117] Figure 10 This is a perspective view of the proximal portion of the minimally invasive surgical instrument 2402. Figure 10As shown, the instrument 2402 includes a transmission mechanism 2404 connected to the proximal end of the instrument body tube 2406. Components on the distal end 2408 of the body tube 2406 are omitted for clarity and may include, for example, two DOF parallel motion mechanisms, a wrist joint and end-actuator assembly as described above; a connector and endoscopic imaging system as described above; etc. In the illustrated embodiment, the transmission mechanism 2404 includes six interface disks 2410. One or more disks 2410 are associated with a DOF of the instrument 240. For example, one disk may be associated with an instrument body roll DOF, and a second disk may be associated with an end-actuator clamp DOF. As shown, in one case, the disks are arranged in a hexagonal grid for compactness—in this case, the six disks are in a triangular shape. Other grid patterns or more arbitrary arrangements may be used. Mechanical components within the transmission mechanism 2404 (e.g., gears, levers, universal joints, cables, etc.) transmit the rolling torque on the disk 2410 to, for example, the main tube 2406 (for rolling) and components connected to the remote mechanism. Cables and / or cable-and-tube combinations controlling the remote DOF travel through the main tube 2406. In one case, the main tube has a diameter of approximately 7 mm, and in another, it has a diameter of approximately 5 mm. When mating with the associated actuator disk, centrifugally spaced protruding pins 2412 provide the appropriate orientation of the disk 2410. One or more electronic interface connectors 2414 provide an electronic interface between the device 2402 and its associated actuator mechanism. In some cases, the device 2402 may transmit information stored in a semiconductor memory integrated circuit to the control system via its associated actuator mechanism. This transmitted information may include device type identification, the number of times the device has been used, etc. In some cases, the control system may update the stored information (e.g., record the number of times used to determine routine maintenance schedules or prevent the device from being used after a predetermined number of uses). U.S. Patent No. 6,866,671 (Tierney et al.) discussing the storage of information on the instrument is incorporated herein by reference. The electronic interface may also include, for example, a power supply for an electrocautery actuator. Optionally, such a power connection may be placed elsewhere on the instrument 2402 (e.g., on the housing of the drive mechanism 2404). Other connectors may be included for, for example, fiber lasers, fiber optic distal bending or force sensors, flushing, suction, etc. As shown, the housing of the drive mechanism 2404 is generally wedge-shaped or disc-shaped to allow it to be placed close to similar housings, as described below.

[0118] Figure 11A perspective view of a portion of the actuator assembly 2420 for mates with and actuates components of surgical instrument 2402. Actuator discs 2422 are arranged to mate with interface discs 2410. Holes 2424 in the discs 2422 are aligned with receiving pins 2412 in a single 360-degree orientation only. Each disc 2422 is rotated by an associated rotary servo motor actuator 2426, which receives servo control input as described above. A generally wedge-shaped mounting bracket 2428, shaped to correspond to the transmission housing of instrument 2402, supports the discs 2422, the servo motor actuators 2426, and the electronic interface 2430 of the interface connector 2414 that mates with instrument 2402. In one case, instrument 2402 is held against actuator assembly 2420 by spring clips (not shown) to allow for easy removal. Figure 11 As shown, portion 2432 of the actuator assembly housing 2428 is truncated to allow passage of the instrument body tube 2406. Optionally, an opening may be placed in the actuator assembly to allow passage of the body tube. Sterilized spacers (reusable or disposable; typically plastic) may be used to separate the actuator assembly and the instrument's drive mechanism to maintain a sterile surgical environment. Sterile thin plastic sheets or "drapes" (e.g., 0.002-inch thick polyethylene) are used to cover portions of the actuator assembly not covered by the spacers, as well as portions of the manipulator arm. Such spacers and drapes are discussed in U.S. Patent No. 6,866,671, which is incorporated herein by reference.

[0119] Figure 12 A perspective view illustrating aspects of mounting minimally invasive surgical instruments and their associated actuator assemblies on the end of an assembly / manipulation arm. (See diagram.) Figure 12 As shown, surgical instrument 2502a is mounted on actuator assembly 2504 so that the transmission mechanism engages with the actuator assembly as described above (optional spacer / curtain not shown). The body tube 2506 of instrument 2502a extends through actuator assembly 2504 and enters an opening in rigid guide tube 2508. As described, although substantially rigid, reference... Figure 5As discussed above, the main body tube 2506 is slightly bent between the drive mechanism housing and the guide tube. This bend allows the holes in the instrument main body tubes entering the guide device to be closer together than the dimensions of their drive mechanisms would otherwise allow. Because the bend angle in the rigid instrument main body tube is smaller than that in a flexible (e.g., relaxed) instrument body, the cable can be more rigid than in a flexible body. High cable stiffness is important because the number of distal DOFs is controlled within the instrument. Similarly, a rigid instrument body is easier to insert into the guide tube compared to a flexible body. In one embodiment, the bend is elastic so that the main body tube retains its straight shape when the instrument is withdrawn from the guide tube (the main body tube can form a permanent bend that will prevent the instrument body from rolling). The actuator assembly 2504 is mounted to a linear actuator 2510 (e.g., a servo-controlled lead screw and nut or ball screw and nut assembly) that controls the insertion of the main body tube 2506 into the guide tube 2508. The second instrument 2502b is mounted with a similar mechanism as shown. Additionally, an imaging system (not shown) can be mounted similarly.

[0120] Figure 12 The guide tube 2508 is further shown to be removably mounted to the support platform 2512. This mounting can be, for example, similar to that used to hold the sleeve in place. Mounting on the surgical system manipulator arm. Removable and replaceable guide tubes allow for the design of the same remote manipulator system using different guide tubes for different procedures (e.g., guide tubes with different cross-sectional shapes or multiple numbers and shapes of working and auxiliary channels). Furthermore, the actuator platform 2512 is mounted to the robot manipulator arm 2514 (e.g., 4 DOF) using one or more additional actuator mechanisms (e.g., for pitch, deflection, roll, insertion). The manipulator arm 2514 can then be mounted to a passive assembly arm, as referenced above. Figure 1 As mentioned above.

[0121] Figure 13 To illustrate from different perspectives and with reference to patients, such as Figure 12 A perspective view illustrating the aspects shown. Figure 13 In the middle, arm 2514 and platform 2512 are placed so that guide tube 2508 enters the patient's abdomen through the umbilicus. This entry is an example of various natural orifice and incision entry, including percutaneous and transcavitary (e.g., transgastric, transcolonic, transrectal, transvaginal, transrectourethrical (Douglas pouch) etc.) incisions. Figure 13This also illustrates how the linear actuators of each instrument / imaging system operate independently by showing the inserted imaging system 2518 and the withdrawn instruments 2502a, 2502b. These aspects can be applied to other surgical instrument assemblies described herein (e.g., flexible guide tubes with end or side exits, side-working tools, etc.). It can be seen that in some cases, the manipulator arm moves to rotate the guide tube 2508 about a remote center 2520 at the inlet into the patient. However, if intermediate tissue restricts movement about the remote center, the arm can hold the guide tube 2508 in place.

[0122] Figure 14 A schematic view illustrating aspects of the transmission mechanism associated with the flexible coaxial guide tube and the instrument. Figure 14 A main guide tube 2702 is shown, coaxially traveling through and exiting the distal end of the secondary guide tube 2704. Similarly, the secondary guide tube 2704 coaxially travels through and exits the distal end of the tertiary guide tube 2706. A drive and actuator mechanism 2708 is associated with the tertiary guide tube 2706. A drive and actuator mechanism 2710 is associated with the secondary guide tube 2704, and the proximal section of the guide tube 2704 extends through the drive and actuator mechanism 2710 (optionally, adjacent to it) before entering the tertiary guide tube 2706. Similarly, a drive and actuator mechanism 2712 is associated with the main guide tube 2702, and the proximal section of the guide tube 2702 extends through the drive and actuator mechanisms 2708, 2710 (optionally, adjacent to them) before entering the secondary and tertiary guide tubes 2704, 2706. The drive mechanisms of instruments and imaging systems (not shown) that travel through and exit the distal end of channel 2714 in the main guide tube 2702 can similarly be arranged (stacked) along the longitudinal axis of the instrument assembly, or they can be arranged around the extended longitudinal axis of the guide tube 2702 on its proximal end as described above. Alternatively, controller positions can be combined and arranged in parallel, such as for side exit assemblies in which drive mechanisms of side exit components are placed in parallel, and both are arranged behind the guide tube drive mechanism. Intermediate exit assemblies can be similarly configured. Instrument and / or imaging system actuators and controls can also be combined within the same housing as actuators and drive mechanisms of the guide tube.

[0123] In many respects, the device described herein is used as a single-port device—all components necessary to complete the surgical procedure enter the body through a single inlet. However, in some respects, multiple devices and ports may be used. Figure 15This is a schematic view illustrating the multi-port aspect when three surgical instrument assemblies enter the body through three different openings. Instrument assembly 2802 includes a main guide tube, a secondary guide tube, and two instruments, along with associated transmission and actuator mechanisms, as described above. In this illustrative example, instrument assembly 2804 includes a main guide tube, a secondary guide tube, and a single instrument, along with associated transmission and actuator mechanisms, as described above. Imaging system assembly 2806 includes a guide tube and an imaging system, along with associated transmission and actuator mechanisms, as described above. Each of these mechanisms 2802, 2804, and 2806 enters the body 2808 through a separate, unique opening as shown. The illustrated device is an example of the various rigid and flexible aspects described herein.

[0124] Figure 16 Another diagrammatic view to illustrate the multi-port aspect. Figure 16 The illustration shows three illustrative instruments or components 2810 entering different natural orifices (nostrils, mouth) and then continuing through a single body cavity (pharynx) to the surgical site.

[0125] Figure 17-19 This is a schematic plan view illustrating aspects of preventing unintended instrument-tissue collisions. Instruments may collide with patient tissue outside the imaging system's field of view within a space limited by patient anatomy (e.g., laryngeal surgery). Such collisions can damage the tissue. With multi-DOF surgical instruments, some DOFs may be within the field of view, while others, more proximal DOFs, may be outside. Therefore, when these proximal DOFs move, the surgeon may not be aware of the tissue damage occurring. Figure 17 As shown, for example, the endoscopic imaging system 2920 extends from the end of the guide tube 2922. The left working instrument 2924a is positioned such that all points of view (DOF) are within the field of view 2926 (bounded by the dashed line) of the imaging system 2920. However, even though the end actuator of instrument 2924b is within the field of view 2926, the right working instrument 2924b has a proximal DOF located outside the field of view 2926 (illustrating parallel motion mechanism and wrist joint as described above). This instrument positioning is exemplified for tasks such as suture ligation.

[0126] On one hand, when manufacturing a camera, the field of view boundary can be determined so that this boundary is known relative to the camera head (image capturing component). This boundary information is then stored in non-volatile memory associated with the imaging system incorporated into the camera head. Therefore, the control system can use the kinematic and connector position information of the imaging system apparatus to position the camera head relative to the working apparatus, and thus the control system can determine the field of view boundary relative to the working apparatus. In this way, the apparatus is controlled to operate within the boundary.

[0127] In another aspect of stereoscopic imaging systems, the field of view boundary can be determined relative to the instrument using machine imaging algorithms to identify the instrument and its position within the field of view. This subject matter of “tool tracking” is disclosed in U.S. Patent Application Publication No. US2006 / 0258938 A1 (Hoffman et al.), which is incorporated herein by reference.

[0128] like Figure 18 As shown, the imaging system 2920 is positioned such that the camera head is positioned precisely at the distal end of the guide tube 2922. Instruments 2924a and 2924b extend from the distal end of the guide tube and are located within the field of view of the imaging system 2920. The "allowable volume" is defined as being aligned with the boundary of the field of view. The control system prevents any part of instruments 2924a and 2924b from moving outside the allowable volume. Because the surgeon can see all distal moving portions of instruments 2924a and 2924b, the surgeon can then move the instruments without colliding with surrounding tissue. The movement of the instruments is recorded, and the "instrument volume" 2928 (bounded by dotted lines) is determined, extending through the furthest point of instrument movement. The instrument volume is a convex volume within which the instruments can be moved without colliding with tissue.

[0129] Next, as Figure 19 The imaging system 2920 is inserted as shown. Therefore, a field of view 2926 is also inserted, and a portion of instruments 2924a and 2924b is located outside the inserted field of view 2926. The new permissible volume is determined as the newly inserted field of view plus the previously determined volume of instruments located outside the field of view. Therefore, the control system will allow the surgeon to move the instruments anywhere within the new permissible volume. This process can be repeated for further insertion of fields of view or movement of the guide tube 2922. This scheme allows the surgeon to define the permissible range of instrument movement in real time without the need for a tissue model. The surgeon only needs to track the boundaries of the range of movement of the instruments within the field of view, and the control system records this information when the field of view changes.

[0130] Another way to prevent unwanted instrument / tissue collisions is to use image mosaicking. Figure 20 A graphical view of the monitor (e.g., stereoscopic) that the surgeon sees during a surgical procedure. Figure 20As shown, an image from the new, more inserted field of view 2940 (bounded by a dashed line) is recorded and mosaicked with an image from the older, more withdrawn field of view 2942. Image mosaicking is known (see, for example, U.S. Patent Nos. 4,673,988 (Jansson et al.) and 5,999,662 (Burt et al.), which are incorporated herein by reference) and has been applied to medical devices (see, for example, U.S. Patent No. 7,194,118 (Harris et al.), which is incorporated herein by reference). Therefore, the surgeon sees a larger area than the current more inserted field of view. A kinematically accurate illustrative simulation of the instrument is shown in the older field of view 2942 so that the surgeon can see possible collisions in that area as the instrument moves.

[0131] Figure 21 This is a schematic perspective view illustrating aspects of a minimally invasive surgical instrument assembly containing a multi-joint instrument designed for traction. (See diagram for reference.) Figure 21 As shown, the guide tube 3102 includes a channel 3104 through which an imaging system is inserted, and three channels 3106a, 3106b, and 3106c through which surgical instruments can be inserted. A traction device 3108 is shown extending through channel 3106c.

[0132] As depicted, the traction device 3108 comprises a proximal device body 3110 and four series of links 3112a-d. Four connectors 3114a-d connect the proximal device body 3110 and the links 3112a-d together. In one aspect, each connector 3114a-d is an independently controllable single-DOF pitch connector. In other aspects, the connectors may have additional DOF. An actively controlled (hand- or remotely operated) gripper 3116 is mounted on the distal end of the distalmost link 3112d via a passive roll connector 3118. In some aspects, other end actuators or no end actuators may replace the gripper. In one aspect, the combined length of the links 3112a-d and the gripper 3116 is sufficient to traction tissue beyond the working envelope of the device extending through channels 3106a and 3106b. For example, the combined length of the links and gripper may be approximately equal to the entire insertion range of the device (e.g., approximately 5 inches). Four links and connectors are shown, but other numbers of links and connectors can be used. Traction is performed using different combinations of pitch connectors 3114a-d and roll mechanism 3108 within channel 3106c.

[0133] For traction, instrument 3108 is inserted so that each connector 3114a-d is exposed one after another. The insertion depth can be varied so that traction can begin at different distances from the distal end of the guide tube with different numbers of connectors as the connectors emerge from the distal end of the guide tube. That is, for example, traction can begin as soon as connector 3114d is inserted past the distal end of the guide tube. For traction, clamp 3116 can clamp the tissue. As instrument 3108 rolls within channel 3106c, passively rolling connector 3118 prevents the clamped tissue from being twisted. In one aspect, the control system connects the movement of instrument 3108 and guide tube 3102. This linked motion control allows the tissue to be held in the correct position by clamp 3116 as the guide tube moves left and right "below" the traction tissue. For example, when the distal end of guide tube 3102 is moved to the left, rolling instrument 3108 (and changing the pitch of connectors 3114a-d) moves clamp 3116 to the right.

[0134] Figure 21 This further clarifies aspects of instrument positioning and control within the guide tube. Working surgical instruments do not necessarily need to be inserted through or aligned with the guide tube channel corresponding to their working position. For example, as... Figure 31 As shown, the left-side instrument does not need to be inserted through the leftmost channel 3106c. Instead, the left-side instrument can be inserted through the "bottom" channel 3106b. The right-side instrument can then be inserted through the rightmost channel 3106a. Subsequently, the left and right instruments can be controlled to work at the surgical site, aligned with the field of view of the imaging system inserted through the channel 3104, which has not been rolled or deviated. In other words, the left-right axis between the insertion channels of the instruments does not need to be aligned with the left-right axis between the actuators at the instrument ends at the surgical site, or with the left-right axis between the pupils of the stereoscopic imaging system. Furthermore, the left-right instrument positions can be changed by the control system identifying which instrument is connected to each specific actuator. For example, the traction instrument 3108 can be inserted through channel 3106a, the right-side instrument through channel 3106b, and the left-side instrument through channel 3106c. In some aspects, utilizing appropriately shaped channels and / or imaging systems, the imaging system can be inserted through one of several channels. For example, the "top" channel 3104 and the "bottom" channel 3106b can be rectangular, with a central hole for fixing a cylindrical instrument body. Therefore, the imaging system can be inserted through the "top" or "bottom" channel, and the working instrument can be inserted through other "top" or "bottom" channels.

[0135] Figure 22This is a block diagram of the components of input devices 203, 204 used to control and selectively associate medical devices on the patient-side support system 2104 with the operator-operated control console 2102 of the surgeon. Various surgical tools, such as grippers, cutters, and needles, can be used to perform medical procedures at the work site within the patient's body. In this example, three surgical tools (tool 1, tool 2, tool 3) 2231, 2241, 2251 are used to robotically perform the procedure, and an imaging system (IS) 2261 is used to view the procedure. Tools 2231, 2241, 2251 and the imaging system 2261 can be placed in a guide tube (GT) 2271 so that it can extend beyond the distal end of the guide tube 2271. The guide tube 2271 can be inserted into the patient through an access port such as a minimally invasive incision or a natural orifice, using the assembly portion of a robotic arm assembly, and is motorized by the guide tube manipulator 2272 toward the work site where the medical procedure is performed.

[0136] Each of devices 2231, 2241, 2251, 2261, and 2271 is operated by its own manipulator. Specifically, imaging system 2261 is operated by imaging system manipulator (PSM4) 2262, first surgical tool 2231 is operated by first tool manipulator (PSM1) 2232, second surgical tool 2241 is operated by second tool manipulator (PSM2) 2242, third surgical tool 2251 is operated by third tool manipulator (PSM3) 2252, and guide tube 2271 is operated by guide tube manipulator 2272.

[0137] Each of the instrument manipulators 2232, 2242, 2252, and 2262 is a mechanical assembly carrying an actuator and providing a sterile mechanical interface to transmit motion to its respective pivoted instrument. Each instrument 2231, 2241, 2251, and 2261 is a mechanical assembly that receives motion from its manipulator and transmits that motion via a cable drive to its distal pivot (e.g., a connector). Such connectors may be prismatic (e.g., linear motion) or rotary (e.g., they rotate about a mechanical axis pivot). Furthermore, the instruments may have internal mechanical constraints (e.g., cables, gears, cams, belts, etc.) that force multiple connectors to move together in a predetermined manner. Each set of mechanically constrained connectors performs a specific axial motion, and the constraints may be designed to pair the rotary connectors (e.g., engaging connectors). Also note that in this way, the instruments may have more connectors than are available actuators.

[0138] In direct control mode, each of the input devices 203, 204 can be selectively associated with one of the devices 2261, 2231, 2241, 2251, 2271 via a multiplexer (MUX) 2290, so that the associated device can be controlled by the input device via its controller and manipulator. For example, a surgeon can assign associations to the left and right input devices 203, 204 via a graphical user interface (GUI) 2291 on the surgeon's console 2102 to associate them with the first and second surgical tools 2231, 2241, respectively, which are controlled in a remote-controlled robotic manner via their respective controllers 2233, 2243 and manipulators 2232, 2242, so that the surgeon can perform medical procedures on the patient while the surgical tool 2251, imaging system 2261, and guide tube 2271 are each controlled via their respective controllers (such as in...). Figure 24 , 25 (As shown) the soft lock is in the correct position. If the surgeon wishes to control the movement of the surgical instrument 2251 using one of the input devices 203, 204, the surgeon can do so by simply deassociating the input device to the instrument 2251 without associating it with the currently associated device. Similarly, if the surgeon wishes to control the movement of the imaging system 2261 or the guide tube 2271 using one or both of the input devices 203, 204, the surgeon can do so by simply deassociating the input device to the imaging system 2261 or the guide tube 2271 without associating it with the currently associated device.

[0139] As an alternative to using GUI 2291 to provide MUX 2290 with selectable input SEL, the selective association of input devices 203, 204 with devices 2251, 2241, 2231, 2261, 2271 can be implemented by the surgeon using voice commands understood by a voice recognition system, or by the surgeon pressing a button on one of input devices 203, 204, or by the surgeon pressing a foot pedal on the surgeon's console 2102, or by the surgeon using any other well-known mode-switching technique. Although such mode-switching is described herein as being implemented by the surgeon, it can optionally be implemented by an assistant under the surgeon's guidance.

[0140] Each of controllers 2233, 2243, 2253, 2263, and 2273 includes a master / slave control system. As an example, Figure 23A block diagram illustrating a master / slave control system 300 is provided. When associated with input device 203, the master / slave control system 300 controls the movement of the tool slave manipulator 2232 and thus controls the position and orientation of its attached tool 2231, as instructed by the surgeon to move the master manipulator 203. A similar master / slave control system may be provided for each of the other slave manipulators (e.g., 2241, 2251, 2261, 2271) in system 2100.

[0141] Both the master and slave manipulators include a number of connection systems that connect to the connectors to facilitate multi-degree-of-freedom movement. When the surgeon moves the master manipulator 203 from one position to another during a surgical procedure, sensors associated with the master manipulator connector provide information indicating such directed movement in the master connector space, and sensors associated with the slave manipulator connector provide information indicating the movement of the slave manipulator and thus the tool 2231 in the slave connector space, for feedback purposes.

[0142] The main input processing unit 301 receives main connector position information from the main connector sensor in the main manipulator 203, which is sampled at the control system processing rate (e.g., 1300 Hz in this example), and calculates the connector velocity from the sensed connector position. The main forward kinematics processing unit 302 receives the main connector position and velocity from the main input processing unit 301, and uses, for example, the Jacobian matrix and eye-related information determined and provided in block 303 respectively, to transform them from the main connector space into the corresponding position and velocity of the main frame (i.e., the frame associated with the main manipulator 203) in Cartesian space relative to the eye reference frame (i.e., the reference frame associated with the surgeon's eye position).

[0143] The scaling and compensation processing unit 304 receives Cartesian position and velocity commands from the main forward kinematics processing unit 302, scales the movement of the commands according to the selected scaling factor for the surgical procedure, and considers compensation to produce the desired position and velocity of the driven tool frame (i.e., the frame associated with tool 2231). For the sake of brevity, Cartesian position is interpreted as including Cartesian orientation where appropriate in this specification, and Cartesian velocity is interpreted as including translational velocity and angular velocity where appropriate. Scaling adjustment is useful where a small movement of the driven tool 2232 is desired relative to a large movement of the master manipulator 203, so as to allow for more precise movement of the driven tool 2231 at the surgical site. On the other hand, compensation is performed relative to the position and orientation of the master frame in the eye reference frame to determine the corresponding position and / or orientation of the end-actuator frame (e.g., the frame associated with the end-actuator of tool 2231) in, for example, a camera reference frame (i.e., the reference frame associated with the image capture end of the imaging system).

[0144] The simulated follower processing unit 308 (also referred to as the "simulation domain") receives the desired follower tool holder position and speed commands from the scaling and compensation processing unit 304, and limits the desired follower tool holder position, orientation, and speed to specified Cartesian limits, for example, to ensure correct and intuitive operation of tool 2231 by keeping tool 2231 within its sensitive working area, and to prevent movement that would result in excessive force applied through the end effector of tool 2231. The simulated follower processing unit 308 generates simulated follower joint position and speed corresponding to the limited follower tool holder position and speed, while ensuring that the generated follower joint position and speed do not exceed the actual range of motion and maximum speed of the follower joint (i.e., joint limits), even near the kinematic singularity of the follower kinematics.

[0145] The reverse scaling and compensation processing unit 306 receives analog joint position and speed commands from the analog slave processing unit 308 and performs reverse functions on the scaling and compensation processing unit 304 thereon. The Cartesian controller 307 receives the input to the scaling and compensation processing unit 304 as a first input and the output of the reverse scaling and compensation processing unit 306 as a second input. The Cartesian controller 307 then generates an error signal as the difference between the first and second inputs, and generates a Cartesian force "F" from the error signal using a formula such as the following. CART ":

[0146]

[0147] Where "K" is the spring constant and "B" is the damping constant. "Δx" represents the difference between the Cartesian velocity inputs of the Cartesian controller 307 and the Cartesian position inputs of the Cartesian controller 307. For orientation errors, the corresponding torque in the Cartesian space is determined.

[0148] The main transpose kinematics processing unit 315 receives the Cartesian force F through the summation node 314. CART The corresponding torque is generated in the joint space using, for example, the Jacobian transpose matrix and kinematic relationships associated with the master manipulator 203. The master output processing unit 316 receives the master torque signal from the master transpose kinematic processing unit 315, generates a current corresponding to the master torque signal, and supplies the current to the corresponding master joint motor of the master manipulator 203. Therefore, whenever the surgeon's command exceeds the system Cartesian or driven joint limit or generates a position or velocity that produces a kinematic singularity condition from the manipulator 2232, the surgeon operating the master manipulator 203 senses a Cartesian force F. CART .

[0149] When the main input processing unit 301 receives the main connector position from the sensor in the main controller 203, the driven input processing unit 309 also receives the driven connector position from the position sensor in the driven controller 2232 at the control system processing rate. The connector control unit 320 receives the driven connector position from the driven input processing unit 309 and the simulated connector position command provided by the simulated driven processing unit 308, and generates the driven torque command signal of the driven connector engine and the main torque feedback command signal of the main connector engine.

[0150] The driven torque command signal is generated by the joint control unit 320 to drive the joint of the slave actuator until the feedback error calculated in the joint control unit 320 is zero. The driven output processing unit 310 receives the driven torque command signal from the joint control unit 320, converts it into an appropriate current, and supplies current to the joint motor of the slave actuator so as to drive the motor.

[0151] The primary torque feedback command signal is generated by the connector control unit 320 as a function of the driven connector position and speed tracking error, so as to reflect the forces applied to the tool 2231 or its slave manipulator 2232 to the primary manipulator 203 so that they can be sensed by the surgeon. The kinematic drawing unit 311 receives the primary torque feedback command signal from the connector control unit 320 and generates a corresponding Cartesian force, which is applied to the end of the tool 2231 relative to the camera mount of the imaging system using the driven kinematic configuration provided in block 312 and the previously calculated driven reference mount position information.

[0152] Gain 313 adjusts the magnitude of the Cartesian force to ensure system stability while providing sufficient force sensing to the surgeon. The gain-adjusted Cartesian force is then processed through summing node 314 and along with the Cartesian force provided by Cartesian controller 307 via main transposed kinematics processing unit 315 and main output processing 316, as previously described regarding the processing of the Cartesian force provided by Cartesian controller 307.

[0153] Additional details concerning the conventional aspects of the master / slave control system 300 are described, such as the calculation of surgeon eye-related information provided in block 303 and slave reference frame information provided in block 312, based on well-known mathematics, for example, in previously cited and incorporated by reference, U.S. Patent No. 6,424,885, “Camera Referenced Control in a Minimally Invasive Surgical Apparatus,” in which the concept of reference frame is referred to as “slave fulcrum.”

[0154] The joint control unit 320 includes joint controllers for each active joint and gear of the slave manipulator 2232, which is controlled by the master / slave control system 300. Specifically, in cases where the slave manipulator 2232 comprises multiple joints to move the tool 2231 through its operable work area, each of these joints will have its own controller. For the sake of simplicity in the description herein and in the claims, the term "joint" is understood as a connection (translation or rotation) between two links and may include gears (or prismatic joints) and any other controllable components connected to a linear drive mechanism that can be used to control the robot arm assembly.

[0155] Direct control mode is the control mode in which the user directly controls a specific slave manipulator. All other slave manipulators (i.e., slave manipulators not connected to the master manipulator) are soft-locked (i.e., all their connectors are held in the correct position by their respective controllers). As an example, in a single-port system such as those described herein, three direct control modes are defined as: a direct “tool tracking” mode, where two hand-operable input devices are associated with two tool slave manipulators and their respective tools; a direct “imaging system” mode, where one or two hand-operable input devices are associated with an imaging system; and a direct “guide tube” mode, where one or two hand-operable input devices are associated with a guide tube. For example, Figure 24-25 This illustrates a direct "tool tracking" mode, in which the left and right main input devices 204 and 203 are associated with the first and second tools, respectively, while the third tool, imaging system, and guide tube are kept in the correct position by their respective controllers; Figure 26-27 This describes a direct "imaging system" mode, in which the left main input device 204 is associated with the imaging system, while the first tool, second tool, third tool, and guide tube are held in the correct position via their respective controllers; and Figures 28-29 The direct “guide tube” mode is described, in which the left and right main input devices 204, 203 are associated with the guide tube, while the first tool, second tool, third tool and imaging system are kept in the correct position by their respective controllers.

[0156] like Figure 24 , 26As shown in Figure 28, the data pick-off / receive points (located at the input of the reverse scaling and compensation block 306 and the output of the scaling and compensation block 304, respectively, in the master / slave control system executed in the associated device controller) can be used to provide state information of instructions to the non-associated controller for linked control modes and to receive state information returned from the non-associated controller, as described herein. For simplicity of the figures, both the reverse scaling and compensation block 306 and the scaling and compensation block 304 are encapsulated in a single block diagram named “Scaling and Compensation”. Although the data pick-off / receive points located at the input to the reverse scaling and compensation block 306 and the output to the scaling and compensation block 304, respectively, are used in these examples, it should be understood that other data pick-off and receive points can be used to practice various aspects of the invention.

[0157] For the sake of simplicity, the master / slave control system 300 has been divided into a master-side part and a slave-side part (on opposite sides of the "scaling and compensation" blocks). The PSM1* controller 248, PSM2* controller 247, PSM4* controller 268, and GT* controller 288 include slave-side components (e.g., Figure 23 The control system 300 blocks 308, 320, 309, 310, 311, 312, 313), and the MTM controllers 241, 242, 262, 281, 282 include main-side components (e.g., Figure 23 The control system consists of blocks 301, 302, 303, 307, 314, 315, and 316. Figure 25 , 27 The position-keeping blocks 251, 252, 253, 271, and 272 in 29 indicate status instructions (each indicating the constant position and orientation of its respective device), which are stored in one or more storage devices and provided to the slave-side PSM3* controller 258, GT* controller 288, PSM4* controller 268, PSM1* controller 248, and PSM2* controller 247, respectively, while tracking data generated in these controllers is ignored (or otherwise discarded), as indicated by the downward dotted arrows from these controllers, so that their respective manipulators and devices remain in the state of the instructions.

[0158] In the linked control mode, the surgeon directly controls the movement of associated manipulators (e.g., one of manipulators 2232, 2242, 2252, 2262, 2272) while indirectly controlling the movement of one or more unassociated manipulators in response to direct manipulator movement commands to achieve secondary objectives. Examples of secondary objectives include optimizing the device work area (i.e., maximizing their range of motion), optimizing the view of the imaging system for other devices and / or work positions, minimizing the chance of collisions between devices and / or the patient's anatomy, and driving unassociated devices to the desired posture. By automating the implementation of secondary tasks via the linked control mode, the system's usability is enhanced by reducing the need for the surgeon to switch to another direct mode to manually achieve the desired secondary objectives. Thus, the linked control mode allows the surgeon to focus more on performing medical procedures and less on managing the system. As described below, the user interface has three linked control modes: a mode for instruments (one or more), a mode for the imaging system, and a mode for the guide tube (i.e., as many modes as the number of manipulators designed to perform different functions within the surgical system).

[0159] Providing tactile cues to surgeons to indicate when to move a connected manipulator is useful, as surgeons might otherwise be unaware of the movement of any device indirectly controlled via a linkage control mode. This is not a problem for directly controlled devices, as the master / slave control system of such devices typically provides tactile feedback pathways. Therefore, tactile cues such as detents can provide this signal to the surgeon when a linkage mode is initiated.

[0160] The GUI 2291, which specifies the associations between the input devices 203, 204 and devices 2231, 2241, 2251, 2261, and 2271 used by the surgeon, can also be used by the surgeon to specify various parameters of the linkage control modes. For example, the surgeon can use GUI 2291 to select which device manipulators participate in different linkage control modes and to define secondary objectives associated with the linkage control modes and / or prioritize them.

[0161] Figure 30 This is a schematic view illustrating the linkage control aspect of the centralized motion control and coordination system architecture of the minimally invasive remote-controlled surgical system incorporating surgical instrument components and parts described herein. The motion coordinator system 2202 receives main input 2204, sensor input 2206, and optimization input 2208.

[0162] The main input 2204 may include the movement of the surgeon's arm, wrist, hand, and fingers on the main control mechanism. Input may also come from other movements (e.g., pressing or moving buttons, levers, switches, etc. with fingers, feet, knees, etc.) and commands (e.g., voice) to control the position and orientation of specific components or to control specific task operations (e.g., energizing the electrocautery actuator or laser, imaging system, etc.).

[0163] Sensor input 2206 may contain position information from, for example, a measured servo motor position or sensed bending information. The use of Bragg fiber gratings for position sensing is described by reference to U.S. Patent Application No. 11 / 491,384 (Larkin et al.), entitled "Robotic surgery system including position sensors using fiber Bragg gratings." Such bending sensors can be incorporated into various instruments and imaging systems described herein for use when determining the position and orientation information of components (e.g., end effector ends). Position and orientation information may also be generated by one or more sensors (e.g., fluoroscopy, MRI, ultrasound, etc.) placed outside the patient, and these sensors sense changes in the position and orientation of components within the patient in real time.

[0164] Optimization input 2208 involves secondary objectives. These can be high-level instructions, or the input can contain more detailed instructions or perceived information. An example of a high-level instruction would be a command from an intelligent controller to optimize the operating area. An example of a more detailed instruction would be directed at an imaging system to start or stop its camera for optimization. An example of a sensor input would be a signal that the operating area limit has been reached.

[0165] The motion coordinator 2202 outputs command signals to the actuator controllers and actuators (e.g., servo motors) associated with the manipulators of each remote surgical system arm. Figure 30 Examples of output signals sent to two instrument controllers 2210, imaging system controller 2212, and guide tube controller 2214 are described. Other numbers and combinations of controllers may be used. Motion coordinator 2202 determines how to utilize all system kinematics (i.e., the total degrees of freedom of the system) to achieve the secondary objective indicated by optimization input 2208.

[0166] As an example, this motion coordination system can be used to control surgical instrument assembly 1700 ( Figure 6Instrument controller 2210 is associated with instruments 1702a and 1702b, imaging system controller 2212 is associated with imaging system 1704, and guide tube controller 2214 is associated with guide tube 1708. Therefore, in some aspects, the surgeon operating the remote-controlled surgical system will simultaneously and automatically enter at least three control modes described above: instrument control mode for moving instruments, imaging system control mode for moving the imaging system, and guide tube control mode for moving the guide tube. A similar centralized system can be adapted to work with a variety of other mechanisms described herein.

[0167] Figure 31 This is a diagrammatic view illustrating aspects of the distributed motion control and coordination system architecture of a minimally invasive remote-controlled surgical system incorporating the surgical instrument components and parts described herein. Figure 31 In the illustrative aspects shown, the control and switching processor 2220 exchanges information with two master arm optimizers / controllers 2222a and 2222b, three surgical instrument optimizers / controllers 2224a, 2224b, and 2224c, an imaging system optimizer / controller 2226, and a guide tube optimizer / controller 2228. Each optimizer / controller is associated with a master or slave arm (which includes, for example, a camera (imaging system) arm, a guide tube arm, and an instrument arm) in the remote surgical system. Each of the optimizers / controllers receives arm-specific optimization targets 2230a-2230g.

[0168] The double-headed arrows between the control and transition processor 2220 and the various optimizers / controllers represent the exchange of tracking data associated with the arms of the optimizers / controllers. The tracking data contains the complete Cartesian configuration of the entire arm, including the base frame and the distal end frame. The control and transition processor 2220 sends the tracking data received from each optimizer / controller to all optimizers / controllers so that each optimizer / controller has data on the existing Cartesian configuration of all arms in the system. Additionally, each arm's optimizer / controller receives an optimization objective unique to that arm. As it seeks its optimization objective, each arm's optimizer / controller subsequently uses the other arm positions as inputs and constraints. In one aspect, each optimization controller uses an embedded local optimizer to seek its optimization objective. The optimization module for each arm's optimizer / controller can be independently turned on or off. For example, an optimization module specific to the imaging system and guide tube can be turned on.

[0169] Compared to centralized systems, distributed control systems offer greater flexibility, although there is a possibility of reduced performance. Using a distributed control system instead of a centralized one makes it easier to add new arms and change the entire system configuration. However, in this distributed system, optimization is localized compared to the comprehensive optimizations that can be implemented in a centralized system, where a single module is aware of the entire system's state.

[0170] Figures 32-34 This describes specific aspects of a linkage control mode, in which related devices are directly controlled to achieve a primary objective, while non-related devices are indirectly controlled to achieve a secondary objective. Specifically, Figure 32 This illustrates the implementation of a linked "tool tracking" mode, in which the left and right main input devices 204 and 203 are associated with the first and second tools, respectively. At the same time, by linking the blocks 3202 and 3201 connected to the data removal points of their respective master / slave control systems and the linked controllers 3204 and 3203 of the imaging system and guide tube, information for their movement instructions is made available so that they can carry out the desired "secondary target". Figure 33 An example of a linked "imaging system" mode is illustrated, in which the left master input device 204 is associated with the imaging system, and its command movement information is made available by linking the block 3302 connected to its master / slave control system data removal point with the linked controllers 3304, 3305, and 3303 of the first tool, second tool, and guide tube, so that they can implement the desired "secondary" objective; and Figure 34 An example of a linked "guide tube" mode is illustrated, in which the left and right main input devices 204, 203 are associated with the guide tube, and information for its movement is made available by linking the block 3402 connected to the data removal point of its master / slave control system with the linked controllers 3404, 3405, 3403 of the first tool, second tool, and imaging system, so that they can implement the desired "secondary" objectives. Note that in these linked mode examples, it is assumed that the third tool is not deployed to simplify the diagram.

[0171] Figures 32-34 The controllers for the interconnected blocks and devices described herein can be executed in a distributed manner, such that they are integrated into their respective controllers or executed outside their respective controllers, or they can be executed in a centralized manner, such that they are integrated into a single unit outside their respective controllers. To transform the direct "tool tracking" mode into the corresponding interconnected "tool tracking" mode, interconnected blocks 3201 and 3202 (as shown) are... Figure 32 (as shown) and data removal / receiving point (e.g.) Figure 24(As shown) are connected. Controllers 3203 and 3204, which instruct and control their respective device controllers to implement secondary objectives of device linkage, are connected at their respective device controller data removal / receiving points and to connection blocks 3202 and 3201 so that they can receive and send information back and forth, as shown by... Figure 32 The arrows indicate this. Similarly, to convert the direct "imaging system" mode to the corresponding linked "imaging system" mode, connect block 3302 (as shown in the image). Figure 33 (as shown) connected to the data removal / receiving point (e.g.) Figure 26 (As shown). Controllers 3303, 3304, and 3305, which instruct and control their respective device controllers to implement secondary objectives, are connected at their respective device controller data removal / receiving points and connected to connection block 3302 so that they can receive and send information back and forth, as shown by... Figure 33 The arrow in the diagram indicates this. Finally, to convert the direct "guide tube" mode to the corresponding linked "guide tube" mode, connect block 3402 (as shown in the diagram). Figure 34 (as shown) connected to the data removal / receiving point (e.g.) Figure 28 (As shown). Controllers 3403, 3404, and 3405, which instruct and control their respective device controllers to implement secondary objectives, are connected at their respective device controller data removal / reception points and to connection block 3402 so that they can receive and send information back and forth, as shown by... Figure 34 The arrow in the image indicates this.

[0172] Figure 35-40 The flowchart illustrates an example of a linkage control mode. As previously explained, the user interface has three linkage control modes: a mode for instruments (one or more), a mode for the imaging system, and a mode for the guide tube. Figures 35-37 Examples of control for inter-device linkage are shown in Figure 39. Figure 38 An example of control for the linkage of the guide tube, and Figure 40 This is an example of coordinated control within an imaging system. (Reference) Figure 35-40 The methods described herein, as well as the various controllers and other processing units described herein, are preferably referenced in the document. Figure 4 Executed in the described processor 220.

[0173] Figures 35-36 The first part illustrates an example of instrument linkage control, in which the working area of ​​a pivot device connected to a guide tube and extending beyond the distal end of the guide tube is optimized. Figure 6This is one example of such a device assembly. Although this example describes optimizing the working area of ​​a pivoting device for performing medical procedures using a guide tube, it should be understood that aspects of the invention are also applicable to any base connected to the pivoting device so that the pivoting device moves when the base moves. As an example, if the rotary assembly joints 2114a, 2114b are actively actuated, refer to... Figure 1-3 The patient-side support system 2104 described can also serve as such a base.

[0174] First, refer to Figure 35 In 3501, the controller 3203 (e.g., motion coordinator 2202 or guide tube optimizer / controller 2228, depending on whether a centralized or distributed linkage mode system is used), which operates in a device linkage control mode at that time, receives instructions on the device end position from the connection blocks and linkage controllers of all devices connected to the guide tube (i.e., devices that move when the guide tube moves). For example, a device may be linked to the guide tube if it is placed inside the guide tube or if it is otherwise physically attached to the guide tube. As used herein (except where clearly indicated otherwise in the context of the description), the phrase "device end position" refers to information indicating the Cartesian coordinates in a fixed reference frame representing the farthest joint of the device and the orientation determined by the angular position of the farthest joint.

[0175] In 3502, the guide tube linkage controller 3203 uses the received instructions regarding the device end position to determine the optimal guide tube end position for the working area of ​​the device connected to the guide tube, while their respective controllers maintain their device end positions. Because the optimization function requires knowledge of the motion limits and kinematic ranges of the devices, as well as the current end positions of the guide tube and the devices, this range of motion and kinematic information is preferably provided to the guide tube linkage controller 3203 at system startup or at other convenient times in a conventional manner, while the current end position of the devices is provided during operation of the device connection block and the linkage controller, as previously described. To determine the desired guide tube end position, each of the device controllers can provide its device with a desired Cartesian pose so that the control of the guide tube linkage solves the kinematics in such a way that the guide tube end is positioned to allow the configurable device connector to be as close as possible to its desired pose without moving its end from the desired end position.

[0176] Preferably, this optimization is implemented using the range of motion of the devices and a selected weighted minimization value function. For example, the weighting value can be selected to maximize the range of motion of the directly controlled instruments 2231, 2241 (i.e., with higher priority) more than any other device that maximizes the range of motion of the imaging system 2261 and keeps its end in the correct position (i.e., held or “soft-locked” in place by its controller). In 3503, the determined guide tube end position is then provided to the guide tube controller 2272 to drive the guide tube 2271 to the determined end position and to the device controllers 2233, 2243, 2263 so that they can drive their respective devices 2231, 2241, 2261 to a pivoted joint configuration that optimizes their respective working areas, as referenced below. Figure 36 Described.

[0177] Now see Figure 36 This describes complementary actions performed by the device controllers of devices linked to the guide tube. In 3601, the commanded position of the guide tube 2271 is received from the controller linked to the guide tube. In 3602, the device controllers generate updated connector position commands to their respective slave actuators to accommodate the new guide tube position while satisfying the commanded device end position. For devices 2231 and 2241 associated with input devices 204 and 203 in the device-linked control mode, the commanded device end position corresponds to the end position commanded by input devices 204 and 203. For imaging system 2261 or another device 2251 not associated with input devices 204 and 203 at this time, the commanded device end position is their current end position so that the ends of these non-associated devices are effectively held in the correct position. In 3603, the device controllers provide updated connector position commands to their respective slave actuators to optimize the device operating area.

[0178] Figure 37 An optional second part of an example is described, in which the movement of imaging system 2261 is connected to the movement of instruments 2231, 2241, so that the instruments are well positioned within the field of view of the imaging system. However, reference... Figures 35-36 The first part of the described example addresses the optimization of a secondary objective of the working area of ​​a device connected to and extending beyond the distal end of the guide tube, while the second part of the example addresses the optimization of a secondary objective of the view of the device's distal end in an image captured by the imaging system.

[0179] Now for reference Figure 37In 3701, the imaging system linkage controller 3204 (e.g., motion coordinator 2202 or imaging system optimizer / controller 2226, depending on whether a centralized or distributed linkage architecture is used), operating in instrument linkage control mode at this time, receives instructions regarding the device end-effector position from all device controllers. In 3702, the imaging system linkage controller determines the centroid of the instructed instrument end-effector position, and in 3703, it determines the centroid velocity using the difference between the centroid positions determined during current and previous digital processing. In 3704 and 3705, a vibration filter is implemented to determine the desired imaging system end-effector position and velocity by applying dead-zone behavior to the centroid position and a low-pass filter to the centroid velocity, respectively.

[0180] In 3706, the imaging system linkage controller 3204 then uses the reverse kinematics of the current end positions of the pivot imaging system 2261 and the guide tube 2271 to determine the desired connector position of the imaging system 2261. In 3707, the imaging system linkage controller uses the forward kinematics of the imaging system 2261 to determine the imaging system end position corresponding to the changed driven connector position, and provides the determined imaging system end position to the guide tube linkage controller 3203. Note that the imaging system end position determined in 3707 should be the same as the desired imaging system end position in 3704, unless a connector limit or singularity occurs in 3707—in which case they will be different to avoid a limit or singularity. The guide tube linkage controller then... Figure 35 The first part of the described example processes the imaging system end position together with the instrument end position to produce an optimized guide tube end position for the instrument and imaging system working areas. In 3708, the imaging system-linked controller 3204 receives the commanded guide tube end position from the guide tube-linked controller 3203 and uses it in 3709 to determine the commanded slave connector position by applying the imaging system end position determined in 3707 and the modified guide tube end position to the same equations and limits used in implementation 3706. In 3710, the commanded slave connector position determined by the imaging system controller 2263 is then provided as an actuator command for the imaging system manipulator 2262 to manipulate or move the imaging system 2261 accordingly.

[0181] After the medical procedure is completed, all medical devices used during the procedure should be retrieved from the patient. It is advantageous to use a synchronized control mode to retrieve all devices simultaneously, rather than one at a time using a direct control mode. Specifically, by retrieving one device under direct control, it is expected that all other devices will follow suit under synchronized control, while addressing secondary objectives such as avoiding collisions with each other and / or with the patient's anatomy during retrieval. Additionally, after each device is retrieved into its guide tube, it is necessary to first position the device in a retrieval configuration so that it can be retrieved into the guide tube. For example, Figure 21 The described traction device 3108, after each of its links 3112a-3112d is aligned with the channel 3106c, can be fully retracted into the channel 3106c of the guide tube 3102. Therefore, it is desirable to automatically drive each of the devices into its retraction configuration before it enters its guide tube. This applies to devices retracted under direct control as well as devices retracted indirectly through a linkage control mode.

[0182] Conversely, all medical devices to be used during the procedure should be inserted into the patient before the procedure is performed. It is advantageous to insert all devices simultaneously using a linkage control mode, rather than one at a time using a direct control mode. Specifically, by inserting one device under direct control, it is expected that all other devices will follow suit under linkage control, while addressing secondary objectives such as avoiding collisions with each other and / or with the patient's anatomy during insertion. Additionally, it is useful to position the devices in a configuration that optimizes their working area after they are inserted into the patient and reach their working site. It is also useful to position the working ends of the devices well within the imaging system's field of view. Therefore, once the imaging system reaches the desired observation point at the working position, it is desirable to automatically drive each of the devices into its optimal configuration.

[0183] Figure 38 This illustrates an example of using linkage control to retract a medical device into the guide tube. Although any one device can be directly controlled while others are indirectly controlled for retraction into the guide tube, this example uses actual degrees of freedom (DOF) for the guide tube manipulators used to control the retraction. Because all devices are connected to the guide tube, all devices move when the guide tube moves. However, the guide tube manipulators in this example do not have actuators for insertion / retraction; therefore, actual insertion / retraction DOF is achieved by sending a guide tube insertion / retraction command to each device controller while the guide tube is held in the correct position, causing the device to move in the desired insertion / retraction direction.

[0184] In 3801, the guide tube connecting block 3402 periodically receives routinely time-sampled output from its associated surgeon-operated input device (one or more), in which case it instructs the guide tube to retract backward along its longitudinal axis (e.g., away from the work site). In 3802, the connecting block 3402 relays the received retraction command to controllers of other devices, so that they in turn instruct their respective device manipulators to retract their respective devices from their positions in the desired retraction direction.

[0185] In 3803, each of the device controllers (i.e., not the guide tube controller) determines when the proximal end of the nearest rotating link of its respective device is within a threshold distance "TH" from the distal end of the guide tube. The threshold distance "TH" can be determined, for example, by considering the current rotation angle of the nearest rotating link, the retraction rate commanded by the surgeon on the input device, and the clearance between the "straightened out" device and the channel through which the device extends in the guide tube. Specifically, the threshold distance "TH" is selected so that each of the devices can be retracted into the guide tube without impacting the end or side of its respective channel where it is placed.

[0186] The distance between the proximal end of the nearest rotating link of the device and the distal end of the guide tube can be determined in a conventional manner by: determining a first vector extending from the remote center "RC" (i.e., the pivot point of the guide tube) to the distal end of the guide tube; determining the nearest rotating link of the device; determining a second vector extending from the remote center "RC" to the nearest end joint of the nearest rotating link of the rotating device; and determining the distance between the proximal end of the nearest rotating link of the device and the distal end of the guide tube from the difference between the first and second vectors.

[0187] In 3804, after determining that the proximal end of the nearest rotating link of its respective device is within a threshold distance "TH" from the distal end of the guide tube, each of the device controllers (i.e., the non-guide tube controllers) drives its device to a retraction configuration (i.e., a connector and link configuration that allows the device to be fully retracted into the guide tube). The rate at which the device is driven to its retraction configuration is determined at least in part by a rate at which the output of the input device changes the direction of the insertion / retraction command to avoid collision between the device and the guide tube. Additionally, when each of the device controllers drives its device to its retraction configuration, potential collisions with other devices and / or the patient are also avoided and considered. In 3805, once each device is determined by its respective device controller to be in its retraction configuration, the device controller allows its respective device to be retracted into its channel in the guide tube in response to a retraction command issued from one or more input devices associated with the guide tube at that time.

[0188] Because the image-capturing end of an imaging system is typically positioned closer to the distal end of the guide tube than an instrument, ensuring that the working end and working part of the instrument are well placed within the imaging system's field of view, the nearest rotating link of the imaging system will typically be the first rotating link of the device assembly, extending beyond the distal end of the guide tube to reach a threshold distance "TH" from the distal end when the device assembly is retracted. When the nearest rotating link of each of the other devices reaches the threshold distance "TH" from the distal end of the guide tube, its device controller drives its device to its retracted configuration.

[0189] As an alternative to the method described with reference to 3803-3804, instead of waiting until the nearest rotating link of each device reaches a threshold distance "TH" from the far end of the guide tube before the device controller begins to drive the device to its retraction configuration, each of the device controllers may immediately begin to drive its device to the retraction configuration upon receiving an instruction indicating the desired movement in the retraction direction. In this case, each device controller is configured to drive its device to its retraction configuration in such a way that any rotating link of the device is properly aligned so as to freely enter the channel of the device before it enters the channel, while avoiding injury to the patient and collisions with other devices.

[0190] When driving the imaging system to its retracted configuration, it is important to remember that the imaging system controller uses the position information of the received associated instrument's end actuator to instruct the movement of its image capture end, thereby keeping the end actuator within its field of view. Because the operator observes the image captured by the image capture end on the display screen while simultaneously moving the input device, the operator may become unoriented and / or move the input device incorrectly in order to correctly instruct the retraction of its associated instrument. To compensate for this non-intuitive experience, the calculation of the master-slave manipulator (i.e., Figure 23 The kinematic reference frame of blocks 302 and 311 (i.e. Figure 23 Blocks 303 and 312) are configured such that the position / orientation of the master manipulator relative to the display screen seen by the operator constantly corresponds to the position and orientation of the end of the associated instrument (e.g., a point on the end actuator) relative to the end of the imaging system (e.g., a point on the image capture end).

[0191] Upon receiving an insertion command (i.e., a command to move the device in a direction extending away from the distal end of the guide tube), the device controller can automatically actuate the device to a desired operating configuration. The desired operating configuration may be a preferred configuration stored in a storage device associated with one or more processors executing the various controllers and processes described herein. Alternatively, it may be a previously assumed operating configuration already stored in the storage device. As an example of the latter, the device connector position of the operating configuration of the device just before its retraction toward the guide tube begins can be stored in the storage device so that if the surgeon decides to reinsert the device (or their replacement device after a tool-changing procedure), their device controller can automatically actuate the device back to the stored operating configuration.

[0192] In some cases, surgical instruments are detachable and replaceable by different surgical instruments having similar instrumental structures but different end actuators to perform different surgical tasks. Therefore, a single guide tube can be used for one or more replaceable surgical instruments. In one case, the end actuator of the surgical instrument is detachable so that it can be easily replaced with another. In another case, surgical accessories such as clips or sutures can be provided to the clamped end actuator for delivery to the work site while the guide tube remains inside the patient. A convenient method for implementing such end actuator replacement (also referred to herein as “tool replacement”) or providing such surgical accessories to the retracted clamped end actuator is to use a perforated guide tube in which one or more incisions are provided in the guide tube in a manner that partially extends outward away from the patient, while another portion of the guide tube extends inward into the patient through an access port.

[0193] Figure 39 The following example illustrates the use of a linkage control for retracting medical devices into a perforated guide tube for tool changes or other purposes, such as delivering surgical accessories to the work site. In this example, multiple devices, including an imaging system and at least two instruments, extend through and beyond the distal end of the guide tube.

[0194] In 3901, a retraction command is received from an input device associated with the instrument to be retracted (also referred to herein as the "associated instrument"). The retraction command is indicated by movement of the input device in a direction that would cause the associated instrument to be retracted toward and / or into the distal end of the guide tube. As previously referenced Figure 23 and 32As described, the sensored connector movement of the input device is processed by the master / slave control system of the instrument, and the state of the command generated by the associated instrument's distal end is acquired on the output of the scaling and compensation block, and provided to the linkage controller block of other devices in the system via the connection block along with information identifying the associated instrument (specifically, its position in the guide tube).

[0195] In 3902, each link controller block determines whether its associated device is retracted along with the associated instrument. In the case of the link controller block for the imaging system, this decision is affirmative, so that the operator can continuously observe the working end of the associated instrument when it is retracted into the guide tube. In the case of the link controller blocks for other devices, the decision considers whether their respective instruments will obstruct access to the end actuator of the associated instrument from the opening of the guide tube for tool changing and delivery of surgical accessories. If an unretracted instrument would obstruct access to the end actuator of the associated instrument through the opening, the link controller block's decision to obstruct the instrument will also be affirmative. On the other hand, the link controller block's decision not to obstruct the instrument will be negative.

[0196] In 3903, the linked controller blocks that make a positive decision then relay the received retraction command to their respective controllers, which in turn instruct their respective device manipulators to retract their devices (referred to herein as "linked devices") from their positions in the desired retraction direction.

[0197] In 3904, each of the retractable device controllers (both associated and linked devices) determines when the proximal end of its respective device's nearest rotating link is referenced. Figure 38 The method described in 3803 is within the threshold distance "TH" from the far end of the guide tube.

[0198] In 3905, the proximal end of the rotating link of each device is used as a reference. Figure 38 As described in 3804, after reaching a threshold distance “TH” from the distal end of the guide tube (including the compensated motion imaging system as described herein), each of the device controllers subsequently instructs its respective device manipulator to drive its device to a retracted configuration (i.e., a configuration that allows the device to retract completely into the connector and linkage configuration of the guide tube).

[0199] In 3906, once each retracted device is determined to be in its retracted configuration by its respective device controller, the device controller allows its respective device to retract into its channel in the guide tube in response to a retracting command issued from the input device.

[0200] In 3907, once the operator determines that the end actuator of the associated instrument is in the correct position relative to the opening in the perforated guide tube, the movement of the input device and therefore the associated instrument is stopped. However, the imaging system may continue to move to ensure that the end actuator of the associated instrument is correctly within its field of view. Additionally, any obstructing instruments continue to move until they no longer obstruct access to the end actuator of the associated instrument through the opening in the perforated guide tube. Once the access path to the end actuator of the associated instrument is clear with the opening, the end actuator can be replaced and / or surgical accessories can be supplied to the end actuator while the imaging system observes the movement of the end actuator.

[0201] Figure 40 This illustrates an example of using linked control to extend a medical device through a guide tube and insert it into the working site. While any one device can be directly controlled while the others are indirectly controlled for insertion into the working site, this example assumes that the imaging system is directly controlled for insertion, while the instrument is indirectly controlled via linked control to follow the image capture end of the imaging system. Using an imaging system to guide insertion is advantageous because it allows the surgeon to visualize the pathway toward the working site.

[0202] For example, similar to da The remote-controlled endoscope in the surgical system allows for control during insertion—on one hand, the surgeon virtually moves the image using one or two master manipulators; she uses the master manipulators to move the image from side to side and pull it toward herself, thus instructing the imaging system and its associated instrument components (e.g., flexible guide tubes) to be directed toward a fixed center point on the output display and to advance within the patient. On the other hand, camera controls are designed to provide the master manipulators with the impression that the image is stationary so that the image moves in the same direction as the master manipulators, as in the case of a da... This is similar to a surgical system. The design positions the master manipulator correctly so that the instrument can be controlled when the surgeon leaves camera control, and therefore it avoids engaging (disengaging), moving, and separating (joining) the master manipulator back to its original position before initiating or resuming instrument control. In some aspects, the master manipulator position can be proportional to the insertion speed to avoid using a large master working area. Alternatively, the surgeon can engage and disengage the master manipulator to utilize a ratchet effect for insertion. In some aspects, insertion can be manually controlled (e.g., via a hand-operated wheel) (e.g., through the glottis when entering via the esophagus), and then automatically inserted (e.g., by a servo-driven motor driven roller) as the distal end of the surgical instrument assembly approaches the surgical site. Preoperative or real-time image data (e.g., MRI, X-ray) of the patient's anatomy and space can be used to assist insertion.

[0203] In step 4001, the imaging system controller receives an insertion command from the associated input device. In step 4002, the imaging system controller instructs the imaging system manipulators to move the imaging system in response to the insertion command. In step 4003, the imaging system controller provides movement commands to controllers of other linked devices so that they can also instruct their respective devices to move in response to the imaging system's movement command. In step 4004, the imaging system controller determines whether the imaging system's image capture end has reached its desired position. This determination can be implemented automatically based on programming standards or it can be indicated by an action taken by the surgeon, such as pressing a button on the input device associated with the imaging system at this time. In step 4005, after the imaging system controller has determined that the imaging system's image capture end has reached its desired position, it provides this indication to controllers linked to instruments (e.g., motion controller 2202 or instrument optimizer / controller 2224a, 2224b, 2224c, depending on which instrument is deployed and whether a centralized or distributed linkage mode architecture is used) so that, in response to their instrument controllers, their respective instrument manipulators instruct them to move their instruments into their optimal operating configuration. In this case, placing the devices in their optimal operating configuration typically involves positioning the working end of the instrument within the field of view of the imaging system and optimizing the instrument's working area (e.g., for example...). Figure 18 (as shown in the image).

[0204] As can be seen from the examples of the linked control modes described in this article, not all position information provided to the motion coordinator or device optimizer / controller is used. Therefore, more information than necessary is transmitted between device controllers, while some is ignored, or only the necessary information is transmitted. Although Figures 30-31 The description may refer to the former, but it should be understood that the execution method described herein can also be applied to the latter.

[0205] Furthermore, whenever the image capture end of the imaging system moves as a linked device, the image reference frame used by the surgeon for master / slave remote operation changes, and this change can affect the surgeon's ability to perform precise surgical activities. In this context, many actions can be taken in response to large movements of the imaging capture end of the imaging system. For example, haptic feedback can be provided on the input device to assist the surgeon in taking appropriate actions, or a computer-generated auxiliary view of the device extending distal to the guide tube can be provided from a stable (e.g., fixed) perspective view and relied upon by the surgeon for master / slave remote operation, or the image captured by the imaging system can be changed in real time to maintain an intuitively accurate master / slave drawing, with the changed image displayed on the surgeon's console.

[0206] Examples of different minimally invasive surgical systems, components, and instruments, as well as these descriptions of related parts, are not to be considered limiting. It should be understood that many variations are possible in conjunction with the aspects described herein. For example, different combinations of rigid and flexible instruments and instrument components, and different combinations of guide tubes and guide tube components, fall within the scope of this specification. The claims define the invention.

Claims

1. A system comprising: Input devices; A first manipulator is connected to a base and configured to move a device, wherein movement of the base causes movement of the first manipulator. A second manipulator configured to move the base; A means for commanding the second manipulator to move at least partially in response to a commanded movement of the first manipulator when the first manipulator moves in response to a movement of the input device; and A means for commanding the first manipulator to move at least partially in response to a command movement of the second manipulator that moves the base, when the second manipulator moves in response to a movement of the input device; When the second manipulator moves the base in response to a motion command from the input device, the device for commanding the first manipulator commands the first manipulator to hold the working end of the instrument in a desired position and orientation.

2. The system according to claim 1, wherein, When the first manipulator moves the instrument in response to a motion command from the input device, the device for commanding the second manipulator commands the second manipulator to move the base in order to optimize the workspace of the instrument.

3. The system according to claim 1 or 2, further comprising: The third manipulator is configured to move the imaging device; and A means for commanding the third manipulator; When the first manipulator moves the instrument in response to a motion command from the input device, the device for commanding the third manipulator commands the third manipulator to move the imaging device so as to keep the working end of the instrument within the field of view of the imaging device.

4. The system according to claim 1 or 2, further comprising: The third manipulator is configured to move the imaging device; and A means for commanding the third manipulator; When the third manipulator moves the imaging device in response to a motion command from the input device, the device for commanding the second manipulator commands the second manipulator to move the base in order to optimize the workspace of the imaging device.

5. The system according to claim 1 or 2, further comprising: The third manipulator is configured to move the imaging device; and A means for commanding the third manipulator; When the second manipulator moves the base in response to a motion command from the input device, the device for commanding the third manipulator commands the third manipulator to move the imaging device so as to keep the image capture end of the imaging device in a desired position and orientation.

Images