Reducing energy build-up in servo control
By using the combination of non-clutch mode and clutch mode in the robot manipulator arm, the command status is updated using servo loop and error threshold, the problem of energy accumulation of the manipulator arm in servo control is solved, and the stability and safety are improved, so as to avoid There is a risk of sudden movement and collision.
Patent Information
- Application Number
- CN202410979594.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-03-29
- Filing Date
- 2020-03-27
- Publication Date
- 2024-11-12
AI Technical Summary
The robot manipulator arm may accumulate energy during servo control, resulting in sudden and large-scale movement, which may cause collision risks, and the difference between the commanded state and the actual state is difficult to detect and correct in time.
Using a combination of non-clutch mode and clutch mode, the difference between the actual state of the manipulator arm and the command state is obtained through the servo loop. If the difference exceeds the error threshold, the command state is updated to reduce the difference and the connector is made in the clutch mode. Float to avoid energy accumulation.
It effectively limits the difference between the command state and actual state of the manipulator arm, prevents sudden movement, reduces the risk of collision, and improves the stability and safety of the manipulator arm.
Smart Images

Figure CN118927238A_ABST
Abstract
Description
[0001] This application is a divisional application of Chinese patent application 202080008137.5, filed on March 27, 2020, entitled “Reducing Energy Accumulation in Servo Control”.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS
[0003] This application claims the benefit of priority under 35 USC §119(e) to U.S. Provisional Patent Application Serial No. 62 / 826,780, filed on March 29, 2019, which is hereby incorporated by reference in its entirety. Technical Field
[0004] The present invention generally provides improved robotic and / or medical (including surgical) devices, systems and methods. Background Art
[0005] Overview
[0006] A system of robotic devices can be used to perform tasks at a work site. For example, a robotic system can include a robotic manipulator to manipulate an instrument used to perform a task. The robotic manipulator can include two or more links coupled together by one or more joints. The joints can be active joints that are actively moved and controlled. The joints can also be passive joints that follow the movement of the active joints or follow external manipulation. Such active and passive joints can be revolute joints or prismatic joints. The configuration of the robotic manipulator can then be determined by the position and orientation of the joints, the structure of the robotic manipulator, and the coupling of the links.
[0007] Robotic systems include industrial robotic systems and entertainment robotic systems. Robotic systems also include medical robotic systems for procedures such as diagnosis, non-surgical treatment, surgical treatment, etc. As a specific example, robotic systems include minimally invasive robotic telesurgery systems, in which surgeons can perform surgery on patients from the bedside or a remote location. Telesurgery generally refers to a surgical procedure performed using a surgical system, in which the surgeon uses some form of remote control (such as a servo mechanism) to manipulate the movement of surgical instruments rather than directly holding and moving the instruments by hand. A robotic medical system that can be used for telesurgery or other telemedicine procedures may include a remotely controllable robotic manipulator. The operator can remotely control the movement of the remotely controllable robotic manipulator. The operator can also manually move the workpiece of the robotic medical system to a position or orientation within its environment. Summary of the invention
[0008] In general, in one aspect, one or more embodiments relate to a computer-assisted medical system comprising a manipulator arm and a controller comprising a computer processor, the controller being configured with at least a non-clutch mode and a clutch mode. When in the non-clutch mode, the controller is configured to servo at least one joint associated with at least one manipulator arm segment of the manipulator arm, the servo comprising executing a servo loop comprising: obtaining an actual state of the manipulator arm; calculating a difference between a command state and an actual state, the command state being used to servo the at least one joint; determining whether the difference exceeds an error threshold; based on determining that the difference does exceed the error threshold, updating the command state using an offset to reduce the difference, and applying the command state to control the actual state; and based on determining that the difference does not exceed the error threshold, not updating the command state, and applying the command state to control the actual state. When in the clutch mode, the controller is configured to float the at least one joint.
[0009] In general, in one aspect, one or more embodiments relate to a method for operating a medical system. The method includes, when in a non-clutch mode, servoing at least one joint associated with at least one manipulator arm segment of a manipulator arm of the medical system, the servo including executing a servo loop, the servo loop including: obtaining an actual state of the manipulator arm; calculating a difference between a command state and an actual state, the command state being used to serve the at least one joint; determining whether the difference exceeds an error threshold; based on determining that the difference does exceed the error threshold, updating the command state using an offset to reduce the difference, and applying the command state to control the actual state; and based on determining that the difference does not exceed the error threshold, not updating the command state, and applying the command state to control the actual state. The method further includes, when in the clutch mode, floating the at least one joint.
[0010] In general, in one aspect, one or more embodiments relate to a non-transitory machine-readable medium comprising a plurality of machine-readable instructions executed by one or more processors associated with a medical system, the plurality of machine-readable instructions causing the one or more processors to perform a method comprising: in a non-clutch mode, servoing at least one joint associated with at least one manipulator arm segment of a manipulator arm of the medical system, the servoing comprising executing a servo loop, the servo loop comprising: obtaining an actual state of the manipulator arm; calculating a difference between a command state and an actual state, the command state being used to servo the at least one joint; determining whether the difference exceeds an error threshold; based on determining that the difference does exceed the error threshold, updating the command state using an offset to reduce the difference, and applying the command state to control the actual state; and based on determining that the difference does not exceed the error threshold, not updating the command state, and applying the command state to control the actual state. The method further comprises: in a clutch mode, floating the at least one joint.
[0011] Other aspects of the invention will become apparent from the following description and appended claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1A A top view of a robotic procedure scenario including a robotic assembly using a robotic manipulator for robotically moving a tool or instrument having an end effector at a work site is shown in accordance with one or more embodiments.
[0013] Figure 1B Schematically illustrates a method according to one or more embodiments Figure 1A The various components of the robot program scene.
[0014] Figure 2 A diagram is shown according to one or more embodiments for Figure 1A A perspective view of the main operator console or workstation where program commands are entered into the robotic assembly.
[0015] Figure 3 According to one or more embodiments, Figure 1A Perspective view of an electronics cart.
[0016] Figure 4 A perspective view of a robotic assembly having four manipulator arms is shown in accordance with one or more embodiments.
[0017] Figure 5 An example of a manipulator assembly is shown in accordance with one or more embodiments.
[0018] Figure 6 A perspective view of a tool or instrument is shown in accordance with one or more embodiments.
[0019] Fig. 7A and Figure 7B A control architecture for controlling a robotic assembly according to one or more embodiments is shown.
[0020] Figure 8 A diagram conceptually illustrating a method for avoiding excess energy accumulation in a servo control mechanism is shown in accordance with one or more embodiments.
[0021] Fig. 9 A flow chart is shown describing a method for limiting or reducing energy accumulation in a servo-control mechanism in accordance with one or more embodiments.
[0022] Fig. 10A A manipulator assembly is schematically illustrated in accordance with one or more embodiments in a position holding task.
[0023] Fig.10B1 and Fig.10B2 A manipulator assembly is schematically illustrated in accordance with one or more embodiments in a teleoperation mission. DETAILED DESCRIPTION
[0024] Specific embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. For consistency, the same elements in various drawings are represented by the same reference numerals.
[0025] In the following detailed description of the embodiments of the present disclosure, many specific details are set forth in order to provide a more thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other cases, well-known features are not described in detail to avoid unnecessarily complicating the description.
[0026] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as adjectives for elements (i.e., any noun in the application). Unless explicitly disclosed, such as by using the terms "before," "after," "single," and other such terms, the use of ordinal numbers does not imply or create any particular order of elements, nor is it meant to limit any element to only a single element. Rather, the use of ordinal numbers distinguishes between elements. For example, a first element is different from a second element, and a first element may include more than one element and be after (or before) a second element in the order of the elements.
[0027] Although some examples described herein relate to surgical procedures or tools, or medical procedures and medical tools, the disclosed technology is applicable to medical and non-medical procedures, and medical and non-medical tools. For example, the tools, systems and methods described herein can be used for non-medical purposes including industrial uses, general robotic uses, and sensing or manipulating non-tissue workpieces. Other example applications relate to cosmetic improvements, imaging of human or animal anatomical structures, collecting data from human or animal anatomical structures, setting up or disassembling systems, and training medical or non-medical personnel. Additional example applications include using procedures on tissue removed from human or animal anatomical structures (not returned to human or animal anatomical structures), and performing procedures on human or animal corpses. In addition, these technologies can also be used for medical treatment or diagnostic procedures that may or may not include surgical aspects.
[0028] In general, embodiments of the present disclosure can facilitate the use of robotic systems and improve workflows under various conditions. For example, the manipulator arm of a robotic system may not always be able to immediately follow a command to reach a command position, velocity, or generally a command state. Various reasons may prevent the manipulator arm from following the command. The result may be a difference between the command state and the actual state. This difference may be undesirable for various reasons. For example, when the difference between the command state and the actual state is significant, the manipulator arm may perform sudden and large movements once it is no longer prevented from moving. In addition, although the actual state of the manipulator arm can be directly observed, the command state is generally not observable by the operator or assistant. Therefore, the accumulation of differences may not be noticed, thereby possibly shocking the assistant when the movement of the manipulator arm toward the command state eventually occurs. The movement may further cause the manipulator arm to collide violently with the assistant, the patient, or other components of the robotic system.
[0029] Embodiments of the present disclosure provide the ability to limit the difference between the commanded state and the actual state of a manipulator arm to an acceptable level. According to one or more embodiments, the limitation on the difference is applicable to different scenarios, including but not limited to scenarios in which the manipulator arm is blocked (e.g., blocked by an obstacle), scenarios in which the manipulator arm is externally backdriven, scenarios in which a tele-manipulation task (i.e., a "following" task, in which the manipulator arm moves according to a command issued by an input device that is not the manipulator arm and is manipulated by an operator), etc. Each of these scenarios is described below.
[0030] Reference is now made to the drawings, wherein like reference numerals refer to like parts throughout the several views. Figure 1A A top view of a computer-assisted medical system (100) (hereinafter referred to as system (100)) in a robotic program scenario is shown. Figure 1AIn the example, a minimally invasive robotic surgical system is shown as a computer-assisted medical system (100), but the following description is applicable to other scenarios and systems, such as non-surgical scenarios or systems, non-medical scenarios or systems. In this example, a diagnostic or surgical procedure is performed on a patient (190) lying on an operating table (110). The system may include a user control system (120) for use by an operator (192) (e.g., a clinician such as a surgeon) during the procedure. One or more assistants (194) may also participate in the procedure. The system (100) may further include a robotic manipulation system (130) (e.g., a patient-side robotic device) and an auxiliary system (140). The robotic manipulation system (130) may include at least one manipulator arm (150), each of which may support a removably coupled tool (160) (also referred to as an instrument (160)). In the illustrated procedure, the tool (160) may be introduced into the body of the patient (190) through a natural orifice such as the throat or anus or through an incision, while an operator (192) views a working site (e.g., a surgical site in a surgical setting) through a user control system (120). An image of the working site may be acquired through an imaging device (e.g., an endoscope, an optical camera, or an ultrasound probe), which imaging device, i.e., the tool (160) for imaging the working site, may be manipulated by a robotic manipulation system (130) to position and orient the imaging device. An auxiliary system (140) may be used to process the image of the working site for display to the operator (192) through the user control system (120) or other display system located locally or remotely from the procedure. The number of tools (160) used at one time generally depends on the task and space constraints, among other factors. If it is appropriate to replace, clean, inspect, or reload one or more of the tools (160) during the procedure, the assistant (194) can remove the tool (160) from the manipulator arm (150) and replace it with the same tool (160) or another tool (160), for example from a tray (180) or other type of tool storage compartment.
[0031] Figure 1B The system (102) is schematically shown. The system (102) may include one or more computing systems (142). The computing system (142) may be used to process input from an operator provided by the user control system (120). The computing system may further be used to provide output, such as a video image, to a display (144). The one or more computing systems (142) may further be used to control the robotic manipulation system (130).
[0032] The computing system (142) may include one or more computer processors, non-persistent storage (e.g., volatile memory such as random access memory (RAM), cache), permanent storage (e.g., a hard disk, an optical drive such as a compact disk (CD) drive or a digital versatile disk (DVD) drive, flash memory, etc.), communication interfaces (e.g., a Bluetooth interface, an infrared interface, a network interface, an optical interface, etc.), and many other elements and functions.
[0033] The computer processor of the computing system (142) may be an integrated circuit for processing instructions. For example, the computer processor may be one or more cores or microcores of a processor. The computing system (142) may also include one or more input devices, such as a touch screen, a keyboard, a mouse, a microphone, a touchpad, an electronic pen, or any other type of input device.
[0034] The communication interface of the computing system (142) may include an integrated circuit for connecting the computing system (142) to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or to another device (e.g., another computing system (142)).
[0035] In addition, the computing system (142) may include one or more output devices, such as a display device (e.g., a liquid crystal display (LCD), a plasma display, a touch screen, an organic LED display (OLED), a projector, or other display device), a printer, a speaker, external storage, or any other output device. One or more of the output devices may be the same as or different from the input device(s). There are many different types of computing systems, and the aforementioned input and output device(s) may take other forms.
[0036] The software instructions in the form of computer-readable program code for executing the embodiments of the present disclosure may be stored in whole or in part, temporarily or permanently, on a non-transitory computer-readable medium (e.g., a CD, a DVD, a storage device, a floppy disk, a tape, a flash memory, a physical memory, or any other computer-readable storage medium). Specifically, the software instructions may correspond to a computer-readable program code that, when executed by (one or more) processors, is configured to execute one or more embodiments of the present invention.
[0037] The computing system (142) may be connected to or part of a network. The network may include multiple nodes. Each node may correspond to the computing system (142) or a group of nodes. For example, embodiments of the present disclosure may be implemented on a node of a distributed system connected to other nodes. As another example, embodiments of the present invention may be implemented on a distributed computing system having multiple nodes, wherein each part of the present disclosure may be located on a different node within the distributed computing system. In addition, one or more elements of the aforementioned computing system (142) may be located at a remote location and connected to other elements via a network.
[0038] The robotic manipulation system (130) can use a tool (160) including an imaging device (e.g., a monocular or stereoscopic endoscope, an ultrasound probe) to capture images of the work site and output the captured images to the auxiliary system (140). Similar to other tools (160), the tool with the imaging device has a mechanical interface (not shown) that allows the imaging device to be coupled to the manipulator arm (150). The mechanical interfaces of different tools (160) (e.g., tools with imaging devices) can be the same or different from each other. Therefore, where applicable, a mechanical adapter can be used to couple the tool (160) to the manipulator arm (150). Alternatively, a specific tool (160) such as a dedicated imaging tool can be installed on a manipulator arm (150) specially designed to accommodate such a tool (160). The auxiliary system (140) can process the captured image in various ways before any subsequent display. For example, the auxiliary system (140) can overlay the captured image with a virtual control interface before displaying the combined image to the operator via the user control system (120). The robotic manipulation system (130) may output the captured images for processing external to the auxiliary system (140). One or more separate displays (144) may also be coupled to the computing system (142) and / or the auxiliary system (140) for local and / or remote display of images (such as images of the procedural site) or other relevant images.
[0039] Figure 2 A perspective view of a user control system (120) is shown. The user control system (120) includes a left-eye display (202) and a right-eye display (204) for providing an operator (192) ( Figure 1A The user control system (120) further includes one or more input control devices (210) that in turn cause the robotic manipulation system (130) ( Figure 1A ) to manipulate one or more tools. The input control device (210) may provide a tool (160) ( Figure 1A) to provide a remote presence to the operator, or to allow the input control device (210) to be coupled to the tool (160) ( Figure 1A ) into one sense, so that the operator has a strong sense of direct control of the tool (160). To this end, position, force and / or tactile feedback sensors (not shown) can be used to transmit the position, force and / or tactile sensation from the tool (160) back to the operator's hand through the input control device (210).
[0040] Figure 3 A perspective view of the auxiliary system (140) is shown. The auxiliary system (140) can be used with an imaging device type tool (160) ( Figure 1A The auxiliary system (140) may be coupled to the image processing unit (shown in Figure 1) and may include a processor (not shown) to process the captured images for subsequent display, such as to an operator at an operator console or another suitable display located locally and / or remotely. For example, in the case of a stereo endoscope, the auxiliary system (140) may process the captured images to present a coordinated stereo image of the working site to the operator. Such coordination may include alignment between relative images and may include adjusting a stereo working distance of the stereo endoscope. As another example, the image processing may include using previously determined camera calibration parameters to compensate for imaging errors of the image capture device, such as optical aberrations.
[0041] Figure 4 A robotic manipulation system (130) is shown having a plurality of manipulator arms (150), each manipulator arm (150) supporting an instrument or tool (160) at a distal end of the manipulator arm. The robotic manipulation system (130) shown includes four manipulator arms (150) that can be used to support tools (160) for manipulation or tools (160) for imaging, such as a stereo endoscope for capturing images of a procedure site. Figure 5 A more detailed description of the manipulator arm 150 is provided and referenced below. Figure 6 A more detailed description of the tool (160) is provided. In a minimally invasive scenario, the tool (160) can be positioned and manipulated through an incision in a patient so that the kinematic distal center is maintained at the incision to minimize the size of the incision or the forces applied to the tissue surrounding the incision. When the tool (160) is positioned within the field of view of the tool operating as an imaging device, the image of the working site can include an image of the distal end of the instrument or tool (160).
[0042] Various tools (160) or instruments of different types and different end effectors may be used. During a procedure, at least some of the tools (160) may be removed and replaced. In a surgical scenario, the end effectors may include, but are not limited to, DeBakey forceps, micro forceps, Potts scissors, clip appliers, scalpels, and electrocautery probes. Some of these end effectors may have a single end effector element, while others may include several end effector elements, such as a first end effector element and a second end effector element, which may pivot relative to each other to define a pair of end effector jaws.
[0043] In a surgical scenario, the elongated shaft of the tool (160) allows the distal end of the end effector and the shaft to be inserted distally into a surgical working site through a minimally invasive hole (typically through a body wall such as the abdominal wall). The surgical working site can be insufflated. Movement of the end effector within the patient's body is typically achieved at least in part by pivoting the tool (160) about the position at which the shaft passes through the minimally invasive hole. Thus, the manipulator arm (150) can move the proximal housing of the instrument outside the patient's body so that the shaft extends through the minimally invasive hole to provide the desired movement of the end effector. Thus, the manipulator arm (150) can undergo movement outside the patient's body.
[0044] Figure 5 An example of a manipulator assembly (500) according to an embodiment of the present disclosure is shown in FIG. The manipulator assembly (500) includes a manipulator arm (502) and may further include a tool (520) (also referred to as an instrument (520)) (in Figure 5 , only the axis of the tool (520) is shown, not the tool (520) itself). In other words, in some embodiments, the manipulator assembly (500) can be a manipulator arm (502) with a tool (520), and in other embodiments a manipulator arm (502) without a tool (520). Other components may be included in the manipulator assembly (500). As described above, during operation, the manipulator arm (502) generally supports the distal instrument or tool (520) and enables movement of the tool (520). Since several different tools (520) with different end effectors can be sequentially mounted on the manipulator arm (502), or the tool (520) needs to be removed and reinstalled during a procedure, the distal tool holder facilitates removal and replacement of the mounted instrument or tool (520). As described in reference Figure 4 As will be appreciated, the manipulator arm (502) is proximally mounted to the base of the robotic assembly. Alternatively, the manipulator arm (502) may be mounted to a separate base that can be independently moved, for example, by mounting the manipulator arm (502) to a single manipulator arm cart that provides a means of allowing the manipulator arm (502) to be mounted directly or indirectly to the operating table ( Figure 1A Typically, the manipulator arm (502) includes a plurality of manipulator arm segments and associated joints extending between a proximal base and a distal tool support.
[0045] In e.g. Figure 5 In the illustrated embodiment, the manipulator arm (502) includes a plurality of joints (such as rotary joints J1, J2, J3, J4, and J5 and a prismatic joint J6) and links or manipulator arm segments (504, 506, 508, and 510). The joints of the manipulator arm (502) may or may not have redundant degrees of freedom in combination. A manipulator arm having one or more redundant degrees of freedom has a plurality of joints such that the plurality of joints can be driven into a range of different configurations for a given position and / or orientation of a portion of the manipulator arm. For example, while a distal member (512) supported within a tool holder (510) maintains a particular state and may include a given position or velocity of an end effector, Figure 5 The manipulator arm (502) can be maneuvered into different configurations. The tool holder (510) can include a cannula (516) through which a tool shaft of a tool (520) extends, and the tool holder (510) can include a carriage (514) shown as a box-shaped structure that translates on a crossbar to which the tool is attached before extending through the cannula (516) toward the work site.
[0046] Actuation of the degrees of freedom of the tool (520) is typically provided by actuators of the manipulator. These actuators may be integrated into the carriage (514). The distal wrist of the tool (520) may allow the end effector of the tool (520) to pivot and / or linearly move about the tool joint axis of one or more joints at the tool wrist. The angle between the end effector jaw elements may be controlled independently of the position and orientation of the end effector. Figure 6 A detailed description of the tool (520) is provided.
[0047] Figure 6An example of a tool (600) (also referred to as an instrument (600)) that can be used for surgical procedures according to one or more embodiments is shown. The tool (600) includes a shaft (610), a wrist (620) located at a working end of the shaft (610), and an end effector (640) disposed on the wrist (620). The wrist (620) can enable the end effector (640) to pivot relative to the shaft (610). The wrist (620) can have at least one degree of freedom. A housing (630) is located at an opposite end of the shaft (610), and the housing (630) is releasably arranged to couple the tool (600) to a manipulator arm (502). The shaft (610) can be rotatably coupled to the housing (630) so that the shaft (610) can be angularly displaced relative to the housing (630) as indicated by arrow (692), thereby allowing the end effector (640) coupled to the shaft to rotate via the wrist (620). In Figure 5 , when the tool (520) is coupled or mounted on the manipulator arm (502), the shaft (610) extends through the sleeve (516). The tool (520) is typically releasably mounted on a tool holder (510) of the manipulator arm (502), which can be driven to translate along a linear guide formed by the prismatic joint (J6). Return Figure 6 , which may also be referred to as the IO, in / out movement or insertion axis (612). As described in U.S. Patent No. 6,394,998, entitled "Surgical Tools for Use in Minimally Invasive Telesurgical Applications," the housing (630) may include a spool that is rotatable to control a cable to actuate a linkage of an end effector (640). The tool holder (510) of the manipulator arm (502) may include a disk for coupling to the spool to drive the spool when the tool (600) is connected to the manipulator arm (502). The disk may be driven by an actuator (e.g., a motor) that responds to input from an associated input control device (e.g., Figure 2 The input of the input control device (210) in the embodiment of the present invention is used to drive the end effector (640) to a desired orientation as specified by the movement of the input control device (210) or any other control signal. In addition, appropriately positioned sensors (e.g., encoders, potentiometers, etc.) can be provided to enable measurement of joint position. Figure 5 As shown, the actuator and sensor may be disposed in a carriage (514) of a tool holder (510).
[0048] Different tools (600) can be equipped with different end effectors (640) having different geometries, degrees of freedom and / or functions. The end effector can have a single component or two or more components. Examples of end effectors having a single component include, but are not limited to, scalpels, cautery electrodes, irrigation or suction equipment, endoscopes (which may or may not have a wrist), etc. Examples of end effectors having two components include, but are not limited to, forceps, clip appliers, scissors, dissecting tools, forceps, etc. The components of the end effector (640) can be individually angularly displaced, thereby not only allowing the end effector to be opened and closed, but also angularly displaced to change the orientation of the end effector (640) as a whole relative to the wrist (620).
[0049] although Figure 1A , Figure 1B , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 Various configurations of components are shown, but other configurations may be used without departing from the scope of the invention. For example, various components may be combined to create a single component. As another example, the function performed by a single component may be performed by two or more components. Furthermore, although the components are described in the context of a surgical scenario, embodiments of the present disclosure may be equally applicable to other fields involving robotic manipulation.
[0050] Go to Fig. 7A , shows a control architecture for controlling a manipulator assembly including a manipulator arm and an instrument or tool mounted thereon according to one or more embodiments. One control architecture is taken as an example. Those skilled in the art will appreciate that other control architectures may be used without departing from the present disclosure. In addition, in the illustrated control architecture, specific signals (e.g., position) are exchanged between the frames of the control architecture. Other signals (e.g., velocity, acceleration, force, etc.) may be used without departing from the present disclosure. Similarly, the control architecture may implement different modes (not shown). For example, Figure 2As shown, various joints of the robotic manipulation assembly can be position controlled during the execution of a robotic task under the control of an input control device (210) operated by a user. Similarly, in another control mode, such as in a tool change mode, one or more joints can be "floating", thereby allowing an assistant to easily externally articulate these one or more joints, such as by reverse driving these one or more joints. The floating joint can be reverse driven by an externally applied force without the need for a control algorithm to offset the reverse driving force or a braking force configured to offset such reverse driving force. For example, the floating joint can be controlled to be in a "clutch mode" in which the command position for the floating joint is the current position to assist external manipulation. As a further example, the user can apply a force to a connecting rod distal to the floating joint, resulting in reverse driving of the floating joint. In particular, when the joint can move in the direction of gravity pull (for example, the axis of the joint is not vertical), the floating joint can further compensate for gravity. In addition, friction compensation can facilitate reverse driving. Additionally or alternatively, the floating joint can also be controlled to apply other characteristics, such as a certain degree of damping.
[0051] Multiple control modes can be combined during operation of the manipulator assembly, for example, some joints are position controlled to resist or rebound the external articulation of those joints, while other joints can be floating and contribute to the external articulation of those other joints. In addition, one or more joints of the manipulator assembly can be passive, that is, not controlled by position or speed at all (but brakes can be partially or completely applied). Passive joints can be manually operated. Passive joints can also include joint sensors so that the complete kinematics of the manipulator assembly can be obtained. In addition, in some embodiments, passive joints can include actuators for providing gravity compensation, friction compensation, or other uses that do not include actively driving the motion of passive joints.
[0052] In one or more embodiments, the joint movement of the manipulator assembly is controlled by a controller using an actuator (e.g., motor, solenoid, etc.) of the manipulator assembly to drive one or more joints, which are calculated by a processor of the controller. Mathematically, the controller can use vectors and / or matrices to perform at least some calculations of joint commands, some of which can have elements corresponding to the position, speed, and / or force / torque of the joint. The range of alternative joint configurations available to the processor can be conceptualized as joint space. The joint space can, for example, have as many dimensions as the degrees of freedom that the manipulator assembly has, and the specific configuration of the manipulator assembly can represent a specific point in the joint space, wherein each coordinate corresponds to the joint state of the associated joint of the manipulator arm assembly.
[0053] As used herein, the term "state" of a joint or multiple joints refers to a control variable associated with the joint or multiple joints, respectively. For example, the state of an angular joint can refer to the angle defined by the joint within its range of motion, and / or to the angular velocity of the joint. Similarly, the state of an axial or prismatic joint can refer to the axial position of the joint, and / or to its axial velocity. Although one or more of the controllers described herein include a position controller, they typically also have a speed control aspect. Without departing from the present disclosure, alternative embodiments may rely primarily or entirely on a speed controller, a force controller, an acceleration controller, etc. Many aspects of the control system that can be used in such a device are fully described in U.S. Patent No. 6,699,177, which is incorporated herein by reference in its entirety. Therefore, as long as the motion described is based on associated calculations, position control algorithms, speed control algorithms, combinations of the two, etc. can be used to perform calculations of the movement of the joints described herein and the movement of the end effector.
[0054] Fig. 7A The control architecture (700A) includes a controller (710) that drives joint actuators (790) of a manipulator assembly based on commanded movement (720). Any number of joint actuators (790) may be driven.
[0055] The commanded movement (720) may be a commanded position and / or velocity of one or more features of the workspace in a Cartesian coordinate space (referred to herein as Cartesian space). The commanded movement (720) may be, for example, a movement command (e.g., in the form of a position and / or velocity) received from a user control system (120), or any other movement command of one or more features of a manipulator arm. A feature may be any feature physically on or physically not on a manipulator assembly that may be used to define a control frame to be articulated using a control input. Examples of features on a manipulator assembly include features of a tool (e.g., an end effector tip, a center point on an end effector, or a U-clip of an end effector), features of a manipulator arm (e.g., an instrument support configured to be physically coupled to a movable instrument), and the like. Another example of a feature of a manipulator assembly is a reference point in empty space that is exactly a certain distance and angle from the tool tip. Another example of a feature outside of a manipulator assembly is a target tissue whose position relative to a portion of the manipulator assembly may be established.
[0056] The controller (710) may include a top-level controller (730), an inverse kinematics controller (740), a joint controller (750), and a forward kinematics model (760). Each of these components will be described later.
[0057] The top-level controller (730) according to one or more embodiments includes instructions in the form of computer-readable program code to receive a command movement (720) and convert the command movement (720) into a position in a Cartesian reference frame. The steps performed to convert the command movement (720) into a Cartesian position depend on the format of the command movement (720) provided. For example, if the command movement (720) specifies a desired end effector position, the top-level controller (730) can perform trajectory planning using, for example, position time (PT) or position velocity time (PVT) interpolation. Alternatively, if the command movement (720) includes a velocity command, the top-level controller (730) can operate as an integrator. Those skilled in the art will appreciate that the top-level controller (730) can perform any operations required to obtain a position signal in a Cartesian reference frame. In one or more embodiments, the top-level controller (730) generates a Cartesian position based on the command movement (720) while taking into account a command joint position (742). The command joint position (742) can enable the top-level controller to determine the actual state of the joint to be controlled (e.g., including a current position and / or velocity, etc.). The actual state may affect the control task and may therefore be considered by the top-level controller. For example, a commanded move may be undesirable or unsafe for a particular configuration of the manipulator assembly and may therefore not be executed, or may alternatively be converted into an alternative commanded move that may be safely executed.
[0058] According to one or more embodiments, the inverse kinematics controller (740) includes instructions in the form of computer-readable program code to convert the command Cartesian position (732) to a command joint position (e.g., a joint angle for a rotational joint) (742). The operation of the inverse kinematics controller can be performed in the velocity domain. In other words, the inverse kinematics controller (740) can seek to determine or solve for a joint velocity vector that can be used to drive the joints of the manipulator arm assembly in a manner that the end effector accurately follows the command Cartesian position. The inverse kinematics controller (740) can integrate the calculated joint velocities to obtain the command joint position (742).
[0059] The Cartesian error (732) may be a combination of the Cartesian position provided by the top-level controller (730) (as previously discussed) and the Cartesian position provided by the forward kinematics model (760) (as discussed below). More specifically, the Cartesian position provided by the forward kinematics model (760) may represent an estimate of the actual or current position of the manipulator assembly (e.g., the end effector) in Cartesian space. This estimate may be subtracted from the Cartesian position representing the commanded movement to obtain a difference to be compensated for use as a control input to the inverse kinematics controller (740).
[0060] While there is typically no closed-form relationship that maps a desired Cartesian space position to an equivalent joint space position, a closed-form relationship between Cartesian space velocity and joint space velocity typically exists. The kinematic Jacobian matrix is a matrix of partial derivatives of Cartesian space position elements with respect to joint space position elements. In this way, the kinematic Jacobian matrix captures the kinematic relationship between, for example, an end effector and a joint. In other words, the kinematic Jacobian matrix captures the effects of joint motion on the end effector. The kinematic Jacobian matrix (J) can be used to map joint space velocity (dq / dt) to Cartesian space velocity (dx / dt), such as end effector velocity.
[0061] Thus, even when there is no closed-form mapping between input positions and output positions, the inverse kinematics controller (740) can iteratively use the mapping of velocities to achieve movement of the manipulator assembly based on the command trajectory. One such implementation is described in simplified terms below. Assume that the command movement (720) includes a time step t i Provides the Cartesian position. At each time step (t i ), the Cartesian velocity (dx / dt) is calculated by the inverse kinematics controller (740) to perform the desired movement and correct the accumulated deviation from the desired Cartesian position (obtained by subtracting the Cartesian position produced by the forward kinematics model (760)). This commanded Cartesian position (or the Cartesian error (732) after subtracting the output of the forward kinematics model) is then converted to a commanded joint position (q) (742) using the pseudo-inverse of the Jacobian matrix (J#) in the velocity domain. The commanded joint positions are used to recalculate the Jacobian matrix (J), which can be used in calculations performed at the next time step. The steps described can be performed for any number of joints.
[0062] Some exemplary manipulator assemblies described herein have more degrees of freedom than are required to position and move the end effector within a work site. For example, in some embodiments, a surgical end effector that can be positioned with six degrees of freedom at an internal surgical site through a minimally invasive hole can have a nine-degree-of-freedom task space (six end effector degrees of freedom (three for position, three for orientation) plus three degrees of freedom to comply with entry site constraints), but can have ten or more degrees of freedom. A highly configurable manipulator arm assembly with more degrees of freedom than required for a given end effector position can be described as having or providing sufficient degrees of freedom to allow a series of joint states for an end effector position in a workspace. For example, for a given end effector position, the manipulator assembly can occupy any one of a series of alternative manipulator arm configurations (and be driven between them). Similarly, for a given end effector velocity vector, the manipulator assembly can have a series of different joint movement velocities for each joint of the manipulator assembly within the null space of the Jacobian matrix.
[0063] When guiding the movement of a highly configurable manipulator with redundant degrees of freedom, the inverse Jacobian matrix often does not fully define the joint vector solution. For example, the mapping from Cartesian commands (x) to joint positions (q) in the system may occupy a series of joint states for a given end effector state, which is a one-to-many mapping. In other words, because the mechanism is redundant, there are mathematically infinitely many solutions, represented by the subspace in which the inverse solution resides. Other constraints may be imposed to obtain a unique solution. Those skilled in the art will appreciate that various methods can be used to perform inverse kinematics, including inverse kinematics for manipulators with and without redundant degrees of freedom.
[0064] According to one or more embodiments, each joint controller (750) includes instructions in the form of computer-readable program code to convert the received command joint position (742) into a command current (752) to drive one of the joint actuators (790) to produce a joint movement (792). Each joint actuator (790) can use a joint controller (750). Through a kinematic manipulator assembly, the joint movement (792) of all joint actuators can produce a manipulator arm movement that reflects the command movement (720). In one embodiment of the present disclosure, the joint controller controls the position or angle of the joint. Alternatively, the joint controller can control other variables, such as joint speed, joint torque, or joint force (in the case of a linear joint). Figure 7B An example of a joint controller (750) is provided in .
[0065] According to one or more embodiments, the forward kinematics model (760) includes instructions in the form of computer-readable program code, as previously discussed, to convert the sensed joint positions (754) to Cartesian positions.
[0066] The controller (710) may be implemented on one or more computing systems. These one or more computing systems may be based on a digital signal processor (DSP), a central processing unit (CPU), etc. Figure 1B Example computing systems are described. Each computing system can perform the described operations at a cycle time that depends on the nature of the operation. In one embodiment, the cycle time of the inverse kinematics controller (740) and the joint controller (750) is the same. Communications between the computing systems implementing the top-level controller (730), the inverse kinematics controller (740), and the joint controller (750) can be performed using any type of electrical or optical communication network (including Ethernet, optical fiber, various bus systems, and / or any other type of digital or analog signal).
[0067] In one or more embodiments, the controller (710) is further configured to perform Fig. 9 At least one of the steps described in .
[0068] Go to Figure 7B , shows a control architecture (700B) of a joint controller according to one or more embodiments. In an example, a closed-loop PD control structure is used to control a joint position based on a commanded joint position input.
[0069] The joint controller may receive a feedback signal in the form of a sensed joint state (754) from an associated joint actuator (790) (specifically, in the case of the control architecture (700B), a sensed joint position in this example) to enable closed-loop control. The sensed joint position including the joint state (754) may be derived from a signal acquired by a sensor attached to the joint. Such a sensor may be, for example, an incremental encoder, a shape sensor, or a Hall sensor of the joint or joint actuator. A state observer or estimator (not shown) may be used. The PD control structure uses two control gains (K P , K D ), respectively, operates on the difference between the command joint position (742) and the sensed joint position (i.e., error signal) and its derivative to generate a command current (752). In one or more embodiments of the present disclosure, the command current (752) is limited, and thus the resulting joint torque or force is also limited. The limitation can be based on hardware limitations, such as a maximum acceptable motor current. The limitation can further be software configurable.
[0070] Thus, the motor current (and resulting motor torque or force) can be proportional to the commanded joint position (742) and the sensed joint position (754) as determined by the proportional control gain (K P ) increases linearly until the saturation limit is reached. Beyond the limit, the motor current is constant. A higher Kp can result in a relatively small position error sufficient to reach the saturation limit, while a lower Kp can result in a relatively large position error required to reach the saturation limit. In one embodiment of the present disclosure, a relatively high Kp is used to obtain responsive position control of a joint with a limited steady-state error. Therefore, an increase in position error can quickly lead to reaching the saturation limit. Although the control architecture (700B) uses a proportional-derivative (PD) controller, other controllers such as proportional-integral-derivative (PID), full state feedback, sliding mode, or various other control schemes may be used without departing from the present disclosure. In addition, although control of the joint position is illustrated, other variables such as speed, torque, or force may be controlled without departing from the present disclosure.
[0071] The nature of the command signal may further vary during operation. Figure 2 As shown, various joints of the robotic manipulator assembly can be position controlled during the execution of a robotic task under the control of an input control device (210) operated by a user. However, in another control mode, for example, during a tool change, one or more joints can be "floating", allowing an assistant to freely backdrive these joints. In this mode, one or more joints can still be position controlled even though the command joint position (742) is the sensed joint position (754).
[0072] Go to Figure 8 , a diagram conceptually illustrating a method for avoiding excessive energy accumulation in a servo control mechanism is shown.
[0073] Assume that Figure 7B The PD controller shown is used to control the joint actuators of a manipulator arm or a manipulator arm assembly including a manipulator arm and a tool. One PD controller can be used for each joint actuator of a manipulator arm segment. Figure 7BAs discussed, the proportional control gain of the PD controller can operate on the difference between the command position and the sensed joint position to generate a command current to drive the joint actuator, and the joint actuator can produce a torque proportional to the command current. Therefore, a larger difference between the command joint position and the sensed joint position results in a larger torque produced by the joint actuator. Therefore, the control loop based on the PD controller can have the characteristics of a linear spring: the farther the command position and the actual joint position are apart, the more energy is stored in the spring because the PD controller needs to expend greater effort to compensate for the difference. The coefficient of the spring can be determined by the proportional control gain (K P )control.
[0074] Although the difference between the command joint position and the sensed joint position usually results in a command current that compensates for the difference by driving the joint actuator in the opposite direction to the difference, compensation may not occur in some cases. For example, suppose (a) the manipulator arm segment attached to the joint by the actuator encounters an obstacle that blocks movement, or (b) the manipulator arm segment attached to the joint by the actuator is reversely driven by the user. In either case, the difference between the command joint position and the sensed joint position can be prevented from decreasing, or even increasing. As a result, in scenario (a), the manipulator arm may move suddenly when the obstacle is removed. The sudden movement may surprise or shock the user, possibly leading to a potential strong collision, etc. In scenario (b), even if the user intentionally reverse drives the manipulator arm under the assumption that the reverse-driven manipulator arm will remain in the reverse-driven position, once the user stops the reverse drive of the manipulator arm, the manipulator arm may return to the command position. The user may reverse drive the manipulator arm for various reasons, for example, when trying to avoid a dangerous or inconvenient position.
[0075] To address the undesirable accumulation of excessive differences between commanded and sensed joint positions, the Figure 8 The method for avoiding or limiting excessive energy accumulation in a servo control mechanism is shown in FIG. Figure 8 As shown, the error threshold is used to determine whether the difference between the command position and the sensed joint position (or more generally, between the command joint state and the actual joint state) is acceptable. The error threshold can be set relative to the command position or more generally the command state. If the difference between the command state and the actual state exceeds the error threshold, the command state can be updated to be closer to the actual state, thereby reducing the difference to an acceptable level below the error threshold. Conversely, if the difference between the command state and the actual state is below the error threshold, the command state may not be updated.
[0076] Referring to the scenarios (a) and (b) described earlier, the impact is as follows. For (a), when an obstacle is present, thereby preventing the movement of the manipulator arm segment, the joint actuator can generate a force acting on the obstacle. Because even though the position error will assume a higher command current, the saturation limit of the joint will be through K P To limit the command current, the force can be the result of the maximum force (or torque) generated by the joint when the saturation limit is reached. Once the obstacle is removed, only limited movement of the connecting rod may occur due to the update of the command joint position to be close to the sensed joint position. This movement is almost imperceptible. Therefore, removing the obstacle does not lead to undesirable consequences, such as sudden movement of the connecting rod. For (b), based on the update of the command state, the manipulator arm segment can be backdriven and can be maintained at or near the backdriven position.
[0077] Although the above discussion introduces a method for avoiding excessive energy accumulation in a servo control mechanism using position control based on the difference between the command position and the actual position of a joint, those skilled in the art will appreciate that similar examples are also applicable to other configurations. For example, alternative configurations can be applied to speed-controlled joints. In addition, although the above discussion is based on control of a joint or multiple joints, where variables (e.g., command joint position, actual joint position) are considered in a joint reference frame, the method can also be applied in a Cartesian reference frame, such as when a similar example is applied to the entire manipulator arm as discussed below.
[0078] Fig. 9 A flow chart is shown according to one or more embodiments. Fig. 9 The flowchart of depicts a method for reducing or limiting energy accumulation in a servo-controlled mechanism in a clutchless mode according to one or more embodiments. The clutchless mode can be used to perform any task involving servo control of one or more joints of a manipulator arm, such as position holding tasks, teleoperation (i.e., "following") tasks, and positioning tasks; these tasks may involve manipulating or imaging an environment using a manipulator assembly. As previously described with reference to Figure 1A , Figure 1B , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 As described, Fig. 9One or more of the steps in can be performed by various components of the system. These figures describe a particular manipulator arm and a particular tool having certain degrees of freedom. However, the methods described subsequently are not limited to a particular configuration of manipulator arms, tools, and / or degrees of freedom. Instead, the methods are applicable to any type of manipulator arm paired with any type of tool used in any type of scenario. Furthermore, the methods described can be applied to a variety of signals and spaces. For example, the methods can be applied to position, velocity, or force signals in joint space and / or Cartesian space. To further illustrate the application of the method, in Fig. 10A , Fig.10B1 and Fig.10B2 Examples are provided in . Various applications and benefits are discussed next.
[0079] Although the various steps in these flowcharts are presented and described sequentially, it will be understood by those of ordinary skill in the art that some or all of the steps may be performed in a different order, may be combined or omitted, and some or all of the steps may be performed in parallel. Additional steps may be further performed. In addition, the steps may be performed actively or passively. For example, according to one or more embodiments of the present invention, some steps may be performed using polling or interrupt driven. For example, according to one or more embodiments of the present invention, the determination step may not require the processor to process instructions unless an interrupt is received to indicate that a condition exists. As another example, according to one or more embodiments of the present invention, the determination step may be performed by performing a test (such as checking a data value to test whether the value is consistent with the condition being tested). Therefore, the scope of the present disclosure should not be considered to be limited to Fig. 9 A specific arrangement of the steps shown in .
[0080] Go to Fig. 9 , showing a series of steps that, when performed repeatedly, can form a servo loop for servoing at least one joint of a manipulator arm assembly. The servo can cause the manipulator assembly to enter a manipulation mode, thereby allowing the manipulator assembly to perform a robotic task. Control of a manipulator arm or manipulator arm assembly can involve a hierarchy of such control loops. As previously described in Fig. 7A As shown in , a control loop can be used to generate a command joint position from a command movement that is provided as an input. In addition, as previously described in Figure 7B As shown in , additional control loops can be used at the joint level to convert command joint positions into command currents to drive joint actuators. As described later, methods for reducing energy accumulation in servo control mechanisms can be integrated with these control loops in various ways.
[0081] At step 900, a desired state is obtained. The desired state may be for one or more joints associated with one or more manipulator arm segments of the manipulator assembly, or for the entire manipulator assembly. The desired state may be a desired position, a desired velocity, a desired force, etc. Depending on the implementation of the method for reducing energy accumulation, the desired state may be provided in a Cartesian reference frame or in a joint reference frame.
[0082] In embodiments where the method for reducing energy accumulation is applied to an entire manipulator assembly or multiple manipulator arm segments of a manipulator assembly, a desired state may be provided in a Cartesian reference frame, for example to guide an end effector of the manipulator assembly. Fig. 7A With the control architecture shown, a command state can affect the state of multiple (or all) manipulator segments and their associated joints.
[0083] In embodiments where the method for reducing energy accumulation is applied to one or more manipulator arm segments at the joint level, a desired state may be provided in the joint reference frame based on Figure 7B The control architecture shown in Figure 1 guides the affected manipulator arm segment by controlling the position, angle, torque, or force of the associated joints.
[0084] In step 902, the actual state is obtained. The actual state can be an actual position, an actual speed, an actual torque, an actual force, etc. In one or more embodiments, the actual state for subsequent steps is represented by a sensed state. The sensed state can be obtained from a sensor (e.g., a sensor in a joint actuator). Therefore, the sensed state can be a measurement of the actual state. Alternatively, the sensed state can be an estimate of the actual state. For the purpose of subsequent discussion, the sensed / estimated state and the actual state are considered equivalent. Depending on the implementation of the method for reducing energy accumulation, the actual state can be provided in Cartesian space or joint space.
[0085] In an embodiment where the method for reducing energy accumulation is applied to an entire manipulator assembly or multiple manipulator arm segments of a manipulator assembly, an actual state can be provided in Cartesian space. The actual state can represent, for example, the position, velocity, force, or torque of an end effector of the manipulator assembly. In this embodiment, to obtain the actual state, the state of the manipulator arm assembly in Cartesian space can be reconstructed using the individual sensed joint states (e.g., joint positions or velocities).
[0086] In embodiments where the method for preventing energy accumulation is applied to one or more manipulator arm segments at the joint level, the actual state may be provided in the joint space of the considered joint to reflect the state of the associated manipulator arm segment.
[0087] At step 904, the command state is obtained. According to one or more embodiments, the command state is obtained by subtracting an offset from the desired state. As described below, the offset may be reflected in Fig. 9 The difference between the desired state and the actual state obtained during the previous execution of step 908 of the method. Initially, the offset may be zero, so that the commanded state is equal to the desired state. However, over time, as Fig. 9 The offset can be increased or decreased if the method addresses energy accumulation (due to the difference between the command state and the actual state).
[0088] In step 906, the difference between the command state and the actual state is compared to an error threshold to determine whether the difference exceeds the error threshold. The absolute difference can be used to make this determination, regardless of whether the difference is positive or negative. The error threshold may have been pre-established. For scenarios where less energy accumulation is acceptable, a reduced error threshold may be preferred, while for scenarios where more energy accumulation is acceptable, an increased error threshold may be used. In situations where it is less desirable to change the command state, an increased error threshold may also be used. If the difference reaches or exceeds the error threshold, execution of the method may proceed to step 908. If the difference is below the error threshold, execution of the method may bypass step 908 and proceed directly to step 910.
[0089] In step 908, an updated offset is obtained and the command state is updated using the updated offset. In one or more embodiments, the updated offset is an offset that reduces the difference between the command state and the actual state, thereby reducing energy accumulation in the servo loop, and the offset is proportional to the control gain K that operates on the difference between the command state and the actual state. P More specifically, an updated offset can be calculated to achieve adjustment of the command state to the actual state without completely reaching the command state. Therefore, the command state can eventually approach the actual state. Based on the proportional control gain (K P), the resulting difference between the command state and the actual state may not be sufficient to reach the saturation limit of the associated actuator, i.e., the command state may be set to a minimum distance (or a small multiple of the minimum distance) from the actual state required to reach saturation. In other words, a barely sufficient difference between the command state and the actual state is a difference that drives the servo loop to saturation without being significantly greater than the difference required to reach the saturation limit. The updating of the command state may be performed more gradually over time (e.g., in fractions of a second or seconds, etc.) in a single instant of time or in a single step or multiple steps. Multi-step or more gradual updates are more likely to be used in conjunction with algorithms that check for unexpected jumps in the command state (e.g., seemingly unbelievable jumps, such as jumps that are beyond what the system can achieve or beyond the expected maximum value for the operating conditions). In such cases, multi-step or more gradual updates will help avoid erroneously triggering such algorithms. Multi-step or more gradual updates are also more likely to be used in scenarios where a large difference between the actual state and the command state has been established (e.g., as may be more common during certain mode switching).
[0090] In step 910, the command state is applied to control the actual state of the manipulator arm assembly. The control task of step 910 can be performed by a method similar to Fig. 7A and Figure 7B The control architecture shown in FIG.
[0091] if Fig. 9 The method is performed in Cartesian space, such as Fig. 7A The entire control loop shown may be affected by the command state. One or more joints of the manipulator arm assembly may be driven based on the command state. However, the manipulator arm assembly may also include joints that are not affected by the command state, such as passive joints (i.e., joints that are not equipped with joint actuators).
[0092] Alternatively, if Fig. 9 The method is executed in the joint space, then Fig. 7A The control loop of the connector to be controlled may be affected by the command status. Fig. 9 methods, such as Fig. 9 As described, multiple joints can be controlled.
[0093] Whether the steps described are performed for the entire manipulator arm assembly or for a single joint, reducing or limiting energy accumulation can reduce the possibility of undesirable situations. More specifically, due to the difference between the command state and the actual state of the proportional control gain provided to one or more joint actuators associated with the manipulator segment is reduced (by setting the command state to be closer to the actual state), the energy accumulation in these manipulator segments is limited. Limiting energy accumulation can be used in conjunction with various control tasks that may be beneficial (except for control tasks specifically used to promote external reverse drive, such as "floating" of one or more joints). Examples of control tasks that can benefit from the described method for reducing or limiting energy accumulation in a servo control mechanism include, but are not limited to, position holding tasks, remote operation tasks, and positioning tasks (including positioning tasks using position time (PT) and position velocity time (PVT) interpolation). Two specific examples are subsequently described.
[0094] although Fig. 9 The above description of the flowchart covers a method for servoing at least one joint of a manipulator arm assembly in a non-clutch mode, but other methods may be performed for the same manipulator arm assembly without departing from the present disclosure. In one embodiment of the present disclosure, one or more joints of the manipulator assembly or the entire manipulator assembly may be operated in a clutch mode. In addition, one or more joints may be in a clutch mode while other joints may be in a manipulation mode. In clutch mode, one or more joints may be "floating", allowing an assistant to easily articulate these one or more joints externally (e.g., by back-driving these one or more joints). The floating joint may be back-driven by an external force without a control algorithm to counteract the back-drive. For example, a user may apply a force to a link distal to the floating joint, resulting in back-drive. As previously discussed, the floating joint may be compensated by gravity and / or friction to further facilitate back-drive. Various clutch modes may be implemented. In one embodiment of the present disclosure, the clutch mode allows a user to move the remote center of the manipulator arm while allowing or preventing the end effector from moving. In one embodiment of the present disclosure, the clutch mode allows a user to move at least some joints of the manipulator arm while preventing the tool or end effector from moving. Combinations of the described clutch modes and additional clutch modes may be implemented for the manipulator assembly without departing from the present disclosure.
[0095] The following examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Fig. 10ASchematically illustrates a manipulator assembly (1000) in a position holding task, i.e., the controller is in a maneuvering mode, in which the manipulator assembly is servo-controlled to hold position, according to one or more embodiments. In the position holding task, the manipulator assembly is controlled to remain stationary at a command position (1002) (the configuration of the manipulator assembly at the command position is shown using dashed lines). Reference Figure 1A and Figure 1B The configuration shown in , can use position holding tasks during phases of the robotic task in which the user control system (120) does not provide any input (e.g., because the manipulator assembly is not currently under teleoperation control) and the manipulator assembly is required to remain stationary. Thus, in the absence of external interference, the commanded position is maintained. Fig. 10A In , an external force is applied (1004) to backdrive the manipulator assembly. As a result, the actual position becomes the backdrive position (1006) (the configuration of the manipulator assembly at the actual position is shown using a solid line). When the manipulator assembly is backdriven by an external force, energy begins to accumulate in the joints of the manipulator assembly as the actual positions of the joints deviate from the joint positions required to maintain the commanded position. Once the error threshold is exceeded due to backdrive, the commanded position is updated. In Fig. 10A In the example of , the command position is updated to be close to the backdrive position (1006). Therefore, even in the backdrive position (1006), the joint actuator can still produce maximum torque / force, based on the fact that the saturation limit of the joint actuator has been reached due to the small residual difference between the updated commanded position and the backdrive position. Once the backdrive stops, the manipulator arm can perform very small movements to reach the updated command position. The small movement is barely noticeable to an operator observing the manipulator arm, or is considered acceptable by the operator given the operating environment. Although in this example, the position holding task is described in the context of an end effector controlled in Cartesian space, the position holding task can be performed in a similar manner for a single joint.
[0096] According to one or more embodiments, the described method can be particularly beneficial when switching from a control mode that does not implement the method to a control mode that implements the method for limiting energy accumulation. For example, assume that the manipulator joint is controlled in a remote operation mode (e.g., using input provided by a user via an input control device). In addition, assume that the remote operation mode does not implement the method for limiting energy accumulation. When switching to a control mode that implements the method for limiting energy accumulation (e.g., a holding mode), excess energy in the control loop is dissipated. More specifically, the possible difference between the actual position and the command position that exceeds the difference that provides joint torque saturation is reduced to a level that is barely sufficient to maintain saturation. Therefore, the joint torque remains constant during the switch (the joint torque is in saturation before and after the switch), but the excess energy is dissipated.
[0097] According to one or more embodiments, Fig.10B1 schematically illustrates a manipulator assembly (1050) in a teleoperation (ie, "following") task, Fig.10B2 A trajectory (1070) in a teleoperation mission is schematically shown (e.g., the controller is in a maneuvering mode in which the manipulator assembly is servo-controlled to follow the incremental changes in position specified by the trajectory (1070). Consider, for example Figure 1A and Figure 1B The master-slave configuration shown in Figure 2 As shown, the command trajectory can be obtained from the input control device (210) under the remote control of the operator. The command trajectory can also be obtained from any other source. The command trajectory can be specified by at least the end points and can further include intermediate points. Position time (PT) and position velocity time (PVT) path interpolation algorithms can be used to establish the command trajectory based on the specified points.
[0098] In the following tasks, the manipulator assembly is controlled to follow a command position (1052) (the configuration of the manipulator assembly at the command position is shown using dashed lines). Fig.10B1 In FIG. 1 , it is assumed that the command position (1052) is only one point of the command trajectory. The next command position may cause movement elsewhere, or alternatively, the next command position may be in the same position to keep the manipulator assembly stationary. In the absence of external interference (e.g., obstacles), the manipulator arm assembly will reach the command position (1052).
[0099] However, in Fig.10B1, an obstacle (1054) prevents the manipulator arm from reaching the command position (1052). The actual position of the manipulator assembly is stuck in the blocking position (1056) (the manipulator assembly configuration in the blocking position is shown using solid lines). Since the manipulator assembly cannot reach the command position, the command torque of the joint actuator of the manipulator assembly may be saturated. When the obstacle (1054) is removed, the manipulator assembly can move to the command position (1052). This may occur if the difference between the command position and the blocking position remains below the error threshold, thereby not resulting in an update of the command position. However, if the error threshold is exceeded, the command position will be updated. In this case, the manipulator assembly does not move to the command position. Instead, the manipulator assembly can remain at or near the blocking position, which now corresponds to the updated command position. As a result, when the manipulator arm assembly is under the control of an input control device operated by an operator, the manipulator arm may eventually follow the operator's command with an offset (or an updated offset, if an offset already existed before hitting the obstacle).
[0100] Go to Fig.10B2 , the figure shows the additional subsequent tasks. Initially, the command state of the manipulator assembly tracks the command trajectory (1072). However, due to the presence of an obstacle (1074), the command trajectory may only be partially completed. More specifically, the movement in the y-direction is blocked due to the obstacle, but the movement in the x-direction is not blocked. The portion of the command trajectory that cannot be completed is shown using a dotted style. When the manipulator assembly contacts the obstacle, the actual y-position no longer follows the command y-position on the command trajectory. Eventually, the difference between the command y-position and the actual y-position exceeds the error threshold, and the command y-position is updated as a result. The updated command y-position is close to the actual y-position. Therefore, compared with the initial command y-position, the updated command y-position has a tracking offset. If there is already a previous tracking offset, the tracking offset may change due to encountering an obstacle. As subsequent tasks continue to be performed on the command trajectory, the tracking offset can be maintained. Therefore, in this example, the command state of the manipulator assembly tracks the command trajectory with a tracking offset in the y-direction. No tracking offset is introduced in the x-direction. Alternatively, as the subsequent tasks continue to be executed, the tracking offset can be gradually reduced, and thus the tracking offset can be eliminated over time. The tracking offset can be reduced by any function, such as an exponential decay function. The time constant of the exponential decay function can be selected as needed.
[0101] Additionally, a ratchet feature can be used to reduce tracking offset. The ratchet feature reduces tracking offset in a direction-dependent manner. Fig. 9 , when the desired state moves toward Fig. 9As the actual state in the desired state moves, the offset used to calculate the command state can be gradually reduced, thereby allowing the command state to gradually approach the desired state until the command state finally reaches the desired state. The reduction of the offset can be performed exponentially, linearly, or using any other function in a manner that depends on the offset. As the desired state moves away from the actual state, the offset can be kept constant. A ratchet behavior is thereby obtained, wherein when moving in one direction, the offset is reduced, while when moving in the opposite direction, the offset is prevented from increasing.
[0102] As Fig.10B2 Although the command state may initially move on the command trajectory, once an obstacle is encountered, the command state may be prevented from advancing on the command trajectory. Once the obstacle is removed, the command state may resume moving on the command trajectory. In this alternative implementation, no tracking offset will be introduced.
[0103] Assuming that for the following tasks described, the command trajectory is provided by an operator controlling the input control device, the user may receive a tactile cue when the actual position of the manipulator assembly no longer corresponds to the command position on the command trajectory. The tactile cue may be, for example, a vibration of the input control device.
[0104] Although in the examples the following tasks are described in the context of an end effector controlled in Cartesian space, they can be performed in a similar manner for a single joint.
[0105] The embodiments of the present disclosure are suitable for various applications. Figure 1A , Figure 1B , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6The manipulator assembly described in the invention can promote and / or improve the use of the robot system and the interaction with the robot system. In one embodiment, these methods can make the manipulator assembly maintain its position or follow a trajectory unless an error threshold is exceeded. This can allow the manipulator assembly to perform a task (such as a remote surgical task). Once the error threshold is exceeded, the manipulator assembly can be allowed to slide, thereby allowing reverse driving. Therefore, if the error threshold is properly set, a continuously available sliding behavior that does not interfere with the execution of the task is provided. The sliding behavior may be advantageous in various situations. Specifically, sliding can reduce the risk associated with the manipulator assembly colliding with another object. Once the obstacle blocking the movement is removed, the sudden movement of the manipulator assembly will not occur. Sliding can further allow the manipulator arm to be reverse driven. In addition, for example, when the joint is switched from a floating control mode to a position holding control, sliding can allow a seamless transition from one control mode to another control mode. In order to obtain the desired characteristics, the sliding behavior can be customized.
[0106] As a first example, whether in Cartesian space or in joint space, the error threshold can be adjusted upward to prevent slippage at small energy accumulations. Similarly, the error threshold can be adjusted downward to allow slippage at relatively small energy accumulations.
[0107] As a second example, the error threshold may be set differently depending on the type of manipulator assembly (e.g., the type of manipulator arm and / or the type of tool supported by the manipulator arm). For example, assume a scenario involving multiple manipulator arms. Such a scenario is common in telesurgery applications. One or more manipulator arms may be equipped with a manipulator tool, while another manipulator arm may be equipped with a tool including an imaging device (e.g., an endoscope). In such a scenario, different error thresholds for different manipulator arms may be advantageous. It may be desirable that the manipulator arm with the imaging device is more resistant to sliding than any other manipulator arm to ensure that the field of view provided to the surgeon by the imaging device does not change when a collision occurs between one of the manipulator arms carrying the tool and the manipulator arm equipped with the imaging device. Similarly, the use of certain tools uses greater forces than other tools, and the selection of the error threshold may be based at least on the expected forces associated with the tool and reflect that difference.
[0108] As a third example, different joints of the same manipulator assembly may be configured differently to obtain desired characteristics of the associated manipulator arm segment. For some joints, a relatively low error threshold may be selected, while for other joints, a higher error threshold may be selected. In addition, for some joints, such as for joints configured to float, a sliding feature may not be available at all. In one example, a sliding feature may not be applicable to the remote center of the manipulator arm. This design choice can ensure that the controller always attempts to align the remote center with a hole through which the tool is attached to the manipulator arm. Control of the remote center can be performed in Cartesian space. Therefore, the sliding feature may not be applicable to joints in the direction of the joint velocity that affects or may affect the displacement of the remote center. However, the sliding feature may be applicable to one or more joints along the direction of the joint velocity, for example, which allows the end effector or another part of the manipulator assembly to slide without sliding the remote center. Consider, for example, a manipulator assembly with built-in kinematic redundancy (e.g., Figure 5 ). Redundancy can allow reconfiguration of some or all of the manipulator arm joints while keeping the tool or end effector of the tool fixed, or while keeping the remote center fixed. Such reconfiguration can be beneficial when the assistant notices an increased risk of collision of the manipulator arm with an object (e.g., another manipulator arm). The reconfiguration will change the configuration of the manipulator arm to reduce the risk of collision. Instant reconfiguration can be facilitated by setting a relatively low error threshold in Cartesian space. The assistant can simply push the manipulator arm toward the desired configuration. While the manipulator arm remains under active control to maintain the position of the end effector (or remote center), some joints may be allowed to slide in response to the assistant pushing the manipulator arm, thereby completing the reconfiguration move without disturbing the position and orientation of the end effector (or remote center position).
[0109] As a fourth example, sliding features can be implemented on one or more joints of a manipulator assembly in Cartesian space or joint space to address erroneous or unexpected behavior of the manipulator assembly, which may be caused, for example, by a user specifying an incorrect command via a user control system. Specifically, an assistant close to the manipulator assembly can prevent such behavior by manually blocking erroneous or unexpected movements. The controller can be configured to stop executing erroneous or unexpected movements when the movement is blocked for at least a specified time interval. If the blocking is short-lived (i.e., less than a specified time interval, which may be caused by an unexpected interference caused by the assistant to the movement of the manipulator assembly), the controller can be configured to continue executing the movement. Conversely, when the blocking lasts for at least a specified time interval, the controller can be configured to abandon the movement, for example, by stopping the execution of the movement while partially executing the movement.
[0110] While the present invention has been described with respect to a limited number of embodiments, those skilled in the art having benefit of this disclosure will appreciate that other embodiments can be designed which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the present invention should be limited only by the appended claims.
Claims
1. A computer-assisted medical system, include: Manipulator arm; and a controller including a computer processor, the controller being configured with at least a non-clutch mode, wherein when in the non-clutch mode, the controller is configured to servo at least one joint associated with at least one manipulator arm segment of the manipulator arm, the servoing comprising executing a servo loop, the servo loop comprising: Get the actual state of the manipulator arm, calculating a difference between a command state and the actual state, the command state being used to servo the at least one joint, determining whether the difference exceeds an error threshold, Based on determining that the difference does exceed the error threshold: updating the command state using the offset to reduce the difference, and applying the command state to control the actual state, and Based on determining that the difference does not exceed the error threshold: does not update the command status, and The command state is applied to control the actual state.
2. The computer-aided medical system according to claim 1, wherein said servoing of said at least one joint is performed by a feedback controller including a proportional control gain, and Wherein updating the commanded state reduces energy accumulation in the servo loop, the energy accumulation associated with the proportional control gain operating on the difference between the commanded state and the actual state. 3 . The computer-assisted medical system of claim 1 , wherein the command state and the actual state each comprise a joint state. 4 . The computer-assisted medical system according to claim 3 , wherein the joint state comprises a joint position, a joint speed, a joint torque or a joint force.
5. The computer-assisted medical system of claim 1, wherein the commanded state and the actual state each comprise a state in a Cartesian reference frame.
6. The computer-assisted medical system according to claim 5, wherein the command state and the actual state are respectively include: End effector position, end effector orientation, end effector velocity, or end effector force.
7. The computer-assisted medical system according to claim 1, wherein the at least one joint comprises a first joint and a second joint, wherein determining whether the difference exceeds the error threshold is performed for the first joint and the second joint respectively, wherein the error threshold comprises a first error threshold for the first joint and a second error threshold for the second joint, and The first error threshold is different from the second error threshold.
8. The computer-assisted medical system of claim 1, wherein the error threshold is selected based on at least one of: a type of tool supported by the manipulator arm and desired characteristics of the at least one manipulator arm segment.
9. The computer-assisted medical system according to any one of claims 1 to 8, wherein the non-clutch mode is a mode for performing a task selected from the group consisting of: a position keeping task, a following task, and a positioning task.
10. The computer-assisted medical system according to claim 9, wherein the non-clutch mode is a mode for performing the position holding task or the following task, and Wherein when performing the position keeping task or the following task, updating the command state includes setting the command state to be very close to the actual state.
