Systems and methods for controlling a robotic manipulator or associated tool

Abstract

The present application is titled "Systems and methods for controlling a robotic manipulator or associated tool". A system includes a robotic manipulator for controlling motion of a medical tool. The robotic manipulator includes a joint and a link connected to the joint. The link is configured to be connected to the medical tool. A processing unit of the system is configured to receive first data from an encoder of the joint. A first tool tip estimate of a first parameter of a tool tip coupled at a distal end of the medical tool is generated using the first data. The first parameter of the tool tip is a position or a velocity of the tool tip. Second data is received from a sensor system located at a sensor portion of the link or at the medical tool. The joint is controlled based on a first difference between the first tool tip estimate and a second tool tip estimate generated using the first data and the second data.

Description

[0001] This application is a divisional application of Chinese patent application 2018800719815, filed on November 9, 2018, entitled "System and method for controlling a robot manipulator or related tool".

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Application 62 / 584,377, filed November 10, 2017, the entire contents of which are incorporated herein by reference. Technical Field

[0004] This disclosure relates to systems and methods for executing robot programs, and more particularly to systems and methods for control, and even more particularly to systems and methods for controlling robot manipulators or tools associated with robot manipulators. Background Technology

[0005] Robotic manipulator components can be operated to control the movement of tools within a workspace. For example, such manipulators can be used to perform both non-medical and medical procedures. As a specific example, remotely operated surgical manipulators can be used to perform minimally invasive medical procedures.

[0006] In medical technology, the aim is to reduce the amount of tissue damaged during medical procedures, thereby minimizing patient recovery time, discomfort, and harmful side effects. For example, minimally invasive techniques can be performed through natural openings in the patient's anatomy or through one or more incisions. Through these natural openings or incisions, clinicians can insert medical instruments to reach target tissue locations. Minimally invasive medical tools include instruments such as therapeutic, diagnostic, and surgical tools. Minimally invasive medical tools can also include imaging tools such as endoscopes, which provide the user with visualization within the patient's anatomy.

[0007] Robotic manipulators can be remotely operated or computer-aided. For example, a tool can be held by a robotic manipulator assembly to execute a program. However, the flexibility of such robotic manipulator assemblies and tools can lead to under-damped vibrations and undesirable long settling times. Flexibility (compliance) can be used to measure physical compliance, mechanical compliance, structural compliance, and the ability to deflect under load. In examples where the robotic manipulator assembly (including its base, joints, and links) has relatively large flexibility and / or relatively large link mass or inertia, commanded movements or external disturbances to the robotic manipulator assembly can cause such vibrations. Thus, a combination of physical parameters including compliance (e.g., physical, mechanical, and structural compliance), damping (e.g., physical damping including viscous damping, where viscous damping elements (e.g., lubricant friction) resist movement with a force proportional to the velocity of motion), and mass / inertia results in lower mechanical resonances, while the damping is less than desired. During programming, command movements or external disturbances may excite these low-level mechanical resonances, resulting in undesirable vibrations. Such vibrations experienced at the tool tip or other control points on the robot manipulator during program execution can degrade system performance. For example, this vibration may make it difficult for a computer-aided system to achieve or follow the tool's command trajectory.

[0008] Such vibrations can negatively impact the control of all types of robotic systems, including medical robotic systems. In the example of medical robots, such vibrations can make it difficult for the medical robotic system to perform commanded manipulation of tissues, movement of imaging systems, insertion of needles, application of sutures, etc. For another example, in some implementations, the tool moves around a remote center of motion (also referred to as the "remote center") during part or all of the procedure. In some cases, vibrations can cause the remote center of motion to shift during surgery or other medical procedures and exert undesirable forces on the body wall at the entry port. Vibrations can cause the actual posture or movement of the tool (or the manipulator holding the tool) to deviate from the commanded posture or movement to such an extent that the tool (or manipulator) behaves as if the remote center has moved beyond a predetermined tolerance from its defined position. That is, the range of positioning of the virtual pivot point associated with the movement of the manipulator and / or tool is caused by vibration, exceeding the tolerance amount from the defined remote center of motion.

[0009] Therefore, systems and methods are needed to provide better control over robotic systems, such as better control of the manipulators of robotic systems by mitigating unwanted vibrations, better control of the tips of tools held by robotic systems, and better control of the movement of manipulators or tools supported by manipulators around the remote center of motion of the tool. Summary of the Invention

[0010] Embodiments of the invention are summarized by the appended claims.

[0011] In one illustrative embodiment, a system includes a robotic manipulator configured to control the movement of a medical tool, the robotic manipulator including a joint and a link connected to the joint. The link is configured to connect to the medical tool. The system also includes a processing unit with one or more processors. The processing unit is configured to: receive first data from an encoder of the joint; and use the first data to generate a first tool tip estimate of a first parameter of a tool tip coupled to a distal end of the medical tool, wherein the first parameter of the tool tip is the position or velocity of the tool tip. The processing unit is further configured to receive second data from a sensor system located at a sensor portion of the link or at the medical tool; use the first data and the second data to generate a second tool tip estimate of the first parameter of the tool tip; and control the joint based on a first difference between the first tool tip estimate and the second tool tip estimate.

[0012] In another illustrative embodiment, a method includes receiving first data from an encoder at a joint of a robotic manipulator. The robotic manipulator includes a link connected to the joint. A medical tool is connected to the link. The first data is used to generate a first tool tip estimate of a first parameter coupled to a tool tip at the distal end of the medical tool. The first parameter of the tool tip is either the position or velocity of the tool tip. Second data is received from a sensor system located at a sensor portion of the link or at the medical tool. The first and second data are used to generate a second tool tip estimate of the first parameter of the tool tip. The joint is controlled based on a first difference between the first and second tool tip estimates.

[0013] In another illustrative embodiment, the non-transitory machine-readable medium includes a plurality of machine-readable instructions, which, when executed by one or more processors, are adapted to cause the one or more processors to perform a method. The method includes receiving first data from an encoder at a joint of a robotic manipulator. The robotic manipulator includes a link connected to the joint. A medical tool is connected to the link. A first tool tip estimate is generated using the first data, specifying a first parameter of a tool tip coupled to a distal end of the medical tool. The first parameter of the tool tip is either the position or velocity of the tool tip. Second data is received from a sensor system located at a sensor portion of the link or at the medical tool. A second tool tip estimate is generated using the first data and the second data, specifying a second tool tip estimate of the first parameter of the tool tip. The joint is controlled based on a first difference between the first tool tip estimate and the second tool tip estimate.

[0014] It should be understood that the foregoing overview and the following detailed description are exemplary and illustrative in nature and are intended to provide an understanding of this disclosure without limiting its scope. In this regard, other aspects, features, and advantages of this disclosure will be apparent to those skilled in the art from the following detailed description. Attached Figure Description

[0015] The various aspects of this disclosure can be best understood in conjunction with the accompanying drawings, based on the following detailed description. It should be emphasized that, in accordance with standard industry practice, the various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or decreased for clarity. Furthermore, reference numerals and / or letters may be repeated in various examples within this disclosure. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed.

[0016] Figure 1A This is a schematic diagram of a robotic medical system according to an embodiment of the present disclosure.

[0017] Figure 1B This is a perspective view of a manipulator assembly according to an embodiment of the present disclosure.

[0018] Figure 1C This is a perspective view of a console for an operator of a robotic medical system according to an embodiment of the present disclosure.

[0019] Figure 2A , Figure 2B and Figure 2C Various embodiments of a robot arm assembly with a load-side sensor according to this disclosure are shown.

[0020] Figure 3A block diagram of a master / slave control system according to an embodiment of the present disclosure is shown.

[0021] Figure 4A A block diagram of a connector controller according to an embodiment of the present disclosure is shown.

[0022] Figure 4B A block diagram of a connector controller according to an embodiment of the present disclosure is shown.

[0023] Figure 5 This is a flowchart of a method for controlling a manipulator and associated tools according to embodiments of the present disclosure. Detailed Implementation

[0024] For the purpose of promoting an understanding of the principles of this disclosure, reference will now be made to the embodiments illustrated in the accompanying drawings, and they will be described using specific language. However, it will be understood that this is not intended to limit the scope of this disclosure. Numerous specific details are set forth in the following detailed description of various aspects of the invention in order to provide a thorough understanding of the disclosed embodiments. However, it will be apparent to those skilled in the art that embodiments of this disclosure can be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the invention.

[0025] As would be commonly apparent to those skilled in the art in relation to this disclosure, any changes and further modifications to the apparatus, tools, and methods described in connection with the principles of this disclosure, and any further applications, are contemplated. In particular, it is fully contemplated that features, components, and / or steps described with respect to one embodiment can be combined with features, components, and / or steps described with respect to other embodiments of this disclosure. Furthermore, the dimensions provided herein are for specific examples, and it is contemplated that the concepts of this disclosure can be implemented using different sizes, dimensions, and / or ratios. To avoid unnecessary descriptive repetition, one or more components or actions described according to one illustrative embodiment may be used or omitted where applicable in other illustrative embodiments. For brevity, many iterations of these combinations will not be described separately. For simplicity, in some cases, the same reference numerals are used in all figures to refer to the same or similar parts.

[0026] While some of the examples described herein frequently involve medical procedures and medical tools, the disclosed techniques are also applicable to non-medical procedures and non-medical tools. For example, the tools, systems, and methods described herein can be used for non-medical purposes, including industrial applications, general robotic applications, manipulation of non-tissue artifacts, and / or cosmetic enhancements. Other non-surgical applications include use on tissue removed from human or animal anatomy (without returning the human or animal anatomy) or on human or animal cadavers.

[0027] The following examples will describe various tools and parts thereof according to their state in three-dimensional space. As used herein, the term "position" refers to the location (place) of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom, such as along the Cartesian X, Y, Z axes, which can be described using variations in Cartesian X, Y, Z coordinates). As used herein, the term "orientation" refers to the rotational placement of an object or part of an object (three rotational degrees of freedom, such as roll, pitch, and yaw, which can be described using roll, pitch, and yaw). As used herein, the term "pose" refers to the position of an object or part of an object in at least one translational degree of freedom, and the orientation of an object or part of an object in at least one rotational degree of freedom. For an asymmetric rigid body in three-dimensional space, a complete pose can be described using six total degrees of freedom.

[0028] See attached diagram. Figure 1A This illustrates an exemplary robotic system. Specifically, in Figure 1A In this context, computer-aided robotic medical systems, typically denoted by reference numeral 10, can be remotely operated and used, for example, in medical procedures including diagnostic, therapeutic, or surgical procedures. As will be described, the remote operating system of this disclosure is under the remote operational control of an operator. In some embodiments, the manipulators or other parts of the robotic system can be directly controlled through manual interaction with the manipulators (or other parts) themselves. Thus, the term "remotely operated manipulator" as used in this application includes manipulators that can be controlled solely through remote operation, as well as manipulators that can be controlled through both remote operation and direct manual control. Furthermore, in some embodiments, non-remotely operated or remotely operated medical systems may be under partial control of a computer programmed to execute a program or subroutine. In other alternative embodiments, a fully automated medical system under the complete control of a computer programmed to execute a program or subroutine may be used to execute the program or subroutine.

[0029] like Figure 1AAs shown, the remotely operated medical system 10 typically includes a manipulator assembly 12 mounted on or near an operating table O, where a patient P is positioned. The manipulator assembly 12 may be referred to as a patient-side trolley in this example, as it includes a trolley and is designed for use alongside the patient. A first medical device system 14 and a second medical device system 15 are operatively coupled to the manipulator assembly 12. In this disclosure, the first medical device system 14 may also be referred to as medical tool 14, and the second medical device system 15 may also be referred to as medical tool 15. Furthermore, in the remainder of this disclosure, for ease of illustration, the second medical device system 15 is generally described as having imaging capabilities; in those cases, the second medical device system 15 may also be referred to as imaging system 15. However, it is contemplated that either or both of medical devices 14 and 15 may not have non-imaging capabilities and / or imaging capabilities. Imaging system 15 may include an endoscopic imaging system using optical imaging techniques, or other types of imaging systems using other techniques (e.g., ultrasound, fluorescein, etc.). The operator input system 16 allows operators such as surgeons or other types of clinicians S to view images of the surgical site or images representing the surgical site and to control the operation of medical tools 14 and / or medical tools 15.

[0030] The operator input system 16 for remotely operating the medical system 10 can be "mechanically grounded" by connecting to a base with an linkage device such as an operator's console, or it can be "mechanically ungrounded" and therefore not connected. Figure 1AAs shown, the operator input system 16 is connected to an operator console, which is typically located in the same room as the operating table O during the procedure. However, it should be understood that the operator S may be located in a different room or a completely different building from the patient P. The operator input system 16 typically includes one or more control devices for controlling the medical tool 14. (The operator input system 16 is also referred to herein as a "master manipulator," a "master input device," and an "input device.") The control devices may include any one or more of a variety of input devices, such as handles, joysticks, trackballs, data gloves, trigger guns, foot pedals, manual controllers, voice recognition devices, touchscreens, body motion or presence sensors, etc. In some embodiments, the control devices(s) are provided with the same degrees of freedom as the medical tool as the robotic component to provide telepresence to the operator. That is, the operator feels that the control devices(s) are integrated with the tool, giving the operator the feeling of directly controlling the tool as if present in the surgical field. In other embodiments, the control devices(s) may have more or fewer degrees of freedom than the associated medical tool and still provide telepresence to the operator. In some embodiments, the control device(s) are manual input devices that move in six degrees of freedom and may also include an actuable handle for actuating medical tools (e.g., for closing a jaw end effector, applying a potential to an electrode, delivering medication, etc.).

[0031] While the operator S views the surgical site via the operator's console, the manipulator assembly 12 supports and manipulates the medical instrument 14. Images of the surgical site can be acquired by the medical instrument 15, such as via an imaging system including a monocular or stereoscopic endoscope, which can be manipulated by the manipulator assembly 12 to orient the medical instrument 15. An electronic trolley can be used to process the images of the surgical site for subsequent display to the operator S via the operator's console. The number of medical instruments 14 used at one time will typically depend on the medical diagnostic or treatment (e.g., surgery) procedure, space constraints within the operating room, and other factors. The manipulator assembly 12 may include kinematic structures and robotic manipulators of one or more non-servo-controlled linkages (e.g., one or more linkages that can be manually positioned and locked in place). The manipulator assembly 12 includes multiple motors that drive inputs on the medical instrument 14. These motors move in response to commands from a control system (e.g., control system 20). The motors include a drive system that, when coupled to the medical instrument 14, can advance the medical instrument into a naturally occurring or surgically created anatomical opening. Other motorized drive systems can enable the distal end of a medical device to move in multiple degrees of freedom, including three linear motions (e.g., linear motion along the X, Y, Z Cartesian axes) and three rotational motions (e.g., rotation about the X, Y, Z Cartesian axes). Additionally, motors can be used to actuate articulated end effectors of tools to grip tissue in the jaws of biopsy devices, etc. Medical tool 14 may include an end effector with a single working component, such as a scalpel, blunt blade, needle, imaging sensor, optical fiber, electrode, etc. Other end effectors may include multiple working components, and examples include forceps, grippers, scissors, applicators, staplers, bipolar cautery instruments, etc.

[0032] The telemedicine system 10 also includes a control system 20. The control system 20 includes at least one memory 24 and at least one processor 22, and typically includes multiple processors for implementing control between the medical instrument 14, the operator input system 16, and other auxiliary systems 26, which may include, for example, imaging systems, audio systems, fluid delivery systems, display systems, lighting systems, steering control systems, flushing systems, and / or suction systems. The control system 20 also includes programmable instructions (e.g., a computer-readable medium storing the instructions) to implement some or all of the methods described according to the aspects disclosed herein. Figure 1AIn the simplified schematic, the control system 20 is shown as a single block, but the system may include two or more data processing circuits, with some processing optionally executed on or near the manipulator component 12, others at the operator input system 16, and so on. Any of a wide variety of centralized or distributed data processing architectures can be employed. Similarly, the programmed instructions can be implemented as multiple separate programs or subroutines, or they can be integrated into multiple other aspects of the remote operating system described herein. In one embodiment, the control system 20 supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry.

[0033] In some embodiments, the control system 20 may include one or more servo controllers that receive force and / or torque feedback from the medical tool 14 or from the manipulator assembly 12. In response to this feedback, the servo controllers send signals to the operator input system 16. The servo controllers (one or more) may also send signals to instruct the manipulator assembly 12 to move (one or more) the medical tool 14 and / or 15, which extends through an opening into the body to an internal surgical site within the patient. Any suitable conventional or specialized controller may be used. The controller may be separate from or integrated with the manipulator assembly 12. In some embodiments, the controller and the teleoperation assembly are part of an integrated system, such as a teleoperated arm cart located near the patient's body during medical procedures.

[0034] The control system 20 can be coupled to the medical instrument 15 and can include a processor to process the captured images for subsequent display, such as displaying them to a console used by an operator or an operator wearing a head-mounted display system, displaying them to one or more stationary or movable monitors located near the control system, or displaying them on another suitable display located locally and / or remotely. For example, in the case of using a stereoscopic endoscope, the control system 20 can process the captured images to present the operator with a coordinated stereoscopic image of the surgical site. This coordination can include alignment between the stereoscopic images and can include adjusting the stereoscopic working distance of the stereoscopic endoscope.

[0035] In alternative embodiments, the robotic system may include more than one manipulator component and / or more than one operator input system. Among other factors, the exact number of manipulator components will depend on the surgical procedure and space constraints within the operating room. Operator input systems may be juxtaposed or placed in separate locations. Multiple operator input systems allow more than one operator to control one or more manipulator components in various combinations.

[0036] Figure 1BThis is a perspective view of one embodiment of the manipulator assembly 12, which is constructed in the form of a trolley and positioned near the patient during a medical procedure. Therefore, Figure 1B This remote manipulation component can also be referred to as a patient-side trolley. The illustrated manipulator assembly 12 provides manipulation of three medical tools 30a, 30b, 30c (e.g., medical tool 14) and a medical tool 28 (e.g., medical tool 15) including imaging equipment, such as for capturing images of the workpiece or surgical site (also referred to as the “working site”). Medical tool 28 can transmit signals to control system 20 via cable 56. Manipulation is provided by a robotic manipulator with multiple joints. The kinematic remote center can be maintained at the incision or natural orifice by positioning and manipulating medical tool 28 and surgical tools 30a-c within the patient's incision or natural orifice. When surgical tools 30a-c are within the field of view of medical tool 28, the image of the working site can include an image of the distal end of surgical tools 30a-c.

[0037] The manipulator assembly 12 includes a driveable base 58. The driveable base 58 is connected to a telescopic column 57, which allows adjustment of the height of an arm 54 (also referred to as "manipulator 54"). The arm 54 may include a rotary joint 55 that rotates and translates parallel to the column 57. The arm 54 may be connected to a directional platform 53. The directional platform 53 may be capable of rotating 360 degrees. The manipulator assembly 12 may also include a telescopic horizontal cantilever 52 for moving the directional platform 53 in a horizontal direction.

[0038] In this example, each arm 54 includes a manipulator arm portion 51. The manipulator arm portion 51 can be directly connected to the medical tool 14. The manipulator arm portion 51 can be remotely operable. In some examples, the arms 54 connected to the orientation platform are not remotely operable. Instead, these arms 54 will be positioned as desired before the operator S begins to operate using the remotely operated components.

[0039] Endoscopes and other imaging systems (e.g., medical tool 15) can be provided in various configurations, including those with rigid or flexible structures. Rigid endoscopes include a rigid tube housing a relay lens system for transmitting images from the distal to the proximal end of the endoscope. Flexible endoscopes use one or more flexible optical fibers to transmit images. Digital imaging-based endoscopes may have a “cutting-edge chip” design, where a distal digital sensor (e.g., one or more charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) devices) stores image data. Endoscopic imaging systems can also utilize other imaging techniques, such as ultrasound, infrared, and fluorescence endoscopy. Endoscopic imaging systems can provide viewers with two-dimensional or three-dimensional images. Two-dimensional images provide limited depth perception. Three-dimensional stereoscopic images provide viewers with more accurate depth perception. Stereoscopic endoscope tools employ stereo cameras to capture stereoscopic images of the patient's anatomy. Endoscopic tools can be fully sterilizable components, where the endoscope cables, handles, and shafts are securely connected and hermetically sealed.

[0040] Figure 1C This is a perspective view of the operator's console 38. The operator's console 38 includes a left-eye display 32 and a right-eye display 34 for presenting the operator S with a coordinated stereoscopic view of the surgical environment capable of depth perception. The operator input system 16 of the operator's console 38 includes one or more input control devices 36, which in turn enable the manipulator assembly 12 to manipulate one or more medical instruments 14 and / or 15. The input control devices 36 can be used, for example, to close a clamping jaw end effector, apply a potential to an electrode, deliver medication, etc. In various alternatives, the input control devices 36 may additionally or alternatively include joystick devices, trackballs, data gloves, trigger guns, manual controllers, voice recognition devices, touchscreens, body motion or presence sensors, etc. In some embodiments, the input control devices 36 will provide the same degrees of freedom as their associated medical instruments 14 to provide a remote presentation to the operator S, or to perceive that the input control devices 36 are integrated with the instruments 14, so that the operator S has the feeling of direct control over the instruments 14. In other embodiments, the input control device 36 may have more or fewer degrees of freedom than the associated medical tool, while still providing remote presentation to the operator S. For this purpose, position, force, and tactile feedback sensors (not shown) may be employed to transmit position, force, and tactile feedback from the tool 14 back to the operator S's hand via the input control device 36. The input control device 37 is a foot pedal that receives input from the user's feet.

[0041] As described above, physical compliance in the combination of robotic arm components and tools can cause underdamped vibrations and long settling times. Such vibrations can occur in systems where the robotic arm components are flexible (e.g., including flexible joints) and / or have unbalanced link masses and inertia, caused by operator-input system commands. Such vibrations experienced on the manipulator or the tool supported by the manipulator during medical or non-medical procedures can lead to control problems. For example, such vibrations experienced on the tip of a medical tool during surgery can cause control problems. As a concrete example, such vibrations can make the system difficult to perform, and clinicians may find it difficult to manipulate tissues, needles, and sutures as required. As will be discussed in detail below, by simultaneously using motor-side measurements from motor-side sensors (e.g., joint encoders) and load-side measurements from load-side sensors (e.g., sensors located at the links of the robotic arm components, at the tool body, and / or at the end effector), the master / slave system can control the robotic arm components to reduce those vibrations (e.g., smaller vibration amplitudes, faster settling times, etc.).

[0042] Reference Figure 2A , Figure 2B and Figure 2C In various embodiments, the load-side sensor system can be attached to different load-side locations of the robot system (e.g., on a link of the robot arm assembly, on a tool supported by the robot arm assembly, or on one or more end effectors of the tool). Such a load-side sensor system can transmit signals to the control system (e.g., Figure 1A-1C The example control system 20) provides load-side positioning measurement data, including, for example, the position and motion of the load-side positioning (e.g., linear velocity, linear acceleration, angular velocity). The control system can use the load-side measurement data to control the robot arm assembly to reduce tip vibration and settling time.

[0043] Figure 2A An arm 200 (e.g., arm 54) is shown, on which interchangeable tools 250 are mounted. Tool 250 may also be referred to herein as tool 250. In some embodiments, tool 250 may be configured for manipulating industrial workpieces or for manipulating human or animal tissue for reasons other than medical or diagnostic. In some embodiments, tool 250 may include tools for performing medical procedures. Tool 250 includes a mounting portion 252 and a shaft 254. Figures 2A to 2CIn the example shown, mounting portion 252 includes a mounting member located on the proximal portion of tool 250. The mounting member is configured to removably couple tool 250 to a carrier 253 of a manipulator. Shaft 254 is coupled to end effector 260 via wrist 258. End effector 260 has a tool tip 262. In some embodiments, arm 200 may include a support for a port device (e.g., a cannula for certain medical procedures) that restricts movement of tool 250 relative to arm 200. The tool 250 associated with each arm 200 may also be controlled by an operator via an operator input system (e.g., Figure 1A-1C The example operator inputs the system at point 16 for control.

[0044] More specifically, arm 200 includes a vertical setting link 202 connected via a setting joint 204 to the farthest setting link 206, and a manipulator arm portion 207 (e.g., Figure 1A-1C The example is a control arm 51), which is connected to the farthest setting link 206. The control arm portion 207 may also be referred to herein as a control portion 207. The control arm portion 207 includes a yaw joint 208, a parallelogram pitch mechanism 210, and a spar 214. The yaw joint 208 connects the farthest setting link 206 to the parallelogram pitch mechanism 210. The parallelogram pitch mechanism 210 includes multiple pitch joints 212a, 212b, 212c and links connecting these pitch joints. The spar 214 is connected to the parallelogram pitch mechanism 210 at a spar joint 216. The mounting portion 252 of the tool 250 is mounted on the spar 214.

[0045] Each of the set joint 204, yaw joint 208, pitch joints 212a, 212b, 212c, and spar joint 216 is controlled by a motor, referred herein as the set joint motor, yaw joint motor, pitch joint motor, and spar joint motor. The arm 200 may also include an insertion gear 218 providing insertion and retraction movements. Thus, at least a portion of the arm 200 is configured to move in a fully motorized manner. In this embodiment, the motors are under the control of a control system (e.g., control system 20) and can operate in conjunction with the motors of other arms to take desired postures that facilitate movement over a workpiece (or patient in a medical procedure), tool installation, preparation steps or storage, and other activities. Additionally, encoders and other sensors associated with each motor or joint provide feedback to the control system, allowing the control system to receive data regarding the sensed or detected or determined position, state, and set of the arm 200.

[0046] In some embodiments, arm 200 is mechanically constrained to move tool 250 about a stationary telemotion center 256 (also referred to as "telemotion center 256"). In some embodiments, the telemotion center 256 coincides with the axis of the tool mounted to arm 200. A yaw joint motor provides yaw motion about the telemotion center 256, and a pitch joint motor and a spar joint motor provide pitch motion about the telemotion center 256. For minimally invasive medical procedures, the telemotion center 256 is typically locked at the incision site in the patient's body wall during the procedure, allowing yaw and pitch motions about the telemotion center 256 to perform the intended surgical task. Alternatively, the telemotion center may be located outside the body to allow a greater range of motion without contact with the patient. For various procedures, the telemotion center 256 may be located anywhere suitable for the procedure, such as within a natural orifice or lumen in some medical procedures, or in a convenient location near the workpiece in non-medical procedures. Those with knowledge will understand that motion about the telemotion center may be constrained by the use of software or by physical constraints defined by mechanical components.

[0047] Although each of the yaw joint 208, pitch joints 212a, 212b, 212c, spar joint 216, and insert gear 218 can be controlled by a separate joint or gear controller, the joints and gear controllers can be controlled by a common joint control unit of the common control system 20 (e.g., a master / slave control system), thereby allowing the user (e.g., operator S) to control their associated control devices (e.g., ...). Figure 1A-1C The example operator input system) is used to manipulate the tip or end effector of tool 250 and the manipulator arm portion 207.

[0048] exist Figure 2A In the example, the load-side sensor system 264 is attached to the load-side positioning 266 of the spar 214, which is the last link of the manipulator arm section 207. Figure 2A In a specific example, the distance D is between load-side positioning 266 and spar joint 216. In various examples, load-side positioning 266 can be located at any part of spar 214. Load-side sensor system 264 may include one or more sensors, including, for example, inertial measurement unit (IMU), electromagnetic sensors, shape sensors, torque sensors, optical tracking systems, image tracking systems, hybrid sensor systems, other suitable sensor systems, and combinations thereof. In some examples, the IMU may include an accelerometer configured to measure linear acceleration at load-side positioning 266 and a gyroscope configured to measure angular velocity at load-side positioning 266. In some examples, the IMU may include a magnetometer configured to measure magnetism at load-side positioning 266.

[0049] like Figure 2B and Figure 2C As shown, the load-side sensor system 264 can be located at load-side positioning points on various parts of the arm 200, the tool 250 held by the arm 200, and / or the end effector 260 coupled to the tool 250. Figure 2B In one example, the load-side sensor system 264 is located at a load-side positioning 266 at the proximal end of the tool 250 (e.g., on the mounting portion 252). In other examples, the load-side sensor system 264 may be located at any part of the tool 250, including, for example, the middle of the shaft and the distal end of the tool 250.

[0050] exist Figure 2C In this example, the load-side sensor system 264 may be located at the load-side positioning 266 (also referred to as sensor section 266) on the end effector 260. The load-side sensor system 264 may also be located at the load-side positioning 266 on other parts of the arm 200, such as the links including the parallelogram pitch mechanism 210.

[0051] like Figure 2A , Figure 2B and Figure 2C As shown in the example, different tools 250 and / or end effectors 260 can be mounted to the arm 200 to perform different functions. Figure 2A In the example, because the load-side sensor system 264 is attached to the manipulator arm portion 207, such a single load-side sensor system 264 can be used with different tools 250 and / or end effectors 260. On the other hand, in Figure 2B and Figure 2C In this example, to provide load-side measurement data, it may be necessary to attach the load-side sensor system 264 to each of those tools 250 and / or end effectors 260. Thus, using the load-side sensor system 264 attached to the manipulator arm portion 207 (such as...) Figure 2A As shown, this can be more cost-effective. In some embodiments, the location of the load-side sensor system 264 can be determined based on various system requirements. In some examples, the load-side sensor system 264 is located (e.g., at the manipulator arm portion 207) closer to the motor to improve controllability and allow for higher bandwidth (more aggressive) interference suppression. In other examples, the load-side sensor system 264 is located (e.g., at the tool 250 or end effector 260) closer to the controlled object (e.g., the tool tip) to improve the detection of interference and the observability of the tool tip's state.

[0052] However, in embodiments where the load-side sensor system 264 is not juxtaposed with the tool tip, the control system (e.g., Figure 1A-1CThe example control system 20) may not be able to adjust the tool tip state estimate based on the load-side sensor system 264 by simply combining measurement data from the load-side sensor system 264 and the state estimate based on the motor-side encoder. See below for reference. Figure 3 , Figure 4A , Figure 4B and Figure 5 To address this challenge, control systems (e.g., Figure 1A-1C The example control system 20) can use both a fusion block and a dynamic model to generate tool tip position and motion estimates. By utilizing measurement data from both motor-side and load-side sensors, while taking into account the dynamic relationship between load-side positioning 266 and tool tip 262, more accurate tool tip position and motion estimates can be achieved, resulting in better tool tip control, reduced vibration, and reduced settling time.

[0053] Reference Figure 3 The control system 300 is shown therein (e.g., Figure 1A-1C Example of a control system 20). Control system 300 can be used to control the movement of the arm 200 from the manipulator 207 portion, and thus control the posture and movement of its attached tool 250 and end effector 260, which is controlled by the operator inputting information into the system (e.g., ...). Figure 1A-1C The example operator inputs commands to the system 16). In the following description, the control system 300 is also referred to as a master / slave control system 300.

[0054] Both the master input device and the slave manipulator can include a number of links connected via joints to allow for multiple degrees of freedom of movement. When the operator S moves the operator input system from one position to another during surgery, sensors associated with the operator input system joints provide signals indicative of such commanded movement in the joint space of the master input device (“master joint space”), and sensors associated with the slave manipulator joints provide information indicative of slave manipulator movement (and thus, tool 250 movement) in the slave joint space for feedback purposes. To better detect and control the fine movements of their respective joints (e.g., within a target speed range of 0.0005 to 0.01 radians per second at the joint, including movement during the transition from zero speed to the target speed range), high-resolution encoders can be used for the motor-side sensors.

[0055] The main input processing unit 301 receives main connector position information sampled at a system control processing rate (e.g., 1300 Hz) from the main connector encoder in the operator input system and calculates the connector velocity based on the sensed connector position. The main forward kinematics processing unit 302 receives the main connector position and velocity from the main input processing unit 301 and transforms them from the main connector space to the corresponding position and velocity in the main reference frame (i.e., the reference frame associated with the operator input system) in Cartesian space relative to the operator's reference frame. In some embodiments, the main forward kinematics processing unit 302 performs this transformation using a Jacobian determinant and reference frame-related information. Example operator reference frames that can be used include an eye reference frame (i.e., a reference frame associated with the position of the operator's eyes).

[0056] The scaling and offset processing unit 304 receives Cartesian position and velocity commands from the main forward kinematics processing unit 302, scales the command movement according to a scaling factor selected for the execution procedure, and takes offset into account to generate the desired position and velocity from the tool reference frame (i.e., the reference frame associated with tool 250). Scaling adjustment is useful in cases where a larger movement of the arm 200 relative to the operator input system 16 is desired, but a smaller movement of the manipulator portion 207, to allow for more precise movement of tool 250 at the surgical site. On the other hand, offset determines the corresponding position and / or orientation of the end effector reference frame (e.g., the reference frame associated with the end effector 260 at the distal end of tool 250) relative to the main reference frame in the eye reference frame, for example, in the camera reference frame (i.e., the reference frame associated with the distal tip of the endoscope).

[0057] The simulated slave processing unit 308 receives the desired slave tool reference frame position and velocity commands from the scaling and offset processing unit 304, and restricts the desired slave tool reference frame position, orientation, and velocity to assigned Cartesian limits to perform correct and intuitive operation of tool 250, for example, by keeping tool 250 within its dexterity workspace. The simulated slave processing unit 308 generates simulated slave joint position and velocity corresponding to the restricted slave tool reference frame position and velocity, while ensuring that the generated slave joint position and velocity do not exceed the actual range of motion and maximum velocity of the slave joint (i.e., joint limits), even near the kinematic singularity of the slave kinematics.

[0058] The inverse scaling and offset processing unit 306 receives simulated joint position and velocity commands from the simulated slave processing unit 308 and performs inverse functions on them (opposite to the function of the scaling and offset processing unit 304). The Cartesian controller 307 receives the input from the scaling and offset processing unit 304 as a first input and receives the output from the inverse scaling and offset processing unit 306 as a second input. Then, the Cartesian controller 307 generates an error signal as the difference between the first and second inputs and generates a Cartesian force "F" based on the error signal. CART For example, using the following formula:

[0059]

[0060] Where "K" is the spring constant and "B" is the damping constant. "Δx" is the difference between the Cartesian speed input to the Cartesian controller 307 and the Cartesian position input to the Cartesian controller 307. For orientation errors, the corresponding torque in Cartesian space is determined.

[0061] The main transpose kinematics processing unit 315 receives the Cartesian force F through the summation node 314. CART And using, for example, the Jacobian transpose matrix and kinematic relationships associated with the operator input system, a corresponding torque is generated in the joint space. In a system where the operator input system has a motor-driven joint for range of motion limitation or force feedback, the main output processing unit 316 receives the main torque signal from the main transpose kinematic processing unit 315, generates a current corresponding to the main torque signal, and sends it to the operator input system (e.g., Figure 1A-1C The corresponding main connector motor of the example operator input system 16) supplies current. As a result, whenever the operator S is commanding a position or velocity that exceeds the system's Cartesian or slave connector limits or that would cause the slave actuator portion 207 of arm 200 to kinematically exhibit singularity conditions, the operator S operating this motor-driven operator input system (e.g., operator input system 16) will feel a Cartesian force F. CART .

[0062] While the main input processing unit 301 is receiving the main connector position from the sensors in the operator input system, the slave input processing unit 309 is also receiving slave positions from the sensors in the slave actuator section 207 at a control system processing rate. The slave input processing unit 309 includes: a motor-side input processing unit 320, which receives slave connector measurement data (e.g., slave connector position and motion data) from motor-side sensors (e.g., connector encoders); and a load-side input processing unit 322, which receives load-side measurement data (e.g., position and motion data of load-side positioning 266) from load-side sensors (e.g., load-side sensor system 264). The connector control unit 318 receives slave connector measurement data and load-side measurement data from the slave input processing unit 309, as well as simulated connector commands provided by the simulated slave processing unit 308, and generates a slave torque command signal for the slave connector motor and a main torque feedback command signal for the main connector motor.

[0063] The torque command signal is generated by the connector control unit 318 to drive the connector of the slave actuator until the feedback error calculated in the connector control unit 318 is zero. The output processing unit 310 receives the torque command signals from the connector control unit 318, converts them into appropriate current, and supplies the current to the connector motor of the slave actuator to drive the motor accordingly.

[0064] In some embodiments, the main torque feedback command signal is generated by the joint control unit 318, which reflects the forces acting on the tool 250 or the slave manipulator supporting the tool 250 back to the operator input system (e.g., operator input system 16) so that the operator S can feel them in some form. In some embodiments, the joint control unit 318 generates the main torque feedback command signal based on slave joint position and speed tracking errors. In various embodiments, the slave tracking error can be determined using motor-side tracking error, tool tip tracking error determined by positive kinematics and motor position, tool tip tracking error determined using load-side sensor system 264, load-side tracking error from joint position and speed estimated using load-side sensor system 264, and / or any combination thereof. In some embodiments, the joint control unit 318 generates the main torque feedback command signal based on measurement data from both the slave joint encoder and the load-side sensor system 264. The kinematic mapping unit 311 receives the main torque feedback command signal from the joint control unit 318 and uses the kinematic configuration and previously calculated position information of the fulcrum (e.g., remote center 256) to generate a corresponding Cartesian force at the tip of the tool 250 relative to the camera reference frame of the endoscope.

[0065] Gain 313 adjusts the magnitude of the Cartesian force to ensure system stability while providing sufficient force feedback to the operator S. The gain-adjusted Cartesian force is then passed through summing node 314 and, together with the Cartesian force provided by Cartesian controller 307, is processed by main transpose kinematics processing unit 315 and main output processing 316, as previously described with reference to the processing of the Cartesian force provided by Cartesian controller 307.

[0066] The joint control unit 318 includes a joint controller for each active joint of the slave manipulator section 207 of the arm 200, which is controlled by the master / slave control system 300. Specifically, in cases where the slave manipulator section 207 includes a yaw joint 208, pitch joints 212a, 212b, 212c, a spar joint 216, and an insert gear 218, such as... Figure 2A , 2B In the example arm 200 of 2C, each of these joints or gears may have its own controller, and each driveable mechanical element for the tool wrist and end effector mechanism will also have its own controller.

[0067] Figure 4 shows a block diagram of a connector controller unit 318 (e.g., for controlling the movement of yaw connector 208, pitch connectors 212a, 212b, 212c, spar connector 216, and insertion gear 218 from manipulator section 207, or for manipulating a tool wrist or end effector mechanism, any one or more of several driveable mechanical elements). For the sake of simplicity in the description herein and in the claims, the term "connector" should be understood to include a connector drivetrain, which may include cables, pulleys, gears, spools, and any other driveable mechanical elements that can be used to control degrees of freedom of movement or other mechanical actions associated with the tool or a robotic arm that holds and / or moves the tool. This can be used to perform non-medical or medical procedures. Exemplary medical procedures include, for example, biopsies, imaging procedures, diagnostic procedures, and surgical procedures such as minimally invasive laparoscopic or endoscopic surgery.

[0068] As discussed in detail below, the connector controller unit 318 can use a fusion block to generate a fused estimate of the position and motion of the load-side positioning 266 by fusing both motor-side and load-side measurement data. This fused estimate of the load-side positioning 266 can be more accurate than an estimate based solely on motor-side measurement data, leading to a more accurate estimate of the position and motion of the tool tip 262. Furthermore, in embodiments where the load-side sensor system 264 is not juxtaposed with the tool tip 262, the connector controller unit 318 can use a corresponding dynamic model to consider the dynamic relationship between the load-side positioning 266 and the tool tip 262, as well as the dynamic relationship of the load-side sensor system 264. This can further improve the accuracy of the estimation of the position and motion of the tool tip 262, thereby enabling better control of the tool tip.

[0069] The connector controller unit 318 includes a positive kinematic block 402 that receives connector encoder data 404 (denoted as Θ) from the motor-side input processing unit 320 (e.g., provided by the connector encoder from the manipulator section 207). enc Connector encoder data 404Θ enc This may include joint position data, joint motion (e.g., velocity, acceleration) data, or combinations thereof. In various embodiments, the kinematic equations of the kinematic chain forming the manipulator arm portion 207 can be used to map joint parameters to the configuration of the robot system. The dimensions of the manipulator arm portion 207 and its associated kinematic equations define the volume of the manipulator arm portion 207 and the space accessible to the features associated with it, commonly referred to as the workspace. The forward kinematics block 402 can use forward kinematics to calculate the positions of specific features associated with the manipulator arm portion 207 in the workspace (e.g., links of the manipulator arm portion 207 including the spar 214, tool 250, end effector 260, and tool tip 262).

[0070] In some embodiments, the positive kinematics block 402 can apply the positive kinematics associated with the manipulator arm portion 207 to the joint encoder data 404Θ. enc And generate an estimate 406 (denoted as) for the position and / or motion (e.g., velocity, acceleration) of the load-side positioning 266. In one example, the forward kinematics block 402 can transmit the connector encoder data 404Θ. enc (For example, using a Cartesian transformation) transform from the world reference frame to the load-side sensor system box reference frame (e.g., the reference frame associated with load-side sensor system 264). In the estimation of 406 In the example of velocity estimation including load-side positioning 266, the forward kinematics block 402 can apply the Jacobian function 430 to the joint encoder data 404Θ.enc The corresponding joint speed data is used to generate a speed estimate for load-side positioning 266.

[0071] As shown in the example in Figure 4, based on the connector encoder data 404Θ enc The estimated load-side positioning of 266 is 406. The data is sent to fusion block 408. Fusion block 408 includes a state estimator 410 and a dynamic model unit 412. The state estimator 410 can receive load-side measurement data 414 of the load-side positioning 266 from the load-side input processing unit 322 (e.g., provided by the load-side sensor system 264). In one example, the load-side measurement data 414 may include linear acceleration data of the load-side positioning 266. And the angular velocity data ω, the load-side measurement data 414 are represented as:

[0072] In the example of Figure 4, the state estimator 410 can receive an estimate 406. and measurement data 414 associated with load-side positioning 266 And generate the Cartesian transform 416 of the fused state estimate (represented as) Fusion state estimation 416 This can include estimation of the position / motion of the load-side positioning 266. Various state estimator algorithms can be used to generate fused state estimates 416. The state estimator algorithm includes, for example, Kalman filters and their variants (e.g., extended Kalman filters, lossless Kalman filters, steady-state Kalman filters, etc.), H... ∞ Filters, particle filters, Luenberger observers, Madgwick filters, α-β-γ filters, sliding mode observers, etc. Based on 404Θ data from the connector encoder, a fusion block is used to combine these components. enc The generated estimate 406 The position and motion of the load-side positioning 266 are compared with the measurement data 414 from the load-side sensor system 264 using only the connector encoder data 404Θ. enc The generated estimate 406 Compared to being better captured.

[0073] In some embodiments, the load-side sensor system 264 is not juxtaposed with the tool tip 262. For example, the load-side sensor system 264 may be located on a link (e.g., spar 214) of the manipulator arm portion 207 or on the tool 250 (e.g., at the mounting portion 252 of the tool 250). In such embodiments, the fusion block 408 of the connector controller unit 318 may include a dynamic model unit 412 to take into account the dynamic relationship between the load-side positioning 266 and the tool tip 262. As shown in the example of FIG4, the state estimator 410 may estimate the fusion state 416 of the load-side positioning 266. The data is sent to dynamic model unit 412. Dynamic model unit 412 determines a dynamic model that models the dynamic relationship between load-side positioning 266 and tool tip 262. In various embodiments, the dynamic model can be determined based on the position of load-side positioning 266, various physical characteristics of the arm 200, the tool 250 body, and the end effector 260, such as mass, stiffness, friction, damping, elastic deformation of bearings and gears, deflection of links under load, vibration, etc. The dynamic model can be derived from physical modeling of individual components, dynamic experimental identification, derivation, and a combination of empirical data, and / or dynamic online adaptive identification. Dynamic model unit 412 can generate a Cartesian transformation from a world reference frame to a tool tip reference frame (e.g., a reference frame associated with tool tip 262) and a fused state estimate 416 based on the dynamic model and load-side positioning 266. Generate tool tip 262 fusion state estimation Cartesian transform 418 Then the fusion block 408 can output the fusion state estimate 418 of the tool tip 262.

[0074]

[0075] In some embodiments, the fusion state estimation 418 of the tool tip 262 Fusion state estimation of load-side positioning 266 416 This can include estimations for the same state parameters (e.g., position, velocity, or combinations thereof). In one example, the fused state estimate 418 for the tool tip 262. Fusion state estimation of load-side positioning 266 and load-side positioning 266 416 Each of these includes only position estimates, without any motion (e.g., velocity) estimates. In another example, the fusion state estimate of tool tip 262 is 418. Fusion state estimation of load-side positioning 266 416 All include velocity estimation, but not position estimation. In yet another example, the fusion state estimate 418 for tool tip 262... Fusion state estimation of load side point 266 416 Each of these includes both position and velocity.

[0076] In some embodiments, the positive kinematics block 402 can apply the positive kinematics associated with the manipulator arm 207, tool 250, and end effector 260 to the joint encoder data 404Θ. enc And based on the connector encoder data 404Θ enc The state estimate 420 of the tool tip 262 is generated (represented as) The forward kinematic block 402 can transmit the connector encoder data 404Θ. enc (For example, using a Cartesian transformation) Transform from the world reference frame to the tool tip reference frame. State estimation 420 This can include an estimate of the position / motion of the tool tip 262. In some examples, the positive kinematics block 402 can be based on the connector encoder data 404Θ. enc The joint speed data in the data were used to generate a speed estimate of the tool tip 262 using the Jacobian function 430.

[0077] In some embodiments, the connector controller unit 318 includes a comparator 422. The comparator 422 receives a fusion state estimate 418 of the tool tip 262 from the fusion block 408. and receiving state estimates 420 from the tool tip 262 from the positive kinematics module 402 And generate and The difference in tool tip condition estimates is 424δT. tip The tool tip condition estimation error is 424δT. tip It may include the position estimation difference of the tool tip 262, the velocity estimation difference of the tool tip 262, or a combination of both.

[0078] like Figure 4A As shown, the connector controller unit 318 may include an inverse kinematics block 426. The inverse kinematics block 426 can receive the tool tip state estimation difference 424δT from the comparator 422. tip and receive connector encoder data Θ from connector encoder 404. enc The inverse kinematics block 426 can be used to apply inverse kinematics to estimate the tool tip state difference 424δT. tip Converted to joint adjustment estimate 432 (represented as δΘ) CMD ). Joint adjustment estimate 432δΘ CMD This may include estimates of joint position and / or motion (e.g., speed) adjustments, which are needed to compensate for tool tip condition estimation errors. tipIn some examples, the inverse kinematics block 426 can be estimated based on the tool tip state difference 424δT. tip The tool tip speed estimation difference is used to generate an estimate of the joint speed adjustment using the inverse Jacobian function 428.

[0079] Joint adjustment estimate 432δΘ CMD It can be sent to combiner 434, which will then command header data 436 (represented as Θ′) CMD ) and joint adjustment estimate 432δΘ CMD Combined to generate the adjusted command header data 438 (represented as Θ) CMD In one example, in response to input from the operator system (e.g., Figure 1A-1C The operator input system 16) in the example receives input from the simulated slave processing unit 308, which provides command header data 436Θ′. CMD Adjusted command connector data 438Θ CMD This may include the adjusted command connector position, the adjusted command connector speed, or a combination of both.

[0080] In some embodiments, the connector control unit 318 can be based on the tool tip state estimation difference 424δT tip To generate the torque command signal. This is achieved by estimating the difference of 424δT using the tool tip state. tip The joint control unit 318 generates a torque command signal, which takes into account the tool tip state estimation difference of 424δT. tip The torque command signal can be used to drive the motors of the actuator 207 (e.g., motors for yaw joint 208, pitch joints 212a, 212b and 212c, spar joint 216, insert gear 218).

[0081] In various embodiments, the connector control unit 318 may use a feedback controller to control the connector motor. The feedback controller may include a proportional-derivative (PD) controller, a proportional-integral-derivative (PID) controller, any linear controller (e.g., a lead-lag controller, H... ∞ The controller can be a linear parameter variation controller, sliding mode or other adaptive controller, or any other suitable feedback controller. The torque command signal can be used to drive the actuator joint until the feedback error calculated by the feedback controller returns to zero.

[0082] In some embodiments, the torque command signal can be further adjusted by a torque saturation block, which can limit the command torque value so that the command torque value does not exceed the maximum desired torque value of its respective motor.

[0083] Reference Figure 4B The connector control unit 450 shown is illustrated in the figure. Figure 3 The joint control unit 318 uses link data provided by the link sensor system to provide feedback on the joint acceleration feedback error. The joint control unit 450 and... Figure 3 The connector control unit 400 is basically similar, except for the differences described below. Figure 4B As shown, the multivariate controller 452 (e.g., multivariate H) ∞ The controller can receive connector adjustment estimates 432 (e.g., from inverse kinematics block 426) and command connector data 436Θ′. CMD (For example, from the analog processing unit 308) and receiving connector encoder data 404Θ enc (For example, input processing unit 320 from the motor side). The multivariable controller 452 can generate a motor torque command based on the joint adjustment estimate 432, command joint data 436, and joint encoder data 404. The output processing unit 310 receives the torque command signal from the multivariable controller 452, converts it into an appropriate current, and then supplies the current to the joint motor of the slave actuator to drive the motor accordingly.

[0084] Figure 5 A method 500 is shown for controlling a tool tip based on measurement data from a motor-side sensor (e.g., a connector encoder) and a load-side sensor. Method 500 in... Figure 5 The process is shown as a set of operations or procedures 502 to 514. In all embodiments of method 500, not all of the illustrated procedures 502 to 514 can be performed. Additionally, in Figure 5 One or more processes not explicitly shown may be included before, after, between, or as part of processes 502 to 514. In some embodiments, one or more processes may be implemented at least in part in the form of executable code stored on a non-transitory tangible machine-readable medium, which, when run by one or more processors (e.g., a processor of a control system), may cause one or more of the processes to be executed by one or more processors.

[0085] As shown in method 500, the connector controller can control the movement of the connector of the manipulator arm based on measurement data from both the motor-side sensor and the load-side sensor. In some embodiments, the load-side sensor may be located at a load-side positioning on the link of the manipulator arm or at another positioning not juxtaposed with the distal tool tip (e.g., a tool supported by the distal portion of the manipulator arm). In those embodiments, the connector controller may include a dynamic modeling unit to take into account the dynamic relationship between the load-side positioning and the tool tip. By utilizing measurement data from both the motor-side sensor and the load-side sensor, and taking into account the dynamic relationship between the load-side sensor and the tool tip, better tool tip control and reduced vibration and settling time can be achieved.

[0086] Method 500 begins at process 502, where the joint controller receives data from the robot system (e.g., ...). Figure 1A-1C The connector encoder of the manipulator arm (as shown in the remote-operated medical system, medical system, or non-medical system) receives motor-side (e.g., connector) measurement data. For example, the connector controller unit 318 of the master / slave control system 300 can receive connector measurement data from the connector encoder of the manipulator arm portion 207. Method 500 can proceed to process 504, where the connector controller generates a first tool tip estimate based on the connector measurement data without using any load-side measurement data. For example, the positive kinematics block 402 of the connector controller unit 318 can apply positive kinematics to the connector measurement data to generate the first tool tip estimate. Method 500 can proceed to process 506, where the connector controller receives load-side measurement data of load-side positioning from a load-side sensor system. For example, the connector controller unit 318 can receive measurement data of load-side positioning 266 provided by the load-side sensor system 264.

[0087] Method 500 can then proceed to process 508, where the connector controller generates estimates of the position and motion of the load-side positioning 266 based on both the connector measurement data and the load-side measurement data. At process 508, the connector controller unit 318 can apply positive kinematics to the connector measurement data using the positive kinematics block 402 and generate estimates of the position and / or motion of the load-side positioning 266. The state estimator 410 can perform the first estimate The load-side measurement data (e.g., acceleration data and angular velocity data) of the load-side positioning 266 are fused to generate a fusion estimate of the load-side positioning 266.

[0088] Method 500 can then proceed to process 510, where the connector controller uses the fused load-side positioning estimate and dynamic model to generate a second tool tip estimate (fused tool tip estimate). The dynamic model can be determined based on the dynamic relationship between the manipulator arm, the tool attached to the manipulator arm, and the end effector attached to the tool. For example, at process 510, dynamic model unit 412 can use the dynamic model to generate the fused tool tip estimate based on the fused load-side positioning estimate provided by state estimator 410.

[0089] Method 500 can then proceed to process 512, where the connector controller determines connector adjustment data based on the tool tip estimation difference between the first and second tool tip estimates. For example, at process 512, connector controller unit 318 can determine tool tip state estimation difference 424 by comparing the first and second tool tip estimates using comparator 422. Inverse kinematics block 426 can apply inverse kinematics to tool tip state estimation difference 424 to generate corresponding connector adjustment data.

[0090] Method 500 can then proceed to process 514, where the connector controller can control the movement of the connector based on the connector adjustment data. For example, in process 514, the connector controller unit 318 can use the connector adjustment data to adjust the command connector data Θ′. CMD (For example, in response to input from the operator input system, it is provided by the analog processing unit 308). The connector controller unit 318 can then estimate the difference 424 (denoted as δT) based on the obtained tool tip state. tip This generates a torque command signal from the joint used to control the manipulator arm. Better tool tip control is achieved by using both motor and load-side measurement data, and by considering the dynamic relationship between the load-side sensor system and the tool tip.

[0091] Any reference to surgical tools and methods is non-limiting, as the tools and methods described herein can be used on animals, human cadavers, animal carcasses, parts of human or animal anatomy, non-surgical diagnostics, industrial systems, and general-purpose robots or remote operating systems.

[0092] One or more elements in embodiments of the present invention can be implemented in software to execute on a processor of a computer system, such as a control processing system. When implemented in software, the elements of embodiments of the present invention are essentially code segments for performing necessary tasks. The program or code segment can be stored in a processor-readable storage medium or device that can be downloaded via computer data signals embodied in the form of a carrier wave over a transmission medium or communication link. A processor-readable storage device can include any medium capable of storing information, including optical, semiconductor, and magnetic media. Examples of processor-readable storage devices include electronic circuits; semiconductor devices, semiconductor storage devices, read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM); floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segment can be downloaded via a computer network (e.g., the Internet, intranet, etc.).

[0093] Note that the presented processes and displays may not inherently relate to any particular computer or other device. Various general-purpose systems may be used with programs based on the teachings herein, or it may prove convenient to construct more specialized devices to perform the described operations. The necessary structures for various such systems will appear as elements in the claims. Furthermore, embodiments of the invention are described without reference to any particular programming language. It should be understood that the teachings of the invention as described herein can be implemented using various programming languages.

[0094] Although certain exemplary embodiments of the invention have been described and illustrated in the accompanying drawings, it should be understood that such embodiments are merely illustrative of the invention and not limiting of it, and that the embodiments of the invention are not limited to the specific embodiments and arrangements shown and described, as various other modifications can be made by those skilled in the art.

Claims

1. A robot manipulator system, comprising: A robotic manipulator configured to control the movement of a tool, the robotic manipulator including a joint and a link connected to the joint, wherein the link is configured to be connected to the tool; A processing unit comprising one or more processors, the processing unit being configured to: Receive first data from the encoder of the connector; Use the first data to generate a first estimate of the first parameter of the tool; Receive second data from a sensor system located at the link or the tool; Using the first data and the second data, a second estimate of the first parameter of the tool is generated based on the dynamic model between the sensor system and the tool; as well as The connector is controlled based on the first difference between the first estimate and the second estimate.

2. The robot manipulator system according to claim 1, wherein, In order to generate the second estimate, the processing unit is configured to: A sensor system estimate is generated using the first data and the second data to produce a first parameter of the sensor system, wherein the first parameter of the sensor system is the position or velocity of the sensor system. as well as The second estimate is generated based on the sensor system estimate and the dynamic model.

3. The robot manipulator system according to claim 2, wherein, The state estimator algorithm is used to generate an estimate of the sensor system, which is selected from a group consisting of a Kalman filter, a particle filter, a nonlinear observer, and an α-β-γ filter.

4. The robot manipulator system according to claim 1, wherein, The processing unit is further configured to: A third estimate of the second parameter of the tool is generated using the first data; A fourth estimate of the second parameter of the tool is generated using the first data and the second data; as well as The connector is controlled based on the first difference and the second difference between the third estimate and the fourth estimate.

5. The robot manipulator system according to claim 1 or any one of claims 2 to 4, wherein, The first data includes data associated with at least one of the position and speed of the joint.

6. The robot manipulator system according to claim 1 or any one of claims 2 to 4, wherein, The second data includes data associated with at least one of translational acceleration data and angular velocity data.

7. The robot manipulator system according to claim 1 or any one of claims 2 to 4, wherein, The sensor system includes at least one of an accelerometer and a gyroscope.

8. The robot manipulator system according to claim 1 or any one of claims 2 to 4, further comprising: An actuator assembly is coupled to the connector to drive the movement of the connector; In order to control the connector based on the first difference, the processing unit is configured as follows: Based on the first difference, generate connector adjustment data; and Based on the joint adjustment data, a control signal is generated to control the actuation component.

9. The robot manipulator system according to claim 8, wherein, The joint adjustment data is generated by applying inverse kinematics to the first difference.

10. The robot manipulator system according to claim 9, wherein, The first difference includes the difference between the first velocity estimate of the first estimate and the second velocity estimate of the second estimate. The joint adjustment data includes joint speed adjustment data, and The inverse Jacobian function is applied to the first difference to generate the joint speed adjustment data.

11. A method for controlling a robot manipulator, the method comprising: The robot manipulator receives first data from an encoder at a connector, the robot manipulator including a link connected to the connector; A first estimate of a first parameter using the first data generation tool, wherein the tool is connected to the link; Receive second data from a sensor system located at the link or the tool; Using the first data and the second data, a second estimate of the first parameter of the tool is generated based on the dynamic model between the sensor system and the tool; as well as The connector is controlled based on the first difference between the first estimate and the second estimate.

12. The method for controlling a robot manipulator according to claim 11, wherein, Generating the second estimate includes: A sensor system estimate is generated using the first data and the second data to produce a first parameter of the sensor system, wherein the first parameter of the sensor system is the position or velocity of the sensor system; and The second estimate is generated based on the sensor system estimate and the dynamic model.

13. The method for controlling a robot manipulator according to claim 12, wherein, The state estimator algorithm is used to generate an estimate of the sensor system, which is selected from a group consisting of a Kalman filter, a particle filter, and an α-β-γ filter.

14. The method for controlling a robot manipulator according to claim 11, further comprising: Using the first data, a third estimate of the second parameter of the tool is generated; A fourth estimate of the second parameter of the tool is generated using the first data and the second data; as well as The connector is controlled based on the first difference and the second difference between the third estimate and the fourth estimate.

15. The method for controlling a robot manipulator according to claim 11 or any one of claims 12 to 14, wherein, The first data includes data associated with at least one of the position and speed of the joint.

16. The method for controlling a robot manipulator according to claim 11 or any one of claims 12 to 14, wherein, The second data includes at least one of translational acceleration data and angular velocity data.

17. The method for controlling a robot manipulator according to claim 11 or any one of claims 12 to 14, wherein, The sensor system includes at least one of an accelerometer and a gyroscope.

18. The method for controlling a robot manipulator according to claim 11 or any one of claims 12 to 14, wherein, Based on the first differential control, the connector also includes: Based on the first difference, generate connector adjustment data; and Based on the joint adjustment data, a control signal is generated to control the actuator connected to the joint to drive its movement.

19. The method for controlling a robot manipulator according to claim 18, wherein, The joint adjustment data is generated by applying inverse kinematics to the first difference.

20. The method for controlling a robot manipulator according to claim 19, wherein, The first difference includes the difference between the first velocity estimate of the first estimate and the second velocity estimate of the second estimate. The joint adjustment data includes joint speed adjustment data, and The inverse Jacobian function is applied to the first difference to generate the joint speed adjustment data.

21. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, said plurality of machine-readable instructions, when executed by one or more processors, adapted to cause said one or more processors to perform a method, said method comprising: The robot manipulator receives first data from an encoder at a connector, the robot manipulator including a link connected to the connector; A first estimate of a first parameter using the first data generation tool, wherein the tool is connected to the link; Receive second data from a sensor system located at the link or the tool; Using the first data and the second data, a second estimate of the first parameter of the tool is generated based on the dynamic model between the sensor system and the tool; as well as The connector is controlled based on the first difference between the first estimate and the second estimate.

Images