Inverse kinematics for surgical robots operating remotely with hardware constraints

Abstract

Various methods for solving inverse kinematics can be used for teleoperation of surgical robotic systems. In one method, an iterative solver solves the angular components of motion independently of solving the linear components of motion. One solver can be used to solve both together. In another method, all limits (e.g., position, velocity, and acceleration) are handled in one solution. In the event a limit is reached, the limit is used as a boundary in the intermediate solution, accepting the solution even if the boundary is reached. In another method, position limits are used to more naturally control motion than creating a deceleration zone near the boundary. In yet another method, medical-based teleoperation uses a bounded Gauss-Sedel solver, such as with successive hyper-relaxation.

Description

Background Technology

[0001] This embodiment relates to a robotic system for minimally invasive surgery (MIS). MIS can be performed using a robotic system comprising one or more robotic arms for manipulating surgical instruments based on commands from a remote operator. For example, the robotic arms may support various surgical end effectors at their distal ends, including scalpels, imaging devices (e.g., endoscopes), clamps, and scissors. Using the robotic system, the surgeon remotely controls the robotic arms to perform operations during MIS.

[0002] Teleoperation translates user commands into commands for the robotic arm and surgical tool joints as inverse kinematics problems. Inverse kinematics problems may require handling varying numbers of degrees of freedom (DOFs), depending on the end effector used. For example, a clamp has six DOFs in teleoperation. After one or more joints are engaged, once positioned for manipulation, six active joints and corresponding six DOFs are provided via three joints on the surgical tool (e.g., rotation, pitch, and yaw) and three joints on the robotic arm (e.g., spherical rotation, spherical pitch, and tool translation). Similarly, a suture device has five DOFs in teleoperation. These five active joints and corresponding DOFs include two joints on the instrument (e.g., rotation and articulation) and three joints on the robotic arm (e.g., spherical rotation, spherical pitch, and tool translation). Furthermore, an endoscope or ultrasonic scalpel has four DOFs in teleoperation. The four active joints and their corresponding DOFs include one joint on the instrument (e.g., rotation) and three joints on the robot arm (e.g., spherical rotation, spherical pitch, and tool translation).

[0003] During remote operation, the user inputs commands at six degrees of freedom (DOF), which are translated into joint motions, enabling the instrument to execute user commands with controlled precision. To translate user commands into joint commands, the inverse kinematics function is solved. An example is shown below:

[0004] Δ 命令i =J i Δθ i ,

[0005] Where Δ 命令i J is the user command matrix (6×1) in the task space at the current time i. i It is the Jacobian matrix (6×n) of the current time i, and Δθ iIt is the joint motion matrix (n×1) at the current time i. Due to physical constraints on the robot arm and surgical tool (e.g., joint position limits, joint velocity limits, and joint acceleration limits), an exact solution may not be achievable when the solution violates one or more of these constraints. Summary of the Invention

[0006] By way of introduction, the preferred embodiments described below include methods, systems, instructions, and computer-readable media for remote operation of surgical robotic systems. Various methods can be used to solve inverse kinematics. In one approach, an iterative solver solves for the angular components of the motion, independently of solving for the linear components. A single solver can be used to solve both together. In another approach, all limits (e.g., position, velocity, and acceleration) are handled in a single solution. If a limit is reached, that limit is used as a boundary in an intermediate solution, and the solution is accepted even if the boundary is reached. In yet another approach, using position limits provides more natural motion control than creating deceleration regions near the boundaries. In yet another approach, medical-based remote operation uses a bounded Gauss-Seidel solver, such as one with successive over-relaxation. Any of these methods can be used together or separately.

[0007] In a first aspect, a method for remote operation of a surgical robotic system is provided. During remote operation, a user command is received to move a robotic arm or a surgical tool of the robotic arm. In a first phase, the linear motion of the movement is solved. In a second phase, separate from the first phase, the angular motion of the movement is solved. The robotic arm or the surgical tool of the robotic arm moves based on the solutions for the linear and angular motions.

[0008] In a second aspect, a method for remote operation of a surgical robotic system is provided. During remote operation, a user command is received to move a robotic arm or a surgical tool of the robotic arm. The motion of the robotic arm or the surgical tool of the robotic arm is solved using an iterative solution, wherein the iterative solution includes solutions that consider all limits regarding the motion. The robotic arm or the surgical tool of the robotic arm moves based on the iterative solution.

[0009] In a third aspect, a surgical robotic system for remote medical manipulation is provided. Surgical instruments are attached to a robotic arm. A controller is configured to solve for the motion of the robotic arm and / or surgical instruments during remote medical manipulation of a patient and in response to movement commands input by a user. The solution is obtained using a bounded Gaussian-Seidel solver with successive over-relaxation.

[0010] This invention is defined by the following claims, and the content of this section should not be construed as limiting those claims. Any teachings relating to one class of claims (e.g., method) may be applied to another class of claims (e.g., computer-readable storage medium or system). Other aspects and advantages of the invention are discussed below in conjunction with preferred embodiments and may subsequently be claimed independently or in combination. Attached Figure Description

[0011] The components and illustrations are not necessarily drawn to scale, but rather to illustrate the principles of the invention. Furthermore, throughout the drawings, similar reference numerals indicate corresponding parts.

[0012] Figure 1 A diagram illustrating one embodiment of an operating room environment equipped with a surgical robot system, according to one embodiment.

[0013] Figure 2 An exemplary surgical robotic arm and surgical tools are shown;

[0014] Figure 3 A flowchart of an implementation scheme for a method of remote operation of a surgical robot system;

[0015] Figure 4 An exemplary robotic arm and surgical tools with six degrees of freedom (DOF) are shown;

[0016] Figure 5 An exemplary robotic arm and surgical tools with four degrees of freedom (DOF) are shown;

[0017] Figure 6 This is a flowchart of an implementation scheme for a nested control loop for inverse kinematics iterative solutions in remote operation; and

[0018] Figure 7 A block diagram of one implementation scheme for a surgical robot system. Detailed Implementation

[0019] Iterative inverse kinematics provides a unified framework for solving six, five, and four DOF problems. It handles the joint position limit, joint velocity limit, and joint acceleration limit in a single solution. It also incorporates hardware-based joint limits, velocity limits, and acceleration limits. Six, five, and four DOF inverse kinematics can use iterative solutions in a decoupled manner, where linear motion commands and angular motion commands are solved independently. It provides an exact solution (within tolerance) when no limit violations are found. Even when limit violations are found, a solution can still be provided, albeit a suboptimal one. The values ​​of the violated limits are used in the solution, thus clamping the position, velocity, and / or acceleration to the limits, at least in intermediate solutions within the iterative inverse kinematics.

[0020] Figure 1 and Figure 2 An exemplary surgical robot system is illustrated. A method for iterative inverse kinematics solutions in remote operation is discussed below with reference to this exemplary system. This method can also be used in other surgical robot systems and surgical or non-surgical robot systems and robots.

[0021] Figures 3 to 6 This involves iterative inverse kinematics solutions in remote operations. Figure 7 This involves a system that uses an iterative inverse kinematics solver with a medical robot system for remote operation.

[0022] Figure 1 This is a schematic diagram of an exemplary operating room environment configured with a surgical robot system 100, which translates commands from a user into movements of a surgical robot arm 122 based on iterative inverse kinematics. The surgical robot system 100 includes a user console 110, a control tower 130, and a surgical robot 120 having one or more surgical robot arms 122 mounted on a surgical platform 124 (e.g., a table or bed), wherein surgical tools with end effectors are attached to the distal ends of the robot arms 122 for performing surgical procedures. Additional, different, or fewer components may also be provided, such as combining the control tower 130 with the console 110 or the surgical robot 120. The robot arm 122 is shown as table-mounted, but in other configurations, the robot arm 122 may be mounted on a trolley, ceiling, sidewall, or other suitable support surface.

[0023] Generally, a user (such as a surgeon or other operator) can sit at user console 110 to remotely manipulate the robotic arm 122 and / or surgical instruments (e.g., remote operation). User console 110 may be located in the same operating room as the robotic system 100, such as... Figure 1As shown. In other environments, the user console 110 may be located in an adjacent or nearby room, or operated remotely from a remote location in a different building, city, or country. The user console 110 may include a seat 112, a pedal 114, one or more handheld user interface devices (UIDs) 116, and an open display 118 configured to display, for example, a view of a surgical site within a patient's body and a graphical user interface. As illustrated by the exemplary user console 110, a surgeon seated on the seat 112 and viewing the open display 118 can manipulate the pedal 114 and / or the handheld user interface device 116 to remotely and directly control the robotic arm 122 and / or surgical instruments mounted to the distal end of the arm 122. The user inputs commands to move the surgical arm 122 and / or the end effector. This user control determines changes in the position, rate, and rate of movement of the robotic arm 122. The rate and rate changes result in the expected dynamic torque force provided by the robotic arm 122. The surgeon, seated in chair 112, can view and interact with display 118 to input commands during surgery to move the robotic arm 122 and / or surgical instruments under remote control.

[0024] In some slightly different scenarios, the user can also operate the surgical robot system 100 in "on-bed" (OTB) mode, where the user is positioned to the patient's side and simultaneously manipulates both robot-driven tools / end effectors attached to the patient (e.g., holding a handheld user interface device 116 with one hand) and manual laparoscopic tools. For example, the user's left hand can manipulate the handheld user interface device 116 to control the robotic surgical components, while the user's right hand can manipulate the manual laparoscopic tools. Thus, in these different scenarios, the user can perform both robot-assisted MIS and manual laparoscopic surgery on the patient.

[0025] During an exemplary procedure or surgical operation, the patient is aseptically prepared and covered for anesthesia. Initial approach to the surgical site can be manually performed using the robotic system 100 in a retracted or collapsed configuration to facilitate access. Once approach is complete, initial positioning and / or preparation of the robotic system can be performed. During the procedure, the surgeon at the user console 110 can use pedals 114 and / or user interface devices 116 to manipulate various end effectors and / or imaging systems to perform surgery remotely. The aforementioned movements vary depending on the specific surgeon, patient, and / or situation. Manual assistance can also be provided at the operating table by a person wearing sterile gowns, whose tasks include, but are not limited to, retracting tissue or performing manual repositioning or tool changes involving one or more robotic arms 122. Some surgical tasks, such as retraction, suturing, or other manipulations of tissue, can alternatively be performed by one or more robotic arms 122 (e.g., a third or fourth arm). Non-sterile personnel may also be present to assist the surgeon at the user console 110. When the procedure or surgery is completed, the robotic system 100 and / or user console 110 may be configured or set to a state that facilitates one or more postoperative procedures, including but not limited to cleaning and / or sterilizing the robotic system 100, and / or medical record input or printout, whether electronic or paper, via the user console 110.

[0026] In some respects, communication between the surgical robot 120 and the user console 110 can be achieved via a control tower 130, which translates user input from the user console 110 into robot control commands and transmits these commands to the surgical robot 120. The control tower 130 performs iterative inverse kinematics. The control tower 130 can also transmit status and feedback from the robot 120 back to the user console 110. The connection between the surgical robot 120, the user console 110, and the control tower 130 can be wired and / or wireless, and can be proprietary and / or implemented using any of a variety of data communication protocols. Any wired connection can optionally be integrated into the operating room floor and / or walls or ceiling. The surgical robot system 100 can provide video output to one or more displays, including displays within the operating room and remote displays accessed via the Internet or other networks. Video output or feeds can also be encrypted to ensure privacy, and all or part of the video output can be stored on a server or electronic healthcare record system.

[0027] Before initiating surgery using a surgical robotic system, the surgical team can perform preoperative setup. During preoperative setup, the main components of the surgical robotic system (e.g., stage 124 and robotic arm 122, control tower 130, and user console 110) are positioned in the operating room, connected, and powered. The stage 124 and robotic arm 122 can be configured to be fully retracted, with the arm 122 positioned below the stage 124 for storage and / or transport. The surgical team can extend the arm 122 from its retracted position for sterile draping. After draping, the arm 122 can be partially retracted until needed for reuse. Several routine laparoscopic steps may need to be performed, including cannula placement and insufflation. For example, each cannula may be inserted into a small incision and through the body wall using a tampon. The cannulas and tampon allow light to enter to visualize tissue layers during insertion, thereby minimizing the risk of injury during placement. Typically, an endoscope is first placed to provide visualization for a handheld camera for placing other cannulas. After insufflation, manual instruments can be inserted through the sleeve if needed to perform any laparoscopic steps manually.

[0028] Next, the surgical team can position the robotic arms 122 above the patient and attach each arm 122 to the corresponding cannula. The surgical robotic system 100 can uniquely identify each tool (endoscope and surgical instrument) upon attachment and display the tool type and arm position on an open or immersive display 118 at the user console 110 and on a touchscreen display on the control tower 130. The corresponding tool function is enabled and activated using the master UID 116 and foot switch 114. The patient-side assistant can attach and detach tools as needed throughout the procedure. The surgeon, seated at the user console 110, can begin surgery remotely using tools controlled by two master UIDs 116 and foot switch 114. The system precisely translates the surgeon's hand, wrist, and finger movements into real-time movements of the surgical instruments via the master UID 116. Therefore, in direct remote operation, the system continuously monitors each surgical action of the surgeon and pauses instrument movement if the system cannot accurately reflect the surgeon's hand movements. During surgical procedures, when the endoscope is moved from one arm to the other, the system can adjust the master UID 116 for instrument calibration and continue to control the instrument's movement. The foot switch 114 can be used to activate various system modes, such as endoscope control and various instrument functions, including monopolar and bipolar ablation, without requiring the surgeon's hand to be removed from the master UID 116.

[0029] Figure 2 This is a schematic diagram illustrating an exemplary design of a robotic arm, tool actuator, and cannula loaded with robotic surgical tools, according to various aspects of the subject matter. Figure 2As shown, the exemplary surgical robot arm 122 may include a plurality of connectors (e.g., connector 202) and a plurality of engagement modules (e.g., engagement 204, see also engagements J1 to J8) for actuating the plurality of connectors relative to each other. The engagement modules may include various engagement types, such as pitch engagements or roll engagements, which can substantially constrain the movement of adjacent connectors about certain axes relative to other axes. Figure 2 An exemplary design also illustrates a tool actuator 210 attached to the distal end of a robotic arm 122. The tool actuator 210 may include a cannula 214 coupled to its end for receiving and guiding surgical instruments or end effectors 220 (e.g., endoscopes, sutures, scalpels, scissors, clamps, retractors, etc.). The surgical instrument (or “tool”) 220 includes an end effector 222 located at the distal end of the tool. A plurality of actuating joint modules of the robotic arm 122 are used to position and orient the tool actuator 210, which actuates the end effector 222 to perform robotic surgery. The end effector 222 is located at the end of the tool shaft. In other embodiments, the end of the tool shaft is the tip of a needle or other object.

[0030] exist Figure 2 In the example, joint J0 is a benchtop pivot joint located below the top of the surgical table. Joint J0 typically remains stationary during surgery. Joints J1 through J5 form a structure or Cartesian robotic arm and also typically remain stationary during surgery, thus offering no assistance in movement during remote surgical manipulation. Joints J6 and J7 form a spherical arm that can be actively moved during surgery or remote manipulation. Joint J8 converts tool 220 (such as end effector 222) into part of a tool actuator. Joint J8 can be actively moved during surgery. During surgery, joints J6 through J8 actively position the tool shaft end (i.e., end effector 222) while maintaining the patient's surgical entry point in a fixed or stable position (i.e., the remote motion center) to avoid stress on the patient's skin. During setup, any of the joints J0 through J8 can be moved. During surgical procedures, joints J6 to J8 may be moved according to safety limits regarding hardware or position, speed, acceleration, and / or torque. Surgical instrument 220 may include zero, one, or more (e.g., three) joints, such as joints for instrument rotation plus any number of additional joints (e.g., a wrist joint for rotation about a longitudinal axis or other types of movement). Any degree of freedom may be provided, such as three degrees from joints J6 to J8 and zero, one, or more degrees of freedom from the surgical instrument 220.

[0031] Figure 3This is a flowchart of one implementation of a method for remote operation of a surgical robotic system. Iterative inverse kinematics is used to translate user commands into solutions for the movement of the robotic arm and surgical instruments. Separate and independent linear and angular solutions can be used; all limits can be included in a single solution; the boundaries of intermediate solutions can be set as one or more limits when limits are reached; the ratio of positional boundaries to yield movements near the limits can be used; and / or a bounded Gaussian-Seidel solver can be used in medical teleoperation.

[0032] Figure 3 The method illustrated is implemented by a control processor such as control tower 130, a computer, workstation, server, or another processor. Any computer within the surgical robot system 100 may be used. The user interface provides movement commands received from the user during action 300. During action 320, the robotic arm 122 and / or surgical tools 220 are moved via instructions or control from the control processor. Other devices may perform and / or be used for any of the aforementioned actions.

[0033] The actions described above may be performed in the order shown or in another order. Various actions 311-316 that are part of the solution to action 310 may be performed in any order and / or simultaneously (e.g., as part of the same solution that translates user commands into movements of the robotic arm and surgical tools).

[0034] Additional, different, or fewer actions may also be employed. For example, any of actions 311-316 may be performed individually in the solution of action 310. Any combination of two or more actions from any of actions 311-316 may be performed without other actions. As another example, actions may be provided for initially positioning the surgical instrument 220 on the patient, surgical planning actions, and / or removing the surgical instrument 220 from the patient.

[0035] In action 300, the control processor receives a user command to move the robotic arm 122 or its surgical tools 220 during remote operation. The control processor receives this user input from a user console 110, such as a foot switch 114 or a user interface device 116, via a wireless or wired interface. In other embodiments, the user command is received by loading from memory or by transmission over a computer network.

[0036] In preparation for remote operation, the user sits at the surgeon's console 110. After positioning the robotic arm 122 for remote operation, one or more joints are locked in place with the fixed remote center of motion (RCM) at the patient's skin or incision entry point. For example, joints J0 to J5 (see...) Figure 2The joints are locked. This locking is achieved by braking and / or preventing the motor from actuating the joints. These joints remain locked during remote operation.

[0037] During remote operation, the user inputs commands to move the robotic arm 122 and / or the surgical tool 220. These commands are used to achieve movement. Different commands can be provided for different movements. These commands can be used for movement of the end effector 222. These commands may not be applicable to movement of a specific joint. The control processor translates the movement commands into control of the specific joint of the robotic arm 122 and / or the surgical tool 220.

[0038] In one implementation, sensors are used to track user movement. For example, the user holds a device such as a pen or UID116. Magnetic position sensors and / or inertial measurement units can be used to determine the position and / or changes in position of the pen or UID116. As another example, the user holds a marker with structures that allow for visual tracking, such as optical patterns or structures on one or more parts of the marker. Stereo cameras and / or depth cameras track the movement of the marker.

[0039] User commands can be used to move at any number of DOFs. Figure 4 A portion of a robotic arm 122 and a surgical tool 220 providing six degrees of freedom (DOF) are shown. The six DOFs correspond to the movement of six movable joints during remote manipulation. The movable joints include three joints on the surgical tool 220—a rotational joint J9, a wrist pitch joint J10, and a wrist deflection joint J11. The movable joints include three joints on the robotic arm 122—a spherical rotational joint J6, a spherical pitch joint J7, and a tool translation joint J8. Additional joints providing six degrees of freedom may be used.

[0040] User commands can be used for movements with fewer than six degrees of freedom. Five, four, or fewer active joints are available. Figure 5 A robotic arm 122 and a surgical tool 220 providing four DOFs are shown, such as an endoscope or an ultrasonic scalpel. The active joints include three joints on the robotic arm 122—a spherical rotation joint J6, a spherical pitch joint J7, and an tool translation joint J8. The active joints include one joint on the surgical tool 220—a rotation joint J9. In the example of five DOFs, the active joints include three joints on the robotic arm 122—a spherical rotation joint J6, a spherical pitch joint J7, and an tool translation joint J8—and two active joints on the surgical tool—a rotation joint J9 and an articulation joint as another joint. Other active joint arrangements can be used, such as providing two or fewer DOFs on the robotic arm during remote operation.

[0041] In action 310, the control processor translates the movement commands from the user into movement of the joint. The control processor uses iterative solutions to solve for the motion of the robot arm or its surgical tool. An iterative inverse kinematics solution is found.

[0042] Any control process can be used. The control process can be the input of user commands, an iterative solution using inverse kinematics with a given termination check for the iteration, and the output of the final result of the junction command. Iterative inverse kinematics uses one or more of the following: separate and independent linear and angular solutions; including all limits in a single solution; setting the boundaries of intermediate solutions as one or more limits when limits are reached; using the ratio of positional boundaries to yield movements close to the limits; and / or using a bounded Gaussian-Seidel solver in medical telemedicine.

[0043] Figure 6 An implementation of a nested loop control process is shown. The outer control loop 600 is a master iterative inverse kinematics loop, and the inner control loop 602 is an iterative solver loop. The outer control loop 600 includes the inner control loop 602. The outer control loop 600 also includes forward kinematics determination 604 for the position of the surgical tool 220, torsion command generation 606 based on user commands and the movement of the surgical tool's position, termination verification 608, and convergence verification 610. User pose commands or movements 605 are input to the outer control loop. The torsion command 606 is input to the solver 614, and the final solution is output to the termination verification 608. Forward kinematics 604 calculates the pose (i.e., position and orientation) of the end effector 222. This pose is input to the torsion command generator 606. The torsion command generator 606 calculates the torsional motion from the current end effector 222 pose to the pose command 605. The torsion is a six-DOF command (e.g., three linear or translational DOFs and three angular DOFs). To achieve better convergence, linear interpolation is performed on the linear torsion using the step size ratio, and interpolation is performed on the diagonal torsion using spherical linear interpolation (SLERP). Other interpolation methods can also be used.

[0044] The termination check 608 of the outer control loop 600 verifies the completion of the junction command 611 to be output to the robotic arm 122 and the surgical tool 220. If not, the outer control loop 600 iterates through the forward kinematics determination 604, the torsional command generation 606, and the inner control loop 602. The termination check 608 of the outer control loop 600 uses one or more criteria. For example, three termination checks are performed, any one of which terminates the outer loop 600. The checks may include: (1) comparing the command pose with the current pose for a convergence check 610, (2) reaching the maximum number of steps or iterations, and (3) the timer expiring. The convergence check 610 determines any error between the calculated pose command and the input pose command 605. The error will be below a threshold level (i.e., the calculated pose output by the solution from the solver 614 matches the input pose command 605 within tolerance).

[0045] The following text is in the context of internal control loop 602 or in this document. Figure 6 Various methods for solving nested control processes are discussed, including: separate and independent linear and angular solutions; including all limits in a single solution; setting the boundaries of intermediate solutions as one or more limits when limits are reached; using the ratio of positional boundaries to yield movements close to the limits; and / or using a bounded Gaussian-Seidel solver in medical telemedicine. Any of these methods can be used in other control processes, such as in conjunction with iterating through a single control loop of solutions.

[0046] The iterative inverse kinematics solver 614 is part of the inner control loop 602. The inner control loop 602 also includes a boundary generator 612 and a termination check 616. The inner control loop 602 iterates through the solution of solver 614 until the termination check 616 is satisfied. Solver 614 optimizes the solution. Additional, different, or fewer processing units or software modules can be set during the control process.

[0047] In the internal control loop 602, the boundary generator 612 sets boundary limits. These limits are related to hardware, safety, and / or other remote operation requirements to limit the position, velocity, acceleration, torque, and / or other motion characteristics of the robotic arm 122 and / or surgical tool 220.

[0048] In one implementation, a solution includes all of the following limits or restrictions: limits or restrictions with respect to the linear components of the translation, limits or restrictions with respect to the angular components of the translation, and / or limits or restrictions with respect to both the linear and angular components of the translation, as in action 311 (see action 311). Figure 3The single solution represents all motions of the robotic arm 122 and surgical tool 220, all linear motions, or all angular motions. For example, a single solution for linear motion provides all constraints or boundaries for linear motion, and a single solution for angular motion provides all constraints or boundaries for angular motion. In the iterative solution to the conversion of user movement commands to joint motions of the end effector 222, all constraints are considered or addressed in the solution. All constraints are those used, such as all positions, velocities, and accelerations, but no torque, where torque is not used to limit motion. In alternative embodiments, different solutions and solutions are provided for different subsets of the constraints considered.

[0049] In one implementation, the limits include positive and negative limits for each of the position, velocity, and acceleration boundaries. Positive and negative limits may be provided for torque boundaries. The magnitudes of the positive and negative limits may be equal or unequal. Limits are provided for each active joint, and the limits for different joints may be the same or different.

[0050] For each active joint, there are two joint position limits (θ). min ,θ max In action 312 (see...) Figure 3 In this context, the position limit ratio can be used to define the deceleration region approaching any position limit of any moving joint. To decelerate the motion near the limit, two additional boundaries (θ) are defined. min_ ,θ max_ They form a buffer zone according to their positional limits, such that:

[0051] θ min_ =θ max_ -θ 偏移 (Equation 1)

[0052] θ min_ =θ min +θ 偏移

[0053] θ 偏移 >0

[0054] The offset can be any value, such as 5mm.

[0055] These additional limits can be used to determine the joint location boundary ratio. Two joint location boundary ratios r are calculated for each joint, one for each limit:

[0056] When θ min_ <θ<θ max_ At that time, r θ_max =1.0, and r θ_min =-1.0

[0057] When θ max_ ≤θ≤θ max时 r θ_max =(θ-θ) max_ ) / θ 偏移 ,or

[0058] When θ min ≤θ≤θ min_ r θ_min =(θ-θ) min_ )θ 偏移 (Equation 2)

[0059] Based on these two ratios, calculate the motion boundaries of the two joints:

[0060] When θ max_ ≤θ≤θ max When, Δθ maxθ =r θ_max *(θ max -θ), or,

[0061] When θ min ≤θ≤θ min When, Δθ minθ =r θ_min *(θ min -θ) (Equation 3)

[0062] Based on the position limit ratio, [Δθ minθ ,Δθ maxθ This is used as a motion position boundary attributed to the joint limit constraint. The position boundary defines a deceleration region that reduces velocity and acceleration. Other ratios may be used for deceleration and / or position limits or boundaries.

[0063] For each joint, there exists a configurable maximum speed limit ω. max Make the velocity ω of the joint in [-ω max ,ω max Within the range. Using the joint ratio from Equation 2, and attributing it to the joint velocity constraint, the motion boundary is determined as:

[0064] Δθ minω =r θ_min *ω max *dt

[0065] Δθ maxω =r θ_max *ω max *dt (Equation 4)

[0066] [Δθ minω ,Δθ maxω Used as a motion boundary attributed to the velocity constraint of the joint.

[0067] For each joint, there is a configurable maximum acceleration limit α max such that when the joint position is within [θ min_ , θ max_ , the joint acceleration is within [-α max , α max . Using the joint ratio from Equation 2, due to the joint acceleration constraint, the motion boundary is determined as:

[0068]

[0069] [Δθ minα , Δθ maxα is used as the motion boundary due to the acceleration limit.

[0070] The joint torque limit can be integrated into this framework. There are two force (torque) limits - the deceleration force / torque limit (F_decel) at which the motion decelerates when exceeded and the stop force / torque limit (F_stop) at which the motion stops when reached. The force ratio is calculated as:

[0071] When F < F_decel, force_ratio = 1.0,

[0072] When F ≥ F_decel, force_ratio = (F - F_decel) / (F_stop - F_decel).

[0073] Other limits, calculations of limits, and / or ratios can be used. The ratio used to decelerate the motion in terms of position, speed, and / or acceleration can also be applied to torque or may not be used for one or more (e.g., all) limits.

[0074] To consider all the limitations in one solution, these limits are combined. The combined boundary is formed by the joint position boundary, joint velocity boundary, and joint acceleration boundary where the force or torque is not restricted. An exemplary combination is given by:

[0075] Δθ min = max(Δθ min_θ , Δθ min_ω , Δθ min_α )

[0076] Δθ max = min(Δθ max_θ , Δθ max_ω , Δθ max_α ) (Equation 6)

[0077] [Δθ min , Δθ maxIs used as the final combined boundary generated by the boundary generator 612 for input to the solver 614. These combined boundaries can be stored in memory and looked up or generated as needed.

[0078] In the case of including torque or force limits, the combined limits are scaled by the force ratio and are given by:

[0079] [Δθ min , Δθ max fr = force_ratio * [Δθ min , Δθ max ,

[0080] where [Δθ min , Δθ max is calculated from Equation 6, and [Δθ min , Δθ max fr is the final boundary of the solver 614. Other methods including force or torque limits can be used. Other combinations of limits can be used.

[0081] The solver 614 receives boundaries (such as all boundaries to be used in a solution) and motion commands from the generator 606. The solver 614 determines the joint motion as an optimization or iterative fit.

[0082] Any optimization can be used, such as linear programming or quadratic programming. In one embodiment as shown in action 313 as Figure 3 shown, a bounded Gauss - Seidel solver is used. The bounded Gauss - Seidel solver can include successive over - relaxation (bounded Gauss - Seidel solver with successive over - relaxation (BGSR)), but successive over - relaxation may not be set. BGSR is an iterative solver that has the additional feature of clamping intermediate solutions to the boundaries. The main iteration step of the general BGSR solver can be shown as:

[0083]

[0084] If and (Equation 7)

[0085] If then

[0086] where r is the successive over - relaxation (SOR) ratio, and 1.0 < r < 2.0. k is the iteration index, x i is the variable to be solved, and [a ij and [b i are the linear coefficients. If [a ij ​​If is a positive definite matrix, then the BGSR is guaranteed to converge. In this specific implementation, the Jacobian matrix is ​​transformed to satisfy this condition.

[0087] J T Jv 接合部 =J T *v 命令 (Equation 8)

[0088] Where J is the Jacobian matrix (6×n) for 6 DOFs, v 接合部 The solution to be obtained in Equation 7 is x. i The joint velocity (n×1), and v 命令 It is the command speed (6×1). "Joint" refers to the active joint in remote operation, such as J6-11.

[0089] BGSR provides intermediate and final solutions through iteration. One solution requires that position, velocity, acceleration, or other parameters exceed limits, in action 314 (see...). Figure 3 The solution uses values ​​for that limit or boundary. This clamps the intermediate or final solution to the boundary of position, velocity, and / or acceleration. Such clamping can be provided in optimizations using solvers other than BGSR. By clamping in one or more iterations, the solution can continue to find the joint motion as the final solution. The limit can be used in the final solution or replaced by values ​​within the limit based on the optimization. A final solution is possible, even when the limit is clamped to the boundary.

[0090] The inner loop termination check 616 checks one or more criteria to terminate the iteration if one of the criteria is met, or if neither criterion is met. In one example, three criteria are used to terminate the iteration of the inner control loop 602: (1)|J T Jv 接合部 -J T *v 命令 |≤tolerance, (2) the number of iterations is greater than the maximum number of iterations, and (3) the error reduction between adjacent steps or iterations in the BGSR is too small (e.g., 1.0). e-3*容差 ).

[0091] Solver 614 can be implemented as a one-stage solver that solves for both linear and angular motions together. This is illustrated as action 315 (see action 315). Figure 3 In an alternative implementation scheme, linear motion and angular motion are solved separately. Two solution stages are provided. For example, Figure 6The control process is executed separately for each of the linear motion and angular motion phases. For example, the inner control loop 602 executes separately for the linear motion and angular motion phases, and the outputs of both phases are then used together in the outer control loop 600. The linear motion and angular motion phases have independent iterative solutions. The linear motion is solved before the angular motion, but the reverse order can be used.

[0092] The motion components are decoupled and solved independently using BGSR or another solver. The two solved functions can be expressed as:

[0093] (J 上_左3x3 ) T *Δ 线性 =(J 上_左3x3 ) T *J 上_左3x3 *Δθ 6_7_8

[0094] (J 下_右3x3 ) T *Δ 角 =(J 下_右3x3 ) T *J 下_右3x3 *Δθ 9_10_11

[0095] Δθ i_min ≤Δθ i ≤Δθ i_max (Equation 12)

[0096] Where 6, 7, and 8 are the movable joints of the robotic arm 122, and 9, 10, and 11 are the movable joints of the surgical tool 220. "Down," "Up," "Left," and "Right" refer to the portions of the Jacobian matrix corresponding to the joints.

[0097] By decoupling the linear and angular phases, linear and angular motions are solved independently. Due to the decoupling, any limit violations independently affect both linear and angular motions. Limit violations in the linear motion may not occur in the angular motion component. When no limit violations exist, an exact solution satisfying the tolerance requirements is provided. Even when limit violations attributable to clamping to the boundary exist, a suboptimal solution still satisfying the tolerance requirements is provided.

[0098] The solution is obtained for six DOFs. In the case where joint command 611 is a command for each of the six joints, the solver is applied directly. In the case where the robotic arm 122 and surgical tool 220 have fewer DOFs, the solution is obtained... Figure 3Zero-padding can be used in action 316 to still provide a solution. Zeros are used for linear and / or angular motion aspects missing in the six DOFs. For surgical instruments with five DOFs, a linear or angular motion solver can be applied directly (e.g., 3 DOFs for linear motion), and the Jacobian matrix for either angular or linear motion has only two DOFs (e.g., 2 DOFs for angular motion). The Jacobian matrix (used for angles in the example below) is extended to three DOFs, as expressed by:

[0099] [Δ 角- ,0.0]=[A 2x2 0.0 2x1 ]*[Δθ 9_10 ,1.0]

[0100] [0.0 1x2 ,1.0]

[0101] For instruments with four DOFs, linear or angular motion solvers can be applied directly, and the Jacobian matrix for either angular or linear motion has only one DOF. The Jacobian matrix (used for angles in the example below) is extended to three DOFs, as expressed by the following equation:

[0102] [Δ 角_ ,0.0,0.0]=[A 1x1 ,0.0,0.0]*[Δθ9,10,1.0]

[0103] [0.0, 1.0, 0.0]

[0104] [0.0, 0.0, 1.0]

[0105] exist Figure 3 In action 320, the control processor causes the robotic arm 122 and / or surgical tool 220 to move. During remote operation, output movement commands 611 for the active joint cause the joint to change position with velocity and / or acceleration. The results from iterative inverse kinematics control the movement of the joint (e.g., joint J6-11). In the decoupled linear and angular solutions, linear motion and angular motion control the movement of the corresponding joint.

[0106] Even when a suboptimal solution is provided, real-time control of the robotic arm 122 and surgical tools 220 is still possible. The iterative solution translates user commands into robot movements.

[0107] Figure 7 This is a block diagram of one implementation of a surgical robotic system for remote medical operation. The system performs... Figure 3 Methods Figure 6 The control process or another method. It can be used... Figure 3Any one or more (e.g., all) of the solution methods for actions 311-316. Provides an iterative inverse kinematics solution for the control of the robot arm 122.

[0108] The surgical robot system includes one or more robotic arms 122 with corresponding surgical instruments 220 or other types of instruments connected to the robotic arm 122, a controller 702, and a memory 704. A user console 110 is represented or included as part of the surgical robot system. Additional, different, or fewer components may be provided. For example, the robotic arm 122, surgical instruments 220, and / or user console 110 may not be provided.

[0109] Each robotic arm 122 includes one or more connectors and joints. The joint can be a pitch joint or a roll joint. Tool actuators and cannulas for receiving and guiding surgical instruments can be disposed on each robotic arm 122. Different combinations of connectors and joints can define or form different portions of the robotic arm 122, such as different portions with different degrees or types of movement (e.g., translation and / or rotation). Any robotic arm 122 currently known or developed later, having motors, sensors, connectors, joints, controllers, surgical instruments, and / or other structures, can be used.

[0110] One or more robotic arms are provided. For example, three or four robotic arms 122 are provided. The robotic arms 122 are mounted to the base of a platform such as an operating table. Alternatively, a trolley, floor, ceiling, or other base frame can be used. The robotic arms 122 include cables or wireless transceivers for communicating with a processor 206 or middleware (e.g., control tower 130).

[0111] The robotic surgical instrument 220 is one or more grasping forceps, retractors, scalpels, endoscopes, suture devices, scissors, or other surgical instruments used to manipulate patient tissue. Manipulation of tissue can be direct, such as cutting or grasping. Manipulation can also be indirect, such as pressing or contacting the endoscope against or in contact with the tissue to guide imaging or view parts of the patient's body. Different or identical types of instruments 220 can be mounted to different robotic arms 122. For example, two robotic arms 122 may hold grasping forceps, a third robotic arm 122 may hold a scalpel, and a fourth robotic arm 122 may hold an endoscope.

[0112] The robotic surgical instrument 220 is attached to the distal end of the robotic arm 122, but may also be attached at other locations. This attachment provides the actuation force to manipulate the instrument, such as a closing gripper or scissors.

[0113] User console 110 is a graphical user interface for surgeons to interact with a surgical robot system, such as a console that includes a processor for controlling a robotic arm 122. The user interface includes user input 708 and a display 118. User input 708 and / or display 118 are located at user console 110 and / or control tower 130, but may also be located elsewhere.

[0114] User input 708 may be a button, keyboard, rocker arm, joystick, trackball, voice recognition circuitry, mouse, touchpad, touchscreen, slider, switch, UID 116, foot switch 114, or a combination thereof, or any other input device for inputting information onto a surgical robot. Display 118 may be a monitor, liquid crystal display (LCD), projector, plasma display, CRT, printer, or other currently known or later-developed device for outputting visual information. In an alternative embodiment, display 118 may be a head-mounted display. User input 708 may be one or more sensors for detecting eye movement and / or blinking. In yet another embodiment, user input 708 may be a microphone for voice-based input. A speaker may be provided for audio information output, either in place of display 118 or used in conjunction with display 118.

[0115] Optional camera 706 is a digital camera for optically tracking user movements, such as during tracking using UID 116 to control the robotic arm 122. In some embodiments, camera 706 can be a stereo camera and / or a depth camera. Camera 706 is positioned relative to the user and a target pattern or user console 110 for tracking human movements by tracking the target pattern or controlling the robotic arm 122 using user input 708.

[0116] Controller 702 is a controller for driving and / or modeling the robotic arm 122 and / or surgical instruments 220. Controller 702 is a general-purpose processor, central processing unit, control processor, graphics processor, digital signal processor, application-specific integrated circuit, field-programmable gate array, digital circuit, analog circuit, artificial intelligence processor, or a combination thereof, or other currently known or later developed device for translating user commands into joint commands for the robotic arm 122 and / or surgical instruments 220. Controller 702 can be a single device or multiple devices operating in series, parallel, or individually. Controller 702 can be the main processor of a computer, such as a laptop computer, server, workstation, or desktop computer, or a processor for handling several tasks in a larger system. Based on hardware, software, firmware, or a combination thereof, controller 702 is configured to implement instructions or perform actions.

[0117] Controller 702 is configured to solve for the motion of robotic arm 122 and / or surgical instrument 220 during remote medical manipulation of a patient and in response to movement commands input by the user via user console 110. Controller 702 is configured to implement a solver, such as BGSR. Iterative solutions are provided using a bounded Gaussian-Seidel solver with successive over-relaxation. This solution includes all limits regarding the position, velocity, and acceleration of the robotic arm and surgical instrument. All limits used in the transformation are processed in the solver to determine a solution for the inverse kinematics. Controller 702 can be configured to solve for both linear and angular components, where the solution for the linear components is independent of the solution for the angular components. The solution handles limit violations without failure. For example, an intermediate solution from the bounded Gaussian-Seidel solver sets the position, velocity, or acceleration of the robotic arm 122 and / or surgical instrument 220 to a boundary. The solver determines that the joint motion does not violate the boundary.

[0118] Controller 702 is configured to control robotic arm 122 and surgical tool 220. Based on the solution, one or more joints are moved in response to user commands. Iterative inverse kinematics solutions control the joints.

[0119] Memory 704 or another memory is a non-transitory computer-readable storage medium that stores data representing instructions executable by a programmable controller 702. Instructions for implementing the processes, methods, and / or techniques discussed herein may be provided on a computer-readable storage medium or memory such as a cache, buffer, RAM, removable media, hard disk drive, or other computer-readable storage medium. Computer-readable storage media include various types of volatile and non-volatile storage media. The functions, actions, or tasks shown in the figures or described herein are performed in response to one or more sets of instructions stored in or on a computer-readable storage medium. Functions, actions, or tasks are independent of a particular type of instruction set, storage medium, processor, or processing strategy, and may be executed by software, hardware, integrated circuits, firmware, microcode, etc., operating individually or in combination. Similarly, processing strategies may include multi-processing, multi-tasking, parallel processing, etc.

[0120] In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored at a remote location for transmission over a computer network or telephone line. In still other embodiments, the instructions are stored within a given computer, CPU, GPU, or system.

[0121] Instructions are provided for any of the actions discussed herein. Instructions are used to determine the dimensions of a surgical robot, such as for designing or testing surgical robots. Instructions are used to generate relationships between velocities and accelerations, generate synthetic trajectories, and / or generate virtual trajectories.

[0122] While the invention has been described with reference to various embodiments, it should be understood that many changes and modifications can be made without departing from the scope of the invention. Therefore, the detailed description above should be considered illustrative rather than restrictive, and it should be understood that the following claims, which include all equivalents, are intended to define the spirit and scope of the invention.

Claims

1. A non-transitory computer-readable storage medium storing instructions executable by a controller of a surgical robotic system for teleoperation of the surgical robotic system, the controller executing the instructions to: receive user commands to move a surgical tool mounted on a robotic arm during the teleoperation; solve for linear motion of the movement in a first phase; solve for angular motion of the movement in a second phase separate from the first phase; and move the robotic arm and / or the surgical tool based on the solutions for the linear motion and the angular motion.

2. The non-transitory computer-readable storage medium of claim 1, wherein receiving during the teleoperation includes receiving with the robotic arm positioned with one or more joints locked in place with a fixed remote center of motion at a patient entry point.

3. The non-transitory computer-readable storage medium of claim 1, wherein receiving includes receiving the user commands with the teleoperation for less than six degrees of freedom, and wherein solving for the linear motion and / or solving for the angular motion includes solving with a Jacobian matrix for six degrees of freedom, wherein the linear motion and / or the angular motion for less than six degrees of freedom has zero terms.

4. The non-transitory computer-readable storage medium of claim 1, wherein solving for the linear motion includes solving in a first single solution with all limits on linear components of the movement of the robotic arm, and wherein solving for the angular motion includes solving in a second single solution with all limits on angular components of the movement of the robotic arm.

5. The non-transitory computer-readable storage medium of claim 4, wherein the first single solution includes a first term including positive and negative position bounds, velocity bounds, and acceleration bounds as all the limits, and wherein the second single solution includes a second term including positive and negative position bounds, velocity bounds, and acceleration bounds as all the limits.

6. The non-transitory computer-readable storage medium of claim 1, wherein solving for the linear motion and / or solving for the angular motion includes solving with limit violations that independently affect the linear motion and the angular motion due to the first phase and the second phase being separate.

7. The non-transitory computer-readable storage medium of claim 1, wherein solving for the linear motion and / or solving for the angular motion includes solving with position, velocity, or acceleration limits used as results in the solutions.

8. The non-transitory computer readable storage medium of claim 1, wherein solving the linear motion and solving the angular motion each comprise solving with a first control loop nested in a second control loop, the first control loop including a boundary generator operating with a solver and a termination check for iterative solving by the solver, the second control loop including a forward kinematics determination of a position of the surgical tool, a command generation from the user command and the movement of the position of the surgical tool, the movement provided to the second control loop, and a second termination check receiving termination from the first control loop.

9. The non-transitory computer readable storage medium of claim 1, wherein solving the linear motion and / or solving the angular motion comprises solving with a position limit ratio defining a deceleration zone of a position limit proximate to a joint of the robotic arm.

10. The non-transitory computer readable storage medium of claim 1, wherein solving the linear motion and / or solving the angular motion comprises solving with a bounded Gauss-Seidel solver.

11. The non-transitory computer readable storage medium of claim 10, wherein solving with the bounded Gauss-Seidel solver comprises solving with successive overrelaxation.

12. The non-transitory computer readable storage medium of claim 10, wherein the bounded Gauss-Seidel solver clamps intermediate solutions to a boundary of a position, velocity, or acceleration of the robotic arm.

13. A non-transitory computer readable storage medium storing instructions executable by a controller of a surgical robotic system for teleoperation of the surgical robotic system, the controller executing the instructions to: receive input commands to move a surgical tool mounted on a distal end of a robotic arm during the teleoperation; solve motion of the robotic arm or the surgical tool with an iterative solution, wherein the iterative solution includes a solution that accounts for all limits on the motion; and move the surgical tool and / or the robotic arm based on the iterative solution; wherein solving comprises solving linear motion in one phase and solving angular motion in another phase, each of the phases having the iterative solution independent of the other phase.

14. The non-transitory computer readable storage medium of claim 13, wherein solving comprises solving with a bounded Gauss-Seidel solver with successive overrelaxation.

15. The non-transitory computer readable storage medium of claim 13, wherein solving comprises clamping an intermediate solution of the iterative solution to one of the limits.

16. A surgical robotic system for teleoperation, the surgical robotic system comprising: a robotic arm; a surgical instrument coupled to a distal end of the robotic arm; and a controller configured to solve motion of the robotic arm and / or the surgical instrument during the teleoperation and in response to movement commands, the solution using a bounded Gauss-Seidel solver with successive overrelaxation. ​ wherein the controller is configured to solve for a linear component and an angular component of the motion, the solution of the linear component being independent of the solution of the angular component.

17. The surgical robotic system of claim 16, wherein the controller is configured to clamp an intermediate solution of the bounded Gauss-Seidel solver to a boundary of the robotic arm and / or the surgical instrument, the clamping allowing the bounded Gauss-Seidel solver to determine that the motion does not violate the boundary.

18. The surgical robotic system of claim 16, wherein the controller is configured to solve with the bounded Gauss-Seidel solver, wherein the solution includes all limits on position, velocity, and acceleration of the robotic arm and the surgical instrument in the solution.

Images