Real time trajectory generation based on dynamic control and velocity saturation

Abstract

A surgical robot configured to generate a smooth trajectory in real time to control the pose of an end effector of a robotic arm. The surgical robot includes a plurality of actuators that control the movement of the robotic arm, an end effector coupled to and configured to be driven by the actuators, and a processor coupled to the actuators. The processor is configured to receive an input command to move the end effector from a current pose to a destination pose, compute a next pose on a smooth trajectory between the current pose and the destination pose based on a simulation of a virtual force applied to a virtual mass object to move the virtual mass object from the current pose to the next pose, and drive at least one of the plurality of actuators to transition the end effector to the next pose based on the applied virtual force.

Description

REAL TIME TRAJECTORY GENERATION BASED ON DYNAMIC CONTROL AND VELOCITY SATURATIONPRIORITY

[0001] This application claims priority to U.S. Provisional Application No. 63 / 642,257, filed May 3, 2024, entitled “REAL TIME TRAJECTORY GENERATION BASED ON DYNAMIC CONTROL AND VELOCITY SATURATION,” the disclosure of which is incorporated by reference herein, in its entirety.TECHNICAL FIELD

[0002] The systems and methods disclosed herein are directed to devices and methods for controlling the position and movement of robotic systems, and more particularly to robotic systems for controlling the pose of surgical tools on a trajectory for moving robotic manipulators.BACKGROUND

[0003] A robotically enabled medical system is capable of performing a variety of medical procedures, including both minimally invasive procedures, such as laparoscopy, and non- invasive procedures, such as endoscopy (e.g., bronchoscopy, ureteroscopy, gastroscopy, etc.). Such robotic medical systems may include robotic arms with robotic manipulators attached to and configured to control the position and the orientation of surgical tool(s) during a given medical procedure. The combination of both the position and the orientation of a surgical tool is also referred to as the pose of the surgical tool. The robotic arm may move the surgical tool along a trajectory to an intended destination point.SUMMARY

[0004] During robotic surgery, such as in a teleoperation, the pose of a surgical tool of a robotic medical system can be controlled based on a calculated trajectory for transitioning a robotic manipulator attached to the surgical tool. The trajectory is a path in space calculated based on information in one or more input commands for controlling and moving the surgical tool, such as by a surgeon driving one or more of the robotic arms during surgery. For example, the movement of the robotic arm transitions the robotic manipulator to determine the pose of the surgical tool. Because controlling the movement of the robotic arm to transition the roboticmanipulator is based on the positioning of different parts (e.g., joints, links) of the robotic arm, the trajectory is calculated based on constraints on such movement. These constraints affect the ability to provide a smooth trajectory for controlling the pose of the robotic manipulator of the surgical tool. For example, a smooth trajectory can be calculated between a start pose and a final destination pose of the robotic manipulator or surgical tool which satisfies linear and / or angular velocity and acceleration limits.

[0005] A smooth trajectory is a path and / or motion in space that is continuous and without abrupt changes or jerks. The path or motion of a smooth trajectory can be a seamless transition between a start pose and a destination pose. In embodiments, the smooth trajectory can also include a seamless transition in velocity and / or acceleration between two points provided by a fluid movement of an object in the robotic system. The smooth trajectory can be calculated as a continuous curve designating a path for the transition, also referred to as displacement, of an end effector in three-dimensional (3D) space, such as in Cartesian coordinates, with determined velocity and acceleration. As disclosed herein, the term “end effector” refers to the device (e.g., robotic manipulator) or tool (e.g., surgical tool) located at the end of a robotic arm or manipulator, respectfully. Accordingly, the end effector is the part of a robot system that interacts with the environment, performing certain tasks or actions. For example, in a robotic medical system, the end effector can be the surgical tool that is attached to a robotic manipulator of the robotic arm and configured to perform a surgical operation on a patient. In other examples, the end effector is a robotic manipulator that is attached to the end of the robotic arm with or without the presence of a surgical tool.

[0006] The displacement of the end effector along a smooth trajectory provides pose control for the end effector and is achieved by controlling the movement of the robotic arm. The movement of the robotic arm can be determined based on the position configuration of one or more parts (e.g., joints and / or links) of the robotic arm in space, also referred to as joint space. The continuous curve also designates a path without abrupt change for transitioning the end effector along the smooth trajectory, which moves the end effector without jerks or abrupt movements and without displacing the end effector outside the smooth trajectory. A smooth trajectory is also calculated with velocity and acceleration limits on the movements of the robotic arm to reduce sudden accelerations or decelerations.

[0007] A smooth trajectory provides a precise control for the pose of the end effector during a motion of the robotic arm so that the end effector can accurately reach the final destination pose in a surgical operation. Removing jerks or abrupt movements and reducing sudden accelerations or decelerations by a smooth trajectory also mitigates mechanical stressesand instabilities in manipulating the robotic arm to provide safety in surgical operations. In surgical scenarios, calculating the trajectory in real time is useful to provide sufficient response time to ensure the accuracy and safety of the surgical procedure. For example, the surgeon driving a robotic arm to control a surgical tool during surgery depends on a reliable real time response of a robotic medical system, also referred to herein as a surgical robot, to perform a successful operation.

[0008] This description includes embodiments and examples of improved robotic systems and associated methods for generating a smooth trajectory in real time to control the pose of an end effector of a robotic arm. The robotic system can be a surgical robot and the smooth trajectory can be calculated in real time during robotic surgery, such as in a teleoperation. The smooth trajectory is calculated based on simulating a virtual force to control the position and orientation of the end effector from an initial (or start) pose to a destination (or end) pose. The initial and destination poses can be provided from an input device that drives the end effector. The virtual force is simulated by a computer executed algorithm which is a feedback control algorithm, also referred to herein as a dynamic controller (DC), applied to move a virtual mass, such as a unit mass object, that represents an end effector of the surgical tool or the robotic arm. The virtual force is applied to move the virtual mass object by a determined virtual force from the initial pose to the destination pose according to velocity and / or acceleration limits.

[0009] According to the DC, the virtual mass object is moved by applying a sequence of one or more virtual forces to transition the end effector in a sequence of respective one or more poses from the initial pose to the destination pose. The DC is a control algorithm that is configured to regulate the motion and behavior of a system, based on dynamic characteristics of the system, and generate outputs that achieve a target behavior or trajectory. For example, the dynamic characteristics of the robotic system based on the virtual force model include the virtual object mass, the virtual forces applied to the virtual object mass, and conditions on the movement of the robotic arm such as limits on velocity, acceleration, pose, inertia, etc.

[0010] The DC is used to compute, based on the velocity and / or acceleration, the sequence of one or more poses on a continuous curve path in 3D space without abrupt change, providing a smooth trajectory for the transition between poses. In embodiments, the computation in 3D space is based on Cartesian coordinates, also referred to herein as task space, for calculating vectors of the virtual force with directional velocity and acceleration limits. For example, the dynamic characteristics include a limitation or condition imposed on the maximum velocity of the robotic arm or the end effector, also referred to herein as velocity saturation, that a robotic system can achieve. Velocity saturation is useful to prevent excessive or undesired velocitiesthat can cause instability, safety concerns, or mechanical stresses. After computing each pose in the sequence of poses, the robotic arm is driven and controlled in joint space according to the pose to transition the end effector accordingly along the smooth path.

[0011] The disclosed systems and methods have several advantages over other systems, such as in terms of computation speed, complexity, and / or accuracy. For example, computing the sequence of poses by simulating the virtual force based on the DC reduces the complexity of smooth trajectory computations in comparison to other algorithms such as multi-step optimization algorithms (e.g., dynamic programming or convex optimization solvers). In comparison to other trajectory calculation algorithms that are based in joint space and that provide results with non-isotropic velocity and / or acceleration limits, applying the DC computation in Cartesian coordinates space provides outputs with isotropic velocity and / or acceleration limits and accordingly higher control accuracy for the smooth trajectory. Given a certain real time threshold, such as a limit on the control loop rate, reducing the complexity of the trajectory calculation also increases the accuracy of the outputs for controlling the pose of the end effector on the smooth trajectory.

[0012] The systems, methods, and devices of this disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein.

[0013] In accordance with some embodiments of the present disclosure, a surgical robot comprises a plurality of actuators, an end effector coupled to and configured to be driven by the actuators, and a processor coupled to the actuators. The processor is configured to receive an input command to move the end effector from a current pose to a destination pose, and compute a next pose on a smooth trajectory between the current pose and the destination pose. The computation is based on a simulation of a virtual force applied to a virtual mass object to move the virtual mass object from the current pose to the next pose. The processor is further configured to drive at least one of the plurality of actuators to transition the end effector to the next pose based on the applied virtual force and move the end effector along the smooth trajectory.

[0014] In some embodiments, the virtual force is applied to the virtual mass object to compute the next pose based on a dynamic controller

[0015] In some embodiments, the virtual mass object is a unit mass object simulated as an end point of the end effector.

[0016] In some embodiments, the processor is further configured to drive at least one of the plurality of actuators based on a sequence of simulated virtual forces to transition the endeffector from the current pose to the destination pose in respective increments of poses on the smooth trajectory.

[0017] In some embodiments, the simulated virtual forces control transitioning the end effector along the smooth trajectory without jerks or abrupt movements and without displacing the end effector outside the smooth trajectory.

[0018] In some embodiments, the processor is further configured to simulate at least one of a displacement vector, a velocity vector, or an acceleration vector applied to the virtual mass object to move the virtual mass object on the smooth trajectory.

[0019] In some embodiments, the virtual vector is represented by vector coordinates that are three-dimensional Cartesian coordinates.

[0020] In some embodiments, the virtual force is simulated to compute the next pose on the smooth trajectory and drive at least one of the plurality of actuators in real time during a teleoperation while the surgical robot is in motion.

[0021] In some embodiments, the virtual force is simulated based on velocity saturation and / or acceleration bounding that satisfy determined velocity and / or acceleration limits.

[0022] In some embodiments, a computer-implemented method comprises receiving an input command comprising a start pose and an end pose for an end effector of a surgical robot, and computing a next pose on a smooth trajectory between the start pose to the end pose based on a simulation of a virtual force applied to a virtual mass object to move the virtual mass object from the start pose to the next pose. The method further comprises transitioning the end effector from the start pose to the next pose based on the applied virtual force, and repeating computing a new next pose on the smooth trajectory based on a new simulation of a next virtual force applied to the virtual mass object to move the virtual mass object to the new next pose, and transitioning the end effector to the new next pose based on the applied next virtual force until reaching the end pose.

[0023] In some embodiments, simulating the next virtual force comprises computing a proportional gain and a derivative gain based on a limit on a velocity or an acceleration of the end effector; and simulating the next virtual force based on the simulated virtual force, the proportional gain, and the derivative gain.

[0024] In some embodiments, the method further comprises driving, in joint space and based on the next pose, a robotic arm attached to the end effector.

[0025] In some embodiments, the virtual force and the next virtual force are simulated using a proportional derivative (PD) controller satisfying determined velocity and acceleration limits of a robotic system coupled to the end effector.

[0026] In some embodiments, the virtual force and the next virtual force are simulated using a spring force model that pulls the virtual mass object from the start pose to the next pose.

[0027] In some embodiments, the next virtual force is simulated and the end effector is transitioned within a 1 millimeter time window.

[0028] In some embodiments, simulating the next virtual force and transitioning the end effector is repeated at a control loop rate higher than 1 kilohertz (kHz).

[0029] In some embodiments, simulating the virtual force and the next virtual force comprises applying a PD controller that is configured to damp overshooting in transitioning the end effector along the smooth trajectory.

[0030] In some embodiments, the PD controller is applied based to at least one condition including a limit on a pose, a velocity, or an acceleration associated with the end effector.

[0031] In some embodiments, the computer-implemented method further comprises accelerating an initial velocity of the end effector from the start pose to reach a constant velocity by simulating a first sequence of one or more virtual forces based on first conditions of velocity and / or acceleration, maintaining the constant velocity along the smooth trajectory to the end pose by simulating a second sequence of one or more virtual forces based on second conditions of velocity and / or acceleration, and slowing down the constant velocity to stop the end effector at the end pose by simulating a third sequence of one or more virtual forces based on third conditions of velocity and / or acceleration.

[0032] In some embodiments, the end effector is transitioned based on the virtual force and the next virtual force to prevent sharp or discrete steps in the smooth trajectory.

[0033] In accordance with some embodiments of the present disclosure, a non-transitory computer readable storage medium storing computer-executable instructions, when executed by one or more processors of a robotic system, cause the one or more processors to receive an input command to move a robotic manipulator from a current pose to a destination pose, and compute a next pose on a smooth trajectory between the current pose and the destination pose based on a DC of a virtual force applied to a virtual mass object. The virtual force moves the virtual mass object from the current pose to the next pose. The operations also include sending a transition command to the robotic manipulator based on the virtual force.

[0034] In some embodiments, the computer-executable instructions further cause the one or more processors to compute in real time a sequence of next poses based on the DC of respective virtual forces applied to the virtual mass object to move the virtual mass object on the smooth trajectory to the destination pose.

[0035] In some embodiments, the next pose comprises a position and an orientation, indicated by a pitch, a roll, and a yaw, of the robotic manipulator.

[0036] In some embodiments, the virtual force comprises at least one of vector coordinates, a velocity, or an acceleration applied to the virtual mass object to move the virtual mass object on the smooth trajectory.

[0037] In some embodiments, the computer-executable instructions further cause the one or more processors to simulate the virtual force based on the DC to drive the robotic manipulator in real time during a teleoperation while the robotic system is in motion.

[0038] In some embodiments, the computer-executable instructions further cause the one or more processors to simulate the virtual force in real time based on at least one condition including a limit on a pose, a velocity, or an acceleration of at least one of the robotic manipulator or the robotic system.

[0039] In accordance with some embodiments of the present disclosure, a robotic control system includes one or more processors and memory. The memory stores instructions that, when executed by the one or more processors, cause the one or more processors to perform any of the methods disclosed herein.

[0040] In accordance with some embodiments of the present disclosure, a non-transitory computer readable storage medium stores computer-executable instructions. The computerexecutable instructions, when executed by one or more processors of a robotic control system, cause the one or more processors to perform any of the methods disclosed herein.

[0041] Note that the various embodiments described above can be combined with any other embodiments described herein. The features and advantages described in the specification are not all inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the inventive subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The disclosed aspects will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like designations denote like elements.

[0043] FIG. 1 illustrates an exemplary robotic medical system according to some embodiments.

[0044] FIG. 2 illustrates components of a robotic medical system in accordance with some embodiments.

[0045] FIGS. 3 A and 3B illustrate different views of an exemplary robotic arm according to some embodiments.

[0046] FIG. 4 illustrates a part of a robotic arm and a surgical tool according to some embodiments.

[0047] FIGS. 5 A, 5B, and 5C illustrate an end effector transition on a smooth trajectory based on a virtual force simulation, in accordance with some embodiments.

[0048] FIG. 6 illustrates a flowchart diagram for a method performed by one or more processors of a robotic system, in accordance with some embodiments.

[0049] FIG. 7 illustrates a flowchart diagram for a method performed by one or more processors of a robotic system, in accordance with some embodiments.

[0050] FIG. 8 is a schematic diagram illustrating electronic components of a robotic medical system under some embodiments.DETAILED DESCRIPTION

[0051] In robotic surgery, the ability to precisely control the trajectory of surgical tools promotes the safety and efficacy of medical procedures. Current methods for controlling the trajectory face limitations associated with the mechanical constraints inherent in robotic systems and the complexity of real-time processing requirements. For example, such methods include methods based on joint space trajectory calculations where the trajectory of the surgical tool or end effector is defined by changes in the states of joints (e.g., angle for rotational joints, extension for prismatic joints) over time. At any given time, the calculated set of joint states determine the position and orientation of the surgical tool or end effector. The trajectory is thus obtained as a sequence of joint states over time. The joint space calculations can be restricted by the limits on velocity and acceleration that vary depending on the direction and joint. For example, rotational joints might accelerate faster than prismatic joints due to their different physical and mechanical properties. Calculating trajectories considering these non-uniform constraints becomes complex as each joint may need individual consideration for the joint dynamics. For example, this can lead to a trajectory that is optimal for one joint but suboptimal or unsafe for other joints. To address such issues, embodiments of the disclosure include systems and methods for improving dynamic control of robotic manipulators or end effectors. This dynamic control is achieved through trajectory calculations that incorporates velocitysaturation techniques to ensure both the smooth movement of the surgical tools and adherence to speed and precision constraints.

[0052] Specifically, embodiments of the disclosure use a dynamic controller (DC) to simulate virtual forces that guide the surgical tool or end effector through a calculated trajectory. This simulation can account for various operational parameters, for example the maximum allowable velocities and accelerations, to prevent abrupt changes in motion unlike with using other methods, such as with joint space trajectory calculations. Consequently, the virtual forces simulated using the DC can guide the surgical tool along a smooth trajectory, enhancing control and precision and addressing the real-time needs of surgical operations. The DC guided smooth trajectory also reduces the risk of errors that could jeopardize system and patient safety. The smooth trajectory is generated by the DC simulating virtual forces using isotropic space calculations in 3D Cartesian coordinates. Using the isotropic Cartesian coordinates can simplify trajectory calculation because of uniform control limits on velocity and acceleration across all axes of the Cartesian space. The virtual forces can also be dynamically adjusted based on tracking the position and velocity of the surgical tool or end effector. The dynamic adjustment of virtual forces can maintain the integrity of the trajectory within the operational parameter, for example based on velocity and acceleration limits (e.g., velocity saturation and / or acceleration bounding) of the robotic system. Thus, trajectory control, precision, and safety can be improved in comparison to other methods, such as joint space trajectory calculation methods. The precise movements of the end effector are realized by controlling the actuators to drive the robotic arm in response to the dynamically adjusted virtual forces, ensuring that each motion adheres closely to the smooth trajectory.

[0053] Aspects of the present disclosure may be integrated into a robotically enabled medical system capable of performing a variety of medical procedures, including both minimally invasive, such as laparoscopy, and non-invasive, such as endoscopy, procedures. Among endoscopy procedures, the robotic system may be capable of performing bronchoscopy, ureteroscopy, gastroscopy, etc. In addition to performing such procedures, the system may provide additional benefits, such as enhanced imaging and guidance to assist the physician. Additionally, the system may provide the physician with the ability to perform the procedure from an ergonomic position without the need for awkward arm motions and positions. The system may also provide the physician with the ability to perform the procedure with improved ease of use such that one or more of the instruments of the system can be controlled by a single user.

[0054] Various embodiments will be described below in conjunction with the drawings for purposes of illustration. It should be appreciated that many other embodiments of the disclosed concepts are possible, and various advantages can be achieved with the disclosed embodiments. Headings are included herein for reference and to aid in locating various sections. These headings are not intended to limit the scope of the concepts described with respect thereto. Such concepts may have applicability throughout the entire specification.

[0055] As disclosed herein, embodiments of the disclosure enable a robotic system to control the pose of an end effector (e.g., endoscopes, staplers, scalpel, scissors, clamp, retractor, etc.) along a smooth trajectory in real time during robotic surgery. For example, the end effector can be transitioned along the smooth trajectory to move the end effector from a current pose to a next pose in a continuous control loop of input of commands by a haptic interface device (HID). The end effector is transitioned from a current pose to a next pose along the smooth trajectory by simulating a virtual force applied to a virtual mass object at the location of the end effector. The virtual force is applied to the virtual mass object by the DC to simulate a behavior of transitioning the end effector on a smooth trajectory.

[0056] The DC is a control algorithm that enables the simulation of dynamic behaviors or actions on an object in a virtual environment. The DC is based on the physical laws and principles governing system motion and behavior, such as Newton's laws of motion, conservation of energy, and torque relationships. The DC may apply a virtual force to the virtual mass object based on velocity and acceleration limits such as velocity saturation and acceleration bounding to control that virtual mass object towards the destination pose.

[0057] The DC may be implemented as a computer-executable algorithm, based on dynamic characteristics determined for the robotic arm and the robotic system, including velocity and / or acceleration limits. Accordingly, the motion and behavior of the system and the involved system parts can be regulated and can provide a smooth transition without an abrupt change on a path in 3D space from an initial pose to a destination pose.

[0058] In embodiments, the velocity and / or acceleration limits used in the DC algorithm include conditions for maintaining constant values or limiting minimum or maximum values for the velocity and / or acceleration of the system components (e.g., robotic arm or robotic manipulator) within a determined space. The conditions can be expressed by fixed values or equations that represent the relationships between the velocity and / or acceleration and the displacement of the end effector in 3D space or the configuration of the joints / links of the robotic arm in joint space. The velocity and / or acceleration limits can be based on restrictions, such as safety or physical restrictions, on the motion in the system. For example, the velocityand / or acceleration limits can include conditions for maximum velocity and / or acceleration on the movement of system components or parts (e.g., joints / links) according to weight. The conditions of velocity and / or acceleration can also be set by design, such as to maintain the velocity and acceleration of the system components within certain thresholds for operation safety.

[0059] Using the DC algorithm, the smooth trajectory calculations are carried in isotropic 3D space coordinates, such as the Cartesian coordinates. The actual motion control between each pose and a next pose along this smooth trajectory path can be realized by driving, in joint space, the parts of the robotic arm (e.g., joints / links) or other system part, after the computation of the next pose in the isotropic Cartesian space.

[0060] Controlling the motion of the robotic system to transition the end effector from a current pose to a next pose is achieved by driving at least one of the plurality of actuators of the robotic arm that is coupled to the end effector. The actuators are motor devices, including mechanical devices, hydraulic devices, or otherwise, that control the movement and the positioning of the different parts of the robotic arm, such as joints, links, etc. The actuators are driven and controlled based on determined parameters associated with the different parts of the robotic arm to achieve the transition of the end effector in 3D space from the current pose to the next pose along the smooth trajectory. The parameters are determined in joint space according to information of the current pose and the next pose based on the simulated virtual force.

[0061] The actuators of one or more robotic arms are driven and controlled in joint space based on a sequence of the simulated virtual forces by the DC. In this sequence, the virtual forces can also be dynamically adjusted based on tracking the position and velocity of the surgical tool or end effector. This sequence transitions the end effector from the current pose to the destination pose in respective increments of poses on the smooth trajectory. The DC algorithm can include a PD controller that calculates the sequence of poses as increments in pose along the smooth path. The simulated virtual forces provide a layer of control to the joint space control of the actuators to transition the end effector along the smooth trajectory without jerks or abrupt movements and without displacing the end effector outside the smooth trajectory. The simulated virtual force by the DC can drive and control the robotic arm to prevent sharp or discrete steps or abrupt changes to provide the smooth trajectory on which the end effector is transitioned. This control can be provided in real time within a determined response time threshold during a teleoperation while the surgical robot is in motion.

[0062] In embodiments, the information about the initial pose and the destination pose is obtained by an input command provided though one or more input devices, such as HIDs, operated by a surgeon. In examples, the initial or start pose is a current pose of the end effector obtained after controlling the end effector according to a previous input command that moved the end effector from a past pose to the current pose. After reaching the destination pose according to an input command, the transition is ended. For example, when the system does not obtain or detect a subsequent input command for further moving the end effector, the end effector transition may end.

[0063] In embodiments, the smooth trajectory calculation for transitioning the end effector can be performed in a continuous control loop as multiple input commands are obtained in sequence during a time of a continuous motion of the robotic arm. The control loop can be regulated by a control loop rate, which can be a fixed or a minimum rate. The control loop rate can impose a restriction on the smooth calculation trajectory, such as a limit on the velocity and / or acceleration achieved to compute one or more increments of poses along the smooth trajectory. In examples, the control loop in the robotic system is configured to be greater than or equal to 1 kHz or 2 kHz. In other examples, the next virtual force for the next pose is simulated by the DC to transition the end effector within a certain time limit , such as within a 1 millimeter time window.

[0064] Examples of velocity and acceleration limits that are used in the DC algorithm include velocity saturation and / or acceleration bounding associated with the robotic system, the end effector, the robotic arm parts including joints and links, or other parts of the system. The DC simulates the virtual force based on a determined virtual object (e.g., a unit mass object) and the velocity and acceleration limits to avoid excessive or uncontrollable velocities that negatively affects system instability, safety, or cause mechanical stress.

[0065] In embodiments, the virtual force is simulated (e.g., in real time) based on at least one condition including a limit on a pose, a velocity, or an acceleration of the end effector or the robotic system. For example, if a certain pose of the end effector is to be avoided or restricted, the DC can be configured to simulate the virtual force based on this condition resulting in a smooth trajectory, in which the end effector is limited from being in the restricted pose.

[0066] In embodiments, simulating the vector comprises simulating (e.g., in 3D Cartesian coordinates) any combination of a displacement vector, a velocity vector, or an acceleration vector applied on the virtual mass object. A displacement vector represents the change in position of the virtual mass object from an initial point to a next point in a coordinate system.This vector describes the straight-line path and distance between the initial and next positions. The velocity vector represents the velocity and direction of the virtual mass object in space. The acceleration vector represents the rate of change of velocity of the virtual mass object with respect to time.

[0067] Examples of DC algorithms for simulating and applying the virtual force to the virtual mass object to generate a smooth trajectory include a PD controller which is a type of feedback control algorithm based on proportional control and derivative control of an object. The proportional control component of the PD controller produces an output signal that is directly proportional to an increment in pose between a current pose and a next pose. The next pose can be computed by adding an increment in pose to the current pose and the next pose. The increment in pose represents a difference between one or more respective parameter values of a current pose and a next pose. For example, in embodiments, the increment in pose is a displacement vector indicating the difference in parameter values of position and orientation between a current pose and next pose. The position and direction of the next pose can be calculated by adding the increment in pose, as a displacement vector, to the current pose. According to the PD controller, a proportional gain can be determined indicating the sensitivity of the algorithm to the increment in pose. Increasing the proportional gain amplifies the control signal, leading to an over corrective action that can cause overshooting in the transition of the end effector outside the smooth trajectory. In the context of movement control, overshooting refers to a situation where a moving object is displaced beyond a desired or target position before coming to a stop. To prevent the overshooting, the derivative control of the PD controller component is applied based on a determined derivative gain. The derivative control component produces an output signal based on the rate of change of the increment in pose. The PD controller accounts for the future behavior of the changes in the pose increments and provides a damping effect to reduce over correcting the increment in pose, and accordingly reduce overshooting and improve stability. The derivative gain determines the contribution of the derivative control in the PD controller.

[0068] In embodiments, simulating the virtual force and the next virtual force comprises applying a PD controller that is configured to damp overshooting in the transition of the end effector along a trajectory. This can be achieved by calculating the proportional gain and the derivative gain based on a limit on a velocity or an acceleration associated with the end effector. The next virtual force can be simulated based on the last applied virtual force, the proportional gain, and the derivative gain. Because the proportional gain and the derivative gain are calculated based on the velocity and / or acceleration limits, the virtual force can be simulatedby the PD controller to satisfy determined velocity and acceleration limits of the robotic system driving the end effector. The PD controller can be configured to damp overshooting in the transition of the end effector along the trajectory according to the velocity / accelerating limits to smoothen the trajectory. In embodiments, the virtual force is also simulated by the PD controller based on a limit on a pose of the end effector. For example, the PD controller can be configured to simulate the virtual force based on this condition resulting in a smooth trajectory free of a restricted pose.

[0069] In embodiments, the simulated virtual force is a pull force applied on a virtual mass object of a virtual spring. The virtual spring model can be characterized by the spring position according to the current pose, and a pull force on the virtual spring as a controlled variable. The virtual force can be simulated by a PD controller, where the proportional control generates a control signal based on the increment pose between the current pose and the next pose. The derivative control of the PD controller generates a control signal based on the rate of change of the increment pose, which provides a damping effect to stabilize the transition and reduce overshooting on the smooth trajectory.

[0070] In embodiments, the robotic system is a robotic medical system comprising a patient support platform, such as a table, a bed, etc . The two ends along the length of the patient support platform are respectively referred to as “head” and “leg”. The two sides of the patient support platform are respectively referred to as “left” and “right.” The patient support platform can include a support frame (e.g., a rigid frame). The robotic medical system can include a base for supporting the robotic medical system. For example, the base can include wheels that allow the robotic medical system to be easily movable or repositionable in a physical environment. In some embodiments, the wheels are omitted from the robotic medical system or are retractable, and the base can rest directly on the ground or floor. In some embodiments, the wheels are replaced with feet. The patient support platform can also be supported by a column that extends between the base and the patient support platform.

[0071] The robotic medical system also includes one or more robotic arms. The robotic arms can be configured to perform robotic medical procedures. The robotic arms can be moved between multiple different positions relative to the surgical table, such as, for example, an operating position, a parked position, or a stowed position. The robotic medical system can also include one or more bars that support the robotic arms (e.g., adjustable arm support or an adjustable bar). Each of the robotic arms can be supported on, and movably coupled to, a bar, such as by a respective base joint of the robotic arm. The bar can be configured to provide several degrees of freedom, including lift, lateral transition, tilt, etc. Each of the robotic armsand / or the adjustable arm supports is also referred to as a respective kinematic chain. In some embodiments, the adjustable arm supports are configured to provide a base position for one or more of the robotic arms for a robotic medical procedure. A robotic arm can be positioned relative to the patient support platform by transitioning the robotic arm along a length of its underlying bar and / or by adjusting a position and / or orientation of the robotic arm via one or more joints and / or links.

[0072] During a robotic medical procedure, one or more of the robotic arms can be configured to hold instruments, which can be robotically controlled medical instruments or tools, such as an endoscope and / or any other instruments that may be used during surgery. Examples of such instruments or tools include surgical instruments, illumination instruments, cutting instruments, tool drivers, imaging devices, sensors, etc. In some embodiments, the instruments or tools can be coupled to one or more accessories, such as one or more cannulas. In some embodiments, the links may be detachably coupled to the medical tool, which facilitates mounting and dismounting of the medical tool from the robotic arm.

[0073] In embodiments, the robotic medical system also includes a tower (e.g., tower viewer) and / or a physician console (or both). The tower provides support for controls, electronics, fluidics, optics, sensors, and / or power for the patient support platform and the physician console. The tower can include a display device. The display device or physician console can include a user interface for displaying a surgical view obtained by one or more cameras of the robotic medical system and / or one or more notifications to an operator of the robotic medical system. The user interface can be used by the physician operator to operate the robotic medical system, including the robotic arms, the patient support platform, etc. The physician console can provide both robotic controls and pre-operative and real-time information of a medical procedure to a physician operator. In some embodiments, the robotic medical system includes one or more input devices, including (e.g., buttons, switches, touch- sensitive surfaces, etc.). For example, the robotic medical system can include a pivot and stow keypad and / or a table keypad. The physician console can also include a foot pedal assembly including another set of input devices (e.g., foot pedals) to be positioned near the feet of the physician. In some embodiments, the physician console includes one or more HIDs that provide force and tactile feedback to a user as the user interacts with the physician console.

[0074] FIG. 1 illustrates an exemplary robotic medical system 200 according to some embodiments. In some embodiments, the robotic medical system 200 is a robotic surgery system. In the example of FIG. 1, the robotic medical system 200 comprises a patient support platform 202 (e.g., a patient support platform, a table, a bed, etc.). The two ends along thelength of the patient support platform 202 are respectively referred to as “head” and “leg”. The two sides of the patient support platform 202 are respectively referred to as “left” and “right.” The patient support platform 202 includes a support 204 (e.g., a rigid frame) for the patient support platform 202.

[0075] The robotic medical system 200 also includes a base 206 for supporting the robotic medical system 200. The base 206 includes wheels 208 (208-1 to 208-4) that allow the robotic medical system 200 to be easily movable or repositionable in a physical environment. In some embodiments, the wheels 208 are omitted from the robotic medical system 200 or are retractable, and the base 206 can rest directly on the ground or floor. In some embodiments, the wheels 208 are replaced with feet.

[0076] The robotic medical system 200 includes one or more robotic arms 210. In some embodiments, the robotic arms 210 can be configured to perform robotic medical procedures. Although FIG. 1 shows five robotic arms 210, it should be appreciated that the robotic medical system 200 may include any number of robotic arms, including less than five or six or more.

[0077] The robotic medical system 200 also includes one or more bars 220 (e.g., adjustable arm support or an adjustable bar) that support the robotic arms 210. Each of the robotic arms 210 is supported on, and movably coupled to, a bar 220, by a respective base joint of the robotic arm. In some embodiments, bar 220 can provide several degrees of freedom, including lift, lateral transition, tilt, etc. In some embodiments, each of the robotic arms 210 and / or the adjustable arm supports 220 is also referred to as a respective kinematic chain.

[0078] FIG. 1 shows three robotic arms 210 supported by the bar 220 that is in the field of view of the figure. The two remaining robotic arms are supported by another bar that is located across the other length of the patient support platform 202.

[0079] In some embodiments, the adjustable arm supports 220 can be configured to provide a base position for one or more of the robotic arms 210 for a robotic medical procedure. A robotic arm 210 can be positioned relative to the patient support platform 202 by transitioning the robotic arm 210 along a length of its underlying bar 220 and / or by adjusting a position and / or orientation of the robotic arm 210 via one or more joints and / or links. In some embodiments, the bar pose can be changed via manual manipulation, teleoperation, and / or power assisted motion.

[0080] In some embodiments, the adjustable arm support 220 can be transitioned along a length of the patient support platform 202. In some embodiments, transitioning the bar 220 along a length of the patient support platform 202 causes one or more of the robotic arms 210 supported by the bar 220 to be simultaneously transitioned with the bar 220 or relative to thebar 220. In some embodiments, the bar 220 can be transitioned while keeping one or more of the robotic arms 210 stationary with respect to the base 206 of the robotic medical system 200.

[0081] In the example of FIG. 1, the adjustable arm support 220 is located along a length of the patient support platform 202. In some embodiments, the adjustable arm support 220 may extend across a partial or full length of the patient support platform 202, and / or across a partial or full width of the patient support platform 202.

[0082] During a robotic medical procedure, one or more of the robotic arms 210 can also be configured to hold instruments 212 (e .g . , robotically controlled medical instruments or tools, such as an endoscope and / or any other instruments (e.g., sensors, illumination instrument, cutting instrument, etc.) that may be used during surgery), and / or be coupled to one or more accessories, including one or more cannulas, in accordance with some embodiments.

[0083] FIG. 2 illustrates components of a robotic medical system in accordance with some embodiments. In some embodiments, the robotic medical system 200 includes a tower 230 (e.g., tower viewer) or a physician console 240 (or both), as illustrated in FIG. 2. The tower 230 may provide support for controls, electronics, fluidics, optics, sensors, and / or power for the patient support platform 202 and the physician console 240. In some embodiments, the tower 230 includes a display device 232. The display device 232 can include a user interface for displaying a surgical view obtained by one or more cameras 606 of the robotic medical system and / or one or more notifications to an operator of the robotic medical system 200. In some embodiments, the physician console 240 can include a display device 242 having a user interface used by the physician operator for operating the patient support platform 202. For example, the display device 242 may include a user interface for displaying a surgical view obtained by one or more cameras 606 of the robotic medical system 200 and / or one or more notifications to an operator of the robotic medical system 200. The physician console 240 can provide both robotic controls and pre-operative and real-time information of a medical procedure to a physician operator. In some embodiments, the physician console 240 includes one or more input devices (e.g., buttons, switches, touch-sensitive surfaces, gimbals, etc.), such as a foot pedal 244. In some embodiments, the physician console 240 includes one or more haptic interface devices (HIDs) that provide force and tactile feedback to a user as the user interacts with the physician console 240.

[0084] In embodiments, the robotic arm of the robotic medical system includes a plurality of links that are connected by one or more joints, each comprising one or more degrees of freedom (DoFs), including lift, lateral transition, tilt, etc. The joints can include a base joint that is located at or near a base of the robotic arm. In some embodiments, the base jointcomprises a prismatic joint that allows the robotic arm to transition along the bar. The joints can include a second joint that rotates with respect to the base joint. The joints can comprise multiple degrees of freedom (DoFs) that facilitate both tilt and rotation of the links. For example, the joints can be elbow joints, wrist roll joints, etc. In embodiments, the joints of the robotic arm may move subject to hardware or safety limitations on position, velocity, acceleration, and / or torque.

[0085] In embodiments, a proximal end of the robotic arm may be connected to the base and a distal end of the robotic arm may be connected to a robotic manipulator, also referred to as an advanced device manipulator (ADM) (e.g., a tool driver, an instrument driver, or a robotic end effector, etc.). The ADM may be configured to control the positioning and manipulation of a medical instrument (e.g., a tool, a scope, etc.). The robotic arm can also include a cannula sensor for detecting the presence or proximity of a cannula to the robotic arm. For example, the robotic arm is placed in a docked state (e.g., docked position) when the cannula sensor detects the presence of a cannula (e.g., via one or more processors of the robotic medical system). In some embodiments, the robotic arm includes input devices (e.g., buttons, touchpoints, clutches, etc.).

[0086] During a medical procedure, the ADM of the robotic arm and / or a remote center of motion (RCM) of a medical tool can be kept in a static pose (e.g., position and / or orientation). An RCM refers to a point in space where a cannula or other access port through which a medical tool is inserted is constrained in motion. In some embodiments, the medical tool includes an end effector that is inserted through an incision or natural orifice of a patient while maintaining the RCM. In some embodiments, the medical tool includes an end effector that is in a retracted state during a setup process of the robotic medical system.

[0087] In embodiments, the robotic medical system can be configured to move one or more links of the robotic arm within a “null space” to avoid collisions with nearby objects (e.g., other robotic arms), while the ADM of the robotic arm and / or the RCM are maintained in their respective poses (e.g., positions and / or orientations). The null space can be viewed as the set of joint states through which a robotic arm can move that does not result in movement of the ADM and / or RCM, thereby maintaining the position and / or the orientation of the medical tool (e.g., within a patient). In some embodiments, a robotic arm can have multiple positions and / or configurations available for each pose of the ADM.

[0088] For a robotic arm to move an instrument to a desired pose in space, in certain embodiments, the robotic arm may have at least six DoFs - three DoFs for transition (e.g., X, Y, and Z positions) and three DoFs for rotation (e.g., yaw, pitch, and roll). In someembodiments, each joint may provide the robotic arm with a single DoF, and thus, the robotic arm may have at least six joints to achieve freedom of motion to position the ADM at any pose in space. To further maintain the ADM of the robotic arm and / or the remote center of motion in a desired pose, the robotic arm may further have at least one additional “redundant joint.” Thus, in certain embodiments, the system may include a robotic arm having at least seven joints, providing the robotic arm with at least seven DoFs. In some embodiments, the robotic arm may include a subset of joints each having more than one degree of freedom thereby achieving the additional DoFs for null space motion. Depending on the embodiments, the robotic arm may have a greater or fewer number of DoFs.

[0089] In some embodiments, the ADM may include a cannula coupled to its end to receive and guide a tool (e.g., surgical instrument). The tool may include an end effector at the distal end of the tool. The plurality of the joints of the robotic arm can be actuated to position and orient the ADM, which actuates the end effector for robotic surgeries. In some embodiments, the end effector is at a tool shaft end. In other embodiments, the tool shaft end is a tip of a needle or other object. In some embodiments, the ADM includes a cannula release lever for releasing the cannula from the tool drive.

[0090] FIGS. 3A and 3B illustrate different views of an exemplary robotic arm 350, in accordance with some embodiments. In accordance with aspects of the subject technology, the surgical robotic arm 350 includes a tool drive 352 and a cannula 362 loaded with a robotic surgical tool. FIGS. 3A and 3B show the robotic arm 350 may include links 351 (e.g., link 351- 1, link 351-2, link 351-3, link 351-4, and / or link 351-5) and actuated joint modules (e.g., a joint 353, see also joints JI, J2, J3, J4, J5, J6, J7, and J8) for actuating the plurality of links relative to one another. The j oint modules may include various types, such as a pitch j oint or a roll j oint, which may substantially constrain the movement of the adjacent links around certain axes relative to others. Also shown in the exemplary design of FIGS. 3A and 3B is atool drive 352 attached to the distal end of the robotic arm 350. The tool drive 352 includes a carriage 354 and a stage 356.

[0091] FIG. 3A illustrates that in some embodiments, the tool drive 352 may include a cannula 362 coupled to its end to receive and guide a surgical instrument 360 or end effector (e.g., endoscopes, staplers, scalpel, scissors, clamp, retractor, etc.). The surgical instrument (or “tool”) 360 may include an end effector 364 at the distal end of the tool. The plurality of the joint modules of the robotic arm 350 can be actuated to position and orient the tool drive 352, which actuates the end effector 364 for robotic surgeries. The end effector 364 is at atool shaft end. In other embodiments, the tool shaft end is a tip of a needle or other object. In someembodiments, the tool drive 352 includes a cannula release lever 358 for releasing the cannula 362 from the tool drive 352.

[0092] In some embodiments, the robotic arm 350 includes input devices (e.g., buttons, touchpoints, etc.) FIG. 3B illustrates that in some embodiments, the robotic arm 350 includes a clearance adjustment touchpoint 366, an instrument clutch 368, a port clutch 370, a forearm pivot touchpoint 372, and a forearm multipoint touchpoint 374.

[0093] FIGS. 3A and 3B illustrate that the link 351-1 includes a first end that is coupled to the joint JI . In some embodiments, the robotic arm 350 includes a joint JO that actuates a second end of the link 351-1. In some embodiments, the joint JO is a table pivot joint and resides under the surgical table top. Joint JO is nominally held in place during surgery. Joints JI to J5 form a setup or Cartesian arm and are nominally held in place during surgery, and do not contribute to motion during surgical teleoperation. Joints J6 and J7 (see FIG. 4) form a spherical arm that may actively move during surgery or teleoperation. Joint J8 transitions the surgical tool 360, such as the end effector 364, as part of a tool driver. Joint J8 may actively move during surgery. Joints J6, J7, and J8 actively position a tool shaft end (e.g., end effector 364) during surgery while maintaining an entry point into the patient at a fixed or stable location (e.g., remote center of motion) to avoid stress on the body wall of the patient. During setup, any of the joints J0-J8 may move. During surgery, the joints J6, J7, and J8 may move subject to hardware or safety limitations on position, velocity, acceleration, and / or torque. The surgical tool 360 may include none, one, or more (e.g., three) joints, such as a joint for tool rotation plus any number of additional joints (e.g., wrists, rotation about a longitudinal axis, or other type of motion). Any number of degrees of freedom may be provided, such as the three degrees from the joints J6, J7, and J8 and none, one, or more degrees from the surgical tool 360.

[0094] FIG. 4 illustrates part of the robotic arm 350 and the surgical tool 360 providing six degrees of freedom (DOF). The six DOF correspond to movement of six active joints during teleoperation. In some embodiments, the active joints include three joints on the surgical tool 360 - rotation at joint J9, pitch at wrist joint J 10, and yaw at wrist joint JI 1. The active joints include three joints on the robotic arm 350 - spherical roll joint J6, spherical pitch joint J7, and tool transition joint J8. In some embodiments, other joints providing the six degrees of freedom may be used.

[0095] In some embodiments, a user can command movement for fewer than six degrees of freedom. For example, in some embodiments, five, four, or fewer active joints may be provided. In an example of five DOF, the active joints include three joints on the robotic arm 350 - spherical roll joint J6, spherical pitch joint J7, and tool transition joint J8 - and two activejoints on the surgical tool - rotation at joint J9 and articulation as another joint. In some embodiments, active joint arrangements may be used, such as providing two or fewer DOF on the robotic arm during teleoperation.

[0096] In some embodiments, the robotic medical system includes a coordinate system (e.g., a robot coordinate system, a coordinate frame, a system frame, etc. that may be a Cartesian or non-Cartesian coordinate system), and respective positions of the patient support platform, the robotic arms, the robotic arms, the adjustable arm supports, and / or instruments are represented as coordinates (e.g., x-, y-, and z-coordinates) on the coordinate system. For example, the robotic medical system (e.g., one or more processors of the robotic medical system) may be configured to identify positions and orientations of the patient support platform, the robotic arms, the adjustable arm supports, and / or instruments based on coordinates in the coordinate system.

[0097] In embodiments, the systems described herein can be configured to apply a DC algorithm to generate a trajectory based on a virtual force model for transitioning an end effector of one or more robotic arms. For example, the DC can be a computer executable algorithm applied by a robotic medical system to transition an ADM on one or more robotic arms. The ADM can also be holding an instrument. In other examples, the DC is applied by the robotic medical system to transition an end effector holding a tool drive on one or more robotic arms.

[0098] In examples, if end effectors on two (or more) robotic arms are driven by a surgeon at a same time, a DC algorithm can be executed to transition each end effector individually. In other embodiments, the DC algorithm is configured to be aware of the simultaneous input instructions to drive at least two end effectors on two respective robotic arms. Accordingly, the DC computes, such as in parallel or within the same loop control rate, a smooth trajectory for each respective end effector. In this case, the virtual force simulation can be performed based on system constraints and velocity and / or acceleration limits satisfying restrictions on the movement of multiple robotic arms at a same time.

[0099] FIGS. 5 A, 5B, and 5C illustrate an end effector transition on a smooth trajectory based on a virtual force simulation, in accordance with some embodiments. The virtual force simulation is performed by executing a DC algorithm by a computer system or a processor of any of the systems described herein. The DC algorithm can be executed in an iterative manner to simulate a sequence of virtual forces and compute a sequence of respective poses generating a smooth trajectory to transition the end effector. In embodiments, the DC algorithm is configured to calculate an initial trajectory according to an input for a start pose and adestination pose and based on velocity and / or acceleration limits. The DC algorithm then computes in an iterative manner the sequence of poses according to the initial calculated trajectory. In other embodiments, the DC algorithm begins computing in an iterative manner the sequence of poses according to the input without calculating an initial trajectory. The smooth trajectory is hence generated incrementally by connecting the sequence of computed poses. The system is also configured to drive at least one of the plurality of actuators of a robotic arm (e.g., in joint space) of the end effector according to each computed pose to drive the robotic arm. Based on the outputs of the DC and the dependent motion of the robotic arm, the end effector can be transitioned on a smooth trajectory from a start pose to a destination pose.

[0100] In FIG. 5 A, a start pose 1010 and a destination pose 1020 for the end effector (not shown) of the robotic arm 1000 are determined in 3D space. For example, the start pose 1010 and the destination pose 1020 are determined as Cartesian coordinates. The start pose 1010 and the destination pose 1020 can be obtained from an input command by an input device of the system which instructs moving the end effector from the start pose 1010 to the destination pose 1020. The DC performs the virtual force simulation to compute a sequence of one or more next poses determining a smooth trajectory 1030 from the start pose 1010 to the destination pose 1020. In examples, the DC algorithm is a PD controller. In some examples, the virtual force is also simulated as a pull force for a virtual spring object.

[0101] In FIG. 5B, the DC simulates a virtual force 1050 based on a determined virtual mass object 1040. The values of the virtual force 1050 and the virtual mass object 1040 can be determined based on dynamic characteristics of the system including velocity and / or acceleration limits. In embodiments, the virtual mass object 1040 is set equal to a unit mass object. In other examples, the virtual mass object 1040 can have other values. The virtual mass object 1040 is located in 3D space (indicated by Cartesian coordinates) to match the location of the end effector or a surgical tool. The location of the virtual mass object 1040 matches the position indicated by the start pose 1010, as shown in FIG. 5B. The inputs to the DC algorithm include both the start pose 1010 and the destination pose 1020. Accordingly, the DC iterative algorithm is set to compute a sequence of one or more poses in iterative increments to displace the virtual mass object 1040 from the start pose 1010 to the destination pose 1020 under the velocity and / or acceleration limits and other possible dynamic system constraints.

[0102] In FIG. 5C, the virtual force 1050 is applied on the virtual mass object 1040 to push or pull the virtual mass object 1040 resulting in an increment in pose. A next pose 1070 can be computed by adding the increment in pose to the start pose 1010. The increment in poseprovides a first transition increment, as a smooth step, on the smooth trajectory 1030. After computing the next pose 1070, the actuators can be driven and controlled in joint space to drive the robotic arm providing the actual motion that transitions the end effector from the start pose 1010 to the next pose 1070.

[0103] At a next iteration step, the DC simulates a next virtual force 1080 based on the same virtual mass object 1040, the velocity and / acceleration limits, and other dynamic characteristics of the system that may be set. Similar to the previous iteration of the DC algorithm, the next virtual force 1080 is applied on the virtual mass object 1040 to push or pull the virtual mass object 1040 from the next pose 1070, resulting in a new increment in pose. The new increment in pose provides a next transition increment (not shown) on the smooth trajectory 1030, which is then realized by the joint space control of the actuators. This iterative process of the DC algorithm is repeated to generate the smooth trajectory 1030 in increments of smooth steps accordingly providing, with the parallel joint space control of the actuators, a smooth transition of the end effector.

[0104] FIG. 6 illustrates a flowchart diagram for a method 1100 performed by a robotic system including one or more processors, in accordance with some embodiments. The steps of the method 1100 can be performed by executing a computer executed DC algorithm. Other steps of the DC algorithm not shown in the method 1100 may also be performed as described in the embodiments of this disclosure. At step 1110 of the method 1100, an input command comprising a start pose and an end pose for an end effector of a surgical tool is received. The end pose is the destination pose indicated by the input command to end the transition of the end effector. The input command can be received by the one or more processors from an input device, such as an HID operated by a surgeon during a surgical procedure. For example, the surgeon may be driving a robotic arm of the end effector.

[0105] At step 1120, a next pose is computed on a smooth trajectory between the start pose and the end pose based on a simulation of a virtual force. The next pose is computed as the sum of the start pose and the increment in pose, and the increment in pose may be based on velocity and / or acceleration limits of the end effector or robotic system. The next pose may be represented as coordinates in 3D space (e.g., Cartesian coordinates). The virtual force is applied to a virtual mass object to move the virtual mass object from the start pose to the next pose, in which the virtual mass object is simulated to reflect a pose of the end effector. The virtual force can be simulated by the DC algorithm in an iterative manner. In examples, the DC algorithm can be a PD controller or another feedback control algorithm configured to simulate a virtualforce, compute increments in pose, and regulate motion trajectory to satisfy velocity and acceleration limits of the end effector or robotic system.

[0106] At step 1130, the end effector is transitioned from the start pose to the next pose based on the applied virtual force. In embodiments, after computing the next pose, the next pose coordinates in 3D space (e.g., Cartesian coordinates) are processed in joint space to determine the positioning of the joints / links or parts of the robotic arm and accordingly send control signals to the actuators of the robotic arm. The control signals drive the actuators to transition the end effector from the start pose to the next pose on the smooth trajectory. For example, the joint configurations of the parts of the robotic arm to achieve the desired pose of the end effector can be determined by a mathematical process referred to as inverse kinematics. Inverse kinematics involves solving a set of equations or constraints to find the joint variables (in joint space) that satisfy the desired end effector position or pose (in task space). The set of equations or constraints may depend on the structure and geometry of the robotic system. In examples, inverse kinematics can be applied to a pose using algorithms based on nonlinear equations, which is used to drive at least one of the plurality of actuators to perform joint movements that achieve the desired position and orientation of the end effector and move the end effector along the smooth trajectory.

[0107] At step 1140, the steps 1120 and 1130 are repeated, in real time, until reaching the end pose. The one or more processors repeat computing a new next pose on the smooth trajectory based on a new simulation of a next virtual force applied to the virtual mass object to move the virtual mass object to the new next pose. The one or more processors also repeat transitioning the end effector to the new next pose based on the applied next virtual force until reaching the end pose. For example, at each iteration, a new next pose is computed in task space (e.g., Cartesian space) based on a new simulation of a next virtual force. The end effector is then transitioned by solving, based on the new pose information, the inverse kinematic equations in joint space to determine the positioning of the joints / links of the robotic arm and drive at least one of the plurality of actuators accordingly.

[0108] FIG. 7 illustrates a flowchart diagram for a method 1200 performed by a robotic system including one or more processors, in accordance with some embodiments. The method 1200 is a control method for regulating the velocity and / or acceleration for the transition of the end effector from a start pose to an end pose. The steps of the method 1200 for regulating the velocity and / or acceleration of the transition can be performed according to a computer executed DC algorithm. The DC algorithm can provide transition control for the end effector by changing the applied simulated force to regulate the velocity and / or acceleration along asmooth trajectory. The velocity and / or acceleration limits used in the DC algorithm can include conditions of velocity and / or acceleration that are changed during the iterative execution of the DC algorithm to adjust the virtual forces and accordingly the increments in pose in a manner that regulates the velocity and / or acceleration of the end effector or robotic arm to produce a smooth trajectory. Other steps of the DC algorithm not shown in the method 1200 may also be useful to support implementing the method 1200 as described in the embodiments of this disclosure.

[0109] At step 1210 of the method 1200, an initial velocity of the end effector from the start pose is accelerated to reach a constant velocity by simulating a first sequence of one or more virtual forces based on first conditions of velocity and / or acceleration. The first conditions of velocity and / or acceleration are part of the velocity and / or acceleration limits used in the DC algorithm to simulate the virtual forces. For example, the first conditions of velocity and / or acceleration can include a linear velocity increase function and a maximum velocity equal to the constant velocity. The DC algorithm can be executed in an iterative manner and in parallel with joint space control, as described in the embodiments herein, to transition the end effector from the start pose in a sequence of one or more poses based on a sequence of one or more virtual forces.

[0110] At step 1220, the constant velocity is maintained along a smooth trajectory to the end pose by simulating a second sequence of one or more virtual forces based on second conditions of velocity and / or acceleration. The second conditions of velocity and / or acceleration are part of the velocity and / or acceleration limits in the DC algorithm. For example, the second conditions of velocity and / or acceleration include a fixed velocity value. At step 1230, the constant velocity is slowed down to pause the end effector at the end pose by simulating a third sequence of one or more virtual forces based on third conditions of velocity and / or acceleration. The third conditions of velocity and / or acceleration are also part of the velocity and / or acceleration limits. For example, the slowing of the velocity can be achieved by replacing the second velocity conditions with third velocity conditions that include a linear velocity decrease function and a minimum velocity equal to zero.[oni] FIG. 8 is a schematic diagram illustrating electronic components of a medical robotic system (e.g., a surgical robotic system) in accordance with some embodiments.

[0112] The robotic medical system (e.g., surgical robotic system) includes one or more processors 380, which are in communication with a computer-readable storage medium 382 (e.g., computer memory devices, such as random-access memory, read-only memory, static random-access memory, and non-volatile memory, and other storage devices, such as a harddrive, an optical disk, a magnetic tape recording, or any combination thereof) storing instructions for performing any methods described herein (e.g., operations described with respect to FIGS. 5A-5C, 6, and 7). The one or more processors 380 are also in communication with an input / output controller 384 (via a system bus or any suitable electrical circuit). The input / output controller 384 receives sensor data from one or more sensors 388-1, 388-2, etc., and relays the sensor data to the one or more processors 380. The input / output controller 384 also receives instructions and / or data from the one or more processors 380 and relays the instructions and / or data to one or more actuators, such as first motors 387-1 and 387-2, etc. In some embodiments, the input / output controller 384 is coupled to one or more actuator controllers 386 and provides instructions and / or data to at least a subset of the one or more actuator controllers 386, which, in turn, provide control signals to selected actuators. In some embodiments, the one or more actuator controllers 386 are integrated with the input / output controller 384 and the input / output controller 384 provides control signals directly to the one or more actuators (without a separate actuator controller). Although FIG. 4 shows that there is one actuator controller 386 (e.g., one actuator controller for the entire medical robotic system; in some embodiments, additional actuator controllers may be used (e.g., one actuator controller for each actuator, etc.). In some embodiments, the one or more processors 380 are in communication with one or more displays 381 for displaying information as described herein.

[0113] It should be noted that the terms “couple,” “coupling,” “coupled” or other variations of the word couple as used herein may indicate either an indirect connection or a direct connection. For example, if a first component is “coupled” to a second component, the first component may be either indirectly connected to the second component via another component or directly connected to the second component.

[0114] The functions for determining whether a tool is within or outside a surgical field of view provided by a camera or scope and rendering one or more indicators representing positions or directions of one or more medical tools described herein may be stored as one or more instructions on a processor-readable or computer-readable medium. The term “computer- readable medium” refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such a medium may comprise random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that a computer-readablemedium may be tangible and non-transitory. As used herein, the term “code” may refer to software, instructions, code or data that is / are executable by a computing device or processor.

[0115] The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for proper operation of the method that is being described, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0116] As used herein, the term “plurality” denotes two or more. For example, a plurality of components indicates two or more components. The term “determining” encompasses a wide variety of actions and, therefore, “determining” can include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in atable, a database or another data structure), ascertaining and the like. Also, “determining” can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, “determining” can include resolving, selecting, choosing, establishing and the like.

[0117] The phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” describes both “based only on” and “based at least on.”

[0118] As used herein, the term “exemplary” means “serving as an example, instance, or illustration,” and does not necessarily indicate any preference or superiority of the example over any other configurations or implementations.

[0119] As used herein, the term “and / or” encompasses any combination of listed elements. For example, “A, B, and / or C” includes the following sets of elements: A only, B only, C only, A and B without C, A and C without B, B and C without A, and a combination of all three elements, A, B, and C.

[0120] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the invention. For example, it will be appreciated that one of ordinary skill in the art will be able to employ a number corresponding alternative and equivalent structural details, such as equivalent ways of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for producing particular actuation motions, and equivalent mechanisms for delivering electrical energy. Thus, the present invention is not intended to be limited to theembodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMSWhat is claimed is:

1. A surgical robot, comprising: a plurality of actuators; an end effector coupled to and configured to be driven by the actuators; and a processor coupled to the actuators and configured to: receive an input command to move the end effector from a current pose to a destination pose; compute a next pose on a smooth trajectory between the current pose and the destination pose based on a simulation of a virtual force applied to a virtual mass object to move the virtual mass object from the current pose to the next pose; and drive at least one of the plurality of actuators to move the end effector along the smooth trajectory.

2. The surgical robot of claim 1, wherein the virtual force is applied to the virtual mass object to compute the next pose based on a dynamic controller.

3. The surgical robot of any of claims 1-2, wherein the virtual mass object is a unit mass object simulated as an end point of the end effector.

4. The surgical robot of any of claims 1-3, wherein the processor is further configured to drive at least one of the plurality of actuators based on a sequence of simulated virtual forces to transition the end effector from the current pose to the destination pose in respective increments of poses on the smooth trajectory.

5. The surgical robot of claim 4, wherein the simulated virtual forces control transitioning the end effector along the smooth trajectory without jerks or abrupt movements and without displacing the end effector outside the smooth trajectory.

6. The surgical robot of any of claims 1-5, wherein the processor is further configured to simulate at least one of a displacement vector, a velocity vector, or an acceleration vector applied to the virtual mass object to move the virtual mass object on the smooth trajectory.

7. The surgical robot of any of claims 1-6, wherein the virtual force is represented by vector coordinates that are three-dimensional Cartesian coordinates.

8. The surgical robot of any of claims 1-7, wherein the virtual force is simulated to compute the next pose on the smooth trajectory and drive at least one of the plurality of actuators in real time during a teleoperation while the surgical robot is in motion.

9. The surgical robot of any of claims 1-8, wherein the virtual force is simulated based on at least one of velocity saturation or acceleration bounding that satisfy determined velocity and / or acceleration limits.

10. A computer-implemented method, comprising: receiving an input command comprising a start pose and an end pose for an end effector of a surgical robot; computing a next pose on a smooth trajectory between the start pose to the end pose based on a simulation of a virtual force applied to a virtual mass object to move the virtual mass object from the start pose to the next pose; transitioning the end effector from the start pose to the next pose based on the applied virtual force; and repeating computing a new next pose on the smooth trajectory based on a new simulation of a next virtual force applied to the virtual mass object to move the virtual mass object to the new next pose, and transitioning the end effector to the new next pose based on the applied next virtual force until reaching the end pose.

11. The computer-implemented method of claim 10, wherein simulating the next virtual force comprises: computing a proportional gain and a derivative gain based on a limit on a velocity or an acceleration associated with the end effector; and simulating the next virtual force based on the simulated virtual force, the proportional gain, and the derivative gain.

12. The computer-implemented method of any of claims 10-11, further comprising driving, in joint space and based on the next pose, a robotic arm attached to the end effector.

13. The computer-implemented method of any of claims 10-12, wherein the virtual force and the next virtual force are simulated using a proportional derivative (PD) controller satisfying determined velocity and acceleration limits of a robotic system coupled to the end effector.

14. The computer-implemented method of any of claims 10-13, wherein the virtual force and the next virtual force are simulated using a spring force model that pulls the virtual mass object from the start pose to the next pose.

15. The computer-implemented method of any of claims 10-14, wherein the next virtual force is simulated and the end effector is transitioned within a 1 millimeter time window.

16. The computer-implemented method of any of claims 10-15, wherein simulating the next virtual force and transitioning the end effector is repeated at a control loop rate higher than 1 kilohertz (kHz).

17. The computer-implemented method of any of claims 10-16, wherein simulating the virtual force and the next virtual force comprises applying a proportional-derivative (PD) controller that is configured to damp overshooting in transitioning the end effector along the smooth trajectory.

18. The computer-implemented method of claim 17, wherein the PD controller is applied based on at least one condition including a limit on a pose, a velocity, or an acceleration associated with the end effector.

19. The computer-implemented method of any of claims 10-18, further comprising: accelerating an initial velocity of the end effector from the start pose to reach a constant velocity by simulating a first sequence of one or more virtual forces based on first conditions of velocity and / or acceleration; maintaining the constant velocity along the smooth trajectory to the end pose by simulating a second sequence of one or more virtual forces based on second conditions of velocity and / or acceleration; andslowing down the constant velocity to stop the end effector at the end pose by simulating a third sequence of one or more virtual forces based on third conditions of velocity and / or acceleration.

20. The computer-implemented method of any of claims 10-19, wherein the end effector is transitioned based on the virtual force and the next virtual force to prevent sharp or discrete steps in the smooth trajectory.

21. A non-transitory computer readable storage medium storing computer-executable instructions, when executed by one or more processors of a robotic system, cause the one or more processors to: receive an input command to move a robotic manipulator from a current pose to a destination pose; compute a next pose on a smooth trajectory between the current pose and the destination pose based on a dynamic controller (DC) of a virtual force applied to a virtual mass object, wherein the virtual force moves the virtual mass object from the current pose to the next pose; and send a transition command to the robotic manipulator based on the virtual force.

22. The non-transitory computer readable storage medium of claim 21, wherein the computer-executable instructions further cause the one or more processors to compute in real time a sequence of next poses based on the DC of respective virtual forces applied to the virtual mass object to move the virtual mass object on the smooth trajectory to the destination pose.

23. The non-transitory computer readable storage medium of any of claims 21 -22, wherein the next pose comprises a position and an orientation, indicated by a pitch, a roll, and a yaw, of the robotic manipulator.

24. The non-transitory computer readable storage medium of any of claims 21-23, wherein simulating the virtual force comprises simulating at least one of a displacement vector, a velocity vector, or an acceleration vector applied to the virtual mass object to move the virtual mass object on the smooth trajectory.

25. The non-transitory computer readable storage medium of any of claims 21 -24, wherein the computer-executable instructions further cause the one or more processors to simulate the virtual force based on the DC to drive the robotic manipulator in real time during a teleoperation while the robotic system is in motion.

26. The non-transitory computer readable storage medium of any of claims 21-25, wherein the computer-executable instructions further cause the one or more processors to simulate the virtual force in real time based on at least one condition including a limit on a pose, a velocity, or an acceleration of at least one of the robotic manipulator or the robotic system.

Images