System and method for controlling robotic movement of a tool based on a virtual boundary

Abstract

A system and method are provided for controlling robotic movement of a tool based on one or more virtual boundaries. The system includes a tool and a manipulator for supporting the tool. A control system controls the operation of the manipulator and the movement of the tool based on a relationship between the tool and one or more virtual boundaries associated with a target site. The control system includes a boundary handler for determining whether the tool conforms to or penetrates the one or more virtual boundaries.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the full benefit of U.S. Provisional Patent Application No. 63 / 000,860, filed March 27, 2020, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure generally relates to systems and methods for controlling robotic movement of a tool based on a virtual boundary. [Background technology]

[0003] The surgical system may include a robotic manipulator and a tool coupled to the manipulator for performing a surgical procedure on a patient. During surgery, the surgical system may constrain the movement of the tool to avoid encroaching on a virtual boundary established to protect portions of the patient's anatomy from the tool.

[0004] In some situations, a user may wish to adjust the patient's position to improve access to a target portion of the patient's anatomy, or to improve the user's visualization of the patient's anatomy, or the patient's anatomy may move for other reasons. To maintain compliance with the virtual boundary as the patient's anatomy moves, the surgical system may instruct the manipulator to autonomously move the tool to compensate for such movement of the patient's anatomy. However, some users may prefer to limit such autonomous movement of the tool. For example, the user may not want the tool to move unexpectedly, or may want tool movement to occur only in response to input from the user. Summary of the Invention

[0005] This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to limit the scope of the claimed subject matter, nor is it intended to identify key features or essential features of the claimed subject matter.

[0006] According to a first aspect, a surgical system is provided that includes a tool, a manipulator supporting the tool, and a control system that controls operation of the manipulator and movement of the tool based on a relationship between the tool and a virtual boundary associated with a target site. The control system includes a user input having a first input state and a second input state. The control system is configured to enable autonomous, boundary-compliant movement of the tool when the user input is in the first input state, so that the tool maintains compliance with the virtual boundary. The control system is configured to disable autonomous, boundary-compliant movement of the tool when the user input is in the second input state. The control system includes a boundary handler for determining whether the tool is encroaching on the virtual boundary in response to a transition of the user input from the second input state to the first input state.

[0007] According to a second aspect, there is provided a surgical system including a tool, a manipulator supporting the tool, and a control system for controlling operation of the manipulator and movement of the tool based on a relationship between the tool and a first virtual boundary associated with a target site. The control system includes a virtual boundary selector that allows a user to select a second virtual boundary associated with the target site. The control system is configured to allow the user to select the second virtual boundary using the virtual boundary selector while the control system maintains compliance of the tool with the first virtual boundary. The control system includes a boundary handler for determining whether the tool complies with the second virtual boundary in response to the user selecting the second virtual boundary.

[0008] According to a third aspect, a surgical system is provided that includes a tool, a manipulator supporting the tool, and a control system that controls the operation of the manipulator and the movement of the tool based on a relationship between the tool and a virtual boundary associated with a target site. The manipulator is operable in multiple modes, including a manual mode in which the manipulator moves the tool in response to user forces and torques applied to the tool by a user, and a semi-autonomous mode in which the manipulator moves the tool along a tool path. The control system includes a path handler for generating a lead-in path from the current position of the tool to the tool path in response to the control system transitioning to the semi-autonomous mode. The control system also includes a boundary handler that, prior to moving the tool along the lead-in path, determines whether movement of the tool along the lead-in path maintains compliance with the virtual boundary by modeling the motion of multiple stereotactic interaction features associated with the tool to determine whether the stereotactic interaction features comply with the virtual boundary.

[0009] According to a fourth aspect, there is provided a method for controlling movement of a manipulator supporting a tool based on a relationship between the tool and a virtual boundary associated with a target site. The method includes initiating autonomous, boundary-compliant movement of the tool in response to movement of the virtual boundary relative to the tool when user input is in a first input state, such that the tool maintains compliance with the virtual boundary. The method also includes disabling the autonomous, boundary-compliant movement of the tool when the user input is in a second input state, and determining whether the tool is encroaching on the virtual boundary in response to the user input transitioning from the second input state to the first input state.

[0010] According to a fifth aspect, there is provided a method for controlling movement of a manipulator supporting a tool based on a relationship between the tool and a first virtual boundary associated with a target region. The method includes allowing a user to select a second virtual boundary associated with the target region. The method also includes allowing the user to select the second virtual boundary while maintaining conformance of the tool to the first virtual boundary, and determining whether the tool conforms to the second virtual boundary in response to the user selecting the second virtual boundary.

[0011] According to a sixth aspect, there is provided a method for controlling operation of a manipulator supporting a tool based on a relationship between the tool and a virtual boundary associated with a target site, the manipulator being operable in multiple modes, including a manual mode in which the manipulator moves the tool in response to user forces and torques applied to the tool by a user, and a semi-autonomous mode in which the manipulator moves the tool along a tool path. The method includes transitioning the manipulator to the semi-autonomous mode, and generating a lead-in path from a current position of the tool to the tool path in response to transitioning the manipulator to the semi-autonomous mode. The method also includes, prior to moving the tool along the lead-in path, determining whether movement of the tool along the lead-in path maintains compliance with the virtual boundary by modeling the motion of multiple stereotactic interaction mechanisms associated with the tool to determine whether the stereotactic interaction mechanisms comply with the virtual boundary.

[0012] According to a seventh aspect, there is provided a surgical system including a localization system configured to track a first object and a second object, and at least one controller configured to associate a virtual boundary with one or more first objects, associate a first stereotactic interaction mechanism and a second stereotactic interaction mechanism with one or more second objects, define a first parameter of the first stereotactic interaction mechanism, and define a second parameter of the second stereotactic interaction mechanism, the first parameter being different from the second parameter, and generate a response based on an interaction between at least one of the first and second stereotactic interaction mechanisms and the virtual boundary.

[0013] According to an eighth aspect, there is provided a method of operating a surgical system including a localization system and at least one controller, the method including: tracking a first object and a second object using the localization system; associating, with the at least one controller, a virtual boundary with the one or more first objects; associating, with the at least one controller, a first stereotactic interaction mechanism and a second stereotactic interaction mechanism with the one or more second objects; defining, with the at least one controller, a first parameter for the first stereotactic interaction mechanism; defining, with the at least one controller, a second parameter for the second stereotactic interaction mechanism, wherein the first parameter is different from the second parameter; and generating, with the at least one controller, a response based on an interaction between at least one of the first and second stereotactic interaction mechanisms and the virtual boundary.

[0014] Any of the above aspects may be combined in whole or in part.

[0015] Any of the above aspects may be utilized in any of the following embodiments, regardless of whether such embodiments are utilized in whole or in part.

[0016] In some embodiments, the manipulator is a surgical robotic manipulator comprising a base and a plurality of links and joints forming a robotic arm. In some embodiments, the manipulator is a handheld manipulator, freely supported in a user's hand against gravity, the base being a base portion of a tool (e.g., the portion held by the user in the free hand), and the tip of the tool being movable relative to the base portion. In some embodiments, the tool is a rotary cutting burr, a saw, a cutting guide, an ultrasonic vibration tool, a laser cutting tool, or the like.

[0017] In some embodiments, the control system is configured to initiate a recovery mode in response to the tool encroaching on the virtual boundary. In some embodiments, the encroachment occurs when the user input transitions from the second input state to the first input state. In some embodiments, when the user input is in the first input state, autonomous, boundary-compliant movement of the tool remains disabled in the recovery mode.

[0018] In some embodiments, the tool includes a tool drive, and the control system is configured to disable operation of the tool drive in response to the tool penetrating the virtual boundary. In some embodiments, the penetrating occurs when the user input transitions from the second input state to the first input state.

[0019] In some embodiments, the control system is configured to guide a user to position the tool to conform to the virtual boundary in the recovery mode. In some embodiments, the guidance is performed by generating user feedback. In some embodiments, the feedback includes one or more of audible feedback, visual feedback, and haptic feedback. In some embodiments, the control system is configured to stop generating the user feedback when the tool is positioned to conform to the virtual boundary. In some embodiments, the control system is configured to limit relative movement between the tool and the virtual boundary. In some embodiments, the limiting of relative movement occurs by generating a boundary constraint using a boundary handler when the user input is in a first input state.

[0020] In some embodiments, the control system includes a constraint solver for computing constraint forces adapted to maintain the tool in compliance with the virtual boundary based on the boundary constraints. In some embodiments, the control system includes a virtual simulator for simulating the dynamics of the tool in the virtual simulation based on the constraint forces and outputting a commanded pose. In some embodiments, the control system is configured to command the manipulator to move the tool based on the commanded pose.

[0021] In some embodiments, the boundary handler is operable between a boundary-enabled state and a boundary-disabled state. The boundary-enabled state is a state in which boundary constraints are sent from the boundary handler to the constraint solver, thereby enabling autonomous, boundary-compliant movement of the tool. In some embodiments, autonomous boundary-compliant movement is enabled when the virtual boundary moves relative to the tool in a manner that causes the tool to encroach on the virtual boundary. In some embodiments, a boundary-disabled state in which boundary constraints are no longer sent from the boundary handler to the constraint solver disables autonomous, boundary-compliant movement of the tool, such that the virtual boundary can move relative to the tool in a manner that causes the tool to encroach on the virtual boundary. In some embodiments, the boundary handler is configured to operate in the boundary-disabled state in response to a user input transition from a first input state to a second input state. In some embodiments, the boundary handler is configured to operate in the boundary-enabled state in response to a user input transition from the second input state to the first input state, causing the tool to adhere to the virtual boundary. In some implementations, the boundary handler is configured to operate with the boundary disabled in response to a tool entering the virtual boundary and a user input transition from the second input state to the first input state.

[0022] In some embodiments, the control system is configured to provide haptic feedback to the user to guide the user to conform the tool to the virtual boundary. In some embodiments, the feedback occurs by activating one or more guiding constraints to guide the tool to conform to the virtual boundary. In some embodiments, the control system is configured to provide haptic feedback to the user to guide the user to conform the tool to the virtual boundary. In some embodiments, the haptic feedback occurs by damping the movement of the tool. In some embodiments, the control system is configured to switch the boundary handler from a boundary disabled state to a boundary enabled state when the tool is positioned to conform to the virtual boundary.

[0023] In some implementations, the user input is configured such that a first input state indicates that the user is actively using the tool and a second input state indicates that the user has released the tool.

[0024] In some embodiments, the user input is placed on the tool and configured such that when the user input is actuated the user input is in a first input state and when the user input is released the user input is in a second input state. In some embodiments, the tool has a grip and the user input includes a presence detector for detecting a user's hand on the grip.

[0025] In some embodiments, the control system includes a pendant and the user input is located on the pendant. In some embodiments, the user input is configured such that activation of the user input places the user input in a first input state and release of the user input places the user input in a second input state.

[0026] In some embodiments, the user input is further defined as a tool input on the tool. In some embodiments, the first and second input states are further defined as a tool input first state and a tool input second state. In some embodiments, the control system includes a pendant and a pendant input disposed on the pendant, the pendant input having a pendant input first state and a pendant input second state.

[0027] In some embodiments, the manipulator is operable in a manual mode in which the manipulator moves the tool in response to user forces and torques applied to the tool by the user when the tool input is in a first state of the tool input. In some embodiments, the manipulator is operable in a semi-autonomous mode in which the manipulator moves the tool along the tool path when the pendant input is in a first state of the pendant input. In some embodiments, the boundary handler is configured to determine whether the tool conforms to or penetrates the virtual boundary in response to a control system switch of the manipulator from one of the manual mode and the semi-autonomous mode to the other of the manual mode and the semi-autonomous mode. In some embodiments, the control system includes a path handler configured to generate a lead-in path from the current position of the tool to the tool path. In some embodiments, the lead-in path is generated when the manipulator switches from the manual mode to the semi-autonomous mode. In some embodiments, the boundary handler is configured to determine whether movement of the tool along the lead-in path maintains compliance with or penetrates the virtual boundary. In some embodiments, the boundary handler is configured to determine whether movement of the tool along the lead-in path maintains compliance with or penetrates the virtual boundary by modeling motion of multiple stereotactic interaction mechanisms associated with the tool that determine whether the stereotactic interaction mechanisms maintain compliance with or penetrate the virtual boundary. In some embodiments, the boundary handler is configured to model motion of the multiple stereotactic interaction mechanisms in three or more degrees of freedom. In some embodiments, the control system includes a guide handler configured to generate user feedback to the user in response to the boundary handler determining that the tool will penetrate the virtual boundary if the tool is moved from its current position along the lead-in path to the tool path.

[0028] In some embodiments, the control system is configured to disable boundary-compliant autonomous movement of the tool in response to one or more of the following: the tool coming to a stop; a predetermined period of time elapses following a user input transition from the first input state to the second input state; the linear velocity of the tool falling below one or more thresholds; or the angular velocity of the tool falling below one or more thresholds.

[0029] In some embodiments, the control system is configured to determine whether the tool remains compliant with the virtual boundary based on a tolerance defined for the virtual boundary.

[0030] In some embodiments, the control system is configured to generate a resilient tool path in a resilient mode to move the tool to conform to the virtual boundary.

[0031] In some embodiments, the control system is configured to move the virtual boundary in a recovery mode from a starting position, returning the tool to conform to the virtual boundary, and then allowing autonomous boundary-compliant movement of the tool while returning the virtual boundary to the starting position.

[0032] In some embodiments, the boundary handler is configured to activate the second virtual boundary in response to a user selecting the second virtual boundary and deactivate the first virtual boundary if the boundary handler determines that the tool conforms to the second virtual boundary, such that the control system transitions to controlling the manipulator movement and the tool movement based on the relationship between the tool and the second virtual boundary. In some embodiments, in response to a user selecting the second virtual boundary, the boundary handler is configured to maintain the first virtual boundary as active if the boundary handler determines that the tool has encroached on the second virtual boundary, such that the control system continues to control the manipulator movement and the tool movement based on the relationship between the tool and the first virtual boundary. In some embodiments, the control system is configured to generate user feedback to the user in response to the boundary handler determining that the tool has encroached on the second virtual boundary.

[0033] In some embodiments, the control system includes a user input having a first input state and a second input state, the user input configured to be disposed on the tool and actuated by the user to place the user input in the first input state and released by the user to place the user input in the second input state. In some embodiments, the control system is configured to allow the user to select a second virtual boundary with a virtual boundary selector while the user input is in the first input state. In some embodiments, the virtual boundary selector includes a second user input disposed on the tool. In some embodiments, the tool includes a tool drive, and the control system is configured to continue operation of the tool drive when the user selects the second virtual boundary with the virtual boundary selector. In some embodiments, the control system is configured to limit relative movement between the tool and the first virtual boundary when the user input is in the first input state and the first virtual boundary is active. In some embodiments, the control system is configured to do so by generating a first boundary constraint in the boundary handler and to constrain relative movement between the tool and the second virtual boundary when the user input is in the first input state and the second virtual boundary is active by generating a second boundary constraint in the boundary handler. In some embodiments, the control system includes a constraint solver that calculates constraint forces adapted to maintain the tool in compliance with the first virtual boundary based on the first boundary constraint or in compliance with the second virtual boundary based on the second boundary constraint. In some embodiments, the control system includes a virtual simulator that simulates tool dynamics in the virtual simulation based on the constraint forces and outputs a commanded pose. In some embodiments, the control system is configured to command the manipulator to move the tool based on the commanded pose. In some embodiments, the virtual boundary selector is configured to allow a user to toggle between the first and second virtual boundaries, to toggle sequentially among multiple virtual boundaries, or to select from a list of virtual boundaries.

[0034] In some embodiments, the first and second parameters are each further defined as a stiffness parameter, and the first stereotactic interaction feature is stiffer than the second stereotactic interaction feature. In some embodiments, the at least one controller is configured to identify an event and, in response to the identification of the event, modify at least one of the first parameter or the second parameter. In some embodiments, the first and second parameters are each further defined as a geometric parameter defined as at least one of a size, an area, a volume, or a shape of the stereotactic interaction feature, and the first geometric parameter is different from the second geometric parameter. In some embodiments, one or more of the first and second stereotactic interaction features are associated with one or more second objects by being positioned a distance away from the one or more second objects. In some embodiments, the at least one controller is configured to determine one or more of the first and second parameters by receiving user input defining one or more of the first and second parameters or by being configured to automatically determine one or more of the first and second parameters based on surgical information. In some embodiments, to generate the response based on an interaction between at least one of the first and second stereotactic interaction features and the virtual boundary, the at least one controller is configured to identify a collision or anticipated collision between one or more of the first and second stereotactic interaction features and the virtual boundary and generate the response further defined as the at least one controller configured to perform one or more of the following: adjusting a pose of the one or more second objects; and generating an alert or notification regarding the collision or anticipated collision. In some embodiments, the first or more second objects are further defined as a single second object, and the first stereotactic interaction feature and the second stereotactic interaction feature are associated with the single second object.In some embodiments, the one or more second objects are further defined as separate second objects, the first stereotactic interaction feature is associated with one of the second objects and the second stereotactic interaction feature is associated with another one of the second objects. In some embodiments, the one or more first objects are further defined as bones, the virtual boundary is further defined as a cutting boundary that delineates an anatomical structure to be removed from an anatomical structure that should not be removed, the one or more second objects are further defined as a rotary cutting tool, the first and second stereotactic interaction features are associated with the rotary cutting tool and are positioned at different positions relative to the rotary cutting tool, and the first and second parameters are each further defined as stiffness parameters, and the first stereotactic interaction feature is stiffer than the second stereotactic interaction feature. In some embodiments, the one or more first objects include one or more of the following: a patient's anatomy, a base, a link, a joint, an end effector, a tool, a sterile adapter, any portion of a kinematic chain forming a robotic manipulator including an energy applicator, a handheld tool or device, an operating table, a head-mounted device, a handheld display device or tablet, a surgical tracker, a retractor, an imaging device, a person in the operating room; and the one or more second objects include one or more of the following: a patient's anatomy, a base, a link, a joint, an end effector, a tool, a sterile adapter, any portion of a kinematic chain forming a robotic manipulator including an energy applicator, a handheld tool or device, an operating table, a head-mounted device, a handheld display device or tablet, a surgical tracker, a retractor, an imaging device, a person in the operating room.

[0035] The advantages of the present disclosure will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0036] [Figure 1]FIG. 1 is a perspective view of a surgical system. [Figure 2] FIG. 1 is a block diagram of a control system for controlling a surgical system. [Figure 3] FIG. 2 is a functional block diagram of a software program. [Figure 4] 13 shows the output of a boundary generator for an acetabular surgical procedure. [Figure 5] 13 shows the output of a path generator for an acetabular surgical procedure. [Figure 6] 10 shows the output of a boundary generator for a surgical procedure on a vertebral body. [Figure 7] 13 shows the output of a boundary generator for a femoral surgical procedure. [Figure 8] 13 shows the output of a path generator for a femoral surgical procedure. [Figure 9] 13 shows the output of a boundary generator for a femoral surgical procedure. [Figure 10] FIG. 1 is a diagram of virtual constraints. [Figure 11] FIG. 2 is a block diagram of modules operable by the control system. [Figure 12] A sample constraint equation is shown below. [Figure 13] 1 shows a sample forward dynamics algorithm for performing a virtual simulation. [Figure 14] 1 shows a sample forward dynamics algorithm for performing a virtual simulation. [Figure 15] 1 illustrates an example set of steps performed by a control system to switch between boundary-enabled and boundary-disabled modes. [Figure 16] 1 shows an example set of steps performed by a control system to solve constraints, perform forward dynamics, and determine a commanded pose. [Figure 16A]1 shows an example set of steps performed by a control system to solve constraints, perform forward dynamics, and determine a commanded pose. [Figure 17A] 1 shows how the control system takes collisions into account when determining the commanded pose. [Figure 17B] 1 shows how the control system takes collisions into account when determining the commanded pose. [Figure 17C] 1 shows how the control system takes collisions into account when determining the commanded pose. [Figure 17D] 1 shows how the control system takes collisions into account when determining the commanded pose. [Figure 18A] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18B] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18C] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18D] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18E] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18F] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 18G] 1 shows the movement of tools and virtual boundaries during a surgical procedure on the femur. [Figure 19] 1 illustrates a transition between a first virtual boundary and a second virtual boundary. [Figure 20] 1 illustrates multiple stereotactic interaction mechanisms associated with components of a robotic surgical system. DETAILED DESCRIPTION OF THE INVENTION

[0037] I. Overview Referring to FIG. 1 , a surgical system 10 is shown. The system 10 is useful for treating a target site or anatomical volume A of a patient 12, such as treating bone or soft tissue. In FIG. 1 , the patient 12 is undergoing a surgical procedure. The anatomical structures in FIG. 1 include the femur F, pelvis PEL, and tibia T of the patient 12. The surgical procedure may include tissue removal or other forms of treatment. The treatment may include tissue cutting, coagulation, lesion, or other in situ tissue treatment. In some examples, the surgical procedure includes a partial knee replacement or total knee replacement, a partial hip replacement or total hip replacement, shoulder surgery, spinal surgery, or ankle surgery. In some examples, the system 10 is designed to remove material to be replaced by a surgical implant, such as a unicompartmental, bicompartmental, multicompartmental, or total knee implant, an acetabular cup implant, a femoral stem implant, or a hip and knee implant including screws, anchors, or other fasteners. Some of these types of implants are shown in U.S. Patent Application Publication No. 2012 / 0330429, entitled "Prosthetic Implant and Method of Implantation," the disclosure of which is incorporated herein by reference. The system 10 and techniques disclosed herein may be used to perform other procedures, surgical or non-surgical, or may be used in industrial or other applications.

[0038] System 10 includes a robotic manipulator 14, also referred to as a surgical robot. Manipulator 14 has a base 16 and a plurality of links 18. A manipulator cart 17 supports manipulator 14 such that manipulator 14 is secured to the manipulator cart 17. Links 18 collectively form one or more arms (e.g., robotic arms) of manipulator 14. Manipulator 14 can have a serial arm configuration (as shown in FIG. 1), a parallel arm configuration, or any other suitable manipulator configuration. In other examples, multiple manipulators 14 can be utilized in a multiple arm configuration.

[0039] In the example shown in FIG. 1, the manipulator 14 includes multiple joints J and multiple joint encoders 19 disposed at the joints J to determine position data for the joints J. For simplicity, only one joint encoder 19 is shown in FIG. 1, although other joint encoders 19 may be similarly shown. The manipulator 14 in one example includes six joints J1-J6 that implement at least six degrees of freedom (DOF) for the manipulator 14. However, the manipulator 14 may have any number of degrees of freedom, any suitable number of joints J, and may include redundant joints.

[0040] The manipulator 14 does not require joint encoders 19, but may alternatively or additionally utilize motor encoders present on the motors of each joint J. Also, the manipulator 14 does not require revolute joints, but may alternatively or additionally utilize one or more prismatic joints. Any suitable combination of joint types is contemplated.

[0041] The base 16 of the manipulator 14 is generally a portion of the manipulator 14 that provides a fixed reference frame for the manipulator 14, or for other components of the system 10 in general. Generally, the origin of the manipulator coordinate system MNPL is defined at a fixed reference in the base 16. The base 16 may be defined relative to any suitable portion of the manipulator 14, such as one or more links 18. Alternatively, or additionally, the base 16 may be defined relative to the manipulator cart 17, such as when the manipulator 14 is physically attached to the cart 17. In one example, the base 16 is defined at the intersection of the axes of joints J1 and J2. Thus, although joints J1 and J2 are actually movable components, the intersection of the axes of joints J1 and J2 is nevertheless a virtual fixed reference pose that provides both a fixed position and orientation reference and does not move relative to the manipulator 14 and / or the manipulator cart 17.

[0042] In some examples, the manipulator 14 can be a handheld manipulator in which the base 16 is the base portion of the tool (e.g., the portion held by the user in the freehand) and the tool tip is movable relative to the base portion. The base portion has a tracked coordinate system of reference, and the tool tip has a tool tip coordinate system that is calculated relative to the coordinate system of reference (e.g., via motor and / or joint encoders and forward kinematics calculations). Because the pose relative to a path can be determined, the movement of the tool tip along the path can be controlled. Such a manipulator 14 is shown in U.S. Patent No. 9,707,043, filed August 31, 2012, entitled "Surgical Instrument Including Housing, A Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing," which is incorporated herein by reference.

[0043] The manipulator 14 and / or manipulator cart 17 house a manipulator controller 26 or other type of control unit. The manipulator controller 26 may comprise one or more computers or any other suitable form of controller for directing the motion of the manipulator 14. The manipulator controller 26 may have a central processing unit (CPU) and / or other processor, memory (not shown), and storage (not shown). The manipulator controller 26 is loaded with software, as described below. The processor may include one or more processors for controlling the operation of the manipulator 14. These processors may be any type of processor, multiprocessor, and / or multicore processing system. The manipulator controller 26 may additionally or alternatively include one or more microcontrollers, field programmable gate arrays, systems-on-chips, discrete circuits, and / or other suitable hardware, software, or firmware capable of performing the functions described herein. The term processor is not intended to limit any embodiment to a single processor. The manipulator 14 may also include a user interface UI with one or more displays and / or input devices (e.g., push buttons, sensors, switches, keyboards, mice, microphones (voice activation), gesture control devices, touch screens, joysticks, foot pedals, etc.).

[0044] The surgical tool 20 couples to the manipulator 14 and is movable relative to the base 16 to interact with the anatomical structure in a particular mode. In certain embodiments, the tool 20 is or forms part of an end effector 22 supported by the manipulator 14. The tool 20 can be grasped by a user. One possible arrangement of the manipulator 14 and tool 20 is described in U.S. Patent No. 9,119,655, filed August 2, 2013, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference. The manipulator 14 and tool 20 can be arranged in alternative configurations. The tool 20 can be similar to that shown in U.S. Patent Application Publication No. 2014 / 0276949, filed March 15, 2014, entitled "End Effector of a Surgical Robotic Manipulator," which is incorporated herein by reference.

[0045] The tool 20 includes an energy applicator 24 designed to contact tissue of the patient 12 at a target site. In one example, the energy applicator 24 is a burr 25. The burr 25 may be substantially spherical and may include a spherical center, a radius (r), and a diameter. Alternatively, the energy applicator 24 may be a drill bit, a saw blade 27 (see alternative tool in FIG. 1 ), an ultrasonic vibration tip, or the like. The tool 20 and / or the energy applicator 24 may include any geometric characteristic, such as a perimeter, a circumference, a radius, a diameter, a width, a length, a volume, an area, a surface / plane, an extent of a motion envelope (along any one or more axes), or the like. The geometric characteristic may be considered to determine how to position the tool 20 relative to the tissue at the target site to perform the desired treatment. In some of the embodiments described herein, a spherical burr having a tool center point (TCP) and a sagittal saw blade having a TCP are described for convenience and ease of explanation, but are not intended to limit the tool 20 to any particular configuration.

[0046] The tool 20 may include a tool controller 21 for controlling the operation of the tool 20, such as controlling power to the tool 20 (e.g., to a tool drive such as the tool's rotary motor), controlling the movement of the tool 20, controlling irrigation / aspiration of the tool 20, and / or the like. The tool controller 21 may be in communication with a manipulator controller 26 or other components. The tool 20 may also include a user interface UI with one or more displays and / or input devices (e.g., push buttons, triggers, sensors, switches, keyboards, mice, microphones (voice activation), gesture control devices, touch screens, joysticks, foot pedals, etc.) coupled to the tool controller 21, the manipulator controller 26, and / or other controllers described herein. The manipulator controller 26 controls the state (e.g., position and / or orientation) of the tool 20 (e.g., of the TCP) with respect to a coordinate system such as the manipulator coordinate system MNPL. The manipulator controller 26 may control the velocity (linear or angular), acceleration, or other derivatives of the motion of the tool 20.

[0047] In one example, a tool center point (TCP) is a predetermined reference point defined on the energy applicator 24. The TCP has a pose that is known or calculable (i.e., not necessarily static) relative to another coordinate system. The shape of the energy applicator 24 is known in or defined relative to the TCP coordinate system. The TCP can be located at the spherical center of the burr 25 of the tool 20 or at the distal end of the saw blade 27 so that only a single point is tracked. The TCP can be defined in various ways depending on the configuration of the energy applicator 24. The manipulator 14 can use joint / motor encoders or any other non-encoder position sensing method to determine the pose of the TCP. The manipulator 14 can determine the TCP pose using joint measurements and / or can use techniques to directly measure the TCP pose. Control of the tool 20 is not limited to a center point. For example, any suitable primitive, mesh, etc. can be used to represent the tool 20.

[0048] The system 10 further includes a navigation system 32. An example of a navigation system 32 is described in U.S. Patent No. 9,008,757, filed September 24, 2013, entitled "Navigation System Including Optical and Non-Optical Sensors," which is incorporated herein by reference. The navigation system 32 tracks the movement of various objects. Such objects include, for example, the manipulator 14, the tool 20, and anatomical structures, such as the femur F, pelvis PEL, and tibia T. The navigation system 32 tracks these objects to collect information about the state of each object with respect to a (navigation) localizer coordinate system LCLZ. Coordinates in the localizer coordinate system LCLZ can be transformed to the manipulator coordinate system MNPL, other coordinate systems, and / or vice versa, using transformations.

[0049] The navigation system 32 includes a cart assembly 34 that houses a navigation controller 36 and / or other types of control units. A navigation user interface UI is in operative communication with the navigation controller 36. The navigation user interface includes one or more displays 38. The navigation system 32 can use the one or more displays 38 to display a graphical representation of the relative status of tracked objects to a user. The navigation user interface UI further includes one or more input devices for inputting information into the navigation controller 36 or otherwise selecting / controlling certain aspects of the navigation controller 36. Such input devices include an interactive touchscreen display. However, the input devices may also include any one or more of push buttons, a keyboard, a mouse, a microphone (voice activation), a gesture control device, a foot pedal, etc.

[0050] The navigation system 32 also includes a navigation localizer 44 coupled to the navigation controller 36. In one example, the localizer 44 is an optical localizer and includes a camera unit 46. The camera unit 46 includes an outer housing 48 that houses one or more optical sensors 50. The localizer 44 may include its own localizer controller 49 and may further include a video camera VC.

[0051] The navigation system 32 includes one or more trackers. In one example, the trackers include a pointer tracker PT, one or more manipulator trackers 52A, 52B, a first patient tracker 54, a second patient tracker 55, and a third patient tracker 56. In the illustrated example of FIG. 1 , the manipulator tracker is rigidly attached to the tool 20 (i.e., tracker 52A), the first patient tracker 54 is rigidly attached to the femur F of the patient 12, the second patient tracker 55 is rigidly attached to the pelvis PEL of the patient 12, and the third patient tracker 56 is rigidly attached to the tibia T of the patient 12. In this example, the patient trackers 54, 55, and 56 are rigidly attached to sections of bone. The pointer tracker PT is rigidly attached to a pointer P, which is used to register anatomical structures to the localizer coordinate system LCLZ. Manipulator trackers 52A, 52B can be attached to any suitable component of manipulator 14 in addition to or other than tool 20, such as base 16 (i.e., tracker 52B) or to any one or more links 18 of manipulator 14. Trackers 52A, 52B, 54, 55, 56, PT can be secured to their respective components in any suitable manner. For example, trackers can be rigidly fixed, flexibly connected (fiber optics), or not physically connected at all (ultrasound), so long as there is a suitable (complementary) method (measurement) for determining the relationship between each tracker and the object with which it is associated.

[0052] Any one or more of the trackers may include an active marker 58. The active marker 58 may include a light-emitting diode (LED). Alternatively, the trackers 52A, 52B, 54, 55, 56, PT may have a passive marker, such as a reflector, that reflects light emitted from the camera unit 46. Other suitable markers not specifically described herein may be utilized.

[0053] The localizer 44 tracks the trackers 52A, 52B, 54, 55, 56, and PT to determine a state of each of the trackers 52A, 52B, 54, 55, 56, and PT, each of which corresponds to a state of the object to which it is attached. The localizer 44 may perform known triangulation techniques to determine the state of the trackers 52, 54, 55, 56, and PT and the associated object. The localizer 44 provides the states of the trackers 52A, 52B, 54, 55, 56, and PT to the navigation controller 36. In one example, the navigation controller 36 determines and communicates the states of the trackers 52A, 52B, 54, 55, 56, and PT to the manipulator controller 26. As used herein, object state includes, but is not limited to, data defining the position and / or orientation of the tracked object, or equivalents / derivatives of the position and / or orientation. For example, the state may be the pose of the object, and may include linear velocity data, angular velocity data, and / or the like.

[0054] The navigation controller 36 may comprise one or more computers or any other suitable form of controller. The navigation controller 36 has a central processing unit (CPU) and / or other processors, memory (not shown), and storage (not shown). These processors may be any type of processor, microprocessor, or multiprocessor system. The navigation controller 36 is loaded with software. The software converts signals received from, for example, the localizer 44 into data representing the position and orientation of the object being tracked. The navigation controller 36 may additionally or alternatively include one or more microcontrollers, field programmable gate arrays, systems-on-chips, discrete circuits, and / or other suitable hardware, software, or firmware capable of performing the functions described herein. The term processor is not intended to limit any embodiment to a single processor.

[0055] While an example of a navigation system 32 using triangulation techniques to determine the state of an object is shown, the navigation system 32 may have any other suitable configuration for tracking the manipulator 14, the tool 20, and / or the patient 12. In another example, the navigation system 32 and / or the localizer 44 are ultrasound-based. For example, the navigation system 32 may include an ultrasound imaging device coupled to the navigation controller 36. The ultrasound imaging device images any of the aforementioned objects, e.g., the manipulator 14, the tool 20, and / or the patient 12, and generates a state signal to the navigation controller 36 based on the ultrasound images. The ultrasound images may be 2D, 3D, or a combination of both. The navigation controller 36 may process the images in near real time to determine the state of the object. The ultrasound imaging device may have any suitable configuration and may be different from the camera unit 46 as shown in FIG. 1.

[0056] In another example, the navigation system 32 and / or the localizer 44 are radio frequency (RF) based. For example, the navigation system 32 may include an RF transceiver coupled to the navigation controller 36. The manipulator 14, the tool 20, and / or the patient 12 may include an attached RF emitter or transponder. The RF emitter or transponder may be passively or actively powered. The RF transceiver transmits an RF tracking signal and generates a status signal to the navigation controller 36 based on the RF signal received from the RF emitter. The navigation controller 36 can analyze the received RF signal and associate a relative status therewith. The RF signal may be of any suitable frequency. The RF transceiver may be positioned in any suitable location to effectively use the RF signal to track the object. Furthermore, the RF emitter or transponder may have any suitable structural configuration, which may differ significantly from the trackers 52A, 52B, 54, 55, 56, PT shown in FIG. 1 .

[0057] In yet another example, the navigation system 32 and / or the localizer 44 are electromagnetic-based. For example, the navigation system 32 may include an EM transceiver coupled to the navigation controller 36. The manipulator 14, the tool 20, and / or the patient 12 may include attached EM components, such as any suitable magnetic, electromagnetic, or inductive trackers. The trackers may be passively or actively energized. The EM transceiver generates an EM field and generates a status signal to the navigation controller 36 based on the EM signals received from the trackers. The navigation controller 36 may analyze the received EM signals and associate a relative status therewith. Again, such an example navigation system 32 may have a structural configuration different from that of the navigation system 32 shown in FIG. 1 .

[0058] The navigation system 32 may have any other suitable components or structures not specifically described herein. Additionally, any of the techniques, methods, and / or components described above with respect to the illustrated navigation system 32 may be implemented or provided for any of the other examples of the navigation system 32 described herein. For example, the navigation system 32 may utilize inertial tracking alone or any combination of tracking technologies, and may additionally or alternatively include fiber optic-based tracking, machine vision tracking, etc.

[0059] 2, system 10 includes a control system 60 including, among other components, a manipulator controller 26, a navigation controller 36, and a tool controller 21. Control system 60 further includes one or more software programs and software modules shown in FIG. 3. The software modules may be part of one or more programs operating in manipulator controller 26, navigation controller 36, tool controller 21, or any combination thereof, to process data to assist in controlling system 10. The software programs and / or modules are stored in non-transitory memory 64 of manipulator controller 26, navigation controller 36, tool controller 21, or a combination thereof, and include computer-readable instructions executed by one or more processors 70 of controllers 21, 26, 36. Memory 64 may be any suitable memory configuration, such as RAM, non-volatile memory, or may be implemented locally or from a remote database. Additionally, software modules for displaying prompts and / or communicating with the user may form part of one or more programs and may include instructions stored in memory 64 in the manipulator controller 26, the navigation controller 36, the tool controller 21, or any combination thereof. The user may interact with any of the input devices of the navigation user interface UI or other user interface UI to communicate with the software modules. The user interface software may execute on a device separate from the manipulator controller 26, the navigation controller 36, and / or the tool controller 21.

[0060] The control system 60 may include any suitable configuration of input, output, and processing devices suitable for performing the functions and methods described herein. The control system 60 may include the manipulator controller 26, the navigation controller 36, or the tool controller 21, or any combination thereof, or may include only one of these controllers. These controllers may communicate via a wired bus or communication network, as shown in FIG. 2, via wireless communication, or otherwise. The control system 60 may also be referred to as a controller. The control system 60 may include one or more microcontrollers, field programmable gate arrays, systems-on-chips, discrete circuits, sensors, displays, user interfaces, indicators, and / or other suitable hardware, software, or firmware capable of performing the functions described herein.

[0061] II. Virtual Boundary and Tool Path Referring to FIG. 3 , the software used by the control system 60 includes a boundary generator 66. As shown in FIG. 4 , the boundary generator 66 is a software program or module that generates a virtual boundary 71 for restricting the movement and / or operation of the tool 20. The virtual boundary 71 can be one-dimensional, two-dimensional, or three-dimensional and can include a point, a line, an axis, a trajectory, a plane, a volume, a triangular mesh, or the like. The virtual boundary 71 can have a simple shape or a complex geometric shape. In some embodiments, the virtual boundary 71 is a surface defined by a triangular mesh. The virtual boundary 71 may also be referred to as a virtual object. The virtual boundary 71 may be, for example, a keep-in boundary, if it is desired that the tool 20 be kept within the volume defined by the boundary, or a keep-out boundary, if it is desired that the tool 20 remain away from the volume defined by the boundary. The virtual boundary 71 may also be a keep-on boundary, if it is desired that the tool 20 be kept at a bounding point, line, plane, surface, or the like, or a keep-off boundary, if it is desired that the tool 20 remain away from a bounding point, line, plane, surface, or the like. The virtual boundary 71 may also be a combination of these types of boundaries. Other types of boundaries are also contemplated.

[0062] The virtual boundary 71 can be defined with respect to an anatomical model AM, such as a 3D bone model. The anatomical model AM is associated with the actual patient's anatomy by being mapped to the patient's anatomy via registration or other processes. In the example of FIG. 4 , the virtual boundary 71 includes a generally spherical mesh that substantially surrounds the acetabulum with an inlet portion 71 a (opening) that provides access to the acetabulum. The inlet portion is funnel-shaped or cone-shaped. This virtual boundary 71 is associated with the 3D model of the acetabulum.

[0063] The anatomical model AM and the associated virtual boundary 71 are registered to one or more patient trackers 54, 55, 56. Thus, the anatomical model AM (and the associated actual patient anatomy) and the virtual boundary 71 fixed to the anatomical model AM can be tracked by the patient trackers 54, 55, 56. The virtual boundary 71 can be implant-specific, e.g., defined based on the size, shape, volume, etc. of the implant, and / or patient-specific, e.g., defined based on the patient anatomy. The virtual boundary 71 can be a boundary created pre-operatively, intra-operatively, or a combination thereof. In other words, the virtual boundary 71 can be defined before the surgical procedure begins, during the surgical procedure (including during tissue removal), or a combination thereof. In either case, the control system 60 obtains the virtual boundary 71 by storing / retrieving the virtual boundary 71 in / from memory, retrieving the virtual boundary 71 from memory, creating the virtual boundary 71 pre-operatively, creating the virtual boundary 71 intra-operatively, etc.

[0064] The manipulator controller 26 and / or the navigation controller 36 track the state of the tool 20 relative to one or more virtual boundaries 71. In one example, the state of the TCP is measured relative to the virtual boundary 71 for purposes of determining forces to be applied to a virtual rigid body model of the tool 20 via a virtual simulation so that the tool 20 remains compliant with the virtual boundary. The tool 20 remains compliant with the virtual boundary 71 by maintaining a desired relationship to the virtual boundary 71, such as not being moved beyond the virtual boundary 71, and / or by maintaining a desired position and / or orientation relative to the virtual boundary 71. It should be appreciated that a predefined, configurable tolerance may also be established for any virtual boundary 71 so that some penetration of the virtual boundary 71 by the tool 20 or deviation from complete compliance with the virtual boundary 71 is not considered to be a violation of the virtual boundary 71. For example, a small tolerance may be set at 0.1 millimeters (mm) so that the tool 20 remains compliant with the virtual boundary 71 as long as the tool 20 penetrates the virtual boundary 71 by less than 0.1 mm (i.e., this is not considered a violation of the virtual boundary 71). In some cases, a larger tolerance may be appropriate. The results of the virtual simulation are instructed to the manipulator 14. The control system 60 controls / positions the manipulator 14 in a manner that emulates how a physical handpiece would respond in the presence of a physical boundary / barrier. The boundary generator 66 may be implemented on the manipulator controller 26. Alternatively, the boundary generator 66 may be implemented in another component, such as the navigation controller 36.

[0065] 3 and 5 , the path generator 68 is another software program or module executed by the control system 60. In one example, the path generator 68 is executed by the manipulator controller 26. The path generator 68 generates a tool path TP for the tool 20 to traverse, such as to remove a section of the anatomy to accept an implant. The tool path TP may include multiple path segments PS or may include a single path segment PS. The path segment PS may be a straight segment, a curved segment, a combination thereof, or the like. The tool path TP may also be defined with respect to the anatomical model AM and tracked via one or more of the patient trackers 54, 55, 56. The tool path TP may be implant-specific, e.g., based on the size, shape, volume, etc. of the implant, and / or patient-specific, e.g., defined based on the patient's anatomy. The tool path TP may be a 3D path along which the TCP of the tool 20 is intended to travel during a particular operation of the system 10.

[0066] In one version described herein, the tool path TP is defined as a tissue removal path, although in other versions, the tool path TP may be used for treatments other than tissue removal. One example of a tissue removal path described herein includes a comminution path 72. It should be understood that the term “comminution path” generally refers to a path of the tool 20 near a target site for comminuting anatomical structures and is not intended to require the tool 20 to operatively comminut the anatomical structures for the entire duration of the path. For example, the comminution path 72 may include sections or segments in which the tool 20 transitions from one location to another without comminution. Additionally, other forms of tissue removal along the comminution path 72, such as tissue resection, may be used. The comminution path 72 may be a predefined path created preoperatively, intraoperatively, or a combination thereof. In other words, the comminution path 72 may be defined before the surgical procedure begins, during the surgical procedure (including while removing tissue), or a combination thereof. In either case, the control system 60 obtains the milling path 72 by storing / retrieving the milling path 72 in / from memory, retrieving the milling path 72 from memory, creating a milling path 72 pre-operatively, creating a milling path 72 intra-operatively, etc. The milling path 72 can have any suitable shape or combination of shapes, such as circular, spiral / bottle opener, linear, curved, combinations thereof, etc. The milling path 72 shown in FIG. 5, when traversed by the tool 20, is intended to remove material from the acetabulum to create space for an acetabular cup implant to be installed in the acetabulum.

[0067] Examples of virtual boundaries 71 and / or comminution paths 72 are shown in Figures 4-9. The particular shapes and arrangements of the virtual boundaries 71 and / or comminution paths 72 shown are for illustrative purposes. Other shapes and arrangements are possible. As previously described, Figures 4 and 5 show virtual boundaries 71 and comminution paths 72 generated for use in a surgical procedure in which the acetabulum is prepared (e.g., comminuted) to receive an acetabular cup implant.

[0068] 6 shows a virtual boundary 71 that includes a generally spherical mesh that substantially surrounds a vertebral body with an entrance portion 71 a (opening) that provides access to the vertebral body. The entrance portion 71 a has a funnel or cone shape and extends to a cylindrical portion 71 b. This virtual boundary 71 is associated with a 3D model of the vertebral body. This virtual boundary 71 is generated for use in a surgical procedure in which the vertebral body is prepared (e.g., milled) to receive a screw or other implant.

[0069] FIG. 7 illustrates a virtual boundary 71 including a generally spherical mesh that substantially surrounds one end of the femur with an entry portion 71a (opening) that provides access to the femur. The entry portion 71a has a funnel or cone shape and extends down the medullary canal of the femur to a continuing canal portion 71b. This virtual boundary 71 is associated with a 3D model of the femur. FIG. 8 illustrates a milling path 72 that has been defined to allow the tool 20 to remove material from the femur to make way for a femoral stem implant. Thus, FIGS. 7 and 8 illustrate the virtual boundary 71 and milling path 72 that are generated for use in a surgical procedure in which the femur F is prepared (e.g., milled) to receive a femoral stem implant.

[0070] FIG. 9 illustrates a series of virtual boundaries 71 generated for five cutting planes through the distal end of a femur. Each of the virtual boundaries 71 in FIG. 9 includes a generally spherical mesh that substantially surrounds the distal end of the femur with an entry portion 71 a (opening) that provides access to the femur. The entry portion 71 a is followed by a cutting slot 71 b defined along one of five cutting planes 73 a–73 e. These virtual boundaries 71 are generated for use in a surgical procedure in which a femur F is prepared (e.g., via a planar resection) to receive a total knee replacement. Other types / shapes of virtual boundaries and / or comminution paths 72 are contemplated for use in other surgical procedures.

[0071] One example of a system and method for generating the virtual boundary 71 and / or the fracture path 72 is described in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference. In some examples, the virtual boundary 71 and / or the fracture path 72 may be generated offline, rather than by the manipulator controller 26 or the navigation controller 36. The virtual boundary 71 and / or the fracture path 72 may then be utilized by the manipulator controller 26 at runtime.

[0072] Referring back to FIG. 3 , two additional software programs or modules execute on the manipulator controller 26 and / or the navigation controller 36. One software module executes behavior control 74. Behavior control 74 is a process that calculates data indicative of the next commanded position and / or orientation (e.g., pose) of the tool 20. In some cases, behavior control 74 outputs only the position of the TCP, while in other cases it outputs the position and orientation of the tool 20. Outputs from one or more sensors, such as boundary generator 66, path generator 68, and force / torque sensor S, may be provided as inputs to behavior control 74 to determine the next commanded position and / or orientation of the tool 20. Behavior control 74 may process these inputs, along with one or more virtual constraints, described further below, to determine the commanded pose.

[0073] A second software module executes the motion control 76. One aspect of motion control is the control of the manipulator 14. The motion control 76 receives data from the behavior control 74 defining a next commanded pose. Based on these data, the motion control 76 determines (e.g., via inverse kinematics and a Jacobian calculator) next positions for the joint angles of the joints J of the manipulator 14 so that the manipulator 14 can position the tool 20 at the commanded pose as commanded by the behavior control 74. In other words, the motion control 76 processes the commanded pose, which may be defined in Cartesian space, into joint angles of the manipulator 14 so that the manipulator controller 26 can command the joint motors accordingly to move the joints J of the manipulator 14 to the commanded joint angles that correspond to the commanded pose of the tool 20. In one version, the motion control 76 adjusts the joint angle of each joint J and continuously adjusts the torque output by each joint motor to ensure as closely as possible that the joint motor drives its associated joint J to the commanded joint angle.

[0074] The boundary generator 66, the path generator 68, the behavior control 74, and the motion control 76 may be subsets of the software program 78. Alternatively, each may be a software program operating independently, separately and / or in any combination thereof. The term "software program" is used herein to describe computer-executable instructions configured to perform various functions of the described technical solutions. For simplicity, the term "software program" is intended to encompass at least any one or more of the boundary generator 66, the path generator 68, the behavior control 74, and / or the motion control 76. The software program 78 may be implemented in the manipulator controller 26, the navigation controller 36, or any combination thereof, or may be implemented in any suitable manner by the control system 60.

[0075] A clinical application 80 may be provided to handle user interactions. The clinical application 80 handles many aspects of user interactions and coordinates surgical workflow, including preoperative planning, implant placement, alignment, bone preparation visualization, postoperative assessment of implant fit, etc. The clinical application 80 is configured to output to the display 38. The clinical application 80 may run on its own separate processor or may run together with the navigation controller 36. In one example, the clinical application 80 interfaces with the boundary generator 66 and / or path generator 68 after implant placement is set by the user, and then sends the virtual boundary 71 and / or tool path TP, to which the boundary generator 66 and / or path generator 68 return, to the manipulator controller 26 for execution. The manipulator controller 26 executes the tool path TP as described herein. The manipulator controller 26 may further create specific segments (e.g., retraction segments) to smoothly return to the generated tool path TP when starting or resuming machining. Manipulator controller 26 may also process virtual boundary 71 to generate corresponding virtual constraints, as further described below.

[0076] III. Operation Mode The system 10 can operate in a manual mode, such as that described in U.S. Patent No. 9,119,655, which is incorporated herein by reference, where a user manually directs and the manipulator 14 executes movement of the tool 20 and its energy applicator 24 at the target site. The user physically contacts the tool 20 to move the tool 20 in manual mode. In one version, the manipulator 14 monitors the forces and torques applied by the user to the tool 20 to position it. For example, the manipulator 14 may include one or more sensors (e.g., force / torque sensors S) that detect and measure the forces and torques applied by the user to the tool 20 and generate corresponding inputs (e.g., one or more corresponding input / output signals) used by the control system 60. The forces and torques applied by the user are used to generate external forces F that are used to determine how to move the tool 20 in manual mode. ext At least partially determine the external force F ext may include other forces and torques other than those applied by the user, such as gravity compensation forces, backdriving forces, etc., as described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. Thus, the forces and torques applied by the user are calculated based on the external force F ext and possibly affecting the overall movement of the tool 20 in manual mode. ext can be completely determined.

[0077] The force / torque sensor S may comprise, for example, a 6-DOF force / torque transducer, as disclosed in U.S. Pat. No. 9,119,655, which is incorporated herein by reference. The force / torque sensor S may form part of the tool 20, the manipulator 14, or both. The force / torque sensor S may form part of the interface between the tool 20 and the manipulator 14, or may be located in any suitable location such that forces and torques applied by a user to the tool 20 are transmitted to the force / torque sensor S. The manipulator controller 26 and / or the navigation controller 36 receive inputs (e.g., signals) from the force / torque sensor S. In response to the forces and torques applied by the user, the manipulator 14 moves the tool 20 in a manner that emulates movements that would have occurred based on the forces and torques applied by the user.

[0078] Movement of tool 20 in manual mode may also be constrained (e.g., limited) with respect to one or more virtual boundaries 71 generated by boundary generator 66. In some versions, measurements taken by force / torque sensor S are transformed from the force / torque coordinate system FT of force / torque sensor S to another coordinate system, such as a virtual mass coordinate system VM, in which a virtual simulation is performed on a virtual rigid body model of tool 20, allowing forces and torques to be virtually applied to the virtual rigid body in the virtual simulation to ultimately determine how these forces and torques (among other inputs) affect the motion of the virtual rigid body, as described below.

[0079] The system 10 can also operate in a semi-autonomous mode in which the manipulator 14 autonomously moves the tool 20 along the milling path 72 (e.g., the active joints J of the manipulator 14 operate to move the tool 20 without requiring a force / torque on the tool 20 from the user). An example of operation in a semi-autonomous mode is also described in U.S. Pat. No. 9,119,655, which is incorporated herein by reference. In some embodiments, when the manipulator 14 operates in a semi-autonomous mode, the manipulator 14 can move the tool 20 without user assistance. Lack of user assistance may mean that the user does not physically contact the tool 20 to move it. Instead, the user can control starting and stopping of the movement using a remote control RC (see FIG. 1 ) in communication with the manipulator 14 (e.g., wired or wireless). The remote control RC can be in the form of a pendant that is held in the user's hand or otherwise grasped or supported by the user. The pendant can have any suitable size and / or shape to allow the user to hold and manipulate the pendant. In some versions, the pendant is a portable electronic device.

[0080] The user interface UI of the tool 20 and the remote control RC may each include one or more user input devices (e.g., push buttons, sensors, switches, keyboards, mice, microphones (voice-activated), gesture control devices, touchscreens, joysticks, foot pedals, etc.) coupled to the tool controller 21, the manipulator controller 26, and / or the navigation controller 36 to control the operation of the manipulator 14. For example, one of the user input devices on the user interface UI of the tool 20 may be a tool input 82 (e.g., a switch or other form of user input device) having first and second input states (see FIG. 1 ). The tool input 82 may be actuated (e.g., pressed and held) by the user to be placed in the first input state and released to be placed in the second input state. The tool 20 may have a grip 83 on which the tool input 82 is disposed. In some versions, the tool input 82 is a presence detector that detects the presence of a user's hand, such as a momentary contact switch that switches between on / off states, a capacitance sensor, an optical sensor, etc. Thus, tool input 82 is configured such that a first input state indicates that the user is actively engaging with tool 20 and a second input state indicates that the user has released tool 20 .

[0081] One of the input devices on the remote control RC can be a pendant input RC1 (e.g., a switch or other form of user input device) having a first and second input state. Similar to the tool input 82, the pendant input RC1 can be actuated (e.g., pressed and held) by a user to assume a first input state and released to assume a second input state. When the pendant input RC1 is actuated, secondary pendant inputs RC2, RC3 (e.g., switches or other forms of user input devices) on the remote control RC can then cause movement of the manipulator 14 by controlling the feedrate of the manipulator 14, e.g., the speed at which the manipulator 14 moves the tool 20. For example, the secondary pendant input RC2 can slow the feedrate, and the secondary pendant input RC3 can increase the feedrate in a semi-autonomous mode. Such a remote control RC embodied as a user pendant is disclosed in U.S. Patent No. 10,117,713 to Moctezuma de La Barrera et al., entitled "Robotic Systems and Methods for Controlling a Tool Removing Material from a Workpiece." This is incorporated herein by reference.

[0082] Tool input 82 and pendant input RC1 may be in the form of continuous actuators, i.e., inputs that must be continuously actuated to enable motion of tool 20 in manual or semi-autonomous mode, depending on which user input is actuated. For example, while the user continuously actuates tool input 82 and manual mode is enabled, manipulator 14 moves in response to the input forces and torques applied by the user, and control system 60 causes virtual boundary 71 to protect the patient's anatomy. When tool input 82 is released, input from force / torque sensor S may be disabled so that manipulator 14 is no longer responsive to forces and torques applied to tool 20 by the user.

[0083] Under normal operating conditions, when the tool input 82 is in a first input state (e.g., actuated), regardless of the state of the pendant input RC1 (the tool input 82 takes precedence), the manipulator 14 operates in manual mode and the control system 60 operates with the boundary enabled to maintain the tool 20 in compliance with the virtual boundary 71 (or boundaries) currently being used by the control system 60. Similarly, under normal operating conditions, when the tool input 82 is in a second input state (e.g., released) and the pendant input RC1 is in the first input state (e.g., actuated), the manipulator 14 operates in semi-autonomous mode and the control system 60 operates with the boundary enabled to maintain the tool 20 in compliance with the virtual boundary 71 and the tool path TP.

[0084] In the boundary-enabled state, the control system 60 controls the movement of the manipulator 14 to keep the tool 20 compliant with the virtual boundary 71. As a result, in the boundary-enabled state, the control system 60 can control the manipulator 14 to cause autonomous movement of the tool 20, and if the virtual boundary 71 moves relative to the tool 20 in a manner that would otherwise cause the tool 20 to encroach on the virtual boundary 71, the control system 60 can compensate for such movement of the virtual boundary 71 by moving the tool 20. Such autonomous movement can be referred to as autonomous, boundary-compliant movement. For example, if the manipulator 14 is operating in manual mode but the user stops causing any movement of the tool 20, e.g., the user is still actuating the tool input 82 but is not applying any user force or torque to the tool 20, and the patient's anatomy is moved such that the virtual boundary 71 (which is fixed relative to the patient's anatomy) moves slightly beyond the TCP of the tool 20, then the control system 60 responds by operating one or more of the joint motors in the manipulator 14 in a manner that results in compensatory movement of the tool 20 to keep the TCP of the tool 20 in compliance with the virtual boundary 71.

[0085] When the tool input 82 and the pendant input RC1 are both in the second input state (e.g., neither is actuated), the manipulator 14 operates in hold mode and with boundaries disabled. In hold mode, movement of the tool 20 is effectively disabled. In this case, the manipulator 14 can continue to be energized and operate to actively maintain the current position and / or orientation of the tool 20 relative to the manipulator coordinate system MNPL by monitoring the encoders 19 and actively driving the joint motors to resist external forces due to gravity or forces inadvertently applied to the manipulator 14 or tool 20 by the user. In some versions, a brake system can be activated to hold the tool 20 in its current position and / or orientation. In hold mode, a user may wish to adjust the patient's anatomy and target site position without unexpected movement of any of the tool 20, for example, such that movement of the tool 20 occurs only in response to input from the user. A user may wish to adjust the patient's anatomy for a variety of reasons, such as visualization, improved access to the target site, target site irrigation, soft tissue removal or cleaning, etc. In either case, if the patient's anatomy moved, any virtual boundaries 71 that were pose-fixed relative to the patient's anatomy also moved.

[0086] In the hold mode and boundary-invalid state, the control system 60 disables any autonomous, boundary-compliant movement of the tool 20. As a result, when the user has moved the patient's anatomy to improve visualization, access, or otherwise, and is ready to resume operation of the manipulator 14 in manual or semi-autonomous mode, the system 10 first checks whether the virtual boundary 71 has moved such that the tool 20 now encroaches on the virtual boundary 71 (e.g., outside the virtual boundary 71, inside the virtual boundary 71, deviating from the virtual boundary 71, etc.). Thus, when returning to manual or semi-autonomous mode by switching the tool input 82 or the pendant input RC1 to the first input state, the control system 60 performs a collision check to determine whether the TCP of the tool 20 is now encroaching on the virtual boundary 71.

[0087] If a collision is detected, the manual mode or semi-autonomous mode (depending on which input was actuated) remains disabled, and the control system 60 functions to provide guidance to the user as to the situation and how to move the tool 20 to bring the tool 20 back into compliance with the virtual boundary 71. Otherwise, if the manual mode or semi-autonomous mode were enabled, this could result in sudden, unexpected movement of the tool 20, particularly when the tool 20 is not fully compliant with the virtual boundary 71. The guidance to the user may be in the form of user feedback, such as visual feedback (e.g., on the display 38, a visual indicator LED on the tool 20), audible feedback (e.g., via a speaker on the manipulator 14, the tool 20), and / or haptic feedback (e.g., by haptically guiding the user to place the tool 20 in a desired relationship with the virtual boundary 71). The collision check may be repeated periodically or continuously, and if the tool 20 returns to compliance with the virtual boundary 71, the manual or semi-autonomous mode may be enabled, and the user feedback ceases. The control system 60 can automatically switch the manipulator 14 from a boundary-invalid state to a boundary-valid state when it detects that the tool 20 has returned to compliance with the virtual boundary 71. In some versions, automated guidance can be provided by the control system 60 to autonomously move the tool 20 to a position that conforms to the virtual boundary 71. In this case, a recovered tool path can be generated by the path generator 68 (or other module) and can be generated based on the current pose of the tool 20 (e.g., from the current pose to a known pose in compliance with the virtual boundary 71), or the recovered path can be pre-defined.

[0088] The system 10 can also operate in a guided haptic mode, as described in U.S. Provisional Patent Application Publication No. 62 / 908,056, filed September 30, 2019, entitled "Systems and Methods for Guiding Movement of a Tool," which is incorporated herein by reference. The guided haptic mode can be used, for example, to help haptically guide the user to place the tool 20 into a goal state that conforms to the virtual boundary 71 if the tool 20 enters the virtual boundary 71 while in hold mode and the boundary is invalid. The guided haptic mode utilizes aspects of the controls used in both the manual and semi-autonomous modes. For example, the forces and torques applied by the user are subsequently detected by the force / torque sensor S and applied to the virtual simulation as external forces F that at least partially influence the overall movement of the tool 20. ext Additionally, in the guided haptic mode, the system 10 applies an external force F to the virtual simulation. ext A virtual constraint force F is applied along with c This generates a virtual attractive (or repulsive) force and torque embodied in

[0089] IV. Constraints and Virtual Simulation Solutions 11 , one or more virtual constraints, such as path constraints, boundary constraints, guide constraints, and other constraints, may be used by control system 60 in various modes to control the movement of tool 20. Generally, virtual constraints are limits on the motion of a rigid body that are considered by control system 60, along with other motion-related information, to determine how to command manipulator 14 to move tool 20. These virtual constraints may affect the position of tool 20 and / or the orientation of tool 20. As described in more detail below, control system 60 uses constraint forces F to satisfy or attempt to satisfy the virtual constraints. c A constraint solver 84 is provided which operates to calculate the constraint force F c incorporates forces and torques that are determined to affect the movement of the tool 20.

[0090] Path constraints may be generated based on the tool path TP provided by the path generator 68. In effect, the path constraints cause virtual forces and / or torques to be calculated by the constraint solver 84 and used in the virtual simulation to pull the tool 20 along the tool path TP so that the TCP of the tool 20 follows along the tool path TP, while the tool 20 maintains a desired orientation. Thus, path constraints may also include orientation constraints, but may also adjust the orientation based on user-applied forces / torques. See, for example, the user reorientation method described in U.S. Pat. No. 9,119,655, incorporated herein by reference. Path constraints may be generated in certain modes of operation, such as semi-autonomous mode, but not in other modes, such as manual mode.

[0091] Boundary constraints may be defined to prohibit the tool 20 from penetrating one or more virtual boundaries 71. Each boundary constraint may be considered a unidirectional virtual constraint that operates to maintain the TCP of the tool 20 in compliance with one or more virtual boundaries 71. For example, a boundary constraint may cause virtual forces and / or torques to be calculated and used in the virtual simulation, such that the TCP of the tool 20 has zero velocity (or near-zero velocity) at the virtual boundary 71, preventing the TCP from penetrating (or penetrating too far) into the virtual boundary 71. Boundary constraints may be active in certain modes, such as manual mode or semi-autonomous mode. However, boundary constraints may be disabled in certain situations, such as when the control system 60 is operating with boundaries disabled, as described above. For example, upon detecting a collision between the tool 20 and the virtual boundary 71 after moving the anatomical structure in hold mode, the boundary constraints are no longer output to the constraint solver 84, thereby disabling the manual or semi-autonomous mode.

[0092] In some cases, even if boundary constraints are no longer output, the user can still move the tool 20 in free mode or other similar modes by applying forces and torques to the tool 20. In free mode, the tool 20 moves relatively freely in response to forces and torques applied to the tool 20 by the user, which allows the user to return the tool 20 to conform to the virtual boundary 71. External forces F, including user-applied forces and torques, ext is an input to the constraint solver 84, which is fed to the virtual simulator 86 and, when enabled, ext At least partially influences the overall movement of the tool 20. ext is typically enabled when the tool input 82 is in a first input state (e.g., actuated), for example, in manual mode. ext can also be enabled in free mode to allow the user to return the tool 20 to conform to the virtual boundary 71 in response to user applied forces and torques. ext is disabled when both user inputs (eg, tool input 82 and pendant input RC1) are in the second input state (eg, neither is activated).

[0093] The guide constraints are defined to generate virtual attractive (or repulsive) forces and torques used in the virtual simulation to guide the user to place the tool 20 into the goal state in accordance with the virtual boundary 71. The guide constraints are defined to ultimately influence the movement of the tool 20 into the goal state, such that the user is provided with one or more haptic interaction effects that guide the user to cause the desired movement of the tool 20. The guide constraints may be active in certain modes, such as the guided haptics mode, but may be inactive in other modes. The guide constraints may also provide the user with other forms of haptic feedback, such as a dampened feel for the movement of the tool 20, to indicate an error or abnormal condition, such as when the user moves the tool 20 further away from the goal state.

[0094] In some versions, the virtual constraints are velocity impulse constraints, where forces and / or torques are calculated to apply virtual impulses to objects in the virtual simulation to change the object's velocity according to the constraint parameters of interest. In some versions, the constraints are similar to those used in impulse modeling described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. In some versions, virtual constraints are used in all modes.

[0095] The virtual constraints employed by the control system 60 are defined primarily by three run-time parameters: a constraint Jacobian J that maps each virtual constraint to the coordinate system employed in the virtual simulation; p , the desired velocity V is the scalar velocity of the virtual constraint in the coordinate system des (or Vp2) (e.g., the desired velocity may be zero when the patient is stationary and the associated virtual constraints defined for the patient are not moving, but may be non-zero when the patient moves, since virtual constraints may be associated with the patient), and a constraint distance Δd, which indicates, for example, how close the TCP is to the constraint and whether it is penetrating the virtual constraint. Δd is also called the penetration depth, i.e., the error distance along the direction of the constraint.

[0096] The virtual constraints are not infinitely rigid; instead, each virtual constraint has an adjustment parameter for adjusting the stiffness of the virtual constraint, for example, by incorporating spring and damping parameters into the constraint. Such parameters may include a constraint force blend parameter (C) and an error reduction parameter (ε). The spring and damping parameters can be adjusted during operation. In some versions, the value of the adjustment parameter may be changed based on a specific relationship, such as the curvature of the tool path TP (for a path constraint), the relationship between the virtual boundary 71 and the TCP (for a boundary constraint), or the relationship between the current state and the goal state (for a guide constraint). The adjustment parameter may be different for different virtual constraints. For example, boundary constraints may be more stringent than other constraints. The virtual constraints may include a first virtual constraint having a first value of the adjustment parameter and a second virtual constraint having a second value of the adjustment parameter, the first value being greater than the second value and affecting the constraint force F. c The resulting virtual forces and / or torques embodied in the adjustment parameters may be adapted to cause stronger movement of the tool 20 as a result of the first virtual constraint compared to the second virtual constraint. The values ​​of the adjustment parameters may be stronger (e.g., stiffer) for the position constraint than for the orientation constraint, or vice versa.

[0097] The tuning parameter can be set to remain constant, to rise / fall exponentially with the constraint distance, to vary linearly with the constraint distance, to vary depending on the constraint direction, to take into account the effect of gravity, etc. The tuning parameter is the constraint force F c Or the constraint force F is calculated based on the virtual constraint, such as increasing or decreasing the stiffness depending on the size of the component. c The tuning parameters and their values, their correlation to particular relationships, and the manner in which they may be scaled may be stored in one or more look-up tables in any suitable memory in control system 60 for later retrieval.

[0098] Each virtual constraint also has constructive settings, which include information about tuning parameters such as the constraint force blend parameter (C) and error reduction parameter (ε), upper and / or lower force limits, and / or upper and lower constraint distance offsets. The upper and lower force limits are ultimately solved by the constraint solver 84 to produce the constraint force F, as further described below. c , which refers to the force limits calculated for each virtual constraint. Virtual constraints can be unidirectional (e.g., the forces calculated to satisfy the constraint are only positive or only negative) or bidirectional (e.g., the forces calculated to satisfy the constraint can be positive or negative). For unidirectional constraints, the upper force limit can be set high in the positive direction (e.g., +100,000 Newtons) and the lower force limit can be set to zero, although the force limits can be set to any desired limits. For bidirectional constraints, the upper and lower force limits can be set high in opposite directions (e.g., + / -100,000 Newtons). The upper and lower constraint distance offsets indicate when the constraint is active. Some constraints may always be active in certain modes. For boundary constraints, the upper constraint distance offset can be zero and the lower constraint distance offset can be a large negative value (e.g., -100,000 mm), so that effectively any boundary intrusion will be within the limits. As further described below, upper and lower limits for the constraint distance offset can be set so that the boundary constraint becomes active when the virtual simulation indicates that the proposed state of the TCP of the tool 20 encroaches on the virtual boundary 71.

[0099] Various virtual constraints can be provided to the constraint solver 84, including guide constraints, path constraints, boundary constraints, and other constraints. These constraints can be turned on and off by the control system 60. For example, in some cases, there may be no path constraints (e.g., manual mode), no boundary constraints (e.g., hold mode, guided manual mode, or free mode), and no other constraints may be generated. Similarly, no guide constraints may be generated unless the user needs to be guided to bring the tool 20 back into compliance with the virtual boundary 71. All virtual constraints used in the behavior control 74 may affect the movement of the tool 20.

[0100] The constraint solver 84 calculates constraint forces F that are virtually applied to the tool 20 in a virtual simulation performed by a virtual simulator 86 based on the virtual constraints sent to the constraint solver 84. c The constraint solver 84 finally calculates the constraint force F that satisfies or attempts to satisfy all the constraints. c The task is to solve the problem, and therefore other constraints are also satisfied by the constraint force F c For example, if boundary constraints are actively communicated to the constraint solver 84, the constraint solver 84 may affect the magnitude / direction of the constraint force F c to have force and / or torque components adapted to keep the tool 20 compliant with the virtual boundary 71 based on the boundary constraints.

[0101] Referring to the constraint equations shown in FIG. 12, the constraint solver 84 calculates the constraint data for each virtual constraint as p To solve, place it in the corresponding row of the matrix form of the constraint equation, where F p is the force vector in the chosen coordinate system, i.e., F p Each component of F is a constraint force scalar acting in the corresponding constraint direction. p To solve for F, the equation shown in Figure 12 is converted into a matrix equation where each row represents a single one-dimensional constraint, as explained below. The constraint data is cgext , damping force F damping , inertia force F inertial , virtual mass matrix M, virtual mass velocity V cg1 , and a time step Δt (e.g., t of 125 microseconds) frame The constraint solver 84 is populated with the constraint equations along with other information known to it, such as:

[0102] The virtual mass matrix M is a combination of the 3x3 mass matrix and the inertia matrix. The damping and inertia forces F damping and F inertial is calculated / known by the virtual simulator 86 and is the virtual mass velocity V output by the virtual simulator 86 at the previous time step.cg1 (e.g., velocity of the virtual mass coordinate system VM) Virtual mass velocity Vcg1 is a 6DOF velocity vector containing linear and angular velocity components. The damping force F damping is the virtual mass velocity V cg1 is a 6DOF force / torque vector calculated as a function of the damping coefficient matrix (linear and rotational coefficients may not be equal). Damping is applied to the virtual mass to improve its stability. Inertial force F inertial is the virtual mass velocity V cg1 and the 6DOF force / torque vector calculated as a function of the virtual mass matrix M. The damping and inertial forces F damping and F inertia is determined in the manner described in Bowling et al., US Pat. No. 9,566,122, which is incorporated herein by reference.

[0103] The constraint solver 84 may comprise any suitable algorithmic instructions (e.g., an iterative constraint solver, a projected Gauss-Seidel solver, etc.) that solves the system of equations for this constraint to find a solution that satisfies the system of equations (e.g., satisfies the various constraints). In some cases, not all constraints may be satisfied simultaneously. For example, if the motion is over-constrained by the various constraints, the constraint solver 84 will find an essentially "optimal" solution, taking into account the relative stiffness / damping of the various constraints. The constraint solver 84 solves the system of equations, ultimately finding the constraint forces F p Output.

[0104] When using the projected Gauss-Seidel solver, the constraint solver 84 creates the A and b matrices based on the constraints, solves the system of equations using the projected Gauss-Seidel solver, and calculates the resulting force vector F p Determine the Gauss-Seidel projected output and transform it from your chosen coordinate system (e.g., the constraint coordinate system) to the virtual mass coordinate system VM. For example, the equation F c =J p T F p When using F in the formula c is the constraint force, and the force vector Fp The components of are equivalent force / torque vectors F applied to the virtual mass frame VM. c is converted to

[0105] Methods for using the Projected Gauss-Seidel to solve systems of equations with multiple constraints are shown, for example, in "Constraint based physics solver" (v1.02) by Marijn Tamis and Giuseppe Maggiore, dated June 15, 2015, which can be found at http: / / www.mft-spirit.nl / files / MTamis_ConstraintBasedPhysicsSolver.pdf, or in "Comparison between Projected Gauss-Seidel and Sequential Impulse Solvers for Real-Time Physics Simulations" (v1.01) by Marijn Tamis, dated July 1, 2015, which can be found at http: / / www.mft-spirit.nl / files / MTamis_PGS_SI_Comparison.pdf, both of which are incorporated herein by reference in their entirety.

[0106] The projected Gauss-Seidel method addresses linear complementarity problems (LCPs). Some constraint types (e.g., one-sided constraints, such as boundary constraints) can only be pushed in one direction, resulting in inequalities related to LCPs. If the calculated force of such a constraint is outside the constraint solver 84's tolerance range for a given iteration, which is invalid, the given constraint must be pruned (or alternately limited / bounded by its upper or lower bounds), and the remaining constraints are solved until a suitable result (i.e., convergence) is found. In this way, the constraint solver 84 determines the active set of constraints for a given time step and then solves for those values. For other constraint types, forces can be applied in both positive and negative directions, e.g., bilateral constraints. Such constraints include guide constraints used to guide a user's tool movement toward a goal state. When enabled, such bilateral constraints are typically active and are not pruned / bounded during the constraint solver 84 iterations.

[0107] Constraint force F calculated by Constraint Solver 84 c The external force F includes three components of force along the x-, y-, and z-axes and three components of torque about the x-, y-, and z-axes. cgext , damping force F damping , and inertial force F inertial (all of which can contain six components of force / torque), along with the constraint force F c In some cases, these components of the force / torque are first transformed into a common coordinate system (e.g., a virtual mass coordinate system VM) and then summed to obtain the overall force F T The resulting 6 DOF forces (i.e., forces and torques) are applied to the virtual rigid body, and the resulting motion is calculated by the virtual simulator 86. Thus, the virtual simulator 86 calculates the total force F TThe virtual simulator 86 effectively simulates how the various constraints reflected in the virtual rigid body affect the motion of the virtual rigid body. The virtual simulator 86 performs forward dynamics to calculate the motion of the virtual rigid body for a given total force F being applied to the virtual rigid body. T In one example, the virtual simulator 86 comprises a physics engine, which is executable software stored in non-transitory memory of any one or more of the aforementioned controllers 21, 26, 36 and implemented by the control system 60.

[0108] For virtual simulation, the virtual simulator 86 models the tool 20 as a virtual rigid body in a virtual mass coordinate system VM, where the origin of the virtual mass coordinate system VM is typically located at the center of mass of the virtual rigid body and the coordinate axes are aligned with the principal axes of the virtual rigid body. The virtual rigid body is a rigid representation of the dynamic object and tool 20 for purposes of the virtual simulation. The virtual rigid body is free to move according to six degrees of freedom (6 DOF) in Cartesian space by the virtual simulation. The virtual simulation may be computationally processed without a visual or graphical representation. Therefore, it is not necessary to display the dynamics of the virtual rigid body in the virtual simulation. In other words, the virtual rigid body does not need to be modeled within a graphics application running on the processing unit. The virtual rigid body may exist solely for the virtual simulation.

[0109] The virtual rigid body and its properties (mass, inertia matrix, center of gravity, principal axes, etc.) determine how the tool 20 moves in response to applied forces and torques (e.g., the total force F that incorporates the forces and torques applied by the user and the constraint forces and torques). T(from). It governs whether the tool 20 feels heavy or light, and how the tool moves (e.g., accelerates in translation and rotation) in response to applied forces and torques. By adjusting the properties of the virtual rigid body, the control system 60 can adjust how the tool 20 feels to the user. To obtain as realistic a motion / feel as possible, it may be desirable to have the properties of the virtual rigid body modeled fairly close to the actual properties of the tool 20, but this is not required. For control stability reasons (finite acceleration of a given manipulator 14, control latency, etc.), the virtual mass and inertia can be modeled to be somewhat higher than the physical tool 20.

[0110] The virtual rigid body may correspond to components that may be on or within the tool 20. Additionally or alternatively, the virtual rigid body may extend partially beyond the physical tool 20. The virtual rigid body may account for the tool 20 with the energy applicator 24 or for the tool 20 without the energy applicator 24. Furthermore, the virtual rigid body may be based on a TCP. In one example, the center of gravity of the virtual rigid body is understood to be the point about which the virtual rigid body would rotate if a virtual force were applied to another point on the virtual rigid body and the virtual rigid body were otherwise unconstrained, i.e., not constrained by the manipulator 14. The center of gravity of the virtual rigid body may be close to, but need not be the same as, the actual center of gravity of the tool 20. The center of gravity of the virtual rigid body can be determined empirically. Once the tool 20 is attached to the manipulator 14, the location of the center of gravity can be reset to accommodate individual practitioner preferences.

[0111] The virtual simulator 86 virtually applies forces and / or torques to the virtual rigid bodies in the virtual simulation, i.e., the total force F T The force and torque components from the virtual mass coordinate system VM are virtually applied to the center of gravity of the virtual rigid body, effectively simulating the rigid body dynamics of the tool 20. Thus, the force / torque virtually applied to the virtual rigid body is the external force F cgextthe force / torque associated with the damping force F (e.g., based on input from one or more sensors) damping , inertia force F inertial , and the constraint forces F associated with the various constraints c Force / torque from constraint force F c (as embodied in

[0112] The rigid body Jacobian can be used to transform velocities and forces from one coordinate system (reference frame) to another on the same virtual rigid body, where F ext can also be used to transform the forces and torques in the virtual mass coordinate system VM (e.g., F used in the constraint equations) cgext Next, the virtual simulator 86 generates the damping force F damping and inertial force F inertial is calculated internally to obtain the total force F T and damping force F damping and inertial force F inertial is output as the simultaneous equations for the next time step.

[0113] As shown in Figures 13 and 14, the virtual forward dynamics algorithm was used in the virtual simulation to calculate the overall force F T Applying F to the virtual rigid body simulates the motion of a moving virtual rigid body. In effect, the virtual forward dynamics algorithm solves the equation F=ma (or a=F / m) with 6 DOF and integrates acceleration to generate velocity, which is used to determine the new pose, as shown in FIG. 14. The control system 60 applies virtual forces and / or torques (e.g., a total force F T) into the virtual simulator 86, and these virtual forces and / or torques are applied to the virtual rigid body at the center of gravity (e.g., CG) in the virtual simulation when the virtual rigid body is at an initial pose with an initial velocity. The virtual rigid body has a different state (i.e., position and / or orientation) depending on the control system 60 satisfying the input virtual forces and / or torques, and is moved to a final pose with a final velocity in Cartesian space. The next commanded pose sent to the motion control 76 is based on the final pose calculated by the virtual simulator 86. Thus, the virtual simulator 86 applies a total force F to the virtual rigid body using virtual forward dynamics, as shown in FIG. 14. T The pose command operates to determine the next commanded pose by simulating the effect of adding

[0114] In a simulation, a velocity limit may be imposed on the virtual rigid body. In some cases, the velocity limit may be set high so that it generally does not affect the simulation, or it may be set to any desired value. The virtual rigid body is in an initial pose (initial state) and has an initial velocity at the beginning of each iteration of the virtual simulation (e.g., each time step / interval dt). The initial pose and initial velocity may be determined as the final pose and final velocity output by the virtual simulator 86 at the previous time step. The virtual simulator 86 then calculates and outputs the next commanded pose based on the virtual simulation. The control system 60 is configured to command the manipulator 14 to move the tool 20 based on the commanded pose.

[0115] Returning to FIG. 11 , a block diagram illustrates the process performed to control the operation of the manipulator 14 and the movement of the tool 20 using the constraints described above. In the version shown, the behavior control 74 includes a path handler 88. The path handler 88 operates to generate / output path constraints according to the tool path TP provided by the path generator 68. The tool path TP is an input to the path handler 88. The path handler 88 also calculates the constraint run-time parameters (e.g., the constraint Jacobian Jp , desired velocity V des 11 is enabled and active in semi-autonomous mode, but disabled in other modes.

[0116] The behavior control 74 further comprises a boundary handler 90 for generating boundary constraints based on the one or more virtual boundaries 71 generated by the boundary generator 66. The boundary constraints ultimately enable the control system 60 to control the operation of the manipulator 14 and the movement of the tool 20 based on the relationship between the tool 20 and the one or more virtual boundaries 71 associated with the target site. For example, the control system 60 restricts the relative movement between the tool 20 and the virtual boundary 71 via the boundary constraints. Inputs to the boundary handler 90 include the last commanded pose of the tool 20 (e.g., treated as the current pose), the virtual boundary 71, and the user input states of the tool input 82 and the pendant input RC1. The boundary handler 90 also processes constraint run-time parameters (e.g., the constraint Jacobian J p , desired velocity V des (or Vp2), and generating boundary constraints based on the constraint parameters described above, including determining the constraint distance Δd).

[0117] The behavior control 74 also includes a guide handler 94. In certain situations, it may be desirable to guide the user to manipulate the tool 20 in a manner that guides the tool 20 to a desired position and / or orientation. For example, in some situations, the TCP of the tool 20 may be in a position that causes it to encroach on the virtual boundary 71. In this case, certain operational modes, such as manual mode or semi-autonomous mode, may be disabled (e.g., no boundary or path constraints affecting the movement of the tool 20 are generated) until the TCP of the tool 20 conforms to the virtual boundary 71. The guide handler 94 may obtain a goal state of the tool 20 that causes the tool 20 to conform to the virtual boundary 71 and generate one or more guide constraints based on the goal state and the current state of the tool 20. As mentioned above, the user may also return the tool 20 to conform to the virtual boundary 71 in a free mode, i.e., without generating any guide constraints, or the control system 60 may automatically return the tool 20 to conform to the virtual boundary 71 via a recovery path. As described further below, other methods of achieving conformance to the virtual boundary 71 are also contemplated.

[0118] Inputs to the guide handler 94 include a recovery signal and a last commanded pose (current state). The boundary handler 90 may identify a target position and / or orientation of the tool 20 that does not violate the virtual boundary 71, so that the goal state (e.g., pose) may be part of the recovery signal from the boundary handler 90. The goal state may be defined in an anatomical coordinate system, an anatomical tracker coordinate system, etc., and transformed to a common coordinate system with the last commanded pose. The guide handler 94 defines one or more guide constraints based on the relationship between the last commanded pose and the goal state. The guide constraints are output from the guide handler 94 to the constraint solver 84. The guide handler 94 is configured to activate the guide constraints to provide haptic feedback to the user to guide the user to conform the tool 20 to the virtual boundary 71, and the constraint solver 84 calculates a constraint force F adapted to attract the tool 20 to conform to the virtual boundary 71 based on the guide constraints. c The method is configured to calculate:

[0119] V. Collision check and bounds valid and invalid states The boundary handler 90 performs various collision checks depending on the operating mode, user input state, etc. A first type of collision check involves checking whether / how much the current state of the tool 20 (e.g., current pose) or a proposed state of the tool 20 (e.g., proposed pose) generated in virtual simulation by the virtual simulator 86 encroaches on the virtual boundary 71. This collision check is performed to determine the boundary constraints generated by the boundary handler 90 and that need to be applied by the constraint solver 84 to modify the current / proposed state in a way that prevents, or at least limits, the tool 20 from encroaching on the virtual boundary 71 during normal operation in manual or semi-autonomous mode. In some versions, this type of collision check is performed each frame during operation in manual or semi-autonomous mode, and is performed before a new commanded pose is generated by the virtual simulator 86, and before the commanded pose ultimately generated and executed by the motion control 76 limits the tool 20 from encroaching on the virtual boundary 71. In some versions, this type of collision check may be performed by boundary handler 90 based on the commanded pose calculated in the previous iteration (e.g., the commanded pose from the previous timeframe is set as the current pose). In that case, boundary handler 90 determines the boundary constraints that need to be generated to at least limit the intrusion of virtual boundary 71. For example, the commanded pose from the previous frame may be a pose that results in tool 20 moving slightly across virtual boundary 71, but boundary handler 90 generates boundary constraints in the current frame to return tool 20.

[0120] A method for performing the first type of collision checking is described in U.S. Patent Application Publication No. 2018 / 0353253 to Bowling, entitled "Robotic Surgical System And Method For Producing Reactive Forces To Implement Virtual Boundaries," which is incorporated herein by reference. Other collision detection methods may also be used. For example, if the virtual boundary 71 is defined by a triangle mesh, collision detection may be performed using wide-phase and narrow-phase searches, as described in U.S. Patent No. 9,119,655, which is incorporated herein by reference.

[0121] The boundary handler 90 performs a second type of collision check when the tool input 82 or the pendant input RC1 switches from a second input state to a first input state, for example, when the system 10 switches from hold mode to manual mode or from hold mode to semi-autonomous mode. The current state of the user inputs and the switch can be detected by the user input state detector 92. The user input state detector 92 provides the current state of the user inputs (e.g., the tool input 82 and the pendant input RC1) and indicates a state change of any of them to the boundary handler 90. The second type of collision check can be a subroutine that is executed every time any of the user inputs switches from one state to another.

[0122] To perform the second type of collision check, the boundary handler 90 checks the geometric definition of the virtual boundary 71 against the current state of the tool 20 (e.g., the last commanded pose) to check whether the tool (e.g., the TCP of the tool 20) conforms to or encroaches on the virtual boundary 71. As previously mentioned, predefined, configurable tolerances may be established for the virtual boundary 71 so that some penetration of the tool 20 into the virtual boundary 71 or deviation of the tool 20 from full conformance with the virtual boundary 71 is not considered an encroachment of the virtual boundary 71; therefore, the second type of collision check must also take these tolerances into account. This may simply require checking the bounding volume of the virtual boundary 71 and comparing the current position of the TCP of the tool 20 to the bounding volume. In some versions, in addition to the TCP, the positions of one or more virtual stereotactic interaction mechanisms (SIFs) belonging to the tool 20 may be compared to the bounding volume. These stereotactic interaction feature SIFs can be points corresponding to actual points on the tool 20, spheres with a specific origin and radius, or other suitable geometric shapes. Each stereotactic interaction feature SIF is compared to a bounding volume to check for collisions. Bounding volumes may be defined, for example, by voxels, and the boundary handler 90 can perform collision detection to determine whether the tool 20 is within any voxels. Other methods of collision detection can utilize ray tracing, configuration space search, bounding volume hierarchies, point membership classification (PMC), etc. The boundary handler 90 can detect collisions between the tool 20 and the virtual boundary 71 using any suitable method.

[0123] If the second type of collision check indicates that the tool 20 has entered the virtual boundary 71, a recovery mode is enabled and a recovery signal and an associated goal state are sent to the guide handler 94, which can generate the previously described user feedback described to guide the user to position the tool 20 to conform to the virtual boundary 71. While the tool 20 is entering the virtual boundary 71, the desired operating mode of the manipulator 14 (e.g., manual mode or semi-autonomous mode) can be disabled. When the user input (e.g., tool input 82 or pendant input RC1) is in the first input state, autonomous, boundary-compliant movement of the tool 20 remains disabled in the recovery mode.

[0124] If the collision check indicates that the tool 20 already complies with the virtual boundary 71, the desired mode of operation of the manipulator 14 can be enabled. If the collision check passes (e.g., the TCP and / or other SIFs are entirely within the allowed area of ​​the virtual boundary 71 or do not penetrate the virtual boundary 71 by more than a set distance (e.g., 0.1 mm)), the boundary constraint is activated and movement of the tool 20 is enabled. The second type of collision check and activation of the boundary constraint are performed atomically (in the same time step) to avoid movement race conditions between the check and activation.

[0125] If the second type of collision check fails, the boundary constraints are not enabled and a recovery sequence is initiated via recovery mode. The recovery sequence may include a user message displayed on one or more displays 38 requesting the user to move the tool 20 away from the cutting area. The tool drive is also disabled (e.g., no machining is allowed). In some versions, recovery mode may allow highly damped movement (e.g., damping constraints may be used by the guide handler 94 to provide highly damped movement of the tool 20). The damping coefficient used in the virtual simulation performed by the virtual simulator 86 is the damping force F dampingcan also be adjusted to change and increase damping. Such movement can be effective by allowing the manipulator 14 to respond to user forces and torques applied to the tool 20, but in a relatively damped manner. Highly damped movement allows the user, through haptic interaction (via the damping constraints), to immediately sense that an abnormal condition exists if the user is not currently looking directly at the display 38.

[0126] Recovery mode may trigger activation of a guided haptic mode to help guide the user during recovery. Alternatively, free mode may be activated in recovery mode, allowing the user to freely move tool 20 to conform to virtual boundary 71. As described above, a recovery path may additionally or alternatively be generated in recovery mode and used to autonomously return tool 20 to conform to virtual boundary 71. In this case, pendant input RC1 may be used to control movement of tool 20 along the recovery path. In some versions, in recovery mode, boundary handler 90 or guide handler 94 may generate recovery constraints associated with virtual boundary 71 that have lower adjustment parameters than those of the original boundary constraints, so that tool 20 can recover from encroaching on virtual boundary 71 more gradually (i.e., virtual boundary 71 is effectively modified to be less rigid). In some versions, in recovery mode, control system 60 may move virtual boundary 71 from its starting position (e.g., change its position / orientation) so that tool 20 no longer encroaches on virtual boundary 71. Once the tool 20 returns to complying with the virtual boundary 71, autonomous, boundary-compliant movement of the tool 20 is enabled by the control system 60, with the virtual boundary 71 gently pushing the tool 20 along with it, slowly transitioning back to the starting position, ensuring that the tool 20 remains compliant with the virtual boundary 71.

[0127] The steps performed in recovery mode are performed relatively quickly, so that there is no noticeable delay in initiating tool motion in response to the user's initial activation of tool input 82 or pendant input RC1. For example, this allows the user / manipulator 14 to relatively efficiently move the tool 20 back into the area allowed for relative to the virtual boundary 71. While the tool 20 is moving in recovery mode, the control system 60 periodically or continuously performs a second type of collision check to detect when the tool 20 has returned to complying with the virtual boundary 71. At that point, the aforementioned boundary constraint activation is performed, damping (if used) is returned to its normal setting (which the user will indicate based on their perception that the condition has been resolved), the ability to manipulate the tool 20 using the tool drive is enabled, and messages on the display 38 are updated / cleared, all without requiring the user to release the tool input 82 or pendant input RC1.

[0128] The recovery sequence can also be initiated if one of the trackers is not valid (i.e., not visible or below a quality threshold) during activation. Once the tracker is visible and the second type of collision check is cleared, the control system 60 switches to a boundary-valid state, allowing operation in manual or semi-autonomous mode.

[0129] 15 illustrates a subroutine executed by boundary handler 90 and associated components to perform a second type of collision check. In step 100, boundary handler 90 determines whether an input state has changed, i.e., whether tool input 82 or pendant input RC1 has switched from one state to another. If not, and if boundary handler 90 is not already in recovery mode, boundary handler 90 returns to checking for a state change. If there has been a state change, boundary handler 90 then determines, in step 102, the nature of the state change, i.e., whether it is a change / transition from a second input state to a first input state (e.g., a user input has been actuated) or vice versa (e.g., a release). If tool input 82 or pendant input RC1 has been actuated, boundary handler 90 performs a collision check in step 104 to determine whether tool 20 conforms to or encroaches on virtual boundary 71.

[0130] If the virtual boundary 71 is violated, (i) the control system 60 switches to a boundary disabled state at step 106, (ii) the tool drive that controls the operation of the tool 20 is disabled at step 108, meaning that the control system 60 effectively ignores any input from the user normally associated with the operation of the tool 20, and (iii) a recovery mode is initiated at step 110. Note that these steps may occur substantially simultaneously, and the order shown in FIG. 15 is for illustrative purposes only. At step 112, the guide handler 94 is instructed, via a recovery signal, to generate user feedback to notify the user that the virtual boundary 71 has been violated and / or to guide the user to position the tool 20 in compliance with the virtual boundary 71. Once in recovery mode, the subroutine continues to check for any subsequent state changes, and if there is no state change (e.g., the user is still actuating the tool input 82 or the pendant input RC1), the boundary handler 90 continues the recovery mode and checks whether the tool 20 remains violated by the virtual boundary 71.

[0131] If the virtual boundary 71 has not been violated, (i) the control system 60 switches to a boundary-enabled state in step 114, (ii) the tool drive controlling operation of the tool 20 is enabled in step 116, and (iii) recovery mode (if active) is exited in step 118. Although not shown, when recovery mode is exited, the user-selected manual or semi-autonomous mode is enabled. Note that these steps may occur substantially simultaneously, and the order shown in FIG. 15 is for illustrative purposes only. Also note that when the tool drive is enabled in step 116, the tool 20 may not be immediately activated after entering recovery mode, but control of the tool is returned to the user to enable operation of the tool via the associated user interface UI (e.g., via a button, footswitch, trigger, etc.). If recovery mode is exited while the user is actuating a user interface UI input causing tool operation (e.g., power cutting, etc.), the control system 60 may ignore this input until the user releases and re-engages the input, ensuring no unexpected operation of the tool 20 after exiting recovery mode.

[0132] If tool input 82 or pendant input RC1 is released, (i) control system 60 switches to hold mode and boundaries are disabled in step 120, (ii) the tool drive that controls the operation of tool 20 is disabled in step 122, and (iii) recovery mode (if active) is exited in step 124. Note that these steps may occur substantially simultaneously, and the order shown in Figure 15 is for illustrative purposes only.

[0133] In some versions, the control system 60 can be configured to disable autonomous, boundary-compliant movement of the tool 20 after a predetermined period of time has elapsed since the user input transitioned from the first input state to the second input state. For example, it may be desirable to leave the virtual boundary 71 enabled for a short period of time (e.g., 100-500 ms) after the tool input 82 or pendant input RC1 is released to allow the manipulator 14 / tool ​​20 to come to a stop. Otherwise, if the boundary constraints were immediately disabled, the tool 20 could coast through the virtual boundary 71 and cut or move in an undesirable manner. In some cases, when the tool input 82 or pendant input RC1 is released, the control system 60 can switch to highly damped movement, as described above, to bring the tool 20 to a stop more quickly. After a certain time interval, or once the tool 20 has come to a stop, the virtual boundary 71 can be disabled. This delay can be pre-determined / pre-set as described above, or it can be automatically controlled by the control system 60 by monitoring the tool 20 (e.g., virtual rigid body) and its velocities (linear, rotational) and keeping the boundary constraints active until the degree of velocity (linear, rotational) falls below a certain threshold (or below a set of thresholds defined for each degree of freedom and / or each type of velocity). In some cases, the virtual boundary 71 can remain active until one or more velocities fall below one or more thresholds or until a maximum time is reached, whichever comes first.

[0134] The process outlined in FIG. 15 shows how the boundary handler 90 can operate between a boundary-enabled state, in which boundary constraints are sent from the boundary handler 90 to the constraint solver 84, and a boundary-disabled state, in which boundary constraints are no longer sent from the boundary handler 90 to the constraint solver 84, thereby disabling autonomous, boundary-compliant movement of the tool 20. The boundary handler 90 (i) operates in a boundary invalid state in response to the tool input 82 or the pendant input RC1 transitioning from a first input state to a second input state, with there being a possible delay in transitioning to the boundary invalid state after the tool input 82 or the pendant input RC1 is switched to allow the movement of the manipulator 14 to stabilize; (ii) operates in a boundary valid state in response to the tool input 82 or the pendant input RC1 transitioning from the second input state to the first input state as long as the tool 20 complies with the virtual boundary 71; (iii) operates in a boundary invalid state in response to the tool input 82 or the pendant input RC1 transitioning from the second input state to the first input state if the tool 20 is encroaching on the virtual boundary 71 at the time of the transition, or shortly thereafter; (iv) if the tool input 82 or the pendant input RC1 is still in the first input state, switches from the boundary invalid state to the boundary valid state once the tool 20 is positioned in compliance with the virtual boundary 71.

[0135] The constraint solver 84, the virtual simulator 86, the path handler 88, the boundary handler 90, the user input state detector 92, and the guide handler 94 each comprise executable software stored in the non-transitory memory of any one or more of the aforementioned controllers and implemented by the control system 60. The constraint solver 84, the virtual simulator 86, the path handler 88, the boundary handler 90, the user input state detector 92, and the guide handler 94 may be embodied in one or more software modules stored in any suitable location for implementation by the control system 60.

[0136] VI. Behavioral Control 16 is an overview of the various steps performed by behavior control 74. These include, as described above, steps performed by constraint solver 84, virtual simulator 86, path handler 88, boundary handler 90, and guide handler 94. In step 130, an external force F is applied based on readings obtained from force / torque sensor S. ext is calculated. In steps 132 and 133, constraint data associated with the various virtual constraints is obtained and active constraints are identified. In step 133, constraints, such as boundary constraints, may be updated according to the results of the virtual simulation, as described further below. While FIG. 16 depicts an embodiment in which the proposed state output by the virtual simulation is evaluated for collisions before the final updated state is calculated, in some versions, the output from the virtual simulation is set as the final updated state without further checking for collisions (see FIG. 16A, described further below).

[0137] In steps 134 to 138, rigid body calculations are performed by the virtual simulator 86, and the inverse mass matrix M of the virtual rigid body is calculated. -1 , inertia force F inertial , and damping force F damping In steps 140-144, the constraint solver 84 performs the constraint force calculations described above using the outputs from the rigid body calculations performed in steps 134-138 and the constraint data obtained in steps 132 and 133, and finally determines the constraint force F c In step 146, the constraint force F c is the virtual mass coordinate system VM(F cgext ) converted into external force F ext , damping force F damping , and inertial force F inertial and the total force F T In step 148, in the virtual simulation performed by the virtual simulator 86, the total force F Tis applied to the virtual rigid body to determine the proposed state (e.g., pose and velocity) of the virtual rigid body, and finally in step 150, the initial state and the proposed state are converted to TCP.

[0138] Step 152 uses a software toggle to first follow one path, then the other path on the next run. In the first path, boundary handler 90 performs a first type of collision check in step 154 ​​to determine whether the proposed state would result in a collision with virtual boundary 71. If no collision is detected, the proposed state is verified and saved as an updated state in step 156 and converted to TCP in step 158. The new commanded pause (T TCP ) and velocity (V TCP ) is output by virtual simulator 86 in step 160. If no collision is detected, the toggle (step 152) remains in its current state, i.e., the toggle is not switched to follow another path.

[0139] If a collision is detected, the boundary handler 90 determines the collision time t collision 17A and 17B, the collision time t collision calculates a first distance between the current state and the proposed state calculated in the virtual simulation (FIG. 17A), calculates a second distance between the current state and the virtual boundary 71 (FIG. 17B), and calculates the ratio of the second distance to the first distance over the entire time frame t frame Thus, for example, if the imaginary boundary 71 is crossed at 50% of the distance to the proposed state, the collision time t collision is the time frame t frame Next, the time Δt used to calculate the constraint force in steps 140 to 144 is the collision time t collision(see also FIG. 17C ) and a new proposed state is calculated. Thus, the virtual simulation is trimmed to the time before the collision occurred. The virtual simulation then determines the new proposed state. Once the new proposed state is determined, the toggle (step 152) causes the other path to be followed (because a collision was detected), and the new proposed state becomes the current state updated in step 166. When the transition to step 166 occurs, the toggle automatically resets to follow the first path.

[0140] Next, referring to FIGS. 16 and 17D, the control system 60 determines the time since the collision occurred, the collision time t collision From the original timeframe t frame In other words, the control system 60 is responsible for the time until the end of the period Δt=t frame -t collision A new virtual simulation is run for a new period t. Boundary constraints are obtained for this next round of virtual simulation in step 168, effectively generating the virtual impulses necessary to prevent the tool 20 from crossing the virtual boundary 71 (or so that the tool 20 crosses the virtual boundary 71 only minimally). The boundary constraints obtained in step 168 are then used to update the constraints in step 133, and the virtual simulation is run for a new period t. frame -t collision(see step 148). A new proposed state is determined and converted back to TCP in step 150. The reset toggle in step 152 again follows the first path, and a collision check is again performed in step 154. For some virtual boundaries 71, if a collision is detected, the above steps will always cause the next collision check to be negative, resulting in a new commanded pose and velocity being output in step 160. However, for complex geometries, even when the first collision is detected and addressed, additional collisions may occur in the next round of virtual simulation. In this case, other steps can be performed to account for multiple collisions, but the process shown in Figure 16 is a simpler case for illustrative purposes. In some cases, a maximum number of iterations of the virtual simulation can be set, and the process will stop once the maximum number of iterations is reached. If a collision still occurs after the maximum number of iterations has been reached, it can be handled in a variety of ways: (i) outputting the most recent proposed state as the final updated state, (ii) outputting the state of the previous frame (i.e., the initial state) as the final updated state, or (iii) signaling an error and stopping the motion of the manipulator 14, tool 20, etc.

[0141] 16A, in some versions, steps 152, 154, 156, 162, 164, 166, and 168 of FIG. 16 are removed, and instead the output from the virtual simulation is set as the new updated state, ultimately generating a new commanded pose. In this version, the first collision check is part of step 132, where boundary handler 90 determines the boundary constraints (if any) that it feeds (along with other constraints) to constraint solver 84.

[0142] VII. EXEMPLARY OPERATIONS 18A-18G illustrate an example of the operation of system 10. In FIG. 18A, a user is shown holding grip 83 of tool 20 with tool input 82 activated and in a first input state. The user operates manipulator 14 in manual mode and with a boundary enabled to remove material from a target site (in this case, femur F). A virtual boundary 71 is shown overlaid on femur F, which may be visually indicated to the user via display 38. In FIG. 18A, the user has already removed a small amount of tissue from femur F using tool 20, but a significant amount of material remains within virtual boundary 71 that has not yet been removed. In this configuration, control system 60 maintains tool 20 within virtual boundary 71 by enabling autonomous, boundary-compliant movement of tool 20 as needed, and is responsible for any movement of virtual boundary 71 that might otherwise cause tool 20 to encroach on virtual boundary 71.

[0143] FIG. 18B shows the user moving the TCP of tool 20 above the femur F in preparation for the user to move the femur F to provide better access.

[0144] FIG. 18C illustrates user movement of femur F. The user releases grip 83 of tool 20, releasing tool input 82. Accordingly, tool input 82 is now in the second input state, and control system 60 is now switched to a hold mode and boundary disabled state. Thus, autonomous, boundary-compliant movement of tool 20 is disabled, and femur F can be moved by the user without causing any corresponding movement of tool 20. Movement of femur F is detected by navigation system 32 by localizer 44, which tracks the movement of femur F via first patient tracker 54, which is securely attached to femur F. As shown, tool 20 has moved outside of virtual boundary 71.

[0145] FIG. 18D shows the user again using tool 20 to continue operation in manual mode. More specifically, the user again actuates tool input 82, placing tool input 82 in the first input state. As previously described, this transition from the second input state to the first input state causes boundary handler 90 to perform a second type of collision check. The result of this collision check is that tool 20 has encroached on virtual boundary 71. Accordingly, guide handler 94 can provide one or more forms of user feedback to indicate this state of tool 20 to the user and / or guide the user to return tool 20 to compliance with virtual boundary 71. Alerts and associated guidance are shown as appearing on one or more displays in FIGS. 18D and 18E . In FIG. 18E , the user has moved tool 20 in guided haptic mode or free mode to conform to virtual boundary 71.

[0146] FIG. 18F shows that the user no longer activates the pendant input RC1 of the remote control RC, placing the pendant input RC1 in a first input state and causing the system 10 to operate in semi-autonomous mode. More specifically, the user releases the tool input 82 to transition from manual mode to semi-autonomous mode. When the control system 60 transitions operation from manual mode to semi-autonomous mode, the control system 60 may perform a third type of collision check (e.g., in addition to performing the second type of collision check). More specifically, before the semi-autonomous mode is enabled, the path handler 88 defines a lead-in path LIP from the current pose of the tool 20 to a start point SP on the tool path TP. This straight-line path may intersect the virtual boundary 71 even if both the current pose of the tool 20 and the start point SP conform to the virtual boundary 71 (as shown in FIG. 18F). The third type of collision check determines whether the lead-in path LIP generated by the path handler 88 encroaches on the virtual boundary 71. If virtual boundary 71 is not violated, control system 60 may autonomously move tool 20 along lead-in path LIP to tool path TP in semi-autonomous mode. If tool 20 violates virtual boundary 71 when moved along lead-in path LIP, control system 60 responds by indicating this to the user and providing guidance to the user on how to move tool 20 to avoid such violation, as shown in the "Change Tool Position" instruction shown on display 38 in FIG. 18F. Semi-autonomous mode remains disabled.

[0147] The third type of collision check may be performed similarly to the first type of collision check. As previously described, the path handler 88 generates a lead-in path LIP from the current position (or pose) of the tool 20 to the start of the tool path TP when the manipulator 14 transitions from manual mode to semi-autonomous mode. The boundary handler 90 determines whether movement of the tool 20 along the lead-in path LIP maintains compliance with the virtual boundary 71 or penetrates the virtual boundary 71.

[0148] VIII. Other Collision Checks In some versions, the boundary handler 90 shown in FIG. 18G is configured to determine whether movement of the tool 20 along the lead-in path LIP maintains compliance with or encroaches on the virtual boundary 71 by modeling the motion of multiple stereotactic interaction mechanisms SIFs associated with the tool 20 to determine whether the stereotactic interaction mechanisms SIFs maintain compliance with or encroach on the virtual boundary 71. This can be done by performing similar collision checks on the motion of straight lines from the stereotactic interaction mechanisms SIFs. Such motion can be determined by transforming the lead-in path LIP and its corresponding orientation to each of the corresponding stereotactic interaction mechanisms SIFs (LIPs in FIG. 18G). T ). In some versions, boundary handler 90 is configured to model more complex motion of multiple stereotactic interaction mechanisms with more than two degrees of freedom. Guide handler 94 generates feedback to the user in response to boundary handler 90 determining that tool 20 (e.g., any modeled point thereof) will encroach on virtual boundary 71 if tool 20 is moved from its current position along lead-in path LIP to tool path TP.

[0149] Collision checking for the lead-in path LIP considers valid stereotactic interaction mechanism SIFs and their shapes, i.e., the sphere configured for tool 20 and its possible location, and has TCP sweep each through the range of motion each may encounter while passing through the proposed lead-in path LIP. Note that the lead-in path LIP includes position and / or orientation alignment (for autonomous machining with more than 3 DOF) to the start point SP, meaning that the resulting motion of each stereotactic interaction mechanism SIF during retraction is not necessarily linear. Thus, control system 60 can (i) accurately model the motion of each stereotactic interaction mechanism SIF and perform continuous collision detection according to applicable translational / rotational trajectories, or as shown, or (ii) approximate the trajectory of the stereotactic interaction mechanism SIF by sweeping the collision shape in translation only between the start and end positions. If the lead-in path collision check fails, the user is alerted prior to any motion of the tool 20 (i.e., the manipulator 14 remains in hold mode, free mode, or manual mode), along with potentially enhanced visual information via the display 38 on how to resolve the situation.

[0150] 19 , boundary handler 90 may be configured to perform a fourth type of collision check when a user desires to switch from a current virtual boundary (shown as first virtual boundary 71) to a second virtual boundary 75 while operating in manual or semi-autonomous mode. If a user desires to change the virtual boundary midway through a surgical procedure, such as switching from the first virtual boundary 71 to the second virtual boundary 75, boundary handler 90 checks to ensure that the second virtual boundary 75 is not violated when the switch occurs (e.g., in the same manner that first virtual boundary 71 is evaluated for violations as described above, including taking into account any tolerances set for the second virtual boundary 75). If the second virtual boundary 75 is violated, control system 60 continues operation with the first virtual boundary 71 enabled and the second virtual boundary 75 disabled. However, if the second virtual boundary 75 is not violated, control system 60 activates the second virtual boundary 75 and deactivates the first virtual boundary 71.

[0151] The virtual boundary selector VBS may be utilized by a user to indicate a desire to switch to a second virtual boundary 75. The virtual boundary selector VBS allows the user to select a second virtual boundary 75, which may also be associated with a target site, so that the control system 60 then controls the operation of the manipulator 14 and the movement of the tool 20 to maintain compliance of the tool 20 with the second virtual boundary 75 in the same manner as the control system 60 used to maintain compliance with the first virtual boundary 71. The virtual boundary selector VBS may include a user input located on the manipulator 14, the tool 20, the remote control RC, etc. The user input for the virtual boundary selector VBS may be any suitable type of input device, including those previously described herein. The virtual boundary selector VBS may allow the user to toggle between the first and second virtual boundaries 71, 75, to toggle sequentially among multiple virtual boundaries, or to select from a list of virtual boundaries.

[0152] The control system 60 allows the user to select a second virtual boundary 75 with the virtual boundary selector VBS while the control system 60 continues to control the movement of the manipulator 14 and the movement of the tool 20 to maintain the tool 20's compliance with the first virtual boundary 71. In response to the user's selection of the second virtual boundary 75, the boundary handler 90 determines whether the tool 20 is complying with or penetrating the second virtual boundary 75, and if the boundary handler 90 determines that the tool 20 is complying with the second virtual boundary 75, the boundary handler 90 activates the second virtual boundary 75 and deactivates the first virtual boundary 71, such that the control system 60 controls the movement of the manipulator 14 and the movement of the tool 20 to transition from being based on the first virtual boundary 71 to being based on the second virtual boundary 75. The boundary handler 90 is configured to maintain the first virtual boundary 71 as active if the boundary handler 90 determines that the tool 20 is entering the second virtual boundary 75 in response to a user selecting the second virtual boundary 75 via the virtual boundary selector VBS.

[0153] The control system 60 generates user feedback to the user in response to the boundary handler 90 determining that the tool 20 has entered the second virtual boundary 75 when the user activates the virtual boundary selector VBS to select the second virtual boundary 75. The control system 60 is configured to allow the user to select the second virtual boundary 75 with the virtual boundary selector VBS while the tool input 82 or the pendant input RC1 is in the first input state. Furthermore, the control system 60 continues operation of the tool drive when the user selects the second virtual boundary 75 with the virtual boundary selector VBS. In some versions, the user may also be able to select the second virtual boundary 75 with the virtual boundary selector VBS while the tool input 82 and the pendant input RC1 are in the second input state.

[0154] In some versions, the user can select the second virtual boundary 75 on the fly with the virtual boundary selector VBS, which means that the manual mode or semi-autonomous mode is active (e.g., the tool input 82 or one of the pendant inputs RC1 goes to the first input state) and the manipulator 14 and tool 20 are moving. If a fourth type of collision check indicates a violation of the second virtual boundary 75, the user / manipulator 14 does not stop or force the manipulation / movement of the tool 20 because a valid boundary constraint remained active (e.g., for the first virtual boundary 71).

[0155] In some examples, the first and second virtual boundaries 71, 75 can be considered standard and extended boundaries for a total knee procedure or other surgical procedure. The standard boundary can have limited dimensions (e.g., height, width, and / or depth) based on the size of the implant, while the extended boundary can have one or more dimensions (e.g., wider width) that are larger than the standard boundary, allowing the user to access more tissue (e.g., bone) with the tool 20. The control system 60 can default operation to the standard boundary. Based on user input via the virtual boundary selector VBS, the user can switch to the extended boundary, which allows more tissue to be reached, if necessary. Once the user finishes machining an area requiring a larger (e.g., wider) boundary, the user may wish to return to the standard boundary and apply the remaining cuts. Using a fourth type of collision check, the user can only return from the extended boundary to the standard boundary if the tool 20 is within the area allowed by the standard boundary. If not, the user interface (e.g., a display) indicates intrusion, the failed switch back to the standard boundary, and the user can try again later by reactivating the virtual boundary selector VBS.

[0156] The control system 60 limits the relative movement between the tool 20 and the second virtual boundary 75 when the tool input 82 or the pendant input RC1 is in the first input state and the second virtual boundary 75 is active by generating a second boundary constraint for the second virtual boundary 75 with the boundary handler 90. The constraint solver 84 generates a constraint force F c to maintain the tool in compliance with the first virtual boundary 71 based on the first boundary constraint, or in compliance with the second virtual boundary 75 based on the second boundary constraint when the user successfully selects the second virtual boundary 75 with the virtual boundary selector VBS.

[0157] Note that the collision check / activation / deactivation steps for transitioning from the first virtual boundary 71 to the second virtual boundary 75 can be performed atomically (i.e., in the same time step) to avoid motion race conditions during the collision check and the actuation or time gap between when both virtual boundaries 71, 75 are active or when both are not. The virtual boundaries 71, 75 are sometimes referred to as collision scenes. If the collision check fails, the collision scene is not updated from the first collision scene to the second collision scene. Because the active collision scene is not deactivated unless the collision check passes, the control system 60 can maintain its operation at the point immediately prior to the transition request. User feedback can be provided to notify the user of the failed transition, such as audible feedback, a user message on the display 38, haptic feedback, etc. Although this can be done automatically, it is typically desired to let the user decide when / if to retry the transition from the “active” to the “new” collision scene as the above sequence is repeated.

[0158] In some circumstances, if the user's attempt to switch to the second virtual boundary 75 fails and the first virtual boundary 71 remains active, an assisted mode may be activated to help bring the tool 20 into compliance with the second virtual boundary 75. In the assisted mode, the same controls as described for the recovery mode may be used to help bring the tool 20 into compliance with the second virtual boundary 75, while the manual or semi-autonomous mode remains active with the first virtual boundary 71 still being used to generate boundary constraints to maintain compliance with the first virtual boundary 71.

[0159] The assisted mode may include the control system 60 (i) generating visual cues on a display or elsewhere to guide the user to cause movements of the tool 20 that position the tool 20 in compliance with the second virtual boundary 75; (ii) generating guide constraints using the guide handler 94 to guide the user to conform to the second virtual boundary 75; (iii) generating an assisted tool path for autonomously moving the tool 20 to conform to the second virtual boundary 75; (iv) moving the second virtual boundary 75 from its starting position so that the tool 20 conforms to it, and then gradually returning the second virtual boundary 75 to its starting position and gently pulling the tool 20 along it; and / or (v) generating assisted constraints associated with the second virtual boundary 75 that have adjustment parameters that are lower than the adjustment parameters of the original boundary constraints of the second virtual boundary 75 so that the tool 20 can more gradually transition to conform to the second virtual boundary 75 (i.e., the second virtual boundary 75 is effectively modified to be less rigid). Other methods for guiding the tool 20 to conform to the second virtual boundary 75 are also contemplated. In some cases, one or more of these methods may be considered by the control system 60 and executed if certain conditions are met. The control system 60 may ascertain how close the tool 20 is to the second virtual boundary 75 and then select an appropriate method. For example, if the tool 20 is within 1-2 mm of the second virtual boundary 75, it may be possible to temporarily activate the second virtual boundary 75 with looser adjustment parameters, or it may be appropriate to temporarily shift the second virtual boundary 75. Once the tool 20 conforms to the second virtual boundary 75, the control system 60 may manually or automatically switch control to the second virtual boundary 75. Other embodiments include backtracking along the axis of the tool, for example, for TKA or THA acetabular preparation for rim osteophytes.

[0160] The remote control RC and / or other input devices on the various user interfaces UI can be used to switch, activate, and / or deactivate various operating modes of the manipulator 14. The control system 60 can be configured to automatically switch modes in certain situations. The control system 60 can also first prompt the user before operating in manual or semi-autonomous mode if the tool 20 is found to be encroaching on the virtual boundary 71 by the boundary handler 90 if these modes were initially disabled when these modes were selected. The control system 60 can also prompt the user before operating in guided haptic mode. Such prompting can include providing a selectable prompt on one or more of the displays 38 to proceed in manual mode, semi-autonomous mode, or guided haptic mode. The user can select to proceed in manual mode, semi-autonomous mode, guided haptic mode, etc. via any suitable input device on any user interface UI, including the remote control RC.

[0161] The current state of the tool 20 relative to the virtual boundary 71, tool path TP, goal state, and / or target region may be output by the navigation system 32 and depicted on the display 38 via a graphical representation of the tool 20, the virtual boundary 71, the tool path TP, the goal state, and / or the target region, e.g., the femur F, the tibia T, the pelvis PEL, the vertebral bodies, or other anatomical structures. These graphical representations may be updated in real time, allowing a user to visualize the movement of the tool 20 relative to the virtual boundary 71, the tool path TP, the goal state, the anatomical structures, etc. For example, the graphical representations of the tool 20 and the anatomical structures may move on the display 38 in real time along with the actual movement of the tool 20 by the manipulator 14 and the actual movement of the anatomical structures.

[0162] The various modes described herein can be used in various types of surgical systems. For example, the manipulator can include a teleoperated robotic arm controlled through a user interface that is remotely located relative to and controls the teleoperated robotic arm. The user interface can include a separate manipulator, such as a 6DOF control unit manually operated by a user, e.g., a separate manipulator with active joints to provide haptic feedback to the user.

[0163] IX. Stereotaxic interaction mechanism As described above, a stereotactic interaction mechanism (SIF) may be utilized by a system for collision detection. This section describes various configurations or embodiments in which a SIF may be implemented or utilized. The following configurations or embodiments of a SIF may be utilized with any of the above-described technologies or components fully incorporated by reference in this section. Alternatively or additionally, the following configurations or embodiments of a SIF may be utilized independently of any of the above-described technologies and for other general purposes such as, but not limited to, robotic control, collision avoidance, user experience, etc.

[0164] The stereotactic interaction mechanism SIF may be attributed to any object capable of interacting with the virtual boundary 71. Optionally, the SIF may be attributed to any component whose pose can be controlled (manually, automatically, or kinematically). For example, the SIF may be attributed to any portion of a robotic surgical system, the tool 20, the end effector 22, the energy applicator 24 or TCP, the manipulator 14, any link 18 or joint J of the manipulator 14, the base 16, or any other part of the kinematic chain forming the manipulator 14. The SIF may be attributed to other objects or components of the operating room or surgery, such as a handheld tool, an operating table, a head-mounted device, a handheld display device or tablet, a tracker, a retractor, a patient, personnel, or staff. The location of the SIF may be known because any object can be tracked using any suitable method, including, but not limited to, the localization techniques described above. The objects to which the SIFs are attributed may be selected based on inputs to the program and / or may be automatically generated based on factors such as, for example, the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0165] Furthermore, the virtual boundary 71 described in this section may be assigned to any object other than an anatomical structure. For example, the virtual boundary 71 may be assigned to any portion of a robotic surgical system, such as the tool 20, the end effector 22, the energy applicator 24 or TCP, the manipulator 14, any link 18 or joint J of the manipulator 14, the base 16, or any other part of the kinematic chain forming the manipulator 14. The virtual boundary 71 may be assigned to other objects in the operating room or components of the procedure, such as an imaging device (e.g., a c-arm, gantry, CT scanner), a handheld tool, an operating table, a head-mounted device, a handheld display device or tablet, a tracker, a retractor, a patient, personnel, or staff. The virtual boundary 71 may have any suitable shape or configuration depending on the object to which it is assigned. For example, if the virtual boundary 71 is assigned to a link 18 of the manipulator 14, the virtual boundary 71 may take the shape of a volume surrounding the link 18, as shown in FIG. 20 , for example. Other configurations are contemplated. The location of the virtual boundary 71 may be known because any object may be tracked using any suitable method, including, but not limited to, the localization techniques described above. The object to which the virtual boundary 71 belongs may be selected based on inputs to the program and / or may be automatically generated based on factors such as the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0166] The SIFs may be defined, positioned, customized, and implemented using the control system 60 described in the previous section, including the software program 78 and the boundary handler 90. The boundary handler 90 may detect collisions between any number of SIFs and the virtual boundary 71 using any suitable method. In response to a collision or anticipated collision between any one or more SIFs and the virtual boundary 71, the control system may adjust the pose of the respective objects and / or generate an alert or notification. Such responses from the control system may be provided to reactively or proactively prevent, avoid, mitigate, or reduce the collision.

[0167] Any number of SIFs can be attributed to any number of objects. In the example of FIG. 20, multiple SIFs are attributed to tool 20, in this case an elongated cutting burr for comminuting femur F for THA. Of course, this example is for illustrative purposes and is not intended to limit the scope of ways in which SIFs may be utilized. The number of SIFs attributed can be determined based on inputs to the program and / or can be automatically generated based on factors such as the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0168] In some embodiments, SIFs can be attributed where they are located directly on the object to which they are attributed. For example, in FIG. 20 , several SIFs identified as SIF-1 are directly attributed to the shaft SH of the tool 20. These SIF-1 are directly located along and spaced apart from one another on the shaft SH. In this example, the SIF-1 are provided to avoid collisions between the shaft SH and a virtual boundary 71 that is attributed to the inner surface of the canal of the femur F. For example, during milling, the tool 20 can periodically reorient as the TCP follows the tool path, which in turn causes reorientation of the shaft SH. Having multiple SIF-1 along the shaft SH reduces the likelihood of physical collisions between the shaft SH and anatomical structures in response to such reorientations.

[0169] In other embodiments, SIFs can be located at locations distant from the object to which they belong. For example, in FIG. 20 , several SIFs identified as SIF-2 are located at the proximal end of tool 20. These SIF-2 are located a specified distance away from tool 20. The spacing of SIF-2 in this example may be appropriate because tool 20 is likely to undergo greater orientational shifts (e.g., angular motion) in response to reorientation near the proximal end compared to near the TCP. Thus, spacing SIF-2 from tool 20 provides additional cushioning or a faster response to a crash or anticipated crash. In this example, SIF-2 are positioned to form a ring of SIFs at the proximal end. The configuration or placement of SIFs can be derived directly from the geometry of the object, e.g., a 2D cross-section of shaft SH or tool 20. SIFs can be positioned directly adjacent to each other or spaced apart from each other.

[0170] The SIF intervals for each object to which the SIF belongs may be determined based on inputs into the program and / or may be automatically generated based on factors such as the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0171] In other embodiments, the SIFs can be configured with any suitable geometry. For example, the SIFs can be points, areas, or volumes. The SIFs can be any suitable shape. In the example of FIG. 20 , the SIFs are spherical. However, other 2D shapes or volumes are contemplated, such as, but not limited to, planes, hyperboloids, paraboloids, cylinders, cubes, pyramids, cones, cuboids, ellipsoids, prisms, or any type of polyhedron. For any type of geometric configuration, the SIFs can also be configured with any suitable size. For example, in FIG. 20 , the spherical SIF-1 increases in volume as the SIF approaches the proximal end of the tool 20. Again, this may be implemented to provide an early response to a collision or anticipated collision. The geometric configuration or size of the SIF can be determined based on inputs to the program and / or can be automatically generated based on factors such as, for example, the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0172] In one embodiment, the SIFs are not infinitely stiff, but instead each SIF may have tuning (stiffness) parameters to tune the stiffness of the SIF relative to the virtual constraint, for example, by incorporating spring and damping parameters into the constraint. Such parameters may include a constraint force mix parameter (C) and an error reduction parameter (ε). The spring and damping parameters may be tuned before or during operation. The tuning parameters of the SIFs may be different for different objects, conditions, locations, or geometric configurations. The SIFs may include a first SIF having a first value of the tuning parameter and a second SIF having a second value of the tuning parameter, the first value being greater than the second value, such that the resulting constraint force F c The virtual forces and / or torques embodied in are adapted to result in a stronger movement of the tool 20 as a result of the first SIF compared to the second SIF. The values ​​of the tuning parameters may be larger (e.g., stiffer) for position constraints than for orientation constraints, or vice versa.

[0173] The tuning parameters of the SIF can be set to remain constant, to rise / fall exponentially with the constraint distance, to vary linearly with the constraint distance, to vary depending on the constraint direction, to take into account the effect of gravity, etc. The tuning parameters are the constraint force F c Or the constraint force F is calculated based on the virtual constraint, such as increasing or decreasing the stiffness depending on the size of the component. cに The tuning parameters of the SIF and their values, their correlation to particular relationships, and the manner in which they may be scaled may be stored in one or more look-up tables in any suitable memory in the control system 60 for later retrieval.

[0174] In one embodiment, a first adjustment parameter can be defined for a first SIF, and a second adjustment parameter can be defined for a second SIF. The first adjustment parameter and the second adjustment parameter are different values. In one example, the first and second differently adjusted SIFs are located on different components of the kinematic chain KC of the robot manipulator 14. The differently adjusted SIFs can also be located at different positions on the same component of the kinematic chain KC. The kinematic chain KC is formed by the manipulator 14, including the base 16, the plurality of links 18 and joints J, the tool 20 (if applicable) including the shaft SH, and any rigidly attached components such as the energy applicator 24. The end effector 22 can also be part of the kinematic chain KC. Additionally, any attachment system or sterile interface coupled between the manipulator 14 and the end effector 22 can be part of the kinematic chain KC. One example of a mounting system and / or sterile interface mechanism that can be part of a kinematic chain is described in U.S. Patent Application Publication No. 2020 / 0170724 A1, entitled "Mounting System With Sterile Barrier Assembly For Use In Coupling Surgical Components," the entire contents of which are incorporated herein by reference. As used herein, the term "kinematic chain" refers to an assembly of rigid bodies connected by joints, where the stiffness of the bodies allows for constrained motion so that the kinematics of the rigid bodies can be determined and related to other rigid bodies in the chain using a mathematical model. In the example of FIG. 20, the kinematic chain further forms a "mechanism" because at least one link is mechanically grounded.

[0175] In the example of FIG. 20 , differently tuned first and second SIFs can be disposed on the tool 20 and / or shaft SH. In this case, the tuning parameters of SIF-2 can be stiffer (e.g., more damping or a larger spring constant) than the tuning parameters of SIF-1 due to the increased likelihood of undesired directional motion of the shaft SH near the proximal end of the tool 20. The greater stiffness of SIF-2 can cause the manipulator 14 to command more non-destructive corrective motion near the proximal end of the tool 20. In other examples, SIF-2 can be less stiff than SIF-1. This tuning configuration can be implemented to provide a smoother transition when a collision occurs between the energy applicator / tool ​​and the virtual boundary 71. This implementation can initially result in greater penetration of the virtual boundary 71 by the shaft SH or tool 20, but other motions of the robotic manipulator can be changed before, during, or immediately after the penetration to account for such greater penetration resulting from the collision. Additionally, having different tuning parameters for the SIFs can reduce the likelihood that a collision with the boundary 71 will cause an error condition and stall the operation of the manipulator 14. Such tuning parameter configurations can be applied to any number of SIFs for the purpose of mitigating undesired object motion or for any other purpose related to robot control and / or user experience.

[0176] Additionally, multiple tuning parameters can be associated with any single SIF. For example, the tuning parameters of one SIF can vary with respect to time, detection state, distance of the SIF to respective objects, distance of the SIF to virtual boundary 71, etc. In another example, one SIF may be large enough to simultaneously include multiple tuning parameters. For example, one portion / surface / volume of the SIF can be configured to be more or less rigid than another portion / surface / volume of the same SIF.

[0177] The characteristics of any of the above SIFs may change dynamically during surgery or while the robotic system is paused. For example, the control system may determine a SIF-changing event in response to some control event or environmental condition. Such conditions may be expected or unexpected and may be detected by the control system using localization data, kinematic data, or any combination thereof. In response, the control system may dynamically change the position, geometric configuration, spacing, or stiffness of one or more SIFs. This dynamically changing mechanism may be managed based on programmatic inputs and / or may be managed automatically based on factors such as the surgical plan, the type or steps of the surgery, surgeon preferences, etc.

[0178] Any of the described features, characteristics, properties, and / or behaviors of a SIF may be referred to as parameters of the SIF. Any of the above embodiments or parameters of a SIF may be used individually or in any combination thereof.

[0179] The foregoing description has set forth several embodiments. However, the embodiments discussed herein are not intended to be exhaustive or to limit the invention to any particular form. The terminology used is intended to be in the nature of words of description rather than of limitation. Many modifications and variations are possible in light of the above teachings, and the invention may be practiced otherwise than as specifically described.

Claims

1. 1. A surgical system comprising: Tools and a manipulator supporting the tool; a control system that controls the operation of the manipulator and the movement of the tool based on a relationship between the tool and a virtual boundary associated with a target portion; Equipped with the control system includes a user input having a first input state and a second input state, and a boundary handler; The control system includes: enabling autonomous, boundary-compliant movement of the tool when the user input is in the first input state, wherein the tool maintains compliance with the virtual boundary; when the user input is in the second input state, disabling autonomous, boundary-compliant movement of the tool and preserving the current pose of the tool; determining, using the boundary handler, that the virtual boundary has moved such that the tool at the current pose penetrates the virtual boundary in response to the user input transitioning from the second input state to the first input state; Surgical system.

2. 2. The surgical system of claim 1, wherein the control system is configured to initiate a recovery mode in response to the tool penetrating the virtual boundary when the user input transitions from the second input state to the first input state, and wherein autonomous, boundary-compliant movement of the tool remains disabled in the recovery mode when the user input is in the first input state.

3. 2. The surgical system of claim 1, wherein the tool has a motor, and the control system is configured to disable operation of the motor in response to the tool penetrating the virtual boundary when the user input transitions from the second input state to the first input state.

4. the control system generates user feedback, including one or more of audible feedback, visual feedback, and haptic feedback, to guide the user to position the tool to conform to the virtual boundary in the recovery mode; The surgical system of claim 2 , wherein the control system stops generating the user feedback when the tool is positioned to conform to the virtual boundary.

5. Where the control system is configured to stop generating the user feedback when the tool is positioned to conform to the virtual boundary, the control system is further configured to limit relative movement between the tool and the virtual boundary when the user input is in the first input state by generating a boundary constraint in the boundary handler; The control system includes: a constraint solver for calculating constraint forces adapted to maintain the tool in compliance with the virtual boundary based on the boundary constraints; a virtual simulator that simulates the dynamics of the tool in a virtual simulation based on the constraint forces and outputs a commanded pose; and The surgical system of claim 4 , wherein the control system is configured to command the manipulator to move the tool based on the commanded pose.

6. the boundary handler is operable in a boundary enabled state, in which boundary constraints are sent from the boundary handler to the constraint solver, thereby enabling autonomous, boundary-compliant movement of the tool when the virtual boundary moves relative to the tool in a manner that would otherwise cause the tool to encroach on the virtual boundary, and a boundary disabled state, in which boundary constraints are no longer sent from the boundary handler to the constraint solver, thereby disabling autonomous, boundary-compliant movement of the tool such that the virtual boundary can move relative to the tool in a manner that would cause the tool to encroach on the virtual boundary; The boundary handler: responsive to a transition of the user input from the first input state to the second input state, the boundary operates in an invalid state; and operating in a state in which the boundary is valid in response to the user input transitioning from the second input state to the first input state while the tool conforms to the virtual boundary; 6. The surgical system of claim 5, wherein the system is configured to operate in an invalid state of the virtual boundary in response to the user input transitioning from the second input state to the first input state while the tool is invading the virtual boundary.

7. 7. The surgical system of claim 6, wherein the control system is configured to provide haptic feedback to the user to guide the user in placing the tool in compliance with the virtual boundary by either activating one or more guide constraints to guide the tool in compliance with the virtual boundary or damping movement of the tool.

8. The surgical system of claim 7 , wherein the control system is configured to switch the boundary handler from the boundary disabled state to the boundary enabled state when the tool is positioned to conform to the virtual boundary.

9. 2. The surgical system of claim 1, wherein the user input is configured such that the first input state indicates a user is actively engaging the tool and the second input state indicates the user has released the tool.

10. the user input is placed on the tool and configured to actuate the user input to place the user input in the first input state and to release the user input to place the user input in the second input state; the tool has a grip, and the user input includes a presence detector for detecting a user's hand on the grip. The surgical system of claim 1 .

11. 2. The surgical system of claim 1, wherein the control system includes a pendant, the user input being in the pendant, and the user input being configured such that the user input is actuated to place the user input in the first input state and the user input is released to place the user input in the second input state.

12. 2. The surgical system of claim 1, wherein the user input is further defined as a tool input disposed on the tool, the first and second input states are further defined as a tool input first state and a tool input second state, the control system includes a pendant and a pendant input disposed on the pendant, and the pendant input has a pendant input first state and a pendant input second state.

13. 13. The surgical system of claim 12, wherein the manipulator is operable in a manual mode in which the manipulator moves the tool in response to user forces and torques applied to the tool by a user when the tool input is in the tool input first state, and wherein the manipulator is operable in a semi-autonomous mode in which the manipulator moves the tool along a tool path when the pendant input is in the pendant input first state.

14. the boundary handler is configured to determine whether the tool conforms to or penetrates the virtual boundary in response to a switching action by the control system of the manipulator from one of the manual mode and the semi-autonomous mode to the other of the manual mode and the semi-autonomous mode; the control system comprises a path handler configured to generate a lead-in path from a current position of the tool to the tool path when the manipulator switches from the manual mode to the semi-autonomous mode; the boundary handler is configured to determine whether movement of the tool along the lead-in path maintains compliance with or encroaches on the virtual boundary; the boundary handler is configured to determine whether movement of the tool along the lead-in path maintains compliance with or penetrates the virtual boundary by modeling motion of a plurality of stereotactic interaction mechanisms associated with the tool to determine whether the stereotactic interaction mechanisms maintain compliance with or penetrate the virtual boundary; the boundary handler is configured to model motion of the plurality of stereotactic interaction mechanisms in three or more degrees of freedom; the control system includes a guide handler configured to generate user feedback to the user in response to the boundary handler determining that the tool will enter the virtual boundary when the tool moves from the current position along the lead-in path to the tool path. The surgical system of claim 13.

15. The control system includes: determining a first parameter for a first stereotactic interaction mechanism of the plurality of stereotactic interaction mechanisms; determining a second parameter for a second stereotactic interaction mechanism of the plurality of stereotactic interaction mechanisms, the first parameter being different from the second parameter; To do The surgical system of claim 14.

16. the control system the tool is stopped; and a predetermined period of time elapses following the transition of the user input from the first input state to the second input state; the linear velocity of the tool is below one or more thresholds; the angular velocity of the tool falls below one or more thresholds; The surgical system of claim 1 , configured to disable autonomous, boundary-compliant movement of the tool in response to one or more of:

17. The surgical system of claim 1 , wherein the control system is configured to determine whether the tool remains compliant with the virtual boundary based on a tolerance defined for the virtual boundary.

18. The surgical system of claim 2 , wherein the control system is configured to generate a restored tool path in the restored mode to move the tool to conform to the virtual boundary.

19. 3. The surgical system of claim 2, wherein the control system is configured to move the virtual boundary in the recovery mode from a start position, return the tool to conform to the virtual boundary, and then return the virtual boundary to the start position while enabling autonomous, boundary-compliant movement of the tool.

20. 1. A method for controlling movement of a manipulator supporting a tool based on a relationship between the tool and a virtual boundary associated with a target site, comprising: a control system initiating autonomous, boundary-compliant movement of the tool such that the tool maintains compliance with the virtual boundary as the virtual boundary moves relative to the tool when the user input is in a first input state; when the user input is in a second input state, the control system disabling autonomous, boundary-compliant movement of the tool and maintaining the current pose of the tool; the control system, in response to the user input transitioning from the second input state to the first input state, determining that the virtual boundary has moved such that the tool at the current pose penetrates the virtual boundary; A method comprising:

Images