System and method for controlling robot movement based on virtual boundary control tool

Abstract

Systems and methods are provided for controlling robotic movement of a tool based on one or more virtual boundaries. The system includes a tool and a manipulator supporting the tool. A control system controls operation of the manipulator and movement of the tool based on a relationship between the tool and one or more virtual boundaries associated with a target site. The control system includes a boundary handler to determine whether the tool is compliant with the one or more virtual boundaries or violates the one or more virtual boundaries.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and all benefits to U.S. Provisional Patent Application No. 63 / 000,860, filed March 27, 2020, the disclosure of which is hereby incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to systems and methods for robot movement based on virtual boundary control tools. Background Technology

[0004] A surgical system may include a robotic manipulator and tools attached to the manipulator to perform surgical procedures on a patient. During the procedure, the surgical system may restrict the movement of the tools to avoid violating virtual boundaries established to protect parts of the patient's anatomy from the tools' influence.

[0005] In some situations, users may want to adjust the patient's position to improve access to target areas on the patient's anatomy or to enhance the user's visualization of the patient's anatomy, or the patient's anatomy may move for some other reason. To maintain compliance with virtual boundaries when the patient's anatomy moves, the surgical system can command the manipulator to autonomously move the tool to compensate for this movement. However, some users may prefer to limit this autonomous movement of the tool. For example, users may not want the tool to move accidentally, or they may want the tool's movement to occur only in response to user input. Summary of the Invention

[0006] This summary presents a simplified version of the selected concepts further described below in the detailed embodiments. This summary is not intended to limit the scope of the claimed subject matter, nor is it intended to identify key or essential features of the claimed subject matter.

[0007] According to a first aspect, a surgical system is provided, the surgical system comprising: a tool; a manipulator for supporting the tool; and a control system for controlling operation of the manipulator and movement of the tool based on a relationship between the tool and a virtual boundary associated with a target site. The control system includes a user input having a first input state and a second input state. The control system is configured to enable autonomous compliance boundary movement of the tool when the user input is in the first input state, such that the tool remains compliant with the virtual boundary. The control system is configured to disable autonomous compliance boundary movement of the tool when the user input is in the second input state. The control system includes a boundary handler to determine whether the tool violates the virtual boundary in response to a transition of the user input from the second input state to the first input state.

[0008] According to a second aspect, a surgical system is provided, the surgical system comprising: a tool; a manipulator for supporting the tool; and a control system for controlling the operation of the manipulator and the movement of the tool based on a relationship between the tool and a first virtual boundary associated with a target site. The control system includes a virtual boundary selector to enable a user to select a second virtual boundary associated with the target site. The control system is configured such that the user can use the virtual boundary selector to select the second virtual boundary while the control system maintains the tool conforming to the first virtual boundary. The control system includes a boundary handler to determine whether the tool conforms to the second virtual boundary in response to the user's selection of the second virtual boundary.

[0009] According to a third aspect, a surgical system is provided, the surgical system comprising: a tool; a manipulator for supporting the tool; and a control system for controlling the operation of the manipulator and the movement of the tool based on a relationship between the tool and a virtual boundary associated with a target site. The manipulator can operate in multiple modes, including a manual mode and a semi-autonomous mode. In the manual mode, the manipulator moves the tool in response to user forces and torques applied to the tool by a user. In the semi-autonomous mode, the manipulator moves the tool along a tool path. The control system includes a path handler to generate an ingress path from the current position of the tool to the tool path in response to the control system switching to the semi-autonomous mode. The control system further includes a boundary handler to determine whether the movement of the tool along the ingress path will maintain compliance with the virtual boundary by modeling the motion of a plurality of stereotactic interactive features associated with the tool before the tool moves along the ingress path, thus determining whether the stereotactic interactive features comply with the virtual boundary.

[0010] According to a fourth aspect, a method is provided for controlling the operation of a manipulator supporting a tool based on a relationship between a tool and a virtual boundary associated with a target location. The method includes initiating an autonomous compliance boundary movement of the tool when user input is in a first input state and in response to movement of the virtual boundary relative to the tool, such that the tool remains compliant with the virtual boundary. The method further includes disabling the autonomous compliance boundary movement of the tool when the user input is in a second input state; and determining whether the tool violates the virtual boundary in response to a change in user input from the second input state to the first input state.

[0011] According to a fifth aspect, a method is provided for controlling the operation of a manipulator supporting a tool based on a relationship between a tool and a first virtual boundary associated with a target region. The method includes enabling a user to select a second virtual boundary associated with the target region. The method further includes enabling the user to select the second virtual boundary while maintaining the tool's compliance with the first virtual boundary; and determining whether the tool conforms to the second virtual boundary in response to the user's selection of the second virtual boundary.

[0012] According to a sixth aspect, a method is provided for controlling the operation of a manipulator supporting a tool based on a relationship between a tool and a virtual boundary associated with a target location, wherein the manipulator can operate in multiple modes, including a manual mode and a semi-autonomous mode, in which the manipulator moves the tool in response to user forces and torques applied to the tool by a user, and in the semi-autonomous mode, the manipulator moves the tool along a tool path. The method includes switching the manipulator to the semi-autonomous mode; and generating an ingress path from the current position of the tool to the tool path in response to switching the manipulator to the semi-autonomous mode. The method further includes determining whether the movement of the tool along the ingress path will maintain compliance with the virtual boundary by modeling the motion of a plurality of stereoscopic interactive features associated with the tool before the tool moves along the ingress path to determine whether the stereoscopic interactive features conform to the virtual boundary.

[0013] According to a seventh aspect, a surgical system is provided, the surgical system comprising: a positioning system configured to track a first object and a second object; and at least one controller configured to: associate a virtual boundary with one or more first objects; associate a first stereoscopic interaction feature and a second stereoscopic interaction feature with one or more second objects; define a first parameter for the first stereoscopic interaction feature; define a second parameter for the second stereoscopic interaction feature, wherein the first parameter is different from the second parameter; and generate a response based on an interaction between at least one of the first stereoscopic interaction feature and the second stereoscopic interaction feature and the virtual boundary.

[0014] According to an eighth aspect, a method for operating a surgical system is provided, the surgical system including a positioning system and at least one controller, the method comprising: tracking a first object and a second object using the positioning system; associating a virtual boundary with one or more first objects using the at least one controller; associating a first stereoscopic interaction feature and a second stereoscopic interaction feature with one or more second objects using the at least one controller; defining a first parameter for the first stereoscopic interaction feature using the at least one controller; defining a second parameter for the second stereoscopic interaction feature using the at least one controller, wherein the first parameter is different from the second parameter; and generating a response using the at least one controller based on an interaction between at least one of the first stereoscopic interaction feature and the second stereoscopic interaction feature and the virtual boundary.

[0015] Any of the above aspects can be combined in whole or in part.

[0016] Any of the foregoing aspects may be used in conjunction with any of the following implementations, whether or not these implementations are used in whole or in part:

[0017] In some embodiments, the manipulator is a surgical robot manipulator, which includes a base and multiple links and joints forming a robotic arm. In some embodiments, the manipulator is a handheld manipulator that is freely supported in the user's hand against gravity, wherein the base is the base portion of the tool (e.g., the portion held by the user's hand) and the tool tip is movable relative to the base portion. In some embodiments, the tool is a rotary cutting file, saw, cutting guide, ultrasonic vibrating tool, laser cutting tool, etc.

[0018] In some implementations, the control system is configured to initiate a recovery mode in response to a tool violating a virtual boundary. In some implementations, the violation occurs when user input transitions from a second input state to a first input state. In some implementations, the tool's autonomous compliance boundary movement remains disabled in recovery mode when user input is in the first input state.

[0019] In some implementations, the tool includes a tool driver and the control system is configured to disable the operation of the tool driver in response to a violation of a virtual boundary by the tool. In some implementations, the violation occurs when user input transitions from a second input state to a first input state.

[0020] In some implementations, the control system is configured to guide the user to place the tool in compliance with a virtual boundary in recovery mode. In some implementations, guidance occurs by generating user feedback. In some implementations, the feedback includes one or more of auditory, visual, and tactile feedback. In some implementations, the control system is configured to stop generating the user feedback when the tool is placed in compliance with the virtual boundary. In some implementations, the control system is configured to restrict relative movement between the tool and the virtual boundary. In some implementations, relative movement restriction occurs when user input is in a first input state and by generating boundary constraints using a boundary handler.

[0021] In some embodiments, the control system includes a constraint solver for calculating constraint forces suitable for maintaining the tool's compliance with the virtual boundary based on the boundary constraints. In some embodiments, the control system includes a virtual simulator for simulating the dynamics of the tool in a virtual simulation based on the constraint forces and outputting a commanded attitude. In some embodiments, the control system is configured to command the manipulator to move the tool based on the commanded attitude.

[0022] In some implementations, the boundary handler can operate between a boundary-enabled state and a boundary-disabled state. The boundary-enabled state is a state in which boundary constraints are transmitted from the boundary handler to the constraint solver to thereby enable autonomous compliant boundary movement of the tool. In some implementations, autonomous compliant boundary movement is enabled when a virtual boundary moves relative to the tool in a manner that would otherwise cause the tool to violate the virtual boundary. In some implementations, the boundary-disabled state is a state in which boundary constraints are no longer transmitted from the boundary handler to the constraint solver to thereby disable autonomous compliant boundary movement of the tool, such that the virtual boundary could cause the tool to move relative to the tool in a manner that causes the tool to violate the virtual boundary. In some implementations, the boundary handler is configured to operate in the boundary-disabled state in response to a change in user input from a first input state to a second input state. In some implementations, the boundary handler is configured to operate in the boundary-enabled state in response to a change in user input from a second input state to a first input state if the tool conforms to the virtual boundary. In some implementations, the boundary handler is configured to operate in the boundary-disabled state in response to a change in user input from a second input state to a first input state if the tool violates the virtual boundary.

[0023] In some embodiments, the control system is configured to provide haptic feedback to a user to guide the user in placing the tool to conform to the virtual boundary. In some embodiments, feedback occurs by activating one or more guiding constraints to guide the tool to conform to the virtual boundary. In some embodiments, the control system is configured to provide haptic feedback to a user to guide the user in placing the tool to conform to the virtual boundary. In some embodiments, haptic feedback occurs by inhibiting movement of the tool. In some embodiments, the control system is configured to switch the boundary handler from a boundary-disabled state to a boundary-enabled state when the tool is placed to conform to the virtual boundary.

[0024] In some implementations, the user input is configured such that the first input state indicates that the user is actively engaging the tool and the second input state indicates that the user has released the tool.

[0025] In some embodiments, the user input is located on the tool and is configured such that the user input is actuated to place the user input in a first input state and released to place the user input in a second input state. In some embodiments, the tool has a grip, and the user input includes a presence detector to detect the user's hand on the grip.

[0026] In some embodiments, the control system includes a teach pendant and user input is located on the teach pendant. In some embodiments, the user input is configured such that the user input is actuated to place the user input in a first input state and the user input is released to place the user input in a second input state.

[0027] In some embodiments, user input is further defined as tool input located on the tool. In some embodiments, the first input state and the second input state are further defined as a tool input first state and a tool input second state. In some embodiments, the control system includes a teach pendant and a teach pendant input located on the teach pendant, the teach pendant input having a teach pendant input first state and a teach pendant input second state.

[0028] In some embodiments, the manipulator can operate in a manual mode, in which, when the tool input is in a tool input first state, the manipulator moves the tool in response to user force and torque applied to the tool by the user. In some embodiments, the manipulator can operate in a semi-autonomous mode, in which, when the teach pendant input is in a teach pendant first state, the manipulator moves the tool along a tool path. In some embodiments, the boundary handler is configured to determine whether the tool conforms to or violates the virtual boundary in response to the control system switching the operation of the manipulator from one of the manual mode and the semi-autonomous mode to the other of the manual mode and the semi-autonomous mode. In some embodiments, the control system includes a path handler configured to generate an inbound path from the tool's current position to the tool path. In some embodiments, the inbound path is generated when the manipulator switches from manual mode to semi-autonomous mode. In some embodiments, the boundary handler is configured to determine whether movement of the tool along the inbound path will maintain conformity to or violate the virtual boundary. In some embodiments, the boundary handler is configured to determine whether the movement of the tool along the introductory path will maintain compliance with or violate the virtual boundary by modeling the motion of a plurality of stereoscopic interactive features associated with the tool. In some embodiments, the boundary handler is configured to model the motion of the plurality of stereoscopic interactive features with three or more degrees of freedom. In some embodiments, the control system includes a guidance handler configured to generate user feedback to the user in response to the boundary handler determining that the tool will violate the virtual boundary if it moves along the introductory path from its current position to the tool path.

[0029] In some implementations, the control system is configured to disable autonomous compliant boundary movement of the tool in response to one or more of the following: the tool begins to stop; a predetermined time period elapses after the user input transitions from the first input state to the second input state; the linear velocity of the tool decreases below one or more thresholds; or the angular velocity of the tool decreases below one or more thresholds.

[0030] In some implementations, the control system is configured to determine whether the tool remains compliant with the virtual boundary based on the tolerances defined for the virtual boundary.

[0031] In some implementations, the control system is configured to generate a recovery tool path in the recovery mode so that the tool moves to conform to the virtual boundary.

[0032] In some implementations, the control system is configured to move the virtual boundary from its starting position in the recovery mode such that the tool returns to conformity with the virtual boundary, and then move the virtual boundary back to the starting position, thereby enabling the tool to autonomously conform to the boundary movement.

[0033] In some embodiments, in response to the user selecting the second virtual boundary, the boundary handler is configured to: if the boundary handler determines that the tool conforms to the second virtual boundary, activate the second virtual boundary and deactivate the first virtual boundary, causing the control system to switch to controlling the operation of the manipulator and the movement of the tool based on the relationship between the tool and the second virtual boundary. In some embodiments, in response to the user selecting the second virtual boundary, the boundary handler is configured to: if the boundary handler determines that the tool violates the second virtual boundary, maintain the first virtual boundary in an active state, causing the control system to continue controlling the operation of the manipulator and the movement of the tool based on the relationship between the tool and the first virtual boundary. In some embodiments, the control system is configured to generate user feedback to the user in response to the boundary handler determining that the tool violates the second virtual boundary.

[0034] In some embodiments, the control system includes a user input having a first input state and a second input state, the user input being located on the tool and configured such that the user input is actuated by the user to place the user input in the first input state and released by the user to place the user input in the second input state. In some embodiments, the control system is configured such that the user can use the virtual boundary selector to select the second virtual boundary when the user input is in the first input state. In some embodiments, the virtual boundary selector includes a second user input located on the tool. In some embodiments, the tool includes a tool driver, and the control system is configured to continue operation of the tool driver when the user selects the second virtual boundary using the virtual boundary selector. In some embodiments, the control system is configured to restrict relative movement between the tool and the first virtual boundary when the user input is in the first input state and the first virtual boundary is active. In some embodiments, the control system does this by generating a first boundary constraint using the boundary handler, and the control system is configured to limit the relative movement between the tool and the second virtual boundary by generating a second boundary constraint using the boundary handler when the user input is in the first input state and the second virtual boundary is active. In some embodiments, the control system includes a constraint solver for calculating constraint forces adapted to maintain the tool's compliance with the first virtual boundary based on the first boundary constraint or to maintain the tool's compliance with the second virtual boundary based on the second boundary constraint. In some embodiments, the control system includes a virtual simulator for simulating the dynamics of the tool in a virtual simulation based on the constraint forces and outputting a commanded posture. In some embodiments, the control system is configured to command the manipulator to move the tool based on the commanded posture. In some embodiments, the virtual boundary selector is configured to allow the user to trigger between the first and second virtual boundaries, trigger sequentially from a plurality of virtual boundaries, or select from a list of virtual boundaries.

[0035] In some embodiments, the first parameter and the second parameter are each further defined as stiffness parameters, and wherein the first stereoscopic interaction feature is more rigid than the second stereoscopic interaction feature. In some embodiments, the at least one controller is configured to: identify an event; and change at least one of the first parameter or the second parameter in response to identifying the event. In some embodiments, the first parameter and the second parameter are each further defined as geometric parameters, wherein the geometric parameters are defined as at least one of the following: size, area, volume, or shape of the stereoscopic interaction feature; and wherein the first geometric parameter is different from the second geometric parameter. In some embodiments, one or more of the first stereoscopic interaction feature and the second stereoscopic interaction feature are associated with the one or more second objects by being positioned at a distance spaced from the one or more second objects. In some embodiments, the at least one controller is configured to define one or more of the first parameter and the second parameter by being configured to: receive user input defining one or more of the first parameter and the second parameter; or automatically define one or more of the first parameter and the second parameter based on surgical information. In some implementations, to generate the response based on the interaction between at least one of the first stereoscopic interaction feature and the second stereoscopic interaction feature and the virtual boundary, the at least one controller is configured to: identify a conflict or anticipated conflict between one or more of the first stereoscopic interaction feature and the second stereoscopic interaction feature and the virtual boundary; and generate a response, the response being further defined as the at least one controller being configured to perform one or more of the following: adjust the pose of the one or more second objects; and generate an alarm or notification regarding the conflict or anticipated conflict. In some implementations, the one or more second objects are further defined as a single second object, wherein the first stereoscopic interaction feature and the second stereoscopic interaction feature are associated with the single second object. In some implementations, the one or more second objects are further defined as separate second objects, wherein the first stereoscopic interaction feature is associated with one of the second objects and the second stereoscopic interaction feature is associated with the other of the second objects.In some embodiments, the one or more first objects are further defined as bones; the virtual boundary is further defined as a cutting boundary, the cutting boundary depicting an anatomical structure to be removed from an anatomy that should not be removed; the one or more second objects are further defined as a rotary cutting tool; the first stereotactic interactive feature and the second stereotactic interactive feature are associated with the rotary cutting tool and located at different positions relative to the rotary cutting tool; and the first parameter and the second parameter are each further defined as stiffness parameters, wherein the first stereotactic interactive feature is more rigid than the second stereotactic interactive feature. In some embodiments, the one or more first objects include one or more of the following: the patient's anatomical structure; any part of the kinematic chain forming the robotic manipulator, including a base, links, joints, end effectors, tools, sterile adapters, power applicators; handheld tools or devices; an operating table; a head-mounted device; a handheld display device or tablet computer; a surgical tracker; a retractor; an imaging device; a person in the operating room; and the one or more second objects include one or more of the following: the patient's anatomical structure; any part of the kinematic chain forming the robotic manipulator, including a base, links, joints, end effectors, tools, sterile adapters, power applicators; handheld tools or devices; an operating table; a head-mounted device; a handheld display device or tablet computer; a surgical tracker; a retractor; an imaging device; a person in the operating room. Attached Figure Description

[0036] The advantages of this disclosure will be readily understood, as they can be better appreciated by referring to the following detailed description when considered in conjunction with the accompanying drawings.

[0037] Figure 1 It is a perspective view of the surgical system.

[0038] Figure 2 It is a block diagram of the control system used to control the surgical system.

[0039] Figure 3 It is a functional block diagram of a software program.

[0040] Figure 4 The output of the boundary generator used for acetabular surgery is shown.

[0041] Figure 5 The output of the path generator used for acetabular surgery is shown.

[0042] Figure 6 The output of the boundary generator used for vertebral surgery is shown.

[0043] Figure 7 The output of the boundary generator used in femoral surgery is shown.

[0044] Figure 8 The output of the path generator used for femoral surgery is shown.

[0045] Figure 9 The output of the boundary generator used in femoral surgery is shown.

[0046] Figure 10 This is a diagram of virtual constraints.

[0047] Figure 11 It is a block diagram of a module that can be operated by a control system.

[0048] Figure 12 The sample constraint equations are shown.

[0049] Figure 13 and Figure 14 The sample forward dynamics algorithm used to perform the virtual simulation is shown.

[0050] Figure 15 A set of exemplary steps performed by the control system to switch between boundary enable mode and boundary disable mode is shown.

[0051] Figure 16 An exemplary set of steps is shown, executed by the control system, to solve for constraints, perform forward dynamics, and determine command attitude.

[0052] Figure 16A An exemplary set of steps is shown, executed by the control system, to solve for constraints, perform forward dynamics, and determine command attitude.

[0053] Figures 17A to 17D This illustrates how the control system resolves conflicts when determining the command attitude.

[0054] Figures 18A to 18G The movement of tools and virtual boundaries is shown during femoral surgery.

[0055] Figure 19 The switching between the first virtual boundary and the second virtual boundary is shown.

[0056] Figure 20 Several stereoscopic interaction features associated with components of the robotic surgical system are shown. Detailed Implementation

[0057] I. Overview

[0058] refer to Figure 1 The surgical system 10 is shown. System 10 can be used to treat a target site or anatomical volume A of a patient 12, such as bone or soft tissue. Figure 1 In the middle, patient 12 is undergoing surgery. Figure 1The anatomical structures included in the procedure are the femur (F), pelvis (PEL), and tibia (T) of patient 12. Surgical procedures may involve tissue removal or other forms of treatment. Treatment may include cutting, coagulation, damaging tissue, other in-situ tissue treatments, etc. In some examples, surgical procedures include partial or total knee or hip replacement surgery, shoulder replacement surgery, spinal surgery, or ankle surgery. In some examples, system 10 is designed to cut away material to be replaced by a surgical implant, such as hip and knee implants, including single-chamber, dual-chamber, multi-chamber, or total knee implants, acetabular cup implants, femoral stem implants, screws, anchors, other fasteners, etc. Some of these types of implants are illustrated in U.S. Patent Application Publication No. 2012 / 0330429 entitled “Prosthetic Implant and Method of Implantation,” the disclosure of which is hereby incorporated herein by reference. The system 10 and techniques disclosed herein can be used to perform other surgical or non-surgical procedures, or for industrial or other applications.

[0059] System 10 includes a robotic manipulator 14, also known as a surgical robot. Manipulator 14 has a base 16 and a plurality of links 18. A manipulator trolley 17 supports manipulator 14, such that manipulator 14 is secured to manipulator trolley 17. The links 18 together form one or more arms (e.g., robotic arms) of manipulator 14. Manipulator 14 may have a tandem arm configuration (e.g.,... Figure 1 (as shown), parallel arm configuration, or any other suitable manipulator configuration. In other examples, more than one manipulator 14 may be used in a multi-arm configuration.

[0060] exist Figure 1 In the example shown, the manipulator 14 includes multiple joints J and multiple joint encoders 19 located at the joints J for determining the position data of the joints J. For simplicity, Figure 1 Only one joint encoder 19 is shown, but other joint encoders 19 may be shown similarly. According to one example, the manipulator 14 has six joints J1 to J6 that implement at least six degrees of freedom (DOF) of the manipulator 14. However, the manipulator 14 can have any number of degrees of freedom and can have any suitable number of joints J and can have redundant joints.

[0061] Manipulator 14 does not require joint encoder 19, but may alternatively or additionally utilize motor encoders present on the motors at each joint J. Furthermore, manipulator 14 does not require rotary joints, but may alternatively or additionally utilize one or more prismatic joints. Any suitable combination of joint types is contemplated.

[0062] The base 16 of the manipulator 14 is typically part of the manipulator 14, providing a fixed reference coordinate system for other components of the manipulator 14 or generally for the system 10. Typically, the origin of the manipulator coordinate system MNPL is defined at a fixed reference point on the base 16. The base 16 can be defined with respect to any suitable part of the manipulator 14, such as one or more links 18. Alternatively or additionally, the base 16 can be defined with respect to the manipulator trolley 17, such as being defined at the location where the manipulator 14 is physically attached to the trolley 17. In one example, the base 16 is defined at the intersection of the axes of joints J1 and J2. Thus, although joints J1 and J2 are moving parts in reality, the intersection of the axes of joints J1 and J2 remains a virtual fixed reference orientation, providing a fixed position and orientation reference and not moving relative to the manipulator 14 and / or the manipulator trolley 17.

[0063] In some examples, the manipulator 14 may be a handheld manipulator, wherein the base 16 is the base portion of the tool (e.g., the portion held by the user's hand) and the tool tip is movable relative to the base portion. The base portion has a tracked reference coordinate system, and the tool tip has a tool tip coordinate system calculated relative to the reference coordinate system (e.g., calculated via a motor and / or joint encoder and positive kinematics). The movement of the tool tip can be controlled to follow a path, as its orientation relative to the path can be determined. Such a manipulator 14 is illustrated in U.S. Patent No. 9,707,043, filed August 31, 2012, entitled "Surgical Instrument Including Housing, A Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing," which is hereby incorporated herein by reference.

[0064] Manipulator 14 and / or manipulator trolley 17 house manipulator controller 26 or other types of control units. Manipulator controller 26 may include one or more computers, or any other suitable form of controller that directs the movement of manipulator 14. Manipulator controller 26 may have a central processing unit (CPU) and / or other processors, memory (not shown), and storage devices (not shown). Manipulator controller 26 is loaded with software as described below. Processor may include one or more processors for controlling the operation of manipulator 14. Processor may be any type of microprocessor, multiprocessor, and / or multicore processing system. Manipulator controller 26 may additionally or alternatively include one or more microcontrollers, field-programmable gate arrays, system-on-a-chip, discrete circuitry, and / or other suitable hardware, software, or firmware capable of performing the functions described herein. The term processor is not intended to limit any embodiment to a single processor. Manipulator 14 may also include a user interface (UI) with one or more displays and / or input devices (e.g., buttons, sensors, switches, keyboards, mice, microphones (voice-activated), gesture controls, touchscreens, joysticks, foot pedals, etc.).

[0065] Surgical instrument 20 is coupled to manipulator 14 and is movable relative to base 16 to interact with anatomical structures in certain modes. In some embodiments, tool 20 is an end effector 22 supported by manipulator 14 or forms part of such end effector. Tool 20 can be gripped by a user. One possible arrangement of manipulator 14 and tool 20 is described in U.S. Patent No. 9,119,655, filed August 2, 2013, entitled “Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes,” the disclosure of which is hereby incorporated by reference. Manipulator 14 and tool 20 can be arranged in alternative configurations. Tool 20 can be similar to the tool shown in U.S. Patent Application Publication No. 2014 / 0276949, filed March 15, 2014, entitled “End Effector of a Surgical Robotic Manipulator,” which is hereby incorporated by reference.

[0066] Tool 20 includes an energy applicator 24 designed to contact the tissue of patient 12 at a target site. In one example, the energy applicator 24 is a file 25. The file 25 may be generally spherical and include a spherical center, radius (r), and diameter. Alternatively, the energy applicator 24 may be a drill bit, a saw blade 27 (see...). Figure 1Alternative tools, ultrasonic vibrating tips, etc. Tool 20 and / or energy applicator 24 may include any geometric features such as perimeter, circumference, radius, diameter, width, length, volume, area, surface / plane, range of motion envelope (along any one or more axes), etc. Geometric features can be considered to determine how to position tool 20 relative to tissue at the target site to perform the desired treatment. In some embodiments described herein, for convenience and ease of illustration, a ball file with a tool center point (TCP) and a sagittal saw blade with a TCP will be described, but it is not intended to limit tool 20 to any particular form.

[0067] Tool 20 may include tool controller 21 to control the operation of tool 20, such as controlling the power of tool 20 (e.g., controlling the power of tool drives (such as a rotary motor of tool 20)), controlling the movement of tool 20, controlling flushing / suction of tool 20, etc. Tool controller 21 may communicate with manipulator controller 26 or other components. Tool 20 may also include a user interface (UI) having one or more displays and / or input devices (e.g., buttons, triggers, sensors, switches, keyboards, mice, microphones (voice-activated), gesture controls, touchscreens, joysticks, foot pedals, etc.), said displays and / or input devices being coupled to tool controller 21, manipulator controller 26, and / or other controllers described herein. Manipulator controller 26 controls (e.g., TCP) the state (e.g., position and / or orientation) of tool 20 relative to a coordinate system such as manipulator coordinate system MNPL. Manipulator controller 26 may control the velocity (linear or angular velocity), acceleration, or other kinematic derivatives of tool 20.

[0068] In one example, the tool center point (TCP) is a predetermined reference point defined at the energy applicator 24. The TCP has a known or computable (i.e., not necessarily static) attitude relative to other coordinate systems. The geometry of the energy applicator 24 is known in or defined relative to the TCP coordinate system. The TCP can be located at the center of the ball of the file 25 of the tool 20 or at the distal end of the saw blade 27, such that only one point is tracked. The TCP can be defined in various ways depending on the configuration of the energy applicator 24. The manipulator 14 can employ a joint / motor encoder, or any other non-encoder position sensing method, to enable the determination of the TCP's attitude. The manipulator 14 can use joint measurements to determine the TCP attitude and / or can employ techniques to directly measure the TCP attitude. Control of the tool 20 is not limited to the center point. For example, any suitable primitive, mesh, etc., can be used to represent the tool 20.

[0069] System 10 also includes a navigation system 32. An example of the navigation system 32 is described in U.S. Patent No. 9,008,757, filed September 24, 2013, entitled "Navigation System Including Optical and Non-Optical Sensors," which is hereby incorporated herein by reference. The navigation system 32 tracks the movement of various objects. Such objects include, for example, manipulators 14, tools 20, and anatomical structures, such as the femur (F), pelvis (PEL), and tibia (T). The navigation system 32 tracks these objects to collect state information about each object with respect to the (navigation) locator coordinate system LCLZ. Coordinates in the locator coordinate system LCLZ can be transformed using transformations to the manipulator coordinate system MNPL, other coordinate systems, and / or vice versa.

[0070] The navigation system 32 includes a cart assembly 34 housing a navigation controller 36 and / or other types of control units. A navigation user interface (UI) communicates operationally with the navigation controller 36. The navigation user interface includes one or more displays 38. The navigation system 32 is capable of displaying a graphical representation of the relative status of the tracked object to a user using the one or more displays 38. The navigation user interface (UI) also includes one or more input devices for inputting information to the navigation controller 36 or otherwise selecting / controlling certain aspects of the navigation controller 36. Such input devices include interactive touchscreen displays. However, input devices may include any one or more of buttons, a keyboard, a mouse, a microphone (voice-activated), gesture controls, foot pedals, etc.

[0071] The navigation system 32 also includes a navigation locator 44 coupled to the navigation controller 36. In one example, the locator 44 is an optical locator and includes a camera unit 46. The camera unit 46 has a housing 48 that houses one or more optical sensors 50. The locator 44 may include its own locator controller 49 and may also include a camera VC.

[0072] Navigation system 32 includes one or more trackers. In one example, the trackers include a pointer tracker PT, one or more manipulator trackers 52A, 52B, a first patient tracker 54, a second patient tracker 55, and a third patient tracker 56. Figure 1In the example shown, the manipulator tracker is securely attached to tool 20 (i.e., tracker 52A), the first patient tracker 54 is securely attached to the femur F of patient 12, the second patient tracker 55 is securely attached to the pelvis PEL of patient 12, and the third patient tracker 56 is securely fixed to the tibia T of patient 12. In this example, patient trackers 54, 55, and 56 are securely fixed to bone segments. The pointer tracker PT is securely fixed to the pointer P, which is used to align anatomical structures to the locator coordinate system LCLZ. Manipulator trackers 52A and 52B can be fixed to any suitable component of manipulator 14 other than or different from tool 20, such as base 16 (i.e., tracker 52B) or any one or more links 18 of manipulator 14. Trackers 52A, 52B, 54, 55, 56, and PT can be fixed to their respective components in any suitable manner. For example, a tracker can be rigidly fixed, flexibly connected (fiber optic), or not physically connected at all (ultrasound), as long as there is a suitable (supplementary) way to determine the relationship (measurement) between the corresponding tracker and the object it is associated with.

[0073] Any one or more of the trackers may include an active marker 58. The active marker 58 may include a light-emitting diode (LED). Alternatively, trackers 52A, 52B, 54, 55, 56, and PT may have passive markers, such as reflectors, that reflect light emitted from camera unit 46. Other suitable markers not specifically described herein may be utilized.

[0074] Positioner 44 tracks trackers 52A, 52B, 54, 55, 56, and PT to determine the state of each of the trackers 52A, 52B, 54, 55, 56, and PT, each state corresponding to the state of the object attached to the tracker. Positioner 44 can perform known triangulation techniques to determine the state of trackers 52A, 52B, 54, 55, 56, and PT and the associated object. Positioner 44 provides the states of trackers 52A, 52B, 54, 55, 56, and PT to navigation controller 36. In one example, navigation controller 36 determines the states of trackers 52A, 52B, 54, 55, 56, and PT and transmits them to manipulator controller 26. As used herein, the state of an object includes, but is not limited to, data or equivalents / derivatives of the position and / or orientation of the tracked object. For example, the state can be the attitude of the object and can include linear velocity data and / or angular velocity data, etc.

[0075] The navigation controller 36 may include one or more computers, or any other suitable form of controller. The navigation controller 36 has a central processing unit (CPU) and / or other processor, memory (not shown), and storage devices (not shown). The processor may be any type of processor, microprocessor, or multiprocessor system. The navigation controller 36 is loaded with software. For example, the software converts signals received from the locator 44 into data representing the position and orientation of the tracked object. The navigation controller 36 may additionally or alternatively include one or more microcontrollers, field-programmable gate arrays, systems-on-a-chip, discrete circuitry, and / or other suitable hardware, software, or firmware capable of performing the functions described herein. The term processor is not intended to limit any implementation to a single processor.

[0076] Although one example of navigation system 32 is shown employing triangulation to determine the state of an object, navigation system 32 can have any other suitable configuration for tracking manipulator 14, tool 20, and / or patient 12. In another example, navigation system 32 and / or locator 44 are ultrasound-based. For example, navigation system 32 may include an ultrasound imaging device coupled to navigation controller 36. The ultrasound imaging device images any of the aforementioned objects (e.g., manipulator 14, tool 20, and / or patient 12) and generates a state signal to navigation controller 36 based on the ultrasound images. The ultrasound images can be 2D, 3D, or a combination of both. Navigation controller 36 can process the images in near real-time to determine the state of the object. The ultrasound imaging device can have any suitable configuration and can differ from... Figure 1 The camera unit 46 shown is shown.

[0077] In another example, navigation system 32 and / or locator 44 are radio frequency (RF) based. For example, navigation system 32 may include an RF transceiver coupled to navigation controller 36. Manipulator 14, tool 20, and / or patient 12 may include RF transmitters or repeaters attached thereto. The RF transmitter or repeater may be passive or actively activated. The RF transceiver transmits RF tracking signals and generates status signals to navigation controller 36 based on the RF signals received from the RF transmitter. Navigation controller 36 may analyze the received RF signals to correlate with relevant statuses. The RF signals may have any suitable frequency. The RF transceiver may be positioned at any suitable location to effectively track objects using the RF signals. Furthermore, the RF transmitter or repeater may have [unclear meaning - possibly related to a specific technology or feature]. Figure 1 The trackers 52A, 52B, 54, 55, 56, and PT shown are significantly different from any suitable structural configuration.

[0078] In yet another example, navigation system 32 and / or locator 44 are electromagnetic. For example, navigation system 32 may include an EM transceiver coupled to navigation controller 36. Manipulator 14, tool 20, and / or patient 12 may include EM components attached thereto, such as any suitable magnetic tracker, electromagnetic tracker, inductive tracker, etc. The tracker may be passive or actively actuated. The EM transceiver generates an EM field and generates a status signal to navigation controller 36 based on the EM signals received from the tracker. Navigation controller 36 can analyze the received EM signals to correlate with the relevant status. Similarly, examples of such navigation systems 32 may have... Figure 1 The navigation system 32 shown has different structural configurations.

[0079] Navigation system 32 may have any other suitable components or structures not specifically listed herein. Furthermore, any of the techniques, methods, and / or components described above with respect to the illustrated navigation system 32 may be implemented or provided with respect to any of the other examples of navigation system 32 described herein. For example, navigation system 32 may utilize only inertial tracking or any combination of tracking techniques, and may additionally or alternatively include fiber-optic tracking, machine vision tracking, etc.

[0080] refer to Figure 2 System 10 includes a control system 60, which includes a manipulator controller 26, a navigation controller 36, a tool controller 21, and other components. Control system 60 also includes... Figure 3 The software program and software module shown are one or more software programs. The software module may be a portion of one or more programs that operate on the manipulator controller 26, navigation controller 36, tool controller 21, or any combination thereof to process data to assist in controlling the system 10. The software program and / or module includes computer-readable instructions stored in a non-transitory memory 64 on the manipulator controller 26, navigation controller 36, tool controller 21, or any combination thereof, for execution by one or more processors 70 of the controllers 21, 26, 36. The memory 64 may be any suitable memory configuration, such as RAM, non-volatile memory, etc., and may be implemented locally or from a remote database. Additionally, software modules for prompting and / or communicating with the user may form part of one or more programs and may include instructions stored in the memory 64 on the manipulator controller 26, navigation controller 36, tool controller 21, or any combination thereof. The user can interact with any of the input devices of the navigation user interface (UI) or other user interface (UI) to communicate with the software module. The user interface software may run on a device separate from the manipulator controller 26, navigation controller 36, and / or tool controller 21.

[0081] The control system 60 may include any suitable configuration of input, output, and processing means adapted to perform the functions and methods described herein. The control system 60 may include manipulator controller 26, navigation controller 36, or tool controller 21, or any combination thereof, or may include only one of these controllers. These controllers can be transmitted via, for example... Figure 2 The wired bus or communication network shown communicates wirelessly or otherwise. Control system 60 may also be referred to as a controller. Control system 60 may include one or more microcontrollers, field-programmable gate arrays, systems-on-a-chip, discrete circuits, sensors, displays, user interfaces, indicators, and / or other suitable hardware, software, or firmware capable of performing the functions described herein.

[0082] II. Virtual Boundaries and Toolpaths

[0083] refer to Figure 3 The software used by the control system 60 includes a boundary generator 66. For example... Figure 4 As shown, boundary generator 66 is a software program or module that generates a virtual boundary 71 for constraining the movement and / or operation of tool 20. Virtual boundary 71 can be one-dimensional, two-dimensional, or three-dimensional, and can include points, lines, axes, trajectories, planes, volumes, triangular meshes, etc. Virtual boundary 71 can have simple shapes or complex geometries. In some embodiments, virtual boundary 71 is a surface defined by a triangular mesh. Virtual boundary 71 can also be referred to as a virtual object. For example, virtual boundary 71 can be a keep-in boundary where tool 20 is desired to remain within the volume defined by the boundary, or a keep-out boundary where tool 20 is desired to remain outside the volume defined by the boundary. Virtual boundary 71 can also be a keep-on boundary where tool 20 is desired to remain on a point, line, plane, surface, etc., defining the boundary, or a keep-off boundary where tool 20 is desired to remain away from a point, line, plane, surface, etc., defining the boundary. Virtual boundary 71 can also be a combination of these types of boundaries. Other types of navigation systems are also envisioned.

[0084] The virtual boundary 71 can be defined with respect to an anatomical model AM, such as a 3D skeletal model. Because the anatomical model AM is mapped to the patient's anatomy via registration or other processes, the anatomical model AM is associated with the real patient's anatomy. Figure 4 In the example, the virtual boundary 71 comprises a generally spherical mesh that substantially surrounds the acetabulum, having an inlet portion 71a (opening) that provides access to the acetabulum. The inlet portion has a funnel or conical shape. This virtual boundary 71 is associated with a 3D model of the acetabulum.

[0085] The anatomical model AM and the associated virtual boundary 71 are registered to one or more patient trackers 54, 55, 56. Therefore, the anatomical model AM (and the associated real patient anatomy) and the virtual boundary 71 fixed to the anatomical model AM can be tracked by the patient trackers 54, 55, 56. The virtual boundary 71 can be implant-specific, such as defined based on the size, shape, volume, etc. of the implant, and / or patient-specific, such as defined based on the patient's anatomy. The virtual boundary 71 can be a boundary created preoperatively, intraoperatively, or a combination thereof. In other words, the virtual boundary 71 can be defined before the start of surgery, during surgery (including during tissue removal), or a combination thereof. In any case, the control system 60 obtains the virtual boundary 71 by storing / retrieving the virtual boundary 71 from memory, obtaining the virtual boundary 71 from memory, creating the virtual boundary 71 preoperatively, creating the virtual boundary 71 intraoperatively, etc.

[0086] Manipulator controller 26 and / or navigation controller 36 track the state of tool 20 relative to one or more virtual boundaries 71. In one example, the state of TCP is measured relative to virtual boundaries 71 to determine, via virtual simulation, the forces to be applied to a virtual rigid body model of tool 20 such that tool 20 remains compliant with the virtual boundaries. Tool 20 remains compliant with virtual boundaries 71 by maintaining a desired relationship with them (e.g., not moving beyond the virtual boundaries) and / or maintaining a desired position and / or orientation relative to virtual boundaries 71. It should be understood that predefined, configurable tolerances can also be established for any virtual boundary 71 such that some penetration or deviation of tool 20 from virtual boundaries 71 is considered a violation of virtual boundaries 71. For example, a small tolerance of 0.1 mm can be set such that as long as tool 20 penetrates into virtual boundaries 71 by less than 0.1 mm, tool 20 remains compliant with virtual boundaries 71 (i.e., this is not considered a violation of virtual boundaries 71). In some cases, larger tolerances may be appropriate. The results of the virtual simulation are commanded to manipulator 14. The control system 60 controls / positions the manipulator 14 in a manner that simulates the response of a physical handheld device in the presence of physical boundaries / obstacles. A boundary generator 66 may be implemented on the manipulator controller 26. Alternatively, the boundary generator 66 may be implemented on other components, such as the navigation controller 36.

[0087] refer to Figure 3 and Figure 5Path generator 68 is another software program or module run by control system 60. In one example, path generator 68 is run by manipulator controller 26. Path generator 68 generates tool paths TP for tool 20 to traverse, such as tool paths for removing segments of anatomical structures to receive implants. Tool path TP may include multiple path segments PS, or may include a single path segment PS. Path segments PS may be straight segments, curved segments, combinations thereof, etc. Tool path TP may also be defined with respect to anatomical model AM and may be tracked via one or more of patient trackers 54, 55, 56. Tool path TP may be implant-specific, such as defined based on the size, shape, volume, etc. of the implant, and / or patient-specific, such as defined based on the patient's anatomy. Tool path TP may be a 3D path along which the TCP of tool 20 is intended to move during certain operations of system 10.

[0088] In one version described herein, the tool path TP is defined as a tissue removal path; however, in other versions, the tool path TP may be used for treatments other than tissue removal. An example of a tissue removal path described herein includes a milling path 72. It should be understood that the term "milling path" generally refers to a path by which the tool 20 mills anatomical structures near the target site, and is not intended to require the tool 20 to operatively mill anatomical structures for the entire duration of the path. For example, a milling path 72 may include portions or segments of the tool 20 that move from one location to another without milling. Additionally, other forms of tissue removal, such as tissue ablation, may be employed along the milling path 72. The milling path 72 may be a predefined path created preoperatively, intraoperatively, or in combination thereof. In other words, the milling path 72 may be defined before the commencement of surgery, during surgery (including during tissue removal), or in combination thereof. In any case, the control system 60 obtains the milling path 72 by storing / retrieving the milling path 72 from memory, obtaining the milling path 72 from memory, creating the milling path 72 preoperatively, creating the milling path 72 intraoperatively, etc. The milling path 72 can have any suitable shape or combination of shapes, such as circular, spiral / helical, straight, curved, or combinations thereof. Figure 5 The milling path 72 shown is intended to remove material from the acetabulum as the tool 20 traverses it to make room for the acetabular cup implant to be fitted into the acetabulum.

[0089] Exemplary virtual boundary 71 and / or milling path 72 in Figures 4 to 9 The specific shape and arrangement of the virtual boundary 71 and / or milling path 72 shown are for illustrative purposes. Other shapes and arrangements are also possible. As previously stated, Figure 4 and Figure 5Virtual boundaries 71 and milling paths 72 are shown, which are generated for use in surgical procedures in which the acetabulum is being prepared (e.g., milled) to receive an acetabular cup implant.

[0090] Figure 6 A virtual boundary 71 is shown, comprising a generally spherical mesh substantially surrounding the vertebral body, having an entrance portion 71a (opening) providing access to the vertebral body. The entrance portion 71a has a funnel or conical shape and extends into a cylindrical portion 71b. This virtual boundary 71 is associated with a 3D model of the vertebral body. This virtual boundary 71 is generated for use in surgical procedures in which the vertebral body is being prepared (e.g., milled) to receive screws or other implants.

[0091] Figure 7 A virtual boundary 71 is shown, comprising a generally spherical mesh substantially surrounding one end of the femur, having an entrance portion 71a (opening) providing access to the femur. The entrance portion 71a has a funnel or conical shape and extends to a tube portion 71b extending downward along the medullary canal of the femur. This virtual boundary 71 is associated with a 3D model of the femur. Figure 8 A milling path 72 is shown, which is defined to allow tool 20 to remove material from the femur to make room for the femoral stem implant. Therefore, Figure 7 and Figure 8 Virtual boundaries 71 and milling paths 72 are shown, which are generated for use in a surgical procedure in which the femur F is being prepared (e.g., milled) to receive a femoral stem implant.

[0092] Figure 9 A series of virtual boundaries 71 are shown for five cutting planes passing through the distal end of the femur. Figure 9 Each of the virtual boundaries 71 comprises a generally spherical mesh substantially surrounding the distal end of the femur, having an entry portion 71a (opening) providing access to the femur. The entry portion 71a continues into a cutting groove 71b defined along one of five cutting planes 73a to 73e. These virtual boundaries 71 are generated for use in surgical procedures in which the femur F is being prepared (e.g., via planar resection) to receive a total knee implant. Other types / shapes of virtual boundaries and / or milling paths 72 are envisioned for other surgical procedures.

[0093] An example of a system and method for generating virtual boundary 71 and / or milling path 72 is described in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is hereby incorporated herein by reference. In some examples, virtual boundary 71 and / or milling path 72 may be generated offline, rather than on manipulator controller 26 or navigation controller 36. Thereafter, virtual boundary 71 and / or milling path 72 may be utilized by manipulator controller 26 at runtime.

[0094] Return to reference Figure 3 Two additional software programs or modules run on the manipulator controller 26 and / or the navigation controller 36. One software module performs behavior control 74. Behavior control 74 is the process of calculating data indicating the next commanded position and / or orientation (e.g., attitude) of tool 20. In some cases, only the position of TCP is output from behavior control 74, while in other cases, the position and orientation of tool 20 are output. Outputs from boundary generator 66, path generator 68, and one or more sensors (such as force / torque sensors S) can be fed as inputs into behavior control 74 to determine the next commanded position and / or orientation of tool 20. Behavior control 74 can process these inputs, as well as one or more virtual constraints described further below, to determine the commanded attitude.

[0095] The second software module executes motion control 76. One aspect of motion control is the control of manipulator 14. Motion controller 76 receives data from behavior controller 74 defining the next command posture. Based on this data, motion controller 76 determines the next position of the joint angles of joint J of manipulator 14 (e.g., via inverse kinematics and Jacobian matrix calculators) so that manipulator 14 can position tool 20 according to the command of behavior control 74 (e.g., in the commanded posture). In other words, motion controller 76 processes the commanded posture, which can be defined in Cartesian space, into joint angles of manipulator 14 so that manipulator controller 26 can accordingly command the joint motors to move joint J of manipulator 14 to the commanded joint angle corresponding to the commanded posture of tool 20. In one version, motion controller 76 adjusts the joint angle of each joint J and continuously adjusts the torque output of each joint motor to ensure, as closely as possible, that the joint motor drives the associated joint J to the commanded joint angle.

[0096] Boundary generator 66, path generator 68, behavior control 74, and motion control 76 may be subsets of software program 78. Alternatively, each may be a software program that operates independently, either alone or in any combination thereof. The term "software program" is used herein to describe computer-executable instructions configured to perform various capabilities of the described technical solutions. For simplicity, the term "software program" is intended to include at least one or more of boundary generator 66, path generator 68, behavior control 74, and / or motion control 76. Software program 78 may be implemented on manipulator controller 26, navigation controller 36, or any combination thereof, or may be implemented by control system 60 in any suitable manner.

[0097] A clinical application 80 may be provided to handle user interactions. The clinical application 80 handles many aspects of the user interaction and coordinates the surgical workflow, including preoperative planning, implant placement, registration, bone preparation visualization, and postoperative assessment of implant fit. The clinical application 80 is configured to output to a display 38. The clinical application 80 may run on its own separate processor or may run in conjunction with a navigation controller 36. In one example, after the user sets the implant placement, the clinical application 80 interacts with a boundary generator 66 and / or a path generator 68, and then sends the virtual boundary 71 and / or tool path TP returned by the boundary generator 66 and / or path generator 68 to a manipulator controller 26 for execution. The manipulator controller 26 executes the tool path TP as described herein. The manipulator controller 26 may additionally create certain segments (e.g., import segments) at the start or resumption of processing to smoothly return to the generated tool path TP. The manipulator controller 26 may also process the virtual boundary 71 to generate corresponding virtual constraints, as further described below.

[0098] III. Operating Mode

[0099] System 10 can operate in manual mode, as described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. Here, the user manually guides tool 20 and its energy applicator 24, and manipulator 14 performs movement of the tool and its energy applicator at a target location. The user physically contacts tool 20 to move tool 20 in manual mode. In one version, manipulator 14 monitors the forces and torques applied by the user to tool 20 to position tool 20. For example, manipulator 14 may include one or more sensors (e.g., force / torque sensors S) that detect and measure the forces and torques applied by the user to tool 20 and generate corresponding inputs (e.g., one or more corresponding input / output signals) used by control system 60. The forces and torques applied by the user at least partially define the external force F used to determine how to move tool 20 in manual mode. ext External force Fext This can include forces and torques other than those applied by the user, such as gravity-compensating forces, reverse driving forces, etc., as described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. Therefore, the forces and torques applied by the user at least partially define the external force F. ext Furthermore, in some cases, the external force F affecting the overall movement of tool 20 can be fully defined in manual mode. ext .

[0100] The force / torque sensor S may include a 6-DOF force / torque sensor, such as that disclosed, for example, in U.S. Patent No. 9,119,655, which is incorporated herein by reference. The force / torque sensor S may form part of the tool 20, the manipulator 14, or both. The force / torque sensor S may form part of the interface between the tool 20 and the manipulator 14, or may be placed in any suitable location such that forces and torques applied to the tool 20 by the user are transmitted to the force / torque sensor S. The manipulator controller 26 and / or the navigation controller 36 receive inputs (e.g., signals) from the force / torque sensor S. In response to forces and torques applied by the user, the manipulator 14 moves the tool 20 in a manner that simulates the movement that would occur based on the forces and torques applied by the user.

[0101] The movement of tool 20 in manual mode can also be constrained (e.g., restricted) relative to one or more virtual boundaries 71 generated by boundary generator 66. In some versions, measurements taken by force / torque sensor S are transformed from the force / torque coordinate system FT of force / torque sensor S to another coordinate system, such as virtual mass coordinate system VM, in which a virtual simulation is performed on the virtual rigid body model of tool 20, so that forces and torques can be virtually applied to the virtual rigid body in the virtual simulation to ultimately determine how these forces and torques (and other inputs) will affect the movement of the virtual rigid body, as described below.

[0102] System 10 can also operate in a semi-autonomous mode, in which manipulator 14 autonomously moves tool 20 along milling path 72 (e.g., the active joint J of manipulator 14 operates to move tool 20 without the user applying force / torque to tool 20). Examples of operation in semi-autonomous mode are also described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. In some embodiments, when manipulator 14 operates in semi-autonomous mode, manipulator 14 is able to move tool 20 without user assistance. No user assistance may mean that the user does not physically touch tool 20 to move tool 20. Instead, the user can use a remote control RC (see [link to remote control]) that communicates with manipulator 14 (e.g., wired or wireless). Figure 1The remote control (RC) is used to control the start and stop of movement. The RC can be a teach pendant held in the user's hand or otherwise grasped or supported by the user. The teach pendant can have any suitable size and / or shape to allow the user to hold and operate it. In some versions, the teach pendant is a portable electronic device.

[0103] The user interface (UI) of tool 20 and the remote controller (RC) may each include one or more user input devices (e.g., buttons, sensors, switches, keyboards, mice, microphones (voice-activated), gesture controls, touchscreens, joysticks, foot pedals, etc.) coupled to tool controller 21, manipulator controller 26, and / or navigation controller 36 to control the operation of manipulator 14. For example, one of the user input devices on the user interface (UI) of tool 20 may be a tool input 82 (e.g., a switch or other form of user input device) having a first input state and a second input state (see...). Figure 1 The tool input 82 can be actuated by the user (e.g., pressed and held) to be placed in a first input state, and can be released to be placed in a second input state. The tool 20 may have a grip 83 on which the tool input 82 is located. In some versions, the tool input 82 is a presence detector that detects the presence of the user's hand, such as a momentary contact switch that toggles between on / off states, a capacitive sensor, an optical sensor, etc. The tool input 82 is thus configured such that the first input state indicates that the user is actively engaging the tool 20 and the second input state indicates that the user has released the tool 20.

[0104] One of the input devices on the remote controller RC can be a teach pendant input RC1 (e.g., a switch or other form of user input device) having a first input state and a second input state. Similar to tool input 82, teach pendant input RC1 can be actuated by the user (e.g., pressed and held) to be placed in the first input state and can be released to be placed in the second input state. When teach pendant input RC1 is actuated, auxiliary teach pendant inputs RC2, RC3 (e.g., switches or other form of user input devices) on the remote controller RC can then cause movement of manipulator 14 by controlling the feed rate of manipulator 14 (e.g., the speed at which manipulator 14 moves tool 20). For example, in semi-autonomous mode, auxiliary teach pendant input RC2 may slow down the feed rate, and auxiliary teach pendant input RC3 may increase the feed rate. Such a remote controller RC embodied as a user teach pendant is disclosed in U.S. Patent No. 10,117,713 to Moctezuma de LaBarrera et al. entitled "Robotic Systems and Methods for Controlling a Tool Removing Material from a Workpiece," which is hereby incorporated herein by reference.

[0105] The tool input 82 and the teach pendant input RC1 can be in the form of continuously activated devices, i.e., they must be continuously actuated to allow the tool 20 to move in manual or semi-autonomous mode depending on which user input is actuated. For example, when the user continuously actuates the tool input 82 and activates manual mode, the manipulator 14 will move in response to the input force and torque applied by the user, and the control system 60 will enforce the virtual boundary 71 to protect the patient's anatomy. When the tool input 82 is released, the input from the force / torque sensor S can be disabled, so that the manipulator 14 no longer responds to the force and torque applied to the tool 20 by the user.

[0106] Under normal operating conditions, when tool input 82 is in the first input state (e.g., actuated), and regardless of the state of teach pendant input RC1 (tool input 82 has priority), manipulator 14 operates in manual mode, and control system 60 operates in boundary-enabled state to maintain tool 20 conformity to the virtual boundary 71 (or multiple boundaries) then employed by control system 60. Similarly, under normal operating conditions, when tool input 82 is in the second input state (e.g., released) and teach pendant input RC1 is in the first input state (e.g., actuated), manipulator 14 operates in semi-autonomous mode, and control system 60 operates in boundary-enabled state to maintain tool 20 conformity to virtual boundary 71 and tool path TP.

[0107] In the boundary-enabled state, the control system 60 controls the operation of the manipulator 14 to maintain the tool 20 in compliance with the virtual boundary 71. As a result, in the boundary-enabled state, the control system 60 is able to control the manipulator 14 to induce autonomous movement of the tool 20, such that if the virtual boundary 71 moves relative to the tool 20 in a manner that would otherwise cause the tool 20 to violate the virtual boundary 71, the control system 60 can compensate for such movement of the virtual boundary 71 by moving the tool 20. This autonomous movement can be referred to as autonomous compliance movement. For example, if the manipulator 14 is operated in manual mode, but the user has stopped inducing any movement of the tool 20—for example, the user is still actuating the tool input 82 but is not applying any user force or torque to the tool 20—and the patient's anatomy moves slightly beyond the TCP of the tool 20 relative to the virtual boundary 71 (which is fixed relative to the patient's anatomy), the control system 60 will react by actuating one or more of the joint motors on the manipulator 14 to provide compensating movement of the tool 20 to maintain the TCP of the tool 20 in compliance with the virtual boundary 71.

[0108] When both tool input 82 and teach pendant input RC1 are in the second input state (e.g., neither is actuated), manipulator 14 operates in hold mode and boundary disabled state. In hold mode, movement of tool 20 is effectively disabled. In this case, manipulator 14 can still be actuated and operated to actively maintain the current position and / or orientation of tool 20 relative to manipulator coordinate system MNPL by monitoring encoder 19 and actively driving joint motors to resist external forces caused by gravity or forces accidentally applied to manipulator 14 or tool 20 by the user. In some versions, a braking system can be engaged to hold tool 20 in its current position and / or orientation. In hold mode, the user may wish to adjust the positioning of the patient's anatomy and target site without causing any accidental movement of tool 20, for example, such that movement of tool 20 occurs only in response to input from the user. The user may wish to adjust the patient's anatomy for various reasons, such as visualization, improved access to the target site, allowing cleaning of the target site, removal or cleaning of soft tissue, etc. In any case, if the patient's anatomy has been moved, any virtual boundary 71 fixed relative to the patient's anatomy will also move.

[0109] In hold mode and with boundary disabled, control system 60 disables any autonomous compliant boundary movement of tool 20. As a result, once the user has completed moving the patient's anatomy to improve visualization, palpation, or other aspects, and is ready to restart operation of manipulator 14 in manual or semi-autonomous mode, system 10 first checks to see if virtual boundary 71 has been moved in a way that tool 20 now violates virtual boundary 71 (e.g., outside virtual boundary 71, inside virtual boundary 71, deviating from virtual boundary 71, etc.). Therefore, when switching back to manual or semi-autonomous mode by switching tool input 82 or teach pendant input RC1 to the first input state, control system 60 performs a conflict check to determine if the TCP of tool 20 now violates virtual boundary 71.

[0110] If a conflict is detected, manual or semi-autonomous mode (depending on which input is actuated) remains disabled, and the control system 60 provides guidance to the user regarding the situation and how the tool 20 can be moved to restore its compliance with the virtual boundary 71. Otherwise, enabling manual or semi-autonomous mode, especially when the tool 20 has become completely non-compliant with the virtual boundary 71, could lead to sudden and unintended movement of the tool 20. User guidance can take the form of user feedback, such as visual feedback (on the display 38 on the tool 20, visual indicator LEDs, etc.), auditory feedback (via the speaker on the manipulator 14, the tool 20, etc.), and / or tactile feedback (e.g., by tactilely guiding the user to position the tool 20 in the desired relationship with the virtual boundary 71). Conflict checks can be periodic or continuous, and if the tool 20 regains compliance with the virtual boundary 71, manual or semi-autonomous mode can be enabled and user feedback will cease. The control system 60 can automatically switch the manipulator 14 from a boundary-disabled state to a boundary-enabled state when it detects that the tool 20 has regained compliance with the virtual boundary 71. In some versions, the control system 60 can provide automatic guidance to autonomously move the tool 20 to the position of complying with the virtual boundary 71. In this case, the recovery tool path can be generated by the path generator 68 (or other modules) and can be generated based on the current pose of the tool 20 (e.g., a known pose from the current pose to complying with the virtual boundary 71), or the recovery path can be predefined.

[0111] System 10 can also operate in a guided haptic mode, as described in U.S. Provisional Patent Application No. 62 / 908,056, filed September 30, 2019, entitled "Systems and Methods for Guiding Movement of a Tool," which is hereby incorporated herein by reference. The guided haptic mode can be used to haptically guide the user to place the tool 20 in a target state that complies with the virtual boundary 71, for example, when the tool 20 is in a hold mode and a boundary disabled state, in cases of violation of the virtual boundary 71. In the guided haptic mode, the control aspects used in both manual and semi-autonomous modes are utilized. For example, the force and torque applied by the user are still detected by the force / torque sensor S to determine the external force F fed into the virtual simulation to at least partially affect the overall movement of the tool 20. ext Additionally, in the guided haptic mode, system 10 generates virtual constraint force F. c The virtual attractive (or repulsive) force and torque, the virtual constraint force and the external force F ext They were fed into the virtual simulation together.

[0112] IV. Solving Constraints and Virtual Simulation

[0113] refer to Figure 11 One or more virtual constraints (such as path constraints, boundary constraints, guiding constraints, and other constraints) can be used by the control system 60 in various modes to control the movement of the tool 20. Typically, virtual constraints are restrictions on the motion of a rigid body, along with other motion-related information, considered by the control system 60 to determine how to command the manipulator 14 to move the tool 20. These virtual constraints can affect the position and / or orientation of the tool 20. As described in more detail below, the control system 60 includes a constraint solver 84 that operates to calculate constraint forces F that satisfy or attempt to satisfy the virtual constraints. c Constraint force F c Includes forces and torques that are defined as affecting the movement of tool 20.

[0114] Path constraints can be generated based on the tool path TP provided by the path generator 68. In practice, the path constraints cause virtual forces and / or torques to be calculated by the constraint solver 84 and used in a virtual simulation to pull tool 20 along the tool path TP, such that the TCP of tool 20 follows along the tool path TP while tool 20 maintains the desired orientation. Therefore, path constraints can also include orientation constraints, but can also be based on forces / torques applied by the user to adjust the orientation. See, for example, the user reorientation method described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. Path constraints can be generated in certain operating modes (such as in semi-autonomous mode) but may not be generated in other modes (such as in manual mode).

[0115] Boundary constraints can be defined to prevent tool 20 from violating one or more virtual boundaries 71. Each of the boundary constraints can be considered a unidirectional virtual constraint that operates to maintain the TCP of tool 20 conforming to one or more virtual boundaries 71. For example, a boundary constraint may cause virtual forces and / or torques to be calculated and used in the virtual simulation, such that the TCP of tool 20 has zero velocity (or near-zero velocity) at the virtual boundary 71 to prevent the TCP from penetrating into the virtual boundary 71 (or penetrating too deeply). Boundary constraints may be active in certain modes (such as in manual and semi-autonomous modes). However, boundary constraints may be disabled in certain situations (such as when the control system 60 operates in a boundary-disabled state as described above). For example, when a conflict between tool 20 and virtual boundary 71 is detected after moving the anatomical structure in hold mode, manual or semi-autonomous mode is disabled because the boundary constraints are no longer output to the constraint solver 84.

[0116] In some cases, even without outputting boundary constraints, users can still move tool 20 in free mode or other similar modes by applying forces and torques to it. In free mode, tool 20 moves relatively freely in response to the forces and torques applied to it by the user, allowing the user to move tool 20 to restore compliance with virtual boundary 71. The external force F includes the forces and torques applied by the user. ext It is the input to the constraint solver 84 and is fed into the virtual simulator 86 so that when enabled, the external force F ext It at least partially affects the overall movement of tool 20. When tool input 82 is in the first input state (e.g., actuated), the external force F ext Typically enabled in, for example, manual mode. External force F extIt can also be enabled in free mode to allow the user to move tool 20 to restore compliance with virtual boundary 71 in response to force and torque applied by the user. When both user input (e.g., tool input 82 and teach pendant input RC1) are in a second input state (e.g., neither is actuated), the external force F ext Disabled.

[0117] Guiding constraints are defined as virtual attractive (or repulsive) forces and torques generated in a virtual simulation to guide a user to place tool 20 in a target state conforming to virtual boundary 71. Guiding constraints are defined as ultimately influencing the movement of tool 20 toward the target state, such that they provide the user with one or more tactile interactive effects that guide the user to induce the desired movement of tool 20. Guiding constraints may be active in some modes (such as in guided haptic mode) but inactive in others. Guiding constraints may also provide the user with other forms of tactile feedback, such as a diminished sensation of movement of tool 20, to indicate an error or abnormality, such as when the user moves tool 20 further away from the target state.

[0118] In some versions, the virtual constraint is a velocity impulse constraint, where forces and / or torques are computed to apply virtual impulses to objects in a virtual simulation to cause changes in the object's velocity according to desired constraint parameters. In some versions, the constraint is similar to the constraint used in impulse modeling as described in U.S. Patent No. 9,119,655, which is incorporated herein by reference. In some versions, virtual constraints are used in all modes.

[0119] The virtual constraints adopted by the control system 60 are mainly defined by three runtime parameters: the constraint Jacobian matrix J. p It maps each virtual constraint to a coordinate system used for virtual simulation; the desired velocity V des (or Vp2), which is the scalar velocity of the virtual constraint in the coordinate system (e.g., when the patient is stationary and does not move relative to the associated virtual constraint defined by the patient, the expected velocity may be zero, but it may not be zero when the patient moves because the virtual constraint may be patient-dependent); and the constraint distance Δd, which is, for example, how close the TCP is to the constraint and indicates whether the virtual constraint has been violated. Δd can also be referred to as the penetration depth, i.e., the error distance along the constraint direction.

[0120] Virtual constraints are not infinitely rigid; rather, each virtual constraint has a tuning parameter to adjust its stiffness, for example, by incorporating spring and damping parameters into the constraint. Such parameters may include a constraint force mixing parameter (C) and an error reduction parameter (∈). The spring and damping parameters can be adjusted during operation. In some versions, the values ​​of the tuning parameters may be changed based on certain relationships, such as the curvature of the tool path TP (for path constraints), the relationship between the virtual boundary 71 and TCP (for boundary constraints), the relationship between the current state and the target state (for guiding constraints), etc. The tuning parameters may differ for different virtual constraints. For example, boundary constraints may be more rigid than other constraints. A virtual constraint may include a first virtual constraint with a first value of the tuning parameter and a second virtual constraint with a second value of the tuning parameter, the first value being greater than the second value, such that the constraint force F is embodied in... c The resulting virtual forces and / or torques are adapted to influence the movement of tool 20 to a greater extent due to the first virtual constraint compared to the second virtual constraint. The values ​​of the tuning parameters can be larger (e.g., more rigid) for position constraints than for orientation constraints, and vice versa.

[0121] The tuning parameters can also be set as follows: constant; exponentially increasing / decreasing with constraint distance; linearly changing with constraint distance; changing with constraint direction; considering gravitational effects; etc. The tuning parameters can also depend on the constraint force F ultimately calculated based on the virtual constraints. c Scaling, such as by depending on the constraint force F c The size of the tuning parameters or any of their components can be used to increase / decrease stiffness. The tuning parameters and their values, their correlation with a specific relationship, and how they can be scaled can be stored in one or more lookup tables in any suitable memory of the control system 60 for later retrieval.

[0122] Each virtual constraint also has configuration settings. These configuration settings may include: information about tuning parameters, such as constraint force mixing parameters (C) and error reduction parameters (∈); upper and / or lower force limits; and / or upper and lower constraint distance offset limits. The upper and lower force limits refer to restrictions on the forces calculated for each virtual constraint, which are ultimately solved by the constraint solver 84 to generate the constraint force F. cAs further described below. Virtual constraints can be unidirectional constraints (e.g., forces calculated to satisfy the constraint are only positive or only negative) or bidirectional constraints (e.g., forces calculated to satisfy the constraint can be either positive or negative). For unidirectional constraints, the upper limit of force can be set high (e.g., +100,000 Newtons) in the positive direction, and the lower limit of force can be set to zero, but the force limit can be set to any desired limit. For bidirectional constraints, the upper and lower limits of force can be set high in opposite directions (e.g., + / -100,000 Newtons). The upper and lower limit constraint distance offsets indicate when the constraints are active. Some constraints may always be active in certain modes. Regarding boundary constraints, the upper limit constraint distance offset can be zero and the lower limit constraint distance offset can be a large negative value (e.g., -100,000 mm) such that any boundary violation actually falls within the limit. The upper and lower limit constraint distance offsets can be set such that the boundary constraint is active when the TCP proposal state of the virtual simulation indicator 20 will violate the virtual boundary 71, as further described below.

[0123] Various virtual constraints, including guiding constraints, path constraints, boundary constraints, and others, can be fed into the constraint solver 84. These constraints can be enabled / disabled by the control system 60. For example, in some cases, there may be no path constraints (such as in manual mode), no boundary constraints (such as in hold mode, guided manual mode, or free mode), and no other constraints are being generated. Similarly, guiding constraints may not be generated unless it is necessary to guide the user to place the tool 20 at the restored compliant virtual boundary 71. All virtual constraints employed in the behavior control 74 can affect the movement of the tool 20.

[0124] The constraint solver 84 calculates the constraint force F to be virtually applied to the tool 20 in the virtual simulation performed by the virtual simulator 86 based on the virtual constraints fed into the constraint solver 84. c The ultimate task of constraint solver 84 is to solve for the constraint force F. c It provides solutions that satisfy or attempt to satisfy all constraints; therefore, other constraints may also affect the constraint force F. c The magnitude / direction. For example, when boundary constraints are actively transmitted to constraint solver 84, constraint solver 84 calculates the constraint force F based on the boundary constraints. c With components of force and / or torque suitable for maintaining tool 20 in compliance with virtual boundary 71.

[0125] refer to Figure 12 The constraint equations shown are used by constraint solver 84 to place the constraint data of each virtual constraint in matrix form into the corresponding rows of the constraint equations to solve F. p Here, F pIt is the force vector in the selected coordinate system, that is, F p Each component is a scalar constraint force acting in the corresponding constraint direction. To solve F... p As described below, Figure 12 The equations shown are converted into matrix equations, where each row represents a single one-dimensional constraint. The constraint data, along with other information known to the constraint solver 84, such as the external force F, are placed in the constraint equations. cgext Damping force F 阻尼 Inertial force F 惯性 Virtual mass matrix M, virtual mass velocity V cg1 and time step Δt (e.g., t set to 125 microseconds) 帧 ).

[0126] The virtual mass matrix M combines the 3x3 mass and inertia matrices. Damping force F 阻尼 and inertial force F 惯性 Calculated / known by virtual simulator 86, and based on the virtual mass velocity V output by virtual simulator 86 in the previous time step. cg1 (For example, the velocity of the virtual mass coordinate system VM). Virtual mass velocity V cg1 It is a 6-DOF velocity vector, which includes linear velocity and angular velocity components. Damping force F 阻尼 It is a 6-DOF force / torque vector, which is based on the virtual mass velocity V. cg1 The damping coefficient matrix (linear and rotational coefficients may not be equal) is calculated. Damping is applied to the virtual mass to improve its stability. Inertial force F 惯性 It is also a 6-DOF force / torque vector, which is based on the virtual mass velocity V. cg1 Calculate the virtual mass matrix M. Damping force F. 阻尼 and inertial force F 惯性 It can be determined in the manner described in U.S. Patent No. 9,566,122 to Bowling et al., which is hereby incorporated herein by reference.

[0127] The constraint solver 84 can be configured with any suitable algorithmic instructions (e.g., iterative constraint solver, projective Gauss-Seidel solver, etc.) to solve the constraint equations in order to provide a solution that satisfies the equations (e.g., satisfies various constraints). In some cases, not all constraints may be satisfied simultaneously. For example, in cases where motion is over-constrained by various constraints, the constraint solver 84 will essentially find a "best-fit" solution given the relative stiffness / damping of the various constraints. The constraint solver 84 solves the equations and ultimately outputs the constraint force F. p .

[0128] When using the projected Gauss-Seidel solver, the constraint solver 84 constructs matrices A and b based on the constraints and uses the projected Gauss-Seidel equations to determine the resulting force vector F. p The output of the projected Gaussian-Seidel coordinate system is taken and transformed from a selected coordinate system (e.g., a constrained coordinate system) to a virtual mass coordinate system VM. For example, using equation F... c =J p T F p F c It is a constraint force, force vector F p The components are converted into an equivalent force / torque vector F applied to the virtual mass coordinate system VM. c .

[0129] For example, methods for solving systems of equations with multiple constraints using the Projected Gauss-Seidel approach are shown in “Constraint-based physics solver” (v1.02) by Marijn Tamis and Giuseppe Maggiore, dated June 15, 2015 (available at http: / / www.mft-spirit.nl / files / MTamis_ConstraintBasedPhysicsSolver.pdf) or in “Comparison between Projected Gauss-Seidel and Sequential Impulse Solvers for Real-Time Physics Simulations” (v1.01) by Marijn Tamis, dated July 1, 2015 (available at http: / / www.mft-spirit.nl / files / MTamis_PGS_SI_Comparison.pdf), both of which are hereby incorporated in their entirety by reference.

[0130] The projected Gauss-Seidel method solves the linear complementarity problem (LCP). Since some constraint types (e.g., unilateral constraints, such as boundary constraints) can only push (apply force) in one direction, inequalities associated with LCP arise. If the force calculated for such constraints exceeds the force allowable for a given iteration of constraint solver 84 (which is invalid), the given constraints must be pruned (or alternately restricted / constrained at their allowed upper or lower limits), and the remaining constraints are solved until a suitable result (i.e., convergence) is found. In this way, constraint solver 84 determines the effective set of constraints for a given time step and then solves for their values. Other constraint types can apply force in both positive and negative directions; for example, bilateral constraints. Such constraints include guiding constraints used to direct the user to move the tool toward the target state. Such bilateral constraints are generally valid when enabled and are not pruned / restricted during iterations of constraint solver 84.

[0131] The constraint force F calculated by constraint solver 84 c This includes three force components along the x, y, and z axes and three torque components about the x, y, and z axes. The virtual simulator 86 utilizes constraint force F in its virtual simulation. c and external force F cgext Damping force F 虚拟 and inertial force F 惯性 (All of these can include the six components of force / torque). In some cases, these components of force / torque are first transformed into a common coordinate system (e.g., a virtual mass coordinate system VM), and then summed to define the resultant force F. T The resulting 6-DOF forces (i.e., force and torque) are applied to the virtual rigid body, and the resulting motion is calculated by the virtual simulator 86. The virtual simulator 86 is thus used to efficiently simulate how various constraints affect the motion of the virtual rigid body, all of which are reflected in the resultant force F. T The virtual simulator 86 is based on a given resultant force F applied to a virtual rigid body. T Forward dynamics are performed to calculate the resulting 6-DOF attitude and the velocity of the virtual rigid body. In one example, the virtual simulator 86 includes a physics engine, which is executable software stored in the non-transitory memory of any one or more of the aforementioned controllers 21, 26, 36 and implemented by the control system 60.

[0132] For virtual simulation, the virtual simulator 86 models tool 20 as a virtual rigid body in a virtual mass coordinate system VM. Typically, the origin of the VM is located at the center of mass of the virtual rigid body, and the coordinate axes are aligned with the principal axes of the virtual rigid body. For the purposes of virtual simulation, the virtual rigid body is the dynamic object and rigid body representation of tool 20. The virtual rigid body moves freely in Cartesian space with six degrees of freedom (6-DOF) according to the virtual simulation. Virtual simulation can be processed computationally without visual or graphical representation. Therefore, it is not required for the virtual simulation to display the dynamics of the virtual rigid body. In other words, the virtual rigid body does not need to be modeled within a graphics application running on the processing unit. The virtual rigid body may only exist within the virtual simulation.

[0133] The virtual rigid body and its properties (mass, inertia matrix, center of mass, principal axes, etc.) define how tool 20 will respond to applied forces and torques (e.g., from the resultant force F of the combination of forces and torques applied by the user and constraint forces and torques). T The control system 60 adjusts the feel of the tool 20 to the user by modifying the properties of the virtual rigid body. To make the motion / feel as realistic as possible, it may be necessary to model the properties of the virtual rigid body to reasonably approximate the actual properties of the tool 20, but this is not required. For control stability reasons (given the finite acceleration of the manipulator 14, control delay, etc.), the virtual mass and inertia can be modeled to be slightly higher than the virtual mass and inertia of the physical tool 20.

[0134] The virtual rigid body can correspond to a component that can be on or within tool 20. Alternatively, the virtual rigid body can extend partially beyond the physical tool 20. The virtual rigid body can be considered as tool 20 with energy applicator 24 or tool 20 without energy applicator 24. Furthermore, the virtual rigid body can be based on TCP. In one example, the center of mass of the virtual rigid body is understood as the point around which the virtual rigid body will rotate if a virtual force is applied to another point of the virtual rigid body and the virtual rigid body is originally unconstrained (i.e., not constrained by manipulator 14). The center of mass of the virtual rigid body can be close to but not necessarily the same as the actual center of mass of tool 20. The center of mass of the virtual rigid body can be determined empirically. Once tool 20 is attached to manipulator 14, the position of the center of mass can be reset to suit the preferences of individual practitioners.

[0135] The virtual simulator 86 applies forces and / or torques virtually to a virtual rigid body in a virtual simulation (i.e., by applying forces from the resultant force F). TThe force and torque components are virtually applied to the center of mass of the virtual rigid body in the virtual mass coordinate system VM to effectively simulate the rigid body dynamics of tool 20. Therefore, the force / torque virtually applied to the virtual rigid body can include the force / torque related to the external force F. cgext (For example, it is based on the force / torque and damping force F associated with input from one or more sensors) 阻尼 Inertial force F 惯性 , and the constraint forces F from various constraints. c Force / torque (due to the constraint force F) c middle).

[0136] The Jacobian matrix of a rigid body can be used to transform velocity and force from one coordinate system (reference frame) to another coordinate system on the same virtual rigid body, and can also be used here to represent F ext The force and torque are transformed into a virtual mass coordinate system VM (e.g., to generate F used in the constraint equations). cgext Then, the virtual simulator 86 internally calculates the damping force F. 阻尼 and inertial force F 惯性 Determine the resultant force F T It also outputs damping force F 阻尼 and inertial force F 惯性 This is so that the constraint solver 84 can use it in its system of equations in the next time step.

[0137] like Figure 13 and Figure 14 The virtual forward dynamics algorithm shown can be used in virtual simulations to simulate the motion of virtual rigid bodies because it applies a resultant force F. T The movement occurs over time. In practice, the virtual forward dynamics algorithm solves the equation F = ma (or a = F / m) in 6-DOF and integrates the acceleration to generate a velocity, which is then used to determine the new attitude, such as... Figure 14 As shown. The control system 60 will simulate virtual forces and / or torques (e.g., resultant force F). T The virtual forces and / or torques are input into the virtual simulator 86, and when the virtual rigid body is in an initial posture with an initial velocity, these virtual forces and / or torques are applied to the virtual rigid body at its center of mass (e.g., CG) in the virtual simulation. In response to the control system 60 satisfying the input virtual forces and / or torques, the virtual rigid body moves to a final posture with a different state (i.e., position and / or orientation) and a final velocity in Cartesian space. The next command posture to be sent to the motion controller 76 is based on the final posture calculated by the virtual simulator 86. Therefore, the virtual simulator 86 uses, as... Figure 14 The virtual forward dynamics simulation shown applies a resultant force F to a virtual rigid body. T The effect determines the posture for the next command.

[0138] Velocity limits may be imposed on the virtual rigid body in the simulation. In some cases, velocity limits can be set high so that they generally do not affect the simulation, or they can be set to any desired value. The virtual rigid body is in an initial attitude (initial state) and has an initial velocity at the start of each iteration of the virtual simulation (e.g., at each time step / interval dt). The initial attitude and initial velocity can be defined as the final attitude and final velocity output by the virtual simulator 86 in the previous time step. Thereafter, the virtual simulator 86 calculates and outputs the next command attitude based on its virtual simulation. The control system 60 is configured to command the manipulator 14 to move the tool 20 based on the command attitude.

[0139] Return to reference Figure 11 The block diagram illustrates the process executed to control the operation of manipulator 14 and tool 20 using the constraints described previously. In the illustrated version, behavior control 74 includes a path handler 88. The path handler 88 operates to generate / output path constraints based on the tool path TP provided by the path generator 68. The tool path TP is the input to the path handler 88. The path handler 88 generates path constraints based on the constraint parameters described previously, including determining constraint runtime parameters (e.g., the constraint Jacobian matrix J). p Expected speed V des (or Vp2) and constraint distance Δd). In reference Figure 11 In the described version, path handler 88 is enabled and active in semi-autonomous mode, but disabled in other modes.

[0140] Behavior control 74 also includes a boundary handler 90 to generate boundary constraints based on one or more virtual boundaries 71 generated by the boundary generator 66. The boundary constraints ultimately allow the control system 60 to control the operation of the manipulator 14 and the movement of the tool 20 based on the relationship between the tool 20 and one or more virtual boundaries 71 associated with the target location. For example, the control system 60 restricts the relative motion between the tool 20 and the virtual boundaries 71 via boundary constraints. The inputs to the boundary handler 90 include the tool 20's last commanded posture (e.g., considered as the current posture), the virtual boundaries 71, and the user input states of the tool input 82 and the teach pendant input RC1. The boundary handler 90 generates boundary constraints based on the constraint parameters described previously, including determining constraint runtime parameters (e.g., the constraint Jacobian matrix J). p Expected speed V des (or Vp2) and constraint distance Δd).

[0141] Behavior control 74 also includes a guidance processor 94. In certain situations, it may be necessary to guide the user to manipulate tool 20 in a manner that directs tool 20 to a desired location and / or orientation. For example, in some cases, the TCP of tool 20 may be in a position where it violates virtual boundary 71. In such cases, certain operating modes, such as manual or semi-autonomous modes (e.g., not generating boundary constraints or path constraints to influence the movement of tool 20), may be disabled until the TCP of tool 20 conforms to virtual boundary 71. The guidance processor 94 may obtain a target state of tool 20 that places tool 20 in compliance with virtual boundary 71, and the guidance processor may generate one or more guidance constraints based on the target state and the current state of tool 20. As previously described, the user may also be able to move tool 20 in free mode to restore compliance with virtual boundary 71, i.e., without generating any guidance constraints, or the control system 60 may automatically move tool 20 via a recovery path to restore compliance with virtual boundary 71. Other ways of implementing compliance with virtual boundary 71 are also envisioned, as further described below.

[0142] The inputs to the guidance processor 94 include a recovery signal and the last command pose (current state). The target state (e.g., pose) can be a portion of the recovery signal from the boundary processor 90, as the boundary processor 90 can identify the target position and / or orientation of the tool 20 that does not violate the virtual boundary 71. The target state can be defined in an anatomical coordinate system, an anatomical tracker coordinate system, etc., and is transformed to a common coordinate system with the last command pose. The guidance processor 94 defines one or more guidance constraints based on the relationship between the last command pose and the target state. The guidance constraints are output from the guidance processor 94 to the constraint solver 84. The guidance processor 94 is configured to activate the guidance constraints to provide haptic feedback to the user to guide the user to place the tool 20 in compliance with the virtual boundary 71, and the constraint solver 84 is configured to calculate the constraint force F. c The constraint force is adapted to comply with the virtual boundary 71 based on the guided constraint attraction tool 20.

[0143] V. Conflict checking and boundary enabled / disabled status

[0144] Boundary handler 90 performs various conflict checks depending on the operating mode, user input state, etc. The first type of conflict check involves checking whether / how the current state (e.g., current pose) of tool 20 or the proposed state (e.g., proposed pose) of tool 20 generated in virtual simulation by virtual simulator 86 violates virtual boundary 71. This conflict check is performed to determine the boundary constraints that need to be generated by boundary handler 90 and applied by constraint solver 84, such that the current / proposed state is changed in a way that prevents or at least limits the tool 20 from violating virtual boundary 71 during normal operation in manual or semi-autonomous mode. In some versions, this type of conflict check is performed in each frame during operation in manual or semi-autonomous mode and occurs before virtual simulator 86 generates a new command pose, such that the finally generated command pose, executed by motion control 76, limits the tool 20 from violating virtual boundary 71. In some versions, this type of conflict check may be performed by boundary handler 90 based on the command pose calculated in the previous iteration (e.g., the command pose of the previous time frame is set as the current pose). In that case, boundary handler 90 determines that boundary constraints need to be generated to at least limit violations of virtual boundary 71. For example, a command pose from the previous frame could be a pose that causes tool 20 to move slightly across virtual boundary 71, but boundary handler 90 generates boundary constraints in the current frame to make tool 20 return.

[0145] A method for performing a first type of collision detection is described in U.S. Patent Application Publication No. 2018 / 0353253, entitled "Robotic Surgical System and Method For Producing Reactive Forces To Implement Virtual Boundaries," granted to Bowling, which is hereby incorporated by reference. Other collision detection methods may also be employed. For example, if the virtual boundary 71 is defined by a triangular mesh, collision detection using wide-phase and narrow-phase searches can be performed as described in U.S. Patent No. 9,119,655, which is also incorporated by reference.

[0146] When tool input 82 or teach pendant input RC1 switches from a second input state to a first input state, for example, when system 10 is switching from hold mode to manual mode or from hold mode to semi-autonomous mode, boundary handler 90 performs a second type of conflict check. The current state of the user input and its switching can be detected by user input state detector 92. User input state detector 92 feeds the current state of the user input (e.g., tool input 82 and teach pendant input RC1) and indicates any state changes to boundary handler 90. The second type of conflict check can be a subroutine executed whenever each of the user inputs switches from one state to another.

[0147] To perform the second type of conflict check, boundary handler 90 checks the geometric definition of virtual boundary 71 against the current state of tool 20 (e.g., the last commanded pose) to determine whether the tool (e.g., the TCP of tool 20) conforms to or violates virtual boundary 71. As previously mentioned, predefined configurable tolerances can be established for virtual boundary 71, such that tool 20 penetrating into virtual boundary 71 to some extent or deviating from full conformity to virtual boundary 71 is not considered a violation. Therefore, the second type of conflict check also needs to consider these tolerances. This may only require checking the bounded volume of virtual boundary 71 and comparing the current position of the TCP of tool 20 to the bounded volume. In some versions, in addition to TCP, the positions of one or more virtual stereotactic interaction features (SIFs) belonging to tool 20 can also be compared to the bounded volume. These stereotactic interaction features (SIFs) can be points corresponding to actual points on tool 20, spheres with unique origins and radii, or other suitable geometries. Each of the stereotactic interaction features (SIFs) is compared to the bounded volume to check for conflicts. For example, a bounded volume can be defined by voxels, and the boundary handler 90 can perform collision detection to determine whether the tool 20 falls within any of the voxels. Other collision detection methods can utilize ray tracing, configuration space search, bounding volume hierarchy, point membership classification (PMC), etc. The boundary handler 90 can use any suitable method to detect collisions between the tool 20 and the virtual boundary 71.

[0148] If the second type of conflict check indicates that tool 20 violates virtual boundary 71, recovery mode is enabled and a recovery signal and associated target state are sent to bootstrap processor 94, which can then generate user feedback previously described as guiding the user to position tool 20 in compliance with virtual boundary 71. When tool 20 violates virtual boundary 71, the desired operating mode of manipulator 14 (e.g., manual mode or semi-autonomous mode) can be disabled. Autonomous boundary compliance movement of tool 20 remains disabled in recovery mode when user input (e.g., tool input 82 or teach pendant input RC1) is in the first input state.

[0149] If the conflict check indicates that tool 20 has complied with virtual boundary 71, the desired operating mode of manipulator 14 can be enabled. If the conflict check passes (e.g., TCP and / or other SIFs are entirely within the allowed area of ​​virtual boundary 71, or the distance penetrating virtual boundary 71 does not exceed the configured distance (e.g., 0.1 mm)), the boundary constraint is activated and movement of tool 20 is enabled. The second type of conflict check and boundary constraint activation are performed atomically (within the same time step) to avoid movement competition between check and activation.

[0150] If the second type of conflict check fails, boundary constraints are not enabled, and a recovery sequence is initiated via recovery mode. The recovery sequence may include a user message displayed on one or more of the displays 38, requesting the user to remove tool 20 from the cutting area. Tool drivers are also disabled (e.g., machining is not allowed). In some versions, high-damped movement may be enabled in recovery mode (e.g., the boot handler 94 may employ damping constraints to provide high-damped movement of tool 20). The damping coefficient used in the virtual simulation performed by the virtual simulator 86 can also be adjusted to change the damping force F. 阻尼 This results in increased damping. Such movement can be enabled by allowing the manipulator 14 to respond to user forces and torques applied to the tool 20, but in a relatively suppressed manner. High-damped movement allows the user to immediately sense, through tactile interaction (via damping constraints), the presence of an abnormality if the user is not directly looking at the display 38 at that time.

[0151] The recovery mode can lead to the activation of a guided haptic mode to help guide the user during recovery. Alternatively, a free mode can be activated in recovery mode to allow the user to move tool 20 freely to comply with virtual boundary 71. As previously described, a recovery path can be generated additionally or alternatively in recovery mode and used to autonomously move tool 20 to restore compliance with virtual boundary 71. In this case, teach pendant input RC1 can be used to control the movement of tool 20 along the recovery path. In some versions, in recovery mode, boundary handler 90 or guidance handler 94 can generate a recovery constraint associated with virtual boundary 71, the tuning parameters of which are lower than the tuning parameters of the original boundary constraint, so that tool 20 can recover from violating virtual boundary 71 more gradually (i.e., virtual boundary 71 is effectively changed to be less rigid). In some versions, in recovery mode, control system 60 can move virtual boundary 71 from its starting position (e.g., change its position / orientation) so that tool 20 no longer violates virtual boundary 71. Once tool 20 resumes compliance with virtual boundary 71, autonomous compliance boundary movement of tool 20 can be activated by control system 60, and then virtual boundary 71 slowly turns back to its starting position, while gently pushing tool 20 along with it, so that tool 20 remains compliant with virtual boundary 71.

[0152] The steps performed in recovery mode are executed relatively quickly, resulting in no noticeable delay in initiating tool movement in response to the user's initial activation of tool input 82 or teach pendant input RC1. For example, this allows the user / manipulator 14 to move tool 20 relatively efficiently back to the permitted area relative to virtual boundary 71. While tool 20 moves in recovery mode, control system 60 periodically or continuously performs a second type of conflict check to detect when tool 20 resumes compliance with virtual boundary 71. At this time, without requiring the user to release tool input 82 or teach pendant input RC1, the previously described boundary constraint activation is performed, damping (if employed) is restored to its normal setting (providing an indication that the condition has been resolved based on user perception), the ability to use the tool driver and operate tool 20 is enabled, and the message on display 38 is updated / cleared.

[0153] A recovery sequence can also be initiated if one of the trackers is ineffective during activation (i.e., invisible or below a quality threshold). Once the tracker becomes visible and the second type of conflict check is cleared, the control system 60 switches to boundary enabled state to allow operation in manual or semi-autonomous mode.

[0154] Figure 15A subroutine for performing a second type of conflict check, executed by boundary handler 90 and associated components, is shown. In step 100, boundary handler 90 determines whether an input state change exists, i.e., whether tool input 82 or teach pendant input RC1 has switched from one state to another. If not, and if boundary handler 90 is not already in recovery mode, boundary handler 90 returns to check for a state change. If a state change exists, boundary handler 90 then determines the nature of the state change in step 102, i.e., whether it is a change / transition from a second input state to a first input state (e.g., user input has been actuated) or vice versa (e.g., release). If tool input 82 or teach pendant input RC1 has been actuated, boundary handler 90 performs a conflict check in step 104 to determine whether tool 20 complies with or violates virtual boundary 71.

[0155] If virtual boundary 71 is violated, then: (i) the control system 60 is switched to a boundary disabled state in step 106; (ii) the tool driver controlling the operation of tool 20 is disabled in step 108, meaning the control system 60 effectively ignores any input from the user that is normally associated with the operation of tool 20; and (iii) a recovery mode is initiated in step 110. It should be noted that these steps may occur almost simultaneously, and Figure 15 The order presented is for illustrative purposes only. In step 112, the recovery signal instructs the guidance process 94 to generate user feedback to notify the user of a violation of virtual boundary 71 and / or to guide the user to position tool 20 in compliance with virtual boundary 71. Once in recovery mode, the subroutine continues to check for any subsequent state changes, and if there are no state changes (e.g., the user still actuates tool input 82 or teach pendant input RC1), the boundary process 90 continues in recovery mode to check whether tool 20 remains in violation of virtual boundary 71.

[0156] If the virtual boundary 71 is not violated, then: (i) the control system 60 is switched to the boundary-enabled state in step 114; (ii) the tool driver for the operation of the control tool 20 is enabled in step 116; and (iii) the recovery mode ends in step 118 (if it is in an active state). Although not shown, when the recovery mode ends, a manual mode or semi-autonomous mode selected by the user is enabled. It should be noted that these steps may occur almost simultaneously, and Figure 15The order presented is for illustrative purposes only. It should also be noted that when tool drive is enabled in step 116, tool 20 may not be immediately activated after entering recovery mode. Instead, tool control returns to the user to allow tool operation via the associated user interface (UI) (e.g., via buttons, foot switches, triggers, etc.). In cases where the user is actuating the UI to cause tool operation (e.g., electric cutting, etc.) at the end of recovery mode, control system 60 may ignore the input until the user releases and re-engages it, ensuring that tool 20 does not operate unexpectedly after leaving recovery mode.

[0157] If tool input 82 or teach pendant input RC1 has been released, then: (i) in step 120, the control system 60 is switched to hold mode and boundary disabled state; (ii) in step 122, the tool driver controlling the operation of tool 20 is disabled; and (iii) in step 124, the recovery mode ends (if it is active). It should be noted that these steps may occur almost simultaneously, and Figure 15 The order presented is for illustrative purposes only.

[0158] In some versions, the control system 60 can be configured to disable autonomous compliant boundary movement of the tool 20 after a predetermined time period following a transition of the user input from a first input state to a second input state. For example, it might be desirable to keep the virtual boundary 71 enabled for a short period (e.g., 100ms to 500ms) after the tool input 82 or teach pendant input RC1 is released to allow the manipulator 14 / tool ​​20 to begin stopping. Otherwise, if the boundary constraint is disabled immediately, the tool 20 might slide across the virtual boundary 71 and cut or move in an undesirable manner. In some cases, when the tool input 82 or teach pendant input RC1 is released, the control system 60 can switch to highly damped movement as described above, allowing the tool 20 to begin stopping more quickly. The virtual boundary 71 can be disabled after a fixed time interval or once the tool 20 begins to stop. This delay can be predetermined / pre-configured as described above, or it can be automatically controlled by the control system 60 by monitoring the tool 20 (e.g., a virtual rigid body) and its velocity (linear velocity, rotational speed) and keeping boundary constraints enabled until the magnitude of the velocity (linear velocity, rotational speed) falls below a certain threshold (or below a set of thresholds defined for each degree of freedom and / or each type of velocity). In some cases, the virtual boundary 71 can remain active until one or more velocities fall below one or more thresholds or reach a maximum time, whichever occurs earlier.

[0159] exist Figure 15The process outlined in the document illustrates how the boundary handler 90 can operate between a boundary enabled state and a boundary disabled state, in which boundary constraints are transferred from the boundary handler 90 to the constraint solver 84, and in the boundary disabled state, boundary constraints are no longer transferred from the boundary handler 90 to the constraint solver 84, thereby disabling the autonomous compliant boundary movement of the tool 20. Boundary handling procedure 90: (i) operates in a boundary disabled state in response to a change of tool input 82 or teach pendant input RC1 from a first input state to a second input state, wherein there may be a delay in the transition to the boundary disabled state after tool input 82 or teach pendant input RC1 is switched to allow the movement of manipulator 14 to stabilize; (ii) operates in a boundary enabled state in response to a change of tool input 82 or teach pendant input RC1 from a second input state to a first input state, provided that tool 20 conforms to virtual boundary 71; (iii) operates in a boundary disabled state in response to a change of tool input 82 or teach pendant input RC1 from a second input state to a first input state if tool 20 violates virtual boundary 71 during or shortly after the transition; and (iv) switches from the boundary disabled state to the boundary enabled state once tool 20 is placed to conform to virtual boundary 71 if tool input 82 or teach pendant input RC1 is still in the first input state.

[0160] The constraint solver 84, virtual simulator 86, path handler 88, boundary handler 90, user input state detector 92, and bootstrap handler 94 each comprise executable software stored in the non-transitory memory of any one or more of the aforementioned controllers and implemented by the control system 60. The constraint solver 84, virtual simulator 86, path handler 88, boundary handler 90, user input state detector 92, and bootstrap handler 94 may be embodied in one or more software modules stored in any suitable location for implementation by the control system 60.

[0161] VI. Behavioral Control

[0162] Figure 16 The various steps performed by behavior control 74 are summarized. These include steps performed by constraint solver 84, virtual simulator 86, path handler 88, boundary handler 90, and guidance handler 94, as described above. In step 130, the external force F is calculated based on the readings obtained from the force / torque sensor S. ext In steps 132 and 133, constraint data associated with various virtual constraints is obtained and active constraints are identified. In step 133, constraints such as boundary constraints may be updated depending on the results of the virtual simulation, as further described below. Figure 16This indicates an implementation where conflicts with the proposed state output by the virtual simulation are evaluated before calculating the final updated state. However, in some versions, the output from the virtual simulation is set as the final updated state without checking for additional conflicts (see further description below). Figure 16A ).

[0163] In steps 134 to 138, the virtual simulator 86 performs rigid body calculations to determine the mass inverse matrix M of the virtual rigid body. -1 Inertial force F 惯性 and damping force F 阻尼 In steps 140 to 144, the constraint solver 84 uses the output from the rigid body calculations performed in steps 134 to 138 and the constraint data provided in steps 132 and 133 to perform the previously described constraint force calculations to ultimately generate constraint forces. Fc In step 146, the constraint force F is... c Transformed into a virtual mass coordinate system VM(F) cgext External force F ext Damping force F 阻尼 and inertial force F 惯性 The resultant force F is generated by their combination. T In step 148, the resultant force F is simulated in a virtual simulation performed by the virtual simulator 86. T An application is made to the virtual rigid body to determine the proposed state (e.g., attitude and velocity) of the virtual rigid body, and finally the initial state and the proposed state are transformed into TCP in step 150.

[0164] In step 152, software triggering is employed, which initially follows one path and then follows another path in the next execution. In the first path, boundary handler 90 performs a first type of conflict check in step 154 ​​to determine whether the proposed state would result in a conflict with virtual boundary 71. If no conflict is detected, the proposed state is verified in step 156 and saved as an updated state, and then transformed into TCP in step 158. In step 160, virtual simulator 86 outputs a new command pose (T). TCP ) and speed (V) TCP If no conflict is detected, the trigger (step 152) remains in its current state, that is, the trigger is not switched to follow another path.

[0165] If a collision is detected, the boundary handling procedure 90 calculates the collision time t in step 162. 冲突 And switch the trigger. (See reference) Figure 17A and Figure 17B This can be achieved by calculating the first distance between the current state and the proposed state calculated in the virtual simulation. Figure 17A), calculate the second distance between the current state and the virtual boundary 71 ( Figure 17B Then multiply the ratio of the second distance to the first distance by the total time frame t. 帧 Determine the conflict time t 冲突 Therefore, for example, if the virtual boundary 71 crosses at a distance of 50% from the proposed state, the conflict time t 冲突 It is time frame t 帧 50%. Then in step 164, the time Δt used for constraint force calculation in steps 140 to 144 is reset to the conflict time t. 冲突 (See also) Figure 17C The virtual simulation is then adjusted to the time before the conflict occurred. The virtual simulation then determines the new proposal state. Once the new proposal state is determined, the trigger (step 152) causes an alternative path to be followed (because a conflict was detected), and the new proposal state becomes the updated current state in step 166. Once the transition to step 166 occurs, the trigger automatically resets to follow the first path.

[0166] Next, refer to Figure 16 and Figure 17D The control system 60 considers the time of conflict (i.e., conflict time t) 冲突 ), until the original time frame t 帧 End. In other words, control system 60 in the time interval Δt = t 帧 -t 冲突 A new virtual simulation is performed during this period. In step 168, boundary constraints are obtained for this next round of virtual simulation to effectively generate the virtual pulses required to keep tool 20 from crossing virtual boundary 71 (or thus tool 20 only minimally crosses virtual boundary 71). The boundary constraints obtained in step 168 are then used to update the constraints in step 133, and for the new time period t... 帧 -t 冲突 Perform virtual simulation (see step 148). In step 150, determine the new proposal state and transform it into TCP again. The trigger in step 152, after being reset, follows the first path again, and a collision check is performed again in step 154. For some virtual boundaries 71, once a collision is detected, the above steps will cause the next collision check to always be negative, and a new command attitude and velocity will be output at step 160. However, for complex geometries, even if the first collision is detected and resolved, the next round of virtual simulation may lead to yet another collision. In this case, other steps can be performed to resolve multiple collisions, but... Figure 16The process shown is for illustrative purposes and is for a simpler case. In some cases, a maximum number of iterations of the virtual simulation can be set such that the process stops once the maximum number of iterations is reached. If a conflict still exists after the maximum number of iterations is reached, this can be handled in several ways: (i) output the latest proposed state as the final updated state; (ii) output the state of the previous frame (i.e., the initial state) as the final updated state; or (iii) issue an error signal and stop the movement of manipulator 14, tool 20, etc.

[0167] refer to Figure 16A In some versions, Figure 16 Steps 152, 154, 156, 162, 164, 166, and 168 are removed, and the output of the virtual simulation is instead set to the new updated state, which ultimately produces a new command pose. In this version, the first conflict check is part of step 132, where the boundary handler 90 determines the boundary constraints (if any) to be fed into the constraint solver 84 (along with other constraints).

[0168] VII. Exemplary Operation

[0169] Figures 18A to 18G An operational example of system 10 is shown. Figure 18A In this configuration, the user is shown gripping the handle 83 of tool 20 at tool input 82, thus actuating tool input 82 and placing it in a first input state. The user operates manipulator 14 in manual mode and boundary-enabled state to remove material from the target site (in this case, from the femur F). A virtual boundary 71 is shown covering the femur F, which can be visually displayed to the user via display 38. Figure 18A In this configuration, the user has already removed a small amount of tissue from the femur F using tool 20, but a significant amount of material remains within the virtual boundary 71. If necessary, the control system 60 maintains tool 20 within the virtual boundary 71 by enabling autonomous, compliant boundary movement of tool 20, thus mitigating any movement of the virtual boundary 71 that might otherwise cause tool 20 to violate it.

[0170] Figure 18B The image shows the user moving the TCP of tool 20 above the femur F to prepare for moving the femur so that the user can more easily access the femur F.

[0171] Figure 18CThe user movement of the femur F is illustrated. The user has removed their hands from the grip 83 of tool 20 and tool input 82 has been released. Therefore, tool input 82 is now in a second input state and control system 60 is now switched to hold mode and boundary disabled state. Thus, autonomous compliant boundary movement of tool 20 is disabled, and femur F can be moved by the user without causing any corresponding movement of tool 20. Navigation system 32 detects movement of femur F by means of locator 44 via a first patient tracker 54 securely attached to femur F. As shown, tool 20 has moved outside virtual boundary 71.

[0172] Figure 18D The user is shown re-engaging tool 20 to continue operating in manual mode. More specifically, the user re-actuates tool input 82 to place tool input 82 into a first input state. As previously discussed, this transition from the second input state to the first input state causes boundary handler 90 to perform a second type of conflict check. The result of this conflict check is that tool 20 violates virtual boundary 71. Therefore, guidance handler 94 can provide the user with one or more forms of user feedback to indicate this state of tool 20 and / or guide the user to move tool 20 to restore compliance with virtual boundary 71. Alerts and associated guidance are shown being provided in Figure 18D and Figure 18E On one or more monitors. Figure 18E In the process, the user has already moved the tool 20 in either the guided haptic mode or free mode to conform to the virtual boundary 71.

[0173] Figure 18F The diagram shows the user now actuating the teach pendant input RC1 on the remote controller RC to place the teach pendant input RC1 in a first input state to operate the system 10 in semi-autonomous mode. More specifically, the user has released the tool input 82 to switch from manual mode to semi-autonomous mode. When the control system 60 switches operation from manual mode to semi-autonomous mode, the control system 60 can perform a third type of conflict check (e.g., perform a third type of conflict check in addition to performing a second type of conflict check). More specifically, before enabling semi-autonomous mode, the path handler 88 defines the import path LIP from the current attitude of tool 20 to the starting point SP on the tool path TP. Even if the current attitude of tool 20 and the starting point SP both conform to the virtual boundary 71, this straight path may sometimes intersect the virtual boundary 71 (e.g., ...). Figure 18F(As shown). The third type of conflict check determines whether the import path LIP generated by the path handler 88 violates the virtual boundary 71. If there is no violation of the virtual boundary 71, the control system 60 is able to autonomously move the tool 20 along the import path LIP to the tool path TP in a semi-autonomous mode. If moving the tool 20 along the import path LIP would violate the virtual boundary 71, the control system 60 indicates to the user that the virtual boundary will be violated and provides the user with guidance on how to move the tool 20 to avoid such a violation (e.g., ...). Figure 18F The device responds to the instruction "Change tool position" shown on display 38. Semi-autonomous mode remains disabled.

[0174] The third type of conflict checking can be performed similarly to the first type. As mentioned, when the manipulator 14 switches from manual mode to semi-autonomous mode, the path handler 88 generates an import path LIP from the current position (or attitude) of tool 20 to the starting point of the tool path TP. The boundary handler 90 determines whether the movement of tool 20 along the import path LIP will maintain compliance with the virtual boundary 71 or violate the virtual boundary 71.

[0175] VIII. Other Conflict Checks

[0176] exist Figure 18G In some versions shown, boundary handler 90 is configured to determine whether movement of tool 20 along import path LIP will maintain compliance with or violate virtual boundary 71 by modeling the motion of multiple stereoscopic interactive features (SIFs) associated with tool 20. This can be performed by a similar conflict check on linear motion from stereoscopic interactive features (SIFs). Such motion can be determined by transforming the import path LIP and its corresponding orientation to each of the corresponding stereoscopic interactive features (SIFs) (see [link to relevant documentation]). Figure 18G LIP in T In some versions, boundary handler 90 is configured to model more complex motions of multiple stereo-oriented interactive features (SIFs) with three or more degrees of freedom. Guide handler 94 generates feedback to the user in response to boundary handler 90 determining that tool 20 (or any of its modeled points) will violate virtual boundary 71 if it moves from its current position along import path LIP to tool path TP.

[0177] Import path LIP conflict checking may consider enabled stereoscopic interactive features (SIFs) and their shapes, which may be spheres configured for tool 20 and their positions, and ensure that each of them is swept across the respective range of motion that will be encountered as the TCP traverses the suggested import path LIP. Note that the import path LIP may include aligning the position and / or orientation with the starting point SP (for autonomous machining greater than 3-DOF), i.e., the final movement of each stereoscopic interactive feature SIF during import is not necessarily a straight line. Therefore, control system 60 may: (i) accurately model the motion of each stereoscopic interactive feature SIF and perform continuous conflict detection according to the applicable translation / rotation trajectory or as shown, and (ii) approximate the stereoscopic interactive feature SIF trajectory by translating the conflict shape only between the starting and ending positions of the stereoscopic interactive feature SIF. If the import path conflict check fails, an alert is issued to the user before any movement of tool 20 (i.e., the manipulator 14 remains in hold mode, free mode, or manual mode), and the user is provided with potentially enhanced visual information via display 38 about how to resolve the situation.

[0178] refer to Figure 19 Boundary handler 90 can be configured to perform a fourth type of conflict check when the user wishes to switch from the current virtual boundary (shown as first virtual boundary 71) to second virtual boundary 75 during operation in manual or semi-autonomous mode. If the user wishes to change the virtual boundary midway through surgery, such as switching from first virtual boundary 71 to second virtual boundary 75, boundary handler 90 checks to ensure that the second virtual boundary 75 will not be violated if a switch occurs (e.g., in the same manner as assessing whether the first virtual boundary 71 is violated, including taking into account any tolerances set for the second virtual boundary 75). If the second virtual boundary 75 would be violated, control system 60 continues operation, where first virtual boundary 71 is enabled and second virtual boundary 75 is disabled. However, if the second virtual boundary 75 is not violated, control system 60 activates the second virtual boundary 75 and deactivates the first virtual boundary 71.

[0179] A user can use a Virtual Boundary Selector (VBS) to indicate a desire to switch to a second virtual boundary 75. The VBS allows the user to select a second virtual boundary 75, which can also be associated with a target location, so that the control system 60 subsequently controls the operation of the manipulator 14 and the movement of the tool 20 in the same manner as the control system 60 uses to maintain compliance with the first virtual boundary 71, thereby maintaining the tool 20's compliance with the second virtual boundary 75. The VBS can include user input located on the manipulator 14, tool 20, remote controller RC, etc. The user input to the VBS can be any suitable form of input device, including those previously described herein. The VBS allows the user to trigger between the first virtual boundary 71 and the second virtual boundary 75, trigger sequentially from a plurality of virtual boundaries, or select from a list of virtual boundaries.

[0180] The control system 60 enables the user to select a second virtual boundary 75 using the virtual boundary selector (VBS), while the control system 60 continues to control the operation of the manipulator 14 and the movement of the tool 20 to maintain the tool 20's compliance with the first virtual boundary 71. The boundary handler 90, in response to the user's selection of the second virtual boundary 75, determines whether the tool 20 complies with or violates the second virtual boundary 75. If the boundary handler 90 determines that the tool 20 complies with the second virtual boundary 75, it activates the second virtual boundary 75 and deactivates the first virtual boundary 71, causing the control system 60 to switch the control operation of the manipulator 14 and the movement of the tool 20 from being based on the first virtual boundary 71 to being based on the second virtual boundary 75. The boundary handler 90 is configured to maintain the first virtual boundary 71 in an active state if it determines that the tool 20 violates the second virtual boundary 75 in response to the user's selection of the second virtual boundary 75 via the virtual boundary selector (VBS).

[0181] The control system 60 generates user feedback to the user in response to the boundary processing procedure 90 determining that the tool 20 violates the second virtual boundary 75 when the user actuates the virtual boundary selector VBS to select the second virtual boundary 75. The control system 60 is configured such that the user can select the second virtual boundary 75 using the virtual boundary selector VBS when either the tool input 82 or the teach pendant input RC1 is in the first input state. Furthermore, when the user selects the second virtual boundary 75 using the virtual boundary selector VBS, the control system 60 continues the operation of the tool driver. In some versions, the user can also select the second virtual boundary 75 using the virtual boundary selector VBS when both the tool input 82 and the teach pendant input RC1 are in the second input state.

[0182] In some versions, the user can instantly select a second virtual boundary 75 using the Virtual Boundary Selector (VBS), meaning that manual or semi-autonomous mode is active (e.g., one of tool input 82 or teach pendant input RC1 is engaged in the first input state) and the manipulator 14 and tool 20 are moving. If a fourth type of conflict check indicates a violation of the second virtual boundary 75, the user / manipulator 14 will not stop or be forced to stop operating / moving tool 20 because the valid boundary constraint remains active (e.g., for the first virtual boundary 71).

[0183] In some examples, the first virtual boundary 71 and the second virtual boundary 75 can be considered as standard and extended boundaries for total knee surgery or other surgical procedures. The standard boundary may have limited dimensions (e.g., height, width, and / or depth) based on the implant size, and the extended boundary may have one or more dimensions that are substantially larger than the standard boundary (e.g., a larger width) to allow the user to access more tissue (e.g., bone) with tool 20. The control system 60 may default to operating at the standard boundary. Based on user input via the virtual boundary selector VBS, the user can switch to the extended boundary, which allows access to a larger amount of tissue, if needed. Once the user has completed processing an area where the user requires a larger (e.g., wider) boundary, the user may wish to switch back to the standard boundary to request the remaining cuts. Using a fourth type of conflict check, if tool 20 is within the area allowed by the standard boundary, the user can only return from the extended boundary to the standard boundary. If not, the user interface (e.g., the display) indicates a violation and failure to switch back to the standard boundary, and the user can retry later by actuating the virtual boundary selector VBS again.

[0184] When tool input 82 or teach pendant input RC1 is in the first input state and the second virtual boundary 75 is active, the control system 60 restricts the relative movement between the tool 20 and the second virtual boundary 75 by generating a second boundary constraint for the second virtual boundary 75 using the boundary processing program 90. When the user has successfully selected the second virtual boundary 75 using the virtual boundary selector VBS, the constraint solver 84 can calculate the constraint force F. c The first virtual boundary 71 is conformed to by a first boundary constraint maintenance tool or the second virtual boundary 75 is conformed to by a second boundary constraint maintenance tool.

[0185] Please note that the conflict checking / activation / deactivation steps for transitioning from the first virtual boundary 71 to the second virtual boundary 75 can be performed atomically (i.e., within the same time step) to avoid movement race conditions between conflict checks and activation or time gaps where virtual boundaries 71 and 75 are either not active or both are active during this period. Virtual boundaries 71 and 75 may also be referred to as conflict scenarios. If the conflict check fails, the conflict scenario will not be updated from the first conflict scenario to the second conflict scenario. Since an active conflict scenario will not be deactivated unless the conflict check passes, the control system 60 is able to maintain its operation just before the transition request. User feedback (e.g., auditory feedback, user messages on display 38, haptic feedback, etc.) can be provided to notify the user of the transition failure. Although this can be done automatically, it is generally desirable to allow the user to decide when / whether to retry the transition from 'active' to 'new' conflict scenario, in which case the above sequence is repeated.

[0186] In some cases, when a user's attempt to switch to the second virtual boundary 75 fails and the first virtual boundary 71 remains active, an auxiliary mode can be activated to help position the tool 20 in compliance with the second virtual boundary 75. In auxiliary mode, the same controls described for recovery mode can be used to help position the tool 20 in compliance with the second virtual boundary 75, except that in manual or semi-autonomous mode, the first virtual boundary 71 is still used to generate boundary constraints to maintain compliance with the first virtual boundary 71 and remain active.

[0187] The assistive mode may include a control system 60 that: (i) generates visual cues on a display or elsewhere to guide the user to move the tool 20 to conform to a second virtual boundary 75; (ii) generates guidance constraints using a guidance processor 94 to guide the user to conform to the second virtual boundary 75; (iii) generates an assistive tool path to autonomously move the tool 20 to conform to the second virtual boundary 75; (iv) moves the second virtual boundary 75 from its starting position to make the tool 20 conform to it, and then gradually moves the second virtual boundary 75 back to its starting position to gently pull the tool 20 along with it; and / or (v) generates an assistive constraint associated with the second virtual boundary 75, the assistive constraint having a tuning parameter lower than that of the original boundary constraint used for the second virtual boundary 75, so that the tool 20 can more gradually conform to the second virtual boundary 75 (i.e., the second virtual boundary 75 is effectively changed to be less rigid). Other methods for guiding the tool 20 to conform to the second virtual boundary 75 are also contemplated. In some cases, one or more of these methods can be considered by the control system 60 and executed when certain conditions are met. The control system 60 can check to see how close the tool 20 is to the second virtual boundary 75 and then select the appropriate method. For example, if the tool 20 is within 1 mm to 2 mm of the second virtual boundary 75, it may be appropriate to activate the second virtual boundary 75 with temporarily looser tuning parameters, or to temporarily displace the second virtual boundary 75. Once the tool 20 conforms to the second virtual boundary 75, the control system 60 can manually or automatically switch control to the second virtual boundary 75. Other implementations include backtracking along the tool axis, for example, for TKA or THA acetabular preparation for marginal osteophytes.

[0188] Remote control RC and / or other input devices on various user interfaces (UIs) can be used to switch, activate, and / or deactivate various operating modes of the manipulator 14. The control system 60 can be configured to automatically switch modes in certain situations. If these modes are initially disabled when selected, the control system 60 can also prompt the user before operating in manual or semi-autonomous mode because the boundary handler 90 detects that the tool 20 violates the virtual boundary 71. The control system 60 can also prompt the user before operating in guided haptic mode. Such prompts can include providing optional prompts on one or more displays 38 to remain in manual, semi-autonomous, or guided haptic mode. The user can select to remain in manual, semi-autonomous, guided haptic, etc., via any suitable input device on any user interface (including the remote control RC).

[0189] The current state of tool 20 relative to virtual boundary 71, tool path TP, target state, and / or target location can be output by navigation system 32 and displayed on display 38 via a graphical representation of tool 20, virtual boundary 71, tool path TP, target state, and / or target location (e.g., femur F, tibia T, pelvis PEL, vertebral body, or other anatomical structure). These graphical representations can be updated in real time, allowing the user to visualize the movement of tool 20 relative to virtual boundary 71, tool path TP, target state, anatomical structure, etc. For example, the graphical representations of tool 20 and anatomical structures can move in real time on display 38 as the manipulator 14 actually moves tool 20 and the anatomical structure.

[0190] The various modes described herein can be used in a variety of surgical systems. For example, the manipulator may include a remotely operated robotic arm controlled via a user interface remotely positioned relative to the robotic arm. The user interface may include a separate manipulator, such as a 6-DOF control unit manually operated by the user, for example, a separate manipulator with movable joints to provide tactile feedback to the user.

[0191] IX. Stereoscopic Interactive Features

[0192] As described above, the stereoscopic interactive feature (SIF) can be used by the system for collision detection. This section describes various configurations or implementations that can be used to implement or utilize SIF. The following configurations or implementations of SIF can be used in conjunction with any of the techniques or components described above, which are incorporated herein by reference in their entirety. Alternatively or additionally, the following configurations or implementations of SIF can be utilized independently of any of the techniques described above and for other general purposes, such as, but not limited to, robot control, collision avoidance, user experience, etc.

[0193] The stereotactic interactive feature (SIF) can be attributed to any object that can interact with the virtual boundary 71. Optionally, the SIF can be attributed to any component whose posture can be controlled (manually, automatically, or kinematically). For example, the SIF can be attributed to any part of the robotic surgical system, tool 20, end effector 22, power applicator 24 or TCP, manipulator 14, any link 18 or joint J of manipulator 14, base 16, or any other part of the kinematic chain forming manipulator 14. The SIF can be attributed to other objects or surgical components in the operating room, such as handheld tools, operating table, head-mounted device, handheld display device or tablet, tracker, retractor, patient, personnel, or staff. The location of the SIF can be known because any suitable method (including but not limited to the positioning techniques described above) can be used to track any of the objects. The object to which the SIF is attributed can be selected based on input to the program and / or automatically generated, for example, based on factors such as surgical plan, type or procedure of operation, surgeon preferences, etc.

[0194] Furthermore, the virtual boundary 71 described in this section can be attributed to any object other than an anatomical structure. For example, virtual boundary 71 can be attributed to any part of the robotic surgical system, tool 20, end effector 22, power applicator 24 or TCP, manipulator 14, any link 18 or joint J of manipulator 14, base 16, or any other part of the kinematic chain forming manipulator 14. Virtual boundary 71 can be attributed to other objects or surgical components in the operating room, such as imaging devices (C-arm, crane gantry, CT scanner, etc.), handheld tools, operating table, head-mounted devices, handheld display devices or tablets, trackers, retractors, patients, personnel, or staff. Virtual boundary 71 can have any suitable shape or configuration depending on the object to which it is attributed. For example, if virtual boundary 71 is attributed to link 18 of manipulator 14, virtual boundary 71 can take the shape of a volume surrounding link 18, such as... Figure 20 As shown. Other configurations are envisioned. The location of the virtual boundary 71 can be known because any suitable method (including but not limited to the positioning techniques described above) can be used to track any of the objects. The object to which the virtual boundary 71 belongs can be selected based on input to the program and / or automatically generated, for example, based on factors such as surgical plan, operation type or procedure, surgeon preferences, etc.

[0195] The SIFs (Signal Indicators) can be defined, located, customized, and implemented using the control system 60 (including software program 78 and boundary handler 90) described in the preceding sections. The boundary handler 90 can detect conflicts between any number of SIFs and the virtual boundary 71 using any suitable method. In response to any one or more conflicts or anticipated conflicts between SIFs and the virtual boundary 71, the control system can adjust the attitude of the corresponding object and / or generate alarms or notifications. Such responses from the control system are provided to passively or actively prevent, avoid, mitigate, or reduce conflicts.

[0196] Any number of SIFs can belong to any number of objects. Figure 20 In the example, multiple SIFs are assigned to tool 20, which in this case is a slender cutting file used for milling the femur F to perform THA. Of course, this example is for illustrative purposes only and is not intended to limit the scope of how SIFs can be utilized. The number of assigned SIFs can be defined based on input to the procedure and / or automatically generated, for example, based on factors such as surgical plan, type or procedure of operation, surgeon preferences, etc.

[0197] In some implementations, the SIF can be attributed to a location directly located at the object to which the SIF belongs. For example, in Figure 20 In this example, several SIFs, designated SIF-1, are directly attributed to the axis SH of tool 20. These SIF-1s are spaced apart from each other along the axis SH and are located directly above the axis SH. In this example, SIF-1s are provided to avoid conflict between the axis SH and the virtual boundary 71 of the inner surface of the canal belonging to the femur F. For example, as the TCP follows the tool path during milling, tool 20 can be routinely reoriented, which in turn causes the axis SH to be reoriented. Having multiple SIF-1s along the axis SH in response to such reorientation reduces the likelihood of physical conflict between the axis SH and anatomical structures.

[0198] In other implementations, the SIF can be assigned to a position spaced apart from the object to which the SIF belongs. For example, in Figure 20 In this example, several SIFs, designated SIF-2, are assigned to the proximal end of tool 20. These SIF-2s are spaced apart from tool 20 by a specified distance. The spacing of the SIF-2s in this example may be appropriate because, in response to reorientation, tool 20 is more likely to experience greater orientational movement (e.g., angular motion) near the proximal end compared to near the TCP. Therefore, the spacing of the SIF-2s from tool 20 provides increased buffering, or an earlier response to collisions or anticipated collisions. In this example, the SIF-2s are arranged to form a SIF loop at the proximal end. The configuration or arrangement of the SIFs can be derived directly from the geometry of the object (e.g., the axis SH or the 2D section of tool 20). SIFs may be directly adjacent to each other or spaced apart from each other.

[0199] The intervals of SIFs relative to the corresponding objects to which they belong can be defined based on the input to the program and / or automatically generated, for example, based on factors such as surgical plans, operation types or procedures, surgeon preferences, etc.

[0200] In other implementations, the SIF can be configured with any suitable geometry. For example, the SIF can be a point, a surface, or a volume. The SIF can have any suitable shape. Figure 20 In the example, the SIF has a spherical shape. However, other 2D shapes or volumes are envisioned, such as, but not limited to: planes, hyperboloids, parabolas, cylinders, cubes, pyramids, cuboids, ellipsoids, prisms, or any type of polyhedron. For any type of geometric configuration, the SIF can also be configured to any suitable size. For example, in... Figure 20 As the SIF approaches the proximal end of tool 20, the volume of the spherical SIF-1 increases. Similarly, this can be implemented to provide an earlier response to conflict or anticipated conflict. The geometry or size of the SIF can be defined based on input to the procedure and / or automatically generated, for example, based on factors such as surgical plan, type or procedure of operation, surgeon preferences, etc.

[0201] In one implementation, the SIF is not infinitely rigid, but each of the SIFs can have a tuning (stiffness) parameter to adjust the stiffness of the SIF relative to the virtual constraint, for example, by incorporating spring and damping parameters into the constraint. Such parameters may include a constraint force mixing parameter (C) and an error reduction parameter (∈). The spring and damping parameters can be adjusted before or during operation. The tuning parameters of the SIF may differ for different objects, conditions, locations, or geometric configurations. The SIF may include a first SIF with a first value of the tuning parameters and a second SIF with a second value of the tuning parameters, the first value being greater than the second value, such that the constraint force F is embodied in... c The resulting virtual forces and / or torques are adapted to influence the movement of tool 20 to a greater extent due to the first SIF compared to the second SIF. The values ​​of the tuning parameters can be larger (e.g., more rigid) for position constraints than for orientation constraints, and vice versa.

[0202] The tuning parameters of SIF can also be set as follows: constant; exponentially increasing / decreasing with constraint distance; linearly changing with constraint distance; changing with constraint direction; considering gravitational effects; etc. The tuning parameters can also depend on the constraint force F ultimately calculated based on the virtual constraint. c Scaling, such as by depending on the constraint force F cThe size of the SIF or any of its components can be used to increase / decrease stiffness. The tuning parameters of the SIF and their values, their correlation with a specific relationship, and how they can be scaled can be stored in one or more lookup tables in any suitable memory of the control system 60 for later retrieval.

[0203] In one implementation, a first tuning parameter can be defined for the first SIF and a second tuning parameter can be defined for the second SIF. The first tuning parameter and the second tuning parameter have different values. In one example, the first and second SIFs with different tunings are located on different parts of the kinematic chain KC of the robot manipulator 14. The SIFs with different tunings can also be located at different locations on the same part of the kinematic chain KC. The kinematic chain KC is formed by the manipulator 14 (including any rigidly attached components such as the base 16, multiple links 18, and connector J), the tool 20 (including the axis SH (if applicable)), and the energy applicator 24. The end effector 22 can also be part of the kinematic chain KC. Furthermore, any mounting system or sterile interface coupled between the manipulator 14 and the end effector 22 can be part of the kinematic chain KC. An example of a mounting system and / or sterile interface mechanism that can be part of a kinematic chain is described in U.S. Patent Application Publication No. US 2020 / 0170724A1, entitled "Mounting System With Sterile Barrier Assembly For Use In Coupling Surgical Components," the entire contents of which are incorporated herein by reference. As used herein, the term "kinematic chain" refers to an assembly of rigid bodies connected by joints, wherein the stiffness of the rigid bodies enables constrained motion, such that the kinematics of the rigid bodies can be determined and correlated with other rigid bodies in the kinematic chain using mathematical models. Figure 20 In the example, the kinematic chain further forms a “mechanism” because at least one link is mechanically grounded.

[0204] exist Figure 20In the example, the first and second SIFs with different tunings can be located on tool 20 and / or axis SH. In this case, the tuning parameters of SIF-2 may be more rigid (e.g., greater damping or a larger spring constant) than those of SIF-1, because the potential for undesired orientation motion of axis SH is greater near the proximal end of tool 20. The greater stiffness of SIF-2 may cause manipulator 14 to command less destructive corrective motion near the proximal end of tool 20. In other examples, the stiffness of SIF-2 may be lower than that of SIF-1. This tuning configuration can be implemented to provide a smoother transition when there is a conflict between the energy applicator / tool ​​and the virtual boundary 71. Although this implementation may initially result in greater penetration of axis SH or tool 20 into virtual boundary 71, other behaviors of the robot manipulator can be changed immediately before, during, or after penetration to address such greater penetration due to conflict. Furthermore, having different tuning parameters for the SIFs can reduce the probability that conflict with boundary 71 may lead to an erroneous condition that causes the operation of manipulator 14 to stop. Such tuning parameter configurations can be applied to any number of SIFs to mitigate unwanted object motion or for any other purpose related to robot control and / or user experience.

[0205] Furthermore, multiple tuning parameters can be associated with any single SIF. For example, the tuning parameters of an SIF can be varied with respect to time, condition detection, the distance of the SIF to the corresponding object, the distance of the SIF to the virtual boundary 71, etc. In another example, the size of an SIF may be large enough to include multiple tuning parameters simultaneously. For example, a portion / surface / volume of an SIF can be configured to be more or less rigid than another portion / surface / volume of the same SIF.

[0206] Any of the aforementioned characteristics of the SIF can be dynamically changed during surgery, either intraoperatively or during pauses in the operation of the robotic system. For example, the control system can determine SIF change events in response to certain control events or environmental conditions. Such conditions may be anticipated or unexpected and can be detected by the control system using positioning data, kinematic data, or any combination thereof. In response, the control system can dynamically change the position, geometry, spacing, and stiffness of one or more SIFs. This dynamic change characteristic can be managed based on program input and / or automatically managed, for example, based on factors such as surgical plan, type or procedure of operation, and surgeon preferences.

[0207] Any of the features, characteristics, properties, and / or behaviors described in SIF can be referred to as parameters of SIF. Any of the above-described embodiments or parameters of SIF can be used alone or in any combination thereof.

[0208] Several embodiments have been described in the foregoing description. However, the embodiments discussed herein are not intended to be exhaustive or to limit the invention to any particular form. The terminology used is intended to be descriptive in nature and not restrictive. In view of the above teachings, many modifications and variations are possible, and the invention may be practiced in ways other than those specifically described.

Claims

1. A surgical system comprising: tool; A manipulator for supporting the tool; as well as A control system for controlling the operation of the manipulator and the movement of the tool based on a relationship between the tool and a virtual boundary associated with a target area, the control system comprising: User input, which has a first input state and a second input state. The control system is configured to enable autonomous compliant boundary movement of the tool when the user input is in the first input state, such that the tool maintains compliance with the virtual boundary. The control system is configured to disable autonomous compliance boundary movement of the tool when the user input is in the second input state; as well as A boundary handler is configured to determine whether the tool violates the virtual boundary in response to a change in user input from the second input state to the first input state.

2. The surgical system of claim 1, wherein the control system is configured to initiate a recovery mode in response to the tool violating the virtual boundary when the user input changes from the second input state to the first input state, and autonomous compliance boundary movement of the tool remains disabled in the recovery mode when the user input is in the first input state.

3. The surgical system of claim 1, wherein the tool includes a tool driver, and the control system is configured to disable the tool driver in response to the tool violating the virtual boundary when the user input changes from the second input state to the first input state.

4. The surgical system of claim 2, wherein the control system is configured to guide a user in the recovery mode to place the tool in compliance with the virtual boundary by generating user feedback including one or more of auditory feedback, visual feedback and tactile feedback.

5. The surgical system of claim 4, wherein the control system is configured to stop generating the user feedback when the tool is placed to conform to the virtual boundary.

6. The surgical system of claim 5, wherein the control system is configured to limit relative movement between the tool and the virtual boundary by generating boundary constraints using the boundary handler when the user input is in the first input state, wherein the control system comprises: A constraint solver, which is used to calculate, based on the boundary constraints, constraint forces suitable for maintaining the tool's compliance with the virtual boundary; as well as A virtual simulator, used to simulate the dynamics of the tool in a virtual simulation based on the constraints and output commands regarding its posture. The control system is configured to command the manipulator to move the tool based on the gesture of the command.

7. The surgical system of claim 6, wherein the boundary handler is operable between a boundary enabled state and a boundary disabled state, in the boundary enabled state, boundary constraints are transmitted from the boundary handler to the constraint solver, thereby enabling autonomous compliant boundary movement of the tool when the virtual boundary moves relative to the tool in a manner that would otherwise cause the tool to violate the virtual boundary; in the boundary disabled state, boundary constraints are no longer transmitted from the boundary handler to the constraint solver, thereby disabling autonomous compliant boundary movement of the tool, such that the virtual boundary can move relative to the tool in a manner that would cause the tool to violate the virtual boundary, the boundary handler being configured to: The operation is disabled at the boundary state in response to the user input changing from the first input state to the second input state; In response to the user input transitioning from the second input state to the first input state while the tool conforms to the virtual boundary, the operation is performed in the boundary-enabled state; and In response to the user input changing from the second input state to the first input state if the tool violates the virtual boundary, the operation is performed in the boundary disabled state.

8. The surgical system of claim 7, wherein the control system is configured to provide tactile feedback to the user to guide the user to place the tool in accordance with the virtual boundary by activating one or more guiding constraints to guide the tool to conform to the virtual boundary.

9. The surgical system of claim 7, wherein the control system is configured to provide tactile feedback to the user to guide the user to place the tool in compliance with the virtual boundary by inhibiting movement of the tool.

10. The surgical system of claim 8, wherein the control system is configured to switch the boundary handler from the boundary disabled state to the boundary enabled state when the tool is placed to conform to the virtual boundary.

11. The surgical system of claim 1, wherein the user input is configured such that the first input state indicates that the user is actively engaging the tool and the second input state indicates that the user has released the tool.

12. The surgical system of claim 1, wherein the user input is located on the tool and the user input is configured such that the user input is actuated to place the user input in a first input state and the user input is released to place the user input in a second input state.

13. The surgical system of claim 12, wherein the tool has a handle, and the user input includes a presence detector to detect the user's hand on the handle.

14. The surgical system of claim 1, wherein the control system includes a teach pendant, and the user input is located on the teach pendant and the user input is configured such that the user input is actuated to place the user input in a first input state and the user input is released to place the user input in a second input state.

15. The surgical system of claim 1, wherein the user input is further defined as a tool input located on the tool, and the first input state and the second input state are further defined as a tool input first state and a tool input second state, wherein the control system includes a teach pendant and a teach pendant input located on the teach pendant, the teach pendant input having a teach pendant input first state and a teach pendant input second state.

16. The surgical system of claim 15, wherein the manipulator is capable of operating in a manual mode, in which the manipulator moves the tool in response to user force and torque applied to the tool by the user when the tool input is in the tool input first state, and the manipulator is capable of operating in a semi-autonomous mode, in which the manipulator moves the tool along a tool path when the teach pendant input is in the teach pendant first state.

17. The surgical system of claim 16, wherein the boundary handling procedure is configured to determine whether the tool conforms to or violates the virtual boundary in response to the control system switching the operation of the manipulator from one of the manual mode and the semi-autonomous mode to the other of the manual mode and the semi-autonomous mode.

18. The surgical system of claim 17, wherein the control system includes a path processor configured to generate an import path from the current position of the tool to the tool path when the manipulator switches from the manual mode to the semi-autonomous mode.

19. The surgical system of claim 18, wherein the boundary handling procedure is configured to determine whether movement of the tool along the insertion path will maintain compliance with the virtual boundary or violate the virtual boundary.

20. The surgical system of claim 19, wherein the boundary handling procedure is configured to determine whether movement of the tool along the infeed path will maintain compliance with or violate the virtual boundary by modeling the motion of a plurality of stereotactic interactive features associated with the tool.

21. The surgical system of claim 20, wherein the boundary handling procedure is configured to model the motion of the plurality of stereoscopic interactive features with three or more degrees of freedom.

22. The surgical system of claim 21, wherein the control system includes a guidance processor configured to generate user feedback to the user in response to the boundary processor determining that the tool will violate the virtual boundary if the tool moves along the import path from the current position to the tool path.

23. The surgical system of claim 1, wherein the control system is configured to disable autonomous compliant boundary movement of the tool in response to one or more of the following: The tool begins to stop; A predetermined time period elapses after the user input changes from the first input state to the second input state; The linear velocity of the tool decreases to below one or more thresholds; or The angular velocity of the tool decreases to below one or more thresholds.

24. The surgical system of claim 1, wherein the control system is configured to determine whether the tool remains compliant with the virtual boundary based on a tolerance defined for the virtual boundary.

25. The surgical system of claim 2, wherein the control system is configured to generate a recovery tool path in the recovery mode to move the tool to conform to the virtual boundary.

26. The surgical system of claim 2, wherein the control system is configured to move the virtual boundary from the starting position in the recovery mode such that the tool returns to conformity with the virtual boundary, and then move the virtual boundary back to the starting position, while simultaneously enabling autonomous conformity movement of the tool.

27. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement a method for controlling the operation of a manipulator supporting the tool based on a relationship between a tool and a virtual boundary associated with a target part, the method comprising the steps of: When the user input is in the first input state and in response to the movement of the virtual boundary relative to the tool, the tool initiates an autonomous boundary-compliant movement, so that the tool remains compliant with the virtual boundary; When the user input is in the second input state, disable the tool's autonomous compliance boundary movement; as well as The tool is determined to have violated the virtual boundary in response to the user input changing from the second input state to the first input state.

Images