Technology for adaptive reorientation of robotic surgical tools

Abstract

This provides a robotic surgical system and method that facilitates tool reorientation. [Solution] The system includes a tool for working on a surgical site and a manipulator for supporting and moving the tool. A controller controls the manipulator to facilitate the movement of the tool along a tool path for treating the surgical site. While the tool is moving along the tool path, the controller detects the reorientation of the tool and records the location where reorientation occurs. In response to the tool revisiting the recorded location, the controller generates a virtual constraint that causes the manipulator to reorient the tool. In some cases, the controller predictively detects a location along the tool path where the tool should be reoriented, and in response to the tool reaching the predictively detected location, the controller uses the virtual constraint to cause the manipulator to reorient the tool.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 717,640, filed on November 7, 2024, the entire content of which is incorporated herein by reference.

Background Art

[0002] In recent years, medical practitioners have found advantages in using robotic systems to perform surgical procedures. Such systems provide robotic control of surgical tools that move along a tool path to manipulate a surgical site. Very generally, a surgeon may need to manually reorient the surgical tool as the tool moves along the tool path. For example, a surgeon may manually reorient the surgical tool to avoid an obstacle or a sensitive area at the surgical site. When the reoriented tool passes an obstacle or a sensitive area, the surgeon typically has to reorient and return the tool to its original orientation. Even more difficult, in some cases, a surgeon may need to repeatedly pass the surgical tool near an obstacle or a sensitive area, for example, in an attempt to remove all material from a target area. In such cases, the surgeon may need to perform reorientation or back - and - forth reorientation of the surgical tool each time the tool approaches an obstacle. Further, the reorientation of the tool may not necessarily be for avoiding an obstacle but may be required for the ergonomics of the surgeon. For example, a surgeon may desire to hold the tool during its movement, but may need to repeatedly reorient the tool as it crosses a curved portion of the path to maintain a comfortable wrist position.

[0003] Regardless of the cause, such repeated reorientation is cumbersome for surgeons, causing hand or arm fatigue and potentially delaying surgical procedures. Furthermore, such conventional techniques, for example, force surgeons to perform reorientation under time constraints due to tool movement. These time constraints can cause surgeon anxiety and potentially lead to human error. [Overview of the project]

[0004] In a first embodiment, a robotic surgical system is provided, comprising: a manipulator configured to support and move a tool along a tool path to work on a surgical site; and one or more controllers configured to automatically reorient the tool at positions along the tool path, the positions of which are identified based on user input defining the positions.

[0005] In a second embodiment, a robotic surgical system is provided, comprising: a manipulator configured to support and move a tool along a tool path to work on a surgical site; and one or more controllers configured to automatically reorient the tool at positions along the tool path, the positions of which are predictively identified by the one or more controllers.

[0006] In a third embodiment, a robotic surgical system is provided, comprising: a tool configured to work on a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to control the manipulator to facilitate the movement of the tool along a tool path for treating a surgical site; to detect reorientation of the tool while the tool is moving along the tool path and record the location where reorientation occurs; and to generate one or more virtual constraints configured to cause the manipulator to reorient the tool in response to the tool revisiting the recorded location.

[0007] In a fourth aspect, a robotic surgical system is provided, comprising: a tool configured to work on a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to perform: acquire a tool path defined with respect to the surgical site; predictively detect a position in the tool path where the tool should be reoriented; generate one or more virtual constraints configured to cause the manipulator to reoriented the tool at the predictively detected position; control the manipulator to facilitate the movement of the tool along the tool path for treating the surgical site, and utilize one or more virtual constraints to cause the manipulator to reoriented the tool in response to the tool reaching the predictively detected position.

[0008] In a fifth aspect, a robotic surgical system is provided, comprising: a tool configured to work on a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to control the manipulator to facilitate the movement of the tool along a tool path for treating a surgical site; to detect reorientation of the tool while the tool is moving along the tool path, record the location where reorientation occurs, and generate feedback indicating that reorientation is needed, the feedback being configured to be delivered in response to the tool revisiting the recorded location.

[0009] In a sixth aspect, a robotic surgical system is provided, comprising: a tool configured to work on a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to perform the following: acquire a tool path defined with respect to the surgical site; predictively detect a position in the tool path where the tool should be reoriented; and generate feedback indicating that reorienteding is necessary, the feedback being configured to be delivered in response to the tool reaching the predictively detected position.

[0010] In a seventh aspect, a robotic surgical system is provided, comprising: a tool configured to handle a surgical site; a manipulator configured to support and move the tool along a predetermined tool path; a user input device; and one or more controllers configured to receive input from the input device, which identifies one or more locations along a predetermined tool path, where the one or more locations indicate places where the user desires to cause the tool to reorient; generate one or more virtual constraints configured to cause the manipulator to reorient the tool in response to the tool reaching each of the identified one or more locations; and control the manipulator to facilitate the movement of the tool along a predetermined tool path to treat a surgical site, and utilize one or more virtual constraints to automatically reorient the tool in response to the tool reaching each of the identified one or more locations.

[0011] A method for operating a robotic surgical system according to any of the above embodiments is provided. Any of the embodiments can be combined in part or in whole.

[0012] Any embodiment can be combined with any of the following embodiments, and embodiments can be described in part or in whole. A controller(s) may generate at least one virtual constraint at a recorded location. A controller(s) may generate at least one virtual constraint for a tool. The robotic surgical system may include a navigation system that tracks the patient's anatomical structures, including the surgical site. A controller(s) may register virtual constraints(s) to the tracked anatomical structures. A controller(s) may customize the features of the virtual constraints(s). Customized features may include one or more of the location of the virtual constraint(s), the geometry of the virtual constraint(s), and / or the stiffness / damping parameters of the virtual constraint(s). Customization may be based on a recorded location relative to the tool path. The robotic surgical system may include a camera configured to detect obstacles or sensitive areas at a recorded location. The camera may be attached to the manipulator or tool, or positioned away from the manipulator or tool. The controller(s) may customize the characteristics of the virtual constraint(s) based on detected obstacles or sensitive areas. The controller(s) may detect tool parameters during or after reorientation. Detected tool parameters may include one or more of the following: tool displacement, direction of force applied to the tool, magnitude of force applied to the tool, tool velocity, and tool acceleration. The controller(s) may customize the characteristics of the virtual constraint(s) based on the detected tool parameters. The controller(s) may generate virtual constraint(s) in a manner that causes the manipulator to reorient the tool with a magnitude and direction that mimics the detected reorientation. The controller(s) may generate virtual constraint(s) in a manner that takes into account one or more of the manipulator's working space limits, manipulator joint limits, and manipulator singularities.A controller(s) may compare detected tool parameters to a threshold and, in response to the detected parameters meeting the threshold, may determine that the tool reorientation is an intentional manual reorientation in response to an external force applied to the tool by the user. The tool may include a user interface configured to receive input to initiate the tool reorientation. A controller(s) may detect the reorientation in response to receiving input from the user interface. Virtual constraints(s) may include a virtual mesh, which may optionally be defined at a recorded location and / or on the tool. A controller(s) may cause the manipulator to reorient the tool in response to the interaction between the tool and the virtual mesh. Virtual constraints(s) may include at least one orientation interaction, which may optionally be defined at a recorded location and / or on the tool. Controllers may cause the manipulator to reorient the tool in response to an interaction between at least one localization interaction feature and a virtual mesh, or in response to an interaction between two or more localization interaction features. The tool may include a tool shaft, and at least one localization interaction feature may be defined relative to the tool shaft. Virtual constraints may include attractive or repulsive forces, which may optionally be defined in the vicinity of a recorded position. Controllers may cause the manipulator to reorient the tool in response to the tool being subjected to an attractive or repulsive force, or entering a zone defining an attractive or repulsive force. The tool may be in its original position before being reoriented by the virtual constraints. Controllers may reorient the tool back to its original position if the tool leaves the recorded position. Controllers may modify the features of the virtual constraints in response to detection of a second reorientation of the tool occurring in the vicinity of the recorded position. Any of the controller(s) features can be performed automatically, including automatic movement of the tool along the tool path, automatic generation of virtual constraints(s), and automatic reorientation of the tool to the manipulator using virtual constraints(s).The controller(s) may dynamically remove or deactivate the virtual restraint(s) in response to the detection of any condition, such as the completion of a surgical step or the determination that the tool no longer needs to pass near the recorded location.

[0013] Controllers may predictively detect their position based on the identification of obstacles or sensitive areas by the camera. Controllers may predictively detect their position based on the detection of one or more preceding reorientations of the tool. Controllers may predictively detect their position based on one or more preceding detected parameters of the tool. Controllers may predictively detect their position by utilizing machine learning models trained on preceding surgical planning data and preceding manipulator data, such as tool path data, reorientation data, implant data, kinematic data, and camera data. Controllers may predictively customize the features of virtual constraints. Predictively customized features include one or more of the virtual constraint's position, geometry, and / or stiffness / damping parameters.

[0014] The controller(s) may generate feedback indicating that reorientation is needed at a recorded or predictively detected location. The feedback may be delivered in response to the tool revisiting the recorded location. The feedback may be haptic feedback provided by the manipulator. The feedback may be visual feedback provided by a light indicator provided to the tool. The feedback may be visual and provided on a display that also presents representations of the tool, tool path, and surgical site. The feedback may include visual identification of the location relative to the representation of the tool path on the display. The feedback may include a text message or verbal warning indicating that reorientation is needed.

[0015] The advantages of the present invention will become readily apparent as they are better understood by referring to the following detailed description, when considered in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0016] [Figure 1] This is a perspective view of a robot system including a manipulator according to one embodiment. [Figure 2] This is a schematic diagram of a controller (or multiple controllers) for a robot system according to one embodiment. [Figure 3] This is a diagram of a surgical tool moving along a tool path according to one embodiment. [Figure 4] This is a diagram of a tool pathway for treating a patient's anatomical structure according to one embodiment. [Figure 5] This is a flowchart illustrating a method for facilitating the reorientation of surgical tools according to one embodiment. [Figure 6] This figure illustrates the reorientation of a surgical tool while it moves along a tool path during the operation of a manipulator in a semi-autonomous, automatic, or guided manual mode, according to one embodiment. [Figure 7] This diagram shows the reorientation of a surgical tool during operation of a manipulator in manual mode, according to one embodiment. [Figure 8] This is a diagram of a virtual mesh generated during a method for facilitating the reorientation of surgical tools according to one embodiment. [Figure 9] This is a diagram of stereotactic interaction features generated during a method for facilitating the reorientation of a surgical tool according to one embodiment. [Figure 10] This is a diagram of particle guides generated during a method for facilitating the reorientation of surgical tools according to one embodiment. [Figure 11] Figures A and B illustrate a configuration in which a controller(s) modify a virtual constraint by reducing its size, according to one embodiment. [Figure 12]A and B are diagrams of a configuration in which a controller (s) modifies virtual constraints by expanding the size of the virtual constraints according to an embodiment. [Figure 13] A and B are diagrams of a configuration in which a controller (s) integrates a plurality of virtual constraints into a collective virtual constraint according to an embodiment. [Figure 14] A and B are diagrams of a configuration in which a controller (s) automatically generates virtual constraints according to an embodiment.

Best Mode for Carrying Out the Invention

[0017] I. System Overview

[0018] Referring to the drawings, like reference numerals indicate like or corresponding parts throughout the plurality of figures, and a surgical system 10 (hereinafter "the system") and a method of operating the system 10 are described herein and shown throughout the accompanying drawings.

[0019] As shown in FIG. 1, the system 10 is a robotic surgical system for treating an anatomical structure (surgical site) of a patient 12, such as bone or soft tissue. In FIG. 1, the patient 12 is undergoing a surgical procedure. The anatomical structure A in FIG. 1 includes the femur and tibia of the patient 12. The surgical procedure may include tissue removal or treatment. The robotic surgical system 10 described herein can be used to treat any anatomical structure (s), such as joints including the knee, hip, shoulder, ankle, or any other bone structure (s) not described herein. The robotic surgical system 10 can be used to perform any type of procedure, including any spinal procedure, partial knee arthroplasty, total knee arthroplasty, total hip arthroplasty, anatomical shoulder arthroplasty, reverse shoulder arthroplasty, fracture repair surgery, osteotomy, etc. Similarly, the techniques and methods described herein can be used for any type of robotic system and any procedure.

[0020] System 10 includes a manipulator 14, which may also be referred to as a robotic manipulator. In one example, the manipulator 14 has a base 16 and a number of links 18. The number of links 18 may generally be referred to as a robotic arm 18A. In some cases, the manipulator 14 may include a number of robotic arms 18A. A manipulator carriage 17 supports the manipulator 14 so that the manipulator 14 is fixed to the manipulator carriage 17. The links 18 collectively form one or more arms of the manipulator 14. The manipulator 14 may have a series arm configuration (as shown in Figure 1) or a parallel arm configuration. In other examples, multiple manipulators 14 may be used in a multi-arm configuration. The manipulator 14 includes a number of joints (J) and a number of joint encoders 19 positioned at the joints (J) for determining the position data of the joints (J). For simplicity, one joint encoder 19 is shown in Figure 1, but it should be understood that other joint encoders 19 may be shown similarly. In one example, manipulator 14 has six joints (J1-J6) that implement at least six degrees of freedom (DOF) for manipulator 14. However, manipulator 14 may have any number of degrees of freedom and any appropriate number of joints (J) and redundant joints (J). In one example, each joint (J) of manipulator 14 may be actively driven and electrically driven. In another example, each joint (J) may be passively driven. In yet another example, the joints (J) may include a combination of actively driven joints (J) and passively driven joints (J).

[0021] The base 16 of the manipulator 14 is generally the stationary part of the manipulator 14 during use, thereby providing a fixed reference coordinate system (i.e., a virtual zero position) for the other components of the manipulator 14 or the system 10 as a whole. Generally, the origin of the base coordinate system is defined at the fixed reference of the base 16. The base coordinate system may be referred to herein as the manipulator coordinate system MNPL, and the robot arm 18A is configured to support and move the end effector coupled to the robot arm 18A in the manipulator coordinate system MNPL. The fixed reference point of the base 16 may be defined with respect to any suitable part of the manipulator 14, for example, one or more of the links 18. Alternatively or additionally, the fixed reference point of the base 16 may be defined with respect to the manipulator carriage 17, for example, at the point where the manipulator 14 is physically attached to the carriage 17. In one example, the fixed reference point of the base 16 is defined at the intersection of the axes of joints J1 and J2. Therefore, although joints J1 and J2 are actually moving components, the intersection of the axes of joints J1 and J2 is nevertheless a virtual fixed reference point that does not move in the manipulator coordinate system MNPL. The manipulator 14 and / or manipulator carriage 17 house the manipulator computer 26 or other type of control unit. In another example, the manipulator 14 may be a handheld robotic device in which the base 16 is part of a surgical tool 22 that is grasped by a hand, and the surgical tool 22 is attached to the grasped part by one or more actuators that move the surgical tool 22 with one or more degrees of freedom relative to the grasped part.

[0022] System 10 may include an end effector 20 coupled to a robotic arm 18A. The end effector 20 may include any end effector suitable for a surgical procedure. In some cases, the end effector 20 may include a surgical tool 22 such that the surgical tool 22 is supported by the robotic arm 18A. The surgical tool 22 may be any instrument for working with the patient's anatomical structure A, such as a saw, cutting bur, router, reamer, impactor, ultrasonic aspirator, probe, scalpel, trocar, cutting tool, drill, dilator, screwdriver, intervertebral insulator, retractor, ablator, or intervertebral discectomy tool. In the case of Figure 1, the end effector 20 includes a surgical tool 22, shown as a drilling device. Additionally or alternatively, the end effector 20 may include accessories and / or energy applicators, such as saw blades, cutting burs, routers, reamers, impactors, ultrasonic aspirators, probes, scalpels, trocars, cutting tools, drills, dilators, screwdrivers, intervertebral inserters, retractors, ablators, and discectomy tools. The accessories and energy applicators may be integrated with the end effector 20 or mounted separately. In some cases, the end effector 20 may include a shaft 27, and the energy applicator may be located at the end of the shaft 27. The end effector 20 may include a cutting guide. As shown in Figure 1, the end effector 20 may include a tool holder, which may support any of the surgical tools 22 described above. The tool holder may be a guide tube for supporting the surgical tool 22. The surgical tool 22 may be temporarily fixed to the guide tube and / or slidable within the guide tube. The guide tube may also be a guide tube further described in U.S. Provisional Patent Application No. 63 / 612,011, entitled “Magnetic Spine Registration Tool,” which is incorporated herein by reference.In addition, the guide tube may be an anti-skiving guide tube as described in U.S. Provisional Patent Application No. 63 / 454,346, entitled “Anti-Skiving Guide Tube And Surgical System Including The Same,” which is incorporated herein by reference. Additionally or alternatively, the surgical tool 22 may be actively driven by the robotic manipulator 14 or be motorized. The surgical tool 22 may be handheld and may be selectively coupled to the robotic manipulator 14.

[0023] System 10 may include one or more tool trackers. The tool trackers may be temporarily coupled to the end effector 20. For example, the tool tracker may be a trackable array as described in U.S. Patent Application Publication 2022 / 0134569, entitled "Robotic Surgical System With Motorized Movement To A Starting Pose For A Registration Or Calibration Routine," the disclosure of which is incorporated herein by reference; or an end effector tracker as described in U.S. Patent No. 10,350,012, entitled "Method And Apparatus For Controlling A Haptic Device," the disclosure of which is incorporated herein by reference; or a tool tracker as described in U.S. Provisional Patent Application No. 63 / 612,011, entitled "Magnetic Spine Registration Tool," which is incorporated herein by reference. In other examples, the tool tracker may be attachable to or detachable from the end effector 20 and / or from any other component of the manipulator 14, for example, one or more links of the robot arm 18A, for example, the furthest link (J6) of the manipulator. For example, the tool tracker may include components similar to the tracker assembly described in U.S. Patent Application Publication No. 2023 / 0277256, entitled "Robotic System Including a Link Tracker," for attaching the tool tracker to the end effector 20 or any other component of the manipulator 14, the disclosure of which is incorporated herein by reference. For example, the tool tracker may be attachable to / detachable from the end effector 20 or any other component of the manipulator 14 using spring-biased latches, magnetic connections, snap-fit ​​connections using flexible elements, etc. In other examples, the tool tracker may be temporarily coupled to the end effector 20 via components of the end effector 20.For example, if the end effector 20 includes a surgical tool 22, the tool tracker may be coupled to the end effector 20 via the surgical tool 22. In another example, if the end effector 20 includes a guide tube, the tool tracker may be coupled to the end effector 20 via the guide tube. In addition, the system 10 may include multiple tool trackers. For example, if the end effector 20 includes a guide tube configured to support a surgical tool 22 temporarily fixed to the guide tube, a first tool tracker may be coupled to the guide tube, and a second tool tracker may be coupled to the surgical tool 22.

[0024] The tool tracker may be coupled to the end effector 20 such that the relationship between the tool tracker and the end effector 20 can be determined. For example, the tool tracker may include a reference surface configured to contact the end effector 20, such as the reference surface described in U.S. Provisional Patent Application No. 63 / 612,011, entitled "Magnetic Spine Registration Tool," which is incorporated herein by reference. Contact between the reference surface and the end effector 20 can indicate that the tool tracker is properly coupled to the end effector 20 such that the position of the tool tracker relative to the end effector 20 is fixed and the relationship between the tool tracker and the end effector 20 can be determined.

[0025] The tool tracker may include one or more reference markers FM. In some cases, the reference markers FM may be coupled to the components of the end effector 20 and / or the manipulator 14, formed integrally, or manually coupled. The reference markers FM may include any suitable shape. For example, the reference markers FM may include a rectangular parallelepiped or an elliptical shape. The reference markers FM may be an active tracking element or a passive tracking element.

[0026] As shown in Figure 1, system 10 further includes a navigation system 32. An example of the navigation system 32 is described in U.S. Patent No. 9,008,757, filed September 24, 2013, entitled “Navigation System Including Optical and Non-Optical Sensors,” which is incorporated herein by reference. The navigation system 32 may track the movement of various objects. Such objects include, for example, a manipulator 14, an end effector 20, and / or an anatomical structure A. The navigation system 32 tracks these objects in order to collect state information of one or more objects relative to the (navigation) localizer coordinate system LCLZ. Coordinates in the localizer coordinate system LCLZ may be transformed to the manipulator coordinate system MNPL and / or vice versa using the transformation and alignment techniques described in U.S. Provisional Patent Application No. 63 / 552,897 entitled "Systems and Method for Image Based Registration and Calibration", U.S. Provisional Patent Application No. 63 / 612,011 entitled "Magnetic Spine Registration Tool", and U.S. Patent Application No. 17 / 513,324 entitled "Robotic Surgical System with Motorized Movement to a Starting Pose for a Registration or Calibration Routine", which are incorporated herein by reference.

[0027] The navigation system 32 may include a cart assembly 34 that houses a navigation computer 36 and / or other types of control units. The navigation interface operably communicates with the navigation computer 36. The navigation interface includes one or more displays 38. The navigation system 32 may use one or more displays 38 to display a graphical representation of the relative state of the tracked object to the operator.

[0028] The navigation system 32 may depict visual representations of anatomical structures A and manipulators 14, and / or end effectors 20, for visual guidance of any of the techniques described. The visual representations may be actual (camera) images, virtual representations (e.g., 2D and / or 3D computer models), or any combination thereof. The visual representations may be presented on any display visible to the operator, such as the navigation system 32's display 38 or a head-mounted device display. The representations may be augmented reality, mixed reality, or virtual reality. In some embodiments, the display may be configured as an extended reality device configured to perform any of the graphic functions described herein. The extended reality device may be implemented by a handheld device (e.g., a tablet or smartphone) or a head-mounted device. The extended reality device may be configured to superimpose, overlay, or combine any of the computer-generated graphics described onto a real-world view in order to implement an extended reality, augmented reality, and / or mixed reality experience for the operator. The real-world view may be acquired directly by the operator's eyes, or it may be a video stream of the real world captured by one or more cameras of an extended reality device. If a head-mounted device is used, it may include a transparent lens positioned directly in front of the operator's eyes, or one or more display screens, to display computer-generated graphics against the real-world view.

[0029] The navigation system 32 also includes a navigation localizer 44 (hereinafter referred to as "localizer") coupled to the navigation computer 36. In one example, the localizer 44 is an optical localizer and includes a camera unit 46. The camera unit 46 has an outer housing 48 that houses one or more optical sensors 50.

[0030] The navigation system 32 may include one or more trackers that can be tracked by the localizer 44. In one example, the trackers include a tool tracker, a pointer tracker PT, one or more manipulator trackers 52, and / or one or more patient trackers 54, 56. In the example shown in Figure 1, the manipulator tracker 52 is attached to the distal flange of the robotic arm 18A. The manipulator tracker 52 may be fixed to any suitable component of the manipulator 14, in addition to a surgical tool, or otherwise, e.g., the base 16 (i.e., tracker 52B), or to any one or more links 18 or joints J of the manipulator 14. Additionally or alternatively, the manipulator tracker 52 may be fixed to a surgical drape or drape assembly, as described in U.S. Patent Application Publication No. 2023 / 0277256, entitled "Robotic System Including a Link Tracker," the disclosure of which is incorporated herein by reference. For example, the manipulator tracker 52 may be secured to a surgical drape or drape assembly via an elastic band or snap ring. The patient tracker may be secured to an anatomical structure A of patient 12. In the example shown in Figure 1, the first patient tracker 54 is securely secured to the femur of patient 12, and the second patient tracker 56 is securely secured to the tibia of patient 12. In this example, the patient trackers 54 and 56 are securely secured to the bone. The pointer tracker PT is securely secured to a pointer P used to register the anatomical structure A in the localizer coordinate system LCLZ. Those skilled in the art will understand that the trackers described herein may be secured to their respective components in any suitable manner.

[0031] As shown in Figure 1, the base tracker 52B may be coupled to the trolley 17 by an adjustable support arm 102. As shown, the base tracker 52B may be attached to one end of the adjustable support arm 102, and the other end of the adjustable support arm 102 may be attached to the trolley 17. The adjustable support arm 102 may be positioned and locked to place the base tracker 52B in a fixed position relative to the trolley 17. Examples of the base tracker 52B coupled to the adjustable support arm may be similar to those described in U.S. Patent Application No. 17 / 513,324, entitled "Robotic Surgical System With Motorized Movement To A Starting Pose For A Registration Or Calibration Routine," or U.S. Patent Application No. 18 / 198,938, entitled "Robotic System With Improved Configurations For Base Tracker," the entire contents of which are incorporated herein by reference. Alternatively or additionally, the base tracker 52B may be coupled to the robot arm 18A and may be movable with the robot arm 18A. For example, the base tracker 52B may include a plurality of (active or passive) tracking elements positioned on any number of links 18 of the manipulator 14. In this case, the base tracker 52B is formed from tracking geometry from various tracking elements that move with the movement of the robot arm 18A. An example of a base tracker 52B formed by optical markers positioned on the links 18 may be similar to that described in U.S. Patent Application No. 18 / 115964, entitled "Robotic System with Link Tracker," the entirety of which is incorporated herein by reference. Alternatively or additionally, the base tracker 52B may be fixed to a surgical drape or drape assembly, as described in U.S. Patent Application Publication No. 2023 / 0277256, entitled "Robotic System Including A Link Tracker," the disclosure thereof is incorporated herein by reference.For example, the base tracker 52B may be secured to a surgical drape or drape assembly via an elastic band or snap ring.

[0032] However, when optical localization is used, one or more trackers may include an active marker 58. The active marker 58 may include a light-emitting diode (LED). Alternatively, the trackers described herein may have a passive marker, such as a reflector that reflects light emitted from the camera unit 46. Other suitable markers not specifically described herein may be used.

[0033] The localizer 44 tracks the trackers and determines the state of one or more trackers corresponding to the state of the object attached to them. The localizer 44 provides the tracker states to the navigation computer 36. In one example, the navigation computer 36 determines the tracker states and communicates them to the manipulator computer 26. As used herein, the state of an object includes, but is not limited to, data defining the position and / or orientation of the tracked object, or its equivalent / derivative of the position and / or orientation. For example, the state may be the attitude of the object and may include linear data and / or angular velocity data, etc.

[0034] The drawing shows an example of the navigation system 32, but the navigation system 32 may have any other suitable configuration for tracking the manipulator 14 and the patient 12. The shown tracker configuration is provided simply as an example for tracking an object in the working space. Any number of trackers may be used and may be placed at locations or on objects not shown. In other examples, as described below, the localizer 44 may detect an object without any trackers attached to the object.

[0035] In one example, the navigation system 32 and / or localizer 44 are ultrasound-based. For example, the navigation system 32 may include an ultrasound imaging device coupled to a navigation computer 36. The ultrasound imaging device may be robotically controlled or handheld. The ultrasound imaging device images one of the aforementioned objects, e.g., the manipulator 14 and the patient 12, and generates a status signal to the controller 30 based on the ultrasound images. The ultrasound images may be from any ultrasound imaging modality. The navigation computer 36 may process the images in near real-time to determine the state of the objects. Ultrasound tracking may be performed without the use of a tracker fixed to the object being tracked. The ultrasound imaging device may have any suitable configuration and may differ from the camera unit 46 shown in Figure 1. An example of an ultrasonic tracking system may be similar to that described in U.S. Patent Application No. 15 / 999,152, filed on 16 August 2018, entitled "Ultrasound Bone Registration With Learning-Based Segmentation And Sound Speed ​​Calibration," the entire contents of which are incorporated herein by reference.

[0036] In another example, the navigation system 32 and / or localizer 44 are radio frequency (RF) based. For example, the navigation system 32 may include an RF transceiver coupled to the navigation computer 36. The manipulator 14 and patient 12 may include an RF emitter or transponder attached to them. The RF emitter or transponder may be passive or actively energized. The RF transceiver transmits an RF tracking signal and generates a status signal to the controller 30 based on the RF signal received from the RF emitter. The navigation computer 36 and / or controller 30 may analyze the received RF signal and associate a relative state with it. The RF signal may be of any suitable frequency. The RF transceiver may be placed in any suitable location to effectively use the RF signal to track an object. Furthermore, the RF emitter or transponder may have any suitable structural configuration which may differ significantly from the tracker shown in Figure 1.

[0037] In yet another example, the navigation system 32 and / or localizer 44 are electromagnetically based. For example, the navigation system 32 may include an EM transceiver coupled to the navigation computer 36. The manipulator 14 and patient 12 may include EM components attached thereto, such as any suitable magnetic tracker, electromagnetic tracker, inductive tracker, etc. The tracker may be passive or actively energized. The EM transceiver generates an EM field and generates a state signal to the controller 30 based on the EM signal received from the tracker. The navigation computer 36 and / or controller 30 may analyze the received EM signal and associate a relative state thereto. Again, such an example of the navigation system 32 may have a structural configuration different from the configuration of the navigation system 32 shown throughout the drawings.

[0038] In yet another example, the navigation system 32 and / or localizer 44 utilize a mechanical vision system including a vision camera coupled to the navigation computer 36. The mechanical vision system may include any suitable type of imaging modality for scanning a surface, such as structured light, multimodal (visible light / NIR), prenoptic (light field), ultrasound, spectroscopic imaging, LIDAR, photogrammetry, and SfM (structure from motion). The mechanical vision system may include a vision camera directly coupled to or positioned away from the manipulator 14. For example, the vision camera may be coupled to the camera unit of the navigation system, coupled to a surgical tool / pointer, included in a smartphone / tablet, positioned in a head-mounted device, or mounted on a multiview camera / boom. The vision camera may scan the target area, end effector, and any physical object in the target space. The physical object may have a shape represented by virtual object data stored by the navigation computer 36. Detected objects may include tools, obstacles, barriers, sensitive areas, anatomical features, trackers, etc. The vision camera and navigation computer 36 are configured to detect physical objects using image processing techniques such as point cloud generation, pattern, color, or shape recognition, edge detection, pixel analysis, neural network or deep learning processing, optical character recognition, and barcode detection. The navigation computer 36 can identify and track objects by comparing captured images with virtual object data. The mechanical vision system may operate in a trackerless manner. The tracker may or may not be coupled to the physical object. If a tracker is used, the mechanical vision system may also include an infrared detector for tracking the tracker and comparing the tracking data with mechanical vision data. In this case as well, such an example of a navigation system 32 may have a different structural configuration from the configuration of the navigation system 32 shown throughout the drawings.Examples of machine vision tracking systems may be similar to those described in U.S. Patent No. 9,603,665, entitled “Systems and Methods for Establishing Virtual Constraint Boundaries,” and / or similar to those described in U.S. Provisional Patent Application No. 62 / 698,402, entitled “Systems and Method for Image Based Registration and Calibration,” filed on 16 July 2018, the entirety of which is incorporated herein by reference.

[0039] The navigation system 32 and / or the optical localizer 44 may have any other suitable components or structures not specifically enumerated herein. Furthermore, any of the techniques, methods, and / or components described above with respect to the camera-based navigation system 32 shown throughout the drawings may be implemented or provided for any other example of the navigation system 32 described herein. For example, the navigation system 32 may utilize inertial tracking alone or any combination of tracking techniques.

[0040] System 10 may include one or more input / output devices 40, 42, 43. The input / output devices may receive input from an operator interacting with the input / output devices 40, 42, 43. In addition, as will be described in more detail below, the input / output devices may output tactile, audible, and / or visual feedback to the operator of the surgical system 10. In Figure 1, the first and second input / output devices 40, 42 are shown as interactive touchscreen displays, and the third input / output device 43 is shown as a foot switch including a user-operable foot pedal. In other cases, the input / output devices 40, 42, 43 may be any devices for receiving input from the operator and / or for outputting tactile, audible, and / or visual feedback to the operator. For example, the input / output devices 40, 42, 43 may include any one or more of the following: speakers, displays, smartphone devices, keyboards, mice, remote control devices, microphones (voice-activated), gesture control devices, head-mounted devices, etc. In addition, the input / output devices 40, 42, and 43 may be haptic devices as described in U.S. Patent No. 10,350,012, entitled “Method and Apparatus for Controlling a Haptic Device,” the disclosure of which is incorporated herein by reference.

[0041] In addition, the operator may interact with the input / output devices 40, 42, and 43 in any suitable manner, and the input / output devices 40, 42, and 43 are intended to be able to receive corresponding inputs. For example, the operator may interact with the input / output devices 40, 42, and 43 by pressing, holding, clicking, double-clicking, releasing, and / or performing any other suitable interaction with the inputs of the input / output devices 40, 42, and 43. More specifically, the operator may interact with the foot switch 43 in Figure 1 by pressing, holding, clicking, double-clicking, and / or releasing the user-operable foot pedal of the foot switch 43. Similarly, the operator may interact with the interactive touchscreen displays 40, 42 in Figure 1 by pressing, holding, clicking, double-clicking, swiping, and / or releasing parts of the graphical user interface of the interactive touchscreen displays 40, 42. If the input / output devices 40, 42, and 43 include a head-mounted device, the head-mounted device may be configured to receive input by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0042] II. Overview of the Controller

[0043] Referring to Figure 2, the system 10 includes one or more controllers 30 (hereinafter referred to as "controllers"). The controllers 30 include software and / or hardware for controlling the manipulator 14. The controllers 30 direct the movement of components of the manipulator 14, such as the robot arm 18A, and control the attitude (position and / or orientation) of the end effector 20 relative to the coordinate system of the robot arm 18A. In one example, as shown in Figure 1, the coordinate system of the robot arm 18A is the manipulator coordinate system MNPL, and the controllers 30 can control the robot arm 18A to support and move the end effector 20 in the manipulator coordinate system MNPL. The manipulator coordinate system MNPL has an origin located at any appropriate attitude relative to the manipulator 14. The axes of the manipulator coordinate system MNPL can also be arbitrarily selected. Generally, the origin of the manipulator coordinate system MNPL is defined by a fixed reference point on the base 16. An example of the manipulator coordinate system MNPL is described in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference.

[0044] As shown in Figure 2, the controller 30 further includes software modules. These software modules may be part of a computer program(s) that operates on the manipulator computer 26, the navigation computer 36, or a combination thereof, to process data and assist in the control of the system 10. The software modules include instructions stored in one or more non-temporary computer-readable media or memory to be executed by one or more processors of computers 26, 36, or a combination thereof. In addition, software modules for prompting and / or communicating with the operator may form part of a program(s) and include instructions stored in memory on the manipulator computer 26, the navigation computer 36, or a combination thereof.

[0045] Controller 30 includes a manipulator controller 60 for processing data to instruct the movement of the manipulator 14. In one example, as shown in Figure 1, the manipulator controller 60 is implemented on a manipulator computer 26. The manipulator controller 60 may receive and process data from a single source or multiple sources. Controller 30 further includes a navigation controller 62 for transmitting state data regarding anatomical structure A to the manipulator controller 60. The manipulator controller 60 receives and processes state data provided by the navigation controller 62 to instruct the movement of the manipulator 14. In one example, as shown in Figure 1, the navigation controller 62 is implemented on a navigation computer 36. The manipulator controller 60 or the navigation controller 62 may communicate the state of the patient 12 and the manipulator 14 to the operator by displaying 2D and / or 3D images of anatomical structure A and the manipulator 14 on one or more displays 38. The manipulator computer 26 or navigation computer 36 may use the display 38 to instruct the display of commands or to request information in order to interact with the operator and to direct the manipulator 14.

[0046] The controller 30, including the manipulator controller 60 and the navigation controller 62, may be implemented on any suitable one or more devices within the system 10, including but not limited to the manipulator computer 26, the navigation computer 36, and any combination thereof. As described herein, the controller 30 may include multiple controllers for various systems, components, or subsystems of the surgical system 10, and is not limited to a single controller. These controllers may communicate with other components of the surgical system 10 (e.g., directly or indirectly), and / or via physical electrical connections (e.g., tethered wiring harnesses), and / or via one or more types of wireless communication (e.g., WiFi® network, Bluetooth®, wireless network, etc.). Any of the controllers 30 may be implemented as or using various configurations such as computers, processors, control units, etc., and may include discrete components or be integrated (e.g., sharing hardware, software, inputs, outputs, etc.). Any of the one or more controllers may implement their respective functions using hardware only, software only, or a combination of hardware and software. Examples of hardware include, but are not limited to, single-core or multi-core processors, CPUs, GPUs, integrated circuits, microchips or ASICs, digital signal processors, microcontrollers, field-programmable gate arrays, systems on a chip, discrete circuits, and / or other suitable hardware. A controller(s) may implement software programs, software modules, algorithms, logic rules, lookup tables and other reference data, as well as various software layers, to implement any of the capabilities described herein. The software and hardware of controller 30, and their equivalents for connected peripherals, are entirely contrived.

[0047] a. Boundary generator

[0048] As shown in Figure 2, the controller 30 includes a boundary generator 66. The boundary generator 66 is a software module that can be implemented on the manipulator controller 60. Alternatively, the boundary generator 66 may be implemented on any other suitable component of the system 10, such as the navigation controller 62.

[0049] The boundary generator 66 may generate virtual constraints that represent the tracked anatomical structure of patient 12. Such virtual constraints may include virtual boundaries, virtual meshes, virtual objects, etc. For example, the boundary generator 66 may generate a virtual mesh that shows the shape, size, and location of patient 12's bones.

[0050] The boundary generator 66 may generate virtual constraints to constrain the manipulator 14 and / or the end effector 20. Such virtual constraints may include virtual boundaries, virtual meshes, virtual objects, etc. Such virtual constraints may correspond to any object detected by the navigation system 32. Virtual constraints may be defined for tracked anatomical structures of the patient 12. For example, the controller 30 may generate a 3D bone model registered in one or more patient trackers. The controller 30 may register the virtual constraints to the bone model in order to fix the virtual constraints to the bone model. Additionally or alternatively, virtual constraints may be defined for the surgical site and / or the end effector 20.

[0051] In some cases, the virtual constraint may be a particle guide. The particle guide may be generated by the boundary generator 66, and the controller 30 may generate a constraint force adapted to attract the surgical tool 22 toward or repel the particle guide. The constraint force is further described in U.S. Patent Application No. 17 / 701,989, entitled "Systems and Methods for Guiding Movement of a Tool," the disclosure of which is incorporated herein by reference.

[0052] In some cases, the virtual constraint may be a stereotactic interaction feature. A stereotactic interaction feature may be generated by the boundary generator 66 and defined relative to the end effector 20. For example, a stereotactic interaction feature may be generated at a location attributed to the surgical tool 22. These stereotactic interaction features may include any suitable geometric shape and any suitable size. For example, a stereotactic interaction feature may be a point and / or sphere having its own origin and radius. A stereotactic interaction feature may be generated relative to the end effector 20 such that the stereotactic interaction feature may collide with the virtual constraint instead of the end effector 20. Stereotactic interaction features are further described in U.S. Patent No. 11,944,396, entitled "Systems and Methods for Controlling Robotic Movement of a Tool Based on a Virtual Boundary," the disclosure of which is incorporated herein by reference.

[0053] The state of the manipulator 14 and / or end effector 20 may be tracked against a virtual constraint. For example, the state of the manipulator 14 and / or end effector 20 may be tracked to determine whether the surgical tool 22 is interacting with the virtual constraint. In one example, the state of the center point of the end effector 20 and / or surgical tool 22 is measured against the virtual constraint for the purpose of determining whether the surgical tool 22 is interacting with the virtual constraint. Furthermore, the boundary generator 66 may generate a virtual representation of the surgical tool 22 to determine whether the surgical tool 22 is interacting with the virtual constraint. Thus, in response to determining that the surgical tool 22 is interacting with the virtual constraint, the boundary generator 66 may apply haptic feedback force to the manipulator 14, or more specifically, the end effector 20.

[0054] A virtual constraint can be a temporary virtual constraint. A temporary virtual constraint can be defined as a virtual constraint that can be removed / deactivated by the controller 30. For example, in some cases, the controller 30 may remove / deactivate a temporary virtual constraint after a predetermined amount of time has elapsed. As another example, the controller 30 may remove / deactivate a temporary virtual constraint in response to the completion of a step in the surgical procedure. As yet another example, the controller 30 may remove / deactivate a temporary virtual constraint in response to an action by the operator of the surgical system 10. For example, the controller 30 may remove a temporary virtual constraint in response to the operator interacting with the input / output devices 40, 42, and 43.

[0055] As will be described in more detail below, the boundary generator 66 may be configured to generate virtual constraints during any of the methods described herein. For example, the boundary generator 66 may be configured to generate virtual constraints during method 100. In addition, virtual constraints may be generated before method 100.

[0056] In some embodiments, virtual constraints may be generated offline rather than on the manipulator computer 26 or navigation computer 36. The virtual constraints can then be utilized at runtime by the manipulator controller 60.

[0057] b. Tool path generator

[0058] As shown in Figure 2, the controller 30 includes a tool path generator 68. The tool path generator 68 is a software module that can be implemented on the manipulator controller 60. Alternatively, the tool path generator 68 may be implemented on any other suitable component of the system 10.

[0059] The tool pathway generator 68 generates pathways for the manipulator 14 and / or end effector 20 to traverse, such as removing a section of anatomical structure A to receive an implant. One exemplary system and method for generating tool pathways 70 is described in U.S. Patent No. 9,937,014, entitled "System and Method of Controlling a Surgical Tool During Autonomous Movement of the Surgical Tool," the disclosure of which is incorporated herein by reference.

[0060] The tool path generator 68 generates a tool path 70 for the surgical site, and the controller 30 controls the manipulator 14 to facilitate the movement of the end effector 20 and surgical tool 22 along the tool path 70 to treat the surgical site. One exemplary path 70 is shown in Figure 3. The tool path generator 68 determines the tool path 70 for the surgical tool 22 of the end effector 20 to move along in the manipulator coordinate system MNPL. Furthermore, the tool path generator 68 generates path data associated with the tool path 70 and communicates the path data to the manipulator controller 60. The tool path generator 68 may provide the manipulator controller 60 with preoperative and / or intraoperative path data so that the tool path 70 can be updated at any point during the surgical procedure.

[0061] The tool pathway 70 may be defined within or through the tissue of patient 12. For example, the tool pathway 70 may be defined to allow the surgical tool 22 to cut and remove a volume of bone so that the bone can accept an implant. The tool pathway 70 may correspond to a specific volume of bone to be cut to accept an implant of a particular shape (e.g., size and shape). In the case of Figure 4, the surgical site S is the tibia of patient 12, and the tool pathway 70 is shown as being defined on the surface of the tibia. In other cases, the tool pathway 70 may be defined within any other tissue of patient 12, such as any bone or soft tissue of patient 12.

[0062] The tool path 70 can be divided into one or more segments. For example, in Figure 3, the tool path 70 is divided into four segments 70a to 70d, each having a first endpoint 71 and a second endpoint 73. For each segment 70a to 70d of the tool path 70, the tool path generator 68 can process the data provided by the controller 30 to generate path segment data. Although only four segments 70a to 70d are shown and illustrated in Figure 3, those skilled in the art will understand that the tool path 70 can be divided into any suitable number of segments. For example, referring to Figure 5, the shown tool path 70 may include seven segments, where each straight section is a separate segment and each curved section is a separate segment. Each segment may be of similar length or of varying lengths.

[0063] The tool path generator 68 can generate multiple tool path points P along the tool path 70. These multiple tool path points P may be any points along the tool path 70, and each tool path point P corresponds to the target position of the surgical tool 22. In the case of Figure 3, the multiple tool path points P may include endpoints 71 and 73. In addition, the multiple tool path points P may include points P1 to P5, etc., located along each segment 70a to 70d. The tool path 70 may include any appropriate number of tool path points PP. In some cases, the controller 30 may include a tool path interpolator configured to determine the target position of the surgical tool 22 along the tool path 70 and the corresponding tool path points P. Examples of tool path interpolators may be similar to those described in U.S. Patent Application No. 18 / 082,996, entitled "Robotic Systems, Methods and Software Programs For Modifying Tool Operation Based on Tissue Parameters," and U.S. Patent No. 10,117,713, entitled "Robotic Systems and Methods for Controlling a Tool Removing Material from a Workpiece," the disclosures of which are incorporated herein by reference.

[0064] The tool path generator 68 may generate path data using any appropriate input. For example, the tool path generator 68 may generate path data using data provided by the controller 30 about the patient 12, the surgical tool 22, the implant, and / or any other object located at the surgical site S. The data input to the tool path generator 68 may include posture data of the femur and / or tibia, posture data of the surgical tool 22, posture data of other objects, imaging data (e.g., CT / MRI data), data defining the shape of the boundary to which the surgical tool 22 should not extend and / or data defining the volume of tissue to be removed by the surgical tool 22, implant data, and data regarding the setting of the boundary by the surgeon. As another example, the tool path generator 68 may generate path data based on the type of procedure and / or preference for the surgical procedure. For example, the tool path generator 68 may generate path data to optimize the cutting efficiency of the surgical tool 22 along the tool path 70, the cutting speed of the surgical tool 22 along the tool path 70, and / or to reduce the distance traveled by the surgical tool 22 while moving along the tool path 70.

[0065] In addition, the tool pathway 70 may be registered to the tracked anatomical structure A of patient 12. For example, referring to Figure 4, the tool pathway 70 may be registered to the tibia of patient 12. In such a case, the position of the tool pathway 70 is fixed to the tibia during patient 12's movement.

[0066] In some embodiments, the tool path 70 may be generated offline rather than on the manipulator computer 26 or navigation computer 36. The tool path 70 can then be made available at runtime by the manipulator controller 60.

[0067] c. Controller operating mode

[0068] The controller 30 may use various operating modes to control the manipulator 14. For example, the controller 30 may use semi-autonomous, automatic, manual, and guided manual operating modes to control the manipulator 14 / robot arm 18A to interact with the surgical site S.

[0069] In semi-autonomous and automatic modes, the controller 30 directs the movement of the robot arm 18A and / or end effector 20 in the surgical site S. In such cases, the controller 30 obtains a tool path 70 from the tool path generator 68 and controls the manipulator 14 to facilitate the movement of the tool along the tool path 70. More specifically, in semi-autonomous and automatic modes, the controller 30 may model the robot arm 18A and / or end effector 20 as virtual rigid bodies and determine forces and torques to apply to the virtual rigid bodies to advance and constrain the robot arm 18A and / or end effector 20 along the tool path 70. The movement of the tool 20 in semi-autonomous and automatic modes is constrained in relation to virtual constraints generated by the boundary generator 66 and / or path generator 68.

[0070] In semi-autonomous mode, the controller 30 is capable of moving the robot arm 18A and / or end effector 20 without operator assistance. Without operator assistance may mean that the operator does not physically move the robot arm 18A and / or end effector 20 by applying external force / torque to move the robot arm 18A and / or end effector 20. Instead, the operator may use some form of control to manage the start and stop of movement. For example, the operator may press and hold a control button to start the movement of the robot arm 18A and / or end effector 20 and release the button to stop the movement of the robot arm 18A and / or end effector 20. Alternatively, the operator may press a button to start the movement of the robot arm 18A and / or end effector 20 and press a button to stop the motorized movement of the robot arm 18A and / or end effector 20 along the tool path 70 generated by the tool path generator 68. The controller 30 uses electrically driven motion to advance the robotic arm 18A and / or end effector 20 according to pre-planned parameters. An example of a semi-autonomous mode is described in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference.

[0071] In manual mode and guided manual mode, the controller 30 is configured to control the movement of the robot arm 18A and / or end effector 20 based on external forces / torques applied to components of the manipulator 14, such as the robot arm 18A and / or end effector 20. The external forces / torques may also be applied to tactile devices, such as the tactile device described in U.S. Patent No. 10,350,012, entitled "Method and Apparatus for Controlling a Haptic Device," the disclosure of which is incorporated herein by reference.

[0072] In manual mode, the robot arm 18A is freely movable, and the operator can manually instruct the movement of the robot arm 18A and / or end effector 20 at the surgical site S, and the controller 30 can control this movement. For example, the operator may physically contact the components of the manipulator 14 and apply external force / torque to instruct the movement of the robot arm 18A and / or end effector 20. The movement of the robot arm 18A and / or end effector 20 in manual mode can also be constrained in relation to virtual constraints generated by the boundary generator 66 and / or tool path generator 68.

[0073] In guided manual mode, the system 10 guides the movement of the end effector 20 along a tool path 70 generated by the tool path generator 68 in response to external forces / torques applied to the components of the manipulator 14. In such cases, the controller 30 obtains the tool path 70 from the tool path generator 68 and controls the manipulator 14 to facilitate the movement of the tool along the tool path 70 in response to external forces / torques. For example, the operator may apply external forces / torques by physically contacting the components of the manipulator 14 to guide the movement of the end effector 20 along the tool path 70. For example, the operator may grasp the end effector 20 and / or surgical tool 22 and apply external forces / torques to move the end effector 20 along the tool path 70. The guided manual mode relies on an external force / torque applied to the manipulator 14 to advance the tool 20, but such advancement is actively controlled to follow the tool path 70 rather than simply mimicking the movement that would have occurred based on the applied external force / torque. Thus, during the guided manual mode, components of the end effector 20, such as the center point of the surgical tool 22, are constrained along the tool path 70. The guided manual mode is further described in U.S. Patent No. 11,564,761, entitled "Systems and Methods for Controlling Movement of a Surgical Tool Along a Predefined Path," the disclosure of which is incorporated herein by reference.

[0074] d. Orientation generator

[0075] As shown in Figure 2, the controller 30 includes an orientation generator 74. The orientation generator 74 is a software module that can be implemented on the manipulator controller 60. Alternatively, the orientation generator 74 may be implemented on any other suitable component of the system 10.

[0076] The orientation generator 74 can generate a preferred orientation of the end effector 20 and the surgical tool 22 with respect to a predetermined tool path 70. One exemplary system and method for generating a preferred orientation is described in U.S. Patent No. 9,937,014, entitled "System and Method of Controlling a Surgical Tool During Autonomous Movement of the Surgical Tool," the disclosure of which is incorporated herein by reference, where preferred orientation is referred to as acceptable orientation.

[0077] The orientation generator 74 can generate preferred orientations for the surgical tool 22 for each tool path point P generated by the tool path generator 68. For example, in Figure 3, the orientation generator 74 generates preferred orientations OR1 to OR5 corresponding to each of the tool path points P1 to P5. Additionally or alternatively, the orientation generator 74 can also generate a range of preferred orientations for the surgical tool 22 for each tool path point P.

[0078] Referring to Figure 3, while the controller 30 controls the manipulator 14 in semi-autonomous and automatic modes, the controller 30 can autonomously move the surgical tool 22 along the tool path 70 in orientations OR1 to OR5. The manipulator controller 68 can communicate with the tool path generator 68 and orientation generator 74 to instruct the manipulator 14 to autonomously move the surgical tool 22 along the tool path 70 in preferred orientations OR1 to OR5. In semi-autonomous mode, the controller 30 can autonomously move the surgical tool 22 along the tool path 70 in preferred orientations OR1 to OR5 without the operator handling the surgical tool 22 or without the operator's assistance. In addition, in guided manual mode, the controller 30 can constrain the surgical tool 22 so that it moves along the tool path 70 in orientations OR1 to OR5 in response to external forces / torques applied to the components of the manipulator 14.

[0079] The orientation generator 74 can generate a preferred orientation based on any preferred input. The orientation generator 74 may receive as input: posture data of the femur and / or tibia, posture data of other objects, imaging data (e.g., CT / MRI data), data defining virtual constraints to which the surgical tool 22 should not extend and / or data defining the volume of tissue to be removed by the surgical tool 22, implant data, data relating to the surgeon's setting of boundary locations, data relating to preferences for surgical procedures (e.g., optimization of cutting efficiency and / or optimization of cutting speed), data defining the location of the localizer 44, data defining the tool path 70 to which the surgical tool 22 should traverse, and / or the type of surgical procedure.

[0080] The orientation generator 74 may generate a preferred orientation using any preferred method. In some cases, the orientation generator 74 may generate a preferred orientation to optimize the cutting efficiency of the surgical tool 22 along the tool path 70. For example, if the surgical procedure is a bone milling procedure, the surgical tool 22 may have a preferred orientation or a range of preferred orientations to avoid cutting inefficiencies. The surgical tool 22 may cut more efficiently in a particular orientation compared to other orientations. Parameters relating to the cutting efficiency of the surgical tool 22 in various scenarios can be stored in memory and accessed by the orientation generator 74. The cutting efficiency of the surgical tool 22 can be determined according to any preferred method. In one exemplary embodiment when milling a femur F, the orientation generator 74 selects an orientation of the surgical tool 22 such that the orientation of the surgical tool 22 is parallel to the longitudinal axis of the femur F. In this embodiment, the orientation generator 74 receives data defining the tool path 70 and femoral orientation data, including the longitudinal axis orientation.

[0081] The manipulator controller 60 may communicate with an orientation generator 74 to determine a preferred orientation of the surgical tool 22 with respect to the tool path 70. The orientation generator 74 generates orientation data relating to the preferred orientation. For any given moment or tool path point P along the tool path 70, the preferred orientation may be a single preferred orientation or multiple preferred orientations. The orientation generator 74 may then communicate the orientation data relating to the preferred orientation of the surgical tool 22 to the manipulator controller 68. The manipulator controller 68 processes the orientation data and instructs the manipulator 14 to orient the surgical tool 22 as it moves autonomously along the tool path 70, as disclosed in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," which is incorporated herein by reference.

[0082] e. Input / Output Devices

[0083] The input / output devices 40, 42, and 43 may be coupled to the controller 30, which may detect inputs from the input / output devices 40, 42, and 43. For example, a detected input may be a detected operation of a foot switch 43, or a detected interaction with an interactive touchscreen display 40 or 42. A detected input may also be a gesture, gaze, voice, and / or head movement of the user detected by a head-mounted device. Thus, the operator may interact with the input / output devices 40, 42, and 43 to communicate with the software module shown in Figure 2. For example, the input / output devices 40, 42, and 43 may be actuated by the operator to input and / or select / control information to specific embodiments of the manipulator controller 60, manipulator computer 26, navigation controller 62, and / or navigation computer 36. The input / output devices 40, 42, and 43 may communicate with the controller 30 via user interface software. The user interface software may run on the manipulator computer 26 and the navigation computer 36, or it may run on a device separate from the manipulator computer 26 and the navigation computer 36.

[0084] The controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. The controller 30 may provide tactile, auditory, and / or visual feedback to the operator in order to provide guidance to the operator based on virtual restraints and / or to provide notifications / instructions to the operator. The controller 30 may implement tactile, auditory, and / or visual feedback on any suitable component of the surgical system. For example, the controller 30 may implement tactile feedback on one or more of the input / output devices 40, 42, 43, the end effector 20, the robotic arm 18A, and / or handheld robotic devices such as a handheld robotic saw, drill, or tool guide. As another example, the controller 30 may implement tactile feedback via a tactile device such as the tactile device described in U.S. Patent No. 10,350,012, entitled "Method and Apparatus for Controlling a Haptic Device," the disclosure of which is incorporated herein by reference. The controller 30 may implement auditory feedback via speakers of the surgical system 10, such as the speakers of input / output devices 40, 42, and 43. The controller 30 may implement visual feedback via displays of the surgical system 10, such as the displays 38 of input / output devices 40 and 42, and / or the display of the head-mounted device.

[0085] Furthermore, tactile, auditory, and / or visual feedback may be any preferred feedback means. For example, tactile feedback may be a vibration pattern or a button click sensation provided to a component of the surgical system, such as one or more of the input / output devices 40, 42, and 43. Auditory feedback may be any sound provided by a component of the surgical system, such as one or more of the input / output devices 40, 42, and 43. For example, the sound may be a button click sound. Visual feedback may be any notification and / or 2D / 3D image provided by a component of the surgical system, such as one or more of the input / output devices 40, 42, and 43. For example, the notification and / or 2D / 3D image may be provided by a display of the surgical system 10, such as the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device.

[0086] III. Reorientation of Surgical Tools

[0087] a. Overview

[0088] The controller 30 can determine whether the surgical tool 22 has been reoriented. Reoriented can be defined as a change in the orientation of the surgical tool 22. In some cases, the operator may apply an external force to the components of the manipulator 14 to reoriented the surgical tool 22. In some cases, reoriented the surgical tool 22 may occur even without the operator applying an external force. For example, a force may be applied to the surgical tool 22 inadvertently, causing it to reoriented. In one such case, the surgical tool 22 may collide with an obstacle, and the obstacle may apply an external force to the surgical tool 22, causing it to reoriented.

[0089] Reorientation of the surgical tool 22 can prevent collision between the surgical tool 22 and an obstacle located within the surgical site S. The obstacle can be any obstacle that hinders the surgical procedure. For example, the obstacle may be a retractor located within the surgical site S, another surgical tool 22 located within the surgical site S, the arm of a surgical assistant, or the tissue of the patient 12. Referring to Figure 4, for example, the surgical tool 22 may collide with the retractor R while moving along a predetermined tool path 70 in automatic mode, semi-autonomous mode, or guided manual mode. In such a case, the surgical tool 22 may be reoriented to prevent collision between the surgical tool 22 and the retractor R. Such an obstacle may not have been detected by the navigation system 32 and therefore may not have been considered during the generation of the predetermined tool path 70. For example, the obstacles and / or trackers 52, 54, and 56 may not have been located within the line of sight of the localizer 44, and the boundary generator 66 may not have generated virtual constraints corresponding to the obstacles and / or objects coupled to the trackers 52, 54, and 56. As another example, the position of such obstacles may have changed after the tool path 70 was generated and / or while the end effector 20 was moving along the tool path 70.

[0090] The reorientation of the surgical tool 22 may also be based on the operator's mobility / ergonomics and / or operator preference. In such cases, the surgical tool 22 may be reoriented to allow the surgical tool 22 to move according to the operator's desires and / or ability. For example, in guided manual mode, the controller 30 may constrain the surgical tool 22 so that it moves along the tool path 70 in a preferred orientation in response to external forces / torques applied to the components of the manipulator 14. However, in some cases, such a preferred orientation may not be ergonomically suitable because it may place unnecessary stress on the operator. In such cases, the surgical tool 22 may be reoriented to provide an orientation that is more ergonomically suitable for the operator. As another example, the operator may have a preferred wrist position and / or hand preference (e.g., left hand or right hand) for grasping the end effector 20 and applying external forces / torques. In such cases, the surgical tool 22 may be reoriented to allow the operator to grasp the end effector 20 and / or the surgical tool 22 using a preferred wrist position and / or hand preference.

[0091] Reorientation of the surgical tool 22 may occur due to limitations of the manipulator. For example, the controller 30 may be prevented from moving along the tool path 70 in a preferred orientation due to one or more of the following: the working space boundary of the manipulator 14, the range of motion of the manipulator 14, the limits of the joints (J) of the manipulator 14, and singularities of the manipulator 14. For example, the range of motion of the manipulator 14 may be limited at a point along the tool path 70, preventing the surgical tool 22 from achieving a preferred orientation at that point. In such cases, the surgical tool 22 may be reoriented to allow the manipulator 14 to continue moving along the tool path 70 despite the limitation of the range of motion of the manipulator 14 at that point along the tool path 70.

[0092] Referring to Figure 5, Method 100 for facilitating the reorientation of a surgical tool 22. Method 100 includes the steps of: controlling a manipulator 14 to facilitate the movement of a surgical tool 22 for treating a surgical site S; detecting an initial reorientation of the surgical tool 22 during its movement; generating a virtual constraint in 106; and facilitating a subsequent reorientation of the surgical tool 22 during its subsequent movement using the generated virtual constraint in 108. In this specification, “initial reorientation” is used to describe the reorientation detected in step 104. In addition, “subsequent reorientation” is used herein to describe the reorientation that occurs as a result of the virtual constraint interacting with the surgical tool 22 during step 108.

[0093] During step 102, the controller 30 may control the manipulator 14 to facilitate the movement of the surgical tool 22 for treating the surgical site S. The controller 30 may facilitate the movement of the surgical tool 22 according to one of the above-described operating modes of the controller 30, such as automatic mode, semi-autonomous mode, manual mode, or guided manual mode. When the controller 30 controls the manipulator 14 in automatic mode, semi-autonomous mode, or guided manual mode, the controller 30 obtains a tool path 70 from the tool path generator 68 and controls the manipulator 14 to facilitate the movement of the tool along the tool path 70 for treating the surgical site S.

[0094] During step 104, the controller 30 detects the initial reorientation of the surgical tool 22. The initial reorientation may be a manual reorientation in response to an external force / torque applied to the end effector 20 and / or the surgical tool 22 by the operator / user. In such a case, the operator may apply an external force to the end effector 20 and / or the surgical tool 22 to cause the initial reorientation of the surgical tool 22. Initial reorientation may also be caused by a collision with an obstacle. In such a case, the obstacle may apply an external force to the end effector 20 and / or the surgical tool 22 to cause the initial reorientation of the surgical tool 22.

[0095] An example of initial reorientation of the surgical tool 22 is shown in Figures 6 and 7. As shown, the surgical tool 22 is coupled to the robotic arm 18A of the manipulator 14, and an obstacle (indicated as a retractor R) is located within the path of the surgical tool 22. In Figure 6, the surgical tool 22 moves along and is constrained to the tool path 70 in one of the following modes: automatic, semi-autonomous, or guided manual. The tool path 70 in Figure 6 includes tool path points P1, P2 and corresponding predetermined orientations OR1, OR2. In the case of Figure 6, initial reorientation occurs at point P2 on the tool path 70, where an external force F is applied to the surgical tool 22. The external force F reorients the surgical tool 22 to orientation ROR, preventing the surgical tool 22 from being oriented to orientation OR2, and preventing the surgical tool 22 from colliding with the retractor R. In Figure 7, the manipulator 14 controls the movement of the surgical tool 22 in manual mode. In other words, the surgical tool 22 does not move along the tool path 70. Instead, the manipulator 14 controls the movement of the surgical tool 22 based on an external force provided by the operator. As shown, the surgical tool 22 moves from point P1 to point P2. Initial reorientation occurs at point P2, where an external force F is applied to the surgical tool 22, causing it to move from point P1 to point P2, reorienting the surgical tool 22 from orientation OR to orientation ROR, and preventing the surgical tool 22 from colliding with the retractor R.

[0096] In some cases, during step 104, the controller 30 may also detect the location where the initial reorientation occurred. As will be described in more detail below, the controller 30 may detect the location where the initial reorientation occurred by detecting its position relative to the tool path 70, the surgical site S, and / or obstacles to be avoided.

[0097] During step 104, the controller 30 may detect initial reorientation during any of the operating modes of the controller 30 described above. For example, the controller 30 may detect initial reorientation during the movement of the surgical tool 22 along a predetermined tool path 70 while the controller 30 controls the manipulator 14 in automatic mode, semi-autonomous mode, or guided manual mode. Alternatively, the controller 30 may detect initial reorientation during the movement of the surgical tool 22 while the controller controls the manipulator 14 in manual mode.

[0098] In step 106, the controller 30 generates virtual constraints in response to detecting the initial reorientation of the surgical tool 22 during step 104. Specifically, virtual constraints may be generated by the boundary generator 66. The virtual constraints generated by the boundary generator 66 may be any preferred type of virtual constraint, such as a virtual mesh, particle guides, and / or stereotactic interaction features.

[0099] Therefore, the initial reorientation of the surgical tool 22 in step 104 may cause the controller 30 to generate a virtual constraint in step 106. Advantageously, if no obstacle is detected by the navigation system 32, the surgical tool 22 may be reoriented to avoid the undetected obstacle, the reorientation may be detected in step 104, and the controller 30 may generate a virtual constraint corresponding to the undetected obstacle in step 106. For example, if the surgical tool 22 is moving along a predetermined tool path 70, the surgical tool 22 may be reoriented to avoid an obstacle, and the controller 30 may generate a virtual constraint corresponding to the obstacle. Similarly, if the surgical tool 22 is moving in manual mode, the surgical tool 22 may be reoriented to avoid an obstacle, and the controller 30 may generate a virtual constraint corresponding to the obstacle.

[0100] During step 108, the virtual constraint facilitates the reorientation of the surgical tool 22 in response to the interaction between the surgical tool 22 and the virtual constraint during subsequent (e.g., later or future) movements of the surgical tool 22. In some cases, the virtual constraint generated during step 106 may be generated at the detected initial reorientation position. During subsequent movements of the surgical tool 22, the surgical tool 22 may be repositioned at or near the detected initial reorientation position. In such cases, the surgical tool 22 may interact with the virtual constraint generated during step 106, and the generated virtual constraint may facilitate the subsequent reorientation of the surgical tool 22. For example, if the controller 30 controls the manipulator 14 in automatic mode, semi-autonomous mode, or guided manual mode, the controller 30 may control the surgical tool 22 during subsequent movements to continue moving along the same tool path 70, to resume moving along the same tool path 70, or to move along a different tool path. Such movement may cause the surgical tool 22 to be repositioned to or near the detected initial repositioning position. When the controller 30 controls the manipulator 14 in manual mode, the controller 30 may also control the surgical tool 22 based on the external force / torque provided by the operator, thereby causing the surgical tool 22 to be repositioned to or near the detected initial repositioning position.

[0101] Steps 102-108 of method 100 for facilitating reorientation can be repeated any preferred number of times. For example, an instance of step 104 for detecting the initial reorientation of the surgical tool 22 may occur each time an external force / torque is applied to the surgical tool 22. Similarly, an instance of step 106 for generating a virtual constraint may occur each time the initial reorientation of the surgical tool 22 is detected during an instance of step 104, and an instance of step 108 for facilitating the reorientation of the surgical tool 22 may occur each time the virtual constraint generated during an instance of step 106 interacts with the surgical tool 22.

[0102] In one such embodiment, the operator may apply a first external force to the surgical tool 22, and the controller 30 may detect a first initial reorientation during the first instance of step 104. Next, the controller 30 may generate a first virtual constraint during the first instance of step 106. Then, during the first instance of step 108, the first virtual constraint may interact with the surgical tool 22 during its subsequent movement to facilitate a first subsequent reorientation of the surgical tool 22. In addition, the operator may apply a second external force to the surgical tool 22, and the controller 30 may detect a second initial reorientation during the second instance of step 104. Next, the controller 30 may generate a second virtual constraint during the second instance of step 106. Then, during the second instance of step 108, the second virtual constraint may interact with the surgical tool 22 during its subsequent movement to facilitate a second subsequent reorientation of the surgical tool 22. Each of the first and second initial reorientations may occur during the movement of the surgical tool 22 along the tool path 70, or during the movement of the surgical tool 22 in manual mode.

[0103] b. Controller control scheme

[0104] As described above, an external force / torque may be applied to the end effector 20 and / or the surgical tool 22 to cause initial reorientation of the surgical tool 22. The controller 30 may control the manipulator 14 / robot arm 18A based on the external force / torque to implement the initial reorientation. Furthermore, the controller 30 may control the manipulator 14 / robot arm 18A to implement the initial reorientation using any preferred control scheme, such as an impedance control scheme or an admittance control scheme.

[0105] When the controller 30 controls the manipulator 14 based on an impedance control scheme, the controller 30 allows external forces / torques applied to the end effector 20 and / or surgical tool 22 to change the posture of the manipulator 14. In one such embodiment, the joints (J) of the robot arm 18A may be passively driven so that external forces / torques cause displacement of the robot arm 18A and change the posture of the manipulator 14. In some cases, the changed posture may be a posture in which the manipulator 14 interacts with a virtual constraint. Therefore, the controller 30 may determine the changed posture of the manipulator 14 in order to determine whether the changed posture of the manipulator 14 will result in interaction with a virtual constraint. For example, the controller 30 may determine the changed posture of the manipulator 14 by determining the position of the joints (J) of the robot arm 18A. The controller 30 may then command reaction forces / torques based on the changed posture of the manipulator 14. The commanded reaction force / torque may include a restraining force applied by a virtual restraint to constrain the surgical tool 22. The commanded reaction force / torque may be applied to the joint (J) of the robot arm 18A.

[0106] When the controller 30 controls the manipulator 14 based on an admittance control scheme, the controller 30 may detect external forces / torques applied to the end effector 20 and / or surgical tool 22 and control the movement of the manipulator 14 based on the detected forces / torques. In one such embodiment, the manipulator 14 may include a force / torque sensor capable of detecting external forces / torques. In another such embodiment, the manipulator 14 may include a current sensor capable of detecting the current supplied to the motor of the manipulator 12 to detect external forces / torques. Upon detecting an external force / torque, the controller 30 may command the posture of the manipulator 14 based on the detected external force / torque (for example, the controller 30 may command the posture of joint (J) of the robot arm 18A). In addition, the controller 30 may command the posture of the manipulator 14 based on the constraint forces applied by the virtual constraint. For example, in some cases, the external force / torque will displace the manipulator 14 to a posture in which it interacts with the virtual constraint. In such cases, the controller 30 can determine the commanded posture based on the external force / torque and the restraining force applied by the virtual restraint.

[0107] The impedance and admittance control scheme is further described in U.S. Patent No. 10,350,012, entitled “Method and Apparatus for Controlling a Haptic Device,” and U.S. Patent No. 10,327,849, entitled “Robotic System and Method For Backdriving The Same,” the disclosures thereof are incorporated herein by reference.

[0108] c. Detection of reorientation

[0109] The controller 30 may detect initial reorientation during step 104 by detecting parameters of the end effector 20 after the occurrence of initial reorientation. The parameters may be any suitable parameters for detecting the occurrence of initial reorientation. For example, the parameters may be one or more of the following: the direction of the external force / torque applied to the surgical tool 22, the magnitude of the external force / torque applied to the surgical tool 22, the displacement of the surgical tool 22, the velocity of the surgical tool 22, and the acceleration of the surgical tool 22. Referring to Figures 6 and 7, the operator applies an external force F to the surgical tool 22 at point P2 to cause initial reorientation. The controller 30 may detect initial reorientation by detecting the direction of the external force F, the magnitude of the external force F, the displacement of the surgical tool 22 caused by the external force F, the velocity of the surgical tool 22 during displacement, and / or the acceleration of the surgical tool 22 during displacement.

[0110] The controller 30 may detect initial reorientation during step 104 by determining whether the detected parameters of the surgical tool 22 are greater than a threshold. For example, the controller 30 may determine whether the direction of the external force F in Figures 6 and 7 is greater than an angle threshold. In other cases, the controller 30 may additionally or alternatively determine whether the magnitude of the external force F is greater than a magnitude threshold. Similarly, the controller 30 may additionally or alternatively determine whether the displacement of the surgical tool 22 is greater than a displacement threshold, whether the velocity of the surgical tool 22 is greater than a velocity threshold, and / or whether the acceleration of the surgical tool 22 is greater than an acceleration threshold.

[0111] The thresholds described above can be determined by the controller 30. In some cases, the controller 30 may determine the thresholds based on the orientation of the surgical tool 22 and the anatomical structure A of the patient 12, both of which can be tracked by the navigation system 32. For example, the controller 30 may estimate a threshold for the magnitude of the external force F required to excise anatomical structure A at the tracked position of the surgical tool 22. If the external force F is greater than the magnitude threshold, the controller may determine that the external force F is greater than the force required to excise anatomical structure A at the tracked position and that initial reorientation has occurred. In this way, the controller 30 may also estimate thresholds for the angle, displacement, velocity, and / or acceleration of the surgical tool 22.

[0112] The controller 30 may detect initial reorientation during step 104 by determining whether the surgical tool 22 has deviated from the tool path 70. In such cases, an obstacle may prevent the surgical tool 22 from moving along the tool path 70. For example, a retractor R may be directly located on the tool path 70 and obstruct the surgical tool 22 from moving along the tool path 70. In such cases, the surgical tool 22 may be reoriented in a way that displaces it from the tool path 70. The controller 30 may be configured to detect such initial reorientation by detecting the position of the surgical tool 22 relative to the tool path 70. In some cases, the controller 30 may detect initial reorientation by determining whether the amount of displacement of the surgical tool 22 from the tool path 70 is greater than a displacement threshold.

[0113] The controller 30 may detect initial reorientation during operation based on an impedance control scheme. In such cases, an external force / torque applied to the manipulator 14 may move the surgical tool 22 to its current position. The controller 30 may then detect initial reorientation based on the current position. In some cases, the controller 30 may detect initial reorientation by comparing the detected current position with one or more previous positions of the surgical tool 22. For example, the controller 30 may determine the displacement of the surgical tool 22 based on the current position and the previous positions of the surgical tool 22.

[0114] The controller 30 may detect initial reorientation during operation based on an admittance control scheme. In such cases, the controller 30 may detect an external force / torque applied to the manipulator 14 and determine the commanded position of the surgical tool 22. The controller 30 may then move the surgical tool 22 from its current position to the commanded position. In such cases, the controller 30 may detect initial reorientation based on the commanded position. In some cases, the controller 30 may detect initial reorientation by comparing the commanded position with the current position and / or one or more previous positions of the manipulator 14. For example, the controller 30 may determine the displacement of the surgical tool 22 based on the commanded position and the current position of the manipulator 14. In some cases, the controller 30 may detect parameters of the surgical tool 22. For example, the controller 30 may sense the velocity of the surgical tool 22 while it is being moved to the commanded position.

[0115] In some embodiments, the surgical system 10 may include a user interface configured to receive input, and the controller 30 may detect the reorientation of the surgical tool 22 based on the user interface receiving input. The user interface may be any of the input / output devices 40, 42, or 43 described above.

[0116] In exemplary embodiments, the controller 30 may determine that the magnitude of the force applied to the surgical tool 22 exceeds a force threshold and provide the operator with a potential reorientation instruction via one or more of the interactive touchscreen displays 40, 42. In some cases, the controller 30 may notify the operator of the reorientation and request confirmation from the operator via the displays of the surgical system 10, e.g., one or more of the interactive touchscreen displays 40, 42, displays 38 and / or the display of the head-mounted device, before determining that reorientation has been detected. Where the input / output devices 40, 42, 43 include a head-mounted device, the head-mounted device may be configured to receive confirmation by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0117] In another exemplary embodiment, the button may be located on the surgical tool 22, and the button can be pressed when the operator wishes to manually reorient the surgical tool 22. For example, the button may be located on the end effector 20 and / or the surgical tool 22. In some cases, the button may be a foot switch 43. The button may also be located on a handheld pendant device, as shown and described in U.S. Patent No. 9,119,655, titled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference. Pressing the button indicates that the operator wishes to reorient the surgical tool 22, and the controller 30 may detect the reorientation of the surgical tool 22 based on the button press.

[0118] The controller 30 may notify the operator of the detection of initial reorientation. In response to the detection of initial reorientation, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, in response to the detection of initial reorientation, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43. As another example, in response to the detection of initial reorientation, the controller 30 may provide visual notification via the display of the surgical system 10. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device. In some cases, in response to the detection of initial reorientation, the controller 30 may request confirmation of the initial reorientation from the operator. The controller 30 may request confirmation via the display of the surgical system 10. If the input / output devices 40, 42, and 43 include a head-mounted device, the head-mounted device may be configured to receive confirmation by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0119] d. Generation of virtual constraints

[0120] The virtual constraints generated by the controller 30 during step 106 may be any suitable type of virtual constraint, such as a virtual mesh, particle guides, and / or positional interaction features.

[0121] The virtual constraints generated by the boundary generator 66 may be generated for the initial reorientation position of the surgical tool 22. In such cases, the controller 30 may detect the initial reorientation position of the surgical tool 22 during step 104 and generate virtual constraints for the detected position during step 106. For example, referring to Figures 6 and 7, the controller 30 may detect that the initial reorientation of the surgical tool 22 occurs at point P2, and the controller 30 may generate virtual constraints at point P2.

[0122] More specifically, the controller 30 may detect the initial reorientation position relative to the tool path 70, relative to the surgical site S, and / or relative to obstacles to be avoided. In the case of Figure 6, the surgical tool 22 moves along the tool path 70 and is reoriented at point P2 along the tool path 70. The controller 30 can detect the position of point P2 relative to the tool path 70. Additionally, the controller 30 can detect the position of point P2 relative to the surgical site S. If the boundary generator 66 identifies obstacles to be avoided, the controller 30 can detect the position of point P2 relative to the obstacles to be avoided. In the case of Figure 7, the surgical tool 22 does not move along the tool path 70. Therefore, the controller 30 can detect the position of point P2 relative to the surgical site S, since point P2 is a position within the surgical site S. Additionally, if the boundary generator 66 identifies obstacles to be avoided, the controller 30 can detect the position of point P2 relative to the obstacles to be avoided.

[0123] Figures 8 to 10 illustrate the case of step 106, in which the controller 30 generates a virtual constraint in response to the detection of the initial reorientation of the surgical tool 22. As previously mentioned, Figures 6 and 7 illustrate the case where the surgical tool 22 is reoriented at point P2 and the controller 30 detects the initial reorientation. The retractor R is not shown in Figures 8 to 10 for simplification.

[0124] In the cases shown in Figures 8 to 10, the initial reorientation occurs at point P2. Therefore, the controller 30 detects the position of the initial reorientation as the position of point P2. Additionally, in the cases shown in Figures 8 to 10, the controller 30 operates in automatic, semi-autonomous, or guided manual mode, moving the surgical tool 22 along the tool path 70. While moving along the tool path 70, the controller 30 generates a virtual constraint at point P2, which is the detected position of the initial reorientation. In other cases, the controller 30 may generate a virtual constraint at the detected position of the initial reorientation while the controller 30 is operating in manual mode.

[0125] In the case of Figure 8, the controller 30 generates a virtual mesh VM. When the controller 30 generates a virtual mesh, the controller 30 may generate the virtual mesh relative to the detected initial reorientation location. In the case of Figure 8, the controller 30 generates the virtual mesh VM adjacent to point P2, which is the detected initial reorientation location. In other cases, the virtual mesh may be generated at any appropriate distance from the detected initial reorientation location. For example, the virtual mesh VM may be generated at point P2. In addition, the virtual mesh VM may include any appropriate shape and size. For example, in the case of Figure 8, the virtual mesh VM includes a spherical shape. In other cases, the virtual mesh VM may include a planar shape of any appropriate size, or any appropriate polygonal shape of any appropriate size.

[0126] In the case of Figure 9, the controller 30 generates a stereotactic interaction feature SF. When the controller 30 generates a stereotactic interaction feature, the controller 30 may generate the stereotactic interaction feature with respect to the surgical tool 22, with respect to the detected initial reorientation position and / or virtual constraints. In the case of Figure 9, the controller 30 generates a stereotactic interaction feature SF with respect to the surgical tool 22. As shown, the stereotactic interaction feature SF contacts the shaft 27 of the surgical tool 22.

[0127] When the controller 30 generates stereotactic interaction features, the controller 30 may generate two or more stereotactic interaction features. For example, the controller 30 may generate stereotactic interaction features for the surgical tool 22, for the detected initial reorientation position, and / or for the virtual constraint. In one such case, the controller 30 may generate stereotactic interaction features for the surgical tool 22 and for the detected initial reorientation position.

[0128] Stereotactic interaction features can be generated relative to the surgical tool 22, the detected initial reorientation position, and / or virtual constraints, thereby being at any appropriate distance from the surgical tool 22, the detected initial reorientation position, and / or virtual constraints. For example, the stereotactic interaction feature SF in Figure 9 may be generated adjacent to the energy applicator of the surgical tool 22 without contacting the surgical tool 22.

[0129] The localized interaction feature can include any appropriate shape and size. For example, in the case of Figure 9, the localized interaction feature SF includes a spherical shape. However, in other cases, the localized interaction feature SF can include any appropriate polygonal shape of any appropriate size.

[0130] In Figure 10, the controller 30 generates a particle guide having a restraining force applied to the surgical tool 22. Specifically, in Figure 10, the controller 30 is illustrated as generating a first particle guide PG1 having a corresponding repulsive force FR and a second particle guide PG2 having a corresponding attractive force FA. The controller 30 may generate any appropriate number of particle guides having corresponding repulsive or attractive forces. For example, the controller 30 may generate a single particle guide having a corresponding repulsive force.

[0131] When the controller 30 generates particle guides, the controller 30 may generate particle guides relative to the detected initial reorientation location. In the case of Figure 10, the controller 30 generates particle guides PG1 and PG2 adjacent to point P2, which is the detected initial reorientation location. The particle guides may be located at any appropriate distance from the detected initial reorientation location. In addition, the controller 30 may generate particle guides relative to the detected initial reorientation location based on the magnitude of the corresponding repulsive or attractive force. For example, the controller 30 may generate particle guides at a distance from the detected initial reorientation location, where this distance corresponds to the magnitude of the repulsive or attractive force.

[0132] The virtual constraints generated by the controller 30 can be activated or deactivated. For example, a virtual constraint generated during step 106 may be activated when the surgical tool 22 is within proximity to the initial reorientation location, and this proximity can be any suitable distance. For example, the stereotactic interaction feature SF in Figure 9 may be activated by the controller when the surgical tool 22 is within proximity to the detected initial reorientation location, and deactivated when the surgical tool 22 is not within proximity to the detected initial reorientation location. Advantageously, in such cases, the virtual constraint generated by the controller 30 interacts with the surgical tool 22 when it is within proximity to the detected initial reorientation location, and does not interact with the surgical tool 22 when it is not within proximity to the detected initial reorientation location. As another example, transient virtual constraints can be activated or deactivated.

[0133] The controller 30 may notify the operator of the generation of a virtual restraint. In response to the generation of a virtual restraint, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, in response to the generation of a virtual restraint, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43. As another example, the controller 30 may provide a visual notification via the display of the surgical system 10 in response to the generation of a virtual restraint. The controller 30 may present a 2D / 3D image of the virtual restraint to provide the visual notification. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of a head-mounted device. In some cases, the controller 30 may request confirmation from the operator of the generation of a virtual restraint. The controller 30 may request confirmation via the display of the surgical system 10. If the input / output devices 40, 42, and 43 include a head-mounted device, the head-mounted device may be configured to receive confirmation by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0134] e. Interaction between virtual restraints and surgical tools

[0135] In step 108, the virtual constraint generated in step 106 facilitates the subsequent reorientation of the surgical tool 22. Specifically, the virtual constraint facilitates the subsequent reorientation of the surgical tool 22 in response to the interaction between the surgical tool 22 and the virtual constraint at the detected reorientation location during the subsequent movement of the surgical tool 22. As previously mentioned, during the subsequent movement of the surgical tool 22, the surgical tool 22 may be relocated to or near the detected initial reorientation location. Therefore, if a virtual constraint is generated in step 106 and the surgical tool 22 is relocated to or near the detected initial reorientation location, the generated virtual constraint interacts with the surgical tool 22 and can reorient the surgical tool 22.

[0136] Generally, a virtual constraint can interact with the surgical tool 22 to produce movement of the surgical tool 22 that mimics an initial reorientation. For example, referring to Figures 6 and 7, the surgical tool 22 is initially reoriented to orientation ROR. Thus, the virtual constraint generated during step 106 interacts with the surgical tool 22 to subsequently reorient the surgical tool 22 to orientation ROR. The virtual constraint can reorient the tool by producing movement of the surgical tool 22 that mimics the force applied to the surgical tool 22 during the initial reorientation, the velocity of the surgical tool 22, and / or the acceleration of the surgical tool 22. In one such case, the virtual constraint can apply a constraining force to the surgical tool 22 based on an external force applied to the surgical tool 22 during the initial reorientation. In particular, the constraining force can mimic the magnitude and direction of an external force applied to the surgical tool 22 during the initial reorientation.

[0137] Figures 8 to 10 illustrate an example of step 108 in which the virtual constraints generated during step 106 interact with the surgical tool 22, facilitating subsequent reorientation of the surgical tool 22. In the cases of Figures 8 to 10, subsequent reorientation of the surgical tool 22 occurs when the surgical tool 22 is again positioned at point P2, which is the detected initial reorientation position. However, in other cases, subsequent reorientation of the surgical tool 22 may occur when the surgical tool 22 is near the detected initial reorientation position.

[0138] In the case of Figure 8, the virtual mesh VM reorients the surgical tool 22 by contacting the surgical tool 22. Specifically, the virtual mesh VM contacts the surgical tool 22 to reorient the surgical tool 22 to the orientation ROR, mimicking the initial reorientation in Figure 6. During step 106, the virtual mesh may be generated to include any appropriate size or shape so that the interaction between the virtual mesh and the surgical tool 22 mimics the initial reorientation. In addition, the virtual mesh may include any appropriate position relative to the detected initial reorientation location in order to mimic the initial reorientation.

[0139] In the case of Figure 9, the stereotactic interaction feature SF reorients the tool by contacting the virtual constraint. Specifically, the stereotactic interaction feature SF contacts the virtual mesh VM to reorient the surgical tool 22 to orientation ROR, mimicking the reorientation in Figure 6. During step 106, the stereotactic interaction feature may be generated to include any appropriate size or shape so that the interaction between the stereotactic interaction feature and the virtual constraint mimics the initial reorientation. In addition, the stereotactic interaction feature may be generated at any appropriate position relative to the surgical tool 22, the detected initial reorientation position, and / or the virtual constraint in order to mimic the initial reorientation.

[0140] The virtual constraints contacted by the stereotactic interaction features can be any suitable virtual constraints, such as a virtual mesh and / or another stereotactic interaction feature. In addition, the virtual constraints contacted by the stereotactic interaction features may be generated during method 100 to facilitate the reorientation of the surgical tool 22, or they may be generated before the execution of method 100. For example, the stereotactic interaction feature SF and virtual mesh VM in Figure 9 may both be generated during a single instance of step 106. As another example, the stereotactic interaction feature SF may be generated during the instance of step 106, and the virtual mesh VM may be generated before the execution of method 100.

[0141] In the case of Figure 10, particle guides PG1 and PG2 reorient the surgical tool 22 by applying a repulsive force FR and an attractive force FA to the surgical tool 22. Specifically, particle guides PG1 and PG2 apply a repulsive force FR and an attractive force FA to the surgical tool 22 to reorient it from orientation OR2 to orientation ROR, mimicking the reorientation in Figure 6. During step 106, the particle guides may be generated to apply any appropriate constraint force to the surgical tool 22 in order to produce the desired reorientation. In addition, the particle guides may include any appropriate position relative to the detected initial reorientation position in order to produce the desired reorientation.

[0142] The virtual constraints generated by the controller 30 can be activated or deactivated at any time. For example, the virtual constraints generated during step 106 may be activated when the surgical tool 22 is within proximity to the initial reorientation location, and this proximity can be any suitable distance. For example, the stereotactic interaction feature SF in Figure 9 may be activated by the controller when the surgical tool 22 is within proximity to the detected initial reorientation location and deactivated when the surgical tool 22 is not within proximity to the detected initial reorientation location. Advantageously, in such cases, the virtual constraints generated by the controller 30 interact with the surgical tool 22 when it is within proximity to the detected initial reorientation location and do not interact with the surgical tool 22 when it is not within proximity to the detected initial reorientation location.

[0143] The controller 30 may notify the operator of the interaction between the surgical tool 22 and the virtual restraint. In response to the interaction between the surgical tool 22 and the virtual restraint, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43 in response to the interaction between the surgical tool 22 and the virtual restraint. As another example, the controller 30 may provide visual notification via the display of the surgical system 10 in response to the interaction between the surgical tool 22 and the virtual restraint. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device. In some cases, the controller 30 may notify the operator of potential interactions between the surgical tool 22 and the virtual restraint. For example, the controller 30 may present the operator with a virtual restraint adjacent to the current position of the surgical tool 22 via the display of the surgical system 10. In an exemplary embodiment, the controller 30 may present the operator with an activated temporary virtual restraint near the current position of the surgical tool 22 via the display of the head-mounted device.

[0144] f. Other features

[0145] i. Temporary virtual constraints

[0146] The controller 30 may be configured to generate temporary virtual constraints. Temporary virtual constraints may be generated before or during method 100. Temporary virtual constraints may be generated in response to the detection of initial reorientation. For example, the controller 30 may be configured to generate temporary virtual constraints in step 106 in response to the detection of initial reorientation in step 104.

[0147] As an example, the controller 30 may be configured to generate a temporary virtual constraint in response to determining that the surgical tool 22 has deviated from the tool path 70 during the detection of initial reorientation. In such a case, the surgical tool 22 may be prevented from moving between a first point and a second point along the tool path 70 by an obstacle such as a retractor R and / or a surgical assistant. The operator may apply an external force / torque to the surgical tool 22 to reorient and move it away from the tool path 70 at the first point in order to avoid the obstacle. The operator may then apply an external force / torque to the surgical tool 22 to move it toward the tool path 70 at the second point so that the surgical tool 22 can continue along the tool path 70. Thus, the controller 30 may generate a temporary virtual constraint between the first point and the second point on the tool path 70 to displace the surgical tool 22 from the tool path 70 during the subsequent movement of the tool along the tool path. Temporary virtual constraints can be generated to mimic detected initial reorientation, i.e., the movement of the surgical tool 22 away from and toward the tool path 70.

[0148] Generally, a temporary virtual constraint can be defined as a virtual constraint that can be removed / deactivated by the controller 30.

[0149] For example, in some cases, the controller 30 may remove / deactivate the temporary virtual restraint after a predetermined amount of time has elapsed. If the controller 30 generates the temporary virtual restraint in response to determining that the surgical tool 22 has deviated from the tool path 70, the predetermined time may be sufficient time for the completion of the surgical procedure.

[0150] As another example, the controller 30 may remove / deactivate the temporary virtual constraint in response to the completion of a step in the surgical procedure. If the controller 30 generates a temporary virtual constraint in response to determining that the surgical tool 22 has deviated from the tool path 70, the controller 30 may remove / deactivate the temporary virtual constraint in response to the surgical tool 22 completing its movement along the path 70. In such a case, the surgical tool 22 may deviate from the tool path 70 to avoid an obstacle along the tool path 70. The surgical tool 22 may deviate in a manner that allows the surgical tool 22 to continue along the tool path 70 after the obstacle has been avoided. The controller 30 may remove / deactivate the temporary virtual constraint in response to the completion of the movement of the surgical tool 22 along the tool path 70. For example, if the obstacle is a retractor R, the controller 30 may remove / deactivate the temporary virtual constraint in response to the removal of the retractor R. As another example, if the obstacle is the arm of a surgical assistant, the controller 30 may remove / deactivate the temporary virtual constraint in response to the surgical assistant changing position so that the surgical assistant no longer obstructs movement along the tool path 70. The controller 30 may determine that the obstacle has been removed based on input from the user interface of the surgical system 10 (for example, the operator may interact with the user interface to indicate the removal of an object) and / or based on tracking by the navigation system 32.

[0151] As another example, the controller 30 may remove / deactivate a temporary virtual constraint in response to an action by the operator of the surgical system 10. If the controller 30 generates a temporary virtual constraint in response to determining that the surgical tool 22 has deviated from the tool path 70, the controller 30 may remove / deactivate the temporary virtual constraint in response to detecting that an obstacle that was blocking the tool path 70 has been removed. In such a case, the operator may reorient the surgical tool 22 to avoid the obstacle along the tool path 70, and the controller 30 may generate a temporary virtual constraint in response to the reorientation. Once the obstacle is removed, the controller 30 may remove / deactivate the temporary virtual constraint.

[0152] In some cases, upon removal / deactivation of a temporary virtual constraint, the controller 30 may be configured to generate a new tool path 70 for excising previously avoided tissue during reorientation. For example, if the surgical tool 22 deviates from the tool path 70 to avoid an obstacle along the tool path 70, the controller 30 may generate a new tool path 70 for excising the tissue avoided during the deviation. In such cases, the controller 30 may generate a temporary virtual constraint in response to determining that the surgical tool 22 has deviated from the tool path 70. Once the temporary virtual constraint is removed, the controller 30 may generate a new tool path 70 for excising the avoided tissue. The controller 30 may then be configured to facilitate the movement of the end effector 20 and the surgical tool 22 along the new tool path 70 for excising the avoided tissue.

[0153] The controller 30 may notify the operator of the generation of temporary virtual constraints. In response to the generation of temporary virtual constraints, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, in response to the generation of temporary virtual constraints, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43. As another example, the controller 30 may provide visual notification via the display of the surgical system 10 in response to the generation of temporary virtual constraints. The controller 30 may present a 2D / 3D image of the generated temporary virtual constraint to provide visual notification. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device. In some cases, the controller 30 may label the generated temporary virtual constraints. For example, the display of the surgical system 10 may provide a menu of virtual constraints generated by the boundary generator 66. The display may use a first label to indicate a fixed virtual restraint (e.g., a restraint representing a tracked anatomical structure of patient 12) or a second label to indicate a temporary virtual restraint.

[0154] ii. User Interface

[0155] In some embodiments, the surgical system 10 may include a user interface configured to receive input from or provide output to the operator of the surgical system 10. For example, the operator may provide input to the user interface to manage virtual restraints of the surgical system 10. In addition, the user interface may output notifications and / or 2D / 3D images related to virtual restraints to the operator. The user interface may be any of the input / output devices 40, 42, 43 described above. For example, the operator may provide input to one or more of the interactive touchscreen displays 40, 42 to manage virtual restraints. In addition, the operator may display notifications and / or 2D / 3D images related to virtual restraints on a display such as the display 38 of the interactive touchscreen devices 40, 42 and / or the display of a head-mounted device.

[0156] In another exemplary embodiment, the user interface may be a button located on the surgical tool 22, which can be pressed to provide input to the surgical system 10. For example, the button may be located on the end effector 20 and / or the surgical tool 22. In some cases, the button may be a foot switch 43. The button may also be located on a handheld pendant device, as shown and described in U.S. Patent No. 9,119,655, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in Multiple Modes," the disclosure of which is incorporated herein by reference. Pressing the button indicates that the operator desires to reorient the surgical tool 22, and the controller 30 may detect the reorientation of the surgical tool 22 based on the button press.

[0157] The operator can manage virtual constraints in the surgical system 10 by confirming the generation of virtual constraints. For example, the controller 30 may provide the operator with a notification / prompt related to the generation of virtual constraints before the generation of virtual constraints in step 106. In such cases, the operator can confirm the generation of virtual constraints by providing input to the user interface in response to the notification / prompt. Similarly, the operator can reject the generation of virtual constraints by providing input to the user interface in response to a prompt. In some cases, the controller 30 may assume that the operator approves the generation of virtual constraints and provide a notification that virtual constraints have been generated.

[0158] The operator can manage the virtual constraints of the surgical system 10 by modifying the properties of the virtual constraints. For example, the operator may modify the size and / or shape of the virtual mesh and / or stereotactic interaction features by providing input to the user interface. As another example, the operator may modify the direction and / or magnitude of the constraint force generated by the particle guide. The operator may also modify the position of the virtual mesh and / or particle guide relative to the detected initial reorientation position, and the operator may modify the position of the stereotactic interaction features relative to the surgical tool 22. The operator may also merge two or more virtual constraints into a single virtual constraint, or separate virtual constraints into two or more virtual constraints. In one such case, the user interface may provide a menu of virtual constraints on the user interface display, and the operator may modify the properties of the virtual mesh, stereotactic interaction features, and / or particle guide by interacting with the user interface.

[0159] The operator may manage the virtual constraints of the surgical system 10 by removing virtual constraints and / or adding additional virtual constraints. Similarly, the operator may manage the virtual constraints of the surgical system 10 by deactivating / activating virtual constraints. For example, the operator may remove / deactivate / activate virtual meshes, stereotactic interaction features, and / or particle guides, and / or add additional virtual meshes, stereotactic interaction features, and / or particle guides containing any appropriate size / shape at any appropriate location. In one such case, the user interface may provide a menu of virtual constraints on the user interface display, and the operator may remove / activate / deactivate / add virtual meshes, stereotactic interaction features, and / or particle guides by interacting with the user interface.

[0160] The operator may manage virtual constraints generated during method 100 to facilitate the reorientation of the surgical tool 22, and / or virtual constraints generated before the execution of method 100. For example, the operator may modify and / or remove virtual constraints generated during step 106, which generates virtual constraints. The operator may also modify and / or remove virtual constraints generated before step 102, which controls the manipulator 14 to facilitate the movement of the surgical tool 22.

[0161] iii. Virtual constraints based on mobility

[0162] As mentioned above, virtual constraints of the system may be generated based on the operator's mobility and / or preferences. In such cases, the surgical tool 22 may be reoriented to allow the surgical tool 22 to be moved according to the operator's requests and / or abilities. Reoriented the surgical tool 22 allows the surgical tool 22 to be moved in a manner that is more ergonomically comfortable for the operator. For example, in guided manual mode, the controller 30 may constrain the surgical tool 22 so that it moves along the tool path 70 in a preferred orientation in response to external forces / torques applied to the components of the manipulator 14. However, in some cases, such a preferred orientation may be ergonomically unsuitable because it may place unnecessary stress on the operator. Therefore, the operator may apply external forces / torques to move the surgical tool 22 to a more ergonomically comfortable orientation, thereby causing an initial reoriented orientation. Thus, the virtual constraints generated during step 106 may mimic an initial reoriented orientation, facilitating an ergonomically comfortable orientation during subsequent movement of the surgical tool 22. As another example, the operator may have a preferred wrist position and / or hand preference (e.g., left hand or right hand) for grasping the end effector 20 and applying an external force / torque. In such a case, the operator may apply an external force / torque to move the surgical tool 22 to an orientation that allows the operator to grasp the end effector 20 and / or the surgical tool 22 using the preferred wrist position and / or hand preference, thereby causing an initial reorientation. Thus, the virtual constraint generated during step 106 may mimic the initial reorientation, facilitating the preferred orientation during subsequent movement of the surgical tool 22.

[0163] The system's virtual constraints may also be generated based on the mobility of the manipulator 14. In such cases, the surgical tool 22 may be reoriented to allow the manipulator 12 to move the surgical tool 22 without restriction. For example, the controller 30 may be prevented from moving along the tool path 70 in a preferred orientation due to one or more of the following: the working space boundary of the manipulator 14, the range of motion of the manipulator 14, the limits of the joints (J) of the manipulator 14, and singularities of the manipulator 14. For example, the range of motion of the manipulator 14 may be restricted at a point along the tool path 70, preventing the surgical tool 22 from achieving a preferred orientation at that point. In such cases, the surgical tool 22 may be reoriented to allow the manipulator 14 to continue moving along the tool path 70 despite the restriction of the range of motion of the manipulator 14 at that point along the tool path 70. Furthermore, the virtual constraints generated during step 106 may facilitate subsequent reorientation of the surgical tool 22 based on one or more of the working space boundary of the manipulator 14, the range of motion of the manipulator 14, the limits of the joints (J) of the manipulator 14, and the singularities of the manipulator 14. For example, the virtual constraints may apply a constraint force to reorient the surgical tool 22 toward the base 16 of the manipulator 16 so that the manipulator 14 remains within the working space boundary. As another example, the virtual constraints may apply a constraint force to reorient the surgical tool 22 so that the constraint force does not cause the robot arm 18A to exceed the range of motion of the manipulator 14. As yet another example, the virtual constraints may apply a constraint force to reorient the surgical tool 22 so that the constraint force does not cause the robot arm 18A to position the manipulator 14 at a singularity where the degrees of freedom (DOF) of the manipulator 14 are reduced.

[0164] iv. Automatic correction of virtual constraints

[0165] In some embodiments, the controller 30 may automatically modify previously generated virtual constraints. Virtual constraints can be any suitable type of virtual constraint, such as a virtual mesh, a localized interaction feature, and / or a particle guide. The controller 30 may modify previously generated virtual constraints by modifying the parameters of the virtual constraint, such as the constraint force, size, shape, and / or the position of the virtual constraint. Previously generated virtual constraints may be temporary virtual constraints.

[0166] In one such case, the controller 30 can automatically consolidate multiple virtual constraints into a single virtual constraint. For example, referring to Figure 13A, the controller 30 generates a first virtual constraint VM1 in response to detecting a first initial reorientation at point P1, a second virtual constraint VM2 in response to detecting a second initial reorientation at point P2, and a third virtual constraint VM3 in response to detecting a third initial reorientation at point P3. As shown in Figure 13B, the controller 30 can automatically consolidate the first, second, and third virtual constraints VM1, VM2, and VM3 into a single aggregated virtual constraint VMA.

[0167] In another case, the controller 30 may automatically correct a previously generated virtual constraint in response to detecting a reorientation of the surgical tool 22. Such a reorientation may be referred to herein as a corrective reorientation. In such a case, the corrective reorientation results in a corrective interaction with the previously generated virtual constraint, and the controller 30 corrects the previously generated virtual constraint based on the corrective interaction. The corrective interaction may be defined as the interaction between the surgical tool 22 and the previously generated virtual constraint, which differs from the expected interaction between the surgical tool 22 and the previously generated virtual constraint. For example, the controller 30 may predict that during the expected interaction, the surgical tool 22 will apply a predicted amount of force to the virtual constraint. During the corrective interaction, the surgical tool 22 may apply a force to the virtual constraint that differs from the predicted amount of force.

[0168] Figures 11A to 12B show the case where the controller 30 automatically adjusts the size of the virtual mesh VM. In both Figures 11A and 12A, the controller 30 generates the virtual mesh VM in response to detecting the initial reorientation of the surgical tool 22 to orientation ROR1 at point P2, which is the detected reorientation location. As shown, the virtual mesh VM in Figures 11A and 12A interact with the surgical tool 22 to facilitate subsequent reorientation of the surgical tool 22 to orientation ROR1.

[0169] In Figure 11B, the controller 30 reduces the size of the virtual mesh VM in response to detecting a modified reorientation of the surgical tool 22. The modified reorientation of the surgical tool 22 is shown in Figure 11A, where the operator applies an external force F directed towards the virtual mesh VM to the surgical tool 22. The controller 30 may determine that the application of the external force F results in a modified interaction between the surgical tool 22 and the virtual mesh VM, which differs from the expected interaction between the surgical tool 22 and the virtual mesh VM. In addition, the controller 30 may determine that the direction of the external force F is directed towards the virtual mesh VM, which suggests that the operator may desire to reduce the size of the virtual mesh VM. Figure 11B shows a second subsequent reorientation of the surgical tool 22, where the virtual mesh VM reorients the surgical tool 22 to orientation ROR2 instead of orientation ROR1 due to the reduction in the size of the virtual mesh VM. In the case of Figure 11B, the controller 30 may modify the size of the virtual mesh VM based on the external force F. For example, the reduction in the size of the virtual mesh VM may be proportional to the magnitude of the external force F.

[0170] In Figure 12B, the controller 30 increases the size of the virtual mesh VM in response to detecting a second initial reorientation of the surgical tool 22. The second initial reorientation of the surgical tool 22 is shown in Figure 12A, where the operator applies an external force F directed away from the virtual mesh VM to the surgical tool 22. The controller 30 may determine that the application of the external force F results in a modified interaction between the surgical tool 22 and the virtual mesh VM, which differs from the expected interaction between the surgical tool 22 and the virtual mesh VM. Additionally, the controller 30 may determine that the direction of the external force F, which is away from the virtual mesh VM, suggests that the operator may desire to increase the size of the virtual mesh VM. Figure 12B shows a second subsequent reorientation of the surgical tool 22, where the virtual mesh VM reorients the surgical tool 22 to orientation ROR2 instead of orientation ROR1 due to the increased size of the virtual mesh VM. In the case of Figure 12B, the controller 30 may modify the size of the virtual mesh VM based on the external force F. For example, the increase in the size of the virtual mesh VM may be proportional to the magnitude of the external force F.

[0171] The controller 30 may also remove / deactivate virtual constraints based on modified reorientation. For example, the controller 30 may remove / deactivate / activate the virtual mesh, stereotactic interaction features, and / or particle guides in response to detecting external / forces applied to the surgical tool 22 directed towards the virtual mesh, stereotactic interaction features, and / or particle guides.

[0172] Corrective reorientation can be triggered by input from the operator. For example, corrective reorientation may be triggered in response to the user interacting with a user interface (e.g., a button located on the surgical tool 22).

[0173] Modified reorientation can occur at any suitable location. As mentioned above, modified reorientation results in a modified interaction with previously generated virtual constraints. Therefore, modified reorientation can occur at any location where a modified interaction can occur. For example, if a virtual constraint was generated at the location where the initial reorientation was detected, then modified reorientation may occur at and / or near the location where the initial reorientation was detected during the subsequent movement of the surgical tool 22.

[0174] A previously generated virtual constraint may be generated before and / or during the execution of method 100. If a virtual constraint is generated before the execution of method 100, the virtual constraint may be modified in response to the detection of an initial reorientation of the surgical tool 22 during step 104. In other words, instead of generating a new virtual constraint in step 106 in response to the detection of an initial reorientation during step 104, the controller 30 may modify an existing virtual constraint. If a virtual constraint is generated in step 106 in response to the detection of an initial reorientation, the controller 30 may modify the generated virtual constraint in response to the detection of a second initial reorientation of the surgical tool 22. In other words, because a virtual constraint was generated in step 106 in response to the detection of an initial reorientation during step 104, the controller 30 modifies the virtual constraint generated in response to the detection of a second initial reorientation.

[0175] In some cases, the controller may generate new virtual constraints in response to modified reorientation. In one such case, the controller 30 may generate new virtual constraints without removing or modifying previously generated virtual constraints. The newly generated virtual constraints may be of the same type as the previously generated virtual constraints, or of a different type. For example, the controller 30 may generate a stereotactic interaction feature in response to a modified interaction between the surgical tool 22 and the previously generated virtual mesh, and the stereotactic interaction feature may interact with the previously generated virtual mesh during the subsequent movement of the surgical tool 22. In another case, the controller 30 may replace a previously generated virtual constraint with a newly generated virtual constraint. The replaced virtual constraint may be of the same type as the previously generated virtual constraint, or of a different type. For example, the controller 30 may replace a previously generated virtual mesh with a particle guide in response to a modified interaction between the surgical tool 22 and the previously generated virtual mesh, and the particle guide may apply a repulsive force to the surgical tool 22 during its subsequent movement.

[0176] The controller 30 may notify the operator of an automatic correction of the virtual restraint. In response to the automatic correction, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, in response to the automatic correction of the virtual restraint, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43. As another example, the controller 30 may provide a visual notification via the display of the surgical system 10 in response to the automatic correction of the virtual restraint. The controller 30 may present a 2D / 3D image of the corrected virtual restraint to provide the visual notification. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device. In some cases, the controller 30 may request confirmation from the operator of the automatic correction of the virtual restraint. The controller 30 may request confirmation via the display of the surgical system 10. Where input / output devices 40, 42, and 43 include a head-mounted device, the head-mounted device may be configured to receive confirmation by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0177] v. Automatic generation of virtual constraints

[0178] In some embodiments, the controller 30 may generate two or more virtual constraints in response to detecting an initial reorientation of the surgical tool 22 during step 104. Specifically, the controller 30 may generate a virtual constraint for a first position in response to detecting an initial reorientation at a first position, and the controller 30 may automatically generate a second virtual constraint for a second position in response to detecting an initial reorientation at the first position. Advantageously, in such cases, the controller 30 attempts to predict future initial reorientations based on past initial reorientations.

[0179] For example, in Figures 14A and 14B, the operator applies an external force F to the surgical tool 22 at each of points P1 to P4, causing first, second, third, and fourth initial reorientations at points P1 to P4. In such cases, the operator may be applying the external force F at each of points P1 to P4 due to the limits of the manipulator 14's range of motion. The controller 30 can then attempt to predict future initial reorientations at points P5 and P6 based on the first, second, third, and fourth initial reorientations at points P1 to P4. As shown in Figure 14B, the controller 30 generates particle guides PG1 to PG4 in response to each of the first, second, third, and fourth reorientations, and the particle guides PG1 to PG4 include repulsive forces to mimic the first, second, third, and fourth reorientations. In addition, the controller 30 automatically generates particle guides PG5 and PG6 at points P5 and P6 to facilitate the reorientation of the surgical tool 22. In this way, when the surgical tool 22 reaches points P5 and P6, the particle guides PG5 and PG6 facilitate the reorientation of the surgical tool 22 without requiring the operator to provide external force.

[0180] The controller 30 can generate any type of virtual constraint for automatic generation. For example, the controller 30 may generate virtual meshes, localization interaction features, and / or particle guides. In addition, the controller 30 may automatically generate any number of virtual constraints in any combination at any suitable location. For example, the controller 30 may automatically generate any combination of one or more virtual meshes, one or more particle guides, and / or one or more localization interaction features at any suitable location. Furthermore, the controller 30 may generate temporary virtual constraints during automatic generation.

[0181] The controller 30 can automatically generate any appropriate number of virtual constraints. For example, in Figures 14A and 14B, the controller 30 automatically generates two virtual constraints, namely particle guides PG5 and PG6. The number of virtual constraints automatically generated can correspond to the generation parameters of the controller 30. For example, a larger value for the generation parameters results in a larger number of virtual constraints that can be automatically generated by the controller 30, while a smaller value for the generation parameters results in a smaller number of virtual constraints that can be automatically generated by the controller 30. The controller's generation parameters can be adjusted by the operator via the user interface of the system 10.

[0182] The controller 30 can automatically generate virtual constraints in response to any appropriate number of initial reorientations. For example, in Figures 14A and 14B, the controller 30 automatically generates particle guides PG5 and PG6 in response to detecting four initial reorientations, namely the first, second, third, and fourth initial reorientations. In other cases, the controller 30 can automatically generate virtual constraints based on a single initial reorientation. The number of initial reorientations to be detected by the controller 30 before the automatic generation of virtual constraints can correspond to the controller 30's learning parameter. For example, a larger value for the learning parameter results in fewer initial reorientations to be detected by the controller 30 before the automatic generation of virtual constraints, while a smaller value for the generation parameter results in more initial reorientations to be detected by the controller 30 before the automatic generation of virtual constraints.

[0183] The controller 30 may notify the operator of the automatic generation of virtual restraints. In response to the automatic generation, the controller 30 may provide tactile, auditory, and / or visual feedback to the operator of the surgical system 10. For example, in response to the automatic generation of virtual restraints, the controller 30 may provide a vibration pattern to one or more of the input / output devices 40, 42, and 43. As another example, in response to the automatic generation of virtual restraints, the controller 30 may provide visual notification via the display of the surgical system 10. The display may be the display 38 of the interactive touchscreen devices 40, 42, and / or the display of the head-mounted device. In some cases, the controller 30 may request confirmation from the operator of the automatic generation of virtual restraints. The controller 30 may request confirmation via the display of the surgical system 10. If the input / output devices 40, 42, and 43 include a head-mounted device, the head-mounted device may be configured to receive confirmation by tracking the operator's gestures, gaze, and / or head movements. The head-mounted device may also include a microphone and / or audio sensors configured to receive voice instructions / commands as confirmation from the user.

[0184] As can be understood from the above description and figures, the techniques described herein provide significant technological solutions that surpass conventional methods. By generating virtual constraints that facilitate the reorientation of surgical tools, these techniques avoid the need for surgeons to manually perform repeated or anterior-posterior reorientation. The techniques described herein can learn about the obstacle environment from past reorientation(s) and / or adaptively predict when future reorientation will be needed. The solutions provided can make robotic procedures less cumbersome for surgeons and reduce fatigue in the surgeon's hand or arm. These solutions reduce delays in surgical procedures by avoiding the need for surgeons to perform manual reorientation. The solutions provided herein can reduce the time constraints, surgeon anxiety, and human error associated with manual reorientation. Furthermore, the techniques described herein can improve surgeon ergonomics. For example, virtual constraints can be adapted reactively or preventively to provide tool reorientation to maintain a comfortable wrist position for the surgeon.

[0185] Computers or controllers as referred to herein include, unless otherwise specified, memory, storage, a central processing unit (CPU) having one or more processors or microprocessors, input / output devices, etc. The term “memory” is intended to include non-temporary memory associated with a processor or CPU, such as RAM (random access memory), ROM (read-only memory), fixed memory devices (e.g., hard drives), removable memory devices (e.g., diskettes), flash memory, etc.

[0186] As will be understood by those skilled in the art, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media, the computer-readable media having computer-readable program code embodied thereon. Computer software, including instructions or code for performing the methodology of the present invention as described herein, may be stored in one or more associated memory devices (e.g., ROM, fixed or removable memory), and when ready for use, may be loaded partially or entirely (e.g., into RAM) and executed by the CPU. Such software may include, but is not limited to, firmware, resident software, microcode, and the like.

[0187] Several embodiments have been described in the preceding description. However, the embodiments discussed herein are not intended to be exhaustive or to limit the invention to any particular form. The terminology used is intended to be descriptive rather than limiting. In light of the above teachings, many modifications and variations are possible, and the invention may be carried out in ways other than those specifically described.

[0188] This disclosure also includes the following clauses, having certain features as described in the dependent clauses, which may be specifically implemented as described in more detail with reference to the above configuration and drawings.

[0189] Clause C1. A robotic surgical system comprising: a tool configured to handle a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to perform: acquire a tool path defined with respect to the surgical site; predictively detect a position in the tool path where the tool should be reoriented; generate one or more virtual constraints configured to cause the manipulator to reoriented the tool at the predictively detected position; control the manipulator to facilitate the movement of the tool along the tool path for treating the surgical site, and utilize the one or more virtual constraints to cause the manipulator to reoriented the tool in response to the tool reaching the predictively detected position.

[0190] C2. The robotic surgical system according to clause C1, further comprising a camera configured to identify obstacles or sensitive areas in relation to the tool path, wherein one or more controllers predictively detect the position based on the identification of obstacles or sensitive areas by the camera.

[0191] C3. The robotic surgical system according to clause C2, wherein the camera is attached to the manipulator.

[0192] C4. The robotic surgical system according to any one of the clauses C1 to C3, wherein the one or more controllers predictively detect the position based on the detection of one or more preceding reorientations of the tool.

[0193] C5. The robotic surgical system according to any one of the clauses C1 to C4, wherein the one or more controllers predictively detect the position based on the detection of one or more previously detected parameters of the tool, the one or more previously detected parameters include one or more of the displacement of the tool, the direction of the force applied to the tool, the magnitude of the force applied to the tool, the velocity of the tool, and the acceleration of the tool.

[0194] C6. The robotic surgical system according to any one of the clauses C1 to C5, wherein one or more controllers predictively detect the position by utilizing a machine learning model trained on prior surgical planning data and prior manipulator data.

[0195] C7. The robotic surgical system according to any one of the clauses C1 to C6, wherein the one or more controllers predictively customize the features of the one or more virtual constraints, the predictively customized features include one or more of the position of the one or more virtual constraints, the geometry of the one or more virtual constraints, and / or the stiffness / damping parameters of the one or more virtual constraints.

[0196] C8. A method for operating a robotic surgical system as described in any one of the clauses C1 to C7.

[0197] C1A. A robotic surgical system comprising: a tool configured to handle a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to control the manipulator to facilitate the movement of the tool along a tool path for treating the surgical site; to detect reorientation of the tool while the tool is moving along the tool path, record the location where the reorientation occurs, and generate feedback indicating that reorientation is needed, the feedback being delivered in response to the tool revisiting the recorded location.

[0198] C2A. The robotic surgical system according to Clause C1A, wherein the feedback is tactile feedback provided by the manipulator.

[0199] C3A. The robotic surgical system according to any one of the clauses C1A to C2A, wherein the feedback is visual feedback provided by a light indicator provided to the tool.

[0200] C4A. The robotic surgical system according to any one of the clauses C1A to C3A, wherein the feedback is visual and is provided on a display that also presents representations of the tool, the tool path, and the surgical site.

[0201] C5A. The robotic surgical system according to Clause C4A, wherein the feedback includes visual identification of the position of the tool path relative to the representation of the tool path on the display.

[0202] C6A. The robotic surgical system described in any one of the clauses C1A to C5A, wherein the feedback includes a text message or verbal warning indicating that reorientation is necessary.

[0203] C7A. A method for operating a robotic surgical system as described in any one of the clauses C1A to C6A.

[0204] C1B. A robotic surgical system comprising: a tool configured to handle a surgical site; a manipulator configured to support and move the tool; and one or more controllers configured to perform the following actions: acquire a tool path defined with respect to the surgical site; predictively detect a position in the tool path where the tool should be reoriented; and generate feedback indicating that reorienteding is necessary, the feedback being delivered in response to the tool reaching the predictively detected position.

[0205] C2B. The robotic surgical system according to clause C1B, further comprising a camera configured to identify obstacles or sensitive areas in relation to the tool path, wherein one or more controllers predictively detect the position based on the identification of obstacles or sensitive areas by the camera.

[0206] C3B. The robotic surgical system according to clause C2B, wherein the camera is attached to the manipulator.

[0207] C4B. The robotic surgical system according to any one of the clauses C1B to C3B, wherein the one or more controllers predictively detect the position based on the detection of one or more preceding reorientations of the tool.

[0208] C5B. The robotic surgical system according to any one of the clauses C1B to C4B, wherein the one or more controllers predictively detect the position based on the detection of one or more previously detected parameters of the tool, the one or more previously detected parameters include one or more of the displacement of the tool, the direction of a force applied to the tool, the magnitude of a force applied to the tool, the velocity of the tool, and the acceleration of the tool.

[0209] C6B. The robotic surgical system according to any one of the clauses C1B to C5B, wherein one or more controllers predictively detect the position by utilizing a machine learning model trained on prior surgical planning data and prior manipulator data.

[0210] C7B. A method for operating a robotic surgical system as described in any one of the clauses C1B to C6B.

[0211] C1C. A robotic surgical system comprising: a tool configured to handle a surgical site; a manipulator configured to support and move the tool along a predetermined tool path; a user input device; and one or more controllers configured to receive input from the input device specifying one or more locations along the predetermined tool path, the one or more locations indicating locations where the user desires the tool to be reoriented; generate one or more virtual constraints configured to cause the manipulator to reoriented the tool in response to the tool reaching each of the one or more specified locations; and control the manipulator to facilitate the movement of the tool along the predetermined tool path for treating the surgical site, and utilize the one or more virtual constraints to automatically reoriented the tool in response to the tool reaching each of the one or more specified locations.

[0212] C2C. The robotic surgical system according to Clause C1C, wherein one or more controllers receive input before facilitating the movement of the tool along the predetermined tool path.

[0213] C3C. The robotic surgical system according to Clause C1C, wherein one or more controllers receive inputs while the tool is moving along the predetermined tool path.

[0214] C4C. The robotic surgical system according to any one of the clauses C1C to C3C, wherein the input device comprises a display configured to present a representation of the predetermined tool path, and the one or more positions are presented on the display for the predetermined tool path.

[0215] C5C. The robotic surgical system according to any one of the clauses C1C to C3C, wherein the input device comprises a button located on the tool or a foot switch connected to the manipulator.

[0216] C6C. A method for operating a robotic surgical system as described in any one of the clauses C1C to C5C.

Claims

1. It is a robotic surgical system, A tool designed to handle the surgical site, A manipulator configured to support and move the aforementioned tool, One or more controllers, Controlling the manipulator to facilitate the movement of the tool along the tool path for treating the surgical site, During the movement of the tool along the tool path, the reorientation of the tool is detected, and the position where the reorientation occurs is recorded. The tool generates one or more virtual constraints configured to cause the manipulator to reorient the tool in response to the tool revisiting the recorded location, One or more controllers configured to perform the following: The robotic surgical system comprising the above-mentioned components.

2. The robotic surgical system according to claim 1, wherein the one or more controllers are further configured to generate at least one virtual constraint at the recorded position.

3. The robotic surgical system according to claim 1 or 2, wherein the one or more controllers are further configured to generate at least one virtual constraint on the tool.

4. The robotic surgical system according to any one of claims 1 to 3, further comprising a navigation system configured to track the anatomical structure of a patient, including the surgical site, wherein one or more controllers are configured to register one or more virtual restraints to the tracked anatomical structure.

5. The robotic surgical system according to any one of claims 1 to 4, wherein the one or more controllers are configured to customize the features of the one or more virtual constraints based on the recorded positions relative to the tool path, the customized features include one or more of the positions of the one or more virtual constraints, the geometry of the one or more virtual constraints, and the stiffness / damping parameters of the one or more virtual constraints.

6. The robotic surgical system according to any one of claims 1 to 5, further comprising a camera configured to detect an obstacle or sensitive area at the recorded location, wherein one or more controllers are configured to customize the features of the one or more virtual constraints based on the detected obstacle or sensitive area, the customized features including one or more of the location of the one or more virtual constraints, the geometry of the one or more virtual constraints, and the stiffness / damping parameters of the one or more virtual constraints.

7. The robotic surgical system according to any one of claims 1 to 6, wherein the one or more controllers generate the one or more virtual constraints in such a manner that the manipulator reorients the tool to mimic the detected reorientation.

8. The one or more controllers described above are The tool detects a state in which it is no longer necessary to revisit the recorded location, In response to the detection of the aforementioned state, remove or deactivate one or more virtual constraints, A robotic surgical system according to any one of claims 1 to 7, configured to automatically perform the following:

9. The robotic surgical system according to any one of claims 1 to 8, wherein the one or more controllers are configured to detect parameters of the tool during or after the occurrence of the reorientation, the detected parameters include one or more of the displacement of the tool, the direction of the force applied to the tool, the magnitude of the force applied to the tool, the velocity of the tool, and the acceleration of the tool.

10. The robotic surgical system according to claim 9, wherein the one or more controllers are configured to customize the features of the one or more virtual constraints based on the detected parameters of the tool, the customized features include one or more of the positions of the one or more virtual constraints, the geometric shapes of the one or more virtual constraints, and the stiffness / damping parameters of the one or more virtual constraints.

11. The one or more controllers described above are The detected parameters of the tool are compared with a threshold, In response to the detected parameters satisfying the threshold, it is determined that the reorientation of the tool was intentional and in response to an external force manually applied to the tool by the user. Recording the position where the reorientation occurred, only in response to determining that the reorientation of the tool was intentional. A robotic surgical system according to claim 9, configured to perform the following:

12. The tool includes a user interface configured to receive input to initiate the reorientation of the tool, The robotic surgical system according to any one of claims 1 to 11, wherein, in response to receiving the input from the user interface, one or more controllers detect the reorientation and record the position in which the reorientation occurred.

13. The one or more virtual constraints include a virtual mesh defined at the recorded location, The robotic surgical system according to any one of claims 1 to 12, wherein the one or more controllers cause the manipulator to reorient the tool in response to the interaction between the tool and the virtual mesh.

14. The one or more virtual constraints further include at least one spatial interaction feature defined for the tool, The robotic surgical system according to claim 13, wherein one or more controllers cause the manipulator to reorient the tool in response to the interaction between the at least one stereotactic interaction feature and the virtual mesh.

15. The tool comprises a tool shaft, The robotic surgical system according to claim 14, wherein the at least one stereotactic interaction feature is defined with respect to the tool shaft.

16. The one or more virtual constraints include attractive or repulsive forces defined in the vicinity of the recorded position, The robotic surgical system according to any one of claims 1 to 15, wherein the one or more controllers cause the manipulator to reorient the tool in response to the tool receiving the suction force or the repulsive force.

17. The tool is in its original position before being reoriented by the one or more virtual constraints. The robotic surgical system according to any one of claims 1 to 16, wherein the one or more controllers automatically reorient the tool back to its original position after the tool has passed through the recorded position.

18. The robotic surgical system according to any one of claims 1 to 17, wherein the one or more controllers are configured to modify the one or more virtual constraint features in response to the detection of a second reorientation of the tool occurring at a second position adjacent to the recorded position.

19. The one or more controllers described above are Controlling the manipulator to facilitate the automatic movement of the tool along the tool path for treating the surgical site, Automatically generating one or more virtual constraints, In response to the tool revisiting the recorded location, the manipulator is instructed to automatically reorient the tool. A robotic surgical system according to any one of claims 1 to 18, configured to perform the following:

20. A method for operating a robotic surgical system, the robotic surgical system comprising a tool configured to handle a surgical site, a manipulator configured to support and move the tool, and one or more controllers, The aforementioned one-terrain controller, A step of controlling the manipulator to facilitate the movement of the tool along the tool path for treating the surgical site, The steps include detecting the reorientation of the tool while it is moving along the tool path and recording the position where the reorientation occurs, The steps include generating one or more virtual constraints that cause the manipulator to reorient the tool in response to the tool revisiting the recorded location, The method described above, which performs the action.

Images