Systems and methods for providing haptic guidance

Abstract

A method includes defining a virtual object and defining a first point and a second point associated with a virtual representation of a surgical tool. Movement of the virtual representation of the surgical tool corresponds to movement of the surgical tool in real space. The method includes controlling a robotic device coupled to the surgical tool to constrain the first point to the virtual object, determining that the first point is at a threshold position along the virtual object, and controlling the robotic device to direct the second point to the virtual object.

Description

[0001] Cross-citation of related applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application No. 62 / 908,890, filed October 1, 2019, the entire disclosure of which is incorporated herein by reference. Background Technology

[0003] This disclosure generally relates to surgical systems for orthopedic procedures, such as surgical systems that facilitate joint replacement surgery. Joint replacement surgery (arthroplasty) is widely used to treat osteoarthritis and other injuries to a patient's joints, and involves replacing parts of a joint with prosthetic components. Joint replacement surgery may include procedures that replace the hip, knee, shoulder, or other joints with one or more prosthetic components.

[0004] One possible tool for arthroplasty is a robot-assisted surgical system. Robot-assisted surgical systems typically include: a robotic device for preparing the patient's anatomy to receive the implant; a tracking system configured to monitor the position of the robotic device relative to the patient's anatomy; and a computing system configured to monitor and control the robotic device. Robot-assisted surgical systems, in various forms, autonomously perform surgical tasks, provide force feedback to the user manipulating the surgical apparatus to complete the surgical task, enhance the surgeon's dexterity and precision, and / or provide other guiding cues to facilitate safe and accurate surgical procedures.

[0005] A surgical plan is typically determined before surgery is performed using a robot-assisted surgical system. Based on this plan, the surgical system guides, controls, or restricts the movement of surgical instruments during various stages of the surgery. The guidance and / or control of the surgical instruments assist the surgeon in achieving the surgical plan. Attached Figure Description

[0006] Figure 1 This is a perspective view of the femur ready to receive an implant component according to an exemplary embodiment.

[0007] Figure 2 This is an illustration of a surgical system according to an exemplary embodiment.

[0008] Figure 3 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the first process performed by the surgical system.

[0009] Figure 4 It is possible to be based on the exemplary implementation scheme Figure 2a flowchart of a second process performed by the surgical system of

[0010] Figure 5 is a virtual control object usable with the process of Figure 4

[0011] Figure 6 is a flowchart of a third process performed by the surgical system of Figure 2

[0012] Figure 7 is a graphical representation of a virtual control object usable with the process of Figure 6

[0013] Figure 8 is a flowchart of a fourth process performed by the surgical system of Figure 2

[0014] Figure 9 is a flowchart of a fifth process performed by the surgical system of Figure 2

[0015] Figure 10 is a graphical representation of a virtual control object usable with the process of Figure 9

[0016] Figure 11 is a flowchart of a sixth process performed by the surgical system of Figure 2

[0017] Figure 12 is a flowchart of a seventh process performed by the surgical system of Figure 2

[0018] Figure 13 is a flowchart of an eighth process performed by the surgical system of Figure 2 SUMMARY

[0019] One implementation of the present disclosure is a method for controlling a robotic device. The method includes defining a virtual object and defining a first point and a second point associated with a virtual representation of a surgical tool. Movement of the virtual representation of the surgical tool corresponds to movement of the surgical tool in real space. The method includes controlling a robotic device coupled to the surgical tool to constrain the first point to the virtual object, determining that the first point is at a threshold position along the virtual object, and controlling the robotic device to direct the second point to the virtual object.

[0020] ​​​​​​​​​Another implementation of the present disclosure is a system including a robotic device and processing circuitry in communication with the robotic device. The processing circuitry can be configured to define a virtual object and define a first point and a second point associated with a virtual representation of a surgical tool. The processing circuitry is configured such that movement of the virtual representation of the surgical tool corresponds to movement of the surgical tool in real space. The processing circuitry is further configured to control the robotic device coupled to the surgical tool to constrain the first point to the virtual object, determine that the first point is at a threshold location along the virtual object, and control the robotic device to direct the second point to the virtual object.

[0021] Another implementation of the present disclosure is a method of operating a robotic device coupled with a tool. The method includes controlling the robotic device to constrain the surgical tool based on a first haptic object, receiving a signal and a user-defined direction, and adjusting a haptic control interaction in response to the signal by extending the first haptic object in the user-defined direction.

[0022] Another implementation of the present disclosure is a system including a robotic device and processing circuitry in communication with the robotic device. The processing circuitry is configured to control the robotic device to constrain the surgical tool based on a first haptic object, receive a signal and a user-defined direction, and adjust a haptic control interaction in response to the signal by extending the first haptic object in the user-defined direction.

[0023] Another implementation of the present disclosure is a method of operating a robotic device coupled with a tool. The method includes controlling the robotic device to constrain the tool based on a first haptic object, receiving a signal and a user-defined direction, and adjusting a haptic control interaction in response to the signal by adjusting a virtual size of the tool.

[0024] Another implementation of the present disclosure is a system including a robotic device and processing circuitry in communication with the robotic device. The processing circuitry is configured to control the robotic device to constrain the tool based on a first haptic object, receive a signal and a user-defined direction, and adjust a haptic control interaction in response to the signal by adjusting a virtual size of the tool.

[0025] Another implementation of the present disclosure is a method of operating a robotic device. The method includes tracking movement of a tool coupled to the robotic device, determining a direction of movement of the tool, determining whether the direction of movement is directed toward a virtual control object, and controlling the robotic device to direct the tool to the virtual control object in response to determining that the direction of movement is directed toward the virtual control object.

[0026] Another implementation of the present disclosure is a system including a robotic device and processing circuitry that can communicate with the robotic device. The processing circuitry is configured to receive tracking data indicative of movement of a tool coupled to the robotic device, determine a direction of movement of the tool, determine whether the direction of movement is directed toward a virtual control object, and control the robotic device to direct the tool to the virtual control object in response to determining that the direction of movement is directed toward the virtual control object.

[0027] Another implementation of the present disclosure is a method of operating a robotic device. The method includes tracking a tool coupled to the robotic device, controlling the robotic device to constrain the tool within a virtual control object, defining a region of the virtual control object, determining that the tool is in the region, and controlling the robotic device to resist movement of the tool in the region.

[0028] Another implementation of the present disclosure is a system including a robotic device and processing circuitry that can communicate with the robotic device. The processing circuitry is configured to control the robotic device to constrain a tool within a virtual control object, define a region of the virtual control object, determine that the tool is in the region, and control the robotic device to resist movement of the tool in the region.

[0029] Another implementation of the present disclosure is a method of operating a robotic device having a tool coupled thereto. The method includes constraining, by the robotic device, the tool to a virtual control object, detecting a force applied to the tool in a generally predetermined direction, determining whether the force in the generally predetermined direction exceeds a threshold force, and controlling the robotic device to allow the tool to exit the virtual control object in response to determining that the force in the generally predetermined direction exceeds the threshold force.

[0030] Another implementation of the present disclosure is a system including a robotic device and processing circuitry that can communicate with the robotic device. The processing circuitry is configured to control the robotic device to constrain a tool to a virtual control object, detect a force applied to the tool in a generally predetermined direction, determine whether the force in the generally predetermined direction exceeds a threshold force, and control the robotic device to allow the tool to exit the virtual control object in response to determining that the force in the generally predetermined direction exceeds the threshold force. DETAILED DESCRIPTION

[0031] The present preferred embodiment of the application is shown in the drawings. Efforts have been made to use the same or similar referencenumbers throughout the drawings to refer to the same or like parts. Although the present description primarily relates to robotic arms for orthopedic joint replacement, it should be understood that the subject matter described herein is applicable to other types of robotic systems, including robotic systems for non-surgical applications, as well as procedures directed to other anatomical regions, such as spinal or dental procedures.

[0032] Referring now toFigure 1 According to example embodiments, a femur 101 is shown as modified during a knee arthroplasty. As shown in Figure 1 The femur 101 has been modified to have a plurality of planar cuts. In the example shown, the femur 100 has been modified through five substantially planar cuts to create five substantially planar surfaces, namely a distal surface 102, a posterior chamfer surface 104, a posterior surface 106, an anterior surface 108, and an anterior chamfer surface 110. The planar surfaces can be achieved using a sagittal saw or other surgical tool (e.g., as shown in the example described below, a surgical tool coupled to a robotic device). The planar surfaces 102-110 are created such that the planar surfaces 102-110 will mate with corresponding surfaces of a femoral implant component. The location and angular orientation of the planar surfaces 102-110 can determine the alignment and positioning of the implant component. Thus, operating the surgical tool with high precision to create the planar surfaces 102-110 can improve the outcome of the joint replacement surgery.

[0033] As shown in Figure 1 The femur 101 is also modified to have a pair of pilot holes 120. The pilot holes 120 extend into the femur 101 and are created such that the pilot holes 120 can receive screws, protrusions extending from surfaces of an implant component, or other structures configured to facilitate coupling of the implant component to the femur 101. The pilot holes 120 can be created using a drill press, a ball burr, or other surgical tools as described below. The pilot holes 120 can have pre-planned locations, orientations, and depths that facilitate securely coupling an implant component to the bone in a desired location and orientation. In some cases, the pilot holes 120 are designed to intersect with high density regions of the bone and / or to avoid other implant components and / or sensitive anatomical features. Thus, operating the surgical tool with high precision to create the pilot holes 120 can improve the outcome of the joint replacement surgery.

[0034] In some embodiments, the systems and methods described herein provide robotic assistance in creating the planar surfaces 102-110 and the pilot holes 120. It should be understood that Figure 1As shown in FIG. 1, creating five planar cuts and two cylindrical guide holes is merely an example, and the systems and methods described herein can be adapted to plan and facilitate the creation of any number of planar cuts or non-planar cuts, any number of guide holes, any combination thereof, and the like, for preparing any bone and / or joint for a variety of implementations. For example, in a hip or shoulder arthroplasty procedure, a spherical drill can be used to ream a curved surface configured to receive a curved implant cup in accordance with the systems and methods herein. Further, in other implementations, the systems and methods described herein can be used to facilitate placement of implant components relative to a bone (e.g., to facilitate impaction of a cup-shaped implant in a hip arthroplasty). Many such surgical and non-surgical implementations are within the scope of the present disclosure.

[0035] Referring now to Figure 2 , a surgical system 200 for orthopedic surgery is shown, in accordance with an example implementation. Generally, the surgical system 200 is configured to facilitate planning and performance of a surgical plan, such as to facilitate a joint-related procedure. As shown in Figure 2 , the surgical system 200 is set up to treat a leg 202 of a patient 204 who is seated or lying on an operating table 205. In Figure 2 , the leg 202 includes a femur 206 (e.g., the femur 101 of Figure 1 ) and a tibia 208 between which a prosthetic knee implant is to be implanted in a total knee arthroscopic procedure. In other scenarios, the surgical system 200 is set up to treat a patient’s hip, i.e., the patient’s femur and pelvis. Additionally, in other scenarios, the surgical system 200 is set up to treat a patient’s shoulder, i.e., to facilitate replacement and / or augmentation of components of the shoulder joint (e.g., to facilitate placement of a humeral component, a glenoid component, and a graft or implant augment). Various other anatomical regions and procedures are possible. To facilitate the procedure, the surgical system 200 includes a robotic device 220, a tracking system 222, and a computing system 224.

[0036] The robotic device 220 is configured to modify the anatomy of a patient (e.g., the femur 206 of the patient 204) under the control of the computing system 224. One embodiment of the robotic device 220 is a haptics device. "Haptics" refers to the sense of touch, and the field of haptics is particularly concerned with human interactive devices that provide feedback to an operator. The feedback can include haptic sensations, such as vibrations. The feedback can also include providing forces to the user, such as positive forces or resistance to movement. One use of haptics is to provide guidance or limitations to a user of a device in manipulating the device. For example, a haptics device can be coupled to a surgical tool that can be manipulated by a surgeon to perform a surgical procedure. The manipulation of the surgical tool by the surgeon can be guided or limited through the use of haptics to provide feedback to the surgeon during the manipulation of the surgical tool.

[0037] Another embodiment of the robotic device 220 is an autonomous or semi-autonomous robot. "Autonomous" refers to the ability of a robotic device to act independently or semi-independently of human control by gathering information about its situation, determining a course of action, and automatically implementing that course of action. For example, in such an embodiment, the robotic device 220 in communication with the tracking system 222 and the computing system 224 can autonomously complete the series of femur cuts mentioned above without the need for direct human intervention.

[0038] The robotic device 220 includes a base 230, a robotic arm 232, and a surgical tool 234, and can be communicatively coupled to the computing system 224 and the tracking system 222. The base 230 provides a movable foundation for the robotic arm 232, allowing the robotic arm 232 and the surgical tool 234 to be repositioned as needed relative to the patient 204 and the surgical table 205. The base 230 can also contain a power system, computing elements, motors, and other electronic or mechanical systems necessary for the functioning of the robotic arm 232 and the surgical tool 234 described below.

[0039] The robotic arm 232 is configured to support the surgical instrument 234 and provide force as instructed by the computing system 224. In some embodiments, the robotic arm 232 allows a user to manipulate the surgical instrument and provides force feedback to the user. In such embodiments, the robotic arm 232 includes joints 236 and supports 238, which include motors, actuators, or other mechanisms configured to allow the user to freely translate and rotate the robotic arm 232 and the surgical instrument 234 in permissible postures, while providing force feedback to restrain or prevent some movement of the robotic arm 232 and the surgical instrument 234, as instructed by the computing system 224. As described in detail below, the robotic arm 232 thereby allows the surgeon to have complete control of the surgical instrument 234 within the controlled object, while providing force feedback (e.g., vibration, forces that prevent or resist boundary penetration) along the boundaries of the object. In some implementations, the robotic arm is configured to automatically move surgical instruments to new positions as instructed by the computing system 224 without direct user intervention, in order to position the robotic arm as needed and / or perform certain surgical tasks, including, for example, an incision in the femur 206.

[0040] Surgical tool 234 is configured to cut, burr, grind, drill, partially remove, reshape, and / or otherwise modify bone, or to constrain / restrict the movement of devices used for cutting, burring, grinding, drilling, partially removing, reshaping, and / or otherwise modifying bone. Surgical tool 234 may be any suitable tool and may be one of a variety of tools interchangeably connected to robotic device 220. For example, such as... Figure 2 As shown, surgical tool 234 includes a ball drill 244. In other examples, the surgical tool may also be a sagittal saw, for example, having blades aligned parallel to or perpendicular to the tool axis. The surgical tool may also be a drill, for example, having a rotary drill bit aligned parallel to or perpendicular to the tool axis. In various embodiments, surgical tool 234 may be a clamp, drill guide, cutting guide, etc., for example, configured to have a saw, drill, or other instrument inserted therein. Surgical tool 234 may also be a holding arm or other support configured to hold implant components (e.g., cup 28a, implant reinforcement, etc.) in place while screwing the implant components onto bone, adhering (e.g., gluing) them to bone or other implant components, or otherwise mounting them in a preferred location. In some embodiments, surgical tool 234 is a clamping tool configured to provide clamping force to the cup-shaped implant to facilitate fixation of the cup-shaped implant to the pelvis in a planned position and orientation.

[0041] The tracking system 222 is configured to track the patient’s anatomy (e.g., femur 206 and tibia 208) and the robotic device 220 (i.e., the surgical tool 234 and / or the robotic arm 232) to allow control of the surgical tool 234 coupled to the robotic arm 232, to determine the position and orientation of modifications or other results made by the surgical tool 234, and to allow the user to visualize the bone (e.g., femur 206, tibia 208, pelvis, humerus, scapula, etc., as applicable in various surgeries), the surgical tool 234, and / or the robotic arm 232 on a display of the computing system 224. More particularly, the tracking system 222 determines the position and orientation (i.e., pose) of an object (e.g., the surgical tool 234, the femur 206) relative to a reference coordinate system and tracks (i.e., continuously determines) the pose of the object during a surgical procedure. According to various embodiments, the tracking system 222 can be any type of guidance system, including a non-mechanical tracking system (e.g., an optical tracking system), a mechanical tracking system (e.g., tracking based on measuring the relative angles of the joints 236 of the robotic arm 232), or any combination of a non-mechanical tracking system and a mechanical tracking system.

[0042] In the embodiment shown in Figure 2 In the embodiment shown in FIG. 2, the tracking system 222 includes an optical tracking system. Accordingly, the tracking system 222 includes a first fiducial tree 240 coupled to the tibia 208, a second fiducial tree 241 coupled to the femur 206, a third fiducial tree 242 coupled to the base 230, one or more fiducials coupled to the surgical tool 234, and a detection device 246 configured to detect the three-dimensional positions of the fiducials (i.e., the markers on the fiducial trees 240-242). The fiducial trees 240, 241 can be coupled to other bones as appropriate for various surgeries (e.g., the pelvis and femur in a hip replacement surgery). The detection device 246 can be an optical detector such as a camera or an infrared sensor. The fiducial trees 240-242 include fiducials that are markers configured to be clearly displayed to the optical detector and / or easily detected by an image processing system using data from the optical detector, e.g., by highly reflecting infrared radiation (e.g., emitted by elements of the tracking system 222). A stereoscopic arrangement of cameras on the detection device 246 allows the position of each fiducial to be determined in 3D space by a triangulation method. Each fiducial has a geometric relationship to the corresponding object such that tracking of the fiducials allows tracking of the objects (e.g., tracking the second fiducial tree 241 allows the tracking system 222 to track the femur 206), and the tracking system 222 can be configured to implement a registration process to determine or verify the geometric relationship. A unique arrangement of the fiducials in the fiducial trees 240-242 (i.e., the fiducials in the first fiducial tree 240 are arranged in a different geometric shape than the fiducials in the second fiducial tree 241) allows the fiducial trees to be distinguished from one another and thus the objects being tracked to be distinguished from one another.

[0043] use Figure 2 The surgical system 200 can determine the position of the surgical instrument 234 relative to the patient's anatomical features (e.g., femur 206) using a tracking system 222 or some other method of surgical guidance and tracking, because the surgical instrument 234 is used to modify anatomical features or otherwise facilitate surgery. Additionally, using... Figure 2 The tracking system 222 or some other method for surgical guidance and tracking, the surgical system 200 can determine the relative posture of the tracked bones.

[0044] The computing system 224 is configured to create surgical plans, control the robotic device 220 according to the surgical plans to perform one or more bone modifications, and / or facilitate the implantation of one or more prosthetic components. Therefore, the computing system 224 is communicatively coupled to the tracking system 222 and the robotic device 220 to facilitate electronic communication between the robotic device 220, the tracking system 222, and the computing system 224. Furthermore, the computing system 224 can be connected to a network to receive information related to the patient's medical history or other patient profiles, medical imaging, surgical plans, surgical procedures, and to perform various functions related to the execution of surgery, such as by accessing an electronic health record system. The computing system 224 includes processing circuitry 260 and input / output devices 262.

[0045] Input / output device 262 is configured to receive user input and display output as needed for the functions and processes described herein. For example... Figure 2 As shown, the input / output device 262 includes a display 264 and a keyboard 266. The display 264 is configured to display a graphical user interface generated by the processing circuitry 260, which includes information such as surgical planning, medical imaging, settings and other options for the surgical system 200, status information related to the tracking system 222 and the robotic device 220, and tracking visualization based on data supplied by the tracking system 222. The keyboard 266 is configured to receive user input from these graphical user interfaces to control one or more functions of the surgical system 200.

[0046] The processing circuit 260 includes a processor and a memory device. The processor can be implemented as a general processor, an application-specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components. The memory device (e.g., memory, storage, storage device, etc.) is one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and / or computer code for completing or facilitating the various processes described in the present application. The memory device can be or include volatile memory or non-volatile memory. The memory device can include a database component, an object code component, a script component, or any other type of information storage components or structures for supporting the various activities and information structures described in the present application. According to an exemplary embodiment, the memory device is communicably connected to the processor via the processing circuit 260 and includes computer code for implementing one or more processes described herein (e.g., by the processing circuit 260 and / or the processor).

[0047] More particularly, the processing circuit 260 is configured to facilitate the creation of a pre-operative surgical plan prior to a surgical procedure. According to some embodiments, the pre-operative surgical plan is developed using a three-dimensional representation of the patient anatomy, which is also referred to herein as a "virtual bone model." In addition to bone, the "virtual bone model" can also include virtual representations of cartilage or other tissue. To obtain a virtual bone model, the processing circuit 260 receives imaging data of the patient anatomy on which a surgical procedure is to be performed. The imaging data can be created using any suitable medical imaging technique to image relevant anatomical features, including computed tomography (CT), magnetic resonance imaging (MRI), and / or ultrasound. The imaging data is then segmented (i.e., the regions corresponding to different anatomical features in the imaging are distinguished) to obtain the virtual bone model. For example, MRI-based scan data of a joint can be segmented to distinguish bone from surrounding ligaments, cartilage, previously implanted prosthetic components, and other tissue to obtain a three-dimensional model of the imaged bone.

[0048] Alternatively, the virtual bone model can be obtained by selecting a three-dimensional model from a database or library of bone models. In one embodiment, a user can use the input / output device 262 to select an appropriate model. In another embodiment, the processing circuit 260 can execute stored instructions to select an appropriate model based on provided images or other information about the patient. The bone model selected from the database can then be deformed based on specific patient characteristics to create a virtual bone model for surgical planning and implementation as described herein.

[0049] A pre-operative surgical plan can then be created based on the virtual bone model. The surgical plan can be automatically generated by processing circuitry 260, input by a user via input / output devices 262, or some combination thereof (e.g., processing circuitry 260 limits some features of a plan created by a user, generates a plan that a user can modify, etc.). In some embodiments, the surgical plan can be generated and / or modified based on intra-operative force measurements collected.

[0050] The pre-operative surgical plan includes desired incisions, holes, surfaces, feather edges, or other modifications made to the patient’s anatomy using surgical system 200. For example, for a total knee arthroplasty, the pre-operative plan can include incisions necessary to be made on the femur, on the distal surface, on the posterior chamfer surface, on the posterior surface, on the anterior surface, and on the anterior chamfer surface, with the relative orientation and location of the incisions suitable to fit to corresponding surfaces of a prosthesis to be joined to the femur during the surgical procedure, and incisions necessary to be made on the tibia, on the surface, suitable to fit to a prosthesis to be joined to the tibia during the surgical procedure. As another example, the pre-operative plan can include modifications necessary to create holes (e.g., pilot holes 120) in the bone. As another example, during a hip arthroplasty, the surgical plan can include feather edges necessary to form one or more surfaces on the acetabular region of the pelvis to receive a cup and, where appropriate, an implant augment. Accordingly, processing circuitry 260 can receive, access, and / or store a prosthesis model to facilitate generation of the surgical plan.

[0051] Processing circuitry 260 is also configured to generate a control object for robotic device 220 in accordance with the surgical plan. The control object can take various forms depending on the various types of robotic devices possible (e.g., haptic, autonomous). For example, in some embodiments, the control object defines instructions for the robotic device to control the robotic device to move within the control object (i.e., autonomously make one or more incisions of the surgical plan as indicated by feedback from tracking system 222). In some embodiments, the control object includes a visualization of the surgical plan and the robotic device on display 264 to facilitate surgical guidance and help guide the surgeon to follow the surgical plan (e.g., without active control or force feedback of the robotic device). In embodiments where robotic device 220 is a haptic device, the control object can be a haptic object as described in the following paragraph.

[0052] In an embodiment where the robotic device 220 is a tactile device, the processing circuitry 260 is also configured to generate one or more tactile objects based on a preoperative surgical plan to assist the surgeon during the implementation of the surgical plan by constraining the surgical instrument 234 during the surgical procedure. The tactile objects can be formed in one, two, or three dimensions. For example, a tactile object may be a line, a plane, or a three-dimensional volume. A tactile object may be curved, have curved surfaces, and / or have planar surfaces, and may be of any shape, such as a funnel shape. Tactile objects can be created to represent various desired outcomes related to the movement of the surgical instrument 234 during the surgical procedure. One or more of the boundaries of a three-dimensional tactile object may represent one or more modifications to be created on a bone surface, such as incisions. Planar tactile objects may represent modifications to be created on a bone surface, such as incisions. Curved tactile objects may represent a resulting bone surface, such as modified to receive a cup-shaped implant and / or implant reinforcement. Linear tactile objects may correspond to guide holes to be made in the bone to prepare the bone to receive screws or other protrusions.

[0053] In an embodiment where the robotic device 220 is a tactile device, the processing circuitry 260 is also configured to generate a virtual tool representation of the surgical tool 234. The virtual tool includes one or more tactile interaction points (HIPs) that represent and are associated with positions on and on the physical surgical tool 234. In an embodiment where the surgical tool 234 is a ball drill (e.g., as...),... Figure 2 In the embodiment shown, the HIP may represent the center of a spherical drill. In the case where a HIP is used to virtually represent a surgical instrument, the HIP may be referred to herein as the tool center point (TCP). If the surgical instrument 234 is irregularly shaped, such as for a sagittal saw, the virtual representation of the sagittal saw may include numerous HIPs. U.S. Application Serial No. 13 / 339,369, filed December 28, 2011, entitled “System and Method for Providing Substantially Stable Haptics,” and hereby incorporated herein by reference in its entirety, describes the use of multiple HIPs to generate tactile forces (e.g., positive force feedback or resistance to movement) on a surgical instrument. In one embodiment of the invention, the virtual tool representing the sagittal saw includes eleven HIPs. As used herein, reference to “HIP” is considered to also include reference to “one or more HIPs.” As described below, the relationship between the HIP and the tactile object enables the surgical system 200 to constrain the surgical instrument 234.

[0054] Prior to performing a surgical procedure, the patient's anatomy (e.g., femur 206) is registered to a virtual bone model of the patient's anatomy by any known registration technique. One possible registration technique is point-based registration as described in U.S. Patent No. 8,010,180, entitled "Haptic Guidance System and Method," issued August 30, 2011, and hereby incorporated by reference in its entirety. Alternatively, registration can be accomplished by 2D / 3D registration with a handheld radiographic imaging device as described in U.S. Application Serial No. 13 / 562,163, entitled "Radiographic Imaging Device," filed July 30, 2012, and hereby incorporated by reference in its entirety. Registration also includes registering the surgical tool 234 to a virtual tool representation of the surgical tool 234 so that the surgical system 200 can determine and monitor the pose of the surgical tool 234 relative to the patient (i.e., relative to the femur 206). Registration allows for accurate guidance, control, and / or force feedback during the surgical procedure. Additional details related to registration in some embodiments in hip arthroplasty are described in detail below.

[0055] The processing circuit 260 is configured to monitor virtual positions of the virtual tool representation, the virtual bone model, and the control objects (e.g., virtual haptic objects) that correspond to real-world positions of the patient's bone (e.g., femur 206), the surgical tool 234, and one or more lines, planes, or three-dimensional spaces defined by forces created by the robotic device 220. For example, if the patient's anatomy moves during the surgical procedure as tracked by the tracking system 222, the processing circuit 260 moves the virtual bone model correspondingly. Thus, the virtual bone model corresponds to and is associated with the patient's actual (i.e., physical) anatomy and the position and orientation of that anatomy in real / physical space. Similarly, any haptic objects, control objects, or other planned automated robotic device motions that are linked to incisions, modifications, etc. to be made to the anatomy created during the surgical planning are also moved correspondingly with the patient's anatomy. In some embodiments, the surgical system 200 includes a clamp or brace to substantially fix the femur 206 to minimize the need to track and account for motion of the femur 206.

[0056] For embodiments in which the robotic device 220 is a haptic device, the surgical system 200 is configured to constrain the surgical tool 234 based on the relationship between the HIP and the haptic object. That is, when the processing circuit 260 detects that the user is manipulating the surgical tool 234 to bring the HIP into virtual contact with the haptic object using data supplied by the tracking system 222, the processing circuit 260 generates control signals to the robotic arm 232 to provide haptic feedback (e.g., force, vibration) to the user to convey a constraint on movement of the surgical tool 234. In general, the term "constrain" as used herein is used to describe a tendency to constrain movement. However, the form of the constraint imposed on the surgical tool 234 depends on the form of the relevant haptic object. The haptic object can be formed in any desired shape or configuration. As noted above, three exemplary embodiments include a line, a plane, or a three-dimensional volume. In one embodiment, the surgical tool 234 is constrained because the HIP of the surgical tool 234 is constrained to move along a linear haptic object. In another embodiment, the haptic object is a three-dimensional volume, and the surgical tool 234 can be constrained by substantially preventing the HIP from moving outside of a volume enclosed by the walls of the three-dimensional haptic object. In another embodiment, the surgical tool 234 is constrained because the plane haptic object substantially prevents the HIP from moving outside of the plane and outside of the bounds of the plane haptic object. For example, the processing circuit 260 can establish a plane haptic object corresponding to a planned plane distal cut needed to create a distal surface on the femur 206 in order to substantially confine the surgical tool 234 to the plane needed to implement the planned distal cut.

[0057] For embodiments in which the robotic device 220 is an autonomous device, the surgical system 200 is configured to autonomously move and operate the surgical tool 234 in accordance with a control object. For example, the control object can define a region relative to the femur 206 in which a cut should be made. In such a case, the robotic arm 232 and one or more motors, actuators, and / or other mechanisms of the surgical tool 234 are controllable to cause the surgical tool 234 to move and operate within the control object as needed to make the planned cut, e.g., using tracking data from the tracking system 222 to implement closed loop control.

[0058] Referring now to Figure 3 , a flowchart of a process 300 that can be carried out by the surgical system 200 of Figure 2 in accordance with an exemplary embodiment is shown. The process 300 can be adapted to facilitate a variety of surgical procedures, including total and partial joint replacement surgical procedures. The process 300 can be carried out using various steps and features (including combinations thereof) shown in Figures 4 to 13 and described in detail below.

[0059] At step 302, a surgical plan is obtained. The surgical plan (e.g., a computer-readable data file) can define desired results of bone modification, e.g., based on desired locations of prosthetic components relative to a patient’s anatomy. For example, in the case of a knee arthroplasty, the surgical plan can provide planned locations and orientations of planar surfaces 102-110 and guide holes 120 as shown in FIG. 1. The surgical plan can be generated based on medical imaging, 3D modeling, surgeon input, etc. Figure 1

[0060] At step 304, one or more control boundaries, such as haptic objects, are defined based on the surgical plan. The one or more haptic objects can be one-dimensional (e.g., a line haptic), two-dimensional (i.e., planar), or three-dimensional (e.g., cylindrical, funnel-shaped, curved, etc.). The haptic objects can represent planned bone modifications defined by the surgery (e.g., haptic objects for each of planar surfaces 102-110 and each of guide holes 120 as shown in FIG. 1), implant components, surgical method trajectories, etc. The haptic objects can be oriented and positioned in three-dimensional space relative to a tracked position of the patient’s anatomy. Figure 1

[0061] At step 306, a pose of a surgical tool is tracked relative to the haptic objects, e.g., by the tracking system 222 described above. In some embodiments, one point on the surgical tool is tracked. In other embodiments (e.g., in the example of FIG. 1), two points on the surgical tool are tracked, e.g., a tool center point (TCP) at a tip / effective end of the surgical tool and a second interaction point (SIP) positioned along a body or handle portion of the surgical tool. In other embodiments, three or more points on the surgical tool are tracked. The pose of the surgical tool is determined relative to a coordinate system in which the one or more haptic objects are defined, and in some embodiments, also the pose of one or more anatomical features of the patient are tracked.

[0062] At step 308, the surgical tool is guided to the haptic object. For example, the display 264 of the surgical system 200 can display a graphical user interface indicating how (e.g., in which direction) to move the surgical tool and / or a robotic device to bring the surgical tool to the haptic object. As another example, the surgical tool can be guided to the haptic object using the folded haptic boundaries described in U.S. Patent No. 9,289,264, the entire disclosure of which is incorporated herein by reference. As another example, the robotic device can be controlled to automatically move the surgical tool to the haptic object. As another example, step 308 can be carried out using the process 800 described in detail below. Figure 8 At step 310, the surgical tool is moved to the haptic object. For example, the user can move the surgical tool to the haptic object based on the guidance provided in step 308. As another example, the robotic device can be controlled to move the surgical tool to the haptic object. As another example, the user can move the surgical tool to the haptic object using the process 800 described in detail below.​​​

[0063] At step 310, the robotic device is controlled to constrain movement of the surgical tool based on the tracked pose of the surgical tool and the pose of the one or more haptic objects. Confinement of the surgical tool can be implemented as described above with reference to Figure 2 In some embodiments, step 310 includes providing a damping force that resists movement of the surgical tool through a particular region, for example as described in Figures 4 to 5 and with reference to Figures 4 to 5 the two-stage method described below. In some embodiments, step 310 includes providing a damping force that resists movement of the surgical tool through a particular region, for example as described in Figures 9 to 10 and with reference to Figures 9 to 10 described below. In some embodiments, step 310 includes adjusting the haptic interaction in response to user input, for example according to the examples described below with reference to Figures 11 to 13 In some embodiments, step 310 includes adjusting the haptic interaction in response to user input, for example according to the examples described below with reference to Figures 11 to 13 Various combinations of these features are possible at step 310.

[0064] At step 312, the surgical tool is facilitated to exit from the haptic object, i.e., to release the haptic object from the constraint. For example, in some embodiments, the robotic device is controlled to allow the surgical tool to exit the haptic object along an axis of the haptic object. In some embodiments, the surgical tool can be allowed to exit the haptic object in a predetermined direction relative to the haptic object, for example as shown in Figures 6 to 7 and described below with reference to Figures 6 to 7 Thereby, the surgical tool can be removed from the surgical region and the haptic object to facilitate subsequent steps of the surgical procedure. Additionally, it should be appreciated that in some cases, the process 300 can return to step 308 with the surgical tool directed to the same or a different haptic object after exiting the haptic object at step 312.

[0065] Thereby, the process 300 can be carried out by the surgical system 200 to facilitate a surgical procedure. According to some embodiments, features of the process 300 are shown in Figures 4 to 13 and in various embodiments, such features can be combined in various combinations and / or based on settings selected for a particular procedure. Further, it should be appreciated that features of Figures 4 to 13 can be provided while omitting some or all other steps of the process 300. All such possibilities are within the scope of the present disclosure.

[0066] Referring now to Figures 4 to 5 , a two-stage haptic interaction is shown according to an example embodiment. Figure 4 A flowchart of a process 400 for providing a two-stage haptic interaction is shown, while Figure 5A storyboard illustration of a two-stage haptic interaction (i.e., of process 400) is shown. Process 400 can be carried out by surgical system 200, e.g., at step 310 of process 300.

[0067] At step 402, a tool center point (TCP) 500 is constrained to a line haptic 502, while allowing the surgical tool 234 to rotate about the tool center point 500 and the TCP 500 to translate along the line haptic 502. The line haptic 502 can correspond to a planned bone modification, e.g., a planned guide hole 120 as shown in Figure 1 The line haptic 502 can be defined as an axis of the planned guide hole 120, and can extend from the bottom (deepest end) of the planned guide hole 120, extending beyond the anatomical feature (e.g., femur 101). The line haptic 502 is straight in the example of Figure 5 In other embodiments, the line haptic 502 can be curved, e.g., defined as a spline. In other embodiments, Figure 4 Process 400 of

[0068] The TCP 500 is tracked relative to the line haptic 502, and the robotic device is controlled to constrain the TCP 500 to remain on or substantially on the line haptic 502 (e.g., to resist or oppose deviations from the line haptic 502, to provide a spring force to drive the TCP 500 back to the line haptic 502, etc.). At step 402, the TCP 500 is translated along the line haptic 502. The robotic device is controlled (e.g., via admittance control) to allow the surgical tool 234 to rotate about the TCP 500 (e.g., as manipulated by the user). That is, a second interaction point (SIP) 504 positioned along a handle or body portion of the surgical tool 234 is unconstrained at step 402.

[0069] At step 402, rotation of the surgical tool 234 about the TCP 500 can facilitate the surgeon reaching the line haptic 502 along the preferred approach trajectory. In some cases, surrounding soft tissue and / or bone structure can make it difficult or impossible for the surgical tool 234 to be inserted along the line haptic 502 from a position entirely outside the patient's body to the bone surface without causing undesirable or unnecessary damage to surrounding tissue or bone (e.g., without needing to pass through such tissue or bone to create a hole). In such cases, the surgical tool 234 can be inserted along the preferred trajectory until the TCP 500 reaches and is constrained by the line haptic 502. At step 402, the surgical tool 234 can be rotated to displace anatomical features by pushing such features with one side of the shaft or body of the surgical tool. By constraining the TCP 500 to the line haptic 502, the surgeon is allowed to focus on rotating the surgical tool 234 as desired at step 402 without needing to burden themselves with simultaneously monitoring the position of the TCP 500 and / or attempting to manually prevent movement of the TCP 500 away from the desired axis. In this way, step 402 can facilitate insertion and orientation of the surgical tool 234 in various anatomical regions.

[0070] At step 404, it is determined (e.g., by processing circuitry 260) that the TCP 500 has reached a threshold position along the line haptic 502. In some cases, the threshold position is defined based on a distance from the surface of a bone (e.g., femur 101) such that the threshold position is reached before the surgical tool 234 contacts the bone. In such cases, steps 406-408 can be carried out as described below before the surgical tool 234 begins modifying the bone, thereby ensuring a desired orientation of the surgical tool 234 before initiating bone modification.

[0071] In other cases, the threshold position is defined based on a depth below the surface of the bone such that the threshold position is reached after the surgical tool 234 contacts the bone. In such cases, the surgical tool 234 can be allowed to begin modifying the bone in a first orientation before being rotated into alignment with the line haptic 502 as described below with reference to steps 406-408, for example to reduce the risk of gouging or to otherwise facilitate entry of the surgical tool 234 into the bone. Figure 5As shown in the above frame, for example, initial contact can be made between the surgical tool 234 and the bone 101 with the surgical tool generally normal to the surface of the bone (i.e., during step 402), which can improve the likelihood of achieving a clean initial incision / hole / drill hole / etc. at the planned location in the bone 101 (i.e., at the intersection between the line haptics 502 and the bone 101). In such cases, the TCP 500 can then be determined to have reached the threshold location after the surgical tool 234 has initially penetrated the bone 101 (i.e., such that the TCP 500 has crossed the surface of the bone 101).

[0072] At step 406, the SIP 504 is directed toward the line haptics 502 in response to determining at step 404 that the TCP 500 has reached the threshold location. In some embodiments, the robotic device can be controlled to provide an assistive force that assists a user rotating the surgical tool 234 about the TCP 500 to move the SIP 504 toward the line haptics 502. In some embodiments, a fold haptic object is used at step 406, which blocks the SIP 504 from rotating away from the line haptics 502 while allowing the SIP 504 to rotate toward the line haptics 502. In some embodiments, directing the SIP 504 toward the line haptics 502 is achieved by displaying instructions via the display 264. In some embodiments, directing the SIP 504 toward the line haptics 502 is achieved by controlling the robotic device to automatically rotate the surgical tool 234 to align the SIP 504 with the line haptics 502. The TCP 500 is constrained to the line haptics 502 during step 406 (i.e., when the SIP 504 is being directed to the line haptics 502), as described for step 402. In some cases, the robotic device is controlled to block the TCP 500 from translating along the line haptics 502 while the SIP 504 is being directed to the line haptics 502 during step 406.

[0073] At step 408, after the SIP 504 has been directed to the wire haptics 502 as a result of step 406, the robotic device is controlled to constrain the TCP 500 and the SIP 504 to the wire haptics 502. The surgical tool 234 can be translated along the wire haptics 502 to perform the planned bone modification (e.g., to create a pilot hole 120). In the illustrated example, the SIP 504 is positioned along an axis of the surgical tool 234. Thus, by constraining two points of the other surgical tool 234 to the wire haptics 502 (i.e., the TCP 500 and the SIP 504), the alignment of the surgical tool 234 to the wire haptics 502 is maintained. In other embodiments, the SIP 504 is directed to a second haptic (i.e., a different virtual haptic object than the wire haptics 502). In such embodiments, the TCP 500 and the SIP 504 are localized to different haptic objects. For example, where the surgical tool 234 is curved, the SIP 504 can be localized to a haptic object that corresponds to the geometry of the planned incision or drill path, while the TCP 500 is localized to a different haptic object that is configured to prevent or resist collision between the shaft of the surgical tool 234 (or another point on the robotic arm) and one or more objects in the surgical region. For example, the SIP 504 can be localized to a haptic object having a geometry that is based on the location of a retractor or other tool in the surgical region (e.g., a tracked retractor location). As another example, the SIP 504 can be localized to a haptic object having a geometry that is based on the location of an anatomical feature, such as a shape corresponding to a surgical port or other knife port or opening through which the shaft of the surgical tool 234 extends during the planned bone preparation. In this way, control of the robotic device can be configured to localize the TCP 502 to a first haptic object and the SIP 504 to a second haptic object to guide the TCP 502 according to the planned bone preparation, while avoiding unwanted behavior of the tool shaft by localizing the SIP 504. Thus, the process 400 can be performed by the surgical system 200 to provide accurate bone modification in a reliable and intuitive manner.

[0074] Other geometries and behaviors can also be achieved by using different haptic objects for the SIP 504 and the TCP 500. For example, the TCP 500 can be localized to a haptic object that corresponds to the geometry of a planned incision or drill path, while the SIP 504 is localized to a different haptic object that is configured to prevent or resist collision between the shaft of the surgical tool 234 (or another point on the robotic arm) and one or more objects in the surgical region. For example, the SIP 504 can be localized to a haptic object having a geometry that is based on the location of a retractor or other tool in the surgical region (e.g., a tracked retractor location). As another example, the SIP 504 can be localized to a haptic object having a geometry that is based on the location of an anatomical feature, such as a shape corresponding to a surgical port or other knife port or opening through which the shaft of the surgical tool 234 extends during the planned bone preparation. In this way, control of the robotic device can be configured to localize the TCP 502 to a first haptic object and the SIP 504 to a second haptic object to guide the TCP 502 according to the planned bone preparation, while avoiding unwanted behavior of the tool shaft by localizing the SIP 504. Thus, the process 400 can be performed by the surgical system 200 to provide accurate bone modification in a reliable and intuitive manner.

[0075] Referring now to Figures 6 to 7 , a process 600 that facilitates withdrawal of a surgical tool from a haptic object is shown, in accordance with an example embodiment. Figure 6 A flowchart of the process 600 is shown, while Figure 7A haptic object usable with the process 600 of FIG. 6 is shown. The process 600 can be carried out by the surgical system 200, for example at step 312 of the process 300. While the example of FIG. 6 contemplates a cylindrical haptic object, it should be understood that the process 600 can be applied to control objects having various geometries. Figure 6 Figures 6 to 7 The example of FIG. 6 contemplates a cylindrical haptic object, but it should be understood that the process 600 can be applied to control objects having various geometries.

[0076] At step 602, a robotic device is controlled to constrain a surgical tool 234 with a cylindrical haptic object. Figure 7 An example of a cylindrical haptic object 700 centered on a target axis 702 is shown. In some embodiments, the cylindrical haptic object 700 corresponds to a surgical method trajectory and / or a planned bone modification (e.g., a planned guide hole 120). The cylindrical haptic object 700 can extend substantially beyond (away from) a sensitive anatomical region. In the event that the surgical tool 234 is constrained in the cylindrical haptic object 700, the surgical tool 234 can partially obstruct access to the surgical region. Thus, a surgeon can desire that the surgical tool move out of the cylindrical haptic object 700 in a safe direction to facilitate various steps of the surgery.

[0077] At step 604, a force exerted against the surgical tool 234 against a boundary of the cylindrical haptic object is detected. The robotic device 220 can detect the force, for example as a wrench applied on a joint of the robotic arm 232. For example, a HIP associated with the surgical tool 234 can be positioned at a boundary of the cylindrical haptic object 700, and a user exerts a force on the surgical tool 234, pushing the surgical tool 234 against the boundary or into the boundary.

[0078] At step 606, it is determined (e.g., by the processing circuit 260) whether the force detected at step 604 is oriented in a predetermined exit direction. The predetermined exit direction can be selected as a safe and / or convenient direction that can allow the surgical tool 234 to exit the haptic object. For example, the predetermined exit direction is defined by an exit region 704 of the cylindrical haptic object 700 and a wall 706 extending from the cylindrical haptic object 700 at the exit region 704. In such an example, the processing circuit 260 can determine that the force is oriented in the predetermined exit direction if a HIP of the surgical tool 234 is at the exit region 704, as the force is exerted against a boundary of the cylindrical haptic object 700. In some embodiments, the exit region 704 spans only a portion of the length of the haptic object 700, for example as illustrated by the dashed line in FIG. 6. Figure 7 ​The deadband 708 shown in the middle is interrupted. As another example, in some embodiments, the processing circuit 260 can determine a direction vector that points in the direction that the user is pushing the tool. In such a case, it can be determined whether the direction vector is within a threshold angle of the predetermined exit direction. In this example, if the direction vector is within the threshold angle of the predetermined exit direction, then the force is considered to be directed in the predetermined exit direction (i.e., a“yes” at step 606). As such, the resulting exit boundary can take the shape of a funnel.

[0079] If the force is not directed in the predetermined exit direction, then the process 600 returns to step 602 and the surgical tool 234 is constrained with the cylindrical haptic object 700. That is, the robotic device 220 is controlled to provide force feedback to constrain the surgical tool from exiting the cylindrical haptic object 700, for example, to facilitate one or more steps of the surgical procedure.

[0080] If the force is directed in the predetermined exit direction (as determined at step 606), then it is determined (e.g., by the processing circuit 260) at step 608 whether the force is greater than a threshold amount of force. In some embodiments, the amount of force exerted on the surgical tool 234 can be measured by the joints of the robotic arm 232. The user can indicate a desire to exit the haptic object by exceeding the threshold amount of force, while the threshold amount of force can be set high enough to substantially prevent accidental or inadvertent exit from the haptic object.

[0081] If the force is less than the threshold amount of force, then the robotic device 220 is controlled to constrain the surgical tool 234 from exiting the haptic object (e.g., to prevent passing through the exit region 704 of the cylindrical haptic object 700). The process 600 returns to step 602 and the surgical tool 234 continues to be constrained by the cylindrical haptic object 700 to facilitate steps of the surgical procedure being performed using the surgical tool 234.

[0082] If it is determined at step 608 that the force exceeds the threshold amount of force, then the surgical tool is allowed to exit the haptic object in the predetermined exit direction at step 610. In Figure 7 In the example, the constraints associated with the exit region 704 are removed to allow the surgical tool 234 to move through the exit region 704 to exit the cylindrical haptic object 700. The wall 706 can be included as a haptic boundary to guide the surgical tool 234 away from the axis 702 in the predetermined direction. In other words, the surgical tool 234 can be pushed over the exit region 704 of the cylindrical haptic object 700 to exit the cylindrical haptic object in the predetermined direction.

[0083] Accordingly, the surgical tool 234 is allowed to exit the haptic object, such that the robotic device is no longer controlled to constrain the surgical tool 234 with the haptic object. In some cases, the surgical tool 234 can be reinserted into the haptic object (i.e., haptic constraint is resumed) via the exit region 704 and / or using any other haptic initiation procedure (e.g., after the procedure of Figure 8 In some cases, the haptic object is removed (deleted, etc.) when the surgical tool 234 exits the haptic object. In some cases, the haptic object is adjusted or a new haptic object is enabled to facilitate subsequent steps of the surgery. In this way, the procedure 600 can be carried out one or more times by the surgery to facilitate the surgery.

[0084] Referring now to Figure 8 FIG. 8 shows a flowchart of a procedure 800 for guiding a surgical tool to a virtual control object (e.g., a haptic object), according to an example embodiment. The procedure 800 can be carried out by the surgical system 200, e.g., at step 308 of the procedure 300.

[0085] At step 802, a virtual control object is established. That is, the virtual control object is generated and a pose of the virtual control object is defined. The virtual control object can comprise one or more of a point object, a line object, a plane object, or a three-dimensional surface or volume, as described in detail above with reference to Figure 2 In some cases, the virtual control object is a haptic object. The procedure 800 can be adapted for use with various virtual control objects having various shapes.

[0086] At step 804, movement of the surgical tool 234 is tracked (e.g., by the tracking system 222). For example, a position of a point (e.g., a tool center point) associated with the surgical tool 234 can be determined and updated over time. The position of the point can be defined relative to the virtual control object, i.e., in a coordinate system in which the pose of the virtual control object is also defined. The surgical tool 234 can be caused to move by manipulation of the user.

[0087] At step 806, a direction of movement of the surgical tool is determined (e.g., by the processing circuit 260). For example, a position of a point associated with the surgical tool 234 can be repeatedly collected over time to obtain a time series of position data. Given two positions (e.g., for subsequent time steps), a vector (e.g., a velocity vector) can be defined that characterizes a direction of movement of the surgical tool 234. In some cases, a speed of movement (e.g., a magnitude of the velocity vector) is determined based on a distance between the positions used and a time elapsed between the collection of these data points. In some cases, the procedure 800 does not proceed to step 808 unless the magnitude of the velocity vector exceeds a threshold.

[0088] At step 808, it is determined (e.g., by processing circuitry 260) whether the direction of movement is directed toward the virtual control object. For example, the velocity vector determined at step 806 can be extended (e.g., infinitely) in the direction of movement from the most recent tracked position of the surgical tool. If the extended velocity vector intersects the virtual control object, it can be determined that the direction of movement is directed toward the virtual control object. If the extended velocity vector does not intersect the virtual control object, it can be determined that the direction of movement is not directed toward the virtual control object. In various embodiments, various other statistical methods, coordinate transformations, etc. can be used to determine whether the direction of movement of the surgical tool is directed toward the virtual control object. For example, in some embodiments, a target volume is defined at (e.g., around, adjacent to, extending from) the virtual control object, and if the extended velocity vector intersects the target volume, it can be determined that the direction of movement is directed toward the virtual control object. For example, where the virtual control object is a line, the target volume can be defined as a cylinder centered on the line.

[0089] If the direction of movement is not directed toward the virtual control object, process 800 returns to step 804, where movement of the surgical tool 234 is tracked. Steps 804-808 can be repeated until the direction of movement is directed toward the virtual control object.

[0090] If the direction of movement is determined to be directed toward the virtual control object, at step 810 the robotic device 220 is controlled to provide a force that directs the surgical tool 234 toward the virtual control object. For example, a positive assistive force can be provided that assists the user in moving the surgical tool 234 toward the virtual control object. The positive assistive force can not be sufficient to move the surgical tool 234 independently in the absence of externally supplied forces by the user. In some cases, the force applied at step 810 causes the surgical tool to move automatically (without user manipulation) to the virtual control object. As another example, in some embodiments, the force is provided as a haptic boundary (e.g., a folding haptic boundary) that constrains movement of the surgical tool 234 away from the virtual control object and / or away from the direction of movement toward the virtual control object.

[0091] As such, the surgical system 200 can implement the process 800 to facilitate the user moving the surgical tool to the virtual control object in response to the user initiating movement of the surgical tool toward the virtual control object. For example, various movements of the surgical tool away from the virtual control object can be desired before the virtual control object is desired to be used to properly position the surgical tool 234, the robotic arm 232, the anatomy, other surgical equipment, etc. The process 800 provides a user-friendly, efficient workflow in which the surgical tool can be moved freely until the surgical tool is moved toward the virtual control object (e.g., toward the surgical region in which the virtual control object is located), at which point the system 200 automatically begins guiding the surgical tool to the virtual control object.

[0092] Referring now to Figures 9 to 10 , a process for providing haptic interactions including dampened zones is shown, in accordance with example embodiments. Figure 9 A process 900 for providing haptic interactions including dampened zones is shown, in accordance with example embodiments. Figure 10 An example haptic object including a dampened zone is shown. The process 900 can be implemented by the surgical system 200, for example at step 310 of the process 300. While Figure 10 A cylindrical haptic object is shown in FIG. 10 for example, but it should be understood that Figures 9 to 10 The features of FIG. 10 can be adapted for use with virtual control objects of various geometries.

[0093] At step 902, a haptic object having a dampened zone is established (e.g., defined in a virtual space by the processing circuit 260). The dampened zone can be defined as a sub-portion of the haptic object and / or a region within the haptic object. Figure 10 An example haptic object 1000 having a dampened zone 1002 is shown in FIG. 10. As Figure 10 As shown in FIG. 10, the haptic object 1000 is a cylindrical haptic object, and the dampened zone 1002 is a cylindrical disc positioned in the haptic object 1000. In Figure 10 In the example of FIG. 10, the haptic object 1000 and the dampened zone 1002 have equal diameters, while the height of the dampened zone 1002 is significantly less than the height of the haptic object 1000, and are centered about a common axis 1004. In other embodiments, various relative sizes and dimensions are possible.

[0094] In some examples, the dampened zone 1002 is positioned along the haptic object 1000 proximate to a surface of an anatomical feature (e.g., a bone). For example, the dampened zone 1002 can be on the outside of a bone surface. In such a case, when a tracked surgical tool 234 approaches the bone while being constrained by the haptic object 1000, the surgical tool 234 first reaches the dampened zone 1002.

[0095] At step 904, it is determined (e.g., by processing circuitry 260 using data from tracking system 222) that surgical tool 234 has entered a first side of a dampening zone 1002. In Figure 10 In examples where the TCP of trackable surgical tool 234 is tracked relative to dampening zone 1002, and when the TCP intersects with a first surface 1006 of dampening zone 1002, it can be determined that surgical tool 234 has entered a first side of dampening zone 1002.

[0096] At step 906, robotic device 220 is controlled to provide haptic feedback, partially resisting movement of the surgical tool through the dampening zone. For example, when surgical tool 234 passes through the dampening zone, control of robotic device 220 based on the dampening zone can cause movement of surgical tool 234 to slow (e.g., not to exceed a preset speed). In cases where the dampening zone is located at a surface of a bone, the dampening zone can thus be used to manage (e.g., reduce) the translational speed of surgical tool 234 upon initial impact between surgical tool 234 and the bone. By reducing the translational speed of surgical tool 234, the dampening zone as provided by process 900 can thereby reduce gouging, improve the quality of the incision or hole / drill hole, and improve the accuracy of the incision or hole placement.

[0097] While Figures 9 to 10 While dampening zones are shown where movement of robotic device 220 is damped as described above, in other embodiments other types of zones are provided that are associated with other effects, in which case step 906 is adjusted accordingly. For example, in some embodiments, the dampening zone is replaced with an acceleration zone in which, at step 906, robotic device 220 causes the speed of surgical tool 234 to increase as the surgical tool passes through the acceleration zone. As another example, in some embodiments, the dampening zone is replaced with an attraction zone in which robotic device 220 is controlled to provide a force on surgical tool 234 that is oriented towards a location in the attraction zone (e.g., a midpoint of the attraction zone), or a repulsion zone in which robotic device 220 is controlled to provide a force on surgical tool 234 that is oriented away from a location in the repulsion zone.

[0098] At step 908, it is determined (e.g., by processing circuitry 260 using data from tracking system 222) that surgical tool exits a second side of the dampening zone. In Figure 10 In examples where the TCP of trackable surgical tool 234 is tracked relative to dampening zone 1002, and when the TCP passes through a second side 1008 of dampening zone 1002, it can be determined that surgical tool 234 has exited the second side of dampening zone 1002.

[0099] At step 910, the damping zone is removed from the haptic object (e.g., by processing circuitry 260) in response to determining that the surgical tool has exited the second side of the damping zone. The resistance feedback applied at step 906 is no longer applied. In some embodiments, the surgical tool can be repeatedly passed through the area previously occupied by the damping zone without experiencing the resistance of step 906. Thus, surgical system 200 can be configured to provide a damping resistance to facilitate initial contact between surgical tool 234 and bone, and to automatically remove such resistance after initial contact.

[0100] Referring now to Figures 11 to 13 , a flowchart is shown that illustrates a process for adjusting a haptic interaction in response to input received from a user via a button (key, trigger, switch, pressure sensor, hand position sensor, etc.) mounted on a surgical tool or a robotic arm, in accordance with example embodiments. Reference is made to Figure 2 , Figures 11 to 13 The process of FIG. 11 contemplates a button positioned on a handle or grip region of robotic arm 232 or surgical tool 234, such that the button can be easily used by a user (e.g., a surgeon) while manipulating robotic arm 232 and surgical tool 234 during performance of a surgical procedure. The user can select the button without requiring the user to change the user’s grip on robotic arm 232 and / or surgical tool 234 and without requiring the user to remove the user’s line of sight or attention from the surgical region. Figures 11 to 13 The process of FIG. 11 can be performed by surgical system 200, e.g., as part of step 310 of FIG. 10. Figure 3

[0101] Figure 11 A flowchart of process 1100 is shown, in which the button allows the user to switch between a first haptic object and a second haptic object. At step 1102, surgical tool 234 is constrained with a first haptic object. The first haptic object can have any of the dimensions, shapes, etc. described herein. At step 1104, an electrical signal is received (e.g., at processing circuitry 260) indicating that a button mounted on a surgical tool or a robotic arm is pressed. At step 1106, haptic control of surgical tool 234 is switched from the first haptic object to a second haptic object, such that at step 1108, surgical tool 234 is constrained with the second haptic object. In some cases, the button can be pressed again to return to control based on the first haptic object.

[0102] ​The process 1100 can provide various advantages based on the relative size, shape, etc. of the first haptic object and the second haptic object. In some embodiments, the first haptic object is a sub-portion of the second haptic object (i.e., such that the second haptic object allows for a greater range of motion than the first haptic object). In such a case, the user can select the button to allow the surgical tool 234 to reach areas that were inaccessible to the surgical tool under control based on the first haptic object. One example is a set of planar haptic objects, where the first haptic object corresponds to a virtually determined range of a planar incision, and the second haptic object is a larger co-planar object. Based on the surgeon’s experience and intra-operative observations, the surgeon can press the button intra-operatively to extend the incision, if needed. As another example, the first haptic object and the second haptic object can only partially overlap. In such an example, the process 1100 can facilitate switching between different steps of a surgery. In some embodiments, the processing circuit 260 prevents switching between haptic control objects unless the surgical tool is currently located within both haptic objects.

[0103] Figure 12 The process 1200 is shown, where a button allows a user to cause a haptic object to extend in a direction of a force exerted by the user. At step 1202, a surgical tool is constrained with a first haptic object. The first haptic object can have any of the sizes, shapes, etc. described herein. At step 1204, an electrical signal is received (e.g., at the processing circuit 260) indicating that a button mounted on the surgical tool or a robotic arm is pressed.

[0104] At step 1206, a direction of a force exerted on the surgical tool is determined (e.g., by the processing circuit 260) in response to the signal from the button. The HIP of the surgical tool can be positioned at a boundary of the first haptic object such that the haptic control interaction prevents the force from causing the surgical tool to move further in the direction of the force. In such a scenario, the pressing of the button indicates that the user wishes to cause the surgical tool to move further in the direction of the force. Accordingly, at step 1208, the first haptic object is extended in the direction of the force, thereby allowing the surgical tool to move further in the direction before being constrained by the first haptic object. At step 1208, the first haptic object can be extended by a preset distance or volume. The surgical tool is then constrained by the extended first haptic object.

[0105] Thus, the process 1200 can facilitate a user extending a surgical tool beyond a first haptic object in a particular, user-selected direction. For example, in some cases, control based on the first haptic object can constrain a surgical tool to reach an entire range of an anatomical feature that a surgeon wishes to modify with the surgical tool. The surgeon can then push the surgical tool toward the target feature and select a button to cause the haptic object to extend toward the target feature. Thus, the process 1200 facilitates advantageous intraoperative adjustments to the haptic object.

[0106] Figure 13 A process 1300 is shown in which a button allows a user to adjust a virtual dimension of a virtual tool, thereby adjusting a virtual control interaction. In embodiments contemplated by the process 1300, a haptic control interaction is implemented by tracking a haptic interaction point (e.g., TCP) associated with a surgical tool. The HIP is a one-dimensional point. The processing circuitry uses one or more virtual dimensions of the surgical tool to determine a volume occupied by the surgical tool based on the location of the HIP. For example, for a spherical drill, the HIP can be located at the center of the spherical drill, and a radius of the spherical drill can be used to determine a 3-D volume occupied by the spherical drill in virtual space based on the HIP location and the radius dimension. Haptic control is provided based on an interaction between the volume occupied by the surgical tool (or a boundary thereof) and a haptic object. In the example of a spherical drill, the HIP can be constrained to a location that is offset from the haptic boundary by at least the radius dimension of the spherical drill. In such a case, changing the virtual radius of the spherical drill can allow the HIP to be moved closer to the haptic boundary. Thus, control of the robotic device can be changed by modifying a virtual dimension of the surgical tool.

[0107] At step 1302, a surgical tool is constrained with a first haptic object. The first haptic object can have any of the dimensions, shapes, etc. described herein. At step 1304, an electrical signal is received (e.g., at the processing circuitry 260) indicating that a button mounted on the surgical tool or a robotic arm is pressed. At step 1306, a virtual dimension of the surgical tool is adjusted (e.g., by the processing circuitry 260) in response to the signal from the button. For example, in a scenario in which the surgical tool is a spherical drill, a virtual radius of the spherical drill can be decreased.

[0108] At step 1308, the robotic device is controlled to constrain the surgical tool with the first haptic object based on the adjusted virtual dimensions of the surgical tool. In examples where the virtual dimensions of the surgical tool are reduced (e.g., in the case of a reduced radius), a greater range of motion can be provided for the surgical tool at step 1308 compared to step 1302. In such cases, when the surgeon wishes to allow the surgical tool to reach a location that the surgical tool was constrained from reaching at step 1302, the surgeon can engage the button. The position of the virtual center point of the surgical tool can also be shifted (e.g., at step 1306) relative to the tracked position of the surgical tool, for example, to align the boundaries of the reduced-size virtual tool with the original-size virtual tool. Such a shift can provide a greater range of motion in some directions for the surgical tool while preserving the range of motion along the common boundary. For example, shifting the virtual center point toward the distal tip of the surgical tool when reducing the virtual dimensions of the surgical tool can allow for an increased range of left-right motion (i.e., orthogonal to the axis of the surgical tool) while confining the surgical tool to the original cutting depth. In this way, process 1300 can facilitate minor intraoperative adjustments to the range of anatomical features that can be modified by the surgical tool according to the surgical plan.

[0109] In some embodiments, the button can be repeatedly selected to cause repeated adjustments (e.g., stepwise reductions to increasingly smaller sizes, toggling between two available sizes, sequentially traversing three or more available sizes, etc.) to one or more virtual dimensions of the surgical tool. Many such possibilities are within the scope of the present disclosure.

[0110] In other embodiments, the input is received from another source (e.g., a foot pedal, a voice-activated switch, a mouse, a keyboard, a touch screen, etc.). In other embodiments, a user input (described above as originating from a button) is replaced with an automatic response based on tracked positions or behaviors of the surgical tool and / or the robotic device (described in Figures 11 to 13 For example, in some embodiments, a dwell time of the surgical tool at the boundary of the haptic object is detected. When the dwell time exceeds a threshold amount of time, the haptic control can be modified as described in steps 1106-1108, 1206-1210, and / or 1306-1308. Various such modifications are within the scope of the present disclosure.

[0111] As mentioned above, Figures 4 to 13All combinations of the various features described throughout this disclosure are within the scope of the disclosure. For example, the process 300 can be implemented using steps of one or more of the processes 400, 600, 800, 900, 1100, 1200, and 1300. Moreover, it should be understood that the various features, method steps, etc. described herein can be adapted to facilitate a variety of surgical procedures, including total and partial hip, knee, and shoulder arthroplasty, as well as for carrying out non-surgical tasks.

[0112] The configuration and arrangement of the systems and methods as shown in the various example embodiments are merely illustrative. Although only a few embodiments have been described in detail in this disclosure, many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.). For example, the position of the elements can be reversed or otherwise changed and the nature or number of elements or positions can be altered or varied. Accordingly, all such modifications are intended to be included within the scope of the present disclosure. The order or sequence of any process or method steps can be varied or re-sequenced according to alternative embodiments. The systems described herein can be adapted to carry out the methods described herein. Other substitutions, modifications, changes, and omissions can be made in the design, operating conditions, and arrangement of the example embodiments without departing from the scope of the present disclosure.

[0113] As utilized herein the terms “approximately,” “about,” “substantially,” and similar terms are intended to have a broad meaning in harmony with the common and accepted usage of such terms by those of ordinary skill in the art to which the subject matter of this disclosure pertains. It is to be understood by those of ordinary skill in the art that such terms are intended to allow for a certain level of variation as is to be expected by virtue of physical measurements and the other uncertainties inherent in the art of physical testing. Accordingly, these terms should be interpreted as indicating that the described condition, situation, or circumstance is the intended one, but that minor variations are intended to be within the scope of the disclosure.

Claims

1. A method comprising: defining a virtual object; defining a first point and a second point associated with a virtual representation of a surgical tool, wherein movement of the virtual representation of the surgical tool corresponds to movement of the surgical tool in real space; controlling a robotic device coupled to the surgical tool to constrain the first point to the virtual object; determining that the first point is at a threshold position along the virtual object; and controlling the robotic device to direct the second point to the virtual object, wherein the method further comprises: determining that the second point is positioned at the virtual object; constraining the first point and the second point to the virtual object; detecting a force exerted on the tool in a predetermined direction, the predetermined direction being orthogonal to the virtual object; determining whether the force in the predetermined direction exceeds a threshold force; and controlling the robotic device to allow the first point and the second point to deviate from the virtual object in response to determining that the force in the predetermined direction exceeds a threshold force.

2. The method of claim 1, wherein controlling the robotic device to constrain the first point to the virtual object comprises controlling the robotic device to prevent the first point from moving away from the virtual object and to allow the surgical tool to rotate about the first point.

3. The method of claim 1, further comprising: defining a damping zone along the virtual object; determining that the first point is in the damping zone; controlling the robotic device to resist movement of the first point through the damping zone.

4. The method of claim 3, comprising: determining that the first point has passed through the damping zone; and removing the damping zone from the virtual object in response to determining that the first point has passed through the damping zone.

5. The method of claim 1, wherein controlling the robotic device to direct the second point to the virtual object comprises controlling the robotic device to provide an assistive force to the surgical tool, the assistive force being oriented to facilitate movement of the second point to the virtual object.

6. The method of claim 1, comprising: determining that the first point is moving toward the virtual object; and controlling the robotic device to provide an assistive force to the surgical tool in response to determining that the first point is moving toward the virtual object, the assistive force being oriented to facilitate movement of the first point toward a virtual line.

7. The method of claim 1, comprising: receiving a signal; and changing a control interaction between the first point and the virtual object in response to the signal.

8. The method of claim 1, wherein defining the virtual object comprises: obtaining a surgical plan including a planned hole in a target bone; and aligning the virtual object with the planned hole.

9. The method of claim 1, wherein the threshold position along the virtual object corresponds to a threshold distance from an anatomical feature. ​ ​ ​ ​ ​ 10. The method of claim 1, wherein the threshold position along the virtual object corresponds to an amount of penetration into an anatomical feature.

11. The method of claim 1, wherein the virtual object is a virtual line.

Images