Systems and methods for providing haptic guidance

Abstract

The invention relates to a system and method for providing haptic guidance. The method includes defining a virtual object and defining a first point and a second point associated with a virtual representation of a surgical tool. Movement of the virtual representation of the surgical tool corresponds to movement of the surgical tool in real space. The method includes controlling a robotic device coupled to the surgical tool to constrain the first point to the virtual object; determining that the first point is at a threshold location along the virtual object; and controlling the robotic device to guide the second point to the virtual object.

Description

[0001] This application is a divisional application of patent application No. 202080066072.X (International Application No. PCT / US2020 / 052466), international application date September 24, 2020, entitled "System and Method for Providing Tactile Guidance". Technical Field

[0002] This disclosure generally relates to surgical systems for orthopedic procedures, such as surgical systems that facilitate joint replacement surgery. Background Technology

[0003] Joint replacement surgery (arthroplasty) is widely used to treat osteoarthritis and other joint injuries in patients. It involves replacing different parts of a joint with prosthetic components. Joint replacement surgery can include procedures that replace the hip, knee, shoulder, or other joints with one or more prosthetic components.

[0004] One possible tool for arthroplasty is a robot-assisted surgical system. Robot-assisted surgical systems typically include: a robotic device for preparing the patient's anatomy to receive the implant; a tracking system configured to monitor the position of the robotic device relative to the patient's anatomy; and a computing system configured to monitor and control the robotic device. Robot-assisted surgical systems, in various forms, autonomously perform surgical tasks, provide force feedback to the user manipulating the surgical apparatus to complete the surgical task, enhance the surgeon's dexterity and precision, and / or provide other guiding cues to facilitate safe and accurate surgical procedures.

[0005] A surgical plan is typically determined before surgery is performed using a robot-assisted surgical system. Based on this plan, the surgical system guides, controls, or restricts the movement of surgical instruments during various stages of the surgery. The guidance and / or control of the surgical instruments assist the surgeon in achieving the surgical plan. Summary of the Invention

[0006] One implementation of this disclosure is a method for controlling a robotic device. The method includes defining a virtual object and defining a first point and a second point associated with a virtual representation of a surgical instrument. Movement of the virtual representation of the surgical instrument corresponds to movement of the surgical instrument in real space. The method includes: controlling the robotic device coupled to the surgical instrument to constrain the first point to the virtual object, determining the first point at a threshold position along the virtual object, and controlling the robotic device to guide the second point to the virtual object.

[0007] Another implementation of this disclosure is a system including a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to define a virtual object and define a first point and a second point associated with a virtual representation of a surgical instrument. The processing circuitry is configured such that movement of the virtual representation of the surgical instrument corresponds to movement of the surgical instrument in real space. The processing circuitry is also configured to control the robotic device coupled to the surgical instrument to constrain the first point to the virtual object, determine that the first point is at a threshold position along the virtual object, and control the robotic device to guide the second point to the virtual object.

[0008] Another implementation of this disclosure is a method of operating a robotic device coupled with a tool. The method includes controlling the robotic device to constrain a surgical tool based on a first tactile object, receiving signals and a user-defined orientation, and adjusting tactile control interactions in response to the signals by extending the first tactile object in the user-defined orientation.

[0009] Another implementation of this disclosure is a system comprising a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to control the robotic device to constrain surgical instruments based on a first tactile object, receive signals and user-defined orientations, and adjust tactile control interactions in response to signals by extending the first tactile object in the user-defined orientation.

[0010] Another implementation of this disclosure is a method of operating a robotic device coupled with a tool. The method includes controlling the robotic device to constrain the tool based on a first tactile object, receiving signals and user-defined orientations, and adjusting tactile control interactions in response to the signals by adjusting the virtual size of the tool.

[0011] Another implementation of this disclosure is a system comprising a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to control the robotic device to constrain the tool based on a first tactile object, receive signals and user-defined orientations, and adjust the tactile control interaction in response to signals by adjusting the virtual dimensions of the tool.

[0012] Another implementation of this disclosure is a method for operating a robotic device. The method includes tracking the movement of a tool coupled to the robotic device, determining the direction of movement of the tool, determining whether the direction of movement points toward a virtual controlled object, and controlling the robotic device to guide the tool toward the virtual controlled object in response to determining that the direction of movement points toward the virtual controlled object.

[0013] Another implementation of this disclosure is a system comprising a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to receive tracking data indicative of movement of a tool coupled to the robotic device, determine the direction of movement of the tool, determine whether the direction of movement points toward a virtual controlled object, and, in response to determining that the direction of movement points toward the virtual controlled object, control the robotic device to guide the tool toward the virtual controlled object.

[0014] Another implementation of this disclosure is a method for operating a robotic device. The method includes: tracking a tool coupled to the robotic device; controlling the robotic device to constrain the tool within a virtual control object; defining a region of the virtual control object; identifying the tool within that region; and controlling the robotic device to resist movement of the tool within that region.

[0015] Another implementation of this disclosure is a system comprising a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to control the robotic device to constrain a tool within a virtual controlled object, define an area of ​​the virtual controlled object, determine the tool within that area, and control the robotic device to resist movement of the tool within that area.

[0016] Another implementation of this disclosure is a method of operating a robotic device, the robotic device being coupled to a tool. The method includes: constraining the tool to a virtual control object via the robotic device; detecting a force applied to the tool in a generally predetermined direction; determining whether the force in the generally predetermined direction exceeds a threshold force; and controlling the robotic device to allow the tool to exit the virtual control object in response to determining that the force in the generally predetermined direction exceeds the threshold force.

[0017] Another implementation of this disclosure is a system comprising a robotic device and processing circuitry communicable with the robotic device. The processing circuitry is configured to control the robotic device to constrain a tool to a virtual controlled object, detect a force applied to the tool in a generally predetermined direction, determine whether the force in the generally predetermined direction exceeds a threshold force, and, in response to determining that the force in the generally predetermined direction exceeds the threshold force, control the robotic device to allow the tool to exit the virtual controlled object. Attached Figure Description

[0018] Figure 1 This is a perspective view of the femur ready to receive an implant component according to an exemplary embodiment.

[0019] Figure 2 This is an illustration of a surgical system according to an exemplary embodiment.

[0020] Figure 3 It is possible to be based on the exemplary implementation scheme Figure 2The flowchart of the first process performed by the surgical system.

[0021] Figure 4 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the second process performed by the surgical system.

[0022] Figure 5 According to the exemplary implementation scheme Figure 4 A diagram illustrating the process.

[0023] Figure 6 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the third process performed by the surgical system.

[0024] Figure 7 According to the exemplary implementation scheme, it can be used with Figure 6 A diagram of a virtual control object used in the process.

[0025] Figure 8 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the fourth process performed by the surgical system.

[0026] Figure 9 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the fifth process performed by the surgical system.

[0027] Figure 10 It is compatible with the exemplary implementation scheme Figure 9 A diagram of a virtual control object used in the process.

[0028] Figure 11 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the sixth process performed by the surgical system.

[0029] Figure 12 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the seventh process performed by the surgical system.

[0030] Figure 13 It is possible to be based on the exemplary implementation scheme Figure 2 The flowchart of the eighth process performed by the surgical system. Detailed Implementation

[0031] The presently preferred embodiment of the invention is illustrated in the accompanying drawings. Efforts have been made to use the same or similar reference numerals throughout the drawings to refer to the same or similar parts. While this specification relates primarily to robotic arms for orthopedic joint replacement, it should be understood that the subject matter described herein is applicable to other types of robotic systems, including robotic systems for non-surgical applications, and surgeries targeting other anatomical regions, such as spinal or dental procedures.

[0032] Now see Figure 1 According to an exemplary embodiment, a femur 101 as modified during knee arthroplasty is shown. Figure 1 As shown, the femur 101 has been modified to have multiple planar incisions. In the example shown, the femur 100 has been modified with incisions in five basic planes to create five basic planar surfaces: a distal surface 102, a posterior chamfered surface 104, a posterior surface 106, an anterior surface 108, and an anterior chamfered surface 110. Planar surfaces can be achieved using a sagittal saw or other surgical tools (e.g., surgical tools attached to a robotic device, as shown in the example described below). Planar surfaces 102 to 110 are created such that they will mate with corresponding surfaces of the femoral implant component. The position and angular orientation of the flat surfaces 102 to 110 determine the alignment and positioning of the implant component. Therefore, operating surgical tools with high precision to create planar surfaces 102 to 110 improves the outcome of joint replacement surgery.

[0033] like Figure 1 As shown, the femur 101 is also modified to have a pair of guide holes 120. The guide holes 120 extend into the femur 101 and are created such that they can receive screws, protrusions extending from the surface of the implant component, or other structures configured to facilitate attachment of the implant component to the femur 101. The guide holes 120 can be created using a drill, ball drill, or other surgical instruments described below. The guide holes 120 can have a pre-planned location, orientation, and depth, which helps to securely attach the implant component to the bone in the desired location and orientation. In some cases, the guide holes 120 are designed to intersect with high-density areas of the bone and / or avoid other implant components and / or sensitive anatomical features. Therefore, operating surgical instruments with high precision to create the guide holes 120 can improve the outcome of joint replacement surgery.

[0034] In some implementations, the systems and methods described herein provide robot assistance for creating planar surfaces 102 to 110 and guide holes 120. It should be understood that, as Figure 1The five planar incisions and two cylindrical guide holes shown are merely examples, and the systems and methods described herein are adaptable to planning and facilitating the creation of any number of planar or non-planar incisions, any number of guide holes, any combination thereof, etc., for preparing any bone and / or joint in various embodiments. For example, during hip or shoulder arthroplasty, a ball drill can be used to enlarge a curved surface configured to receive a curved implant cup, according to the systems and methods described herein. Furthermore, in other embodiments, the systems and methods described herein can be used to facilitate the placement of implant components relative to bone (e.g., to facilitate the clamping of a cup-shaped implant in hip arthroplasty). Many such surgical and non-surgical implementations are within the scope of this disclosure.

[0035] Now see Figure 2 According to an exemplary embodiment, a surgical system 200 for orthopedic surgery is shown. Generally, the surgical system 200 is configured to facilitate the planning and execution of surgical procedures, such as joint-related surgeries. Figure 2 As shown, a surgical system 200 is configured to treat the legs 202 of a patient 204 who is sitting or lying on an operating table 205. Figure 2 In the illustration shown, leg 202 includes femur 206 (e.g., Figure 1 The femur 101 and tibia 208 are used in total knee arthroscopy, between which a prosthetic knee implant will be inserted. In other scenarios, the surgical system 200 is set up to treat a patient's hip, i.e., the patient's femur and pelvis. Additionally, in other scenarios, the surgical system 200 is set up to treat a patient's shoulder, i.e., to facilitate the replacement and / or enlargement of components of the shoulder joint (e.g., to facilitate the placement of humeral components, glenoid components, and graft or implant reinforcements). Various other anatomical regions and procedures are also possible. To facilitate the surgery, the surgical system 200 includes a robotic device 220, a tracking system 222, and a computing system 224.

[0036] Robotic device 220 is configured to modify the patient's anatomy (e.g., the femur 206 of patient 204) under the control of computing system 224. One embodiment of robotic device 220 is a tactile device. "Tactile" refers to the sensation of touch, and the field of tactile sensing particularly relates to human interactive devices that provide feedback to an operator. Feedback may include tactile sensations, such as vibrations. Feedback may also include providing force to the user, such as positive force or resistance to movement. One use of tactile sensing is to provide guidance or constraints to the user of a device in manipulating it. For example, a tactile device may be coupled to a surgical instrument that can be manipulated by a surgeon to perform surgery. The surgeon's manipulation of the surgical instrument can be guided or constrained by using tactile sensing to provide feedback to the surgeon during manipulation of the surgical instrument.

[0037] Another embodiment of the robotic device 220 is an autonomous or semi-autonomous robot. "Autonomy" refers to the ability of a robotic device to act independently or semi-independently from human control by collecting information about its situation, determining course of action, and automatically implementing that course of action. For example, in this embodiment, the robotic device 220, communicating with the tracking system 222 and the computing system 224, can autonomously perform the aforementioned series of femoral cuts without direct human intervention.

[0038] The robotic device 220 includes a base 230, a robotic arm 232, and surgical instruments 234, and is communicatively coupled to a computing system 224 and a tracking system 222. The base 230 provides a movable foundation for the robotic arm 232, allowing the robotic arm 232 and surgical instruments 234 to be repositioned relative to the patient 204 and the operating table 205 as needed. The base 230 may also contain a power system, computing elements, motors, and other electronic or mechanical systems necessary for the functionality of the robotic arm 232 and surgical instruments 234, as described below.

[0039] The robotic arm 232 is configured to support the surgical instrument 234 and provide force as instructed by the computing system 224. In some embodiments, the robotic arm 232 allows a user to manipulate the surgical instrument and provides force feedback to the user. In such embodiments, the robotic arm 232 includes joints 236 and supports 238, which include motors, actuators, or other mechanisms configured to allow the user to freely translate and rotate the robotic arm 232 and the surgical instrument 234 in permissible postures, while providing force feedback to restrain or prevent some movement of the robotic arm 232 and the surgical instrument 234, as instructed by the computing system 224. As described in detail below, the robotic arm 232 thereby allows the surgeon to have complete control of the surgical instrument 234 within the controlled object, while providing force feedback (e.g., vibration, forces that prevent or resist boundary penetration) along the boundaries of the object. In some implementations, the robotic arm is configured to automatically move surgical instruments to new positions as instructed by the computing system 224 without direct user intervention, in order to position the robotic arm as needed and / or perform certain surgical tasks, including, for example, an incision in the femur 206.

[0040] Surgical tool 234 is configured to cut, burr, grind, drill, partially remove, reshape, and / or otherwise modify bone, or to constrain / restrict the movement of devices used for cutting, burring, grinding, drilling, partially removing, reshaping, and / or otherwise modifying bone. Surgical tool 234 may be any suitable tool and may be one of a variety of tools interchangeably connected to robotic device 220. For example, such as... Figure 2 As shown, surgical tool 234 includes a ball drill 244. In other examples, the surgical tool may also be a sagittal saw, for example, having blades aligned parallel to or perpendicular to the tool axis. The surgical tool may also be a drill, for example, having a rotary drill bit aligned parallel to or perpendicular to the tool axis. In various embodiments, surgical tool 234 may be a clamp, drill guide, cutting guide, etc., for example, configured to have a saw, drill, or other instrument inserted therein. Surgical tool 234 may also be a holding arm or other support configured to hold implant components (e.g., cup 28a, implant reinforcement, etc.) in place while screwing the implant components onto bone, adhering (e.g., gluing) them to bone or other implant components, or otherwise mounting them in a preferred location. In some embodiments, surgical tool 234 is a clamping tool configured to provide clamping force to the cup-shaped implant to facilitate fixation of the cup-shaped implant to the pelvis in a planned position and orientation.

[0041] The tracking system 222 is configured to track the patient's anatomical structures (e.g., femur 206 and tibia 208) and the robotic device 220 (i.e., surgical instrument 234 and / or robotic arm 232) to allow control of the surgical instrument 234 coupled to the robotic arm 232, thereby determining the position and orientation of modifications or other results made by the surgical instrument 234, and allowing the user to visualize the bones (e.g., femur 206, tibia 208, pelvis, humerus, scapula, etc., as applicable in various surgical procedures), surgical instrument 234, and / or robotic arm 232 on the display of the computing system 224. More specifically, the tracking system 222 determines the position and orientation (i.e., pose) of the object (e.g., surgical instrument 234, femur 206) relative to a reference coordinate system and tracks (i.e., continuously determines) the pose of the object during surgery. According to various implementations, the tracking system 222 may be any type of guidance system, including non-mechanical tracking systems (e.g., optical tracking systems), mechanical tracking systems (e.g., tracking based on measuring the relative angles of the joints 236 of the robot arm 232), or any combination of non-mechanical and mechanical tracking systems.

[0042] exist Figure 2 In the embodiment shown, the tracking system 222 includes an optical tracking system. Therefore, the tracking system 222 includes: a first reference tree 240 coupled to the tibia 208; a second reference tree 241 coupled to the femur 206; a third reference tree 242 coupled to the base 230; one or more references coupled to the surgical instrument 234; and a detection device 246 configured to detect the three-dimensional position of the references (i.e., marks on the reference trees 240 to 242). Reference trees 240, 241 may be coupled to other bones suitable for various surgeries (e.g., the pelvis and femur in hip replacement surgery). The detection device 246 may be an optical detector, such as a camera or an infrared sensor. Reference trees 240 to 242 include references that are marks configured to be clearly displayed to the optical detector and / or easily detected by an image processing system using data from the optical detector, for example, by highly reflective infrared radiation (e.g., emitted by elements of the tracking system 222). The stereo arrangement of the cameras on the detection device 246 allows the position of each reference in 3D space to be determined using triangulation methods. Each reference has a geometric relationship with the corresponding object, such that tracking the references allows tracking the object (e.g., tracking the second reference tree 241 allows the tracking system 222 to track the femur 206), and the tracking system 222 can be configured to perform a registration process to determine or verify this geometric relationship. The unique arrangement of the references in reference trees 240 to 242 (i.e., the references in the first reference tree 240 are arranged with a different geometry than the references in the second reference tree 241) allows the reference trees to be distinguished from each other and thus the objects to be tracked to be distinguished from each other.

[0043] use Figure 2 The surgical system 200 can determine the position of the surgical instrument 234 relative to the patient's anatomical features (e.g., femur 206) using a tracking system 222 or some other method of surgical guidance and tracking, because the surgical instrument 234 is used to modify anatomical features or otherwise facilitate surgery. Additionally, using... Figure 2 The tracking system 222 or some other method for surgical guidance and tracking, the surgical system 200 can determine the relative posture of the tracked bones.

[0044] The computing system 224 is configured to create surgical plans, control the robotic device 220 according to the surgical plans to perform one or more bone modifications, and / or facilitate the implantation of one or more prosthetic components. Therefore, the computing system 224 is communicatively coupled to the tracking system 222 and the robotic device 220 to facilitate electronic communication between the robotic device 220, the tracking system 222, and the computing system 224. Furthermore, the computing system 224 can be connected to a network to receive information related to the patient's medical history or other patient profiles, medical imaging, surgical plans, surgical procedures, and to perform various functions related to the execution of surgery, such as by accessing an electronic health record system. The computing system 224 includes processing circuitry 260 and input / output devices 262.

[0045] Input / output device 262 is configured to receive user input and display output as needed for the functions and processes described herein. For example... Figure 2 As shown, the input / output device 262 includes a display 264 and a keyboard 266. The display 264 is configured to display a graphical user interface generated by the processing circuitry 260, which includes information such as surgical planning, medical imaging, settings and other options for the surgical system 200, status information related to the tracking system 222 and the robotic device 220, and tracking visualization based on data supplied by the tracking system 222. The keyboard 266 is configured to receive user input from these graphical user interfaces to control one or more functions of the surgical system 200.

[0046] Processing circuitry 260 includes a processor and a memory device. The processor may be implemented as a general-purpose processor, an application-specific integrated circuit (ASIC), one or more field-programmable gate arrays (FPGAs), a set of processing units, or other suitable electronic processing units. The memory device (e.g., memory, storage cell, storage device, etc.) is one or more means (e.g., RAM, ROM, flash memory, hard disk storage, etc.) for storing data and / or computer code used to perform or contribute to the various processes and functions described herein. The memory device may be or include volatile or non-volatile memory. The memory device may include database components, object code components, scripting components, or any other type of information structure for supporting the various activities and information structures described herein. According to an exemplary embodiment, the memory device is communicatively connected to the processor via processing circuitry 260 and includes computer code for performing (e.g., by processing circuitry 260 and / or the processor) one or more processes described herein.

[0047] More specifically, processing circuitry 260 is configured to facilitate the creation of a preoperative surgical plan prior to the surgical procedure. According to some embodiments, the preoperative surgical plan is developed using a three-dimensional representation of the patient's anatomy, also referred to herein as a "virtual skeletal model." In addition to bone, the "virtual skeletal model" may also include virtual representations of cartilage or other tissues. To obtain the virtual skeletal model, processing circuitry 260 receives imaging data of the patient's anatomy to be operated on. Any suitable medical imaging technique can be used to create the imaging data to image the relevant anatomical features, including computed tomography (CT), magnetic resonance imaging (MRI), and / or ultrasound. The imaging data is then segmented (i.e., distinguishing regions in the imaging corresponding to different anatomical features) to obtain the virtual skeletal model. For example, MRI-based scan data of a joint can be segmented to distinguish the bone from surrounding ligaments, cartilage, previously implanted prosthetic components, and other tissues to obtain an imaged three-dimensional model of the bone.

[0048] Alternatively, a virtual skeletal model can be obtained by selecting a 3D model from a database or a library of skeletal models. In one embodiment, the user can use input / output device 262 to select an appropriate model. In another embodiment, processing circuitry 260 can execute stored instructions to select an appropriate model based on provided images or other information about the patient. The skeletal model selected from the database can then be deformed based on specific patient characteristics to create a virtual skeletal model for surgical planning and implementation as described herein.

[0049] A preoperative surgical plan can then be created based on the virtual skeletal model. The surgical plan can be automatically generated by processing circuitry 260, input by the user via input / output device 262, or some combination of both (e.g., processing circuitry 260 restricts certain features of the user-created plan, generates a user-modifiable plan, etc.). In some implementations, the surgical plan can be generated and / or modified based on force measurements collected intraoperatively.

[0050] Preoperative surgical planning includes desired incisions, holes, surfaces, burrs, or other modifications to the patient's anatomy using the surgical system 200. For example, for total knee arthroscopy, preoperative planning may include: necessary incisions to be formed on the femur, distal surface, posterior chamfered surface, posterior surface, anterior surface, and anterior chamfered surface, the relative orientation and location of which are adapted to fit the corresponding surface of the prosthesis to be engaged with the femur during surgery; and necessary incisions to be formed on the tibia, adapted to fit the prosthesis to be engaged with the tibia during surgery. As another example, preoperative planning may include modifications necessary to create holes (e.g., guide holes 120) in the bone. As another example, during hip arthroplasty, surgical planning may include burrs necessary to form one or more surfaces in the acetabular region of the pelvis to receive a cup and, where appropriate, to receive an implant reinforcement. Therefore, processing circuitry 260 may receive, access, and / or store prosthesis models to facilitate the generation of surgical plans.

[0051] Processing circuitry 260 is also configured to generate control objects for robotic device 220 based on a surgical plan. Control objects can take various forms depending on the possible types of robotic devices (e.g., tactile, autonomous). For example, in some embodiments, the control object defines instructions for the robotic device to control its movement within the control object (i.e., autonomously making one or more incisions according to the surgical plan under feedback instructions from tracking system 222). In some embodiments, the control object includes the surgical plan and visualization of the robotic device on display 264 to aid in surgical guidance and help guide the surgeon to follow the surgical plan (e.g., without active control or force feedback from the robotic device). In embodiments where robotic device 220 is a tactile device, the control object may be a tactile object as described in the following paragraphs.

[0052] In an embodiment where the robotic device 220 is a tactile device, the processing circuitry 260 is also configured to generate one or more tactile objects based on a preoperative surgical plan to assist the surgeon during the implementation of the surgical plan by constraining the surgical instrument 234 during the surgical procedure. The tactile objects can be formed in one, two, or three dimensions. For example, a tactile object may be a line, a plane, or a three-dimensional volume. A tactile object may be curved, have curved surfaces, and / or have planar surfaces, and may be of any shape, such as a funnel shape. Tactile objects can be created to represent various desired outcomes related to the movement of the surgical instrument 234 during the surgical procedure. One or more of the boundaries of a three-dimensional tactile object may represent one or more modifications to be created on a bone surface, such as incisions. Planar tactile objects may represent modifications to be created on a bone surface, such as incisions. Curved tactile objects may represent a resulting bone surface, such as modified to receive a cup-shaped implant and / or implant reinforcement. Linear tactile objects may correspond to guide holes to be made in the bone to prepare the bone to receive screws or other protrusions.

[0053] In an embodiment where the robotic device 220 is a tactile device, the processing circuitry 260 is also configured to generate a virtual tool representation of the surgical tool 234. The virtual tool includes one or more tactile interaction points (HIPs) that represent and are associated with positions on and on the physical surgical tool 234. In an embodiment where the surgical tool 234 is a ball drill (e.g., as...),... Figure 2 In the embodiment shown, the HIP may represent the center of a spherical drill. In the case where a HIP is used to virtually represent a surgical instrument, the HIP may be referred to herein as the tool center point (TCP). If the surgical instrument 234 is irregularly shaped, such as for a sagittal saw, the virtual representation of the sagittal saw may include numerous HIPs. U.S. Application Serial No. 13 / 339,369, filed December 28, 2011, entitled “System and Method for Providing Substantially Stable Haptics,” and hereby incorporated herein by reference in its entirety, describes the use of multiple HIPs to generate tactile forces (e.g., positive force feedback or resistance to movement) on a surgical instrument. In one embodiment of the invention, the virtual tool representing the sagittal saw includes eleven HIPs. As used herein, reference to “HIP” is considered to also include reference to “one or more HIPs.” As described below, the relationship between the HIP and the tactile object enables the surgical system 200 to constrain the surgical instrument 234.

[0054] Prior to surgery, the patient's anatomical structures (e.g., femur 206) are registered to a virtual skeletal model of the patient's anatomical structures using any known registration technique. One possible registration technique is point-based registration, as described in U.S. Patent No. 8,010,180, entitled "Haptic Guidance System and Method," issued August 30, 2011, and hereby incorporated herein by reference in its entirety. Alternatively, registration can be performed using a handheld radiographic imaging device for 2D / 3D registration, as described in U.S. Application Serial No. 13 / 562,163, filed July 30, 2012, entitled "Radiographic Imaging Device," and hereby incorporated herein by reference in its entirety. Registration also includes registering surgical instrument 234 to a virtual tool representation of surgical instrument 234, enabling the surgical system 200 to determine and monitor the posture of surgical instrument 234 relative to the patient (i.e., relative to femur 206). Registration allows for precise guidance, control, and / or force feedback during surgery. Additional details relating to registration in some implementation schemes are described below.

[0055] Processing circuitry 260 is configured to monitor the virtual positions of virtual tool representations, virtual skeletal models, and control objects (e.g., virtual haptic objects) corresponding to the real-world positions of the patient's bones (e.g., femur 206), surgical instruments 234, and forces defined by robotic device 220 in one or more lines, planes, or three-dimensional spaces. For example, if the patient's anatomy moves during surgery, as tracked by tracking system 222, processing circuitry 260 moves the virtual skeletal model accordingly. Thus, the virtual skeletal model corresponds to and is associated with the patient's actual (i.e., physical) anatomy and its position and orientation in real / physical space. Similarly, the movements of any haptic objects, control objects, or other planned movements of the anatomical structure, created during surgical planning and connected to incisions, modifications, etc., to be made to that anatomy, also move correspondingly to the patient's anatomy. In some embodiments, surgical system 200 includes clamps or supports to substantially immobilize femur 206 to minimize the need to track and manipulate the movement of femur 206.

[0056] For the implementation of the robotic device 220 as a tactile device, the surgical system 200 is configured to constrain the surgical tool 234 based on the relationship between the HIP and the tactile object. That is, when the processing circuitry 260 uses data supplied by the tracking system 222 to detect that a user is manipulating the surgical tool 234 to make virtual contact between the HIP and the tactile object, the processing circuitry 260 generates control signals to the robotic arm 232 to provide tactile feedback (e.g., force, vibration) to the user to convey the constraint on the movement of the surgical tool 234. Generally, the term "constraint" as used herein is used to describe a tendency to constrain movement. However, the form of the constraint imposed on the surgical tool 234 depends on the form of the associated tactile object. The tactile object can be formed in any desired shape or configuration. As described above, the three exemplary embodiments include lines, planes, or three-dimensional volumes. In one embodiment, the surgical tool 234 is constrained because the HIP of the surgical tool 234 is constrained to move along a linear tactile object. In another embodiment, the tactile object is a three-dimensional volume and can constrain the surgical tool 234 by substantially preventing the HIP from moving outside the volume surrounded by the walls of the three-dimensional tactile object. In another embodiment, the surgical tool 234 is constrained because the planar tactile object substantially prevents the HIP from moving outside the plane and outside the boundaries of the planar tactile object. For example, processing circuitry 260 can create a planar tactile object corresponding to a planned planar distal incision required to create a distal surface on the femur 206, so as to substantially confine the surgical tool 234 to the plane required to implement the planned distal incision.

[0057] In an implementation where the robotic device 220 is an autonomous device, the surgical system 200 is configured to autonomously move and manipulate the surgical tool 234 according to a controlled object. For example, the controlled object may define an area relative to the femur 206 where an incision is to be made. In this case, one or more motors, actuators, and / or other mechanisms of the robotic arm 232 and the surgical tool 234 are controllable, causing the surgical tool 234 to move and manipulate within the controlled object as needed to make the planned incision, for example, using tracking data from the tracking system 222 to achieve closed-loop control.

[0058] Now see Figure 3 This illustrates a method that can be implemented according to an exemplary embodiment. Figure 2 The flowchart of the procedure 300 performed by the surgical system 200 is shown. Procedure 300 is adaptable to facilitate various surgical procedures, including total and partial joint replacement surgery. It can be used... Figures 4 to 13 The various steps and features (including combinations thereof) shown in the diagram and described in detail below are used to carry out process 300.

[0059] At step 302, a surgical plan is obtained. The surgical plan (e.g., a computer-readable data file) defines the desired outcome of the bone modification, for example, based on the desired location of the prosthetic component relative to the patient's anatomy. For example, in the case of knee arthroplasty, the surgical plan may provide, for example, Figure 1 The planar positions and orientations of the planar surfaces 102 to 110 and the guide hole 120 are shown in the diagram. Surgical plans can be generated based on medical imaging, 3D modeling, surgeon input, etc.

[0060] At step 304, one or more control boundaries, such as tactile objects, are defined based on the surgical plan. The one or more tactile objects may be one-dimensional (e.g., linear tactile), two-dimensional (i.e., planar), or three-dimensional (e.g., cylindrical, funnel-shaped, curved, etc.). The tactile objects may represent planned skeletal modifications defined by the surgical procedure (e.g., for...). Figure 1 The tactile objects (such as each of the planar surfaces 102 to 110 and each of the guide holes 120 shown), implant components, surgical method trajectories, etc., can be oriented and positioned in three-dimensional space relative to the tracked position of the patient's anatomy.

[0061] At step 306, the posture of the surgical instrument is tracked relative to the tactile object, for example, by means of the aforementioned tracking system 222. In some embodiments, a point on the surgical instrument is tracked. In other embodiments (e.g., in...) Figures 4 to 5 In one example, two points on the surgical instrument are tracked, such as the tool center point (TCP) at the tip / effective end of the surgical instrument and a second interaction point (SIP) located along the body or handle portion of the surgical instrument. In other embodiments, three or more points on the surgical instrument are tracked. The posture of the surgical instrument is determined relative to a coordinate system that defines one or more tactile objects, and in some embodiments, the posture of one or more anatomical features of the patient is also tracked.

[0062] At step 308, the surgical instrument is guided to the tactile object. For example, the display 264 of the surgical system 200 may display a graphical user interface instructing the user on how (e.g., in which direction) to move the surgical instrument and / or robotic device to bring the surgical instrument to the tactile object. As another example, the folded tactile boundary described in U.S. Patent No. 9,289,264, the entire disclosure of which is incorporated herein by reference. As another example, the robotic device may be controlled to automatically move the surgical instrument to the tactile object. As another example, the method described in detail below may be used. Figure 8 The process involves step 308, which is carried out in step 800.

[0063] In step 310, the robotic device is controlled to constrain the movement of the surgical tool based on the tracked pose of the surgical tool and the pose of one or more tactile objects. See above for reference. Figure 2 The aforementioned method achieves constraint on the surgical instruments. In some embodiments, step 310 includes, for example, Figures 4 to 5 Shown and referenced Figures 4 to 5 The two-stage method described below. In some embodiments, step 310 includes providing a damping force to resist the movement of surgical instruments through a specific area, such as... Figures 9 to 10 As described herein and referenced below. Figures 9 to 10 The following description is provided. In some implementations, step 310 includes adjusting the tactile interaction in response to user input, for example, according to the reference below. Figures 11 to 13 The description Figures 11 to 13 Examples. Various combinations of these features are possible at step 310.

[0064] At step 312, this facilitates the withdrawal of the surgical instrument from the tactile object, i.e., to release the constraints on the tactile object. For example, in some embodiments, the robotic device is controlled to allow the surgical instrument to withdraw from the tactile object along its axis. In some embodiments, for example, as... Figures 6 to 7 As shown and referenced below Figures 6 to 7 As described, surgical instruments can be withdrawn from a tactile object in a predetermined direction relative to the tactile object. This allows the surgical instruments to be removed from the surgical area and the tactile object to facilitate subsequent surgical procedures. Additionally, it should be understood that in some cases, process 300 can return to step 308, where the surgical instruments are guided to the same or a different tactile object after withdrawal from the tactile object at step 312.

[0065] Thus, procedure 300 can be performed by surgical system 200 to facilitate surgical procedures. According to some embodiments, the characteristics of procedure 300 are as follows: Figures 4 to 13 As shown, and in various embodiments, such features can be combined in various combinations and / or based on settings selected for a particular procedure. Furthermore, it should be understood that some or all other steps of process 300 may be provided while omitting them. Figures 4 to 13 The characteristics of [the subject]. All such possibilities are within the scope of this disclosure.

[0066] Now see Figures 4 to 5 A two-stage haptic interaction is illustrated according to an exemplary implementation. Figure 4 A flowchart of process 400 for providing two-stage haptic interaction is shown, while Figure 5A storyboard illustration of a two-stage tactile interaction (i.e., process 400) is shown. For example, at step 310 of process 300, process 400 can be performed by surgical system 200.

[0067] At step 402, the tool center point (TCP) 500 is constrained to the line haptic 502, while allowing the surgical tool 234 to rotate around the tool center point 500 and the TCP 500 to translate along the line haptic 502. The line haptic 502 may correspond to planned skeletal modifications, such as... Figure 1 The planned guide hole 120 is shown in the diagram. The line tactile sensor 502 may be defined as the axis of the planned guide hole 120 and may extend from the bottom (deepest end) of the planned guide hole 120 beyond anatomical features (e.g., femur 101). The line tactile sensor 502 in... Figure 5 In the example, the line tactile feedback 502 is a straight line. In other embodiments, the line tactile feedback 502 may be curved, for example, defined as a spline. In other embodiments, Figure 4 The process 400 may be adapted to use planar tactile objects, volumetric tactile objects, or tactile objects limited to any combination of lines, planes, and / or volumes to replace line tactile 502.

[0068] The TCP 500 is tracked relative to the line haptic 502, and the robotic device is controlled to constrain the TCP 500 to remain on or substantially on the line haptic 502 (e.g., to prevent or resist deviation from the line haptic 502, to provide elasticity to drive the TCP 500 back to the line haptic 502, etc.). At step 402, the TCP 500 is translated along the line haptic 502. The robotic device is controlled (e.g., via access control) to allow the surgical instrument 234 to rotate around the TCP 500 (e.g., as manipulated by a user). That is, the second interaction point (SIP) 504, located along the handle or body portion of the surgical instrument 234, is unconstrained at step 402.

[0069] At step 402, rotation of the surgical instrument 234 about TCP 500 can help the surgeon reach the line touch 502 along a preferred approach trajectory. In some cases, surrounding soft tissue and / or bone structures may make it difficult or impossible for the surgical instrument 234 to be inserted along the line touch 502 from a position completely outside the patient's body into the bone surface without causing undesirable or unnecessary damage to the surrounding tissue or bone (e.g., it is not necessary to create a hole through this tissue or bone). In such cases, the surgical instrument 234 may be inserted along a preferred trajectory until TCP 500 reaches and is constrained by the line touch 502. At step 402, the surgical instrument 234 may be rotated to transfer the anatomical feature by pushing such a feature with the axis or one side of the body of the surgical instrument. By constraining TCP 500 to the line touch 502, the surgeon is allowed at step 402 to focus on rotating the surgical instrument 234 as needed without the burden of simultaneously monitoring the position of TCP 500 and / or attempting to manually prevent TCP 500 from moving away from the desired axis. Thus, step 402 can facilitate the insertion and orientation of the surgical instrument 234 in various anatomical regions.

[0070] At step 404, it is determined (e.g., by processing circuitry 260) that TCP 500 has reached a threshold position along the line of touch 502. In some cases, the threshold position is defined based on the distance from the surface of the bone (e.g., femur 101), such that the threshold position is reached before the surgical tool 234 contacts the bone. In such cases, steps 406 to 408 may be performed as described below before the surgical tool 234 begins modifying the bone, thereby ensuring the desired orientation of the surgical tool 234 before initiating bone modification.

[0071] In other cases, the threshold position is defined based on the depth beneath the bone surface, such that the surgical instrument 234 reaches the threshold position after contacting the bone. In such cases, it may be permissible for the surgical instrument 234 to begin modifying the bone in a first orientation before rotating to align with the line tactile 502 as described below with reference to steps 406 to 408, for example to reduce the risk of scraping or otherwise facilitate the surgical instrument 234's entry into the bone. Figure 5As shown, for example, initial contact can be made between the surgical tool 234 and the bone 101, wherein the surgical tool is approximately orthogonal to the bone surface as shown in the upper frame (i.e., during step 402). This increases the likelihood of achieving a clean initial incision / hole / drill / etc. at a planned location in the bone 101 (i.e., at the intersection between the online haptic 502 and the bone 101). In this case, it can then be determined that TCP 500 has reached a threshold position after the surgical tool 234 has initially penetrated the bone 101 (i.e., so that TCP 500 has crossed the surface of the bone 101).

[0072] At step 406, in response to determining at step 404 that TCP 500 has reached a threshold position, SIP 504 is guided toward the line tactile 502. In some embodiments, a robotic device may be controlled to provide an assistive force that assists a user rotating the surgical tool 234 around TCP 500 in moving SIP 504 toward the line tactile 502. In some embodiments, a folded tactile object is used at step 406, which prevents SIP 504 from rotating away from the line tactile 502 while allowing SIP 504 to rotate toward the line tactile 502. In some embodiments, guiding SIP 504 toward the line tactile 502 is achieved by displaying instructions via display 264. In some embodiments, guiding SIP 504 toward the line tactile 502 is achieved by controlling a robotic device to automatically rotate the surgical tool 234 to align SIP 504 with the line tactile 502. During step 406 (i.e., when SIP 504 is guided to line haptics 502), TCP 500 is constrained to line haptics 502 (as described for step 402). In some cases, the robot device is controlled to prevent TCP 500 from translating along line haptics 502 while SIP 504 is guided to line haptics 502 during step 406.

[0073] At step 408, after SIP 504 has been guided to line haptics 502 due to step 406, the robotic device is controlled to constrain TCP 500 and SIP 504 to line haptics 502. Surgical tool 234 can be translated along line haptics 502 to perform planned skeletal modifications (e.g., to create guide holes 120). In the example shown, SIP 504 is positioned along the axis of surgical tool 234. Thus, alignment of surgical tool 234 with line haptics 502 is maintained by constraining two points of the other surgical tool 234 to line haptics 502 (i.e., TCP 500 and SIP 504). In other embodiments, SIP 504 is guided to a second haptic (i.e., a virtual haptic object different from line haptics 502). In this embodiment, TCP 500 and SIP 504 are confined to different haptic objects. For example, if the surgical instrument 234 is curved, the SIP 504 may be constrained on a curve, while the TCP 500 may be constrained on a straight line (or vice versa), in order to achieve the desired degrees of freedom and constraints of movement of the surgical instrument 234.

[0074] Other geometries and behaviors can also be achieved by using different haptic objects for SIP 504 and TCP 500. For example, TCP 500 can be limited to a haptic object with a geometry corresponding to a planned incision or drilling path, while SIP 504 can be limited to a different haptic object configured to prevent or resist collisions between the axis of surgical tool 234 (or another point on the robotic arm) and one or more objects in the surgical area. For example, SIP 504 can be limited to a haptic object with a geometry based on the location of a retractor or other tool in the surgical area (e.g., a tracked retractor location). As another example, SIP 504 can be limited to a haptic object with a geometry based on the location of anatomical features, such as a shape corresponding to a surgical incision or other incision or opening through which the axis of surgical tool 234 extends during planned bone preparation. Thus, the control of the robotic device can be configured to confine TCP 502 to a first tactile object and SIP 504 to a second tactile object to guide TCP 502 according to the planned skeletal preparation, while confining SIP 504 to avoid undesirable behavior of the tool axis. Therefore, procedure 400 can be performed by the surgical system 200, thereby providing accurate skeletal modification in a reliable and intuitive manner.

[0075] Now see Figures 6 to 7 This illustrates a process 600 that facilitates the removal of a surgical tool from a tactile object according to an exemplary embodiment. Figure 6 A flowchart of process 600 is shown, while Figure 7It shows that it can be used with Figure 6 The tactile object used in process 600. For example, at step 312 of process 300, process 600 can be performed by surgical system 200. Although Figures 6 to 7 The example envisions a cylindrical tactile object, but it should be understood that process 600 can be applied to control objects with various geometries.

[0076] At step 602, the robotic device is controlled to constrain the surgical tool 234 with a cylindrical tactile object. Figure 7 An example of a cylindrical tactile object 700 centered on a target axis 702 is shown. In some embodiments, the cylindrical tactile object 700 corresponds to a surgical approach trajectory and / or planned skeletal modifications (e.g., planned guide holes 120). The cylindrical tactile object 700 may extend substantially beyond (away from) the sensitive anatomical area. When a surgical tool 234 is confined within the cylindrical tactile object 700, the surgical tool 234 may partially obstruct access to the surgical area. Therefore, the surgeon may wish to move the surgical tool out of the cylindrical tactile object 700 in a safe direction to facilitate various steps of the surgical procedure.

[0077] At step 604, the force exerted on the surgical tool 234 against the boundary of the cylindrical tactile object is detected. The robotic device 220 can detect this force, for example, as a wrench applied to a joint of the robotic arm 232. For instance, a HIP associated with the surgical tool 234 may be positioned at the boundary of the cylindrical tactile object 700, and the user applies force to the surgical tool 234, thereby pushing the tool against or into the boundary.

[0078] At step 606, it is determined (e.g., by processing circuitry 260) whether the force detected in step 604 is oriented in a predetermined withdrawal direction. The predetermined withdrawal direction may be selected as a safe and / or convenient direction that allows the surgical instrument 234 to withdraw from the tactile object. For example, the predetermined withdrawal direction is defined by the withdrawal region 704 of the cylindrical tactile object 700 and a wall 706 extending from the cylindrical tactile object 700 at the withdrawal region 704. In this example, processing circuitry 260 may determine that if the HIP of the surgical instrument 234 is at the withdrawal region 704, the force is oriented in the predetermined withdrawal direction because the force is applied at the boundary of the cylindrical tactile object 700. In some embodiments, the withdrawal region 704 only spans a portion of the length of the tactile object 700, for example, by... Figure 7The dead zone 708 shown is interrupted. As another example, in some implementations, processing circuitry 260 may determine a direction vector pointing in the direction the user is forcefully pushing the tool. In this case, it may be determined whether the direction vector is within a threshold angle of a predetermined exit direction. In this example, if the direction vector is within the threshold angle of the predetermined exit direction, the force is considered to be directed in the predetermined exit direction (i.e., "yes" at step 606). Thus, the resulting exit boundary may exhibit a funnel shape.

[0079] If the force is not directed in the predetermined exit direction, process 600 returns to step 602, and the surgical tool 234 is restrained with the cylindrical tactile object 700. That is, the robotic device 220 is controlled to provide force feedback, thereby restraining the surgical tool from the cylindrical tactile object 700, for example, in one or more steps that facilitate surgery.

[0080] If the force is oriented in a predetermined withdrawal direction (as determined at step 606), then at step 608 (e.g., by processing circuitry 260) it is determined whether the force exceeds a force threshold. In some embodiments, the amount of force applied to the surgical instrument 234 can be measured via the joints of the robotic arm 232. The user can indicate a desire to withdraw from the tactile object by exceeding the force threshold, while the force threshold can be set high enough to substantially prevent accidental or unintentional withdrawal from the tactile object.

[0081] If the force is less than a force threshold, the control robot device 220 restrains the surgical tool 234 from the tactile object (e.g., preventing transmission through the exit region 704 of the cylindrical tactile object 700). Process 600 returns to step 602, and the surgical tool 234 continues to be restrained by the cylindrical tactile object 700 to facilitate the use of the surgical tool 234 to perform surgical procedures.

[0082] If it is determined at step 608 that the force exceeds a force threshold, then at step 610 the surgical instrument is allowed to withdraw from the tactile object in a predetermined withdrawal direction. Figure 7 In the example, the constraints associated with exit region 704 are removed to allow surgical tool 234 to move through exit region 704 to exit the cylindrical tactile object 700. A wall 706 may be included as a tactile boundary to guide surgical tool 234 away from axis 702 in a predetermined direction. In other words, surgical tool 234 can be pushed across exit region 704 of cylindrical tactile object 700 to exit the cylindrical tactile object in a predetermined direction.

[0083] Therefore, the surgical instrument 234 is allowed to exit the tactile object, so that the robotic device is no longer controlled to constrain the surgical instrument 234 with the tactile object. In some cases, this can be achieved via the exit region 704 and / or by using any other tactile initiation procedure (e.g., in...). Figure 8 After the procedure, the surgical tool 234 is reinserted into the tactile object (i.e., the tactile constraint is restarted). In some cases, the tactile object is removed (deleted, etc.) when the surgical tool 234 is withdrawn. In some cases, the tactile object is adjusted or a new tactile object is enabled to facilitate subsequent steps of the surgery. Thus, the surgical procedure 600 can be performed once or multiple times to facilitate the surgery.

[0084] Now see Figure 8 A flowchart of process 800 for guiding a surgical tool to a virtual control object (e.g., a haptic object) according to an exemplary embodiment is shown. For example, at step 308 of process 300, process 800 may be performed by surgical system 200.

[0085] In step 802, a virtual control object is created. That is, a virtual control object is generated and its pose is defined. The virtual control object may include one or more of the following: a point object, a line object, a plane object, or a three-dimensional surface or volume, as referenced above. Figure 2 Detailed description. In some cases, the virtual control object is a tactile object. Process 800 is adaptable for use with a variety of virtual control objects having various shapes.

[0086] At step 804, the movement of the surgical tool 234 is tracked (e.g., via tracking system 222). For example, the position of a point (e.g., the tool center point) associated with the surgical tool 234 may be determined and updated over time. The position of the point may be defined relative to a virtual control object, i.e., in a coordinate system in which the posture of the virtual control object is also defined. The surgical tool 234 may be moved by the user's manipulation.

[0087] At step 806, the direction of movement of the surgical instrument is determined (e.g., by processing circuitry 260). For example, the positions of points associated with the surgical instrument 234 can be repeatedly collected over time to obtain a time series of position data. Given two positions (e.g., for subsequent time steps), a vector (e.g., a velocity vector) characterizing the direction of movement of the surgical instrument 234 can be defined. In some cases, the movement velocity (e.g., the magnitude of the velocity vector) is determined based on the distance between the positions used and the time elapsed between collecting these data points. In some cases, process 800 does not proceed to step 808 unless the magnitude of the velocity vector exceeds a threshold.

[0088] At step 808, it is determined (e.g., by processing circuitry 260) whether the direction of movement points towards the virtual control object. For example, the velocity vector determined at step 806 may extend (e.g., indefinitely) from the latest tracking position of the surgical instrument in the direction of movement. For example, if the extended velocity vector intersects the virtual control object, the direction of movement can be determined to point towards the virtual control object. If the extended velocity vector does not intersect the virtual control object, the direction of movement can be determined not to point towards the virtual control object. In various embodiments, various other statistical methods, coordinate transformations, etc., may be used to determine whether the direction of movement of the surgical instrument points towards the virtual control object. For example, in some embodiments, the target volume is defined at the virtual control object (e.g., around, adjacent to, or extending from it), and if the extended velocity vector intersects the target volume, the direction of movement can be determined to point towards the virtual control object. For example, in the case where the virtual control object is a line, the target volume may be defined as a cylinder centered on the line.

[0089] If the direction of movement does not point to the virtual control object, process 800 returns to step 804, where the movement of the surgical tool 234 is tracked. Steps 804 to 808 may be repeated until the direction of movement points to the virtual control object.

[0090] If the direction of movement is determined to be toward a virtual controlled object, the robotic device 220 is controlled at step 810 to provide a force that guides the surgical tool 234 toward the virtual controlled object. For example, a positive assist force may be provided to help the user move the surgical tool 234 toward the virtual controlled object. Without an external force supplied by the user, the positive assist force may not be sufficient to move the surgical tool 234 independently. In some cases, the force applied at step 810 causes the surgical tool to move automatically (without user intervention) toward the virtual controlled object. As another example, in some embodiments, a force is provided as a tactile boundary (e.g., a folding tactile boundary) to constrain the movement of the surgical tool 234 away from the virtual controlled object and / or deviate from the direction of movement toward the virtual controlled object.

[0091] Thus, the surgical system 200 can implement process 800 to assist the user in moving the surgical tool toward the virtual control object in response to the user initiating movement of the surgical tool toward the virtual control object. For example, various movements of the surgical tool away from the virtual control object may be desired before using it to correctly position the surgical tool 234, robotic arm 232, anatomical structures, other surgical instruments, etc. Process 800 provides a user-friendly and efficient workflow in which the surgical tool can move freely until it moves toward the virtual control object (e.g., toward the surgical area where the virtual control object is located), at which point the system 200 automatically begins guiding the surgical tool toward the virtual control object.

[0092] Now see Figures 9 to 10 This illustrates a process for providing tactile interaction including a damped zone according to an exemplary embodiment. Figure 9 A process 900 for providing tactile interaction including a damping zone is shown, while Figure 10 An example tactile object including a damping zone is shown. For example, at step 310 of process 300, process 900 can be performed by surgical system 200. Although Figure 10 A cylindrical tactile object is shown as an example, but it should be understood that... Figures 9 to 10 Its features make it suitable for use with virtual control objects of various geometries.

[0093] At step 902, a tactile object with a damping region is established (e.g., defined in virtual space by processing circuitry 260). The damping region may be defined as a sub-part of the tactile object and / or a region within the tactile object. Figure 10 An example tactile object 1000 with a damping zone 1002 is shown. For example... Figure 10 As shown, the tactile object 1000 is a cylindrical tactile object, and the damping region 1002 is a cylindrical disk positioned within the tactile object 1000. Figure 10 In the example, the tactile object 1000 and the damping region 1002 have equal diameters, while the height of the damping region 1002 is significantly smaller than the height of the tactile object 1000, and they are centered on a common axis 1004. In other embodiments, various relative sizes and dimensions are possible.

[0094] In some examples, the damping zone 1002 is positioned along the surface of the tactile object 1000 close to an anatomical feature (e.g., bone). For example, the damping zone 1002 may be on the outer side of the bone surface. In this case, when the tracked surgical tool 234 approaches the bone while being restrained by the tactile object 1000, the surgical tool 234 first reaches the damping zone 1002.

[0095] At step 904, it is determined (e.g., by processing circuitry 260 using data from tracking system 222) that surgical instrument 234 has entered the first side of damping zone 1002. Figure 10 In the example, the position of the TCP of the surgical tool 234 relative to the damping region 1002 can be tracked, and when the TCP intersects with the first surface 1006 of the damping region 1002, it can be determined that the surgical tool 234 has entered the first side of the damping region 1002.

[0096] At step 906, the robotic device 220 is controlled to provide tactile feedback, thereby partially resisting the movement of the surgical instrument through the damping zone. For example, when the surgical instrument 234 passes through the damping zone, control of the robotic device 220 based on the damping zone can cause the movement of the surgical instrument 234 to be slowed down (e.g., not exceeding a preset speed). Where the damping zone is located on the bone surface, the damping zone can therefore be used to manage (e.g., reduce) the translational speed of the surgical instrument 234 at the initial collision between the surgical instrument 234 and the bone. By reducing the translational speed of the surgical instrument 234, the damping zone, as provided by process 900, can thereby reduce scraping, improve the quality of the incision or hole / drill, and improve the accuracy of incision or hole placement.

[0097] Although Figures 9 to 10 The movement of the robotic device 220 is shown in a damped zone as described above; however, in other embodiments, other types of zones associated with other effects are provided, in which case step 906 is adjusted accordingly. For example, in some embodiments, the damped zone is replaced by an acceleration zone, in which, at step 906, the robotic device 220 causes the speed of the surgical instrument 234 to increase as the surgical instrument passes through the acceleration zone. As another example, in some embodiments, the damped zone is replaced by an attraction zone, in which the robotic device 220 is controlled to provide a force oriented toward a location (e.g., the midpoint of the attraction zone) on the surgical instrument 234, or by a repulsion zone, in which the robotic device 220 is controlled to provide a force oriented away from a location on the surgical instrument 234.

[0098] At step 908, it is determined (e.g., by processing circuitry 260 using data from tracking system 222) that the surgical instrument has exited the second side of the damping zone. Figure 10 In the example, the position of the TCP of the surgical tool 234 relative to the damping region 1002 can be tracked, and it can be determined that the surgical tool 234 has exited the second side of the damping region 1002 when the TCP passes through the second side 1008 of the damping region 1002.

[0099] At step 910, in response to determining that the surgical instrument has withdrawn from the second side of the damping zone, the damping zone is removed from the tactile object (e.g., via processing circuitry 260). The resistance feedback applied at step 906 is no longer applied. In some embodiments, the surgical instrument can be repeatedly passed through the area previously occupied by the damping zone without experiencing the resistance of step 906. Thus, the surgical system 200 can be configured to provide damping resistance to facilitate initial contact between the surgical instrument 234 and the bone, and to automatically remove this resistance after initial contact.

[0100] Now see Figures 11 to 13 A flowchart according to an exemplary embodiment is shown, illustrating a process for adjusting tactile interaction in response to input received from a user via buttons (keys, triggers, switches, pressure sensors, hand position sensors, etc.) mounted on surgical instruments or robotic arms. Reference Figure 2 , Figures 11 to 13 The process envisions a button positioned on the handle or grip area of ​​the robotic arm 232 or surgical instrument 234, so that the user (e.g., a surgeon) can easily use the button while manipulating the robotic arm 232 and surgical instrument 234 during the performance of the surgery. The user can select the button without having to change their grip on the robotic arm 232 and / or surgical instrument 234 and without having to take their eyes or attention away from the surgical area. Figures 11 to 13 The procedure can be performed by the surgical system 200, for example as... Figure 3 Part of step 310.

[0101] Figure 11 A flowchart of process 1100 is shown, in which a button allows the user to switch between a first tactile object and a second tactile object. At step 1102, the surgical tool 234 is constrained with the first tactile object. The first tactile object may have any of the sizes, shapes, etc., described herein. At step 1104, an electrical signal is received (e.g., at processing circuitry 260) indicating that a button mounted on the surgical tool or robotic arm has been pressed. At step 1106, tactile control of the surgical tool 234 is switched from the first tactile object to the second tactile object, such that at step 1108, the surgical tool 234 is constrained with the second tactile object. In some cases, the button can be pressed again to return to control based on the first tactile object.

[0102] Process 1100 can offer various advantages based on the relative size, shape, etc., of the first and second tactile objects. In some embodiments, the first tactile object is a sub-part of the second tactile object (i.e., such that the second tactile object allows a greater range of motion than the first tactile object). In this case, the user can select a button to allow the surgical instrument 234 to reach areas that would be inaccessible under control based on the first tactile object. One example is a set of planar tactile objects, where the first tactile object corresponds to a virtually defined range of a planar incision, while the second tactile object is a larger coplanar object. Based on the surgeon's experience and intraoperative observation, the surgeon can press a button intraoperatively to extend the incision if necessary. As another example, the first and second tactile objects may only partially overlap. In such an example, process 1100 can facilitate switching between different steps of the surgical procedure. In some embodiments, processing circuitry 260 prevents switching between tactile control objects unless the surgical instrument is currently located within both tactile objects.

[0103] Figure 12 Process 1200 is illustrated, wherein a button allows a user to cause a tactile object to extend in the direction of a force applied by the user. At step 1202, a surgical instrument is constrained with a first tactile object. The first tactile object may have any of the sizes, shapes, etc., described herein. At step 1204, an electrical signal is received (e.g., at processing circuitry 260) indicating that a button mounted on the surgical instrument or robotic arm has been pressed.

[0104] At step 1206, in response to a signal from the button, the direction of the force applied to the surgical instrument is determined (e.g., by processing circuitry 260). The HIP of the surgical instrument may be positioned at the boundary of the first tactile object such that the tactile control interaction prevents the force from causing further movement of the surgical instrument in the direction of the force. In this scenario, pressing the button indicates that the user wishes to move the surgical instrument further in the direction of the force. Therefore, at step 1208, the first tactile object extends in the direction of the force, thereby allowing the surgical instrument to move further in said direction before being constrained by the first tactile object. At step 1208, the first tactile object may extend to a preset distance or volume. The surgical instrument is then constrained by the extended first tactile object.

[0105] Thus, process 1200 can help the user extend the surgical instrument beyond the first tactile object in a specific, user-selected direction. For example, in some cases, control based on the first tactile object can constrain the surgical instrument to reach the entire range of anatomical features that the surgeon wishes to modify with the surgical instrument. The surgeon can then push the surgical instrument toward the target feature and select a button to cause the tactile object to extend toward the target feature. Thus, process 1200 facilitates advantageous intraoperative adjustments to the tactile object.

[0106] Figure 13 Process 1300 is illustrated, in which a button allows a user to adjust the virtual size of a virtual tool, thereby adjusting the virtual control interaction. In an embodiment envisioned by process 1300, tactile control interaction is achieved by tracking tactile interaction points (e.g., TCPs) associated with a surgical tool. The HIP is a one-dimensional point. Processing circuitry uses one or more virtual dimensions of the surgical tool to determine the volume occupied by the surgical tool based on the position of the HIP. For example, for a ball drill, the HIP may be located at the center of the ball drill, and the radius of the ball drill can be used to determine the 3-D volume occupied by the ball drill in virtual space based on the HIP's position and radius. Tactile control is provided based on the interaction between the volume occupied by the surgical tool (or its boundaries) and the tactile object. In the example of the ball drill, the HIP may be constrained to a position offset from the tactile boundary by at least the radius of the ball drill. In this case, changing the virtual radius of the ball drill allows the HIP to move closer to the tactile boundary. Therefore, control of the robotic device can be altered by modifying the virtual size of the surgical tool.

[0107] At step 1302, the surgical tool is constrained by a first tactile object. The first tactile object may have any of the sizes, shapes, etc. described herein. At step 1304, an electrical signal is received (e.g., at processing circuitry 260) indicating that a button mounted on the surgical tool or robotic arm has been pressed. At step 1306, in response to the signal from the button, the virtual size of the surgical tool is adjusted (e.g., by processing circuitry 260). For example, in a scenario where the surgical tool is a ball drill, the virtual radius of the ball drill may be reduced.

[0108] At step 1308, the robotic device controls the surgical tool to constrain it with a first tactile object based on the adjusted virtual size of the surgical tool. In an example where the virtual size of the surgical tool is reduced (e.g., in the case of a reduced radius), a greater range of motion can be provided for the surgical tool at step 1308 compared to step 1302. In this case, the surgeon can engage a button when they wish to allow the surgical tool to reach a position that was constrained and inaccessible at step 1302. The position of the virtual center point of the surgical tool can also be shifted relative to the tracked position of the surgical tool (e.g., at step 1306), for example, to align the reduced-size virtual tool with the boundary of the original-size virtual tool. This shift can provide a greater range of motion for the surgical tool in some directions while maintaining the range of motion along a common boundary. For example, shifting the virtual center point toward the distal tip of the surgical tool when reducing its virtual size allows for an increased range of lateral motion (i.e., orthogonal to the axis of the surgical tool) while confining the surgical tool to the original cutting depth. Thus, process 1300 can facilitate minor intraoperative adjustments to the range of anatomical features that can be modified by surgical instruments according to the surgical plan.

[0109] In some implementations, buttons can be repeatedly selected to cause repeated adjustments to one or more virtual sizes of the surgical instrument (e.g., gradually decreasing to smaller and smaller sizes, switching between two available sizes, sequentially traversing three or more available sizes, etc.). Many such possibilities are within the scope of this disclosure.

[0110] In other implementations, input is received from another source (e.g., foot pedal, voice switch, mouse, keyboard, touchscreen, etc.). In other implementations, user input is replaced by an automated response based on the tracked position or behavior of surgical tools and / or robotic devices. Figures 11 to 13 (Described as originating from a button). For example, in some embodiments, the dwell time of a surgical instrument at the boundary of a tactile object is detected. When the dwell time exceeds a threshold amount, the tactile control may be modified as described in steps 1106 to 1108, 1206 to 1210, and / or 1306 to 1308. Various such modifications are within the scope of this disclosure.

[0111] As mentioned above, Figures 4 to 13All combinations of the various features shown herein are within the scope of this disclosure. For example, process 300 can be implemented using steps of one or more of processes 400, 600, 800, 900, 1100, 1200, and 1300. Furthermore, it should be understood that the various features, method steps, etc., described herein are adaptable to facilitate a variety of surgical procedures, including total and partial hip, knee, and shoulder arthroplasty, as well as for performing non-surgical tasks.

[0112] The construction and arrangement of the systems and methods illustrated in the various exemplary embodiments are merely illustrative. While only a few embodiments are described in detail in this disclosure, many modifications are possible (e.g., variations in the size, dimensions, structure, shape and proportions of various elements, parameter values, use of materials, color, orientation, etc.). For example, the positions of elements may be reversed or otherwise varied, and the nature or number of discrete elements or positions may be altered or changed. Therefore, all such modifications are intended to be included within the scope of this disclosure. Depending on alternative embodiments, the order or sequence of any process or method steps may be changed or reordered. The systems described herein are applicable to implementing the methods described herein. Other substitutions, modifications, alterations, and omissions may be made to the design, operating conditions, and arrangement of the exemplary embodiments without departing from the scope of this disclosure.

[0113] As used herein, the terms “approximately,” “about,” “substantially,” and similar terms are intended to have a broad meaning consistent with common and accepted usage by one of ordinary skill in the art to which the subject matter of this disclosure relates. Those skilled in the art who review this disclosure will understand that these terms are intended to allow for the description of certain features described and claimed, without limiting the scope of those features to the precise numerical ranges provided. Therefore, these terms should be interpreted as indicating non-substantial or irrelevant modifications or alterations to the described subject matter and are considered within the scope of this disclosure.

Claims

1. A method comprising: controlling a robotic device to constrain a surgical tool with a virtual control object; detecting a force exerted on the surgical tool in a predetermined direction; determining whether the force in the predetermined direction exceeds a threshold force; and controlling the robotic device to allow the surgical tool to exit the virtual control object in response to determining that the force in the predetermined direction exceeds the threshold force.

2. The method of claim 1, wherein, the predetermined direction is defined by an exit region of the virtual control object.

3. The method of claim 2, wherein, the exit region spans only a portion of a length of the virtual control object.

4. The method of claim 1, wherein, detecting the force exerted on the surgical tool in the predetermined direction comprises determining whether a vector characterizing the force is within a threshold angle of the predetermined direction.

5. The method of claim 1, comprising: constraining the surgical tool from exiting the virtual control object by the robotic device in response to determining that the force in the predetermined direction is less than the threshold force.

6. The method of claim 1, comprising: enabling a new control object for a subsequent step of a surgical procedure in response to the surgical tool exiting the virtual control object.

7. The method of claim 1, further comprising: defining a damping zone along the virtual control object; determining that a first point associated with the surgical tool is in the damping zone; and controlling the robotic device to resist movement of the first point through the damping zone.

8. The method of claim 7, comprising: determining that the first point has passed through the damping zone; and removing the damping zone from the virtual object in response to determining that the first point has passed through the damping zone.

9. The method of claim 1, comprising: determining that the surgical tool is moving toward the virtual control object; and controlling the robotic device to provide an assistive force to the surgical tool in response to determining that the surgical tool is moving toward the virtual control object, the assistive force being oriented to facilitate movement of the surgical tool toward the virtual control object. adjusting a size of a virtual representation of the surgical tool in response to a request, wherein controlling the robotic device to constrain the surgical tool with the virtual control object is based on the size of the virtual representation of the surgical tool. adjusting a position of a haptic interaction point relative to the surgical tool in response to a request, wherein controlling the robotic device to constrain the surgical tool with the virtual control object is based on an interaction between the haptic interaction point and the virtual control object. adjusting the position of the haptic interaction point relative to the surgical tool comprises displacing the haptic interaction point toward a distal tip of the surgical tool.

10. The method of claim 1, comprising:

13. A surgical system comprising:

11. The method of claim 1, comprising: a robotic device; 12. The method of claim 11, wherein, a surgical tool configured to interface with the robotic device; and a controller programmed to: control the robotic device to constrain the surgical tool with a virtual control object; determine a force exerted on the surgical tool in a predetermined direction; determine whether the force in the predetermined direction exceeds a threshold force; and ​ ​ ​ ​ ​ The robotic device is controlled to allow the surgical tool to exit the virtual control object in response to a determination that the force in the predetermined direction exceeds the threshold force.

14. The surgical system of claim 13, wherein, The controller is programmed to determine that the force applied to the surgical tool is in the predetermined direction using an exit region of the virtual control object.

15. The surgical system of claim 14, wherein, The exit region spans only a portion of a length of the virtual control object.

16. The surgical system of claim 14, wherein, The controller is programmed to determine that the force applied to the surgical tool is in the predetermined direction by determining whether a vector representing the force is within a threshold angle of the predetermined direction.

17. The surgical system of claim 14, wherein, The controller is programmed to control the robotic device to constrain the surgical tool from exiting the virtual control object in response to a determination that the force in the predetermined direction is less than the threshold force.

18. The surgical system of claim 14, wherein, The controller is programmed to enable a new control object for a subsequent step of a surgical procedure in response to the surgical tool exiting the virtual control object.

19. The surgical system of claim 14, wherein, The surgical tool is configured to cut, deburr, grind, drill, partially resect, reshape, and / or otherwise modify bone.

20. The surgical system of claim 14, wherein, The controller is programmed to remove the virtual control object in response to the surgical tool exiting the virtual control object.

Images