System and method for aligning a robot end-effector with an axis

Abstract

A computer-assisted method for aligning an axis of an end-effector with a target axis defined relative to a bone including providing a heads-up-display configured to be viewable within the direct and / or peripheral vision of a user; displaying information on the heads-up-display indicating where the axis of the end-effector is located relative to a location of the target axis in one or more degrees-of-freedom; updating the information on the heads-up-display as the end-effector is moved relative to the target axis; and indicating when the location of the axis of the end-effector is aligned with the location of the target axis in the one or more degrees-of-freedom. Additionally, a computer-assisted surgical system includes a hand-held robot a heads-up-display configured to be worn on a head of a user; and a computing system having a processor and non-transient memory configured to carry out the computer-assisted method.

Description

Docket No. CURE-0232PCT-P0118-WOSYSTEM AND METHOD FOR ALIGNING A ROBOT END-EFFECTOR WITH AN AXISRELATED APPLICATIONS

[0001] This application claims priority benefit of U.S. Provisional Application Serial Number 63 / 689,400, filed August 30, 2024 and U.S. Provisional Application Serial Number 63 / 727,239, filed December 3, 2024, the contents of which are hereby incorporated by reference.TECHNICAL FIELD

[0002] The present invention generally relates to computer-assisted surgery, and more particularly to systems and methods that provide feedback to a user to assist the user in aligning an end-effector of a robot with a target axis.BACKGROUND

[0003] There are numerous medical and surgical procedures that require the alignment of a tool (e.g., drill bit, reamer, impactor, needle, bone pin, or laser) or an implant (e.g., a pedicle screw) with a target axis relative to patient anatomy. These illustratively include the alignment of a reamer for preparing the acetabulum in the context of total hip arthroplasty (THA) or the glenoid in shoulder arthroplasty; drilling holes in the glenoid or humorous for the alignment of shoulder implants in the context of shoulder arthroplasty; drilling a tunnel in anterior cruciate ligament (ACL) procedures; alignment of needles for biopsies; and stereotactic surgery often in the context of neurosurgery, spinal surgery, transplant and graft surgery. Manually aligning hand-held tools such that the tool axis aligns with a target axis has been met with limited success thus affecting the accuracy in revising the bone along said target axis. Such tools are described in U.S. Pat. Nos. 4,257,411; 4,739,751; and 7,972,341.Docket No. CURE-0232PCT-P0118-WO

[0004] Computer- assisted surgical (CAS) systems are regularly deployed to improve the process of aligning a tool axis (or implant axis) with a target axis. For example, conventional navigation CAS systems generally include a tracking system for tracking a tool where the system provides positional feedback (e.g., displaying the real-time position of the target axis relative to the real-time position of the tool axis on a display monitor) to assist the user in aligning the tool axis with the pre-defined location for the target axis. However, positional feedback alone may be distracting to the user because the user has to constantly direct their eyes at a display monitor and away from the surgical site. Other CAS systems may include a surgical robot that directs movement of an end-effector (e.g., a tool, instrument, or implant) to align the end-effector axis with the target axis. However, such robotic systems have a large footprint that consume a large area of the operating room or the surgical site. Smaller robotic systems have been developed, for example, the hand-held surgical robotic system as described in U.S. Pat. No. 11,457,980 assigned to the assignee of the present application. The hand-held robotic system includes a tracking system and a hand-held robotic device (referred to hereinafter as “hand-held robot”). The hand-held robot automatically compensates for surgeon hand movements (or bone movement) to maintain alignment of a bone pin axis coincident with a virtual plane having a pre-defined location relative to the bone for the precise placement of two or more bone pins in the bone coincident with the virtual plane. A guide (e.g., a cut guide, a cutting jig, a slotted guide) is then mounted to the bone pins inserted in the bone where the guide includes a guide slot for receiving a saw blade therethrough for forming a cut surface on the bone at the pre-planned location. However, this handheld robot is configured to operate in only 2 degrees-of-freedom (2-DoF), pitch and translation. In other words, the hand-held robot, due to its configuration (e.g., number of actuators), is limited to controlling movement of an end-effector in only 2 degrees-of-freedom, which is not enoughDocket No. CURE-0232PCT-P0118-WO degrees-of-freedom to align the end-effector axis with a target axis. A robot that operates in at least three or more degrees-of-freedom is required to align an end-effector axis with a target axis.

[0005] Nevertheless, the aforementioned 2-DoF hand-held robot has several advantages over conventional robotic systems including: it’s lightweight (e.g., less components are required to operate a robot in two degrees-of-freedom compared to a robot that operates in more degrees-of- freedom); it’s easy to handle; it has a simpler control scheme which makes it less prone to tool placement errors (e.g., the control system to operate a robot in 2-DoF is more straightforward to compared to a robot that operates in more degrees-of-freedom); it’s cheaper than conventional robotic systems; and it consumes much less space in the operating room and of the surgical site. However, as previously mentioned, the 2-DoF hand-held robot lacks the operational degrees-of- freedom required to align an end-effector axis with a target axis and is therefore not currently capable of performing surgical procedures which require such alignment.

[0006] Thus, there exists a need for a system and method that assists a user in aligning an endeffector axis with a target axis using a robotic system that operates in less than all the degree-of- freedom required to align an end-effector axis with a target axis.SUMMARY OF THE INVENTION

[0007] The present invention provides a computer-assisted surgical system and a computer- assisted method for aligning an axis of an end-effector with a target axis defined relative to a bone. The method includes providing a heads-up-display configured to be viewable within the direct and / or peripheral vision of a user; displaying information on the heads-up-display indicating where the axis of the end-effector is located relative to a location of the target axis in one or more degrees- of-freedom; updating the information on the heads-up-display as the end-effector is moved relativeDocket No. CURE-0232PCT-P0118-WO to the target axis; and indicating when the location of the axis of the end-effector is aligned with the location of the target axis in the one or more degrees-of-freedom. The computer-assisted surgical system includes a hand-held robot a heads-up-display configured to be worn on a head of a user; and a computing system having a processor and non-transient memory configured to carry out the computer-assisted method for aligning an axis of an end-effector with a target axis defined relative to a bone.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention is further detailed with respect to the following drawings that are intended to show certain aspects of the present of invention, but should not be construed as limit on the practice of the invention, wherein:

[0009] FIG. 1A and FIG. IB depict a shoulder joint in need of a should replacement;

[0010] FIG. 2 depicts a hand-held robot aligning an end-effector with a virtual plane having a pre-defined location relative to a desired location of a target axis;

[0011] FIG. 3A depicts a hand-held robot being manually moved by a user with the aid of visual feedback on a heads-up display (HUD) to align an end-effector of the robot with a target axis;

[0012] FIG. 3B depicts a zoomed-in view of the HUD of FIG. 3A;

[0013] FIG. 3C depicts the end-effector axis of FIG. 3A being in closer alignment with the target axis;

[0014] FIG. 3D depicts the end-effector axis of FIG. 3A in alignment with the target axis;

[0015] FIG. 4A depicts examples of visual alignment data for display on the HUD to assist the user in aligning an end-effector of a robot with a target axis;Docket No. CURE-0232PCT-P0118-WO

[0016] FIG. 4B depicts more examples of visual alignment data for display on the HUD to assist the user in aligning an end-effector of a robot with a target axis;

[0017] FIG. 5 depicts a HUD connected to a surgical system and how data may be communicated between various devices of the surgical system;

[0018] FIG. 6 depicts a computer-assisted surgical system for aligning an end-effector of a robot with a target axis; and

[0019] FIGs. 7 A and 7B depict a hand-held robot for controlling movement of an end-effector to maintain alignment of the end-effector coincident with a virtual plane, where FIG. 7A depicts the hand-held robot in a first working POSE, and FIG. 7B depicts the hand-held robot in a second working POSE.DETAILED DESCRIPTION

[0020] The present invention has utility as a system and method that provides feedback to a user to assist the user in aligning an end-effector of a robot with a target axis. In particular, the robot is configured to operate in less than all the degrees-of-freedom required to align the endeffector axis with the target axis, which may be advantageous because the robot, compared to conventional robotic systems, may be more user friendly, lightweight, have a simpler control scheme, cheaper, and consumes less space in the operating room and of the surgical site. The user is then guided to align the end-effector with the target axis in the remaining degrees-of-freedom using feedback, and more specifically, the visual feedback may be provided on a heads-up-display (HUD).

[0021] Definitions. As used in this description and the accompanying claims, the following terms shall have the meanings indicated, unless the context otherwise requires. Unless otherwiseDocket No. CURE-0232PCT-P0118-WO defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0022] All publications, patent applications, patents and other references mentioned herein are incorporated by reference in their entirety.

[0023] The following description provides examples related to shoulder replacement; however, it should be appreciated that the embodiments described herein are readily adapted for use in a myriad of applications where it is desirous to align a tool or an implant with a target axis for executing surgical procedures in other portions of the body. Examples include: drilling holes for anterior-cruciate ligament (ACL) replacement procedures; inserting pedicle screws in spinal applications; placing pins for trauma procedures; aligning needles for biopsies or targeted therapeutic treatment; and any other procedure where a tool, instrument, or implant needs to be aligned with a target axis.

[0024] It is to be understood that in instances where a range of values are provided that the range is intended to encompass not only the end point values of the range but also intermediate values of the range as explicitly being included within the range and varying by the last significant figure of the range. By way of example, a recited range from 1 to 4 is intended to include 1-2, 1- 3, 1-4, 2-3, 2-4, and 3-4.

[0025] Unless indicated otherwise, explicitly or by context, the following terms are used herein as set forth below.

[0026] As used herein, like reference numerals described in with respect to subsequent drawings have the meaning imparted thereto with respect to the previously detailed drawings.Docket No. CURE-0232PCT-P0118-WO

[0027] As used herein, the term “end-effector” refers to a tool, an instrument, a distal end of the robot (e.g., a coupler for coupling a tool to the robot), or an implant. The “end-effector” may be configured to couple to a robot, where the robot is directed to control movement of the endeffector in at least one degree-of-freedom. The “end-effector” may be configured to removably couple to the robot. Examples of “end-effectors” include: a pin (e.g., bone pin); a drill bit; a burr; an end-mill; a probe or pointer, a cutter; a saw-blade; a reamer; a broach; a laser; forceps or graspers; a screw (e.g., a pedicle screw, trauma screw); a nail; a coupler (e.g., chuck, collet, socket) for coupling a tool to the robot, or any other tool, instrument, coupler, or implant. The “endeffector” may be specifically designed to perform work on the environment or be implanted in the body.

[0028] As used herein, the term “bone data” refers to data related to one or more bones. The bone data may be determined: (i) prior to making modifications (e.g., bone cuts, insertion of a pin or screw, etc.) to one or more bones, referred to as pre-operative bone data; and / or (ii) determined after one or more modifications have been made to a bone, referred to as post-modification bone data. The bone data may include: the shapes of the one or more bones; the sizes of the one or more bones; angles and axes associated with the one or more bones (e.g., inclination or anteversion of the glenoid); angles and axes associated with two or more bones relative to one another (e.g., the combined inclination and anteversion of the shoulder joint); anatomical landmarks associated with the one or more bones (e.g., humeral head center, greater tuberosity, lesser tuberosity, intertubercular groove glenoid center,); bone density data; bone microarchitecture data; and stress / loading conditions of the bone(s). By way of example, the bone data may include one or more of the following: an image data set of one or more bones (e.g., an image data set acquired via fluoroscopy, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound,Docket No. CURE-0232PCT-P0118-WO other x-ray modalities, laser scan, etc.); three-dimensional (3-D) bone models, which may include a virtual generic 3-D model of the bone, a physical 3-D model of the bone, a virtual patient-specific 3-D model of the bone generated from an image data set of the bone; and a set of data collected directly on the bone intra-operatively commonly used with imageless CAS devices (e.g., laser scanning the bone, painting the bone with a digitizer). The term “bone model” may refer to a full representation of a bone (e.g., a whole femur bone) or a partial representation of a bone (e.g., only a distal region or fragment of a femur bone). Likewise, the term “bone” may refer to a whole bone, a portion of the bone (e.g., distal portion of the femur), or a fragment of a bone. The term “virtual” may also be referred to herein as “digital”, meaning the data is stored, generated, and / or processed by a computer.

[0029] As used herein, the terms “computer-assisted surgical device” and “CAS device” refer to devices used in surgical procedures that are at least in part assisted by one or more computers. Examples of CAS devices illustratively include tracked / navigated instruments and surgical robots. Examples of a surgical robot illustratively include robotic hand-held devices, serial-chain robots, bone mounted robots, parallel robots, or master-slave robots, as described in U.S. Patent Nos. 5,086,401; 6,757,582; 7,206,626; 8,876,830; 8,961,536; 9,707,043; and 11,457,980; which patents and patent application are incorporated herein by reference. The surgical robot may be active (e.g., automatic / autonomous control), semi-active (e.g., a combination of automatic and manual control), haptic (e.g., tactile, force, and / or auditory feedback), and / or provide power control (e.g., turning a robot or a part thereof on and off). It should be appreciated that the terms “robot” and “robotic” are used interchangeably herein. The terms “computer-assisted surgical system” and “CAS system” refer to a system comprising at least one CAS device and may further include additional computers, software, devices, or instruments. An example of a CAS system mayDocket No. CURE-0232PCT-P0118-WO include: i) a CAS device and software (e.g., cutting instructions, pre-operative bone data) used by the CAS device; ii) a CAS device and software (e.g., surgical planning software) used with a CAS device; iii) one or more CAS devices (e.g., a surgical robot); iv) a combination of i), ii), and iii); and iv) any of the aforementioned with additional devices or software (e.g., a tracking system, tracked / navigated instruments, tracking arrays, bone pins, rongeur, an oscillating saw, a rotary drill, manual cutting guides, manual cutting blocks, manual cutting jigs, etc.). In particular embodiments of the present invention, the CAS device is a surgical robot configured to operate in less than the degrees-of-freedom required to align an axis of an end-effector with a target axis, such as the two degree-of-freedom hand-held robot described herein.

[0030] Also referenced herein is a “surgical plan.” A surgical plan is generated using planning software. The surgical plan may be generated pre-operatively, intra-operatively, or pre-operatively and then modified intra-operatively. The planning software may be used to plan the location for an implant with respect to a bone and / or plan a location to make one or more modifications (e.g., bone cuts, location for inserting bone pins) to the bone. The planning software may include various software tools and widgets for planning the surgical procedure. This may include, for example, planning: (i) a location for implant data (e.g., a 3-D implant model) with respect to bone data (e.g., a 3-D bone model) to define a location for the implant with respect to the bone; (ii) a location for one or more bone cuts (e.g., a location for reaming the bone) to be made relative to bone data to define the locations for one or more cuts to be made on the bone, and / or (iii) one or more locations for inserting hardware (e.g., bone pins, screws) relative to bone data. All of which may be used to define locations for device operating data (e.g., a cut-file, a virtual plane, virtual boundary, a virtual axis) with respect to the bone data, where a CAS device is directed to control movement of an endeffector (e.g., the hardware, a burr, end-mill, drill bit) with respect to the bone according to theDocket No. CURE-0232PCT-P0118-WO device operating data. The device operating data may also be referred to as data used for operating a device. The device operating data may also, or alternatively, be used to provide feedback (e.g., display instructions or alignment data on a HUD or display monitor) to a user to assist the user in aligning an end-effector with a target axis as described herein

[0031] As used herein, the term “digitizer” refers to a device capable of measuring, collecting, recording, and / or designating the position of physical locations (e.g., points, lines, planes, boundaries, etc.) in three-dimensional space. By way of example but not limitation, a “digitizer” may be: a “mechanical digitizer” having passive links and joints, such as the high-resolution electro-mechanical sensor arm described in U.S. Patent No. 6,033,415 (which U.S. patent is hereby incorporated herein by reference); a non-mechanically tracked digitizer probe (e.g., optically tracked, electromagnetic ally tracked, acoustically tracked, and equivalents thereof) as described for example in U.S. Patent 7,043,961 (which U.S. patent is hereby incorporated herein by reference); an end-effector of a robotic device; or a laser scanner.

[0032] As used herein, the term “digitizing” refers to the collecting, measuring, designating, and / or recording of physical locations in space using a digitizer. In some embodiments, “digitizing” may refer to the conversion of a designated location, area, or volume in space to a digital format. For example, a tracking system may determine the location of a digitizer probe tip in contact with a point on the bone, where the determined location of that point is saved to computer memory.

[0033] As used herein, the term “registration” refers to: the determination of the spatial relationship between two or more objects; the determining of a coordinate transformation between two or more coordinate systems associated with those objects; the mapping of an object onto another object; and a combination thereof. Examples of objects routinely registered in an operatingDocket No. CURE-0232PCT-P0118-WO room (OR) illustratively include: CAS systems / devices; anatomy (e.g., bone); bone data (e.g., 3- D virtual bone models); a surgical plan (e.g., location of virtual planes defined relative to bone data, cutting instructions defined relative to bone data, or other device operating data defined relative to bone data); and any external landmarks (e.g., a tracking array affixed to a bone, an anatomical landmark, a designated point / feature on a bone, etc.) associated with the bone (if such landmarks exist). Methods of registration known in the art are described in U.S. Pat. Nos. 6,033,415; 8,010,177; 8,036,441; and 8,287,522; and 10,537,388. In particular embodiments with orthopedic procedures, the registration procedure relies on the manual collection of several points (i.e., point-to-point, point-to- surface) on the bone using a tracked digitizer where the surgeon is prompted to collect several points on the bone that are readily mapped to corresponding points or surfaces on a 3-D bone model. The points collected from the surface of a bone with the digitizer may be matched using iterative closest point (ICP) algorithms to generate a transformation matrix. This transformation matrix and various other transformation matrices provides the mathematical locational relationship between: (i) bone data (e.g., a 3-D bone model, planned location for forming one or more cut surfaces; planned location for an implant model relative to a bone model); and / or a surgical plan (e.g., a pre-defined location for a targeted virtual plane that was defined with respect to bone data, a pre-defined location for a targeted axis that was defined with respect to bone data, a pre-defined location of other device operating data that was defined with respect to bone data); and (ii) the coordinate system of a tracking array affixed to the bone (if present); and / or a CAS device (e.g., the base coordinate system of the CAS device, or a coordinate system of a tracking array affixed to the CAS device and, if needed, calibration data and / or kinematic data that define the location of an end-effector relative to the tracking array); and (iii) any other coordinate systemDocket No. CURE-0232PCT-P0118-WO or object required to perform the procedure. In other embodiments, the registration is performed using imageless registration.

[0034] As used herein, the term “display” is intended to encompass a variety of the digital devices that during operation provide an image (including multiple images in succession to form a video feed) recognizable to human viewing. Digital devices operative herein as displays illustratively include a graphical user interface (GUI), a computer or television (TV) monitor, a 2D or 3D holographic display, a mobile display, a smartphone display, a video wall, a headmounted display, a heads-up display (HUD), a virtual reality and / or augmented reality headset, a virtual reality and / or augmented reality glasses, a virtual reality and / or augmented reality Bluetooth compatible contact lens with a display, a virtual reality and / or augmented reality monocular’ display system, a broadcast reference monitor, any of the aforementioned with touchscreen capabilities, and a combination thereof. One or more computers comprising a processor may be operatively coupled to the display for controlling the output of the display. In particular embodiments of the present invention, the “display” is a heads-up-display (HUD) provided on glasses that are worn on a user’s head and positioned / oriented thereon for viewing by the user’s eyes. The heads-up-display may be viewable within the direct and / or peripheral vision of a user while the user is observing the target anatomy. The HUD has particular advantages in that the user can always focus on the surgical site and does not have to avert their eyes to other displays (e.g., a display monitor) located in the operating room. Various types of feedback including text and graphics can be displayed on the HUD to provide the user with detailed information, which is an improvement over other simpler forms of feedback which provide limited information (e.g., a simple light, like a light-emitting-diode, that turns on and off, changes color, or blinking frequency).Docket No. CURE-0232PCT-P0118-WO

[0035] As used herein, the term “feedback” may refer to visual feedback provided on a display. This “feedback” may also be provided in other forms, which may be in lieu of or in addition to visual feedback. For example, the “feedback” may include audio feedback, or haptic / tactile feedback (e.g., a buzz or vibration when a digitizer tip is located at in an area of max deviation).

[0036] Embodiments of the present invention describe a system and method that provides feedback to a user to assist the user in aligning an end-effector of a robot with a target axis. The system and method may include glasses or contacts that provide a heads-up display (HUD) giving the user real-time visual guidance for alignment of the end-effector with the target axis. The realtime visual guidance may be provided on the HUD in the form of virtual reality, augmented reality, graphics, text, holographies, as well as other forms of visual indicia. As the user manipulates the robot (e.g., the user wielding a hand-held robot), the HUD displays virtual alignment data, which may represent the current position and / or orientation (POSE) of the end-effector relative to the target axis. The virtual alignment data on the display may be overlaid on the natural environment as seen by the user’s eyes through the lens / display thereby giving the user a near unobstructed view of the surgical site, anatomy, or whatever the user is naturally viewing through the glasses. The virtual alignment data may include alignment markers, degree indicators, and / or color-coded regions to indicate ideal placement angles and / or positions for the end-effector relative to the target axis. The system and method allow the user to maintain direct visual contact with the surgical site while simultaneously receiving precise digital guidance for aligning the end-effector with the target axis, thereby improving the placement accuracy of the end-effector and reducing the risk of misalignment. The combination of the robotic precision in the robotically controlled degrees-of- freedom and the intuitive visual feedback from the HUD for alignment in the remaining degrees- of- freedom enhances the user’s ability to achieve precise and accurate alignment of the end-Docket No. CURE-0232PCT-P0118-WO effector axis with the target axis leading to better patient outcomes and reduced procedure times. According to embodiments, additional visual medical information can be provided in the display of the HUD, for example a visual feed from a medical scope or ultrasound procedure can be overlaid onto the display so that the user is able to reference this visual medical information while allowing the user to maintain direct visual contact with the surgical site.[00371 With reference now to the figures, FIGs. 1A and IB depict a shoulder joint comprising a humerus bone ‘H’, a scapula bone having a glenoid ‘G’ (or glenoid cavity), and a planned location for inserting a pin 10 in the glenoid for aligning one or more shoulder implants in the glenoid, ‘G’. The location for the pin 10 may be planned using planning software operatively coupled to a computing system. The planning software may include bone data, such as a three- dimensional (3-D) virtual model of the target bone (e.g., glenoid, humerus) generated from an image data set (e.g., CT scan) of the patient’s bone. The planning software may further include 2- D images or 3-D models of implants (e.g., CAD files) and various software tools to position the 3-D implant models relative to the bone data to determine the best fit, fill, and / or alignment for mounting an implant relative to the bone. The software tools may further be used to determine a location of a target axis relative to the bone for performing a procedure on the bone along said target axis. For example, in total shoulder arthroplasty, an image data set (e.g., CT scan data) of the patient’s scapula and humerus may be uploaded to the planning software. The planning software may include image segmentation tools to segment the scapula and the humerus in each of the images to generate a 3-D model of the scapula and a 3-D model of the humerus, or relevant portions thereof (e.g., the glenoid cavity and proximal humerus). A 3-D model of a socket implant may be positioned relative to the glenoid ‘G’ of the scapula model to define the location for mounting the socket in the glenoid. Likewise, a 3-D model of a stem and ball implant may beDocket No. CURE-0232PCT-P0118-WO positioned relative to the humerus bone model to define the location for mounting the stem and ball implant in the humerus. In some embodiments, the planning software may automatically position the implant models relative to the bone models, while in other embodiments, this may be performed by the user using the various software tools, or a combination of the two. In this way, the user and / or software can position the implants according to a desired anteversion and inclination angle that may produce the best clinical outcomes. A location for a target axis 12 may then be defined relative to the scapula model based on the defined location for the socket implant model relative to the scapula model. The location for the target axis 12 is the axis along which a pin 10 is inserted in the glenoid ‘G’ to assist in aligning and mounting the socket implant in the glenoid ‘G’ in the planned location (or pre-determined location). This target axis 12 may be defined as an axis extending through the center, or apex, of the socket implant model and perpendicular to a plane formed by the rim of the socket implant model. Another target axis may be defined for the inserting the stem in the humerus. It should be appreciated, that the above steps may also apply for reverse total shoulder arthroplasty, where one or more target axes are defined for drilling one or more holes in the glenoid to align and mount a glenoid sphere implant in the glenoid ‘G’ in a planned location (or pre-determined location).

[0038] FIG. 2 depicts the shoulder joint, a hand-held robot 14, and the location of a virtual plane 16 that is defined relative to the scapula model. The hand-held robot 14 is configured to automatically control movement of an end-effector 18 (e.g., bone pin, drill bit) coupled to the hand-held robot 14 to maintain alignment of the end-effector 18 coincident with a virtual plane 16 for performing work, or applying energy, on the bone (e.g., inserting bone pins in the bone, drilling holes in the bone). The hand-held robot 16 generally includes a working portion 20 movably coupled to a hand-held portion 22. The hand-held robot 14 is controlled by one or more computersDocket No. CURE-0232PCT-P0118-WO that provide control signals to two or more actuators (307a, 307b in FIG. 7A) housed in the handheld portion 22 to move the working portion 20 in two or more degrees-of-freedom (e.g., pitch and translation) relative to the hand-held portion 22. In specific embodiments, the hand-held robot 14 is advantageously designed to operate in no more than two degrees-of-freedom, where two degrees-of-freedom is the minimum number of degrees-of-freedom required to align an endeffector 18 coincident with a virtual plane 16 and affords the aforementioned advantages over robotic systems with more than two degrees-of-freedom (e.g., easy to handle, lightweight, simpler control scheme, etc.). However, a robot that operates in no more than two degrees-of-freedom is unable to move and align an end-effector with the target axis 12. Embodiments of the present invention advantageously provide a user with the ability to use said 2-DoF hand-held robot 14 and still align an end-effector 18 with the target axis 12 as further described below. A detailed description of the hand-held robot 14 is described below with reference to FIGs. 6, 7A, and 7B.

[0039] As previously stated, the hand-held robot 14 is configured to control movement of an end-effector 18 to maintain alignment of the end-effector 18 coincident with a virtual plane 16. The hand-held robot 14 may therefore control movement of the end-effector 18 to align the endeffector 18 with the target axis 12 in two of the three or more required degrees-of-freedom, which is at least half the battle. The location for the virtual plane 16 is pre-defined with respect to the bone using planning software. The location for the virtual plane 16 may be defined based on the planned location of the target axis 12 and at least one additional point to fully define a plane. The virtual plane 16 is preferably defined as being coincident with the planned location of the target axis 12 (e.g., all the points of the target axis 12 belong to the virtual plane 16 or are pail of the plane) and then using one more point that is defined relative the scapula model (e.g., a point on the rim of the glenoid). The one more point may be chosen by the user or by the computer andDocket No. CURE-0232PCT-P0118-WO defines the orientation of the virtual plane 16 about the target axis 12. Therefore, the one more point may be chosen strategically such that orientation of the virtual plane 16 provides the best trajectory for the end-effector 18 to access the one or more operative bones. The defined locations of the target axis 12 and the virtual plane 16 relative to the scapula model may then be saved as part of a surgical plan.[00401 In the operating room (OR), the scapula model is registered to the scapula in a coordinate system of a tracking array affixed to the scapula, or a base coordinate system of a surgical robot, using techniques known in the art (e.g., collecting points on the scapula with a digitizer and matching those points to corresponding points on the scapula model using an iterative closest point algorithm). The registration also allows the system to determine or locate the planned locations of the target axis 12 and the virtual plane 16 relative to the scapula in the coordinate system of the tracking array (or base coordinate system). After registration, a computing system can provide control signals to the actuators of the hand-held robot 14 to maintain alignment of the end-effector axis coincident with the virtual plane 16 based on: (i) the tracked location of the tracking array affixed to the scapula; (ii) the registered location of the virtual plane 16 to the scapula in the coordinate system of the tracking array; and (ii) the tracked location of the hand-held robot 14.

[0041] With reference now to Figs. 3A to 3D, particular embodiments of a system and method for providing feedback to a user to assist the user in aligning an end-effector 18 of a hand-held robot 14 with a target axis 12 is shown. The system generally includes the hand-held robot 14, and contact lenses or glasses 30 providing a heads-up-display (HUD) 32. The glasses 30 are configured to be worn on a user’s head and include lenses that act as displays to be viewed by the user’s eyes. The heads-up-display may be viewable within the direct and / or peripheral vision of a user while the user is observing the target anatomy. The glasses may include only one lens that acts as aDocket No. CURE-0232PCT-P0118-WO display, or both lenses may act as displays. In specific embodiments, the glasses are the Z100 smart glasses manufactured by Vuzix (Rochester, NY). The Z100 smart glasses are particularly suited for embodiments of the present invention due to their simplicity. Other smart glasses that incorporate augmented reality (AR) functionality add to the weight and cost of the glasses due to the additional hardware components required to run the AR software. The visual feedback provided to the user of the present invention does not require extensive AR functionality and is relatively straightforward to implement / program such that these additional hardware components are not required in the glasses. Thus the glasses 30 of the present invention are lightweight and easy-to-wear compared to more complex small glasses.

[0042] As shown in FIG. 3A, the HUD 32 provides visual feedback to the user to assist the user in aligning the axis of the end-effector 18 with the target axis 12 in the remaining degrees-of- freedom. The hand-held robot 14 automatically controls movement of the end-effector 18 to align the end-effector axis coincident with the virtual plane 16. While the hand-held robot 14 maintains this automatic alignment, the HUD 32 concurrently displays virtual alignment data to assist the user in manually (with the user’s hands) moving the hand-held robot 14 to align the end-effector axis 18 with the target axis 12. For example, when the hand-held robot 14 is brought near' the bone, the system may calculate the error between the current location of the end-effector axis 18 and the target axis 12 in one or more degrees-of-freedom (e.g., the error in yaw; error in yaw and translational error; the error in all 6 degrees-of-freedom; the error in all the degrees-of-freedom not automatically controlled by the robot). The HUD may then display the virtual alignment data to indicate the location of the end-effector axis relative to the target axis 12 in those one or more degrees of freedom. Examples of the virtual alignment data is shown in FIG. 3B, which is a zoomed-in view of the HUD 32. In particular embodiments, the virtual alignment data may includeDocket No. CURE-0232PCT-P0118-WO a first alignment indicator in the form of a number 34 indicating the degrees of angular error between the current location of the end-effector axis and the target axis 12. The hand-held robot 12 may control movement of the end-effector in pitch and translation, where this number 34 indicates the degrees of error in yaw, shown as -45°. The yaw rotational direction is shown by arrow 35. The user therefore knows to manually rotate the hand-held robot 14 in yaw to reduce this number 34 thereby reducing the error between the end-effector axis and the target axis 12. A + / - symbol may also be provided with this number 34 to indicate to the user which direction to manually rotate the hand-held robot 14. The virtual alignment data may also include alignment indicators in the form of graphics, such as the line 36, a target symbol 40, and current location symbol 38. The orientation of the line 36 is particularly useful because it indicates the degree-of- freedom that the user is manually controlling. In FIG. 3B, the line 36 is displayed horizontally because the user has to manually move the hand-held robot in the yaw rotational direction, which is horizontal, or perpendicular, to the degrees-of-freedom that are automatically controlled by the hand-held robot 14 (for clarification, the hand-held robot 14 in this example is automatically controlling movement in pitch and translation which are the vertical, or perpendicular DoFs, relative to the yaw rotational direction). The orientation of the line 36 may automatically change or be updated such that it is always perpendicular to the degrees-of-freedom that are automatically controlled by the hand-held robot 14. The target symbol 38 provides an indication of where the target axis 12 is located relative to the current location of the end-effector axis, where the current location of the end-effector axis is indicated by current location symbol 40. The symbols (38, 40) may be shown as, for example, an “X”, a “+”, a square, circle, triangle, or any other shape.

[0043] As the user manually moves (e.g., manually rotates in yaw) the hand-held robot 14, the virtual alignment data updates in real-time as can be seen by the subsequent time points shown inDocket No. CURE-0232PCT-P0118-WOFIGs. 3C and 3D. At the time point in FIG. 3C, the user has manually rotated the hand-held robot 13, as such, the number 34 indicating the rotational error has decreased to -20° and the current location symbol 40 has moved along line 36 and closer to the target symbol 38. The virtual alignment data may also be color coded, or capable of blinking at different frequencies, to further assist the user in manually moving the hand-held robot 14. For example, the current location symbol 40 may change from the color red to the color yellow signifying that the end-effector axis is getting closer in alignment with the target axis 12. FIG. 3D depicts a time point when the user has manually moved the hand-held robot 12 such that end-effector axis is now aligned with the target axis 12. At this point, when the end-effector axis is aligned with the target axis 12, a motor 305 (as shown in FIG. 7A) of the hand-held robot 14 may automatically turn on to activate (or drive) the end-effector 18 (e.g., the drill motor may turn on to rotate a bone pin for inserting the bone pin along the target axis 12). The HUD 32 may also display the current location symbol 40 overlapping with the target symbol 38. The current location symbol 40 may also now change colors from yellow to green. If at any point, the user manually moves the hand-held robot 14 such that the end-effector axis is no longer aligned with the target axis 12, the motor 305 may automatically turn off to keep the end-effector 18 from performing work (e.g., inserting a pin, drilling a hole) on the bone that strays away from the target axis 12. If the user does manually move the end-effector axis away from the target axis 12, the user can use the virtual alignment data to get back in alignment. After the work is performed on the bone, the user can complete the surgical procedure. For example, after a pin is inserted in the bone along the target axis 12, the user may align (using the inserted pin) and mount the socket implant in the glenoid of the scapula in a conventional manner.Docket No. CURE-0232PCT-P0118-WO

[0044] FIGs. 4A and 4B depict other examples of virtual alignment data that may be displayed on HUD 32 to assist the user in manually moving the hand-held robot 14 to align the end-effector axis with the target axis 12. Again, the hand-held robot 14 is automatically controlling movement of the end-effector 18 to maintain alignment of the end-effector axis with the virtual plane 16. The hand-held robot 14 is therefore automatically aligning the end-effector axis in two of the three or more required degrees-of-freedom to align the end-effector axis with the target axis 12. The HUD 32 and virtual alignment data assist the user in manually moving the hand-held robot 14 in the remaining degrees-of-freedom to align the end-effector axis with the target axis 12. The virtual alignment data is concurrently displayed as the hand-held robot is automatically controlling movement of the end-effector 18. FIG. 4A depicts more examples of virtual alignment data that may be displayed on the HUD 32. The HUD 32 may include a first DoF indicator 50, a second DoF indicator 52, and a target indicator 54. The first DoF indicator 50 may assist the user in manually moving the hand-held robot 14 in a first DoF to align the end-effector axis with the target axis 12. For example, the first DoF may assist the user in manually moving the hand-held robot 14 in the roll rotational direction to align the end-effector axis with the target axis, referred to hereinafter as a roll indicator 50. The roll indicator may include a curved line 56, which again represents the direction of movement that the user has to manually move the hand-held robot 14. The roll direction requires the user to manually rotate the hand-held robot 14 about the user’s wrist, thus when a user is viewing the curved line 56 in the HUD 32 it intuitively tells the user that they need to roll or rotate the hand-held robot around the curve (about their wrist) to align the endeffector axis with the target axis 12. The roll indicator 50 may further include a current location symbol 60 indicating the current location of the end-effector axis relative to the target axis 12, where the location of the target axis 12 is indicated by the target symbol 58. The second DoFDocket No. CURE-0232PCT-P0118-WO indicator 52 may assist the user in manually moving the hand-held robot 14 in a second DoF to align the end-effector axis with the target axis 12. For example, the second DoF may assist the user in manually moving the hand-held robot 14 in the pitch rotational direction to align the endeffector axis with the target axis, referred to hereinafter as the pitch indicator 52. In this embodiment, the hand-held robot 14 may automatically control movement of the end-effector in yaw and translation, where the user has to then manually move the hand-held robot in pitch to align the end-effector axis with the target axis 12. The pitch indicator 52 may include a vertical line 62, again representing the direction that the user has to manually move the hand-held robot 14 for the alignment. The pitch indicator 52 may also include a current location symbol 66 indicating the current location of the end-effector axis relative to the target axis 12, where the location of the target axis 12 is indicated by the target symbol 64. The target indicator 54 may include a current location symbol 70 indicating the current location of the end-effector axis relative to the target axis 12, where the location of the target axis 12 is indicated by the target symbol 68. The target indicator 54 shows generally where the current location of the end-effector axis is located relative to the target axis in two or more of the remaining degrees-of-freedom. In other words, the roll indicator 50 and pitch indicator 52 assist the user in manually moving the hand-held robot 14 in single degrees-of-freedom, while the target indicator 54 combines both degrees-of-freedom to provide an overall spatial view of the relative locations between the end-effector axis and the target axis.

[0045] FIG. 4B depicts similar virtual alignment data as FIG. 4A with the following differences. First, the pitch indicator 52 is now a yaw indicator 72 to assist the user in manually moving the hand-held robot in the yaw rotational direction. In this embodiment, the hand-held robot 14 may automatically control movement of the end-effector 18 in pitch and translation, and the user has to manually move the hand-held robot 14 in yaw. The yaw indicator 72 includes a horizontal line 74,Docket No. CURE-0232PCT-P0118-WO which again represents the direction that the user has to manually move the hand-held robot 14 relative to the automatic movements of the end-effector 18 caused by the hand-held robot (e.g., the hand-held robot 14 automatically moves the end-effector 18 in pitch and vertical translation, which are in the vertical direction). The yaw indicator may also include a current location symbol 76 indicating the current location of the end-effector axis relative to the target axis 12, where the location of the target axis 12 is indicated by the target symbol 78. The other difference is the shape of the current location symbol 80 and the target symbol 82 in the target indicator. The current location symbol 80 and the target symbol 82 are in the shape of a rectangular, which may represent the shape of the end-effector 18. For example, the end-effector 18 may be a saw blade having a rectangular profile at its distal end. When making planar cuts on a bone with a saw-blade, it’ s important that the roll of the saw blade being aligned with the desired cut plane. Therefore, the rectangular shapes may assist the user in manually moving the hand-held robot in the roll rotational direction to align or match the current location rectangle 80 with the target rectangle 82.

[0046] FIG. 5 depicts the HUD 32 connected to a surgical system and how data may be communicated between the various devices. The system may include the hand-held robot 14, a tracking system 206, a computer 208, the glasses 30, and an optional mobile smart phone 84. The contact lenses or glasses 30 may be equipped with wireless communication capabilities such as Bluetooth, WiFi, radiofrequency, or optical communication (e.g., modulated infrared light) to communicate with one or more external devices such as computer 208 or mobile phone 84. Wired communication is also contemplated. The glasses 30 may further include hardware (e.g., processors, controllers, memory), software, data, and / or utilities for operating the HUD 32 and displaying the virtual alignment data based on data received by one or more external devices (e.g., tracking system 206, computer 208, and / or mobile phone 84). The HUD 30 may be controlled inDocket No. CURE-0232PCT-P0118-WO the following manner. The tracking system 206 may determine the location of the hand-held robot 14 (and end-effector 18) in space as well as the location of the operative bone (as shown by arrow 86). This tracking data is sent to the computer 208 for processing (as shown by arrow 88). The computer 208 may determine the relative location of the end-effector axis relative to the location of the target axis 12 and calculate the error therebetween. The computer 208 may include HUD hardware and software for generating the virtual alignment data to be displayed on the HUD 32 based on the tracking data and the calculated errors. The computer 208 may then directly communicate with the glasses 30 via Bluetooth, for example, to upload the virtual alignment data on the glasses 30 for display on the HUD 32 (as shown by arrow 94). In other embodiments, the HUD hardware and software for generating the virtual alignment data for display on the HUD 32 is located on a mobile phone 84. In particular, the mobile phone may have an operating system capable of running an application that is already programmed for use with the contact lenses or glasses 30. In which case, the computer 208 may process the tracking data and send the relevant information (e.g., the errors between the end-effector axis and the target axis 12) to the mobile phone 84 (as shown by arrow 90). The mobile phone 84 then generates the virtual alignment data for display on the HUD 32 and communicates this data to the contact lenses or glasses 30 via Bluetooth, for example (as shown by arrow 92).Computer- Assisted Surgical System

[0047] Referring now to FIGs. 6, 7A, and 7B, embodiments of the present inventive system and method generally includes a computer-assisted surgical system. In some inventive embodiments, a 2-DoF hand-held robot 14 is provided for maintaining alignment of an end-effector 18 (e.g., bone pin) coincident with a virtual plane. FIG. 6 is a schematic view showing the computer-assistedDocket No. CURE-0232PCT-P0118-WO surgical system 200 including a 2-DoF hand-held robot 14, a computing system 204, glasses 30, and a tracking system 206. In other inventive embodiments, the system includes an end effector 18 extending from a robotic arm.

[0048] The computing system 204 generally includes hardware and software for executing a surgical procedure. By way of example but not limitation, in one preferred form of the present invention, the computing system 204 is configured to control the actuation of the working portion 20 relative to the hand-held portion 22 of the 2-DoF hand-held robot 14 device to maintain alignment of the end-effector axis 18 (FIG. 7B) coincident with a virtual plane, where the desired location of the virtual plane may be defined in a surgical plan. The end-effector 18 coupled to the working portion 20 in operation performs work (e.g., inserts pins, cuts, mills, etc.) on the subject bone. The computing system 204 may generate control signals to accurately maintain the endeffector axis 18 coincident with a virtual plane defined in the surgical plan based on: a) the location of the virtual plane as registered to the location of the bone (or more specifically to the coordinate system of a tracking array affixed to the bone); and b) the tracked POSE of the 2-DoF hand-held robot 14.

[0049] The computing system 204 of the computer-assisted surgical system 200 may include: one or more device computers (208, 209); a planning computer 210; a tracking computer 211; a glasses computer (not shown); and peripheral devices. Each computer may include one or more processors. Processors operate in the computing system 204 to perform computations and execute software associated with the inventive system and method. The device computer(s) (208, 209), the planning computer 210, and the tracking computer 211 may be separate entities as shown in FIG. 6, or it is also contemplated that operations may be executed on one (or more) computers depending on the configuration of the computer-assisted surgical system 200. For example, theDocket No. CURE-0232PCT-P0118-WO tracking computer 211 may have operational data to control the 2-DoF hand-held robot 14 without the need for a device computer (208, 209). Furthermore, if desired, any combination of the device computers (208, 209), planning computer 210, and / or tracking computer 211 may be connected together via a wired or wireless connection. It is further appreciated that one or more of the computers may be readily located remote from the surgical site. Cloud-based computation is also contemplated in the present invention. In addition, the data gathered by, and / or the operations performed by, the tracking computer 211 and device computer(s) (208, 209) may work together to control the 2-DoF hand-held robot 12 and, as such, the data gathered by, and / or the operations performed by, the tracking computer 211 and device computer(s) (208, 209) to control the 2-DoF hand-held robot 14 may be referred to herein as a “control system.”

[0050] The peripheral devices allow a user to interface with the computing system 204 and may include, but are not limited to, one or more of the following: one or more user-interfaces, such as a display or monitor (212a, 212b) to display a graphical user interface (GUI); and user-input mechanisms, such as a keyboard 214, mouse 222, pendent 224, joystick 226, and foot pedal 228. If desired, the monitor(s) (212a, 212b) may have touchscreen capabilities, and / or the 2-DoF handheld robot 14 may include one or more input mechanisms (e.g., buttons, switches, etc.). The glasses 30 may also include one or more input mechanisms, such as a button or microphone, for receiving input from a user. The glasses 30 may also include one or more output mechanisms, such as a light, a speaker, or haptic feedback device. Another peripheral device may include a tracked digitizer probe 230 to assist in the registration process. A tracking array 220c is assembled to the digitizer probe 230 to permit the tracking system 206 to track the POSE of the digitizer probe 230 in space. The digitizer probe 230 may further include one or more user input mechanisms to provide input to the computing system 204. For example, a button on the digitizer probe 230 mayDocket No. CURE-0232PCT-P0118-WO allow the user to signal to the computing system 204 to digitize a point in space to assist in registering a bone to a surgical plan.

[0051] The device computer(s) (208, 209) may include one or more processors, controllers, software, data, utilities, and / or storage medium(s) such as RAM, ROM or other non-volatile or volatile memory to perform functions related to the operation of the 2-DoF hand-held robot 14. By way of example but not limitation, one or more of the device computers (208, 209) may include software to control the 2-DoF device 202, e.g., generate control signals for the actuators to move the working portion 20 relative to the hand-held portion 22 to a targeted POSE, receive and process tracking data, control the rotational or oscillating speed of the end-effector 18 by controlling motor 305 (FIG. 7A), execute registration algorithms, execute calibration routines, provide workflow instructions to the user throughout a medical procedure, as well as any other suitable software, data or utilities required to successfully perform the procedure in accordance with embodiments of the invention.

[0052] In some embodiments, the system 200 may include a first device computer 208 located separate from the 2-DoF hand-held robot 14 and a second device computer 209 housed in the 2- DoF hand-held robot 14 to provide on-board control. The first device computer 208 may be dedicated to the control of the surgical workflow via a GUI, the registration process and the associated calculations, the display of 3-D models and 3-D model manipulation or animation, processing tracking data from the tracking system, generating virtual alignment data for display on the HUD 32, as well as other processes. The second device computer 209, also referred to herein as an on-board device computer, may be dedicated to the control of the 2-DoF hand-held robot 14. For example, the on-board device computer 209 may compute and generate the control signals for the actuator motors (210a, 210b) based on: i) received signals / data corresponding toDocket No. CURE-0232PCT-P0118-WO the real-time POSE of the 2-DoF hand-held robot 14 from the tracking system; and ii) received signals / data corresponding to the real-time POSE of the virtual plane computed by first device computer 208. The on-board device computer 209 may also send internal data (e.g., operational data, actuator / screw position data, battery life, etc.) via a wired or wireless connection. In some inventive embodiments, wireless optical communication is used to send and receive the signals / data described herein. Details about bi-directional optical communication between a 2-DoF hand-held robot 14 and a tracking system 206 are further described below.

[0053] The planning computer 210 in some inventive embodiments is dedicated to planning the procedure. By way of example but not limitation, the planning computer 210 may contain hardware (e.g., processors, controllers, memory, etc.), planning software, data, and / or utilities capable of: receiving, reading, and / or manipulating medical imaging data; segmenting imaging data; constructing and manipulating three-dimensional (3D) virtual bone models; storing and providing computer-aided design (CAD) files such as implant CAD files; planning the POSE for cut surfaces, virtual planes, screws, pins, implants, grafts, and fixation hardware relative to preoperative bone data; generating the surgical planning data for use with the system 200, and providing other various functions to aid a user in planning the surgical procedure. The planning software is particularly programmed for planning procedures that require the alignment of an endeffector 18 with a target axis (e.g., shoulder replacement procedures, spine procedures). The final surgical plan data may include one or more images of a bone or virtual models of the bone, registration data, subject identification information, the POSE for inserting or mounting one or more pins, screws, implants, grafts, fixation hardware relative to the bone, and / or the POSE of one or more virtual planes defined relative to the bone. The device computer(s) (208, 209) and the planning computer 210 may be directly connected in the operating room, or the planning computerDocket No. CURE-0232PCT-P0118-WO210 may exist as separate entities outside the operating room. The final surgical plan is readily transferred to a device computer (208, 209) and / or tracking computer 211 through a wired (e.g., electrical connection) or a wireless connection (e.g., optical communication) in the operating room; or transferred via a non-transient data storage medium (e.g., a compact disc (CD), or a portable universal serial bus (USB drive)) if the planning computer 210 is located outside the operating room (or if otherwise desired). As described above, the computing system 204 may comprise one or more computers, with multiple processors capable of performing the functions of the device computer 208, the tracking computer 211, the planning computer 210, the glasses computer (not shown), or any combination thereof.

[0054] The tracking system 206 of the present invention generally includes a detection device to determine the POSE of an object relative to the position of the detection device. In particular embodiments of the present invention, the tracking system 206 is an optical tracking system such as the optical tracking system described in U.S. Pat. No. 6,061,644 (which patent is hereby incorporated herein by reference), having two or more optical detectors 207 (e.g., cameras) for detecting the position of fiducial markers arranged on rigid bodies or integrated directly on the tracked object. By way of example but not limitation, the fiducial markers may include an active transmitter, such as an LED or electromagnetic radiation emitter; a passive reflector, such as a plastic sphere with a retro-reflective film; or a distinct pattern or sequence of shapes, lines or other characters. A set of fiducial markers arranged on a rigid body, or integrated on a device, is sometimes referred to herein as a tracking array (220a, 220b, 220c, 312), where each tracking array has a unique geometry / arrangement of fiducial markers, or a unique transmitting wavelength / frequency (if the markers are active LEDS), such that the tracking system 206 can distinguish between each of the tracked objects.Docket No. CURE-0232PCT-P0118-WO

[0055] If desired, the tracking system 206 may be incorporated into an operating room light 218, located on a boom, a stand, or built into the walls or ceilings of the operating room. The tracking system computer 211 includes tracking hardware, software, data, and / or utilities to determine the POSE of objects (e.g., tissue structures, the 2-DoF hand-held robot 14) in a local or global coordinate frame. The output from the tracking system 206 (i.e., the POSE of the objects in 3-D space) is referred to herein as tracking data, where this tracking data may be readily communicated to the device computer(s) (208, 209) through a wired or wireless connection. In a particular embodiment, the tracking computer 206 processes the tracking data and provides control signals directly to the 2-DoF hand-held robot 14 and / or device computer 208 based on the processed tracking data to control the position of the working portion 20 of the 2-DoF hand-held robot 14 relative to the hand-held portion 22. In another embodiment, the tracking computer 206 sends tracking data to a receiver located on the 2-DoF hand-held robot 14, where an on-board device computer 209 generates control signals based on the received tracking data.

[0056] The tracking data is determined in some inventive embodiments using the position of the fiducial markers detected from the optical detectors and operations / processes such as image processing, image filtering, triangulation algorithms, geometric relationship processing, registration algorithms, calibration algorithms, and coordinate transformation processing. These operations / processes may be executed directly on the tracking system computer 211 or executed on a separate computer (e.g., first device computer 208) in communication with the tracking system 206.

[0057] Bi-directional optical communication (e.g., light fidelity or Li-Fi) may occur between the 2-DoF hand-held robot 14 and the tracking system 206 by way of a modulated light source (e.g., light emitting diode (LED)) and a photosensor (e.g., photodiode, camera). The 2-DoF hand-Docket No. CURE-0232PCT-P0118-WO held robot 14 may include an LED and a photosensor (i.e., a receiver) disposed on the working portion 20 or hand-held portion 22, where the LED and photosensor are in communication with a processor such as modem or an on-board device computer. Data generated internally by the 2-DoF hand-held robot 14 may be sent to the tracking system 206 by modulating the LED, where the light signals (e.g., infrared, visible light) created by the modulation of the LED are detected by the tracking system optical detectors (e.g., cameras) or a dedicated photosensor and processed by the tracking system computer 211. The tracking system 206 may likewise send data to the 2-DoF hand-held robot 14 with a modulated LED associated with the tracking system 206. Data generated by the tracking system 206 may be sent to the 2-DoF device 202 by modulating the LED on the tracking system 206, where the light signals are detected by the photosensor on the 2-DoF handheld robot 14 and processed by a processor in the 2-DoF hand-held robot 14. Examples of data sent from the tracking system 206 to the 2-DoF hand-held robot 14 includes operational data, surgical planning data, informational data, control data, positional or tracking data, pre-operative bone data, or instructional data. Examples of data sent from the 2-DoF hand-held robot 14 to the tracking system 206 may include motor position data, battery life, operating status, logged data, operating parameters, warnings, or faults. In some embodiments, data generated by the first device computer 208 is sent to the tracking system 206, where that generated data is transferred to the 2- DoF hand-held robot 14 via the LED on the tracking system 206.

[0058] It should be appreciated that in some embodiments of the present invention, other tracking systems are incorporated with the surgical system 200. By way of example but not limitation, the surgical system 200 may comprise an electromagnetic field tracking system, ultrasound tracking systems, accelerometers and gyroscopes, and / or a mechanical tracking system. The replacement of a non-mechanical tracking system with other tracking systems will be apparentDocket No. CURE-0232PCT-P0118-WO to one skilled in the art in view of the present disclosure. In one form of the present invention, the use of a mechanical tracking system may be advantageous depending on the type of surgical system used such as the computer-assisted surgical system described in U.S. Pat. No. 6,322,567 assigned to the assignee of the present application and incorporated herein by reference in its entirety.[00591 FIGS. 7A and 7B are schematic views showing the 2-DoF hand-held robot 14 in greater detail. More particularly, FIG. 7A shows the 2-DoF hand-held robot 14 in a first working POSE, and FIG. 7B illustrates the 2-DoF hand-held robot 14 in a second working POSE. The 2-DoF hand-held robot 14 comprises a hand-held portion 22 (or handle) and a working portion 20. The hand-held portion 22 comprises an outer casing 303 of ergonomic design which can be held and wielded by a user (e.g., a surgeon). In particular’ embodiments, the 2-DoF hand-held robot 14 is intended to be fully supported by the hands of the user in that there are no additional supporting links connected to the 2-DoF hand-held robot 14 and the user supports the full weight of the 2- DoF hand-held robot 14. The working portion 20 comprises an end-effector 18 having an endeffector axis 307. The end-effector 18 may be removably coupled to the working portion 20 (via a coupler (e.g., chuck)) and driven by a motor 305. The hand-held portion 22 and working portion 20 are connected to one another, for example, by a first linear actuator 307a and a second linear actuator 307b in order to control the pitch and translation of the working portion 20 relative to the hand-held portion 22, as will hereinafter be discussed in further detail. In a particular embodiment, the working portion 20 is removably coupled to the hand-held portion 22 to permit different types of working portions to be assembled to the hand-held portion 22. For example, a first working portion 20 may illustratively be a laser system having components to operate a laser for treatingDocket No. CURE-0232PCT-P0118-WO tissue, a second working portion 20 may illustratively be a drill for rotating a bone pin, and a third working portion 20 may illustratively be an oscillating saw.

[0060] A tracking array 312, having three or more fiducial markers of the sort well known in the ait, is preferably rigidly attached to the working portion 20 in order to permit the tracking system 206 (FIG. 13) to track the POSE of the working portion 20. The three or more fiducial markers may, alternatively, be integrated directly with the working portion 20. The fiducial markers may be active markers such as light emitting diodes (LEDs), or passive markers such as retroreflective spheres. The 2-DoF hand-held robot 14 may further include one or more user input mechanisms such as triggers (e.g., trigger 314) or button(s). The user input mechanisms may permit the user to perform various functions illustratively including: activating or deactivating the motor 305; activating or deactivating the actuation of the working portion 20 relative to the handheld portion 22; notifying the computing system 204 to change from targeting one virtual plane to a subsequent virtual plane; and pausing the surgical procedure.

[0061] Within the outer casing of the hand-held portion 22 is a linear actuator 307a and a second linear actuator 307b. Each linear actuator (307a, 307b) may include a motor (310a, 310b) to power a screw (316a, 316b) (e.g., a lead screw, a ball screw), a nut (318a, 318b), and a linear rail (308a, 308b). In some inventive embodiments, the motors (first motor 310a, second motor 310b) are electric servo-motors that bi-directionally rotate the screws (316a, 316b). Motors (310a, 310b) may also be referred to herein as linear actuator motors. The nuts (318a, 318b) (e.g., ball nuts, elongated nuts) are operatively coupled to the screws (316a, 316b) to translate along the screws (316a, 316b) as each screw is rotated by its respective motor (310a, 310b). A first end of each linear rail (308a, 308b) is coupled to a corresponding nut (316a, 316b) and the opposing end of each linear rail (308a, 308b) is coupled to the working portion 20 via hinges or links (320a, 320b) such that theDocket No. CURE-0232PCT-P0118-WO hinges / links (320a, 320b) allow the working portion 20 to pivot relative to the linear rails (308a, 308b). The motors (310a, 310b) power the screws (316a, 316b) which in turn cause the nuts (318a, 318b) to translate along the axis of the screws (316a, 316b). Translation of nuts 318a, 318b along ball screws 316a, 316b, respectively, causes translation of front lineal’ rail 308a and back linear rail 308b, respectively, whereby to permit (a) selective linear movement of working portion 20 relative to hand-held portion 22, and (b) selective pivoting of working portion 304 relative to handheld portion 302 of 2-DoF hand-held robot 14. Accordingly, the translation “d” and pitch “a” (FIG. 7B) of the working portion 20 may be adjusted depending on the position of each nut (318a, 318b) on their corresponding screw (316a, 316b). A linear guide 322 (FIG. 7A) may further constrain and guide the motion of the linear rails (308a, 308b) in the translational direction “d”. In a particular embodiment, the nuts (316a, 316b) are elongated and couple directly to the working portion 20 via the hinges (320a, 320b), in which case the linear rails (308a, 308b) are no longer a component of the linear actuators (307a, 307b). It should be appreciated that other linear actuation mechanisms / components may be used to adjust the POSE of the working portion 20 relative to the hand-held portion 22 such as linear motors, pneumatic motors, worm drives and gears, rack and pinion gear’s, and other arrangements of motors and transmissions.

[0062] The 2-DoF hand-held robot 14 may receive power via an input / output port (e.g., from an external power source) and / or from on-board batteries (not shown).

[0063] The motors (305, 310a, 310b) of the 2-DoF hand-held robot 14 may be controlled using a variety of methods. By way of example but not limitation, according to one method of the present invention, control signals may be provided via an electrical connection to an input / output port. By way of further example but not limitation, according to another method of the present invention, control signals are communicated to the 2-DoF hand-held robot 14 via a wireless connection,Docket No. CURE-0232PCT-P0118-WO thereby eliminating the need for electrical wiring. The wireless connection may be made via optical communication. In certain inventive embodiments, the 2-DoF hand-held robot 14 includes a receiver for receiving control signals from the computing system 204 (FIG. 6). The receiver may be, for example, an input port for a wired connection (e.g., Ethernet port, serial port), a transmitter, a modem, a wireless receiver (e.g., Wi-Fi receiver, Bluetooth® receiver, a radiofrequency receiver, an optical receiver (e.g., photosensor, photodiode, camera)), or a combination thereof. The receiver may send control signals from the computing system 204 directly to the motors (305, 310a, 310b) of the 2-DoF hand-held robot 14, or the receiver may be in communication with a computer (e.g., an on-board device computer 209) that processes signals received by the receiver and then generates the control signals for the motors (305, 310a, 310b) based on the received signals.Other Embodiments

[0064] While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration of the described embodiments in any way. Rather, the foregoing detailed description will provide those skilled in the ait with a convenient roadmap for implementing the exemplary embodiment or exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the scope as set forth in the appended claims and the legal equivalents thereof.

Claims

Docket No. CURE-0232PCT-P0118-WOCLAIMS1. A computer-assisted method for aligning an axis of an end-effector with a target axis defined relative to a bone, said method comprising: providing a heads-up-display configured to be viewable within the direct and / or peripheral vision of a user; displaying information on the heads-up-display indicating where the axis of the endeffector is located relative to a location of the target axis in one or more degrees-of-freedom; updating the information on the heads-up-display as the end-effector is moved relative to the target axis; and indicating when the location of the axis of the end-effector is aligned with the location of the target axis in the one or more degrees-of-freedom.

2. The method of claim 1 wherein a hand-held robot automatically controls movement of the end-effector in at least one degree-of-freedom relative to the target axis.

3. The method of claim 2 wherein the hand-held robot automatically controls movement of the end-effector to automatically align the end-effector with a plane.

4. The method of claim 1 wherein the displayed information comprises: an endeffector position indicator; a first degree-of-freedom movement indicator; a second degree-of- freedom movement indicator; a target indicator; or a combination thereof.Docket No. CURE-0232PCT-P0118-WO5. The method of claim 1 wherein the hand-held robot is manually moved by a user while the hand-held robot automatically controls movement of the end-effector and the information updates in real-time with the movement.

6. The method of claim 1 wherein the displayed information further comprises at least a first translational error indicator indicating the error between the current location of the endeffector and the location of the target axis in a translational degree-of-freedom.

7. The method of claim 1 wherein the displayed information further comprises a second rotational error indicator indicating the degrees of angular error between the current location of the end-effector and the location of the target axis in a rotational degree-of-freedom.

8. The method of claim 7 wherein the degrees of angular error represent errors in one or more orthogonal rotational degrees of freedom in any definable coordinate system.

9. The method of any one of claims 7 or 8 wherein the second error indicator comprises a symbol indicating whether the degrees of angular’ error are in excess, in deficit, or zero.

10. The method of claim 1 further comprising a graphic being displayed, the graphic being a line, a target symbol, a current position symbol, or a combination thereof , wherein orientation of the line, when present, represents a direction that the hand-held robot is manuallyDocket No. CURE-0232PCT-P0118-WO moved by a user and wherein the target symbol, when present, indicates the location of the target axis relative to a location of the axis of the end-effector in at least one degree-of-freedom.

11. The method of claim 10 further comprising changing the orientation of the line when present.

12. The method of claim 10 further comprising the graphic changing color, blink-rate, or a combination thereof in response to a preselected condition.

13. The method of claim 10 wherein the graphic updates in real-time.

14. The method of claim 1 further comprising automatically energizing the end effector when the axis of the end-effector is aligned with the target axis.

15. The method of claim 14 wherein energizing the end-effector causes the end effector to rotate.

16. The method of claim 1 further comprising automatically de-energizing the end effector when the axis of the end-effector deviates from being aligned with the target axis by a tolerance amount.

17. The method of 16 wherein the tolerance amount is automatically or manually adjustable or computed based on a criteria.Docket No. CURE-0232PCT-P0118-WO18. The method of any one of claims 1 to 17 further comprising communicating between the heads-up display and at least one of, a tracking system, a computer, a mobile smart device, or a combination thereof19. The method of any one of claims 1 to 17 wherein the bone is part of a joint, including a hip, knee, shoulder, spine, or elbow, and where the bone is either an intact bone in its entirety, or any portion thereof, including a detached fragment.

20. The method of any one of claims 1 to 17 further comprising modifying the bone surgically to repair a defect therein or mount an implant thereto.

21. A computer-assisted surgical system, comprising: a hand-held robot, comprising: a hand-held portion; a working portion mo v ably coupled to the hand-held portion for driving an endeffector having an axis; and a plurality of actuators for moving the working portion relative to the hand-held portion; a heads -up -di splay configured to be worn on a head of a user; and a computing system having a processor and non-transient memory configured to: display information on the heads-up-display indicating where the axis of the endeffector is located relative to a location of a target axis in one or more degrees-of-freedom;Docket No. CURE-0232PCT-P0118-WO update the information on the heads-up-display as the end-effector is moved relative to the target axis; and indicate when the location of the axis of the end-effector is aligned with the location of the target axis in one or more degrees-of-freedom.

22. The system of claim 21 wherein the plurality of actuators automatically control movement the working portion in response to control signals from a computing system.

23. The system of claim 22 wherein the plurality of actuators automatically control movement of the working portion in no more than two degrees-of-freedom.

24. The system of claim 22 wherein the plurality of actuators automatically control movement of the end-effector in the two degrees-of-freedom while the hand-held robot is manually moved by a user in the other degrees-of-freedom.

25. The system of claim 21 wherein the information is provided by the computing system.

26. The system of claim 21 further comprising a tracking system in communication with the computing system.

27. The system of claim 21 wherein the computing system is configured to control activation of the end-effector.Docket No. CURE-0232PCT-P0118-WO28. The system of any one of claims 21 to 26 wherein the working portion further comprises a motor configured to provide a rotary or oscillatory motion to the end-effector.

29. The system of any one of claims 21 to 26 wherein the computing system is configured to generate control signals to maintain the axis of the end-effector coincident with a virtual plane defined relative to the bone.

30. The system of any one of claims 21 to 26 wherein the displayed information comprises: an end-effector position indicator; a first degree-of-freedom movement indicator; a second degree-of-freedom movement indicator; a target indicator; or a combination thereof.

31. The system of any one of claims 21 to 26 wherein the displayed information further comprises a first error indicator indicating the error between the current position of the end-effector and the location of the target axis in a translation degree-of-freedom.

32. The system of claim 21 wherein the displayed information further comprises a second error indicator indicating the degrees of angular' error between the current position of the end-effector and the location of the target axis in a rotational degree-of-freedom.

33. The system of claim 32 wherein the degrees of angular error represent degrees of error in one or more orthogonal rotational degrees of freedom in any definable coordinate system.Docket No. CURE-0232PCT-P0118-WO34. The system of any one of claims 32 or 33 wherein the second error indicator comprises a symbol indicating whether the degrees of angular error are in excess, in deficit, or zero.

35. The system of claim 21 further comprising a graphic being displayed wherein the graphic is a line, a target symbol, a current position symbol, or a combination thereof, wherein an orientation of the line, when present, represents a direction that the hand-held robot is manually moved by a user and wherein the target symbol, when present, indicates the location of the target axis relative to the current position of the end-effector axis in one or more degrees-of-freedom.

36. The system of claim 35 further comprising changing the orientation of the line.

37. The system of claim 35 further comprising the graphic changing color, blink-rate, or a combination thereof in response to a preselected condition.

38. The system of claim 35 wherein the graphic updates in real-time.

39. The system of claim 21 further comprising automatically energizing the end effector when the axis of the end-effector is aligned with the target axis.

40. The method of claim 39 wherein energizing the end-effector causes the end effector to rotate.Docket No. CURE-0232PCT-P0118-WO41. The method of claim 21 further comprising automatically de-energizing the end effector when the axis of the end-effector deviates from being aligned with the target axis with a tolerance amount.

Images