Optical and non-optical sensor tracking of robotically controlled instruments

Abstract

The invention relates to optical and non-optical sensor tracking of robotically controlled instruments. Systems and methods for determining the position and / or orientation of an object using optical and non-optical sensors. The navigation system includes an optical sensor for receiving an optical signal from a marker and a non-optical sensor, such as a gyroscope, for generating non-optical data. The navigation computer determines a position and / or orientation of the object based on the optical and non-optical data.

Description

[0001] This application is a divisional application of an application with an application date of September 25, 2013, application number 202110822755.X, and invention name “Navigation system including optical sensors and non-optical sensors”. Application 202110822755.X is a divisional application of an invention patent application with an application date of September 25, 2013, application number 201811105300.0, and invention name “Navigation system including optical sensors and non-optical sensors”. Application 201811105300.0 is a divisional application of an invention patent application with an application date of September 25, 2013, application number 201380050199.2, and invention name “Navigation system including optical sensors and non-optical sensors”.

[0002] Related Applications

[0003] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 61 / 705,804, filed September 26, 2012, and U.S. Non-Provisional Application No. 14 / 035207, filed September 24, 2013, the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present invention generally relates to navigation systems that track objects in space by determining the position and / or orientation of such objects over time. More specifically, the present invention relates to navigation systems that utilize optical sensors and non-optical sensors to determine the position and / or orientation of an object. Background Art

[0005] Navigation systems assist users in accurately positioning objects. For example, navigation systems are used in industrial, aerospace, defense, and medical applications. In the medical field, navigation systems assist surgeons in accurately placing surgical instruments relative to the patient's anatomy.

[0006] Surgical procedures that use navigation systems include neurosurgery and orthopedic surgery. Typically, the instrument and anatomical structure are tracked, along with their relative motion displayed on a display. The navigation system can display instrument movement in conjunction with preoperative images of the anatomical structure or intraoperative images. Typically preoperative images are prepared by MRI or CT scans, while intraoperative images can be prepared using fluoroscopes, low-level X-rays, or any similar device. Optionally, some systems are image-free, where the patient's anatomical structure is "depicted" by a navigation probe and mathematically fitted to an anatomical model for display.

[0007] Navigation systems may employ light signals, sound waves, magnetic fields, RF signals, etc. to track the position and / or orientation of instruments and anatomical structures.Optical navigation systems are widely used due to their accuracy.

[0008] Prior art optical navigation systems typically include one or more camera units housing one or more optical sensors, such as a charge coupled device or CCD. The optical sensors detect light emitted from a tracker that is attached to the instrument and anatomical structure. Each tracker has a plurality of optical emitters, such as light emitting diodes (LEDs), that periodically send light to the sensor to determine the position of the LEDs.

[0009] The position of the LED on the instrument tracker is related to the coordinates of the working end of the instrument relative to the camera coordinate system. The position of the LED on one or more anatomical structure trackers is related to the coordinates of the target region of the anatomical structure in three-dimensional space relative to the camera coordinate system. Therefore, the position and / or orientation of the working end of the instrument relative to the target region of the anatomical structure can be tracked and displayed.

[0010] Navigation systems can be used in a closed loop to control the movement of surgical instruments. In these navigation systems, both the instrument and the anatomical structure being treated are equipped with trackers so that the navigation system can track their position and orientation. The information from the navigation system is then fed to the control system to control or guide the movement of the instrument. In some cases, the instrument is grasped by a robot and the information is sent from the navigation system to the robot's control system.

[0011] In order for the control system to quickly cause relative motion between the instrument and the anatomical structure being treated, the accuracy and speed of the navigation system must meet the desired tolerances of the procedure. For example, the tolerances associated with a cementless knee implant may be very small to ensure adequate fit and function of the implant. Therefore, the accuracy and speed of the navigation system may need to be greater than that in more coarse cutting procedures.

[0012] One of the limitations on the accuracy and speed of optical navigation systems is that the system relies on line of sight between the optical sensor of the camera unit and the LED. When line of sight is broken, the system may not accurately determine the position and / or orientation of the currently tracked instrument and anatomical structure. Therefore, the surgical procedure may encounter many starts and stops. For example, during the control of robot-assisted cutting, when line of sight is broken, the cutting tool must be disabled until line of sight is regained. This may cause significant delays and additional costs to the procedure.

[0013] Another limitation in accuracy occurs when using active LEDs on the tracker. In such systems, the LEDs are typically fired in sequence. In this case, the system only measures and knows the position of the actively fired LEDs, while the positions of the remaining unmeasured LEDs are unknown. In these systems, the positions of the remaining unmeasured LEDs are approximated. The approximation is typically based on linear velocity data inferred from the last known measured position of the currently unmeasured LED. However, since the LEDs are fired in sequence, there may be considerable lag between the measurements of any one LED. This lag increases with each additional tracker used in the system. Furthermore, the approximation does not take into account rotation of the tracker, also leading to possible errors in the tracker's position data.

[0014] Therefore, there is a need in the art for optical navigation systems that utilize additional non-optical data based to improve tracking and provide such levels of accuracy and speed with which the position and / or orientation of an object can be determined for precision surgical procedures (such as robotic-assisted surgical cutting). Summary of the invention

[0015] The present invention relates to systems and methods for determining the position and / or orientation of an object using optical sensors and non-optical sensors.

[0016] In one version of the invention, a navigation system for tracking an object is provided. The navigation system includes an optical sensor that receives an optical signal from one or more markers on a tracker. The tracker also includes a non-optical sensor that generates a non-optical signal. A computing system determines a position of one of the markers at a first time based on a first optical signal. The computing system also determines a position of one or more of the other markers at the first time based on the first optical signal and the non-optical signal from the non-optical sensor. The determined position is then correlated with the object to track the position of the object.

[0017] In another version of the invention, a navigation system for tracking an object is provided. The navigation system includes an optical sensor that sequentially receives optical signals from three markers on the tracker. The tracker also includes a non-optical sensor that generates a non-optical signal. A computing system determines a position of a first marker in the markers at a first time based on a first optical signal from the first marker. The computing system also determines a position of a second marker and a third marker in the markers at the first time based on the first optical signal and the non-optical signal from the non-optical sensor. The determined position is then correlated with the object to track the position of the object.

[0018] In yet another version of the invention, a robotic surgical cutting system is provided. The system includes a robotic manipulator and a cutting tool. The robotic control system controls or constrains the movement of the cutting tool in at least 5 degrees of freedom. A navigation system communicates with the robotic control system. The navigation system includes at least one optical sensor and a tracker mounted to the robotic manipulator. A tracker for mounting to an anatomical structure of a patient is also provided. The anatomical structure tracker includes a non-optical sensor and three markers. The optical sensor receives an optical signal from the marker and the non-optical sensor generates a non-optical signal. The navigation system transmits position data indicating the position of the anatomical structure to the robotic control system to control the cutting of the anatomical structure so that the cutting occurs within predefined boundaries.

[0019] In another version of the invention, a navigation system is provided that includes a locator having at least one optical sensor. A tracker is in communication with the optical sensor. The tracker includes a non-optical sensor and three markers. A computing system determines a position of each of the three markers in a locator coordinate system based on the optical signal and the non-optical signal. The computing system executes a matching algorithm to match the determined positions of one or more of the markers in the locator coordinate system with the positions of one or more of the markers in a model of the tracker established relative to the tracker coordinate system to obtain a transformation matrix that transforms the tracker coordinate system into the locator coordinate system.

[0020] In another version of the invention, a system for tracking an object is provided. The system includes at least two optical sensors and a tracker for mounting to the object. The tracker has a non-optical sensor and three markers. At least two of the optical sensors receive optical signals from the markers at an optical sensing frequency of at least 100 Hz. The non-optical sensor generates a non-optical signal at a non-optical sensing frequency of at least 100 Hz.

[0021] A method for tracking an object is also provided. The method includes operating an optical sensor to sequentially receive optical signals from a marker and operating a non-optical sensor to generate a non-optical signal. Based on a first optical signal from the first marker, a position of a first marker in the marker at a first time is determined. Based on the first optical signal and the non-optical signal from the non-optical sensor, a position of a second marker and a third marker in the marker at the first time is determined. The determined positions of the first marker, the second marker, and the third marker are correlated with the object to track the position of the object during a surgical procedure.

[0022] Another method for tracking an object during a surgical procedure is provided. In the method, three markers are placed in a field of view of an optical sensor so that the optical sensor sequentially receives optical signals from the markers. Then, a computing system is operated to determine the position of a first marker in the markers at a first time based on a first optical signal from the first marker and to determine the position of a second marker and a third marker in the markers at the first time based on the first optical signal and a non-optical signal from a non-optical sensor. Then, the position is correlated with the object to track the position of the object during the surgical procedure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Advantages of the present invention will be readily appreciated and better understood by referring to the following detailed description when considered in conjunction with the accompanying drawings, in which:

[0024] Figure 1 is a perspective view of the navigation system of the present invention when used in conjunction with a robotic manipulator;

[0025] Figure 2 It is a schematic diagram of the navigation system;

[0026] Figure 3 is a schematic diagram of the coordinate system used by the navigation system;

[0027] Figure 4 is a flow chart of steps implemented by a positioning engine of a navigation system;

[0028] Figure 4A is a schematic illustration of matching the measured LEDs with the tracker model to obtain the transformation matrix;

[0029] Figure 5 is a flow chart of steps performed by a positioning engine in a first alternative embodiment;

[0030] Figure 5A is a description of a tracker model including real and virtual LEDs;

[0031] Figure 6 is a flow chart of the steps performed by the positioning engine in a second alternative embodiment; and

[0032] Figure 7 is a flow chart of the steps performed by the positioning engine when one or more LEDs are blocked from measurement. DETAILED DESCRIPTION

[0033] I. Overview

[0034] Reference Figure 1, a surgical navigation system 20 is illustrated. The system 20 is shown in a surgical environment, such as an operating room of a medical facility. The navigation system 20 is set up to track the movement of various objects in the operating room. Such objects include, for example, a surgical instrument 22, a patient's femur F, and a patient's tibia T. The navigation system 20 tracks these objects in order to display their relative positions and orientations to the surgeon, and in some cases, to control or constrain the movement of the surgical instrument 22 relative to a predetermined path or anatomical boundary.

[0035] The surgical navigation system 20 includes a computer cart assembly 24 that houses a navigation computer 26. A navigation interface is in operative communication with the navigation computer 26. The navigation interface includes a display 28 adapted to be located outside the sterile field and a display 29 adapted to be located inside the sterile field. The displays 28, 29 are adjustably mounted to the computer cart assembly 24. Input devices 30, 32 (such as a mouse and keyboard) can be used to input information into the navigation computer 26 or otherwise select / control certain aspects of the navigation computer 26. Other input devices are contemplated, including touch screens (not shown) or voice activation on the displays 28, 29.

[0036] The localizer 34 communicates with the navigation computer 26. In the illustrated embodiment, the localizer 34 is an optical localizer and includes a camera unit 36. The camera unit 36 ​​has a housing 38 that accommodates one or more optical sensors 40. In some embodiments, at least two optical sensors 40 are used, preferably three. The optical sensors 40 can be three separate high-resolution charge coupled devices (CCDs). In one embodiment, three one-dimensional CCDs are used. It should be understood that in other embodiments, separate camera units each having a separate CCD or two or more CCDs can also be arranged around the operating room. The CCD detects infrared (IR) signals.

[0037] The camera unit 36 ​​is mounted on an adjustable arm to position the optical sensor 40 over the area where the procedure will be performed to provide the camera unit 36 ​​with an ideally unobstructed field of view for the tracker discussed below.

[0038] The camera unit 36 ​​includes a camera controller 42 that communicates with the optical sensor 40 to receive signals from the optical sensor 40. The camera controller 42 communicates with the navigation computer 26 via a wired connection or a wireless connection (not shown). One such connection may be an IEEE 1394 interface, which is a serial bus interface standard for high-speed communication and isochronous real-time data transfer. The connection may also use a company-specific protocol. In other embodiments, the optical sensor 40 communicates directly with the navigation computer 26.

[0039] To track the object, position and orientation signals and / or data are transmitted to the navigation computer 26. The computer cart assembly 24, display 28, and camera unit 36 ​​may be as described in U.S. Pat. No. 7,725,162, issued May 25, 2010 to Malackowski et al., entitled “Surgery System,” which is incorporated herein by reference.

[0040] The navigation computer 26 may be a personal computer or a laptop computer. The navigation computer 26 has a display 28, a central processing unit (CPU) and / or other processor, a memory (not shown), and a storage device (not shown). As described below, the navigation computer 26 is loaded with software. The software converts the signals received from the camera unit 36 ​​into data representing the position and direction of the object being tracked.

[0041] The navigation system 20 includes a plurality of tracking devices 44, 46, 48 (also referred to herein as trackers). In the illustrated embodiment, one tracker 44 is securely secured to the patient's femur F and another tracker 46 is securely secured to the patient's tibia T. The trackers 44, 46 are securely secured to portions of the bones. The trackers 44, 46 may be attached to the femur F in the manner shown in U.S. Pat. No. 7,725,162 (which is incorporated herein by reference). In further embodiments, an additional tracker (not shown) is attached to the patella to track the position and orientation of the patella. In further embodiments, the trackers 44, 46 may be mounted to other tissue types or portions of the anatomical structure.

[0042] The instrument tracker 48 is securely attached to the surgical instrument 22. The instrument tracker 48 can be integrated into the surgical instrument 22 during manufacture or can be separately mounted to the surgical instrument 22 in preparation for a surgical procedure. The working end of the surgical instrument 22 currently being tracked can be a rotary drill, an electroablation device, or the like. In the illustrated embodiment, the surgical instrument 22 is an end effector of a surgical manipulator. This arrangement is shown in U.S. Provisional Patent Application No. 61 / 679,258, entitled “Surgical Manipulator Capable of Controlling a Surgical Instrument in either a Semi-Autonomous Mode or a Manual, Boundary Constrained Mode,” the disclosure of which is incorporated herein by reference, and also in U.S. Patent Application No. 13 / 958,834, entitled “Navigation System for use with a Surgical Manipulator Operable in Manualor Semi-Autonomous Mode,” the disclosure of which is incorporated herein by reference.

[0043] The trackers 44, 46, 48 may be battery powered with internal batteries or may have leads to receive power through the navigation computer 26 (preferably receiving external power like the camera unit 36).

[0044] In other embodiments, the surgical instrument 22 can be manually positioned solely by the user's hands without the assistance of any cutting guide, cantilever, or other restraining mechanism (such as a manipulator or robot). Such a surgical instrument is described in U.S. Provisional Patent Application No. 61 / 662,070, entitled "Surgical Instrument Including Housing, a Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing," which is incorporated herein by reference, and also in U.S. Patent Application No. 13 / 600,888, entitled "Surgical Instrument Including Housing, a Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing," which is incorporated herein by reference.

[0045] The optical sensor 40 of the locator 34 receives light signals from the trackers 44, 46, 48. In the illustrated embodiment, the trackers 44, 46, 48 are active trackers. In this embodiment, each tracker 44, 46, 48 has at least three active markers 50 for sending light signals to the optical sensor 40. The active markers 50 can be light emitting diodes or LEDs 50. The optical sensor 40 preferably has a sampling rate of 100 Hz or higher, more preferably has a sampling rate of 300 Hz or higher, and most preferably has a sampling rate of 500 Hz or higher. In some embodiments, the optical sensor 40 has a sampling rate of 1000 Hz. The sampling rate is the rate at which the optical sensor 40 receives light signals from the sequentially illuminated LEDs 50. In some embodiments, for each tracker 44, 46, 48, the light signals from the LEDs 50 are illuminated at different rates.

[0046] Reference Figure 2, each of the LEDs 50 is connected to a tracker controller 62 located in the housing (not shown) of the associated tracker 44, 46, 48, which transmits / receives data to / from the navigation computer 26. In one embodiment, the tracker controller 62 transmits data at the order of several megabytes / second via a wired connection to the navigation computer 26. In other embodiments, a wireless connection may be used. In these embodiments, the navigation computer 26 has a transceiver (not shown) for receiving data from the tracker controller 62.

[0047] In other embodiments, the trackers 44, 46, 48 may have passive markers (not shown), such as reflectors that reflect light emitted from the camera unit 36. The reflected light is then received by the optical sensor 40. Active and passive arrangements are well known in the art.

[0048] Each of the trackers 44, 46, 48 also includes a 3-dimensional gyro sensor 60 that measures the angular velocity of the tracker 44, 46, 48. As is well known to those skilled in the art, the gyro sensor 60 outputs readings indicating the angular velocity of the x-axis, y-axis, and z-axis relative to the gyro coordinate system. These readings are multiplied by conversion constants defined by the manufacturer to obtain a measurement in degrees / second relative to each of the x-axis, y-axis, and z-axis of the gyro coordinate system. These measurements can then be converted to an angular velocity vector defined in radians / second

[0049] The angular velocity measured by the gyro sensor 60 provides the navigation system 20 with additional non-optical based kinematic data for tracking the trackers 44, 46, 48. The gyro sensor 60 may be oriented along the axes of each coordinate system of the trackers 44, 46, 48. In other embodiments, each gyro coordinate system is transformed to its tracker coordinate system so that the gyro data reflects angular velocity relative to the x-, y-, and z-axes of the coordinate system of the trackers 44, 46, 48.

[0050] Each of the gyro sensors 60 communicates with a tracker controller located within the housing of the associated tracker, which transmits / receives data to / from the navigation computer 26. The navigation computer 26 has one or more transceivers (not shown) to receive data from the gyro sensors 60. The data may be received via a wired connection or a wireless connection.

[0051] The gyro sensor 60 preferably has a sampling rate of 100 Hz or higher, more preferably has a sampling rate of 300 Hz or higher, and most preferably has a sampling rate of 500 Hz or higher. In some embodiments, the gyro sensor 60 has a sampling rate of 1000 Hz. The sampling rate of the gyro sensor 60 is the rate at which a signal is sent from the gyro sensor 60 to be converted into angular velocity data.

[0052] The sampling rates of the gyroscopic sensor 60 and the optical sensor 40 are established or calibrated so that for every optical measurement of position there is a corresponding non-optical measurement of angular velocity.

[0053] Each of the trackers 44,46,48 also includes a 3-axis accelerometer 70 that measures acceleration along each of the x-axis, y-axis, and z-axis of the accelerometer coordinate system. The accelerometer 70 provides additional non-optical based data for the navigation system 20 to track the trackers 44,46,48.

[0054] Each of the accelerometers 70 communicates with a tracker controller located in the housing of the associated tracker, which transmits and receives data to and from the navigation computer 26. The data from the accelerometers 70 is received by one or more of the navigation computer's transceivers (not shown).

[0055] Accelerometer 70 may be oriented along the axes of each coordinate system of trackers 44, 46, 48. In other embodiments, each accelerometer coordinate system is transformed to its tracker coordinate system so that the accelerometer data reflects acceleration relative to the x-, y-, and z-axes of the tracker's 44, 46, 48 coordinate system.

[0056] The navigation computer 26 includes a navigation processor 52. The camera unit 36 ​​receives optical signals from the LEDs 50 of the trackers 44, 46, 48 and outputs signals to the processor 52 related to the positions of the LEDs 50 of the trackers 44, 46, 48 relative to the locator 34. The gyro sensor 60 transmits non-optical signals to the processor 52, which are related to the 3-dimensional angular velocity measured by the gyro sensor 60. Based on the received optical and non-optical signals, the navigation processor 52 generates data indicating the relative position and orientation of the trackers 44, 46, 48 relative to the locator 34.

[0057] It should be understood that the navigation processor 52 may include one or more processors to control the operation of the navigation computer 26. The processor may be any type of microprocessor or multi-processor system. The term processor is not intended to limit the scope of the present invention to a single processor.

[0058] Prior to commencing a surgical procedure, additional data is loaded into the navigation processor 52. Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, the navigation processor 52 determines the position of the working end of the surgical instrument 22 and the orientation of the surgical instrument 22 relative to the tissue to which the working end is to be applied. In some embodiments, the navigation processor 52 forwards this data to the manipulator controller 54. The manipulator controller 54 can then use this data to control the robotic manipulator 56 as described in U.S. Provisional Patent Application No. 61 / 679,258, entitled "Surgical Manipulator Capable of Controlling a Surgical Instrument in either a Semi-Autonomous Mode or a Manual, Boundary Constrained Mode," the disclosure of which is incorporated herein by reference, and U.S. Patent Application No. 13 / 958,834, also entitled "Navigation System for use with a Surgical Manipulator Operable in Manual or Semi-Autonomous Mode," the disclosure of which is incorporated herein by reference.

[0059] The navigation processor 52 also generates image signals indicating the relative position of the working end of the surgical instrument relative to the surgical site. These image signals are applied to the displays 28, 29. The displays 28, 29 generate images based on these signals that allow the surgeon and staff to view the relative position of the working end of the surgical instrument relative to the surgical site. As discussed above, the displays 28, 29 may include a touch screen or other input / output device that allows command input.

[0060] II. Coordinate Systems and Transformations

[0061] Reference Figure 3 , the object is usually tracked with reference to a localizer coordinate system LCLZ. The localizer coordinate system has an origin and an orientation (a set of x-axis, y-axis, and z-axis). During the procedure, one goal is to keep the localizer coordinate system LCLZ stationary. As will be further described below, an accelerometer mounted to the camera unit 36 ​​can be used to track sudden or unexpected movement of the localizer coordinate system LCLZ (such as may occur when the camera unit 36 ​​is accidentally bumped by surgical personnel).

[0062] Each tracker 44, 46, 48 and the currently tracked object also has its own coordinate system that is independent of the localizer coordinate system LCLZ. The components of the navigation system 20 that have their own coordinate systems are the skeleton trackers 44, 46 and the instrument tracker 48. These coordinate systems are denoted as the skeleton tracker coordinate system BTRK1, BTRK2 and the instrument tracker coordinate system TLTR, respectively.

[0063] The navigation system 20 monitors the position of the patient's femur F and tibia T by monitoring the position of bone trackers 44, 46 securely attached to the bones. The femur coordinate system is FBONE and the tibia coordinate system is TBONE, which are the coordinate systems of the bones to which the bone trackers 44, 46 are securely attached.

[0064] Prior to starting the procedure, preoperative images of the femur F and tibia T (or other tissues in other embodiments) are generated. These images may be based on an MRI scan, a radiological scan, or a computed tomography (CT) scan of the patient's anatomy. These images are mapped to a femur coordinate system FBONE and a tibia coordinate system TBONE using methods known in the art. In one embodiment, a pointer instrument P (referenced to FIG. 1 ) with its own tracker PT is used. Figure 2 ) (such as disclosed in U.S. Pat. No. 7,725,162 to Malackowski et al., which is incorporated herein by reference) can be used to map the femoral coordinate system FBONE and the tibial coordinate system TBONE to the pre-operative images. These images are fixed in the femoral coordinate system FBONE and the tibial coordinate system TBONE.

[0065] During the initial stages of the procedure, the skeletal trackers 44, 46 are securely fixed to the patient's skeleton. The poses (positions and orientations) of the coordinate systems FBONE and TBONE are mapped to the coordinate systems BTRK1 and BTRK2, respectively. Given the fixed relationship between the skeleton and their skeletal trackers 44, 46, the poses of the coordinate systems FBONE and TBONE remain fixed relative to the coordinate systems BTRK1 and BTRK2, respectively, during the entire procedure. Data describing the poses are stored in a memory integrated with both the manipulator controller 54 and the navigation processor 52.

[0066] The working end (also called the energy applicator tip) of the surgical instrument 22 has its own coordinate system EAPP. For example, the origin of the coordinate system EAPP can represent the center of mass of a surgical cutting drill. Before the program starts, the pose of the coordinate system EAPP is fixed to the pose of the instrument tracker coordinate system TLTR. Therefore, the poses of these coordinate systems EAPP, TLTR relative to each other are determined. Data describing the poses are stored in a memory integrated with both the manipulator controller 54 and the navigation processor 52.

[0067] III. Software

[0068] Reference Figure 2 , positioning engine 100 is a software module that may be considered part of navigation system 20. Components of positioning engine 100 run on navigation processor 52. In some versions of the invention, positioning engine 100 may run on manipulator controller 54.

[0069] The positioning engine 100 receives as input optical-based signals from the camera controller 42 and non-optical-based signals from the tracker controller 62. Based on these signals, the positioning engine 100 determines the pose (position and orientation) of the bone tracker coordinate systems BTRK1 and BTRK2 in the localizer coordinate system LCLZ. Based on the same signals received for the instrument tracker 48, the positioning engine 100 determines the pose of the instrument tracker coordinate system TLTR in the localizer coordinate system LCLZ.

[0070] The positioning engine 100 forwards signals representing the pose of the trackers 44, 46, 48 to a coordinate transformer 102. The coordinate transformer 102 is a navigation system software module that runs on the navigation processor 52. The coordinate transformer 102 references data defining the relationship between the pre-operative images of the patient and the patient trackers 44, 46. The coordinate transformer 102 also stores data indicating the pose of the working end of the surgical instrument relative to the instrument tracker 48.

[0071] During the procedure, the coordinate transformer 102 receives data indicating the relative pose of the trackers 44, 46, 48 relative to the localizer 34. Based on these data and previously loaded data, the coordinate transformer 102 generates data indicating the relative position and orientation of the coordinate system EAPP and both the bone coordinate systems FBONE and TBONE relative to the localizer coordinate system LCLZ.

[0072] Thus, the coordinate transformer 102 generates data indicating the position and orientation of the working end of the surgical instrument 22 relative to the tissue (e.g., bone) to which the working end of the instrument is applied. Image signals representing these data are forwarded to the displays 28, 29, allowing the surgeon and staff to view the information. In some embodiments, other signals representing these data may be forwarded to the manipulator controller 54 to control the corresponding movement of the manipulator 56 and the surgical instrument 22.

[0073] Since the steps for determining the pose of each of the tracker coordinate systems BTRK1, BTRK2, TLTR in the locator coordinate system LCLZ are identical, only one will be described in detail. Figure 4The steps shown are based on only one active tracker, tracker 44. In the following description, the LEDs of tracker 44 will be represented by reference numerals 50a, 50b, 50c, identifying a first 50a LED, a second 50b LED, and a third 50c LED.

[0074] Figure 4 The steps set forth in show the use of optical-based sensor data and non-optical-based sensor data to determine the positions of the LEDs 50a, 50b, 50c of the tracker 44. From these positions, the navigation processor 52 can determine the position and orientation of the tracker 44, and therefore the position and orientation of the femur F to which the tracker 44 is attached. The optical-based sensor data derived from the signals received by the optical sensor 40 provides line-of-sight based data that relies on line-of-sight between the LEDs 50a, 50b, 50c and the optical sensor 40. However, the gyroscope sensor 60 that provides the non-optical-based signal used to generate the non-optical-based sensor data does not rely on line-of-sight and can therefore be integrated into the navigation system 20 to better approximate the positions of the LEDs 50a, 50b, 50c when two of the LEDs 50a, 50b, 50c are not currently being measured (because only one LED is measured at a time) or when one or more of the LEDs 50a, 50b, 50c are not visible to the optical sensor 40 during a procedure.

[0075] In a first initialization step 200, the system 20 measures the positions of the LEDs 50a, 50b, 50c for the tracker 44 in the locator coordinate system LCLZ to establish initial position data. These measurements are obtained by sequentially illuminating the LEDs 50a, 50b, 50c, which transmit light signals to the optical sensor 40. Once the light signals are received by the optical sensor 40, corresponding signals are generated by the optical sensor 40 and transmitted to the camera controller 42. The frequency between the illumination of the LEDs 50a, 50b, 50c is 100 Hz or greater, preferably 300 Hz or greater, and more preferably 500 Hz or greater. In some cases, the frequency between illumination is 1000 Hz or 1 millisecond between illuminations.

[0076] In some embodiments, the optical sensor 40 can read only one LED at a time. As described in U.S. Pat. No. 7,725,162 to Malackowski et al. (which is incorporated herein by reference), the camera controller 42 can control the illumination of the LEDs 50a, 50b, 50c via one or more infrared transceivers or RF transceivers (on the camera unit 36 ​​and the tracker 44). Alternatively, in the absence of instructions from the camera controller 42, the tracker 44 can be activated locally (such as by a switch on the tracker 44), which once activated illuminates its LEDs 50a, 50b, 50c sequentially.

[0077] Based on input from the optical sensor 40, the camera controller 42 generates raw position signals which are then sent to the positioning engine 100 to determine the position of each of the corresponding three LEDs 50a, 50b, 50c in the localizer coordinate system LCLZ.

[0078] During the initialization step 200, in order to establish initial position data, the movement of the tracker 44 must be less than a predetermined threshold. The value of the predetermined threshold is stored in the navigation computer 26. The initial position data established in step 200 essentially provides a static snapshot of the position of the three LEDs 50a, 50b, 50c at an initial time t0, from which the remaining steps of the process are based. During initialization, the speed of the LEDs 50a, 50b, 50c is calculated by the positioning engine 100 between cycles (i.e., each group of three LED measurements) and once the speed is sufficiently low (i.e., less than a predetermined threshold indicating that little or no motion is occurring), the initial position data or static snapshot is established. In some embodiments, the predetermined threshold (also referred to as the static speed limit) is 200 mm / s or less, preferably 100 mm / s or less, and more preferably 10 mm / s or less along any axis. When the predetermined threshold is 100 mm / s, then the calculated speed must be less than 100 mm / s to establish a static snapshot.

[0079] Reference Figure 4 and 4A Once the static snapshot is obtained, in step 202, the measured positions of the LEDs 50a, 50b, 50c are compared to the model of the tracker 44. The model is data stored in the navigation computer 26. The model data indicates the positions of the LEDs on the tracker 44 in the tracker coordinate system BTRK1. The system 20 stores the number and position of the LEDs 50 of each tracker 44, 46, 48 in the coordinate system of each tracker. For the trackers 44, 46, 48, the origin of their coordinate systems is set at the centroid of all the LED positions of the tracker 44.

[0080] The positioning engine 100 matches the measured LEDs 50a, 50b, 50c in the locator coordinate system LCLZ with the LEDs in the stored model using a rigid body matching algorithm or a point matching algorithm. Once the best fit is determined, the positioning engine 100 evaluates the deviation of the fit to determine whether the measured LEDs 50a, 50b, 50c are within a predetermined tolerance range of the stored model. The tolerance can be based on the distance between the corresponding LEDs so that if the fit results in a distance that is too large, the initialization step has to be repeated. In some embodiments, the position of the LED cannot deviate from the model by more than 2.0 mm, preferably by no more than 0.5 mm, and more preferably by no more than 0.1 mm.

[0081] In step 204, if the fit is within a predetermined tolerance, a transformation matrix is ​​generated to transform any other unmeasured LEDs in the model from the skeletal tracker coordinate system BTRK1 to the locator coordinate system LCLZ. As further explained below, this step is utilized if more than three LEDs are used or if virtual LEDs are used. In some embodiments, the trackers 44, 46, 48 may have four or more LEDs. Once all positions in the locator coordinate system LCLZ are established, an LED cloud is created. The LED cloud is the arrangement of all LEDs 50a, 50b, 50c on the tracker 44 in the locator coordinate system LCLZ based on the x-axis, y-axis, and z-axis positions of all LEDs 50a, 50b, 50c in the locator coordinate system LCLZ.

[0082] Once the LED cloud is initially established, the navigation system 20 can track the tracker 44 during the surgical procedure. As previously discussed, this includes lighting the next LED in the sequence. For illustration, LED 50a is now illuminated. Therefore, LED 50a transmits a light signal to the optical sensor 40. Once the light signal is received by the optical sensor 40, a corresponding signal is generated by the optical sensor 40 and transmitted to the camera controller 42.

[0083] Based on input from the optical sensor 40, the camera controller 42 generates a raw position signal which is then sent to the positioning engine 100 to determine the new position of the LED 50a relative to the x-, y- and z-axes of the localizer coordinate system LCLZ at time t1. This is shown in step 206 as a new LED measurement.

[0084] It should be understood that time indications such as t0, t1 . . . tn are used to indicate different times or different time ranges or time periods for illustrative purposes and do not limit the present invention to a specific time or explicit time.

[0085] In step 208 , using the determined new position of LED 50 a , a linear velocity vector of LED 50 a may be calculated by positioning engine 100 .

[0086] Tracker 44 is considered a rigid body. Therefore, the linear velocity vector of LED 50a is a vector quantity equal to the time rate of change of its linear position. The velocity of each LED in the localizer coordinate system LCLZ (and even the acceleration of each LED) can be calculated from the previous and current measured position and time of the LED in the localizer coordinate system LCLZ. The previous and current measured position and time of the LED define the position history of the LED. The velocity calculation of LED 50a can take the following simplest form:

[0087]

[0088] in is the previous p The position of LED 50a measured at is the current time t n As is well known to those skilled in the art, the speed and / or acceleration of each LED can also be obtained by performing data fitting on the LED position history of the LED.

[0089] At time t1, in step 210, gyro sensor 60 also measures the angular velocity of tracker 44. Gyro sensor 60 transmits a signal to tracker controller 62 related to the angular velocity.

[0090] Then, the tracker controller 62 transmits corresponding signals to the positioning engine 100 so that the positioning engine 100 can calculate the angular velocity vector according to these signals. In step 210, the gyroscope coordinate system is also transformed into the skeleton tracker coordinate system BTRK1, so that the angular velocity vector calculated by the positioning engine 100 is represented in the skeleton tracker coordinate system BTRK1.

[0091] In step 212, relative to the position vector Calculate the relative velocity vector for the origin of the skeleton tracker coordinate system BTRK1 The position vector It is also stored in the memory in the navigation computer 26 for access by the positioning engine 100 for performing the following calculations. This calculation is performed by calculating the angular velocity vector derived from the gyroscope signal The vector product with the position vector from LED 50a to the origin determines the relative velocity of the origin of the skeleton tracker coordinate system BTRK1.

[0092]

[0093] The positioning engine 100 then calculates the relative velocity vectors for the remaining unmeasured LEDs 50b, 50c (which are unmeasured because these LEDs are not lit and therefore their positions are not measured). These velocity vectors may be calculated relative to the origin of the skeletal tracker coordinate system BTRK1.

[0094] Executed by the positioning engine 100 to determine the relative velocity vector of each unmeasured LED 50b, 50c at time t1 The calculation is based on the angular velocity vector and position vector at time t1 and The vector product of the position vector and is obtained from the origin of the skeleton tracker coordinate system BTRK1 to each unmeasured LED 50b, 50c. These position vectors and Stored in memory in the navigation computer 26 for access by the positioning engine 100 for performing the following calculations:

[0095]

[0096] Additionally, in step 212 , these relative velocities calculated in the bone tracker coordinate system BTRK1 are transformed into the localizer coordinate system LCLZ using the transformation matrix determined in step 202 . The relative velocities in the localizer coordinate system LCLZ are used for the calculations in step 214 .

[0097] In step 214, the positioning engine 100 first calculates the velocity vector of the LED 50a based on the velocity vector measured at time t1. Calculate the velocity vector of the origin of the skeleton tracker coordinate system BTRK1 in the locator coordinate system LCLZ The velocity vector of LED 50a at time t1 is represented by the position vector relative to the origin of LED 50a: The relative velocity vector with respect to the origin at time t1 Add to calculate the velocity vector Therefore, the velocity vector of the origin at time t1 is calculated as follows:

[0098]

[0099] The velocity vector of the remaining unmeasured LEDs in the localizer coordinate system LCLZ at time t1 can now be calculated by the localization engine 100, based on the velocity vector of the origin of the skeleton tracker coordinate system BTRK1 in the localizer coordinate system LCLZ at time t1 And based on their respective relative velocity vectors at time t1 represented by the position vector relative to the remaining unmeasured LED to the origin of the skeleton tracker coordinate system BTRK1. These velocity vectors at time t1 are calculated as follows:

[0100]

[0101] In step 216, the positioning engine 100 calculates the motion (i.e., the change in position (in Cartesian coordinates) Δx) of each of the unmeasured LEDs 50b, 50c from time t0 to time t1 based on the calculated velocity vectors of the LEDs 50b, 50c and the change in time. In some embodiments, the change in time Δt measured for each LED is two milliseconds or less, and in some embodiments one millisecond or less.

[0102]

[0103] The changes in these calculated positions (x, y, z) can then be added to the previously determined positions of each of the LEDs 50b, 50c in the localizer coordinate system LCLZ. Thus, in step 218, the position changes can be added to the previous positions of the LEDs 50b, 50c at time t0 determined during the static snapshot. This is represented as follows:

[0104] x(LED50b) t1 =x(LED50b) t0 +Δx(LED50b)

[0105] In step 220, each of these calculated positions of LEDs 50b, 50c at time t1 is combined with the position of LED 50a determined at time t1. The newly determined positions of LEDs 50a, 50b, 50c are then matched to the model of tracker 44 using a point matching algorithm or a rigid body matching algorithm to obtain a best fit. The result of this best fit calculation (if within the defined tolerances of system 20) is the creation of a new transformation matrix by navigation processor 52 to link the skeletal tracker coordinate system BTRK1 to the locator coordinate system LCLZ.

[0106] The newly calculated positions of the unmeasured LEDs 50b, 50c are adjusted to the model using the new transformation matrix in step 222 to provide the adjusted positions. The measured position of LED 50a may also be adjusted due to the matching algorithm so that the measured position is also recalculated. These adjustments are considered updates to the LED cloud. In some embodiments, during the matching step, the measured position of LED 50a is fixed to the position of the model of LED 50a.

[0107] With the best fit transformation complete, the measured (and possibly adjusted) position of LED 50a in locator coordinate system LCLZ and the calculated (and adjusted) positions of LEDs 50b, 50c enable the coordinate transformer 102 to determine the new position and orientation of femur F based on the previously described relationship between femoral coordinate system FBONE, bone tracker coordinate system BTRK1 and locator coordinate system LCLZ.

[0108] Then, steps 206 to 222 are repeated at the next time t2 and start with the measurement of LED 50b in the locator coordinate system LCLZ, where LEDs 50a, 50c are the unmeasured LEDs. Due to this cycle, at each time t1, t2...tn, the position of each LED 50a, 50b, 50c is either measured (one LED is lit at each time) or calculated based on the measurements made by the optical sensor 40 and the gyro sensor 60 with a calculated position that is being approximated very accurately. This cycle of steps 206 to 222 can be implemented by the positioning engine 100 at a frequency of at least 100 Hz (more preferably at least 300 Hz and most preferably at least 500 Hz) to determine the new position of LED 50.

[0109] Reference Figure 5 The LED cloud may also include virtual LEDs, which are predetermined points identified on the model but do not actually correspond to physical LEDs on the tracker 44. The positions of these points may also be calculated at times t1, t2, ... tn. Figure 4 , these virtual LEDs can be calculated in the same manner as the unmeasured LEDs. The only difference is that since the virtual LEDs do not correspond to any light source, the virtual LEDs are never illuminated or included in the sequence of optical measurements, but are essentially just virtual. Steps 300-322 show the steps for tracking the trackers 44, 46, 48 using real and virtual LEDs. Steps 300-322 generally correspond to steps 200-222, except that the virtual LEDs are added, and the virtual LEDs are treated similarly to the unmeasured LEDs using the same equations described above.

[0110] One purpose of using virtual LEDs other than LEDs 50a, 50b, 50c is, for example, to reduce the effects of errors in the velocity calculations described above. Although these errors may have little effect on the calculated positions of LEDs 50a, 50b, 50c, the farther the point of interest is from LEDs 50a, 50b, 50c, the more likely these errors are to be amplified. For example, when tracking femur F with tracker 44, LEDs 50a, 50b, 50c incorporated into tracker 44 may experience a slight error of about 0.2 millimeters in their calculated positions. However, consider the surface of femur F, which may be located more than 10 centimeters from LEDs 50a, 50b, 50c. A slight error of 0.2 millimeters at LEDs 50a, 50b, 50c may cause an error of 0.4 to 2 millimeters on the surface of femur F. The farther femur F is from LEDs 50a, 50b, 50c, the more the error increases. As described below, at Figure 5 The use of virtual LEDs in the steps can mitigate the potential amplification of this error.

[0111] Reference Figure 5A , one virtual LED 50d may be placed on the surface of the femur F. Other virtual LEDs 50e, 50f, 50g, 50h, 50i, 50j, 50k may be placed anywhere in the bone tracker coordinate system BTRK1, such as along each of the x-axis, y-axis, and z-axis, and on either side of the origin along these axes to produce 6 virtual LEDs. These virtual LEDs are included as Figure 5A 4 and are used in steps 302 and 320. In some embodiments, only virtual LEDs 50e-50k are used. In other embodiments, virtual LEDs may be placed at locations along each of the x-axis, y-axis, and z-axis (but at different distances from the origin of the skeletal tracker coordinate system BTRK1). In further embodiments, some or all of the virtual LEDs may be located outside of the axes x, y, and z.

[0112] Now there are real LEDs 50a, 50b, 50c and virtual LEDs 50d-50k in the model. At each time t1, t2...tn, the extended model is matched in step 320 with the measured / calculated positions of the real LEDs 50a, 50b, 50c and the calculated positions of the virtual LEDs 50d-50k to obtain a transformation matrix that links the bone tracker coordinate system BTRK1 with the locator coordinate system LCLZ. Now, with the virtual LEDs 50d-50k included in the model (located at a position away from the real LEDs 50a, 50b, 50c), the error of the rotation matrix can be reduced. In essence, the rigid body matching algorithm or the point matching algorithm has additional points for matching and some of these additional points are located radially outside the points defining the real LEDs 50a, 50b, 50c, thereby rotationally stabilizing the matching.

[0113] exist Figure 5 In another variation of the process, the positions of the virtual LEDs 50e-50k can be dynamically changed during use based on the movement of the tracker 44. The slower the tracker 44 moves, the more accurate the calculated positions of the unmeasured real LEDs 50b, 50c and the virtual LEDs 50e-50k at time t1. Therefore, the positions of the virtual LEDs 50e-50k along the x-axis, y-axis, and z-axis relative to the origin of the skeletal tracker coordinate system BTRK1 can be adjusted based on the speed of the tracker 44. Therefore, if the positions of the virtual LEDs 50e-50k are represented as (s, 0, 0), (-s, 0, 0), (0, s, 0), (0, -s, 0), (0, 0, s), (0, 0, -s), then s will increase when the tracker 44 moves slowly and s will decrease to a smaller value when the tracker 44 moves faster. This can be handled by an empirical formula for s or s can be adjusted based on an estimate of the speed and the calculated position.

[0114] The determination of the new position of the LED 50 (real and virtual) may be performed at a frequency of at least 100 Hz, more preferably at least 300 Hz and most preferably at least 500 Hz.

[0115] Data from the accelerometer 70 can be used in situations where optical measurement of the LED 50 is obstructed due to line of sight interference. When the LED to be measured is blocked, the positioning engine 100 assumes a constant velocity of the origin to estimate the position. However, the constant velocity assumption in this case may be inaccurate and may lead to errors. The accelerometer 70 mainly monitors whether the constant velocity assumption is accurate during this period of time. Figure 6 and 7 The steps shown illustrate how to check this assumption.

[0116] Continuing with tracker 44 as an example, Figure 6 Steps 400-422 are usually the same as from Figure 5 300-322 of . However, in step 424, system 20 determines whether less than 3 LEDs were measured in the last cycle of measurements (meaning that one or more of the LEDs could not be measured in this cycle). This may be caused by line of sight problems, etc. The cycle for tracker 44 is the last three attempted measurements. If each of LEDs 50a, 50b, 50c was visible and could be measured during the last three measurements, then system 20 proceeds to step 408 and proceeds as previously described with reference to FIG. Figure 5 Continuation of description.

[0117] If the system 20 determines that one or more of the LEDs 50a, 50b, 50c cannot be measured during this cycle (i.e., blocked from measurement), the algorithm still moves to step 408, but if the new LED to be measured in step 406 is an LED that cannot be measured, the system makes some speed assumptions as described below.

[0118] When an LED (such as LED 50a) is not seen by optical sensor 40 at its measurement time tn in step 406, the previously calculated velocity vector of the origin of tracker 44 in localizer coordinate system LCLZ at the previous time t(n-1) is assumed to remain constant. Therefore, the velocity vector in the localizer coordinate system LCLZ can be calculated based on the previously calculated velocity vector The velocity vector of LED 50a, 50b, 50c in the localizer coordinate system LCLZ is calculated using the relative velocity vector of LED 50a, 50b, 50c (which is derived from the newly measured angular velocity vector from gyroscope 60). The new position of LED 50a, 50b, 50c can then be determined using the equations described in steps 316-322.

[0119] Initially, when LED 50a is to be measured at step 406, but is obstructed, the velocity vector of the origin is assumed to be the same as previously calculated. Therefore, no velocity of the new LED is calculated at step 408:

[0120]

[0121] Step 410 is performed in the same manner as step 310 .

[0122] Then, the relative velocity vector of LEDs 50a, 50b, 50c calculated in step 412 is Based on the newly measured angular velocity vector from the gyroscope 60 and the previous velocity vector in the skeleton tracker coordinate system BTRK1

[0123]

[0124] In step 414, the relative velocity vector of LEDs 50a, 50b, 50c may be used and the origin velocity vector Calculate the velocity vector in the locator coordinate system LCLZ:

[0125]

[0126] Steps 416 to 422 are performed the same as steps 316 - 322 .

[0127] If the system 20 determines at step 424 that one or more of the LEDs 50a, 50b, 50c cannot be measured (i.e., blocked from measurement) during the cycle, then Figure 7 Another algorithm is implemented simultaneously at steps 500-506 shown until a full cycle of measurements is made in which all LEDs 50a, 50b, 50c are visible to optical sensor 40. Thus, system 20 is considered to be in a "blocked" state until a full cycle with all visible measurements is made.

[0128] Steps 500 - 506 are performed continuously while the system 20 is in the blocking state.

[0129] In step 500, the navigation processor 52 starts a clock that tracks how long the system 20 has been in the blocked state. The time of the blocked state is referred to below as t(blocked).

[0130] In step 502, the accelerometer 70 measures acceleration along the x, y and z axes of the skeleton tracker coordinate system BTRK1 to track errors in the constant velocity assumption. Like the gyroscope readings, the accelerometer readings are transformed from the accelerometer coordinate system to the skeleton tracker coordinate system BTRK1.

[0131] If the accelerometer 70 detects one or more accelerations that exceed one or more predetermined acceleration tolerances, the navigation computer 26 will place the system into an error state. The acceleration tolerances may be defined differently along each x-axis, y-axis, and z-axis, or the same along each axis. If the measured acceleration exceeds the tolerance, the constant velocity assumption is unreliable and cannot be used for the specific application of surgical navigation. Different tolerances may be used for different applications. For example, during robotic cutting, the tolerance may be very low, but for visual navigation only (i.e., no feedback for the cutting control loop), the tolerance may be set higher.

[0132] In step 504, the speed error associated with the position of the LED 50 relative to the optical sensor 40 is taken into account and monitored during the blocking state. For each of the LEDs 50, the time t of the blocking state is 阻挡状态 Multiply by the velocity error v 误差 must be less than the position error tolerance γ and therefore the following equation must be satisfied to prevent the system 20 from being placed into an error state:

[0133] vError×t 阻挡状态 <γ

[0134] In this equation, the speed error v is calculated for each of the LEDs 50a, 50b, 50c. 误差 as follows:

[0135]

[0136] Position error x 误差(t) and x 误差(t％1) is a predefined position error in the system 20 based on the position at time t and t-1 relative to the optical sensor 40. Essentially, the farther the LEDs 50a, 50b, 50c are from the optical sensor 40, the higher the potential position error. These position errors are experimentally or theoretically derived and put into a lookup table or formula such that at each position of the LEDs 50a, 50b, 50c in Cartesian coordinates (x, y, z) an associated position error is provided.

[0137] In step 504, the positioning engine 100 accesses the lookup table or calculates the formula to determine the position error for each of the LEDs 50a, 50b, 50c at the current time t and the previous time t-1. Therefore, the position error is based on the position in Cartesian coordinates in the locator coordinate system LCLZ calculated by the system 20 in step 422 at the current time t and the previous time t-1. The time variable Δt represents the time taken for the subsequent position calculation, so for illustrative purposes, the difference between t and t-1 can be 1 millisecond.

[0138] The position error tolerance γ is predefined in the navigation computer 26 for access by the positioning engine 100. The position error tolerance γ can be expressed in millimeters. The position error tolerance γ can be in the range from 0.001 mm to 1 mm and is specifically set to 0.5 mm in some embodiments. Therefore, if the position error tolerance γ is set to 0.5 mm, the following formula must be satisfied:

[0139] v 误差 ×t 阻挡状态 <0.5 mm

[0140] As can be seen, the longer the system 20 is in the blocked state, the greater the role of the time variable in the equation and therefore the smaller the velocity error that will be tolerated. In some embodiments, the equation is calculated by the positioning engine 100 for each of the LEDs 50a, 50b, 50c individually in step 504. In other embodiments, due to how closely the LEDs 50a, 50b, 50c are arranged on the tracker 44, the velocity error of only one of the LEDs 50a, 50b, 50c is used in the calculation to determine compliance.

[0141] In step 506, when one or more errors exceed the position error tolerance γ, the system 20 is placed in an error state. In this state, for example, any control or movement of the cutting or ablation tool is stopped and the tool is turned off.

[0142] IV. Other Embodiments

[0143] In one embodiment, when each of the trackers 44, 46, 48 is being actively tracked, the lighting of the LEDs occurs such that one LED from the tracker 44 is lit, then one LED from the tracker 46 is lit, then one LED from the tracker 48 is lit, then a second LED from the tracker 44 is lit, then a second LED from the tracker 46 is lit, and so on until all LEDs are lit, and the sequence is then repeated. This lighting sequence may occur via a command signal sent from a transceiver (not shown) on the camera unit 36 ​​to a transceiver (not shown) on the trackers 44, 46, 48.

[0144] The navigation system 20 can be used in a closed loop manner to control surgical procedures performed by surgical cutting tools. Both the instrument 22 and the anatomical structure being cut are equipped with a tracker 50 so that the navigation system 20 can track the position and orientation of the instrument 22 and the anatomical structure being cut (such as bone).

[0145] In one embodiment, the navigation system is part of a robotic surgical system for treating tissue. In some versions, the robotic surgical system is a robotic surgical cutting system for removing material from a patient's anatomical structure, such as bone or soft tissue. The cutting system can be used to prepare bones for surgical implants, such as hip and knee implants, including single-chamber knee implants, dual-chamber knee implants, or total knee implants. Some of these types of implants are shown in U.S. Patent Application No. 13 / 530,927, entitled "Prosthetic Implant and Method of Implantation," the disclosure of which is incorporated herein by reference.

[0146] A robotic surgical cutting system includes a manipulator (see, e.g., Figure 1 ). The manipulator has a plurality of arms and a cutting tool carried by at least one of the plurality of arms. The robotic control system controls or constrains the motion of the cutting tool in at least five degrees of freedom. Examples of such manipulators and control systems are shown in U.S. Provisional Patent Application No. 61 / 679,258, entitled “Surgical Manipulator Capable of Controlling aSurgical Instrument in either a Semi-Autonomous Mode or a Manual, Boundary Constrained Mode,” which is incorporated herein by reference, and in U.S. Patent Application No. 13 / 958,834, also entitled “Navigation System for use with aSurgical Manipulator Operable in Manual or Semi-Autonomous Mode,” which is incorporated herein by reference.

[0147] In this embodiment, the navigation system 20 communicates with a robotic control system (which may include a manipulator controller 54). The navigation system 20 transmits position and / or orientation data to the robotic control system. The position and / or orientation data represents the position and / or orientation of the instrument 22 relative to the anatomical structure. This communication provides closed-loop control to control the cutting of the anatomical structure so that the cutting occurs within predefined boundaries.

[0148] In this embodiment, the manipulator motion may be consistent with the LED measurements so that for each LED measurement obtained, there is a corresponding motion of the instrument 22 through the manipulator 56. However, this may not always be the case. For example, there may be such a lag between the last LED measurement and the motion through the manipulator 56 that the position and / or orientation data sent from the navigation computer 26 to the manipulator 56 becomes unreliable for controlling the loop motion. In this case, the navigation computer 26 may be configured to also transmit kinematic data to the manipulator controller 54. Such kinematic data includes previously determined linear and angular velocities for the trackers 44, 46, 48. Since the velocity is already known, the position can be calculated based on the lag time. Then, in order to control the motion of the manipulator 56, the manipulator controller 54 can calculate the position and orientation of the trackers 44, 46, 48, and therefore calculate the relative position and orientation of the instrument 22 (or the instrument tip) relative to the femur F and / or tibia T.

[0149] In this embodiment, the instrument 22 is composed of Figure 1The manipulator shown or other robotic grasping that provides some form of mechanical constraint to motion. This constraint limits the motion of the instrument 22 within predefined boundaries. If the instrument 22 deviates outside the predefined boundaries, control is sent to the instrument 22 to stop cutting.

[0150] When both the instrument 22 and the anatomical structure being cut are tracked in real time in these systems, the need to rigidly fix the anatomical structure in place can be eliminated. Since both the instrument 22 and the anatomical structure are tracked, control of the instrument 22 can be adjusted based on the relative position and / or orientation of the instrument 22 relative to the anatomical structure. In addition, the representation of the instrument 22 and the anatomical structure on the display can be moved relative to each other to simulate their real-world motion.

[0151] In one embodiment, each of the femur F and the tibia T has a target volume of material to be removed by the working end of the surgical instrument 22. The target volume is defined by one or more boundaries. The boundaries define the surface of the bone that should remain after the procedure. In some embodiments, as disclosed in provisional patent application No. 61 / 679,258, entitled "Surgical Manipulator Capable of Controlling aSurgical Instrument in either a Semi-Autonomous Mode or a Manual, Boundary Constrained Mode" (which is incorporated herein by reference), the system 20 tracks and controls the surgical instrument 22 to ensure that the working end (e.g., a drill) removes only the target volume of material and does not extend beyond the boundaries,

[0152] In the described embodiment, control of the instrument 22 is accomplished by utilizing data generated by the coordinate transformer 102, which indicates the position and orientation of the drill or other cutting tool relative to the target volume. By knowing these relative positions, the surgical instrument 22 or the manipulator to which the surgical instrument is mounted can be controlled so that only the desired material is removed.

[0153] In other systems, the instrument 22 has a cutting tool that is movable in three degrees of freedom relative to the handheld housing and is manually positioned by the surgeon's hand without the assistance of a cutting boom, guide arm, or other restraining mechanism. Such a system is shown in U.S. Provisional Patent Application No. 61 / 662,070, entitled "Surgical Instrument Including Housing, a Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing," the disclosure of which is incorporated herein by reference.

[0154] In these embodiments, the system includes a handheld surgical cutting instrument with a cutting tool. As shown in U.S. Provisional Patent Application No. 61 / 662,070 entitled "Surgical Instrument Including Housing, a Cutting Accessory that Extends from the Housing and Actuators that Establish the Position of the Cutting Accessory Relative to the Housing", the control system uses an internal actuator / motor to control the movement of the cutting tool in at least 3 degrees of freedom. The navigation system 20 communicates with the control system. A tracker (such as tracker 48) is mounted to the instrument. Other trackers (such as trackers 44, 46) are mounted to the patient's anatomical structure.

[0155] In this embodiment, the navigation system 20 communicates with the control system of the handheld surgical cutting instrument. The navigation system 20 transmits position and / or orientation data to the control system. The position and / or orientation data represents the position and / or orientation of the instrument 22 relative to the anatomical structure. This communication provides closed-loop control to control the cutting of the anatomical structure so that the cutting occurs within a predefined boundary (the term predefined boundary is understood to include a predefined trajectory, volume, line, other shape or geometry, etc.).

[0156] Features of the present invention can be used to track sudden or unexpected movement of the localizer coordinate system LCLZ, such as may occur when the camera unit 36 ​​is bumped by the surgical staff. An accelerometer (not shown) mounted to the camera unit 36 ​​monitors for bumps and stops the system 20 if a bump is detected. In this embodiment, the accelerometer communicates with the camera controller 42 and if the measured acceleration along any of the x-axis, y-axis or z-axis exceeds a predetermined value, the camera controller 42 sends a corresponding signal to the navigation computer 26 to disable the system 20 and wait for the camera unit 36 ​​to stabilize and restart the measurement. In some cases, the initialization steps 200, 300, 400 will have to be repeated before resuming navigation.

[0157] In some embodiments, a virtual LED is positioned at the working tip of instrument 22. In this embodiment, the virtual LED is located at the position of the working tip in the model of instrument tracker 48 so that the working tip position is continuously calculated.

[0158] An object of the contemplated claims is to cover all such modifications and variations within the true spirit and scope of the invention. In addition, although the embodiments described above are related to medical applications, the invention described herein may also be applicable to other applications (such as industry, aerospace, protection, etc.).

Claims

1. A surgical system comprising: Robotic manipulators; an instrument coupled to a robotic manipulator, wherein the robotic manipulator is configured to control movement of the instrument to facilitate a surgical procedure; a non-optical sensor coupled to the instrument; as well as A navigation system configured to obtain readings from a non-optical sensor, and the navigation system includes a camera unit configured to optically track a pose of the instrument, wherein the navigation system is configured to: detecting a state in which a camera unit is blocked from optically tracking the posture of the instrument; and In response to detecting the state, the posture of the instrument is tracked using readings from the non-optical sensor.

2. The surgical system of claim 1 , wherein the navigation system is configured to: detecting that the camera unit is no longer blocked from optically tracking the pose of the instrument; and In response, the position of the instrument is optically tracked using a camera unit.

3. The surgical system of claim 1, wherein: In response to detecting the condition, the navigation system is configured to estimate a pose of the instrument using readings from the non-optical sensor.

4. The surgical system of claim 1, wherein: In response to detecting the state, the navigation system tracks the pose of the instrument by being further configured to: obtaining a previous pose of the instrument, the previous pose being measured based on optically tracking the pose of the instrument by a camera unit before the state is detected; as well as Readings from the non-optical sensor are used to calculate a current pose of the instrument relative to the previous pose.

5. The surgical system of claim 1, wherein the non-optical sensor comprises an accelerometer.

6. The surgical system of claim 1, wherein the non-optical sensor comprises a gyroscope.

7. The surgical system of claim 1, wherein the robotic manipulator is configured to utilize the instrument to prepare the patient's anatomy to receive an implant.

8. The surgical system of claim 1, wherein the robotic manipulator comprises a robotic arm supported by a movable cart.

9. The surgical system of claim 1, wherein: In response to detecting the state, the navigation system is configured to transmit the tracked pose of the instrument to a control system coupled to a robotic manipulator to enable the robotic manipulator to control or constrain the movement of the instrument relative to a predefined path or boundary.

10. A navigation system for use with a robotic manipulator configured to control movement of an instrument to facilitate a surgical procedure, the navigation system comprising: a camera unit configured to optically track the pose of the instrument; a non-optical sensor coupled to the instrument; as well as a computing system coupled to the camera unit and configured to obtain readings from the non-optical sensor, and the computing system is configured to: detecting a state in which a camera unit is blocked from optically tracking the posture of the instrument; and In response to detecting the state, the pose of the instrument is tracked using readings from the non-optical sensor.

11. The navigation system of claim 10, wherein the computing system is configured to: detecting that the camera unit is no longer blocked from optically tracking the pose of the instrument; and In response, the position of the instrument is optically tracked using a camera unit.

12. The navigation system according to claim 10, wherein: In response to detecting the condition, the computing system is configured to estimate a pose of the instrument using readings from the non-optical sensor.

13. The navigation system according to claim 10, wherein: In response to detecting the state, the computing system tracks the pose of the instrument by being further configured to: obtaining a previous pose of the instrument, the previous pose being measured based on optically tracking the pose of the instrument by a camera unit before the state is detected; as well as Readings from the non-optical sensor are used to calculate a current pose of the instrument relative to the previous pose.

14. The navigation system of claim 10, wherein the non-optical sensor comprises an accelerometer.

15. The navigation system of claim 10, wherein the non-optical sensor comprises a gyroscope.

16. The navigation system according to claim 10, wherein: In response to detecting the state, the computing system is configured to transmit the tracked pose of the instrument to a control system coupled to a robotic manipulator to enable the robotic manipulator to control or constrain movement of the instrument relative to a predefined path or boundary.

17. A computer-implemented method of operating a navigation system comprising a camera unit, a non-optical sensor, and a computing system, the non-optical sensor being coupled to an instrument controlled by a robotic manipulator, the method comprising: optically tracking the position of the instrument using a camera unit; detecting, using a computing system, a state in which a camera unit is blocked from optically tracking a posture of the instrument; Using a computing system to obtain readings from non-optical sensors; as well as In response to detecting the state, the computing system tracks the pose of the instrument using readings from the non-optical sensor.

18. The computer-implemented method of claim 17, comprising: detecting, using a computing system, that the camera unit is no longer blocked from optically tracking the pose of the instrument; as well as In response, the position of the instrument is optically tracked using a camera unit.

19. The computer-implemented method of claim 17, comprising: In response to detecting the state, the computing system estimates a pose of the instrument using readings from the non-optical sensor.

20. The computer-implemented method of claim 17, comprising, in response to detecting the state, the computing system tracking the pose of the instrument by further performing: obtaining a previous pose of the instrument, the previous pose being measured based on optically tracking the pose of the instrument by a camera unit before the state is detected; and Readings from the non-optical sensor are used to calculate a current pose of the instrument relative to the previous pose.

21. A surgical system comprising: a robotic device configured to support a cutting tool and to move the cutting tool in multiple degrees of freedom; a control system configured to command the robotic device to control or constrain movement of the cutting tool; a first tracker configured to be coupled to the robotic device; a second tracker configured to couple to the anatomical structure, and comprising an optical marker configured to generate an optical signal and a non-optical sensor configured to generate a non-optical signal; as well as a navigation system in communication with the control system and comprising at least one optical sensor, wherein the navigation system is configured to: receiving the optical signal from one or more of the optical markers using the optical sensor; receiving the non-optical signal from the non-optical sensor; as well as Based on the received optical and non-optical signals, position data indicative of the position of the anatomical structure is transmitted to the control system to control the robotic device for cutting the anatomical structure.

22. The surgical system of claim 21, wherein the navigation system is configured to determine a position of each of the optical markers based on the received optical and non-optical signals.

23. The surgical system of claim 22, wherein the optical sensor is configured to sequentially receive the optical signals from the optical markers, and wherein the navigation system is further configured to: determining a position of a first optical marker of the optical markers at a first time based on a first optical signal from the first optical marker; determining positions of a second optical marker and a third optical marker among the optical markers at the first time based on the non-optical signal from the non-optical sensor and the first optical signal; as well as The determined positions of the first optical marker, the optical second marker, and the third optical marker are correlated to track the position of the anatomical structure.

24. The surgical system of claim 23, wherein the optical sensor does not receive optical signals from the second optical marker and the third optical marker at the first time.

25. The surgical system of claim 23, wherein the navigation system is further configured to determine the positions of the second optical marker and the third optical marker at the first time by calculating a first velocity of the first optical marker based on the first optical signal and by calculating velocities of the second optical marker and the third optical marker based on the first velocity and the non-optical signal.

26. The surgical system of claim 22, wherein the navigation system is further configured to: determining a position of each of the optical markers in a locator coordinate system based on the optical signal and the non-optical signal; and A matching algorithm is executed to match the determined positions of one or more of the optical markers in the locator coordinate system with the positions of one or more of the optical markers in a model of a second tracker established relative to the tracker coordinate system to obtain a transformation matrix that transforms the tracker coordinate system into the locator coordinate system.

27. The surgical system of claim 22, wherein the navigation system is further configured to: determining whether one or more of the optical markers are blocked from line of sight with the optical sensor to generate a blocked state, Tracks the time of blocked state, determining a position error tolerance associated with the one or more blocked markers, determining a position error associated with the one or more blocked markers based on the time of the blocked state, Determine if the position error is within the position error tolerance, and If the position error exceeds the position error tolerance the navigation system is placed in an error state.

28. The surgical system of claim 21, wherein the navigation system transmits position data indicative of the position of the anatomical structure to the control system to further control cutting of the anatomical structure relative to a boundary that is predefined relative to the anatomical structure.

29. The surgical system of claim 28, wherein the cutting tool has a working end, and wherein the navigation system is configured to: monitoring whether the working end of the cutting tool removes a target volume of material defined by the boundary; or A determination is made as to whether the working end of the cutting tool has exceeded the boundary, and in response, the control system is instructed to cause the cutting tool to cease cutting.

30. The surgical system of claim 21, wherein the navigation system transmits position data indicative of the position of the anatomical structure to the control system to further control the cutting of the anatomical structure relative to a tool path defined relative to the anatomical structure.

31. The surgical system of claim 21, wherein the robotic device is further defined as a robotic manipulator comprising one or more arms and comprising a plurality of actuators for moving the one or more arms to move the cutting tool in at least five degrees of freedom.

32. The surgical system of claim 21, wherein the robotic device is further defined as a robotic instrument comprising a handheld housing manually supported and positioned by a surgeon's hands, and a plurality of actuators configured to move the cutting tool relative to the handheld housing.

33. A navigation system for a robotic device, the robotic device being configured to support a cutting tool and to move the cutting tool in multiple degrees of freedom, the navigation system being in communication with a control system, the control system being configured to command the robotic device to control or constrain the movement of the cutting tool, and the navigation system comprising: at least one optical sensor; a first tracker configured to be coupled to the robotic device; a second tracker configured to couple to the anatomical structure and comprising an optical marker configured to generate an optical signal and a non-optical sensor configured to generate a non-optical signal; and A computing system comprising at least one processor, wherein the at least one processor is configured to: receiving the optical signal from one or more of the optical markers using the optical sensor; receiving the non-optical signal from the non-optical sensor; as well as Based on the received optical and non-optical signals, position data indicative of the position of the anatomical structure is transmitted to the control system to enable the control system to control the robotic device for cutting the anatomical structure.

34. A surgical navigation system, comprising: a tracker configured to be coupled to a surgical object, and the tracker includes an optical marker and a non-optical sensor; an optical sensor configured to detect the optical marker; as well as A computing system, comprising at least one processor, configured to: detecting the optical marker using an optical sensor to determine the position of the tracker; determining, using an optical sensor, that at least one of the optical markers can no longer be detected; as well as In response to at least one of the optical markers no longer being detected, determining a position of the tracker based on detection by the non-optical sensor.

Images