Hybrid optical-inertial skeleton tracking

Abstract

A first tracker assembly and a second tracker assembly, each having a single optical marker, are fastened to the skeleton, wherein the first tracker assembly also includes a motion sensor. The locator optically tracks the position of the optical markers and determines the orientation of the tracker coordinate system (TCS) associated with the tracker assembly according to three positional DOFs and two rotational DOFs based on the tracked marker positions. A tilt angle relative to gravity is determined using measurements from the motion sensors, and the orientation of the TCS according to an additional rotational DOF defined by a virtual line extending between the optical markers is determined based on the tilt angle. The skeleton's orientation according to six DOFs is then determined based on the orientation determined by the TCS according to the three positional DOFs and two rotational DOFs, and the orientation of the TCS according to the additional rotational DOFs.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and all benefits to U.S. Provisional Patent Application No. 63 / 611,298, filed December 18, 2023, the entire contents of which are hereby incorporated by reference. Background Technology

[0003] Surgical systems typically employ navigation tracking to detect the posture of the bones undergoing surgery. Depending on the surgery, the tracked bone may be, for example, the knee, hip, shoulder, or part of the spine. The human spine is characterized by specific curvatures that allow the spine to absorb shock, but these curvatures can be disrupted by degenerative changes or spinal deformities. Minimally invasive intervertebral fusion is a well-established treatment option for degenerative spinal diseases and typically involves inserting an implant (spacer, graft, fusion cage) into the intervertebral disc. Intervertebral movement during this procedure, if not considered, can affect the outcome. Computer-assisted surgery with surgical navigation offers the advantage of tracking intraoperative movement of the patient's anatomy in six degrees of freedom (DOF) by working in conjunction with a tracker attached to the bone. However, attaching a standard 6 DOF tracker to individual vertebrae is challenging, especially in a minimally invasive context, due to the relatively small size and close proximity of the vertebrae, particularly in the cephalocaudal direction. Therefore, implant placement and vertebral adjustment within the spinal curvature are often achieved based on the surgeon's experience and skill without surgical navigation tracking. Summary of the Invention

[0004] The present invention presents a series of concepts in a simplified form, which are further described in the detailed embodiments below. The present invention is not intended to limit the scope of the claimed subject matter, nor is it intended to identify key or essential features of the claimed subject matter.

[0005] The first aspect includes a hybrid tracker for tracking an object of interest, such as a patient's skeleton, instrument, or robotic manipulator, in six degrees of freedom during a medical procedure. The hybrid tracker includes two optical markers configured to be coupled to the object and detectable by a locator. The tracker also includes a motion sensor capable of moving with at least one of the two optical markers and configured to generate a measurement indicating a tilt angle relative to gravity.

[0006] The second aspect includes a method for tracking an object of interest, such as a patient's skeleton, instrument, or robotic manipulator, in six degrees of freedom, such as using a tracker of the first aspect, during a medical procedure. The method includes: optically tracking the position of an optical marker in a known coordinate system, such as the coordinate system of the tracker, using a locator. The method further includes: determining the tracker, or more specifically, the orientation of the tracker coordinate system associated with the tracker, in the known coordinate system based on the tracked position of the optical marker, such as according to three positional degrees of freedom and two rotational degrees of freedom, by one or more controllers. The method further includes: determining a tilt angle relative to gravity by the one or more controllers using measurements from the motion sensor. The method further includes: determining the tracker, or more specifically, the orientation of the tracker coordinate system, based on the tilt angle, according to additional rotational degrees of freedom, such as defined by a virtual line extending between the optical markers. The method further includes: determining the pose of the object in the known coordinate system by the one or more controllers based on the pose of the tracker, or more specifically the tracker coordinate system, according to six degrees of freedom, wherein the pose of the tracker, or more specifically the tracker coordinate system, is determined based on the tracked position of the optical marker and the orientation of the tracker, or more specifically the tracker coordinate system, based on the tilt angle.

[0007] Other aspects include corresponding computer systems, devices, and computer programs stored on one or more computer storage devices, each configured to perform the actions described above. A system of one or more computers may be configured to perform specific operations or actions by having software, firmware, hardware, or combinations thereof installed on the system, which, in operation, causes the system to perform the actions. One or more computer programs may be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the actions.

[0008] The third aspect includes a surgical navigation system for tracking an object of interest, such as a patient's skeleton, instrument, or robotic manipulator, in six degrees of freedom during a medical procedure, such as using a tracker of the first aspect. The system includes a locator configured to optically track the position of an optical marker of the tracker in a known coordinate system, such as the locator's coordinate system. The system also includes one or more controllers configured to: determine the orientation of the tracker, or more specifically, the tracker coordinate system associated with the tracker, in the known coordinate system based on the tracked position of the optical marker, such as according to three positional degrees of freedom and two rotational degrees of freedom; determine a tilt angle relative to gravity using measurements from the motion sensors; and determine the orientation of the tracker, or more specifically, the tracker coordinate system, based on the tilt angle and according to additional rotational degrees of freedom, such as defined by a virtual line extending between the optical markers. The one or more controllers are further configured to: determine the pose of the object in the known coordinate system based on the pose of the tracker, or more specifically the tracker coordinate system, according to six degrees of freedom, the pose of the tracker, or more specifically the tracker coordinate system, based on the tracked position of the optical marker and the orientation of the tracker, or more specifically the tracker coordinate system, based on the tilt angle.

[0009] The fourth aspect includes a surgical system comprising a robotic manipulator and the surgical system of the third aspect. The one or more controllers are configured to control the movement of the robotic manipulator relative to the object during the medical procedure based on the determined posture of the object in six degrees of freedom.

[0010] The fifth aspect includes a surgical navigation system comprising: a tracker assembly rigidly attached to a bone and including at least one optical marker; a motion sensor rigidly attached to the bone and configured to measure a tilt angle relative to gravity; a locator configured to optically detect the posture of the at least one optical marker; and one or more controllers coupled to the locator and the motion sensor, the controllers being configured to: receive the measured tilt angle; establish a baseline relationship between the at least one optical marker and the motion sensor by combining the detected posture of the at least one optical marker with the measured tilt angle; monitor the measurement relationship between the detected posture of the at least one optical marker and the measured tilt angle; and detect a deviation between the measurement relationship and the baseline relationship to indicate that the at least one optical marker has moved relative to the bone.

[0011] The sixth aspect includes a method for operating the surgical navigation system of the fifth aspect.

[0012] The seventh aspect includes a surgical navigation system comprising: a tracker assembly rigidly attached to a bone and including: at least one optical marker and a tracker motion sensor configured to measure a first tilt angle relative to gravity; a tracker observation sensor rigidly attached to the bone and configured to measure a second tilt angle relative to gravity; a locator configured to optically detect the posture of the at least one optical marker; and one or more controllers coupled to the locator, the tracker motion sensor, and the tracker observation sensor and configured to: receive the measured first tilt angle and the second tilt angle; establish a baseline relationship between the tracker assembly and the tracker observation sensor by combining the measured first tilt angle and the second tilt angle; track the measured first tilt angle and the second tilt angle to monitor the measurement relationship between the tracker assembly and the tracker observation sensor; and detect a deviation between the measurement relationship and the baseline relationship to indicate that the at least one optical marker has moved relative to the bone.

[0013] The eighth aspect includes a method for operating the surgical navigation system of the seventh aspect.

[0014] The ninth aspect includes a tracker observation device configured to monitor a tracker assembly mounted to a skeleton at a first location, the tracker observation device comprising: a body; an attachment coupled to the body and configured to be mounted to the skeleton at a second location spaced apart from and different from the first location; a motion sensor coupled to the body and configured to measure a tilt angle relative to gravity; and a communication device coupled to the body and configured to remotely transmit the measured tilt angle.

[0015] Any of the above aspects may be combined in whole or in part. The above aspects may also be combined in whole or in part with any one or more of the following implementations:

[0016] In some implementations, the tracker includes only two optical markers. In some implementations, the tracker includes a first tracker component having one of the optical markers and a second tracker component having the other of the optical markers. According to previous implementations, each tracker component may therefore have only one optical marker. The first tracker component may be coupled to the object at a first location, and the second tracker component may be coupled to the object at a second location different from the first location. The first tracker component also includes the motion sensor.

[0017] In some implementations, the object being tracked is a skeleton, and the first and second tracker components are removably coupled to a first and a second pedicle screw inserted into the skeleton, respectively. The skeleton may more commonly be a vertebra, femur, tibia, scapula, humerus, pelvis, skull, etc.

[0018] In some implementations, the motion sensor is configured to measure movement relative to an axis substantially perpendicular to a virtual line extending between the optical markers of the tracker. Therefore, the tilt angle relative to gravity used to determine the tracker's orientation based on the additional rotational degrees of freedom can be defined as the tilt angle of such an axis relative to gravity.

[0019] In some implementations, the motion sensor is defined as an accelerometer. In some implementations, the tracker that includes the first tracker component and the second tracker component does not include a magnetometer, and / or does not include a gyroscope.

[0020] In some implementations, the initial attitude of the tracker, or more specifically the tracker coordinate system, is determined in the known coordinate system according to six degrees of freedom. In some implementations, an initial tilt angle relative to gravity is obtained using measurements from the motion sensor corresponding to the initial attitude. In some implementations, the attitude of the tracker, or more specifically the tracker coordinate system, in the known coordinate system according to three positional degrees of freedom and two rotational degrees of freedom, is determined based on the tracked position of the optical marker and the initial attitude. In some implementations, the orientation of the tracker, or more specifically the tracker coordinate system, according to the additional rotational degrees of freedom, is determined based on the determined tilt angle and the initial tilt angle.

[0021] In some implementations, a first vector of the optical marker connecting the first tracker component and the second tracker component is calculated in the known coordinate system based on the initial attitude; a second vector of the optical marker connecting the first tracker component and the second tracker component is calculated in the known coordinate system based on the tracked position of the optical marker; a rotation matrix is ​​calculated based on the first vector and the second vector; and the tracker, or more specifically the attitude of the tracker coordinate system in the known coordinate system according to the three positional degrees of freedom and the two rotational degrees of freedom, is determined based on the rotation matrix.

[0022] In some implementations, a rotation matrix is ​​calculated based on the initial tilt angle and the determined tilt angle; and the tracker is determined based on the rotation matrix calculated based on the initial tilt angle and the determined tilt angle, or more specifically, the tracker coordinate system is oriented according to the orientation of the additional rotational degrees of freedom.

[0023] In some implementations, the tracker (such as at least one of the first tracker component and the second tracker component) includes features for temporary contact with an additional tracker component including at least one optical marker detectable by the locator, to determine the initial pose of the tracker based on six degrees of freedom, or more specifically, the tracker coordinate system in the known coordinate system.

[0024] In some implementations, a second motion sensor can move with the positioner. A second initial tilt angle relative to gravity is obtained using measurements from the second motion sensor corresponding to the initial posture. In some implementations, the tracker, or more specifically the tracker coordinate system, is determined in a known coordinate system based on the initial posture of the six degrees of freedom, according to the tracked position of the optical marker corresponding to the initial posture, the initial tilt angle relative to gravity indicated by the tracker's motion sensor, and the second initial tilt angle relative to gravity indicated by the second motion sensor. In some implementations, the orientation of the tracker, or more specifically the tracker coordinate system, according to the additional rotational degrees of freedom, is determined based on the determined tilt angle, the initial tilt angle, and the second initial tilt angle.

[0025] In some implementations, such as when the motion sensor measures that the axis of its movement is not substantially perpendicular to the virtual line extending between the optical markers, an angle between the axis and a plane normal to the virtual line extending between the optical markers is determined; and the tracker is determined based on the determined tilt angle and the determined angle between the axis and the plane, or more specifically, the orientation of the tracker coordinate system according to the additional rotational degrees of freedom defined around the virtual line extending between the optical markers. In some implementations, measurements from the motion sensor are monitored to determine whether the tracker assembly has moved relative to the skeleton.

[0026] The motion sensor may be coupled to the tracker assembly. The tracker assembly may include an attachment portion coupled to the bone, a rod extending from the attachment portion, and a tracking head coupled to the rod and supporting the at least one optical marker. The motion sensor may be coupled to any component of the tracker assembly. The tracker assembly may be attached to the bone at a first location. The motion sensor may be rigidly attached to the bone at a second location spaced apart from and different from the first location. A second tracker assembly may be rigidly attached to the bone at the second location. The second tracker assembly may include at least one optical marker, and the motion sensor is coupled to the second tracker assembly. The motion sensor may be coupled to a tracker observation device at the second location.

[0027] The tracker observation sensor is coupled to the tracker assembly at the first location. The tracker observation sensor may be coupled to any component of the tracker assembly. The tracker observation sensor may be coupled to a component of the tracker assembly different from the component to which the motion sensor is attached. The tracker observation sensor may be rigidly attached to the skeleton at a second location spaced apart from and different from the first location. The second tracker assembly may include at least one optical marker, and the tracker observation sensor is coupled to the second tracker assembly. The tracker observation sensor may be coupled to a tracker observation device at the second location. The one or more controllers may establish the baseline relationship between the tracker assembly and the tracker observation sensor by further combining the measured first tilt angle and second tilt angle with the tracked posture of the at least one optical marker of the tracker assembly. The one or more controllers are configured to generate feedback in response to detecting a deviation between the measured relationship and the baseline relationship. Implementations of the described techniques may include hardware, methods, or processes, or computer software on a computer-accessible medium. Attached Figure Description

[0028] Figure 1An example of a surgical system for performing spinal surgery is illustrated, which includes a robotic manipulator for guiding an end effector to manipulate spinal tissue and a navigation system for tracking vertebrae during the procedure.

[0029] Figure 2 Examples of objects that can be combined with are shown. Figure 1 Hybrid trackers in navigation systems.

[0030] Figure 3 yes Figure 2 A magnified view of the hybrid tracker.

[0031] Figure 4 Examples of what can be combined with Figure 1 Components in a surgical system.

[0032] Figure 5 Examples are shown for use Figure 2 The method of using a hybrid tracker to track objects.

[0033] Figure 6 Examples are shown for use Figure 2 Another method for tracking objects using hybrid trackers. Detailed Implementation

[0034] The following description provides exemplary implementations with reference to the accompanying drawings. The same or similar reference numerals will be used to denote the same or similar structural features.

[0035] Figure 1 A surgical system 10 is illustrated for treating the anatomical structures (i.e., target sites) of patient P. The surgical system 10 may be located in a surgical environment (such as an operating room in a medical facility). The surgical system 10 may include a surgical navigation system 12 and a robotic manipulator 14.

[0036] The robotic manipulator 14 may be coupled to the surgical instrument 16 and configured to manipulate the surgical instrument 16 to treat a target volume of patient tissue, such as under the guidance of a surgeon and / or a surgical navigation system 12. For example, the surgical navigation system 12 may enable the robotic manipulator 14 to manipulate the surgical instrument 16 to manipulate and / or remove a target volume of patient tissue while avoiding other objects adjacent to the target volume in the surgical workspace, such as other medical instruments and adjacent anatomical structures to be avoided. Alternatively or additionally, the surgeon may manually grasp and manipulate the surgical instrument 16 while receiving guidance from the surgical navigation system 12 and / or from the robotic manipulator 14. For example, the robotic manipulator 14 may be configured to provide tactile feedback and / or constrain the movement of the surgical instrument 16 relative to the target volume to avoid adjacent objects.

[0037] Surgical instrument 16 may be configured to manipulate and / or remove tissue at a target site, and / or insert an implant into tissue at a target site, which may include soft and / or hard tissue, such as bone. For example, and without limitation, surgical instrument 16 may be a drill, electrosurgical instrument, ultrasonic instrument, reamer, impactor, or sagittal saw. As described in more detail below, surgical instrument 16 may also be an instrument specifically designed for performing spinal surgery, such as a cobb chisel, box chisel, curette, cutter, dilator, or inserter.

[0038] The surgical navigation system 12 can be configured to use tracker-based positioning to track the pose (i.e., position and orientation) of objects of interest within the surgical workspace. Tracked objects may include, but are not limited to, the patient's anatomy, surgical instruments (such as surgical instrument 16), and the surgeon's anatomy (such as the surgeon's hand or fingers). The patient's tracked anatomy may include soft tissues such as ligaments, muscles, and skin, and / or hard tissues such as bones. Tracked surgical instruments may include retractors, cutting tools, inserters, implants, and waste management devices used during surgical procedures.

[0039] I. Hybrid Optical-Inertial Skeleton Tracking

[0040] Each object of interest can be attached to a tracker configured to emit light toward the surgical navigation system 12. At least one of the attached trackers may also include a motion sensor configured to generate motion measurement data indicating the angle of tilt relative to gravity. For example, in Figure 1 In the illustrated example, surgical system 10 is being used to perform spinal procedures on patient P, such as minimally invasive intervertebral fusion, in which an implant (e.g., a spacer, graft, fusion device) is inserted into the intervertebral disc space. To support optimal implant positioning and optimal alignment of vertebral Vs according to the target spinal curvature, surgical navigation system 12 can be configured to continuously track the posture of individual vertebral Vs at 6 DOF during the spinal procedure. One possible method to provide such tracking is to mount a 6 DOF optical tracker comprising at least three optical markers onto a spinal clamp attached to the spinous process of the vertebral V. However, given the relatively small size and close arrangement of the vertebral Vs, this method would result in crowding of the surgical site and make it difficult to perform the procedure as a minimally invasive surgery.

[0041] This disclosure describes an implementation of a small, lightweight, minimally invasive 6 DOF tracker optimized for tracking individual vertebral Vs. Such a tracker may include two or more specifically only two optical markers and a motion sensor configured to be coupled to the vertebral V. The optical markers may emit light to determine the tracker's attitude at 5 DOF, while the motion sensor may generate data to determine the tracker's orientation at an additional rotational DOF.

[0042] Although described primarily in the context of tracking the vertebral body, it should be understood that the tracking techniques and advantages described herein are not limited to the vertebral body and can be used to track any type of skeletal structure in patient P, including but not limited to the femur, tibia, scapula, humerus, pelvic bones, ribs, or skull. Furthermore, the tracker can be used in any appropriate surgical procedure in which one or more bones are tracked. Such surgical procedures include, but are not limited to: spinal procedures, partial or total knee replacement, shoulder replacement, hip replacement, skull procedures, etc.

[0043] The surgical navigation system 12 can be configured to detect light signals emitted from a given tracker by imaging it, and to receive motion measurement data from the tracker via a wired or wireless data connection to the given tracker. Alternatively, the given tracker with a motion sensor can be configured to encode motion measurement data in light signals emitted by optical markers on the tracker, such as by setting the intensity, frequency, and / or pattern of the emitted light according to the motion measurement data. The surgical navigation system 12 can then be configured to determine the tracker's orientation in a known coordinate system based on the imaging and motion data, according to 6DOF, and subsequently determine the orientation of the object to which the tracker is attached, according to 6DOF, based on the determined orientation of the tracker and a predetermined positional relationship between the object and the tracker.

[0044] In response to determining the pose of the object of interest according to 6 DOF, the surgical navigation system 12 can display the relative pose of the tracked object to assist the surgeon. The surgical navigation system 12 can also control and / or constrain the movement of the robotic manipulator 14 and / or surgical instruments 16 based on virtual boundaries associated with the tracked object. For example, the surgical navigation system 12 can identify the target volume of patient tissue to be treated and potential obstacles in the surgical workspace based on the tracked object. The surgical navigation system 12 can then restrict surgical tools (e.g., the end effector 18 of the surgical instrument 16) from contacting anything outside the target volume of the patient tissue to be treated, thereby improving surgical accuracy. The surgical navigation system 12 can also eliminate damage to the surgical instruments caused by accidental contact with other objects, which could result in unwanted debris at the target site.

[0045] Still referencing Figure 1 The surgical navigation system 12 may include a locator camera 20 and a navigation cart assembly 22. The navigation cart assembly 22 may house a navigation controller 24 configured to implement the functions, features, and processes of the surgical navigation system 12 described herein. For example, the navigation controller 24 may be configured to convert optical-based image data received from the locator camera 20 into object pose data indicating the posture of the tracked object in the surgical workspace, and to control the operation of the robotic manipulator 14 based on the indicated posture, as described herein.

[0046] The navigation controller 24 can operationally communicate with the user interface 28 of the surgical navigation system 12. The user interface 28 facilitates user interaction with both the surgical navigation system 12 and the navigation controller 24. For example, the user interface 28 may include one or more output devices that provide the user with information such as information from the navigation controller 24. Output devices may include a display 30 adapted to be located outside a sterile area including the surgical workspace, and may include a display 32 adapted to be located within the sterile area. Displays 30, 32 may be adjustably mounted to the navigation cart assembly 22. The user interface 28 may also include one or more input devices for user input to the surgical navigation system 12. Input devices may include a keyboard, mouse, and / or touchscreen 34, with which the user can interact to input surgical parameters to the navigation controller 24 and control various aspects of the navigation controller. Input devices may also include a microphone for user input via voice recognition technology.

[0047] The locator camera 20 can be configured to facilitate the identification of the orientation of a tracked object relative to a known coordinate system in a surgical workspace by generating image data indicating the orientation of the tracker attached to the object in a known coordinate system (such as the locator coordinate system LCLZ of the locator camera 20) according to at least 5 DOF. Specifically, the locator camera 20 can be communicatively coupled to the navigation controller 24 of the surgical navigation system 12 and can be configured to generate image data and transmit it to the navigation controller 24, which indicates the orientation of the tracker in the known coordinate system according to at least 5 DOF. The navigation controller 24 can also receive motion measurement data from at least one of the trackers, which indicates the tilt angle of the tracker, or more specifically, the motion sensor of the tracker, relative to gravity. As described in more detail below, the navigation controller 24 can be configured to generate tracker orientation data indicating the orientation of the tracker in the known coordinate system according to 6 DOF based on the image data and motion measurement data, and subsequently determine object orientation data indicating the orientation of the object in the known coordinate system according to 6 DOF based on the tracker orientation data and a predetermined positional relationship between the object and the tracker.

[0048] The locator camera 20 may have a housing 36 that houses at least two optical sensors 38. Each of the optical sensors 38 is adapted to detect light signals of a specific frequency band emitted by the tracker, such as invisible light signals (e.g., infrared or ultraviolet). Although Figure 1 The locator camera 20 is illustrated as a single unit with multiple optical sensors 38, but in an alternative example, the locator camera 20 may include separate units arranged around a surgical workspace, each with a separate housing 36 and one or more optical sensors 38.

[0049] The optical sensor 38 can be a one-dimensional or two-dimensional charge-coupled device (CCD). For example, the housing 36 can accommodate two two-dimensional CCDs or three one-dimensional CCDs for triangulation of the position of the tracker's optical markers in the surgical workspace. Alternatively, the locator camera 20 can employ other optical sensing technologies, such as complementary metal-oxide-semiconductor (CMOS) active pixels.

[0050] The locator camera 20 can be mounted to an adjustable arm to selectively position the optical sensor 38 within the field of view of the surgical workspace and the target volume, ideally unobstructed. The locator camera 20 may be adjustable in at least one degree of freedom by rotating about a rotary joint, and may be adjustable about two or more degrees of freedom.

[0051] Generally, the object to which each tracker is attached can be rigid and inflexible, such that movement of the object cannot or is unlikely to change the positional relationship between the object and the tracker. In other words, the relationship between the tracker and the object to which the tracker is attached in the surgical workspace can remain fixed, even if the position of the object changes within the surgical workspace. For example, the tracker can be securely attached to the patient's bones and surgical instruments, such as retractors and surgical instruments 16. In this way, in response to determining the position of the tracker in the surgical workspace using the locator camera 20 and / or motion measurement data received from the tracker, the navigation controller 24 can infer the position of the object to which the tracker is attached based on the determined position of the tracker.

[0052] For example, when the target volume to be treated is located on the spine of patient P, tracker 40 can be securely attached to each of one or more vertebrae V of patient P, and tracker 42 can be securely attached to surgical instrument 16. Tracker 42 can be integrated into surgical instrument 16 during manufacturing or can be installed separately to surgical instrument 16 during preparation for surgical procedures. Each vertebra tracker 40 can be directly mounted to a different vertebra V and / or can be removably coupled to a pedicle screw already inserted in a vertebra V.

[0053] Figure 2 and Figure 3 An example implementation of a vertebral tracker 40 is illustrated, which can be used in applications such as... Figure 1 The illustrated spinal procedure tracks individual vertebrae V. As shown in the illustrated example, the vertebral tracker 40 may include a first tracker assembly 44 and a second tracker assembly 46, each tracker assembly being configured to be rigidly fastened to the vertebrae V at different locations. Tracker assemblies 44, 46 may each include a single marker 48 that directs an optical signal to the optical sensor 38 of the locator camera 20. In some implementations, the marker 48 may be implemented as an active marker that receives current from a power source to generate an optical signal and emits it to the optical sensor 38. In this case, tracker assemblies 44, 46 may each be powered by an internal battery or may have leads that receive power through the navigation controller 24. For example, the marker 48 may be a light-emitting diode (LED) that emits light toward the optical sensor 38, such as invisible light (e.g., infrared or ultraviolet light).

[0054] Each tracker component 44, 46 may also include a tracker controller 50 communicatively coupled to the marker 48 and the navigation controller 24. The tracker controller 50 may be configured to control the rate and sequence of triggering of the marker 48, such as under the guidance of the navigation controller 24. For example, the tracker controller 50 may cause the marker 48 of each component 44, 46 to trigger at different rates and / or times to help the navigation controller 24 distinguish the marker 48. In some examples, the navigation controller 24 may form a bidirectional infrared communication channel with each tracker controller 50 to control the timing of triggering of the active marker 48 operated by the tracker controller 50, write / read non-volatile data, and obtain the status (e.g., battery level, LED damage) of the tracker components 44, 46 or objects attached to the tracker components 44, 46.

[0055] Instead of active markers, the marker 48 of tracker components 44, 46 can be implemented as a passive marker, such as a reflector that reflects light emitted from locator camera 20. For this purpose, locator camera 20 may include a light source 52 ( Figure 1 The light source illuminates tracker components 44 and 46 with light (such as invisible light, e.g., infrared or ultraviolet light). Marker 48 can be configured to reflect light back toward locator camera 20, which can then be detected by optical sensor 38.

[0056] In some cases, the surgical workspace, or more specifically, tracker 40, may include a combination of active and passive markers 48 for tracking various objects within the surgical workspace. For example, the markers 48 of tracker component 44 may be implemented as passive markers, in which case tracker component 44 may omit tracker controller 50. Conversely, the markers 48 of tracker component 46 may be implemented as active markers.

[0057] The two markers 48 enable the navigation controller 24 to determine the attitude of the tracker 40 at 5 DOF. More specifically, the two markers 48 enable the navigation controller 24 to determine the attitude of the tracker 40 based on three positional degrees of freedom and two rotational degrees of freedom, but not the rotational degrees of freedom relative to the virtual line VL extending between the markers 48 of the tracker assemblies 44 and 46. To enable tracking of the tracker 40 and the corresponding vertebra V at 6 DOF, and thereby provide a comprehensive understanding of the attitude of the vertebra V, at least one of the tracker assemblies 44 and 46 (e.g., tracker assembly 46) may also include a motion sensor 54. The motion sensor 54 is typically configured to generate measurement data indicating the tilt angle of the tracker 40 corresponding to the remaining rotational degrees of freedom, which can be transmitted to the navigation controller 24 via the tracker controller 50 of the tracker assembly 46.

[0058] Motion sensor 54 can be configured to measure motion sensor 54 relative to axis m z The movement, or more specifically, the movement of the axis m z The tilt angle relative to gravity, where the axis m of motion sensor 54 is... z The orientation of the virtual line VL extending between the optical markers 48 of the first tracker assembly 44 and the second tracker assembly 46 is such that the axis m z The tilt angle corresponds to the rotation of tracker 40 around the virtual line VL. Therefore, navigation controller 24 can be configured to obtain axis m using measurements from motion sensor 54. z The angle of inclination relative to gravity, such as the axis m z The angle between the tracker 40 and the plane normal to the gravity vector, and the orientation of the tracker 40 in the known coordinate system based on the tilt angle and the rotational degrees of freedom defined around the virtual line VL.

[0059] In some implementations, tracker components 44 and 46 can be fastened to the vertebral V, such that the axis m z Essentially perpendicular to the virtual line VL (e.g., enabling the navigation controller 24 to track the tracker 40 with sufficient accuracy for a given application, for example within 3 mm, by assuming an axis m). z Perpendicular to the virtual line VL). Alternatively or alternatively, such as when the axis mz When not substantially perpendicular to the virtual line VL, the navigation controller 24 can be configured to determine the indicating axis m before starting to track the vertebra V during the surgical procedure. z Compensation data for the angle between the plane and the plane normal to the virtual line VL. More specifically, the navigation controller 24 can be configured to determine the axis m. z With axis m z The angle between the projections on this plane. Subsequently, when tracking vertebra V, the navigation controller 24 can be configured to determine the orientation of the tracker 40 based on the tilt angle relative to gravity and compensation data obtained during the procedure, according to additional rotational degrees of freedom defined around the virtual line VL.

[0060] In some implementations, the motion sensor 54 may be implemented as an accelerometer, or more specifically, a three-axis accelerometer, which is configured to indicate the axis m of the accelerometer. z The tilt relative to gravity is used to determine the remaining rotational degrees of freedom not provided by the two optical markers 48; this tilt can be set to correlate with rotations around the remaining degrees of freedom. Correspondingly, tracker 40 can perform tracking at 6 DOF without incorporating gyroscopes, magnetometers, or redundant markers and / or sensors, nor with Kalman-like filters, which would require more space and computation and may be consequently subject to unbounded drift and magnetic interference in the operating room. In this way, tracker 40 offers the advantages of minimal setup and footprint, lower computational requirements, and less drift compared to alternative solutions.

[0061] Given the rigidity of vertebra V, once coupled to the vertebra, tracker components 44 and 46 will have a fixed positional relationship relative to each other, and thus define a tracker coordinate system VTRK, which includes features of the vertebral tracker 40 located at fixed coordinates, such as markers 48 and motion sensors 54. The tracker coordinate system VTRK may also have a fixed relationship relative to the vertebral coordinate system VBRA of the vertebral V. Correspondingly, in response to the orientation of the vertebral coordinate system VTRK being determined relative to a known coordinate system according to 6 DOF, the vertebral coordinate system VBRA, and the corresponding orientation of the vertebral V relative to the known coordinate system according to 6 DOF, can be inferred based on the fixed positional relationship between the vertebral coordinate system VBRA and the tracker coordinate system VTRK. In some implementations, tracker components 44 and 46 may be removably and rigidly coupled to the corresponding pedicle screws 55 inserted into the vertebral V during the procedure to provide such a fixed relationship.

[0062] By implementing the vertebral tracker 40 as two separate tracker components 44, 46, each including a single marker 48 and a motion sensor 54, the space occupied by the tracker 40 is reduced compared to providing a single tracker component including three or even two optical markers. Therefore, the surgical workspace is less crowded, simplifying the manipulation of the surgical instruments 16 relative to the surgical site and thus making minimally invasive surgery increasingly suitable for surgical navigation.

[0063] As a supplement to or alternative to the accelerometer, the motion sensor 54 may include a sensor for measuring the axis m. z A gyroscope for measuring the tilt angle, and / or a magnetometer for measuring the rotation of motion sensor 54 relative to gravity. The latter measurement can be used to determine the axis m. z The angle between the plane and the plane normal to the virtual line VL is used by the navigation controller 24 to guide the user in placing the tracker 40 so that the axis m of the motion sensor 54 is aligned. z Basically perpendicular to the virtual line VL, or use that angle to determine the compensation data described above.

[0064] It is further conceivable that tracker 40 may incorporate one or more redundant markers and / or motion sensors, the latter including any one or more accelerometers, gyroscopes, or magnetometers, such as to support a Kalman filter implemented by surgical navigation system 12 and / or to aid tracking tracker 40 when one of the markers 48 is occluded. For example, in some implementations, each of tracker components 44, 46 may include a motion sensor 54 for measuring the motion of tracker components 44, 46 relative to one or more axes, or more specifically, the tilt angle of each of the axes relative to gravity. In some implementations, motion sensor 54 of one of the tracker components 44, 46 may be typically de-energized during tracking and may be energized by navigation controller 24 when needed, such as upon detection of occlusion of the corresponding optical marker 48. It should be understood that in an implementation where only one of the tracker assemblies 44, 46 includes an active motion sensor 54, the navigation controller 24 may still be able to track the tracker 40 at least five degrees of freedom (DOF) after one of the optical markers 48 is occluded (e.g., by determining three position DOFs based on the tracked position of the unoccluded marker 48 and two rotation DOFs based on tilt angle measurements from the motion sensor 54).

[0065] It is also conceivable that the two markers 48 and the motion sensor 54 may be part of the same tracker assembly configured for mounting to a patient's anatomical structure, such as bone. In other words, each of these components may be supported by the same carrier structure and may have a known relationship relative to the other components before mounting to the patient's anatomical structure. As an example, at least two optical markers 48 (such as LEDs) may be positioned along a single pin structure in a straight line with the motion sensor 54, which is also incorporated into the pin structure, such that the line of the optical markers 48 is substantially perpendicular to the axis of the motion sensor 54, as described herein. In some implementations, instead of LEDs, the optical markers 48 may be formed of optical fibers extending along the pin structure. The optical fibers may be coupled to a light source (such as a laser) and have transparent portions / apertures spaced along their length to emit light and thus act as optical markers 48. In yet another implementation, the optical fibers may include continuous transparent portions extending along their length to emit light in the form of a continuous line extending along the pin structure, such that the illuminated line is substantially perpendicular to the axis of the motion sensor 54 incorporated into the pin-like structure.

[0066] Refer again Figure 1 Before initiating a surgical procedure using the surgical system 10, preoperative images can be generated for the anatomical structures of interest, such as defining the target volume of patient tissue to be treated by the surgical instruments 16 and / or the anatomical structures adjacent to that target volume. For example, when the target volume of patient tissue to be treated is located in the patient's spinal region, preoperative images of the patient's spine can be taken. These images can be based on MRI scans, radiographic scans, or computed tomography (CT) scans of the patient's anatomy and can be used to develop a virtual model of that anatomy.

[0067] Each virtual model of an anatomical structure may include a three-dimensional model (e.g., point cloud, mesh, CAD) containing data representing all or at least a portion of the anatomical structure, and / or data indicating the portion of the anatomical structure to be treated relative to the three-dimensional coordinate system of that anatomical structure. These virtual models may be provided to and stored in the navigation controller 24 prior to the surgical procedure. Treatment planning may be developed in the operating room based on kinematic studies, bone tracking, and other methods, supplementing or replacing preoperative imaging. These same methods can also be used to generate the virtual models described above.

[0068] In addition to the virtual model corresponding to the anatomical structures of interest to the patient, navigation controller 24 may also receive and store virtual models of other tracked objects of interest prior to the surgical procedure, such as surgical instruments and other objects potentially present in the surgical workspace (e.g., the surgeon's hand and / or fingers). Navigation controller 24 may also receive and store virtual models of each tracker positioned in the surgical workspace, as well as relational data for each tracker indicating the positional relationship between the tracker and the object to which it is attached. This positional relationship can be defined relative to the virtual models of the trackers and objects. For example, relative to a given tracker 40 attached to vertebra V, navigation controller 24 may receive data indicating the orientation of the vertebral coordinate system VBRA relative to the tracker coordinate system VTRK. In this way, in response to identifying the orientation of the tracker coordinate system VTRK relative to a known coordinate system at 6 DOF, navigation controller 24 may refer to the relational data of tracker 40 to determine the vertebral coordinate system VBRA and the corresponding orientation of vertebra V relative to the known coordinate system.

[0069] In some examples, the positional relationship between each tracker and the object to which it is attached can be manually indicated via interaction with a patient image depicting the tracker on the user interface 28. Alternatively, the positional relationship between each tracker and the object to which it is attached can be determined by tracing the object using a pointer instrument 56 having its own fixed tracker 58, which is tracked by a surgical navigation system 12 during tracing. The surgical navigation system 12 also tracks the trackers attached to the object to correlate the pose of the tracing object with the pose of the attached tracker.

[0070] The navigation controller 24 can also receive and store surgical planning data before the procedure. The surgical planning data can identify the patient's anatomy involved in the surgical procedure, identify the instruments used in the surgical procedure, and define the planned trajectory of the instruments and the planned movement of the patient's tissues during the surgical procedure.

[0071] Before continuously tracking the tracker 40 attached to the vertebral V, the navigation controller 24 can be configured to perform an initialization phase to help determine a virtual model of the tracker 40. For this purpose, the tracker 40 may include an interface 60 for temporarily coupling at least one additional optical marker 48 after the tracker assemblies 44, 46 are secured to the vertebral V. Alternatively, the navigation controller 24 may prompt a user to touch the interface 60 using a pointer instrument 56. In either case, the navigation controller 24 can be configured to determine the position of the interface 60 relative to the optical marker 48 of the tracker assemblies 44, 46 based on the emitted light signal.

[0072] Motion sensor 54, and more specifically, the axis m of motion sensor 54. x At least one of the optical markers 48 relative to interface 60 and tracker components 44, 46 may have a known positional relationship. Therefore, navigation controller 24 can detect the positions of optical markers 48 and interface 60 of tracker components 44, 46 in a known coordinate system as described herein, and such detected positions can be used to define the tracker coordinate system VTRK of vertebral tracker 40, which in turn defines a virtual model of tracker 40. The detected positions of optical markers 48 and interface 60 of tracker components 44, 46 can also define the initial attitude of the coordinate system VTRK of vertebral tracker 40 relative to the known coordinate system, which can be stored by navigation controller 24 for tracking the attitude of tracker 40 during the tracking phase, along with the stored axis m determined from motion sensor 54 and corresponding to the initial attitude of tracker coordinate system VTRK. z The initial tilt angle relative to gravity, as described in more detail below.

[0073] In some implementations, the navigation controller 24 may also be configured to determine the compensation data described above based on the detected positions of the optical marker 48 of the tracker components 44, 46 and the interface 60, such as based on a predefined relationship between the motion sensor 54, the optical marker 48 of the tracker component 46, and the interface 60. In an alternative implementation, based on the detected positions of the optical marker 48 of the tracker components 44, 46 and the interface 60, the navigation controller 24 may be configured to guide an operator to orient the tracker component 46 such that the axis m of the motion sensor 54... z Basically perpendicular to the virtual line VL extending between optical marks 48.

[0074] Refer again Figure 1In some implementations, the locator camera 20 may also include a motion sensor 62, similar to motion sensor 54, configured to generate data indicating the tilt angle of the locator camera 20's axis (such as defined by the locator coordinate system LCLZ) relative to gravity. After the tracker 40 is secured to the vertebral V and during the initialization phase, the navigation controller 24 may be configured to similarly utilize the measurement data from motion sensor 62 to determine the initial tilt angle of motion sensor 62 relative to gravity, and based on the initial tilt angle of motion sensor 54, the initial tilt angle of motion sensor 62, and the position of the optical marker 48 of the tracker components 44, 46 detected by the navigation controller 24 as described herein, determine the initial attitude of the tracker coordinate system VTRK in a known coordinate system according to 6 DOF. More specifically, the navigation controller 24 can be configured to determine the initial attitude of the tracker coordinate system VTRK in the known coordinate system based on the tracked position of the optical markers 48, according to three position DOFs and two rotation DOFs, and to determine the initial attitude of the tracker coordinate system VTRK in the known coordinate system based on the following additional rotation DOFs: the initial tilt angle of the motion sensor 54, the initial tilt angle of the motion sensor 62, and the virtual line VL extending between the optical markers 48 and the axis m. z The predetermined relationship between them (e.g., the calibration data described above). Using motion sensor 62 to determine the initial pose of the tracker coordinate system VTRK can serve as a redundancy or alternative to using interface 60 as described above.

[0075] Furthermore, during the tracking phase of tracker 40, navigation controller 24 can be configured to determine the orientation of tracker coordinate system VTRK in a known coordinate system based on the following rotational degrees of freedom defined about the virtual line VL: the updated tilt angle obtained from motion sensor 54, the initial tilt angle determined from motion sensor 54, and the initial tilt angle determined from motion sensor 62. As an example, navigation controller 24 can be configured to utilize motion sensor 62 as a collision detector, such as by determining whether measurement data from motion sensor 62 indicates a change in the pose of locator camera 20 and the corresponding locator coordinate system LCLZ. If so, navigation controller 24 can be configured to re-establish the initial pose of tracker coordinate system VTRK, as described above. Navigation controller 24 can be configured to indicate when such a process occurs via user interface 28. As will be described later, motion sensor 62 can be used in techniques for determining whether either the first tracker component 44 or the second tracker component 46 has collided, moved, dislocated, or detached relative to the skeleton.

[0076] As described above, the surgical instrument 16 may form part of the end effector 18 of the robotic manipulator 14. The robotic manipulator 14 may include a base 64, a plurality of links 66 extending from the base 64, and a plurality of movable joints 68 for moving the surgical instrument 16 relative to the base 64. The links 66 may be configured as follows: Figure 1 The illustrated tandem arm configuration, parallel arm configuration, or other suitable configuration may be used. The robot manipulator 14 may include the ability to operate in a manual mode, in which a user grasps the end effector of the robot manipulator 14 to induce movement of the surgical instrument 16 (e.g., direct movement, or measured by force / torque sensors that generate active drive of the robot manipulator 14). The robot manipulator 14 may also include a semi-autonomous mode, in which the surgical instrument 16 is moved by the robot manipulator 14 along a predefined toolpath (e.g., the active joint 68 of the robot manipulator 14 is actuated to move the surgical instrument 16 without the user applying force / torque to the end effector). An example of operation in semi-autonomous mode is described in U.S. Patent No. 9,119,655 to Bowling et al., which is hereby incorporated by reference in its entirety. A separate tracker 42 may be attached to the base 64 of the robot manipulator 14 to track movement of the base 64 via the locator camera 20.

[0077] Similar to the surgical navigation system 12, the robotic manipulator 14 may house a manipulator controller 70 configured to implement the functions, features, and processes of the robotic manipulator 14 described herein. The surgical instrument 16 may be a powered surgical instrument and may similarly include a positioner controller 72 configured to implement the functions, features, and processes of the surgical instrument 16, including controlling the actuation of the end effector 18 to treat target tissue, such as based on the attitude or position of the end effector 18 relative to the target tissue, under the guidance of the manipulator controller 70 and / or the navigation controller 24.

[0078] During a surgical procedure, the manipulator controller 70 may be configured to determine, for example, based on navigation data received from the navigation controller 24, the desired location to which the surgical instrument 16 should be moved. Based on this determination, and information related to the current position of the surgical instrument 16, the manipulator controller 70 may be configured to determine the extent to which link 66 needs to be moved to reposition the surgical instrument 16 from its current position to the desired position. Data indicating where link 66 will be repositioned may be forwarded to the joint motor controllers (e.g., one joint motor controller for each motor) that control the active joints 68 of the robot manipulator 14. In response to receiving such data, the joint motor controllers may be configured to move link 66 according to the data, and thus move the surgical instrument 16 to the desired position.

[0079] Now for reference Figure 4 The locator camera 20 may include a locator controller 74 communicatively coupled to the optical sensor 38 and the navigation controller 24. The tracker 42 may include at least three optical markers 48 arranged according to a predefined layout, and, assuming the optical markers 48 are active, may also include a controller communicating with the navigation controller 24 to regulate the triggering of the optical markers 48. During surgical procedures, the locator controller 72 may be configured to operate the optical sensor 38 to generate an optically based signal indicating detected light signals received from the trackers 40, 42, or more specifically, the image plane position of the optical sensor 38 in which such light signals are detected. The sampling rate of the optical sensor 38 is the rate at which the optical sensor 38 detects light signals from the sequentially triggered markers 48. The sampling rate of the optical sensor 38 may be 100 Hz or higher, more preferably 300 Hz or higher, or most preferably 500 Hz or higher. In one example, the sampling rate of the optical sensor 38 may be 8000 Hz.

[0080] In response to receiving light signals from trackers 40 and 42, optical sensor 38 may output an optical-based signal to locator controller 74 indicating the position of marker 48 of trackers 40 and 42 relative to locator camera 20. Navigation controller 24 may use these optical-based signals to determine the attitude of objects attached to trackers 40 and 42 relative to locator camera 20. Specifically, each optical sensor 38 may include a one-dimensional or two-dimensional sensor region (also referred to as an "image plane") that detects light signals from trackers 40 and 42 and outputs an optical-based signal in response indicating the pixel coordinates within the sensor region where each light signal is detected. Thus, the optical-based signal output by each optical sensor 38 may represent an image of trackers 40 and 42 generated by optical sensor 38 based on the detected light signals, wherein the image includes blobs in pixel coordinates corresponding to the positions where light signals are detected in the image plane of optical sensor 38. The detected position of each light signal can be based on the angle at which the optical sensor 38 receives the light signal, and thus can correspond to the position of the marker 48 in the surgical workspace that emits the detected light signal toward the optical sensor 38.

[0081] Optical sensor 38 can transmit optical-based signals to positioner controller 74, which can then generate image data for each optical sensor 38 based on the optical-based signals received from the optical sensor 38, and transmit such image data to navigation controller 24. The image data of optical sensor 38 can indicate the image and / or image plane position represented by the optical-based signals received from optical sensor 38. Simultaneously with the operation of optical sensor 38 to image trackers 40, 42, navigation controller 24 can communicate with motion sensor 54 of each tracker 40, such as via tracker controller 50 of tracker assembly 46, to receive motion measurement data indicating the tilt angle of motion sensor 54 relative to gravity.

[0082] Then, the navigation controller 24 can generate tracker attitude data indicating the attitude of trackers 40, 42 relative to the locator camera 20 based on the received image data and motion measurement data. More specifically, the navigation controller 24 can determine the position of marker 48 in the locator coordinate system LCLZ based on the image data. For example, the navigation controller 24 can be configured to associate patches in the image data corresponding to the same optical marker 48, triangulate the position of the optical marker 48 relative to the locator camera 20 based on the position of the associated patches in the image data and the known positional relationship between the optical sensors 38, and assign the triangulated position to the marker 48 of each tracker 40, 42 based on the known geometry of the marker 48 of each tracker 40, 42, or alternatively based on the known frequency, timing, or intensity associated with each marker 48.

[0083] The navigation controller 24 can then be configured to generate tracker attitude data, indicative of the attitude of each tracker 40, 42 relative to the locator coordinate system LCLZ, according to 6 DOF. The determined position of each tracker 42's markers 48 relative to the locator coordinate system LCLZ, relative to each tracker 42 including at least three markers 48, can define the attitude of tracker 42 according to 6 DOF. Conversely, the determined position of each tracker 40's markers 48 relative to the locator coordinate system LCLZ, relative to each vertebral tracker 40 having two markers 48, can define the attitude of tracker 40 according to 5 DOF. The navigation controller 24 can be configured to fuse the 5 DOF attitude of each tracker 40 with motion measurement data received from the tracker 40 to determine the attitude of the tracker 40 according to 6 DOF.

[0084] In some implementations, the navigation controller 24 may be configured to access previously stored swing data 76 based on motion measurement data, which indicates the orientation of the tracker 40 relative to a tilt angle indicated by the motion measurement data, according to the rotational degrees of freedom defined about a virtual line VL. More specifically, after production of the tracker 40 but before distribution, the tracker 40 may be set on a swing table configured to position the tracker 40 in various orientations relative to gravity. For each orientation, the position of the tracker 40's optical markers 48 and / or interfaces 60 can be obtained using a coordinate measuring machine (CMM) and stored in association with a tilt angle indicated by the motion sensor 54. Such data can be used to define the positional relationship of the motion sensor 54 relative to one or more of the optical markers 48 and / or interfaces 60, and / or to develop formulas for estimating the orientation of the tracker coordinate system VTRK relative to the tilt angle indicated by the motion sensor 54, each of which may be indicated by the swing data 76.

[0085] Based on the determination of the attitudes of trackers 40 and 42 in a known coordinate system according to 6 DOF, the navigation controller 24 can be configured to generate object attitude data, according to 6 DOF, indicating the attitude of objects attached to trackers 40 and 42 relative to the locator camera 20, based on the tracker attitude data. Specifically, the navigation controller 24 can retrieve previously stored relationship data 78 indicating the relationship between trackers 40 and 42 and the objects (including vertebrae V) attached to trackers 40 and 42, and can apply these positional relationships to the tracker attitude data to determine the attitude of the objects fixed to trackers 40 and 42 relative to the locator camera 20 according to 6 DOF.

[0086] In an alternative implementation, the locator controller 74 may be configured to determine tracker attitude data and / or object attitude data based on optical signals generated by the optical sensor 38, and transmit the tracker attitude data and / or object attitude data to the navigation controller 24 for further processing.

[0087] Each of the controllers described herein may include a processor, memory operatively coupled to the processor, and non-volatile memory. The processor may be programmed to perform the functions, features, and processes of the controllers described herein, and may include one or more devices selected from: microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field-programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other means of manipulating signals (analog or digital) based on operating instructions stored in memory. The memory may include a single memory device or multiple memory devices, including but not limited to: read-only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other means capable of storing information. Non-volatile memory may include one or more persistent data storage devices, such as hard disk drives, optical drives, magnetic tape drives, non-volatile solid-state devices, or any other means capable of persistently storing information.

[0088] Non-volatile memory can store software, which may include one or more applications and / or modules embodied by a set of computer-executable instructions compiled or interpreted by various programming languages ​​and / or technologies, individually or in combination, including but not limited to Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL. A processor can operate under the control of the software stored in non-volatile memory. Specifically, the processor can be configured to read the computer-executable instructions embodying the software into memory and execute the computer-executable instructions. When executed by the processor, the computer-executable instructions can be configured to cause the processor to implement the configured functions, features, and processes of the controller described herein.

[0089] The non-volatile memory can also store data that aids in the operation of the controller. Specifically, the controller's software can be configured to access this data during execution to help implement the functions, features, and processes of the navigation controller 24 described herein. For example, the data stored in the non-volatile memory relative to the navigation controller 24 may include the swing data 76 and relational data 78 described herein.

[0090] Figure 5 A method 200 for determining the pose of vertebra V at 6 DOF using tracker 40 is illustrated, and Figure 6Method 300 illustrates an exemplary implementation of method 200. Methods 200 and 300 may each be implemented by one or more of the controllers described herein (such as navigation controller 24).

[0091] In block 202, method 200 may include: determining initial attitude data of tracker 40. In some implementations, determining the initial attitude data may include one or more of blocks 302, 304, and 306 of method 300. In block 302, method 300 may include: determining the initial attitude of tracker coordinate system VTRK in a known coordinate system (such as locator coordinate system LCLZ) according to 6 DOF. As described above, when additional optical markers 48 are temporarily set relative to other markers 48 of tracker 40, such as via interface 60 of tracker 40, this initial attitude can be determined by imaging tracker 40 fastened to vertebra V using locator camera 20. In block 304, method 300 may include: determining, for example, an initial tilt angle relative to gravity corresponding to the initial attitude using measurements from motion sensor 54. More specifically, the axis m of motion sensor 54 may be determined. z The tilt angle of this axis relative to gravity has a known relationship with the rotation of the tracker coordinate system VTRK about the virtual line VL connecting the marker 48 of the tracker 40. In block 306, method 300 may include: calculating an initial vector for the optical marker 48 connecting the first tracker assembly 44 and the second tracker assembly 46 based on the initial attitude of the tracker coordinate system VTRK determined above.

[0092] In block 204, methods 200 and 300 may each include: determining whether to activate tracking of tracker 40. For example, a user may interact with the user interface 28 of the surgical navigation system 12 to indicate the start of a surgical procedure and / or the initiation of the tracking phase of tracker 40. In response, it may be determined whether to activate tracking of tracker 40.

[0093] In response to determining that tracking is active (the "Yes" branch of box 204), in box 206, methods 200 and 300 may each include: tracking the position of optical markers 48 of the first tracker assembly 44 and the second tracker assembly 46 relative to a known coordinate system (such as the locator coordinate system LCLZ) using the locator camera 20. In box 208, method 200 may include: determining 5 DOF optical attitude data based on the tracked position and initial attitude data of the optical markers 48, according to three positional degrees of freedom and two rotational degrees of freedom, which indicates the updated attitude of the tracker coordinate system VTRK associated with the first tracker assembly 44 and the second tracker assembly 46 in the known coordinate system.

[0094] Block 208 of method 200 may include one or more of blocks 308, 310, and 312 of method 300. In block 308, method 300 may include: calculating a 3 DOF translation vector from an initial position of a marker 48 of tracker 40, as indicated by an initial attitude, to a tracked position of marker 48, as determined in a known coordinate system in block 206. In block 310, method 300 may include: calculating an update vector for the optical marker 48 connecting the first tracker assembly 44 and the second tracker assembly 46 based on the tracked position of the optical marker 48. In block 312, method 300 may include: calculating a 2 DOF rotation matrix from the initial vector connecting the optical marker 48 to the update vector in a known coordinate system.

[0095] In block 210, methods 200 and 300 may each include: obtaining an updated tilt angle indicated by motion sensor 54. As previously described, the tilt angle may have a known relationship with the rotation of the tracker coordinate system VTRK about a virtual line VL connecting the tracker 40 to the marker 48. More specifically, the difference between the updated angle and the initial tilt angle determined in block 304 may indicate a change in the orientation of the tracker coordinate system VTRK relative to its initial orientation, based on the rotational degrees of freedom defined about the virtual line VL. Therefore, in block 212, method 200 may include: determining 1 DOF motion attitude data indicating a change in the orientation of the tracker coordinate system VTRK based on the updated tilt angle and the initial attitude data, according to additional rotational degrees of freedom defined about the virtual line VL.

[0096] More specifically, box 212 may include one or more of boxes 314 and 316 of method 300. In box 314, method 300 may include: calculating the difference between the updated tilt angle and the initial tilt angle. In box 318, method 300 may include: calculating a 1 DOF rotation matrix based on the difference. This calculation may be based on oscillation data 76, as described above, which may indicate the rotation of the tracker coordinate system VTRK relative to the virtual line VL extending between the markers 48 of the tracker 40.

[0097] In block 214, method 200 may include: fusing 5 DOF optical attitude data with 1 DOF motion attitude data to determine the attitude of the tracker coordinate system VTRK in 6 DOF. More specifically, block 214 may include: determining the attitude of the tracker coordinate system VTRK associated with tracker components 44, 46 in the locator coordinate system based on the tracked position of the optical markers, according to 3 positional degrees of freedom and 2 rotational degrees of freedom, such as by applying 5 DOF optical attitude data to the initial attitude of the tracker coordinate system VTRK; and determining the orientation of the tracker coordinate system VTRK based on the obtained tilt angle, according to an additional rotational DOF defined by a virtual line VL extending between the optical markers 48 of tracker components 44, 46, such as by applying 1 DOF motion attitude data to the initial attitude of the tracker coordinate system VTRK.

[0098] Box 214 may include box 318 of method 300. In box 318, method 300 may include: translating the tracker coordinate system VTRK from an initial attitude using a 3 DOF translation vector; and rotating the tracker coordinate system VTRK from the initial attitude based on a 2 DOF rotation matrix. Correspondingly, the attitude of the tracker coordinate system VTRK in a known coordinate system according to three position DOFs and two rotation DOFs may be determined. Then, box 318 may include rotating the tracker coordinate system VTRK from the initial attitude using a 1 DOF rotation matrix. Correspondingly, the orientation of the tracker coordinate system VTRK according to additional rotational degrees of freedom defined by a virtual line VL extending between optical markers 48 of tracker assemblies 44, 46 may also be determined, thereby producing the attitude of the tracker coordinate system VTRK in a known coordinate system according to 6 DOFs.

[0099] In box 216, methods 200 and 300 may each include: determining the pose of vertebra V in the known coordinate system based on the 6DOF pose of the tracker coordinate system VTRK in the known coordinate system and the relational data 78 indicating the pose of the vertebral coordinate system VBRA relative to the tracker coordinate system VTRK.

[0100] Following box 216, methods 200 and 300 can each return to box 204 to determine whether to continue tracking tracker 40. For example, a user may interact with user interface 28 to indicate the end of a surgical procedure, in which case it can be determined to interrupt tracking. Conversely, in response to determining to continue tracking (the "Yes" branch of box 204), methods 200 and 300 can each continue tracking as described above.

[0101] Although illustrated as occurring in parallel, it should be understood that the block 206 branch of methods 200 and 300 and the block 210 branch of methods 200 and 300 may occur sequentially and / or in a different order. For example, blocks 206 and 210 may be executed, and subsequently, blocks 308, 310, and 312 may be executed, followed by blocks 314 and 316. Similarly, in block 318, the initial pose of the tracker coordinate system VTRK may be transformed in a different order than described above.

[0102] II. Monitoring Tracker Accuracy Using Motion Sensors

[0103] This document describes systems, methods, and solutions in which one or more controllers utilize one or more motion sensors 62 to monitor the accuracy of one or more tracker assemblies 44, 46. Specifically, the motion sensors 62 can be used to monitor whether one or more tracker assemblies 44, 46 have been dislocated, dislodged, collided with, or moved relative to the skeleton. The motion sensors 62 can also detect whether a part or portion of a tracker assembly has been dislocated, dislodged, collided with, or moved relative to other parts or portions of the tracker assembly, which also indicates undesired movement relative to the skeleton.

[0104] By monitoring these undesirable conditions, the techniques described herein better ensure the accuracy of the corresponding tracker component's tracking of the bone. Furthermore, by providing tilt angle measurements (relative to Earth's gravity), motion sensor 62 can establish accuracy detection in a manner that does not require additional optical markers / trackers, and is therefore less susceptible to optical tracking errors. Additionally, in some solutions, motion sensor 62 can be seamlessly integrated or coupled to a single tracker component, without necessarily requiring additional fixation points on the bone separate from the tracker component being monitored. Monitoring via motion sensor 62 is continuous, and therefore, the need for surgeons to perform separate verification steps to confirm tracker accuracy by touching individual checkpoints on the implanted tracker bone using navigation pointer 56 can be reduced or eliminated. Thus, the described solution provides continuous and robust tracker accuracy monitoring in a less invasive manner for the patient, less susceptible to additional optical tracking errors, seamlessly integrated with the tracker configuration, and reduced surgical procedure time.

[0105] These technologies can be used in conjunction with the solutions described above, or entirely independently of them. The described implementation can be used in conjunction with the surgical navigation system 12 described above. Therefore, all features, capabilities, and components of the surgical navigation system 12 are incorporated herein by reference and will not be repeated for simplicity. Any one or more controllers described above, such as navigation controller 24 and locator controller 74, can be used in the solutions described herein.

[0106] The solutions described in this section are not intended to be limited to the implementations described above. As will be illustrated in the following description, the motion sensor 62 described in this section may be configured and positioned differently from the solutions described in the preceding section (I). For example, the motion sensor 62 may be located anywhere on the tracker assemblies 44, 46 or on the bone. Furthermore, one or more tracker assemblies 44, 46 may have configurations different from those described in the preceding sections. Any of the tracker assemblies 44, 46 may include any suitable optical tracking configuration, such as any number of optical markers 48 (e.g., one, two, three, four, etc.). In some cases, any of the tracker assemblies 44, 46 may employ tracking modes other than optical tracking, such as radio frequency tracking, electromagnetic tracking, inertial tracking, etc. Any of the tracker assemblies 44, 46 may be mounted to the bone using any suitable tracker mounting system. For example, the tracker mounting system may include an attachment portion that is directly attached to the bone. The attachment portion may be a bone plate, bone pin, bone fastener, clamp, limb strap, etc. Rods or posts may extend from the attachment portion. The optical marker 48 can be attached to the rod using a tracking head or housing that supports the optical marker 48. The tracking head can be coupled to the rod in a fixed or adjustable manner. In other cases, the optical marker 48 can be directly supported by the rod or extend from the rod (e.g., using a post).

[0107] Two example implementations of this solution will now be described. In the first example, at least one motion sensor 62 is used in conjunction with an optical marker 48 to provide hybrid inertial tracker accuracy monitoring. In another example, two motion sensors 62 are used, with or without the optical marker 48, to provide inertial-to-inertial tracker accuracy monitoring. In both cases, the motion sensors 62 can detect whether tracker assemblies 44, 46 have been dislocated, detached, collided with, or moved relative to the skeleton, or whether a part of one tracker assembly 44, 46 has been dislocated, detached, collided with, or moved relative to other parts of the tracker assemblies 44, 46.

[0108] A. Accuracy monitoring of hybrid inertial-optical trackers

[0109] In one implementation of tracker accuracy monitoring, at least one motion sensor 62 is used in conjunction with optical markers 48 of tracker assemblies 44, 46. In this example, tracker assemblies 44, 46 are rigidly attached to the skeleton at a first location. Tracker assemblies 44, 46 include at least one optical marker 48. The motion sensor 62 is rigidly attached to the skeleton and configured to measure tilt angles relative to gravity (e.g., the gravity vector), as described above. The locator 20 is configured to optically detect the pose of at least one optical marker 48. One or more controllers 24, 74 are coupled to the locator 20 and the motion sensor 62.

[0110] One or more controllers 24, 74 implement the method described below, and are not limited to the specific order of steps unless required by its logical sequence. One or more controllers 24, 74 receive a measured tilt angle from motion sensor 62. Motion sensor 62 may include a communication device (such as Wi-Fi, Bluetooth, RF, infrared, or other near-field communication technologies) to remotely transmit the measured tilt angle to one or more controllers 24, 74. One or more controllers 24, 74 acquire the tracked pose of at least one optical marker 48. During the initialization phase of tracker setup, the acquisition of the measured tilt angle and the tracked pose may be performed, wherein tracker components 44, 46 remain stationary and rigidly attached to the skeleton as intended at a first position. One or more controllers 24, 74 establish a baseline relationship between at least one optical marker 48 and motion sensor 62 by combining the detected pose of at least one optical marker 48 with the measured tilt angle. The baseline relationship may be a geometric relationship. For example, coordinates representing the pose of at least one optical marker 48 and a virtual line representing the measured tilt angle may be combined in a common coordinate system. In some cases, the pose of the skeleton (if known) can also be combined with the tracked pose and the measured tilt angle. However, these techniques can operate without relying on the pose of the skeleton because the motion sensor 62 provides a second reference (in addition to the optical marker) rigidly attached to Google.

[0111] After the initialization phase is completed, a baseline relationship is established to provide a reference for monitoring the relative movement of tracker components 44, 46, parts of tracker components 44, 46, or at least one optical marker 48 relative to the bone. One or more controllers 24, 74 monitor the measurement relationship between the detected pose and the measured tilt angle of at least one optical marker 48. That is, one or more controllers 24, 74 receive the detected pose and the measured tilt angle of at least one optical marker 48 over time. These detected poses and measured angles can be received at any suitable interval and for any duration, and can be monitored continuously or discretely. For example, one or more controllers 24, 74 can obtain the detected pose and measured angle per camera frame, per second, or per N seconds. This monitoring can be performed for any duration after the initialization step and can be performed continuously throughout the entire surgical procedure.

[0112] Through this monitoring, one or more controllers 24, 74 can detect deviations between the measured relationship and the baseline relationship to indicate that the tracker assemblies 44, 46, components of the tracker assemblies 44, 46, or at least one optical marker 48 has moved relative to the skeleton. That is, even if the measured tilt angle changes relative to the baseline, the change in the relative relationship between the measured angle and the tracked posture of the optical marker 48 indicates an error. Upon detecting a deviation, one or more controllers 24, 74 can be configured to identify the error condition.

[0113] In some cases, to avoid false error alarms or overly sensitive monitoring, one or more controllers 24, 74 may implement thresholds, limits, or detection ranges to be applied to detected deviations. For example, a threshold may be a distance error measurement of at least one optical mark 48, where an error is not triggered unless the distance error measurement is greater than the threshold (such as 1 mm or 2 mm). In another example, a threshold error angle (e.g., 0.5 degrees) may be set between the measured tilt angle and at least one optical mark 48.

[0114] One or more controllers 24, 74 may be configured to generate feedback to a user in response to the detection of an error. In one example, the feedback is provided via tracker components 44, 46. Tracker components 44, 46 may include audible alarms, visual indicators, or vibration feedback to alert the user. For example, tracker components 44, 46 may include a visual indicator, which may be an optical marker 48 or a separate visual indicator. The visual indicator may change color (e.g., from green to red) in response to an error. One or more controllers 24, 74 may communicate with navigation system 12 to command any such feedback to tracker components 44, 46. Alternatively or additionally, the feedback may be provided via motion sensor 62, for example using an audible alarm, or via navigation system 12, for example using visual or audiovisual alerts, messages, or notifications provided by display units 32, 34.

[0115] After describing the solution, several configurations of the motion sensor 62 are envisioned, for which the solution can be fully implemented. In one example, the motion sensor 62 is coupled to the tracker assemblies 44, 46 themselves. In other words, the motion sensor 62 is rigidly attached to the skeleton via the tracker assemblies 44, 46 to which it is attached. The motion sensor 62 can be integrally coupled to the tracker assemblies 44, 46, or detachably coupled to (and removable from) the tracker assemblies. In this implementation, in addition to being rigidly fixed to at least one optical mark 48, the motion sensor 62 can also be coupled to the attachment portion, rod, or any other component of the tracker assemblies 44, 46. In this way, the motion sensor 62 is located at a different component or position than the at least one optical mark 48. In this configuration, the motion sensor 62 can detect whether a component of the tracker assembly (to which it is attached) has moved relative to the at least one optical mark 48, thereby indicating that the tracker assemblies 44, 46 have undergone deformation relative to their original configuration.

[0116] In another implementation, motion sensor 62 is rigidly attached to the bone at a second location spaced apart from and different from the first location (where the "first" tracker assembly 44 is attached). In this configuration, by being spaced apart from and separated from the first tracker assembly 44, motion sensor 62 can detect whether any part of the first tracker assembly 44 (including at least one optical marker 48) has moved relative to the bone.

[0117] In one example, motion sensor 62 is coupled to a second tracker assembly (e.g., 46) rigidly attached to the skeleton at a second location. The second tracker assembly 46 may include any described tracking configuration or modality (e.g., optical tracking, etc.). Motion sensor 62 may be integrally coupled to the second tracker assembly 46 or detachably coupled to (and removable from) the second tracker assembly 46. Motion sensor 62 may be coupled to any part of the second tracker assembly 46, including directly adjacent to an optical marker. In such cases, the tracked poses of the optical markers of the second tracker assembly 46 may optionally be combined to determine baseline relationships. However, this is not necessarily required, as the relationship between the measured tilt angles and the tracked poses of the optical markers of the second tracker assembly 46 can be fixed or movable.

[0118] In another example, motion sensor 62 is embodied in a tracker observation device rigidly attached to the bone at a second location. The tracker observation device may be a dedicated device specifically intended for remote monitoring of the first tracker assembly 44. The tracker observation device may be used as a supplement to or replacement for the second tracker assembly 46. In one configuration, the tracker observation device includes a body supporting motion sensor 62. The body of the tracker observation device supports an attachment configured to be mounted to the bone at the second location. The attachment may be a bone plate, bone pin, bone fastener, clamp, limb strap, etc. The body also supports communication devices for motion sensor 62 to remotely transmit measured tilt angles to one or more controllers 24, 74. Motion sensor 62 may be integrally coupled to tracker observation device 46 or detachably coupled to (and removable from) tracker observation device.

[0119] In one example, the tracker observation device can be screwed into the bone at any desired landmark, such as a "checkpoint." The tracker observation device can have a small footprint to minimize interference with the surgical site. For example, the tracker observation device can have a width of 10 mm or less and, when mounted to the bone, can extend only minimally from the surface of the bone, such as 5 mm or less. When the tracker observation device is mounted to the bone, a distal portion of the tracker observation device can extend from the surface of the bone. This distal portion may include a divot formed therein. This divot can be precisely machined to mate with the distal tip of the navigation pointer 56. Thus, by rigidly attaching to the bone, the tracker observation device can optionally and additionally be used as a "manual" checkpoint to verify the accuracy of the tracker assembly. This can be implemented for redundant checks, for example, if the surgeon wishes to verify the operation of the tracker observation device or the accuracy of the tracker.

[0120] B. Inertial-Inertial Tracker Accuracy Monitoring

[0121] In another implementation of tracker accuracy monitoring, at least two motion sensors 62 are used with one or more tracker components 44, 46. In this example, tracker components 44, 46 are rigidly attached to the skeleton. Tracker components 44, 46 include at least one optical marker 48. Tracker components 44, 46 also include a (tracker) motion sensor 62 configured to measure a first tilt angle relative to gravity in the manner described above. Tracker motion sensor 62 can be coupled to tracker components 44, 46 in any of the described manners. A second motion sensor (i.e., tracker observation sensor 62') is also rigidly attached to the skeleton. Tracker observation sensor 62' is configured to measure a second tilt angle relative to gravity. Positioner 20 is configured to optically detect the posture of at least one optical marker 48. One or more controllers 24, 74 are coupled to positioner 20, tracker motion sensor 62, and tracker observation sensor 62'.

[0122] One or more controllers 24, 74 implement the method described below, and are not limited to the specific order of steps unless required by its logical sequence. One or more controllers 24, 74 receive measured first tilt angles and second tilt angles from tracker motion sensor 62 and tracker observation sensor 62'. Both sensors 62, 62' may include the described communication means to remotely transmit their respective measured tilt angles to one or more controllers 24, 74. During the described initialization phase when the tracker is set up, the measured first tilt angles and second tilt angles may be acquired, wherein tracker components 44, 46 remain stationary and rigidly attached to the skeleton as intended at a first position. One or more controllers 24, 74 establish a baseline relationship between tracker components 44, 46 and tracker observation sensor 62' by combining the measured first tilt angles and second tilt angles. The baseline relationship may be geometric. For example, virtual lines representing the measured first tilt angles and second tilt angles may be combined in a common coordinate system. Optionally, one or more controllers 24, 74 may combine the tracked pose of at least one optical marker 48 with the measured first and second tilt angles (e.g., in a common coordinate system). Additionally, the pose of the skeleton (if known) may also be combined with the measured first and second tilt angles. However, these techniques can operate independently of the pose of the skeleton or at least one optical marker 48, since the two sensors 62, 62' represent two separate references rigidly attached to the skeleton.

[0123] After the initialization phase is completed, a baseline relationship is established to provide a reference for monitoring the relative movement of tracker components 44, 46, parts of tracker components 44, 46, or at least one optical marker 48 relative to the bone. One or more controllers 24, 74 track the measured first tilt angle and second tilt angle to monitor the measurement relationship between tracker components 44, 46 and tracker observation sensor 62'. That is, one or more controllers 24, 74 receive the measured first tilt angle and second tilt angle over time. These measured first tilt angles and second tilt angles can be received at any appropriate interval and for any duration, and they can be monitored continuously or discretely. This monitoring can be performed for any duration after the initialization step and can be performed continuously throughout the entire surgical procedure.

[0124] Through this monitoring, one or more controllers 24, 74 can detect deviations between the measured relationship and the baseline relationship to indicate that the tracker components 44, 46, parts of the tracker components 44, 46, or at least one optical marker 48 have moved relative to the skeleton. That is, the relative change in the relationship between the measured first tilt angle and the second tilt angle indicates an error. Upon detecting a deviation, one or more controllers 24, 74 can be configured to identify the error condition.

[0125] In some cases, to avoid false alarms or overly sensitive monitoring, one or more controllers 24, 74 may implement thresholds, limits, or detection ranges to be applied to detected deviations. For example, a threshold may be a distance error measurement between a measured first tilt angle and a second tilt angle, where an error is not triggered unless the distance error measurement is greater than the threshold (such as 1 mm or 2 mm). In another example, a threshold error angle (e.g., 0.5 degrees) may be set between the measured first tilt angle and the second tilt angle.

[0126] One or more controllers 24, 74 may be configured to generate feedback to the user in response to the detection of an error. In one example, auditory, visual, and / or tactile feedback is provided via tracker components 44, 46 in the manner described above. Alternatively or additionally, this feedback may be provided via one or two sensors 62, 62', for example using an auditory alarm or via navigation system 12, for example using visual or audiovisual alerts, messages, or notifications provided by display units 32, 34.

[0127] After describing the solution, several configurations of the tracker observation sensor 62' are envisioned, for which the solution can be fully implemented. In one example, the tracker observation sensor 62' is coupled to the tracker assemblies 44, 46 themselves. Here, the tracker observation sensor 62' is rigidly attached to the skeleton via the tracker assemblies 44, 46 to which it is attached. The tracker observation sensor 62' can be integrally coupled to the tracker assemblies 44, 46, or detachably coupled to (and removable from) the tracker assemblies. In this configuration, the tracker assemblies will support both the tracker motion sensor 62 and the tracker observation sensor 62'. The tracker motion sensor 62 can be coupled to the attachment portion, rod, or any other component of the tracker assemblies 44, 46 (including being fixedly coupled to at least one optical marker 48). Simultaneously, the tracker observation sensor 62' can be coupled to any different component of the tracker assemblies 44, 46 than the component to which the tracker motion sensor 62' is attached (e.g., an attachment, rod, tracking head, or fixedly coupled to at least one optical mark 48). Accordingly, the two tracker components supporting the two sensors 62, 62' should be able to move relative to each other to optimize the robustness of error detection. In this configuration, the sensors 62, 62' can detect whether a component of the tracker assembly (to which it is attached) has moved relative to other components, thereby indicating that the tracker assemblies 44, 46 have undergone deformation relative to their original configuration.

[0128] In another implementation, the tracker observation sensor 62' is rigidly attached to the bone at a second location spaced apart from and different from the first location (where the "first" tracker assembly 44 and the tracker motion sensor 62 are attached). In this configuration, by being spaced apart from and separated from the first tracker assembly 44, the tracker observation sensor 62' can detect whether any part of the first tracker assembly 44 (including at least one optical marker 48) has moved relative to the bone.

[0129] In one example, tracker observation sensor 62' is coupled to a second tracker assembly (e.g., 46) rigidly attached to the skeleton at a second location. The second tracker assembly 46 may include any described tracking configuration or modality (e.g., optical tracking, etc.). Tracker observation sensor 62' may be integrally or removably coupled to any component of the second tracker assembly 46, including directly adjacent to an optical marker. In such cases, the tracked pose of the optical marker of the second tracker assembly 46 may optionally be combined to determine a baseline relationship. However, this is not necessarily required, as the relationship between the tilt angle of the second measurement and the tracked pose of the optical marker of the second tracker assembly 46 can be fixed or movable.

[0130] In another example, tracker observation sensor 62' is detachably coupled to or integrated with a tracker observation device rigidly attached to the bone at a second location. The tracker observation device may be a dedicated device specifically intended for remote monitoring of the first tracker assembly 44. The tracker observation device may be used as a supplement to or replacement for the second tracker assembly 46. In one configuration, the tracker observation device includes a body supporting the tracker observation sensor 62'. The body of the tracker observation device supports an attachment configured to be mounted to the bone at the second location. The attachment may be a bone plate, bone pin, bone fastener, clamp, limb strap, etc. The body also supports communication devices for the tracker observation sensor 62' to remotely transmit a second measured tilt angle to one or more controllers 24, 74.

[0131] The foregoing description is illustrative in nature and is in no way intended to limit the disclosure, its application, or use. The broad doctrines of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be limited thereto, as other modifications will become apparent upon examination of the drawings, specification, and appended claims. It should be understood that one or more steps in the method may be performed in different orders (or simultaneously) without altering the principles of this disclosure. Furthermore, while each of the examples is described above as having certain features, any one or more of those features described with respect to any example of this disclosure may be implemented in any other example and / or combined with features of any other example, even if such combination is not explicitly described. In other words, the described examples are not mutually exclusive, and permutations and combinations of one or more examples with each other remain within the scope of this disclosure.

[0132] Spatial and functional relationships between components (e.g., between controllers, circuit elements, semiconductor layers, etc.) are described using various terms, including “connected,” “joined,” “coupled,” “adjacent,” “next to,” “on top of,” “above,” “below,” and “set.” Unless explicitly described as “direct,” when the relationship between a first component and a second component is described in the foregoing disclosure, the relationship can be a direct relationship, where no other intermediate components exist between the first component and the second component, or it can be an indirect relationship, where one or more intermediate components exist between the first component and the second component (spatially or functionally).

[0133] As used herein, the phrase "at least one of A, B, and C" should be interpreted as meaning the logic of using non-exclusive OR (A or B or C), and should not be interpreted as meaning "at least one of A, at least one of B, and at least one of C". The term subset does not necessarily require a proper subset. In other words, a first subset of a first set is completely equivalent to (equal to) the first set.

[0134] In the accompanying drawings, the direction of the arrows (as indicated by the arrowheads) typically shows the flow of information (such as data or instructions) of interest to the illustration. For example, when components A and B exchange various types of information, but the information transmitted from component A to component B is relevant to the illustration, the arrow may point from component A to component B. This unidirectional arrow does not imply that no other information is transmitted from component B to component A. Furthermore, for information sent from component A to component B, component B may send a request for that information to component A or receive acknowledgment of that information.

[0135] In this application (including the definitions below), the terms "controller" or "module" may be replaced by the term "circuit". The term "controller" may refer to, be a subset of, or include: at least one application-specific integrated circuit (ASIC); at least one programmable system-on-chip (PSoC); at least one digital, analog, or mixed-signal analog / digital discrete circuit; at least one digital, analog, or mixed-signal analog / digital integrated circuit; at least one combinational logic circuit; at least one field-programmable gate array (FPGA); at least one processor (shared, dedicated, or grouped) that executes code; at least one memory circuit (shared, dedicated, or grouped) that stores code executed by at least one processor; other suitable hardware components that provide the described functionality; or combinations of some or all of the foregoing, such as in a system-on-chip.

[0136] The controller may include one or more interface circuits having one or more transceivers, such as radio frequency (RF) or optical-based transceivers (e.g., infrared (IR)). In some examples, the interface circuitry may implement a wired or wireless interface for connection to a local area network (LAN) or a wireless personal area network (WPAN). Examples of LANs are the Institute of Electrical and Electronics Engineers (IEEE) standard 802.11-2016 (also known as the Wi-Fi wireless network standard) and the IEEE standard 802.3-2015 (also known as the Ethernet wired network standard). Examples of WPANs are the Bluetooth wireless network standard from the Bluetooth Special Interest Group and the IEEE standard 802.15.4.

[0137] The controller can communicate with other controllers using interface circuitry. Although the controller may be depicted in this disclosure as logically communicating directly with other controllers, in various implementations, the controller may actually communicate via a communication system. The communication system may include physical and / or virtual network devices such as hubs, switches, routers, gateways, and transceivers. In some implementations, the communication system is connected to or traverses a wide area network (WAN), such as the Internet. For example, the communication system may include multiple LANs interconnected via the Internet or peer-to-peer leased lines using technologies including Multiprotocol Label Switching (MPLS) and Virtual Private Networks (VPNs).

[0138] In various implementations, controller functionality can be distributed among multiple controllers connected via a communication system. For example, multiple controllers can implement the same functionality assigned by a load balancing system. In another example, controller functionality can be split between a server (also known as a remote or cloud) controller and a client (or user) controller.

[0139] Some or all of the hardware characteristics of the controller can be defined using a hardware description language, such as IEEE Standard 1364-2005 (often referred to as "Verilog") and IEEE Standard 1076-2008 (often referred to as "VHDL"). Hardware description languages ​​can be used to fabricate and / or program hardware circuits. In some implementations, some or all of the controller's characteristics can be defined using a language such as IEEE 1666-2005 (often referred to as "SystemC"), which encompasses both code and hardware description as described below.

[0140] As used above, the term "code" can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuitry" covers a single processor circuitry that executes some or all of the code from multiple controllers. The term "grouped processor circuitry" covers processor circuitry that, in conjunction with additional processor circuitry, executes some or all of the code from one or more controllers. References to multiple processor circuitry cover multiple processor circuitry on a discrete die, multiple processor circuitry on a single die, multiple cores of a single processor circuitry, multiple threads of a single processor circuitry, or a combination of the foregoing. The term "shared memory circuitry" covers a single memory circuitry that stores some or all of the code from multiple controllers. The term "grouped memory circuitry" covers memory circuitry that, in conjunction with additional memory, stores some or all of the code from one or more controllers.

[0141] The term memory circuit is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not cover transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); therefore, the term computer-readable medium can be considered tangible and non-transitory. Non-limiting examples of non-transitory computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or masked read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0142] The apparatus and methods described in this application can be implemented, partially or completely, by a dedicated computer created by configuring the computer to perform one or more specific functions embodied in a computer program. The function blocks and flowchart elements described above can be used as software specifications that can be translated into computer programs through the routine work of technicians or programmers.

[0143] A computer program may include processor-executable instructions stored on at least one non-transitory computer-readable medium. A computer program may also include or depend on stored data. A computer program may encompass a basic input / output system (BIOS) for interacting with the hardware of a special-purpose computer, device drivers for interacting with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0144] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from the source code by a compiler; (iv) source code for execution by an interpreter; and (v) source code for compilation and execution by a just-in-time (JIT) compiler, etc. As an example only, source code may be written using the syntax of the following languages: C, C++, C#, Objective C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, JavaScript®, HTML5 (Hypertext Markup Language Version 5), Ada, ASP (Dynamic Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.

Claims

1. A surgical navigation system for tracking bones during medical procedures, the surgical navigation system comprising: A first tracker assembly having a first single optical marker and a motion sensor, the first tracker assembly being coupled to the bone at a first location; A second tracker assembly having a second single optical marker, the second tracker assembly being coupled to the bone at a second location different from the first location; A locator configured to optically track the positions of the first single optical marker and the second single optical marker in the locator's coordinate system; as well as One or more controllers, wherein the one or more controllers are configured to: Based on the tracked positions of the first single optical marker and the second single optical marker, the attitude of the tracker coordinate system associated with the first tracker assembly and the second tracker assembly in the locator coordinate system is determined according to three positional degrees of freedom and two rotational degrees of freedom; The tilt angle relative to gravity is determined using measurements from the motion sensor. Based on the tilt angle, the orientation of the tracker coordinate system is determined according to additional rotational degrees of freedom defined by a virtual line extending between the first single optical marker and the second single optical marker; as well as Based on the determined pose and orientation in the tracker coordinate system, the pose of the skeleton in the locator coordinate system is determined according to six degrees of freedom.

2. The surgical navigation system of claim 1, wherein the motion sensor is configured to measure movement relative to an axis substantially perpendicular to the virtual line extending between the first single optical marker and the second single optical marker, and the tilt angle relative to gravity is defined as the tilt angle of the axis relative to gravity.

3. The surgical navigation system of claim 1, wherein the motion sensor is configured to measure movement relative to an axis, and the tilt angle relative to gravity is defined as the tilt angle of the axis relative to a plane normal to the gravity vector.

4. The surgical navigation system of claim 1, wherein the motion sensor is configured to measure movement relative to an axis, the tilt angle relative to gravity is defined as the tilt angle of the axis relative to gravity, and the one or more controllers are configured to: Determine the angle between the axis and the plane normal to the virtual line extending between the first and second individual optical marks; and Based on the determined tilt angle and the angle between the axis and the plane normal to the virtual line, the orientation of the tracker coordinate system is determined according to the additional rotational degrees of freedom defined around the virtual line extending between the first single optical mark and the second single optical mark.

5. The surgical navigation system of claim 1, wherein the motion sensor is defined as an accelerometer.

6. The surgical navigation system of claim 1, wherein neither the first tracker assembly nor the second tracker assembly includes a magnetometer and neither includes a gyroscope.

7. The surgical navigation system of claim 1, wherein the first tracker assembly and the second tracker assembly are removably coupled to a first pedicle screw and a second pedicle screw inserted into the bone, respectively.

8. The surgical navigation system of claim 1, wherein the skeleton is a vertebra.

9. The surgical navigation system of claim 1, wherein the one or more controllers are configured to: The initial attitude of the tracker coordinate system in the locator coordinate system is determined based on six degrees of freedom. The initial tilt angle relative to gravity is obtained using measurements from the motion sensor corresponding to the initial posture. Based on the tracked positions of the first and second single optical markers and the initial attitude, the attitude of the tracker coordinate system in the locator coordinate system is determined according to three positional degrees of freedom and two rotational degrees of freedom; as well as Based on the determined tilt angle and the initial tilt angle, the orientation of the tracker coordinate system is determined according to the additional rotational degrees of freedom.

10. The surgical navigation system of claim 9, wherein the one or more controllers are configured to: Calculate a first vector in the locator coordinate system that connects the first single optical marker and the second single optical marker based on the initial attitude; A second vector connecting the first and second single optical markers in the locator coordinate system is calculated based on the tracked positions of the first and second single optical markers. Calculate the rotation matrix based on the first vector and the second vector; as well as Based on the rotation matrix, the attitude of the tracker coordinate system in the locator coordinate system is determined according to the two rotational degrees of freedom.

11. The surgical navigation system of claim 9, wherein the one or more controllers are configured to: Calculate the rotation matrix based on the initial tilt angle and the determined tilt angle; and The orientation of the tracker coordinate system is determined based on the rotation matrix calculated according to the initial tilt angle and the determined tilt angle, according to the additional rotational degrees of freedom.

12. The surgical navigation system of claim 9, wherein the surgical navigation system includes a third tracker assembly, the third tracker assembly including at least one optical marker detectable by the locator, wherein at least one of the first tracker assembly and the second tracker assembly includes features for temporary contact by the third tracker assembly to determine the initial attitude of the tracker coordinate system in the locator coordinate system according to six degrees of freedom.

13. The surgical navigation system of claim 9, wherein the surgical navigation system includes a second motion sensor capable of moving with the locator, wherein the one or more controllers are configured to: A second initial tilt angle relative to gravity is obtained using measurements from the second motion sensor corresponding to the initial posture; and Based on the determined tilt angle, the initial tilt angle, and the second initial tilt angle, the orientation of the tracker coordinate system is determined according to the additional rotational degrees of freedom.

14. The surgical navigation system of claim 1, wherein the one or more controllers are configured to monitor measurements from the motion sensors to determine whether either the first tracker component or the second tracker component has moved relative to the bone.

15. A surgical system comprising a robotic manipulator and a surgical navigation system as claimed in claim 1, wherein one or more controllers are configured to control the movement of the robotic manipulator relative to the skeleton during the medical procedure based on a determined posture of the skeleton.

16. A method for tracking bones during a medical procedure using a first tracker assembly and a second tracker assembly, the first tracker assembly including a first single optical marker and a motion sensor, the second tracker assembly including a second single optical marker, the method comprising: The first tracker assembly is fastened to the bone at a first location, and the second tracker assembly is fastened to the bone at a second location different from the first location; The positions of the first single optical marker and the second single optical marker in the coordinate system of the locator are optically tracked using the locator; One or more controllers determine the attitude of the tracker coordinate system associated with the first tracker component and the second tracker component in the locator coordinate system based on the tracked positions of the first single optical marker and the second single optical marker, according to three positional degrees of freedom and two rotational degrees of freedom; The tilt angle relative to gravity is determined by the one or more controllers using measurements from the motion sensors; The orientation of the tracker coordinate system is determined by the one or more controllers based on the tilt angle, according to additional rotational degrees of freedom defined by a virtual line extending between the first single optical marker and the second single optical marker; as well as The posture of the skeleton in the locator coordinate system is determined by the one or more controllers based on the determined pose and orientation of the tracker coordinate system, according to six degrees of freedom.

17. The method of claim 16, wherein securing the first tracker assembly and the second tracker assembly to the bone comprises removably securing the first tracker assembly and the second tracker assembly to a first pedicle screw and a second pedicle screw inserted into the bone, respectively.

18. The method of claim 16, wherein the method comprises: The initial attitude of the tracker coordinate system in the locator coordinate system is determined based on six degrees of freedom. The initial tilt angle relative to gravity is obtained using measurements from the motion sensor corresponding to the initial posture. Based on the tracked positions of the first and second single optical markers and the initial attitude, the attitude of the tracker coordinate system in the locator coordinate system is determined according to three positional degrees of freedom and two rotational degrees of freedom; as well as Based on the determined tilt angle and the initial tilt angle, the orientation of the tracker coordinate system is determined according to the additional rotational degrees of freedom.

19. A hybrid tracker for tracking a skeleton in six degrees of freedom, the hybrid tracker comprising: A first tracker assembly having a first single optical marker and a motion sensor, the first tracker assembly being coupled to the bone at a first location; as well as A second tracker assembly having a second single optical marker, the second tracker assembly being coupled to the bone at a second location different from the first location; The motion sensor is configured to generate a measurement indicating the tilt angle relative to gravity and to measure movement relative to an axis substantially perpendicular to a virtual line extending between the first and second single optical markers.

20. The hybrid tracker of claim 19, wherein the skeleton is a vertebra, and wherein: The first tracker assembly includes a first pedicle screw configured to be inserted into the vertebra, and the first single optical marker is coupled to the first pedicle screw; and The second tracker assembly includes a second pedicle screw configured to be inserted into the vertebra, and the second single optical marker is coupled to the second pedicle screw.

21. A surgical navigation system, the surgical navigation system comprising: A tracker assembly, which is rigidly attached to the bone and includes at least one optical marker; A motion sensor, which is rigidly attached to the bone and configured to measure the tilt angle relative to gravity; A locator configured to optically detect the attitude of the at least one optical marker; as well as One or more controllers, coupled to the positioner and the motion sensor, are configured to: Receive the measured tilt angle; A baseline relationship between the at least one optical marker and the motion sensor is established by combining the detected posture of the at least one optical marker with the measured tilt angle; The measurement relationship between the detected posture and the measured tilt angle of the at least one optical marker is monitored; as well as The deviation between the measured relationship and the baseline relationship is detected to indicate that the at least one optical marker has moved relative to the bone.

22. The surgical navigation system of claim 21, wherein the motion sensor is coupled to the tracker assembly.

23. The surgical navigation system of claim 22, wherein: The tracker assembly includes an attachment portion coupled to the bone, a rod extending from the attachment portion, and a tracking head coupled to the rod and supporting the at least optical marker; and The motion sensor is coupled to the attachment portion or the rod.

24. The surgical navigation system of claim 21, wherein: The tracker assembly is attached to the bone at a first location; and The motion sensor is rigidly attached to the bone at a second location that is spaced apart from and different from the first location.

25. The surgical navigation system of claim 24, further comprising a second tracker assembly rigidly attached to the bone at the second location and including at least one optical marker, wherein the motion sensor is coupled to the second tracker assembly.

26. The surgical navigation system of claim 24, wherein the surgical navigation system includes a tracker observation device rigidly attached to the bone at the second location, the tracker observation device comprising: main body: An attachment, which is coupled to the body and configured to be mounted to the skeleton at the second location; A motion sensor, the motion sensor being coupled to the main body; as well as A communication device coupled to the main body and configured to remotely transmit the measured tilt angle to the one or more controllers.

27. The surgical navigation system of claim 21, wherein the one or more controllers are configured to generate feedback in response to detecting the deviation.

28. A method of operating a surgical navigation system, the surgical navigation system comprising: A tracker assembly, which is rigidly attached to the bone and includes at least one optical marker; A motion sensor, rigidly attached to the bone and configured to measure tilt angles relative to gravity; a locator configured to optically detect the posture of the at least one optical marker; The method includes one or more controllers coupled to the positioner and the motion sensor. The measured tilt angle is received from the motion sensor; A baseline relationship between the at least one optical marker and the motion sensor is established by combining the detected posture of the at least one optical marker with the measured tilt angle; The measurement relationship between the detected posture and the measured tilt angle of the at least one optical marker is monitored; as well as The deviation between the measurement relationship and the baseline relationship is detected to indicate that the at least one optical marker has moved relative to the bone.

29. A surgical navigation system, the surgical navigation system comprising: A tracker assembly, rigidly attached to a bone, includes: at least one optical marker, and a tracker motion sensor configured to measure a first tilt angle relative to gravity. A tracker observation sensor is rigidly attached to the bone and configured to measure a second tilt angle relative to gravity; A locator configured to optically detect the attitude of the at least one optical marker; One or more controllers, coupled to the locator, the tracker motion sensor, and the tracker observation sensor, and configured to: Receive the measured first tilt angle and second tilt angle; A baseline relationship between the tracker assembly and the tracker observation sensor is established by combining the measured first tilt angle and second tilt angle. Tracking the measured first and second tilt angles to monitor the measurement relationship between the tracker assembly and the tracker observation sensor; and The deviation between the measured relationship and the baseline relationship is detected to indicate that the at least one optical marker has moved relative to the bone.

30. The surgical navigation system of claim 29, wherein the tracker observation sensor is coupled to the tracker assembly.

31. The surgical navigation system of claim 30, wherein: The tracker assembly includes an attachment portion coupled to the bone, a rod extending from the attachment portion, and a tracking head coupled to the rod and supporting the at least optical marker; The tracker motion sensor is coupled to one of the following: the attachment portion, the rod, or the tracking head; and The tracker observation sensor is coupled to another of the following: the attachment portion, the rod, or the tracking head.

32. The surgical navigation system of claim 29, wherein: The tracker assembly is attached to the bone at a first location; and The tracker observation sensor is rigidly attached to the bone at a second location that is spaced apart from and different from the first location.

33. The surgical navigation system of claim 32, further comprising a second tracker assembly rigidly attached to the bone at the second location and including at least one optical marker, wherein the tracker observation sensor is coupled to the second tracker assembly.

34. The surgical navigation system of claim 32, wherein the surgical navigation system includes a tracker observation device rigidly attached to the bone at the second location, the tracker observation device comprising: main body: An attachment, which is coupled to the body and configured to be mounted to the skeleton at the second location; A tracker observation sensor, the tracker observation sensor being coupled to the main body; as well as A communication device coupled to the main body and configured to remotely transmit the measured second tilt angle to the one or more controllers.

35. The surgical navigation system of claim 29, wherein the tracker motion sensor is in a fixed relationship with the at least one optical marker.

36. The surgical navigation system of claim 29, wherein the one or more controllers establish the baseline relationship between the tracker assembly and the tracker observation sensor by further combining the measured first tilt angle and second tilt angle with the tracked attitude of the at least one optical marker of the tracker assembly.

37. The surgical navigation system of claim 29, wherein the one or more controllers are configured to generate feedback in response to detecting the deviation.

38. A method of operating a surgical navigation system, the surgical navigation system comprising: A tracker assembly rigidly attached to a bone, the tracker assembly comprising: at least one optical marker and a tracker motion sensor configured to measure a first tilt angle relative to gravity; a tracker observation sensor rigidly attached to the bone and configured to measure a second tilt angle relative to gravity; a locator configured to optically detect the posture of the at least one optical marker; and one or more controllers coupled to the locator, the tracker motion sensor, and the tracker observation sensor, the method comprising the one or more controllers: Receive the measured first tilt angle and second tilt angle; A baseline relationship between the tracker assembly and the tracker observation sensor is established by combining the measured first tilt angle and second tilt angle. The measured first and second tilt angles are tracked to monitor the measurement relationship between the tracker assembly and the tracker observation sensor; and The deviation between the measurement relationship and the baseline relationship is detected to indicate that the at least one optical marker has moved relative to the bone.

39. A tracker observation device configured to monitor a tracker assembly mounted to a skeleton at a first location, the tracker observation device comprising: main body: An attachment, which is coupled to the body and configured to be mounted to the skeleton at a second location spaced apart from and different from the first location; A motion sensor, coupled to the body and configured to measure the tilt angle relative to gravity; as well as A communication device coupled to the main body and configured to remotely transmit the measured tilt angle.

Images