Fixing device for fluoroscopic imaging system and related navigation system and method

Abstract

The present application relates to a fixture for a fluoroscopic imaging system and related navigation systems and methods. The present invention discusses a fixture for a fluoroscopic x-ray imaging system, wherein the fluoroscopic imaging system includes a C-arm, an x-ray source at a first end of the C-arm, and an x-ray detector at a second end of the C-arm. The fixture includes a processor and a memory coupled to the processor. The memory includes instructions executable by the processor, so that the processor is configured to detect an x-ray emission from the x-ray source toward the x-ray detector, determine an offset of the x-ray source relative to the x-ray detector in response to detecting the x-ray emission, and provide an indication of the offset of the x-ray source to a medical navigation system. Related methods and robotic systems are also discussed.

Description

Technical Field

[0001] The present disclosure relates to medical devices, and more particularly, to fluoroscopic x-ray imaging systems and related methods and devices. Background Art

[0002] Before and / or during the performance of a surgical procedure using surgical navigation, registration may be required between the coordinate system of the fluoroscopic x-ray imaging system, the coordinate system of the tracking system (e.g., the camera coordinate system), and / or the coordinate system of the pre-operative image of the anatomical structure (e.g., an MRI or CT scan). Because the fluoroscopic imaging system is a large piece of equipment having an x-ray source and an x-ray detector supported at opposite ends of a large C-arm, the C-arm may flex differently at different locations, so that the relative positions of the x-ray source and the x-ray detector may be different at different locations of the C-arm, making it difficult to properly register the resulting image with respect to other coordinate systems. Summary of the invention

[0003] According to some embodiments of the inventive concept, a surgical imaging system is configured to be used with a fluoroscopic imaging system. The fluoroscopic imaging system includes a C-arm, an x-ray source at a first end of the C-arm, and an x-ray detector at a second end of the C-arm, and the fluoroscopic imaging system is configured to generate an x-ray image based on x-rays received from the x-ray source at the x-ray detector. The surgical imaging system includes a fixture and a medical navigation system. The fixture includes an x-ray opaque reference pattern in a single plane, and the fixture is coupled to the x-ray detector so that the x-ray opaque reference pattern is located on a surface of the x-ray detector between the x-ray detector and the x-ray emitter. The medical navigation system is configured to receive a first patient image corresponding to a first orientation of the C-arm from the fluoroscopic imaging system, wherein the first patient image includes a first shadow corresponding to the x-ray opaque reference pattern. The medical navigation system is also configured to receive a second patient image corresponding to a second orientation of the C-arm from the fluoroscopic imaging system, wherein the second patient image includes a second shadow corresponding to the x-ray opaque reference pattern. The medical navigation system is also configured to provide alignment between a tracking coordinate system of a physical space monitored by the tracking camera and an image coordinate system of the first patient image and the second patient image based on a correlation of an offset of the x-ray source relative to the x-ray detector as a function of an orientation of the x-ray detector relative to gravity and based on a first shadow in the first patient image and a second shadow in the second patient image.

[0004] According to some embodiments of the present inventive concept, a fixture is provided for a fluoroscopic x-ray imaging system, wherein the fluoroscopic x-ray imaging system includes a C-shaped arm, an x-ray source at a first end of the C-shaped arm, and an x-ray detector at a second end of the C-shaped arm. The fixture includes a processor and a memory coupled to the processor. The memory includes instructions executable by the processor such that the fixture is configured to detect an x-ray emission from the x-ray source toward the x-ray detector, determine an offset of the x-ray source relative to the x-ray detector in response to detecting the x-ray emission, and provide an indication of the offset of the x-ray source to a medical navigation system.

[0005] According to some other embodiments of the inventive concept, a method for operating a fixture of a fluoroscopic x-ray imaging system is provided, the fluoroscopic x-ray imaging system comprising a C-shaped arm, an x-ray source at a first end of the C-shaped arm, and an x-ray detector at a second end of the C-shaped arm. An x-ray emission from the x-ray source toward the x-ray detector is detected. An offset of the x-ray source relative to the x-ray detector is determined in response to detecting the x-ray emission. The offset of the x-ray source is provided to a medical navigation system.

[0006] According to other embodiments of the inventive concept, a medical navigation system includes a processor and a memory coupled to the processor, wherein the memory includes instructions executable by the processor, so that the medical navigation system is configured to perform the following operations. The processor is configured to receive information defining a first image from a fluoroscopic x-ray imaging system, wherein the fluoroscopic x-ray imaging system includes a C-arm, an x-ray source at a first end of the C-arm, and an x-ray detector at a second end of the C-arm. The processor is configured to provide / receive an indication of a first offset of the x-ray source relative to the x-ray detector, wherein the first offset is associated with the first image. The processor is configured to receive information defining a second image from the fluoroscopic x-ray imaging system, wherein the first image and the second image are different. The processor is configured to provide / receive an indication of a second offset of the x-ray source relative to the x-ray detector, wherein the second offset is associated with the second image, and the first offset and the second offset are different. The processor is configured to provide a registration between a tracking coordinate system of a physical space monitored by a tracking sensor and an image coordinate system of a first image and a second image from the fluoroscopic x-ray imaging system, wherein the registration is provided based on an indication of the first offset and the second offset.

[0007] According to yet another embodiment of the inventive concept, a method for operating a medical navigation system is provided. Information defining a first image is received from a fluoroscopic x-ray imaging system, the fluoroscopic x-ray imaging system comprising a C-arm, an x-ray source at a first end of the C-arm, and an x-ray detector at a second end of the C-arm. An indication of a first offset of the x-ray source relative to the x-ray detector is provided / received, wherein the first offset is associated with the first image. Information defining a second image is received from the fluoroscopic x-ray imaging system, wherein the first image and the second image are different. An indication of a second offset of the x-ray source relative to the x-ray detector is provided / received, wherein the second offset is associated with the second image, and the first offset and the second offset are different. Based on the indication of the first offset and the second offset, a registration is provided between a tracking coordinate system of a physical space monitored by a tracking sensor and an image coordinate system of the first image and the second image from the fluoroscopic x-ray imaging system.

[0008] By reading the following figures and detailed description, other methods and related systems according to embodiments of the subject matter of the present invention and corresponding methods and computer program products will become apparent to those skilled in the art. It is intended that all such systems and corresponding methods and computer program products are included in this specification, are within the scope of the subject matter of the present invention, and are protected by the appended claims. Moreover, it is intended that all embodiments disclosed herein can be implemented individually or in combination in any manner and / or combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, and illustrate certain non-limiting embodiments of the disclosed concepts. In the drawings:

[0010] Figure 1 is a top view of a possible arrangement of the positions of the robotic system, the patient, the surgeon, and other medical staff during a surgical procedure;

[0011] Figure 2 A robotic system including a surgical robot and positioning of a camera relative to a patient is shown according to one embodiment;

[0012] Figure 3 A surgical robotic system according to an exemplary embodiment is shown;

[0013] Figure 4 shows a portion of a surgical robot according to an exemplary embodiment;

[0014] Figure 5 shows a block diagram of a surgical robot according to an exemplary embodiment;

[0015] Figure 6 A surgical robot according to an exemplary embodiment is shown;

[0016] 7A to 7C An end effector according to an exemplary embodiment is shown;

[0017] Figure 8 shows a surgical instrument and an end effector before and after inserting the surgical instrument into a guide tube of the end effector according to one embodiment;

[0018] 9A to 9C shows portions of an end effector and a robotic arm according to an exemplary embodiment;

[0019] Fig.10 shows a dynamic reference array, imaging array, and other components according to an exemplary embodiment;

[0020] Fig.11 A registration method according to an exemplary embodiment is shown;

[0021] FIG. 12A to FIG. 12B An embodiment of an imaging device according to an exemplary embodiment is shown;

[0022] Fig.13A shows a portion of a robot including a robot arm and an end effector according to an exemplary embodiment;

[0023] Fig. 13B yes Fig.13A a close-up view of an end effector shown having a plurality of tracking markers rigidly attached thereto;

[0024] Fig. 13C is a tool or instrument having a plurality of tracking markers rigidly attached thereto according to one embodiment;

[0025] Fig.14A is an alternative version of the end effector having a movable tracking marker in the first configuration;

[0026] Fig. 14B is in the second configuration with a removable tracking mark Fig.14A The end effector shown in ;

[0027] Fig. 14C Shown from Fig.14A a template for tracking markers in a first configuration;

[0028] Fig.14D Shown from Fig. 14B A template for tracking markers in a second configuration;

[0029] Fig.15A An alternative version of the end effector is shown with only a single tracking marker attached;

[0030] Fig. 15B Show Fig.15A an end effector, wherein the instrument is disposed through the guide tube;

[0031] Fig. 15C Show Fig.15A an end effector of the invention, wherein the instrument is in two different positions, and resulting logic for determining whether the instrument is positioned within or outside of the guide tube;

[0032] Fig.15D Show Fig.15A an end effector with an instrument at two different frames in a guide tube and its relative distance from a single tracking marker on the guide tube;

[0033] Fig.15E Shows relative to the coordinate system Fig.15A The end effector;

[0034] Fig.16 is a block diagram of a method for navigating and moving an end effector of a robot to a desired target trajectory;

[0035] Figures 17A-17B depicts an apparatus for inserting an expandable implant having fixed and removable tracking markers in collapsed and expanded positions, respectively;

[0036] Figures 18A-18B depicts instruments for inserting an articulated implant having fixed and removable tracking markers in inserted and angulated positions, respectively;

[0037] Fig.19A Describing embodiments of robots having interchangeable or alternative end effectors;

[0038] Fig.19B An embodiment of a robot having an instrument-type end effector coupled thereto is depicted;

[0039] Fig. 20 is a photograph showing a fixture containing multiple BB / datum planes;

[0040] Fig.21 is a schematic diagram illustrating a pinhole camera model;

[0041] Fig. 22 is a diagram illustrating a configuration of a fluoroscopic x-ray imaging system according to some embodiments of the present inventive concept;

[0042] Fig.23A , Fig. 23B and Fig.23C is a schematic diagram illustrating a gravity vector according to some embodiments of the present inventive concept;

[0043] Fig.24A and Fig. 24B is a photograph illustrating gravity vectors according to some embodiments of the inventive concept;

[0044] Fig.25 and Fig.26 is a flow chart illustrating calibration operations according to some embodiments of the inventive concept;

[0045] Fig.27A , Fig.27B and Fig.27C is a photograph illustrating a fixture that can be used for calibration according to some embodiments of the inventive concept;

[0046] Fig.28 FIG. 1 is a diagram showing some embodiments of the present invention. Fig.25 a photograph of the reference fluoroscopic image / snapshot in question;

[0047] Fig.29A , Fig.29B and Fig.29C is a photograph illustrating the use of BB / fiducials in a fixture to provide up / down / left / right alignment according to some embodiments of the present inventive concept;

[0048] Fig.30 depicts the use of a crosshair as a reference according to some embodiments of the inventive concept;

[0049] Fig.31 depicts the use of monitoring markers on a fluoroscopic x-ray imaging system according to some embodiments of the present inventive concepts;

[0050] Fig.32 is a flow chart illustrating the use of a calibration fixture according to some embodiments of the inventive concept;

[0051] Fig.33 is a schematic diagram illustrating a fixing device for an x-ray detector according to some embodiments of the present inventive concept;

[0052] Fig.34 is a block diagram illustrating a fixture controller according to some embodiments of the inventive concept;

[0053] Fig.35 is a block diagram illustrating a robot controller according to some embodiments of the inventive concept.

[0054] Fig.36A and Fig.36B is a flow chart illustrating the operation of a fixing device according to some embodiments of the present inventive concept;

[0055] Fig.37is a flow chart illustrating the operation of a robotic system according to some embodiments of the inventive concept;

[0056] Fig.38A and Fig.38B Some embodiments of the present invention are shown in FIG. Fig.33 a plan view of the fixtures;

[0057] Fig.39 is a plan view of a fluoroscopic x-ray imaging system including a mounting socket for a fixture according to some embodiments of the present inventive concepts;

[0058] Fig.40 is a plan view of a fixture mounted on an x-ray detector of a fluoroscopic x-ray imaging system according to some embodiments of the present inventive concepts;

[0059] Fig.41 is a plan view of a fixture and a temporary radiopaque fiducial pattern mounted on an x-ray detector of a fluoroscopic x-ray imaging system according to some embodiments of the inventive concepts;

[0060] Fig.42 is an enlarged plan view of an x-ray detector of a fluoroscopic x-ray imaging system including a mounting socket according to some embodiments of the inventive concepts;

[0061] Fig.43 is an enlarged plan view of a fixture mounted on an x-ray detector of a fluoroscopic x-ray imaging system according to some embodiments of the present inventive concepts;

[0062] Fig.44 is an enlarged plan view of a fixture and a temporary radiopaque fiducial pattern mounted on an x-ray detector of a fluoroscopic x-ray imaging system according to some embodiments of the inventive concepts;

[0063] Fig.45A , Fig.45B and Fig.45C An enlarged plan view of a fixture mounted on an x-ray detector of a fluoroscopic imaging system, a sterile drape covering the fixture and the x-ray detector, and the application of tracking markers on / through the sterile drape is depicted in accordance with some embodiments of the concepts of the present invention. DETAILED DESCRIPTION

[0064] It should be understood that the present disclosure is not limited in its application to the details of the construction and arrangement of the components set forth in the description herein or illustrated in the accompanying drawings. The teachings of the present disclosure may be used and practiced in other embodiments, and practiced or implemented in various ways. Also, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of "includes," "comprising," or "having" and their variations herein is intended to cover the items listed thereafter and their equivalents and additional items. Unless otherwise specified or limited, the terms "install," "connect," "support," and "couple" and their variations are widely used and cover both direct and indirect installations, connections, supports, and couplings. In addition, "connect" and "couple" are not limited to physical or mechanical connections or couplings.

[0065] The following discussion is presented to enable those skilled in the art to make and use embodiments of the present disclosure. Those skilled in the art will readily appreciate various modifications to the illustrated embodiments, and without departing from the embodiments of the present disclosure, the principles herein may be applied to other embodiments and applications. Therefore, the embodiments are not intended to be limited to the embodiments shown, but should be given the widest scope consistent with the principles and features disclosed herein. The following detailed description should be read with reference to the accompanying drawings, in which similar elements in different figures have similar reference numerals. The figures, which are not necessarily to scale, depict the selected embodiments and are not intended to limit the scope of the embodiments. Those skilled in the art will recognize that the embodiments provided herein have many useful alternatives and fall within the scope of the embodiments.

[0066] Now turning to the accompanying drawings, Figure 1 and Figure 2 A surgical robotic system 100 is shown according to an exemplary embodiment. The surgical robotic system 100 may include, for example, a surgical robot 102, one or more robotic arms 104, a base 106, a display 110, an end effector 112, such as including a guide tube 114, and one or more tracking markers 118. The surgical robotic system 100 may include a patient tracking device 116, which also includes one or more tracking markers 118, which are suitable for being directly fixed to a patient 210 (e.g., fixed to a bone of the patient 210). The surgical robotic system 100 may also use, for example, a camera 200 positioned on a camera support 202. The camera support 202 may have any suitable configuration to move, orient, and support the camera 200 in a desired position. The camera 200 may include any suitable one or more cameras, such as one or more infrared cameras (e.g., bifocal or stereo photogrammetry cameras) that are capable of identifying, for example, active and passive tracking markers 118 (shown as Figure 2 A portion of the patient tracking device 116 in Figures 13A-13B ). The camera 200 can scan a given measurement volume and detect light from the markers 118 to identify and determine the location of the markers 118 in three dimensions. For example, active markers 118 can include infrared emitting markers activated by an electrical signal (e.g., an infrared light emitting diode (LED)), and / or passive markers 118 can include retroreflective markers that reflect infrared light (e.g., they reflect incoming IR radiation into the direction of the incident light), such as emitted by an illuminator on the camera 200 or other suitable device.

[0067] Figure 1 and Figure 2 Possible configurations for placing the surgical robotic system 100 in an operating room environment are shown. For example, the robot 102 can be located near or beside the patient 210. Although depicted near the head of the patient 210, it should be understood that the robot 102 can be located in any suitable position near the patient 210, depending on the area of ​​the patient 210 being operated on. The camera 200 can be separate from the robotic system 100 and positioned at the foot of the patient 210. The position allows the camera 200 to have a direct line of sight to the surgical field of view 208. Again, it is contemplated that the camera 200 can be located in any suitable position with a line of sight to the surgical field of view 208. In the configuration shown, the surgeon 120 can be positioned opposite the robot 102, but still be able to manipulate the end effector 112 and the display 110. The surgical assistant 126 can again be positioned opposite the surgeon 120 and can reach both the end effector 112 and the display 110. If desired, the positions of the surgeon 120 and the assistant 126 can be reversed. The traditional area for the anesthesiologist 122 and nurse or scrub technician 124 may remain unobstructed by the location of the robot 102 and camera 200 .

[0068] With respect to other components of the robot 102, the display 110 may be attached to the surgical robot 102, and in other exemplary embodiments, the display 110 may be separate from the surgical robot 102, whether in an operating room with the surgical robot 102 or at a remote location. The end effector 112 may be coupled to the robot arm 104 and controlled by at least one motor. In an exemplary embodiment, the end effector 112 may include a guide tube 114 that is capable of receiving and directing a surgical instrument 608 (described further herein) for performing a surgical procedure on a patient 210. As used herein, the term "end effector" is used interchangeably with the terms "end implementer" and "implementer element". Although generally shown with a guide tube 114, it should be understood that the end effector 112 may be replaced with any suitable instrument suitable for surgical procedures. In some embodiments, the end effector 112 may include any known structure for achieving movement of the surgical instrument 608 in a desired manner.

[0069] The surgical robot 102 is capable of controlling the translation and orientation of the end effector 112. The robot 102 is capable of moving the end effector 112, for example, along the x-axis, y-axis, and z-axis. The end effector 112 can be configured for selective rotation around one or more of the x-axis, y-axis, and z-axis and the Z-frame axis (so that one or more Euler angles (e.g., roll, pitch, and / or yaw) associated with the end effector 112 can be selectively controlled). In some exemplary embodiments, the selective control of the translation and orientation of the end effector 112 can allow the performance of a medical procedure with significantly improved accuracy compared to a conventional robot using, for example, a six-degree-of-freedom robotic arm that includes only a rotation axis. For example, the surgical robotic system 100 can be used to operate on a patient 210, and the robotic arm 104 can be positioned above the body of the patient 210, with the end effector 112 selectively angled toward the body of the patient 210 relative to the z-axis.

[0070] In some exemplary embodiments, the position of the surgical instrument 608 can be dynamically updated so that the surgical robot 102 can always know the position of the surgical instrument 608 during surgery. Therefore, in some exemplary embodiments, the surgical robot 102 can quickly move the surgical instrument 608 to the desired position without any further assistance from the physician (unless the physician requires it). In some further embodiments, the surgical robot 102 can be configured to correct the path of the surgical instrument 608 if the surgical instrument 608 deviates from the selected pre-planned trajectory. In some exemplary embodiments, the surgical robot 102 can be configured to allow stopping, modifying and / or manually controlling the movement of the end effector 112 and / or the surgical instrument 608. Therefore, in an exemplary embodiment, in use, a physician or other user can operate the system 100 and can choose to stop, modify or manually control the autonomous movement of the end effector 112 and / or the surgical instrument 608. Additional details of the surgical robotic system 100 , including control and movement of the surgical instrument 608 by the surgical robot 102 , may be found in co-pending US Pat. No. 9,782,229 , the entire disclosure of which is incorporated herein by reference.

[0071] The robotic surgical system 100 may include one or more tracking markers 118 configured to track the movement of the robotic arm 104, the end effector 112, the patient 210, and / or the surgical instrument 608 in three dimensions. In an exemplary embodiment, a plurality of tracking markers 118 may be mounted (or otherwise affixed) on an external surface of the robot 102, such as, but not limited to, the base 106 of the robot 102, the robotic arm 104, and / or the end effector 112. In an exemplary embodiment, at least one tracking marker 118 of the plurality of tracking markers 118 may be mounted or otherwise affixed to the end effector 112. One or more tracking markers 118 may further be mounted (or otherwise affixed) to the patient 210. In an exemplary embodiment, the plurality of tracking markers 118 may be located on the patient 210 spaced apart from the surgical field 208 to reduce the possibility of being obstructed by the surgeon, surgical tools, or other parts of the robot 102. In addition, one or more tracking markers 118 may also be mounted (or otherwise affixed) to a surgical tool 608 (e.g., a screwdriver, dilator, implant inserter, etc.). Thus, the tracking markers 118 enable the robot 102 to track each of the tagged objects (e.g., the end effector 112, the patient 210, and the surgical tool 608). In an exemplary embodiment, the system 100 may use the tracking information collected from each tagged object to calculate, for example, the orientation and position of the end effector 112, the surgical instrument 608 (e.g., positioned in the tube 114 of the end effector 112), and the relative position 210 of the patient.

[0072] The markers 118 may include radiopaque or optical markers. The markers 118 may be in suitable shapes, including spherical, spherical, cylindrical, cubic, cuboid, etc. In an exemplary embodiment, one or more of the markers 118 may be optical markers. In some embodiments, the positioning of one or more tracking markers 118 on the end effector 112 can increase / maximize the accuracy of position measurement by being used to check or verify the position of the end effector 112. Further details of the surgical robot system 100 including the control, movement and tracking of the surgical robot 102 and the surgical instrument 608 can be found in U.S. Patent Publication No. 2016 / 0242849, the entire disclosure of which is incorporated herein by reference.

[0073] The exemplary embodiment includes one or more markers 118 coupled to the surgical instrument 608. In the exemplary embodiment, for example, these markers 118 coupled to the patient 210 and the surgical instrument 608, as well as the markers 118 coupled to the end effector 112 of the robot 102 may include conventional infrared light emitting diodes (LEDs) or may be capable of being tracked using a commercially available infrared optical tracking system (e.g., )Tracking diode. is a registered trademark of Northern Digital Inc. of Waterloo, Ontario, Canada. In other embodiments, the marker 118 may include a conventional reflective sphere that can be tracked using a commercially available optical tracking system (e.g., Polaris Spectra). Polaris Spectra is also a registered trademark of Northern Digital. In an exemplary embodiment, the marker 118 coupled to the end effector 112 is an active marker that includes an infrared light emitting diode that can be turned on and off, and the marker 118 coupled to the patient 210 and the surgical instrument 608 includes a passive reflective sphere.

[0074] In an exemplary embodiment, light emitted and / or reflected by the marker 118 can be detected by the camera 200 and can be used to monitor the position and movement of the tagged object. In alternative embodiments, the marker 118 can include a radio frequency and / or electromagnetic reflector or transceiver, and the camera 200 can include or be replaced by a radio frequency and / or electromagnetic transceiver.

[0075] Similar to the surgical robot system 100, Figure 3 A surgical robot system 300 and a camera support 302 are shown in a docked configuration consistent with an exemplary embodiment of the present disclosure. The surgical robot system 300 may include a robot 301 including a display 304, an upper arm 306, a lower arm 308, an end effector 310, a vertical column 312, casters 314, a cabinet 316, a flat drawer 318, a connector panel 320, a control panel 322, and an information ring 324. The camera support 302 may include a camera 326. These components are referred to as Figure 5 Describe in more detail. Figure 3 The surgical robotic system 300 is shown in a docked configuration, with the camera support 302 nested with the robot 301, for example when not in use. Those skilled in the art will recognize that the camera 326 and the robot 301 can be separated from each other during a surgical procedure and positioned in any suitable position, such as Figure 1 and 2 shown.

[0076] Figure 4 A base 400 consistent with an exemplary embodiment of the present disclosure is shown. The base 400 may be part of the surgical robot system 300 and include a cabinet 316. The cabinet 316 may house certain components of the surgical robot system 300, including but not limited to a battery 402, a power distribution module 404, a platform interface board module 406, a computer 408, a handle 412, and a flat panel drawer 414. The connections and relationships between these components refer to Figure 5 Describe in more detail.

[0077] Figure 5 A block diagram of certain components of an exemplary embodiment of a surgical robot system 300 is shown. The surgical robot system 300 may include a platform subsystem 502, a computer subsystem 504, a motion control subsystem 506, and a tracking subsystem 532. The platform subsystem 502 may further include a battery 402, a power distribution module 404, a platform interface board module 406, and a tablet charging station 534. The computer subsystem 504 may further include a computer 408, a display 304, and a speaker 536. The motion control subsystem 506 may further include a driver circuit 508, motors 510, 512, 514, 516, 518, stabilizers 520, 522, 524, 526, an end effector 310, and a controller 538. The tracking subsystem 532 may further include a position sensor 540 and a camera converter 542. The system 300 may also include a foot pedal 544 and a tablet 546.

[0078] Input power is supplied to the system 300 via a power supply 548 that may be provided to the power distribution module 404. The power distribution module 404 receives the input power and is configured to generate different supply voltages that are provided to other modules, components, and subsystems of the system 300. The power distribution module 404 may be configured to provide different voltage supplies to the platform interface module 406 that may be provided to other components such as the computer 408, the display 304, the speaker 536, the drive 508, etc., to power, for example, the motors 512, 514, 516, 518 and the end effector 310, the motor 510, the ring 324, the camera converter 542, and other components for the system 300, such as fans for cooling electrical components within the cabinet 316.

[0079] The power distribution module 404 may also provide power to other components, such as a tablet charging station 534 that may be located within the tablet drawer 318. The tablet charging station 534 may communicate wirelessly or wired with the tablet 546 for charging the tablet 546. The tablet 546 may be used by a surgeon consistent with the present disclosure and as described herein.

[0080] The power distribution module 404 may also be connected to the battery 402, which serves as a temporary power source in the event that the power distribution module 404 is not receiving power from the input power 548. At other times, the power distribution module 404 may be used to charge the battery 402 if necessary.

[0081] Other components of the platform subsystem 502 may also include a connector panel 320, a control panel 322, and a ring 324. The connector panel 320 may be used to connect different devices and components to the system 300 and / or associated components and modules. The connector panel 320 may include one or more ports that receive wires or connections from different components. For example, the connector panel 320 may have a ground terminal port that can ground the system 300 to other devices, a port to connect a foot pedal 544 to the system 300, a port to connect to a tracking subsystem 532, which may include a position sensor 540, a camera converter 542, and a camera 326 associated with the camera bracket 302. The connector panel 320 may also include other ports to allow USB, Ethernet, HDMI communication with other components (e.g., computer 408).

[0082] The control panel 322 may provide various buttons or indicators for controlling the operation of the system 300 and / or providing information about the system 300. For example, the control panel 322 may include buttons for powering the system 300 on or off, raising or lowering the vertical column 312, and raising or lowering the stabilizers 520-526, which may be designed to engage the casters 314 to lock the system 300 from physical movement. Other buttons may stop the system 300 in an emergency, which may remove all motor power and apply mechanical brakes to prevent all movement from occurring. The control panel 322 may also have indicators for notifying the user of certain system conditions, such as a line power indicator or the charge status of the battery 402.

[0083] Ring 324 may be a visual indicator to inform a user of system 300 of the different modes in which system 300 is operating as well as certain warnings to the user.

[0084] The computer subsystem 504 includes a computer 408, a display 304, and a speaker 536. The computer 504 includes an operating system and software for operating the operating system 300. The computer 504 can receive and process information from other components (e.g., the tracking subsystem 532, the platform subsystem 502, and / or the motion control subsystem 506) to display information to the user. In addition, the computer subsystem 504 may also include a speaker 536 to provide audio to the user.

[0085] Tracking subsystem 532 may include position sensor 504 and transducer 542. Tracking subsystem 532 may correspond to camera support 302 including camera 326, as shown in FIG. Figure 3The position sensor 504 may be a camera 326. The tracking subsystem may track the position of certain markers located on different components of the system 300 and / or on instruments used by a user during a surgical procedure. This tracking may be performed in a manner consistent with the present disclosure, including the use of infrared technology that tracks the position of active or passive elements such as LEDs or reflective markers, respectively. The location, orientation, and position of structures having these types of markers may be provided to the computer 408 and may be presented to the user on the display 304. For example, a surgical instrument 608 having these types of markers and tracked in this manner (which may be referred to as a navigation space) may present a three-dimensional image of the associated patient's anatomical structure to the user.

[0086] The motion control subsystem 506 can be configured to physically move the vertical column 312, the upper arm 306, the lower arm 308, or the rotary end effector 310. The physical movement can be performed by using one or more motors 510-518. For example, the motor 510 can be configured to vertically raise or lower the vertical column 312. Figure 3 As shown, the motor 512 can be configured to move the upper arm 308 laterally about the point of engagement with the vertical column 312. Figure 3 As shown, motor 514 can be configured to move lower arm 308 laterally about a point of engagement with upper arm 308. Motors 516 and 518 can be configured to move end effector 310 in a controllable tilt and controllable tilt manner, thereby providing a plurality of angles at which end effector 310 can be moved. These movements can be accomplished by controller 538, which can control these movements to move system 300 in a desired manner via load cells disposed on end effector 310 and activated by a user engaging these load cells.

[0087] In addition, the system 300 can provide for automatic movement of the vertical column 312, upper arm 306, and lower arm 308 by the user indicating on the display 304 (which can be a touch screen input device) the position of the surgical instrument or assembly on the three-dimensional image of the patient's anatomy on the display 304. The user can initiate this automatic movement by stepping on the foot pedal 544 or some other input device.

[0088] Figure 6A surgical robotic system 600 consistent with an exemplary embodiment is shown. The surgical robotic system 600 may include an end effector 602, a robotic arm 604, a guide tube 606, an instrument 608, and a robotic base 610. The instrument tool 608 may be attached to a tracking array 612 containing one or more tracking markers (e.g., marker 118) and have an associated trajectory 614. The trajectory 614 may represent a movement path that the instrument tool 608 is configured to travel once it passes through or is secured in the guide tube 606, such as a path that the instrument tool 608 is inserted into a patient's body. In exemplary operation, the robotic base 610 may be configured to electronically communicate with the robotic arm 604 and the end effector 602 so that the surgical robotic system 600 can assist a user (e.g., a surgeon) in operating on a patient 210. The surgical robotic system 600 may be consistent with the surgical robotic systems 100 and 300 described previously.

[0089] Tracking array 612 can be mounted on instrument 608 to monitor the position and orientation of instrument tool 608. Tracking array 612 can be attached to instrument 608 and can include tracking markers 804. Figure 8 As best seen in FIG. 1 , the tracking marker 804 may be, for example, a light emitting diode and / or other type of reflective marker (e.g., marker 118 as described elsewhere herein). The tracking device may be one or more line of sight devices associated with the surgical robotic system. As an example, the tracking device may be one or more cameras 200, 326 associated with the surgical robotic system 100, 300, and may also track the tracking array 612 to obtain a defined domain or relative orientation of the instrument 608 relative to the robotic arm 604, the robotic base 610, the end effector 602, and / or the patient 210. The tracking device may be consistent with those structures described in conjunction with the camera support 302 and the tracking subsystem 532.

[0090] Fig. 7A , 7B7C show top, front, and side views, respectively, of an end effector 602 consistent with an exemplary embodiment. The end effector 602 may include one or more tracking markers 702. The tracking markers 702 may be light emitting diodes or other types of active and passive markers, such as the tracking markers 118 described previously. In an exemplary embodiment, the tracking markers 702 are active infrared emitting markers that are activated by an electrical signal, such as an infrared light emitting diode (LED). Thus, the tracking markers 702 may be activated such that the infrared markers 702 are visible to the camera 200, 326, or may be deactivated such that the infrared markers 702 are not visible to the camera 200, 326. Thus, when the markers 702 are active, the end effector 602 may be controlled by the system 100, 300, 600, and when the markers 702 are deactivated, the end effector 602 may be locked in place and cannot be moved by the system 100, 300, 600.

[0091] The marker 702 may be disposed on or within the end effector 602 in a manner such that the marker 702 may be visible to one or more cameras 200, 326 or other tracking devices associated with the surgical robotic system 100, 300, 600. The camera 200, 326 or other tracking device may track the end effector 602 as it moves to different positions and viewing angles by following the movement of the tracking marker 702. The position of the marker 702 and / or the end effector 602 may be displayed on a display 110, 304 associated with the surgical robotic system 100, 300, 600, for example. Figure 2 The display 110 and / or Figure 3 The display 304 is shown. This display 110, 304 can allow the user to ensure that the end effector 602 is in a desired position relative to the robotic arm 604, the robotic base 610, the patient 210 and / or the user.

[0092] For example, if Fig. 7A As shown, the markers 702 can be placed around the surface of the end effector 602 so that a tracking device placed away from the surgical field of view 208 and facing the robot 102, 301 and the camera 200, 326 can view at least three of the markers 702 through a series of common orientations of the end effector 602 relative to the tracking device. For example, distributing the markers 702 in this manner allows the end effector 602 to be monitored by the tracking device as it translates and rotates in the surgical field of view 208.

[0093] Additionally, in an exemplary embodiment, the end effector 602 may be equipped with an infrared (IR) receiver that can detect when the external camera 200, 326 is ready to read the marker 702. Upon this detection, the end effector 602 may then illuminate the marker 702. Detection by the IR receiver that the external camera 200, 326 is ready to read the marker 702 may signal the need to synchronize the duty cycle of the marker 702 (which may be a light emitting diode) to the external camera 200, 326. This may also allow the robotic system as a whole to achieve lower power consumption, whereby the marker 702 will only be illuminated at appropriate times rather than being illuminated continuously. Additionally, in an exemplary embodiment, the marker 702 may be turned off to prevent interference with other navigation tools (e.g., different types of surgical instruments 608).

[0094] Figure 8 A type of surgical instrument 608 is depicted that includes a tracking array 612 and a tracking marker 804. The tracking marker 804 can be of any type described herein, including but not limited to a light emitting diode or a reflective ball. The marker 804 is monitored by a tracking device associated with the surgical robotic system 100, 300, 600, and the marker can be one or more of the line-of-sight cameras 200, 326. The cameras 200, 326 can track the positioning of the instrument 608 based on the position and orientation of the tracking array 612 and the marker 804. A user, such as a surgeon 120, can orient the instrument 608 in a manner such that the tracking array 612 and the marker 804 are sufficiently discernible by the tracking device or camera 200, 326 to display the instrument 608 and the marker 804, for example, on the display 110 of the exemplary surgical robotic system.

[0095] Surgeon 120 can place instrument 608 into guide tube 606 of end effector 602 and adjust instrument 608 in this way. Figure 8606 is obvious. The size and configuration of the hollow tube or conduit 114, 606 of the end effector 112, 310, 602 is set to receive at least a portion of the surgical instrument 608. The conduit 114, 606 is configured to be oriented by the robot arm 104 so that the insertion and trajectory of the surgical instrument 608 can reach the desired anatomical target in or on the patient 210. The surgical instrument 608 may include at least a portion of a generally cylindrical instrument. Although a screwdriver is used as an example as a surgical tool 608, it should be understood that any suitable surgical tool 608 can be positioned by the end effector 602. For example, the surgical instrument 608 may include one or more of a guide wire, a cannula, a retractor, a drill, a reamer, a screwdriver, an insertion tool, a removal tool, etc. Although the hollow tube 114, 606 is generally shown as having a cylindrical configuration, it will be understood by those skilled in the art that the conduit 114, 606 can have any suitable shape, size and configuration desired to accommodate the surgical instrument 608 and access the surgical site.

[0096] 9A to 9C A portion of an end effector 602 and a robotic arm 604 are shown consistent with an exemplary embodiment. The end effector 602 may further include a body 1202 and a clamp 1204. The clamp 1204 may include a handle 1206, a ball 1208, a spring 1210, and a lip 1212. The robotic arm 604 may further include a recess 1214, a mounting plate 1216, a lip 1218, and a magnet 1220.

[0097] The end effector 602 can mechanically interface and / or engage the surgical robot system and the robotic arm 604 through one or more couplings. For example, the end effector 602 can be engaged with the robotic arm 604 through a positioning coupling and / or a reinforcement coupling. Through these couplings, the end effector 602 can be fastened to the robotic arm 604 outside the flexible sterile barrier. In an exemplary embodiment, the positioning coupling can be a magnetic kinematic bracket, and the reinforcement coupling can be a five-bar cross-center clamp linkage.

[0098] With respect to the positioning coupling, the robotic arm 604 may include a mounting plate 1216 (which may be a non-magnetic material), one or more recesses 1214, a lip 1218, and magnets 1220. The magnets 1220 are mounted below each recess 1214. Portions of the fixture 1204 may include a magnetic material and be attracted by the one or more magnets 1220. Through the magnetic attraction of the fixture 1204 and the robotic arm 604, the ball 1208 becomes seated in the corresponding recess 1214. For example, as Fig. 9B The ball 1208 shown will be placed in the recess 1214, as shown in FIG. Fig. 9AAs shown. This arrangement can be considered a magnetically assisted kinematic coupling. The magnets 1220 can be configured to be strong enough to support the entire weight of the end effector 602 regardless of the orientation of the end effector 602. The positioning coupling can be any type of kinematic mounting that uniquely constrains six degrees of freedom.

[0099] With respect to the enhanced coupling, portions of the clamp 1204 can be configured as fixed ground linkages, and such a clamp 1204 can be used as a five-bar linkage. When the lip 1212 and the lip 1218 engage the clamp 1204 in a manner that secures the end effector 602 and the robotic arm 604, closing the clamp handle 1206 can secure the end effector 602 to the robotic arm 604. When the clamp handle 1206 is closed, the spring 1210 can be stretched or stressed when the clamp 1204 is in the locked position. The locked position can be a position that provides a linkage that passes through the center. Because the closed position passes through the center, the connection will not open without a force applied to the clamp handle 1206 to release the clamp 1204. Therefore, in the locked position, the end effector 602 can be securely secured to the robotic arm 604.

[0100] The spring 1210 may be a tensioned bending beam. The spring 1210 may be composed of a material exhibiting high stiffness and high yield strain, such as virgin PEEK (polyetheretherketone). The connection between the end effector 602 and the robotic arm 604 may provide a sterile barrier between the end effector 602 and the robotic arm 604 without interfering with the fastening of the two couplings.

[0101] The reinforced coupling may be a linkage with multiple spring members. The reinforced coupling may lock with a cam or friction based mechanism. The reinforced coupling may also be a sufficiently powerful electromagnet that will support the fastening of the end effector 102 to the robotic arm 604. The reinforced coupling may be a multi-piece collar that is completely separate from the end effector 602 and / or robotic arm 604 that slides over the interface between the end effector 602 and the robotic arm 604 and is fastened with a screw mechanism, a cross-center connector, or a cam mechanism.

[0102] refer to Fig.10 and Fig.11 Before or during the surgical procedure, certain registration procedures may be performed to track the target anatomy of the object and patient 210 in both navigation space and image space. To perform such registration, a system such as a Fig.10 Registration system 1400 is depicted.

[0103] To track the position of the patient 210, the patient tracking device 116 may include a patient fixation device 1402 to be fixed to a rigid anatomical structure of the patient 210, and a dynamic reference base (DRB) 1404 may be securely attached to the patient fixation device 1402. For example, the patient fixation device 1402 may be inserted into an opening 1406 of the dynamic reference base 1404. The dynamic reference base 1404 may include markers 1408 that are visible to the tracking device (e.g., the tracking subsystem 532). These markers 1408 may be optical markers or reflective balls, such as the tracking markers 118, as previously described.

[0104] Patient fixation apparatus 1402 is attached to a rigid anatomical structure of patient 210 and may remain attached throughout the surgical procedure. In an exemplary embodiment, patient fixation apparatus 1402 is attached to a rigid area of ​​patient 210, such as a bone located away from a target anatomical structure undergoing the surgical procedure. To track the target anatomical structure, dynamic reference base 1404 is associated with the target anatomical structure by using a registration fixture temporarily placed on or near the target anatomical structure to register dynamic reference base 1404 with the location of the target anatomical structure.

[0105] The registration fixture 1410 is attached to the patient fixation apparatus 1402 by using the pivot arm 1412. The pivot arm 1412 is attached to the patient fixation apparatus 1402 by inserting the patient fixation apparatus 1402 through an opening 1414 of the registration fixture 1410. The pivot arm 1412 is attached to the registration fixture 1410 by, for example, inserting a knob 1416 through an opening 1418 of the pivot arm 1412.

[0106] Using the pivot arm 1412, the registration fixture 1410 can be placed over the target anatomy and its position can be determined in imaging space and navigation space using tracking markers 1420 and / or fiducials 1422 on the registration fixture 1410. The registration fixture 1410 can include a collection of markers 1420 that are visible in the navigation space (e.g., the markers 1420 can be detected by the tracking subsystem 532). As previously described herein, the tracking markers 1420 can be optical markers that are visible under infrared light. The registration fixture 1410 can also include a collection of fiducials 1422 that are visible in the imaging space (e.g., a three-dimensional CT image), such as bearing balls. As described with respect to Fig.11 In more detail, using the registration fixture 1410, the target anatomy can be associated with the dynamic reference base 1404, allowing the depiction of the object in the navigation space to be superimposed on the image of the anatomy. The dynamic reference base 1404 located at a location away from the target anatomy can become a reference point, allowing the registration fixture 1410 and / or the pivot arm 1412 to be removed from the surgical field.

[0107] Fig.11 An exemplary method 1500 for registration consistent with the present disclosure is provided. The method 1500 begins at step 1502, where a graphical representation (or image) of a target anatomical structure can be imported into the system 100, 300, 600, such as the computer 408. The graphical representation can be a three-dimensional CT or fluoroscopic scan of the target anatomical structure of the patient 210, which contains a detectable imaging pattern of the registration fixture 1410 and the fiducials 1420.

[0108] At step 1504, the imaged pattern of fiducials 1420 is detected and registered in imaging space and stored in computer 408. Optionally, at this point at step 1506, a graphical representation of registration fixture 1410 may be superimposed on the image of the target anatomy.

[0109] At step 1508, the navigation pattern of the registration fixture 1410 is detected and registered by identifying the marker 1420. The marker 1420 can be an optical marker that is recognized in the navigation space by the tracking subsystem 532 via the position sensor 540 through infrared light. Therefore, the location, orientation, and other information of the target anatomical structure are registered in the navigation space. Therefore, the registration fixture 1410 can be recognized in the imaging space by using the reference point 1422, and the registration fixture can be recognized in the navigation space by using the marker 1420. At step 1510, the registration of the registration fixture 1410 in the imaging space is transferred to the navigation space. For example, this transfer is completed by using the relative position of the imaging pattern of the reference point 1422 compared to the position of the navigation pattern of the marker 1420.

[0110] At step 1512, the registration of the navigation space of the registration fixture 1410 (already registered with the imaging space) is further transferred to the navigation space of the dynamic registration array 1404 attached to the patient fixation apparatus 1402. Thus, the registration fixture 1410 can be removed, and because the navigation space is associated with the imaging space, the dynamic reference base 1404 can be used to track the target anatomy in both the navigation space and the imaging space.

[0111] At steps 1514 and 1516, the navigation space may be superimposed on the imaging space, and the object may have a marker visible in the navigation space (e.g., surgical instrument 608 with optical marker 804). The object may be tracked by the graphical representation of surgical instrument 608 on the image of the target anatomy.

[0112] Figures 12A-12BAn imaging device 1304 is shown that can be used in conjunction with the robotic system 100, 300, 600 to obtain pre-operative, intra-operative, post-operative, and / or real-time image data of the patient 210. Using the imaging system 1304, any suitable subject can be imaged for any suitable procedure. The imaging system 1304 can be any imaging device, such as an imaging device 1306 and / or a C-arm 1308 device. It may be desirable to take x-rays of the patient 210 from a plurality of different positions without requiring frequent manual repositioning of the patient 210 that may be required in an x-ray system. Fig. 12A As shown, the imaging system 1304 may be in the form of a C-arm 1308 that includes an elongated C-shaped member terminating at a relatively distal end 1312 of the "C" shape. The C-shaped member 1300 may also include an x-ray source 1314 and an x-ray detector 1316 (also referred to as an image receptor). The space within the C-arm 1308 of the arm may provide a physician with space to care for a patient without being substantially disturbed by an x-ray support structure 1318. Fig. 12B As shown, the imaging system may include an imaging device 1306 having a gantry housing 1324 attached to a support structure imaging device support structure 1328, such as a wheeled mobile cart 1330 with wheels 1332, which may surround an image capture portion (not shown). The image capture portion may include an x-ray source and / or emitting portion and an x-ray receiving and / or image receiving portion, which may be positioned approximately one hundred eighty degrees apart from each other and mounted on a rotor (not shown) relative to a track of the image capture portion. The image capture portion may be operable to rotate three hundred sixty degrees during image acquisition. The image capture portion may rotate about a center point and / or axis, thereby allowing image data of the patient 210 to be acquired from multiple directions or multiple planes. Although certain imaging systems 1304 are exemplified herein, it should be understood that a person of ordinary skill in the art may select any suitable imaging system.

[0113] Now turn to see Figures 13A-13C The surgical robotic system 100, 300, 600 relies on accurate positioning of the end effector 112, 602, surgical instrument 608, and / or patient 210 (e.g., patient tracking device 116) relative to a desired surgical area. Figures 13A-13C In the illustrated embodiment, the tracking markers 118 , 804 are rigidly attached to the instrument 608 and / or a portion of the end effector 112 .

[0114] Fig.13A A portion of a surgical robotic system 100 is depicted having a robot 102 including a base 106, a robotic arm 104, and an end effector 112. Other elements not shown, such as displays, cameras, etc., may also be present as described herein. Fig. 13BA close-up view of the end effector 112 is depicted with the catheter 114 and a plurality of tracking markers 118 rigidly secured to the end effector 112. In this embodiment, the plurality of tracking markers 118 are attached to the catheter 112. Fig. 13C An instrument 608 (in this case, stylet 608A) is depicted having a plurality of tracking markers 804 rigidly affixed to the instrument 608. As described elsewhere herein, the instrument 608 may include any suitable surgical instrument, such as, but not limited to, a guidewire, a cannula, a retractor, a drill, a reamer, a screwdriver, an insertion tool, a removal tool, and the like.

[0115] When tracking an instrument 608, an end effector 112, or other object to be tracked in 3D, an array of tracking markers 118, 804 can be rigidly attached to a portion of the tool 608 or end effector 112. Preferably, the tracking markers 118, 804 are attached so that the markers 118, 804 do not interfere (e.g., do not interfere with surgical manipulation, visibility, etc.). The markers 118, 804 can be attached to the instrument 608, the end effector 112, or other object to be tracked with the array 612, for example. Typically, three or four markers 118, 804 are used with the array 612. The array 612 may include linear segments, cross-pieces, and may be asymmetric so that the markers 118, 804 are in different relative positions and locations with respect to each other. For example, as Fig. 13C , a probe 608A having a 4-marker tracking array 612 is shown, and Fig. 13B An end effector 112 with a different 4-marker tracking array 612 is depicted.

[0116] exist Fig. 13C , the tracking array 612 serves as the handle 620 of the probe 608A. Thus, the four markers 804 are attached to the handle 620 of the probe 608A, out of the way of the shaft 622 and the tip 624. Stereophotogrammetric tracking of these four markers 804 allows the instrument 608 to be tracked as a rigid body and allows the tracking system 100, 300, 600 to accurately determine the position of the tip 624 and the orientation of the shaft 622 as the probe 608A moves around in front of the tracking camera 200, 326.

[0117] In order to be able to automatically track one or more tools 608, end effectors 112, or other objects to be tracked in 3D (e.g., multiple rigid bodies), the markers 118, 804 on each tool 608, end effector 112, or the like are arranged asymmetrically at known marker spacings. The reason for the asymmetrical alignment is to make it clear which marker 118, 804 corresponds to a specific location on the rigid body and whether the marker 118, 804 is viewed from the front or back (i.e., mirrored). For example, if the markers 118, 804 are arranged in a square on the tool 608 or end effector 112, it will be unclear to the system 100, 300, 600 which marker 118, 804 corresponds to which corner of the square. For example, for the probe 608A, it is unclear which marker 804 is closest to the shaft 622. Therefore, the direction in which the shaft 622 extends from the array 612 is unknown. Therefore, each array 612, and therefore each tool 608, end effector 112, or other object to be tracked, should have a unique marker pattern to enable it to be distinguished from other tools 608 or other objects being tracked. The asymmetric and unique marker pattern allows the system 100, 300, 600 to detect individual markers 118, 804 and then check the marker intervals against a stored template to determine which tool 608, end effector 112, or other object they represent. The detected markers 118, 804 can then be automatically classified and assigned to each tracked object in the correct order. Without this information, rigid body calculations cannot be performed to extract key geometric information, such as the alignment of the tool tip 624 and the shaft 622, unless the user manually specifies which detected marker 118, 804 corresponds to which location on each rigid body. These concepts are generally known to those skilled in 3D optical tracking methods.

[0118] Now turn to see Figures 14A to 14D , shows an alternative version of the end effector 912 with movable tracking markers 918A-918D. Fig.14A In FIG. 1 , an array with removable tracking marks 918A-918D is shown in a first configuration and in Fig. 14B , the removable tracking marks 918A-918D are shown in a second configuration that is angled relative to the first configuration. Fig. 14C For example, in Fig.14A a template of tracking markers 918A-918D as seen by cameras 200, 326 in a first configuration; and Fig.14D For example, in Fig. 14B A template of tracking markers 918A-918D as seen by camera 200, 326 in a second configuration.

[0119] In this embodiment, 4-marker array tracking is contemplated where the markers 918A-918D are not all located in fixed positions relative to a rigid body, and instead one or more of the array markers 918A-918D may be adjusted, for example during testing, to give updated information about the rigid body being tracked without hampering the process of automatic detection and classification of the tracked markers 918A-918D.

[0120] When tracking any tool, such as a guide tube 914 connected to an end effector 912 of a robotic system 100, 300, 600, the main purpose of the tracking array is to update the position of the end effector 912 in the camera coordinate system. Fig. 13B , an array 612 of reflective markers 118 rigidly extends from the catheter 114. Because the tracking markers 118 are rigidly attached, knowledge of the marker positions in the camera coordinate system also provides the accurate positions of the centerline, tip, and tail of the catheter 114 in the camera coordinate system. Typically, information about the position of the end effector 112 from this array 612 and information about the position of the target track from another tracked source are used to calculate the required movement that must be input for each axis of the robot 102 to move the catheter 114 into alignment with the track and move the tip to a specific position along the track vector.

[0121] Sometimes, a desired trajectory is in an inconvenient or inaccessible location, but if the guide tube 114 can be rotated, then the desired trajectory can be reached. For example, if the guide tube 114 can be rotated upward beyond the limits of the pitch (wrist up-down angle) axis, then a very steep trajectory pointing away from the base 106 of the robot 102 may be reachable, but if the guide tube 114 is attached parallel to the plate connecting it to the end of the wrist, then the very steep trajectory may not be reachable. In order to reach this trajectory, the base 106 of the robot 102 can be moved, or a different end effector 112 with a different guide tube attached can be swapped with the working end effector. Both solutions can be time consuming and cumbersome.

[0122] like Fig.14A and 14BAs best seen in FIG. 1 , if the array 908 is configured so that one or more of the markers 918A-918D are not in a fixed position, and instead, one or more of the markers 918A-918D can be adjusted, rotated, pivoted, or moved, the robot 102 can provide updated information about the object being tracked without hindering the detection and tracking process. For example, one of the markers 918A-918D can be fixed in place and the other markers 918A-918D can be movable; two of the markers 918A-918D can be fixed in place and the other markers 918A-918D can be movable; three of the markers 918A-918D can be fixed in place and the other markers 918A-918D can be movable; or all of the markers 918A-918D can be movable.

[0123] exist Fig.14A and 14B , markers 918A, 918B are rigidly connected directly to the base 906 of the end effector 912, and markers 918C, 918D are rigidly connected to the tube 914. Similar to the array 612, an array 908 can be provided to attach the markers 918A-918D to the end effector 912, the instrument 608, or other objects to be tracked. However, in this case, the array 908 is composed of multiple separate components. For example, the markers 918A, 918B can be connected to the base 906 by a first array 908A, and the markers 918C, 918D can be connected to the tube 914 by a second array 908B. The marker 918A can be attached to the first end of the first array 908A, and the marker 918B can be separated by a linear distance and attached to the second end of the first array 908A. While the first array 908 is generally linear, the second array 908B has a curved or V-shaped configuration with the respective root ends connected to the catheter 914 and radiating therefrom to the distal ends in a V-shape with the marker 918C at one distal end and the marker 918D at the other distal end. While specific configurations are exemplified herein, it should be understood that other asymmetric designs including different numbers and types of arrays 908A, 908B and different arrangements, numbers and types of markers 918A-918D are contemplated.

[0124] The catheter 914 may be movable, rotatable, or pivotable relative to the base 906, for example, across a hinge 920 or other connector to the base 906. Thus, the markers 918C, 918D are movable so that when the catheter 914 pivots, rotates, or moves, the markers 918C, 918D also pivot, rotate, or move. Fig.14A As best seen in FIG. 1 , the catheter 914 has a longitudinal axis 916 that is aligned in a generally orthogonal or vertical orientation such that the markers 918A-918D have a first configuration. Fig. 14B , the catheter 914 is pivoted, rotated or moved such that the longitudinal axis 916 is now angled relative to the vertical orientation such that the markings 918A-918D have a second configuration that is different from the first configuration.

[0125] Compared to Figures 14A-14D In the described embodiment, if there is a rotation between the guide tube 914 and the arm 104 (e.g., wrist attachment), where all four markers 918A-918D remain rigidly attached to the guide tube 914, and this rotation is adjusted by the user, then the robotic system 100, 300, 600 will not be able to automatically detect that the guide tube 914 orientation has changed. The robotic system 100, 300, 600 will track the position of the marker array 908 and will calculate the wrong robot axis movement assuming that the guide tube 914 is attached to the wrist (robotic arm 104) in the previous orientation. By keeping one or more markers 918A-918D (e.g., two markers 918C, 918D) rigidly on the tube 914 and keeping one or more markers 918A-918D (e.g., two markers 918A, 918B) across the rotation, automatic detection of the new position becomes possible and the correct robot movement is calculated based on the detection of the new tool or end effector 112, 912 on the end of the robotic arm 104.

[0126] One or more of the markers 918A-918D are configured to move, pivot, rotate, etc. according to any suitable manner. For example, the markers 918A-918D may be moved by a hinge 920 such as a clamp, spring, lever, slide, toggle, etc., or any other suitable mechanism for moving the markers 918A-918D individually or in combination, moving the arrays 908A, 908B individually or in combination, moving any portion of the end effector 912 relative to another portion, or moving any portion of the tool 608 relative to another portion.

[0127] like Fig.14A and 14B 908 and the conduit 914 can become reconfigurable by simply loosening a clamp or hinge 920, moving portions of the array 908A, 908B relative to other portions 908A, 908B, and re-tightening the hinge 920 so that the conduit 914 is oriented in a different position. For example, two markers 918C, 918D can be rigidly interconnected with the tube 914, and two markers 918A, 918B can be rigidly interconnected across a hinge 920 to a base 906 of an end effector 912 attached to the robotic arm 104. The hinge 920 can be in the form of a clamp, such as a wing nut, that can be loosened and re-tightened to allow the user to adjust the configuration in a first configuration ( Fig.14A ) and the second configuration ( Fig. 14B ) to quickly switch between them.

[0128] Cameras 200, 326 detect e.g. Fig. 14C and 14D If array 908 is in a first configuration ( Fig.14A ) and the tracking camera 200, 326 detects the markers 918A-918D, then the tracked markers are Fig. 14C 1 matches the array template 1 shown in FIG. If array 908 is the second configuration ( Fig. 14B ) and the tracking cameras 200, 326 detect the same markers 918A-918D, then the tracked markers are the same as Fig.14D Array Template 1 and Array Template 2 are recognized by the system 100, 300, 600 as two distinct tools, each with its own uniquely defined spatial relationship between the guide tube 914, markers 918A-918D, and robot attachment. The user can therefore adjust the position of the end effector 912 between the first configuration and the second configuration without notifying the system 100, 300, 600 of the change, and the system 100, 300, 600 will appropriately adjust the movement of the robot 102 to stay on track.

[0129] In this embodiment, there are two assembly locations where the marker array matches a unique template that allows the system 100, 300, 600 to recognize the assembly as two different tools or two different end effectors. Fig. 14C and 14D 1 and array template 2) shown in the figure, or in any position of external rotation, the markers 918A-918D will not match any template, and although the camera 200, 326 detects the individual markers 918A-918D, the system 100, 300, 600 will not detect that any array exists, which has the same result as if the markers 918A-918D were temporarily blocked from the field of view of the camera 200, 326. It should be understood that other array templates may exist for other configurations, such as to identify different instruments 608 or other end effectors 112, 912, etc.

[0130] In the described embodiment, Fig.14A and 14BTwo discrete assembly positions are shown. However, it should be appreciated that there may be multiple discrete positions on a rotational joint, linear joint, combination of rotational and linear joints, pegboard, or other assembly where a unique marking template may be generated by adjusting the position of one or more markings 918A-918D of the array relative to other markings, where each discrete position matches a specific template and defines a unique tool 608 or end effector 112, 912 by a different known attribute. Additionally, while illustrated with respect to an end effector 912, it should be appreciated that movable and fixed markings 918A-918D may be used with any suitable instrument 608 or other object to be tracked.

[0131] When using an external 3D tracking system 100, 300, 600 to track a fully rigid body array of three or more markers attached to the end effector 112 of the robot (e.g., Fig.13A and 13B 106 ), it is possible to directly track or calculate the 3D position of each segment of the robot 102 in the coordinate system of the camera 200, 326. The geometric orientation of the joints relative to the tracker is known by design, and the linear or angular positions of the joints are known from the encoders of each motor of the robot 102, thereby completely defining the 3D position of all moving parts from the end effector 112 to the base 116. Similarly, if the tracker is mounted on the base 106 of the robot 102 (not shown), then it is likewise possible to track or calculate the 3D position of each segment of the robot 102 from the base 106 to the end effector 112 based on the joint geometry and joint positions known from the encoders of each motor.

[0132] In some cases, it may be desirable to track the positions of all segments of the robot 102 from fewer than three markers 118 rigidly attached to the end effector 112. Specifically, if the tool 608 is introduced into the catheter 114, then the fully rigid body motion of the robot 902 may need to be tracked with only one additional marker 118 being tracked.

[0133] Now turn to see Figures 15A-15E , shows an alternative version of an end effector 1012 having only a single tracking marker 1018. The end effector 1012 can be similar to another end effector described herein, and can include a guide tube 1014 extending along a longitudinal axis 1016. Similar to the other tracking markers described herein, the single tracking marker 1018 can be rigidly attached to the guide tube 1014. This single marker 1018 can serve to add a missing degree of freedom to allow for fully rigid body tracking, and / or can serve to act as a surveillance marker to ensure that assumptions about the robot and camera positioning are valid.

[0134] A single tracking marker 1018 may be attached to the robotic end effector 1012 as a rigid extension to the end effector 1012 that protrudes in any convenient direction and does not obstruct the surgeon's field of view. The tracking marker 1018 may be attached to the catheter 1014 or any other suitable location on the end effector 1012. When attached to the catheter 1014, the tracking marker 1018 may be positioned at a location between the first and second ends of the catheter 1014. For example, at Fig.15A , a single tracking marker 1018 is shown as a reflective sphere mounted on the end of a narrow shaft 1017 extending forward from the catheter 1014 and positioned longitudinally above the midpoint of the catheter 1014 and below the entrance of the catheter 1014. This position allows the marker 1018 to be generally visible by the camera 200, 326 and will not block the vision of the surgeon 120 or collide with other tools or objects in the surgical vicinity. In addition, the catheter 1014 with the marker 1018 in this position is designed so that the array of markers on any tool 608 introduced into the catheter 1014 is visible at the same time as the single marker 1018 on the catheter 1014 is visible.

[0135] like Fig. 15B 1014, the instrument 608 becomes mechanically constrained in four of the six degrees of freedom. That is, the instrument 608 cannot rotate in any direction except about the longitudinal axis 1016 of the catheter 1014, and the instrument 608 cannot translate in any direction except along the longitudinal axis 1016 of the catheter 1014. In other words, the instrument 608 can only translate along and rotate about the centerline of the catheter 1014. If two other parameters, such as (1) the rotation angle about the longitudinal axis 1016 of the catheter 1014; and (2) the position along the catheter 1014, are known, then the position of the end effector 1012 in the camera coordinate system is fully defined.

[0136] Reference Fig. 15C , the system 100, 300, 600 should be able to know when the tool 608 is actually positioned inside the catheter 1014, rather than being instead positioned outside the catheter 1014 and simply somewhere within the field of view of the camera 200, 326. The tool 608 has a longitudinal axis or centerline 616 and an array 612 having a plurality of tracked markers 804. Rigid body calculations can be used to determine where the centerline 616 of the tool 608 is located in the camera coordinate system based on the tracked position of the array 612 on the tool 608.

[0137] A fixed normal (vertical) distance D from a single marker 1018 to the centerline or longitudinal axis 1016 of the catheter 1014 Fis fixed and geometrically known, and can track the position of the single marker 1018. Therefore, when the detected distance D from the tool centerline 616 to the single marker 1018 is D The known fixed distance D from the catheter centerline 1016 to the single marker 1018 F When matched, it can be determined that the tool 608 is inside the catheter 1014 (the center lines 616, 1016 of the tool 608 coincide with the catheter 1014) or is exactly at this distance D D With fixed distance D F A point in the possible location of the match. For example, Fig. 15C , the detected normal distance D from the tool centerline 616 to the single marker 1018 D The fixed distance D from the catheter centerline 1016 to a single marker 1018 in two data frames (tracked marker coordinates) represented by the transparent tool 608 in two locations F match, and therefore additional consideration may be required to determine when the tool 608 is positioned in the catheter 1014.

[0138] Now turn to see Fig.15D , programmed logic may be used to find a frame of tracking data where the detected distance D from the tool centerline 616 to a single marker 1018 is D The tool 608 is held fixed at the correct length regardless of the tool 608 moving in space relative to the single spherical surface 1018 by more than a certain minimum distance to satisfy the condition that the tool 608 moves within the catheter 1014. For example, a first frame F1 may be detected by the tool 608 in a first position, and a second frame F2 may be detected by the tool 608 in a second position (i.e., linearly moved relative to the first position). The markers 804 on the tool array 612 may move from the first frame F1 to the second frame F2 by more than a given amount (e.g., more than 5 mm in total). Even in the case of this movement, the detected distance D from the tool centerline vector C' to the single marker 1018 is D It is substantially the same in both the first frame F1 and the second frame F2.

[0139] Logically, the surgeon 120 or user can place the tool 608 within the catheter 1014 and rotate or slide the tool 608 slightly down into the catheter 1014, and the system 100, 300, 600 will be able to detect that the tool is within the catheter 1014 from tracking five markers (the four markers 804 on the tool 608 plus the single marker 1018 on the catheter 1014). Knowing that the tool 608 is within the catheter 1014, all six degrees of freedom that spatially define the position and orientation of the robotic end effector 1012 can be calculated. Without the single marker 1018, even if it is known with certainty that the tool 608 is within the catheter 1014, it is not known where the catheter 1014 is located along the centerline vector C' of the tool and how the catheter 1014 is rotated relative to the centerline vector C'.

[0140] Focus on Fig.15E , in the case where the presence of a single marker 1018 is tracked and there are four markers 804 on the tool 608, it is possible to construct the centerline vector C' and the normal vector of the catheter 1014 and the tool 608 through the single marker 1018 and through the centerline vector C'. This normal vector has an orientation that intersects the centerline vector C' at a known orientation and at a specific fixed position relative to the forearm of the robot distal to the wrist (in this example, parallel to the segment orientation). For convenience, three mutually orthogonal vectors k', j', i' can be constructed, such as Fig.15E , thereby defining the rigid body position and orientation of the catheter 1014. One of the three mutually orthogonal vectors k' is composed of the centerline vector C', the second vector j' is composed of the normal vector passing through the single marker 1018, and the third vector i' is the vector cross product of the first vector k' and the second vector j'. When all joints are at zero, the robot joint positions relative to these vectors k', j', i' are known and fixed, and therefore the rigid body calculation can be used to determine the position of any section of the robot relative to these vectors k', j', i' when the robot is in the starting position. During robot movement, if the position of the tool marker 804 (when the tool 608 is in the catheter 1014) and the position of the single marker 1018 are detected from the tracking system, and the angular / linear position of each joint is known from the encoder, then the position and orientation of any section of the robot can be determined.

[0141] In some embodiments, the orientation of the fixed tool 608 relative to the catheter 1014 may be useful. For example, the end effector catheter 1014 may be oriented in a specific position around its axis 1016 to allow machining or implant positioning. Although the orientation of any object attached to the tool 608 inserted into the catheter 1014 is known from the tracked marker 804 on the tool 608, the rotational orientation of the catheter 1014 itself in the camera coordinate system is unknown without an additional tracking marker 1018 (or multiple tracking markers in other embodiments) on the catheter 1014. This marker 1018 provides a substantially "clock position" from -180° to +180° based on the orientation of the marker 1018 relative to the centerline vector C'. Therefore, a single marker 1018 can provide additional degrees of freedom to allow fully rigid body tracking, and / or can serve as a monitoring marker to ensure that assumptions about robot and camera positioning are valid.

[0142] Fig.16 1 is a block diagram of a method 1100 for navigating and moving the end effector 1012 of the robot 102 (or any other end effector described herein) to a desired target trajectory. Another use of a single marker 1018 on the robot end effector 1012 or guide tube 1014 is to enable automated safe movement of the robot 102 as part of the method 1100 without a full tracking array attached to the robot 102. This method 1100 works when the tracking cameras 200, 326 do not move relative to the robot 102 (i.e., they are in a fixed position), the coordinate system of the tracking system is co-registered with the coordinate system of the robot, and the robot 102 is calibrated so that the position and orientation of the guide tube 1014 can be accurately determined in the robot Cartesian coordinate system based solely on the encoded position of each robot axis.

[0143] For this method 1100, the coordinate system of the tracker and the coordinate system of the robot must be co-registered, which means that a coordinate transformation from the Cartesian coordinate system of the tracking system to the Cartesian coordinate system of the robot is required. For convenience, this coordinate transformation can be a 4×4 matrix of translations and rotations, which is well known in the field of robotics. This transformation will be referred to as Tcr to refer to "Transformation-Camera to Robot". Once this transformation is known, any new frame of tracking data received as x, y, z coordinates in vector form for each tracked marker can be multiplied by the 4×4 matrix, and the resulting x, y, z coordinates will be in the coordinate system of the robot. To obtain Tcr, the full tracking array on the robot is tracked when it is rigidly attached to the robot at a known position in the robot's coordinate system, and then the known rigid body method is used to calculate the transformation of the coordinates. It should be obvious that when the additional markers 1018 are also read, any tool 608 inserted into the catheter 1014 of the robot 102 can provide the same rigid body information as the rigidly attached array. That is, rather than being inserted into a fixed position and orientation, the tool 608 only needs to be inserted into any position and at any rotation within the catheter 1014. Therefore, it is possible to determine the Tcr by inserting any tool 608 with a tracking array 612 into the catheter 1014 and reading the tool's array 612 plus a single marker 1018 of the catheter 1014 while determining the current position of the catheter 1014 in the robot's coordinate system from the encoders on each axis.

[0144] exist Fig.16 Logic for navigating and moving the robot 102 to a target trajectory is provided in method 1100 of . Prior to entering loop 1102, it is assumed that a transformation Tcr has been previously stored. Therefore, prior to entering loop 1102, in step 1104, after securing the robot base 106, when the robot is static, greater than or equal to one frame of tracking data of the tool inserted in the catheter is stored; and in step 1106, a transformation Tcr of the robot catheter position from camera coordinates to robot coordinates is calculated from this static data and previous calibration data. Tcr should remain valid as long as the camera 200, 326 does not move relative to the robot 102. If the camera 200, 326 moves relative to the robot 102 and Tcr needs to be re-acquired, the system 100, 300, 600 can be caused to prompt the user to insert the tool 608 into the catheter 1014 and then automatically perform the necessary calculations.

[0145] In the flow chart of method 1100, each frame of collected data consists of a snapshot of the tracked position of the DRB 1404 on the patient 210, the tracked position of a single marker 1018 on the end effector 1014, and the position of each robot axis. From the positions of the robot's axes, the position of the single marker 1018 on the end effector 1012 is calculated. This calculated position is compared to the actual position of the marker 1018 as recorded from the tracking system. If the values ​​match, then it can be ensured that the robot 102 is in a known position. A transformation Tcr is applied to the tracked position of the DRB 1404 so that a target for the robot 102 can be provided according to the robot's coordinate system. The robot 102 can then be commanded to move to reach the target.

[0146] After steps 1104, 1106, loop 1102 includes step 1108 of receiving rigid body information of DRB 1404 from the tracking system; step 1110 of transforming the target tip and trajectory from image coordinates to tracking system coordinates; and step 1112 of transforming the target tip and trajectory from camera coordinates to robot coordinates (applying Tcr). Loop 1102 further includes step 1114 of receiving a single stray marker position of the robot from the tracking system; and step 1116 of transforming the single stray marker from tracking system coordinates to robot coordinates (applying stored Tcr). Loop 1102 also includes step 1118 of determining the current position of the single robot marker 1018 in the robot coordinate system according to forward kinematics. Information from steps 1116 and 1118 is used to determine step 1120 whether the stray marker coordinates from the transformed tracked position are consistent with the calculated coordinates within a given tolerance. If so, then proceed to step 1122 to calculate and apply robot movement to the target x, y, z and trajectory. If no, then proceed to step 1124: stop and require full array to be inserted into catheter 1014 before proceeding; step 1126: recalculate Tcr after array insertion; and proceed to repeat steps 1108, 1114 and 1118.

[0147] This method 1100 has advantages over methods that omit continuous monitoring of a single marker 1018 to verify position. Without a single marker 1018, it would still be possible to use the Tcr to determine the position of the end effector 1012 and send the end effector 1012 to the target position, but it would not be possible to verify that the robot 102 is actually in the expected position. For example, if the camera 200, 326 has been raised and the Tcr is no longer valid, the robot 102 will move to the wrong position. For this reason, the single marker 1018 provides safety value.

[0148] For a given fixed position of the robot 102, it is theoretically possible to move the tracking camera 200, 326 to a new position where the single tracked marker 1018 remains unmoved, since the single tracked marker is a single point and not an array. In this case, the system 100, 300, 600 will not detect any errors, since there will be agreement in the calculated and tracked positions of the single marker 1018. However, once the robot's axis causes the catheter 1012 to move to a new position, the calculated and tracked positions will not agree and the safety check will come into play.

[0149] For example, the term "monitoring marker" may be used in reference to a single marker in a fixed position relative to the DRB 1404. In this example, if the DRB 1404 is raised or otherwise moved, the relative position of the monitoring marker changes and the surgeon 120 may be alerted that there may be a problem with navigation. Similarly, in the embodiments described herein, the system 100, 300, 600 can continuously check whether the camera 200, 326 has moved relative to the robot 102 through a single marker 1018 on the catheter 1014 of the robot. If the registration of the coordinate system of the tracking system and the coordinate system of the robot is lost (for example, because the camera 200, 326 is raised or malfunctioning or because the robot malfunctions), the system 100, 300, 600 can alert the user and can make corrections. Therefore, this single marker 1018 can also be considered a monitoring marker for the robot 102.

[0150] It should be apparent that in the case where the full array is permanently mounted on the robot 102 (e.g., the plurality of tracking markers 702 are mounted on Figures 7A-7C 602 shown in FIG. 1 ), such functionality of a single marker 1018 as a robot monitoring marker is not needed because the camera 200, 326 is not required to be in a fixed position relative to the robot 102 and update the Tcr at each frame based on the tracked position of the robot 102. The reason for using a single marker 1018 instead of a full array is that the full array is bulkier and obtrusive, thereby blocking the surgeon's field of view and access to the surgical field of view 208 that is larger than the single marker 1018, and the line of sight to the full array is more easily blocked than the line of sight to a single marker 1018.

[0151] Now turn to see Figures 17A-17B18A-18B, depicts an instrument 608 such as an implant holder 608B, 608C, which includes both a fixed tracking marker 804 and a removable tracking marker 806. The implant holder 608B, 608C may have a handle 620 and an outer shaft 622 extending from the handle 620. The shaft 622 may be positioned substantially perpendicular to the handle 620, as shown, or may be positioned in any other suitable orientation. An inner shaft 626 may extend through the outer shaft 622 at one end by a knob 628. Using a typical connection mechanism known to those skilled in the art, at the other end, the implant 10, 12 is connected to the shaft 622 at the tip 624 of the implant holder 608B, 608C. The knob 628 may be rotated, for example, to expand or articulate the implant 10, 12. Expandable fusion devices and installation methods are described in US Pat. Nos. 8,709,086 and 8,491,659, the disclosures of which are incorporated herein by reference.

[0152] When tracking a tool 608, such as an implant holder 608B, 608C, the tracking array 612 may contain a combination of fixed markers 804 and one or more movable markers 806, which constitute the array 612 or are otherwise attached to the implant holder 608B, 608C. The navigation array 612 may include at least one or more (e.g., at least two) fixed position markers 804, which are positioned at known positions relative to the implant holder instruments 608B, 608C. These fixed markers 804 will not be able to move in any orientation relative to the instrument geometry and will be suitable for defining the position of the instrument 608 in space. In addition, there is at least one marker 806 that can be attached to the array 612 or the instrument itself, which can move (e.g., slide, rotate, etc.) within predetermined boundaries relative to the fixed marker 804. The system 100, 300, 600 (e.g., software) aligns the position of the movable marker 806 with a specific position, orientation, or other attribute (e.g., position) of the implant 10. Figures 17A-17B The height of the expandable intervertebral body spacer shown in, or Figures 18A-18B Thus, the system and / or user can determine the height or angle of the implant 10, 12 based on the position of the movable marker 806.

[0153] exist FIG. 17A to FIG. 17B In the embodiment shown in FIG. 6 , four fixed markers 804 are used to define the implant holder 608B, and a fifth movable marker 806 is capable of sliding within a predetermined path to provide feedback regarding the height of the implant (eg, a collapsed position or an expanded position). Fig.17A The expandable spacer 10 is shown at its initial height, and Fig. 17BThe spacer 10 is shown in an expanded state, wherein the movable marker 806 is translated to different positions. In this case, the movable marker 806 moves closer to the fixed marker 804 when the implant 10 expands, but it is expected that this movement can be reversed or otherwise different. The amount of linear translation of the marker 806 will correspond to the height of the implant 10. Although only two positions are shown, it will be possible to use this as a continuous function, by which any given expansion height can be related to a specific position of the movable marker 806.

[0154] Now turn to see Figures 18A to 18B , four fixed markers 804 are used to define the implant holder 608C, and a fifth movable marker 806 is configured to slide within a predetermined path to provide feedback regarding the implant articulation angle. Fig.18A The articulated spacer 12 is shown in its initial linear state, and Fig.18B The spacer 12 is shown in an articulated state at a certain offset angle, wherein the movable marker 806 is translated to different positions. The amount of linear translation of the marker 806 will correspond to the articulation angle of the implant 12. Although only two positions are shown, it is possible to use this as a continuous function, by which any given articulation height can be related to a specific position of the movable marker 806.

[0155] In these embodiments, removable marker 806 continues to slide to provide feedback about the attribute of implant 10,12 based on position.It is also expected that there may be a careful position that removable marker 806 must be in, and the careful position will also be able to provide other information about the implant attribute.In this case, each careful configuration of all marks 804,806 is relevant to the specific geometry of implant holder 608B, 608C and implant 10,12 on or under the specific height with specific orientation.In addition, any movement of removable marker 806 can be used for other variable attributes of the navigated implant of any other type.

[0156] Although depicted and described with respect to the linear movement of movable marker 806, movable marker 806 should not be limited to only sliding, because there may be rotation or other movement of marker 806 that may be applicable to the application of providing information about implant 10, 12. Any relative position change between the group of fixed markers 804 and movable marker 806 may be relevant information of implant 10, 12 or other devices. In addition, although expandable and articulated implants 10, 12 are illustrated, instrument 608 may work on other medical devices and materials, such as spacers, cages, plates, fasteners, nails, screws, rods, latches, wire structures, sutures, anchor clips, staples, stents, bone grafts, biological agents, cements, etc.

[0157] Now turn to see Fig.19A It is contemplated that the robotic end effector 112 may be interchangeable with other types of end effectors 112. Furthermore, it is contemplated that each end effector 112 may be capable of performing one or more functions based on the desired surgical procedure. For example, an end effector 112 having a guide tube 114 may be used to guide an instrument 608 as described herein. Additionally, the end effector 112 may be replaced by a different or alternative end effector 112 that controls, for example, a surgical device, instrument, or implant.

[0158] Alternative end effectors 112 may include one or more devices or instruments coupled to and controllable by the robot. As non-limiting examples, Fig.19A The end effector 112 depicted in the drawings may include a retractor (e.g., one or more retractors disclosed in U.S. Pat. Nos. 8,992,425 and 8,968,363) or one or more mechanisms for inserting or installing a surgical device, such as an expandable intervertebral fusion device (e.g., the expandable implants exemplified in U.S. Pat. Nos. 8,845,734, 9,510,954, and 9,456,903), a stand-alone intervertebral fusion device (e.g., the implants exemplified in U.S. Pat. Nos. 9,364,343 and 9,480,579), an expandable corpectomy device (e.g., the corpectomy implants exemplified in U.S. Pat. Nos. 9,393,128 and 9,173,747), an articulated intervertebral fusion device (e.g., the expandable implants exemplified in U.S. Pat. Nos. 8,845,734, 9,510,954, and 9,456,903), a stand-alone intervertebral fusion device (e.g., the implants exemplified in U.S. Pat. Nos. 9,364,343 and 9,480,579), an expandable corpectomy device (e.g., the corpectomy implants exemplified in U.S. Pat. Nos. 9,393,128 and 9,173,747), a hinged intervertebral fusion device (e.g., the expandable implants exemplified in U.S. Pat. Nos. 9,845,734, 9,510,954, and 9 Spacers (e.g., implants illustrated in U.S. Pat. No. 9,259,327), faceted prostheses (e.g., devices illustrated in U.S. Pat. No. 9,539,031), laminoplasty devices (e.g., devices illustrated in U.S. Pat. No. 9,486,253), spinous process spacers (e.g., implants illustrated in U.S. Pat. No. 9,592,082), inflatables, fasteners, including multi-axial screws, single-sided screws, pedicle screws, posterior screws, etc., bone fixation plates, rod constructs and modification devices (e.g., devices illustrated in U.S. Pat. No. 8,882,803), artificial and natural intervertebral discs, motion preservation devices and implants, spinal cord stimulators (e.g., devices illustrated in U.S. Pat. No. 9,440,076), and other surgical devices. The end effector 112 may include one or more instruments coupled directly or indirectly to the robot for delivering bone cement, bone grafts, living cells, drugs, or other deliverable substances to the surgical target. The end effector 112 may also include one or more instruments designed to perform a discectomy, kyphoplasty, vertebral body stenting, dilation, or other surgical procedures.

[0159] The end effector itself and / or the implant, device or instrument may include one or more markers 118 such that the location and position of the markers 118 may be identified in three dimensions. It is contemplated that the markers 118 may include active or passive markers 118, as described herein, which may be directly or indirectly visible to the camera 200. Thus, for example, one or more markers 118 positioned on the implant 10 may enable tracking of the implant 10 before, during, and after implantation.

[0160] like Fig.19B As shown in FIG. 1 , the end effector 112 may include an instrument 608, or it may be coupled to the robotic arm 104 (for example, the instrument 608 may be Figures 9A-9C The coupling mechanism shown in FIG. is coupled to the robot arm 104) and can be controlled by the robot system 100. Fig.19B In the embodiment shown in , the robotic system 100 is capable of inserting the implant 10 into the patient's body and can expand or contract the expandable implant 10. Therefore, the robotic system 100 can be configured to assist the surgeon or operate partially or completely independently of the surgeon. Therefore, it is contemplated that the robotic system 100 can be capable of controlling each alternative end effector 112 for its designated function or surgical procedure. Although the robot and associated systems described herein are generally described with reference to spinal applications, it is also contemplated that the robotic system is configured for use in other surgical applications, including but not limited to trauma surgery or other orthopedic applications (e.g., placement of intramedullary nails, plates, etc.), cranial, neurological, cardiothoracic, vascular, colorectal, tumor, dental and other surgical operations and procedures. According to some embodiments discussed below, the robotic system can be used in brain surgery applications.

[0161] When a fluorescence-only or preoperative computed tomography (CT) scan workflow is used in surgical navigation, the current 2-BB plane method of registering fluoroscopic x-ray images (also known as fluoroscopic images, fluorescence images, fluoroscopic snapshots, fluorescence snapshots, fluoroscopic x-ray snapshots, etc.) to an optical tracking camera may have disadvantages as discussed below.

[0162] 1) It may be difficult to automatically and / or accurately locate the shadow of the metal BB of the fluoroscopic fixture in the presence of other shadows on the fluoroscopic x-ray image.

[0163] 2) The presence of multiple BBs or other radiopaque fiducials on a fluoroscopic x-ray image may create unnecessary clutter on the image.

[0164] 3) The height of the outer BB plane may make it difficult to get the fluoroscopic x-ray detector close enough to the patient, resulting in the fluoroscopic x-ray image being too magnified (eg, the field of view is too narrow).

[0165] 4) Because the registration algorithm may need to accurately find several BBs on the fluoroscopic x-ray image, it may not be possible to align the image to the desired degree.

[0166] According to some embodiments of the inventive concept, some or all of the above problems may be solved.

[0167] Surgical navigation typically uses / requires the registration of a pair of fluoroscopic x-ray images to an optical tracking system used to provide robotic control / navigation and / or surgical planning. For example, the current ExcelsiusGPS application allows planning of screw placement, robot position, and / or tool navigation based solely on two registered fluoroscopic x-ray image snapshots (fluorescence workflow). The current ExcelsiusGPS application also allows registration of a pair of registered fluoroscopic x-ray image snapshots with a prior CT scan (preoperative CT workflow). In both workflows, it may be necessary to accurately register the pair of fluorescence images to the optical tracking camera.

[0168] exist Fig. 223009 (also referred to as source, emitter, x-ray emitter, etc.) and an x-ray detector 3005 (also referred to as detector, collector, or image intensifier) ​​are mounted on a C-arm 3009. The C-arm 3009 may be mounted to a base 3011 of the system via a positioning mechanism 3015 configured to provide in-out Tx and up-down Ty translational movement, and to provide tilt Rx, sway Ry, and orbit Rz rotational movement of the C-arm 3009 relative to the base 3011. By positioning the source 3001 and the detector 3005 on opposite sides of a patient (e.g., a patient lying on an operating table), x-ray radiation may be transmitted by the source 3001 through the patient to the detector 3005, and the detected radiation from the detector 3005 may be processed to generate a fluoroscopic x-ray image (also referred to as a fluoroscan) of the patient's anatomical structure, and the C-arm 3009 may be rotated about the tilt Rx, swing Ry, and / or track Rz, so that multiple images of the same anatomical structure / tissue may be generated from different angles of the tilt Rx, swing Ry, and / or track Rz. A 3-D reference tracking volume may be determined using two images taken from different angles of the tilt Rx, swing Ry, and / or track Rz, and the 3-D reference tracking volume may be used to predict how an object in the x-ray path will appear on the image, and vice versa. That is, by tracking the positions of the collector and emitter as x-ray snapshots are taken and detecting the shadows of a reference fiducial in these x-ray snapshots, there is a closed-form mathematical solution that allows the position of any new object appearing in the 3-D reference volume (defined in camera coordinates) to be mapped to the 2-D snapshot, and allows the position of any object appearing in two 2-D snapshots simultaneously to be mapped to the camera's 3-D reference space.

[0169] In a fluorescence workflow, the registration of the tracked reference space to the x-ray image is all that is used or required for planning and navigation, where screws or other surgical implants are planned on a 2-D snapshot, and the positions of tools and robots can be driven or observed where they appear on the 2-D snapshot. It is often useful to register the 3-D reference space described above to a 3-D image volume of a CT scan, a cone beam CT volume, or an MRI volume. Once the reference space and image volume are registered, planning and navigation of surgical implants can be performed relative to the 3-D image volume. Some embodiments disclosed herein relate to registering the tracked 3-D reference space to a 2-D fluorescence snapshot.

[0170] One method of recording a pair of fluorescence snapshots to an optical tracking system is to use a fixture 21001 (also called a fluorescence fixture) that is attached to Fig. 22The detector 3005 (also called image intensifier) ​​of the fluoroscopic system. Fig. 20 As shown in , the fixture 21001 has an optical tracking marker 21005 (e.g., a reflective sphere or a light emitting diode), which is used by the optical tracking system to track its position relative to the patient, and the fixture 21001 has a plurality of metal sphere arrays (BB) or other radiopaque references, which are dispersed / embedded in at least two planes to form an x-ray shadow pattern on the fluoroscopic image / snapshot. Fig. 20 In the example of FIG. 3005 , the first BB plane 21009 a is spaced apart from the detector 3005 , and the second BB plane 21009 b is installed adjacent to the detector 3005 such that the second BB plane 21009 b is located between the first BB plane 21009 a and the detector 3005 . Fig. 20 The BBs are not visible but are embedded in a radiolucent material including corresponding BB planes such that each BB blocks x-rays to provide a corresponding shadow in the fluoroscopic image generated using the detector 3005. As the fluoroscopic snapshot / image is taken, the optical tracking system captures the position of the optical tracking markers 21005 of the fluoroscopic fixture, allowing the position of the image plane of the detector 3005 to be accurately tracked. From an image processing perspective, the shadows formed by the BBs and cast on the image plane allow the position of the source 3001 to be accurately determined using the geometric constraints of the pinhole camera model, as shown in FIG. Fig.21 As shown in .

[0171] Fig. 20 A Globus fluoroscopy registration fixture 21001 is depicted. The fixture 21001 clamps onto an x-ray detector 3005 (e.g., an image intensifier) ​​of a C-arm fluoroscopy system. Six reflective spheres 21005 are used to provide optical tracking by an optical tracking system, and two BB planes 21009a and 21009b are shown consisting of black plastic with 161 BBs embedded therein.

[0172] Fig.21 is a schematic diagram illustrating the concept of the pinhole camera model. X-rays from a point source (i.e., x-ray source 3001, also referred to as an x-ray emitter, emitter, or source) travel through tissue (between the x-ray source 3001 and the first BB plane 21009a) until reaching the detector 3005 (image intensifier), where an image is generated. If the spacing between the BB and the plane is known, the pattern of x-ray shadows projected onto the 2D image plane of the detector 3005 indicates where the x-ray source 3001 must be located relative to the detector 3005 to form a snapshot.

[0173] With the positions of the source 3001 and detector 3005 accurately determined in camera space (of the optical tracking system), it is possible to accurately define the mathematical transformations that move back and forth between camera space (of the optical tracking system) and a 2D image plane (one plane in each of a pair of fluoroscopic images). For example, if an object is at a known position in camera space, its projected representation can appear accurately on each image plane, where the projection of the object will appear as if a new pair of fluoroscopic images / snapshots were taken from the same orientation of the C-arm 3009, source 3001, and detector 3005. Conversely, if the projection of the object is added to two 2D image planes (like the projected screw images of the surgical plan added to the anteroposterior (AP) and lateral fluoroscopic images / snapshots), the 3D position of the object in camera space can be accurately determined. Note that there are restrictions on where the projected image of an object can be placed on both 2D image planes simultaneously, which is a valid position that is correctly projected on both images simultaneously. Consider the analogy of viewing a point in space from frontal and lateral perspectives. From a front view, it is unclear how far in front or back the point is, but its position from left to right and top to bottom can be determined. Looking at the same point from the side, it is unclear how far to the left or right the point is in 3D space, but its position from front to back and top to bottom can be determined. If you move the 2D position of a point from a front view, it will be allowed to move to any position from left to right, and from a side view, the position of the point will remain the same, but it cannot be in a different position from top to bottom unless the position of the point on the side view changes its top and bottom positions to match.

[0174] Although the above is used Fig. 20 The method of tracked fixture 21001 and geometric constraints is effective in accurately determining the location of the detector 3005 image plane and source 3001 in optical camera space so that fluoroscopic images / snapshots can be registered to the optical camera space, but the method may have disadvantages. For example, it may be difficult to automatically locate or accurately locate the shadow of the metal BB (or other fiducial) of the fluoroscopic fixture 21001 in the presence of other shadows on the fluoroscopic image, such as caused by metal surgical implants. Some fluoroscopic images / snapshots may have contrasting dark areas that are comparable to metal in the BB or other fiducial. In such cases, the algorithm may not be able to find or use these fiducial shadows for registration, and the accuracy of the registration may be reduced, or the registration may not be performed.

[0175] Another disadvantage of the method using fixture 21001 is that many BBs or other radiopaque fiducials on the fluoroscopic image must be present for registration, but the BBs may be undesirable for surgeons using the image as a visual aid for surgery. For example, the BBs or other fiducials may obscure portions of the anatomy that the surgeon needs to see clearly.

[0176] use Fig. 20 Another disadvantage of the method of fixing the device 21001 is that it may be difficult / impossible to collimate the fluoroscopic image if the fluoroscopic image is used for alignment. Collimation is the movement of lead shielding from one or more sides of the x-ray source 3001 to prevent the x-rays from reaching the outside of the collector 3005 and appearing on the fluoroscopic image. Collimation may be useful when some x-rays pass through tissue and other x-rays only pass through air. For example, when taking a lateral fluoroscopic image / snapshot of a patient, the area to be visualized (torso) is very wide and deep, but very short in the anterior-posterior (AP) direction (from top to bottom). Therefore, it may be useful to collimate the x-ray image from the top (or top and bottom) to reduce / prevent the sharp changes in image contrast between tissue and air that would otherwise be seen. However, the fluoroscopic images / snapshots used for alignment rely on being able to visualize enough reference BBs in each snapshot, and collimation will block some BBs.

[0177] use Fig. 20 Another disadvantage of the method of fixture 21001 is that the two planes of the BB that the pinhole camera model uses / requires to locate the source 3001 are bulky and conspicuous. Although one plane of the BB is flush with the surface of the x-ray detector 3005, the second plane must typically protrude at least 100m outward from the surface of the x-ray detector 3005 to achieve the desired accuracy in locating the source 3001. This protrusion of the outer BB plane may make it difficult to get the fluoroscopy unit close enough to the patient. The farther the fluoroscopy unit is from the patient, the more magnified the fluoroscopy image appears due to parallax (resulting in a narrower field of view). Due to this limitation, the fluoroscopy image may be much more magnified than the surgeon wants, for example, only covering one spinal level instead of two.

[0178] All of the issues discussed above stem from the use of two planes of BB in the calibration algorithm, and therefore eliminating the need for multiple planes of BB may be beneficial. Instead of using the detected BB to determine the position of the detector 3005 image plane and source 3001 in the above method, the position of the source 3001 may be extrapolated based on the position of an optical tracker mounted to the detector 3005 (image intensifier). However, since the x-ray detector 3005 (image intensifier or flat panel), source 3001 (emitter / anode), and C-arm 3009 are all somewhat bulky pieces of equipment themselves, the C-arm 3009 of the fluoroscopic system may flex a small amount in different orientations, and therefore the position of the source 3001 cannot be accurately extrapolated. Due to the flexure of the C-arm 3009, the position of the source 3001 relative to the detector 3005 may vary by several millimeters between the typical positions of the C-arm 3009 (such as anteroposterior AP and lateral), creating unacceptable inaccuracies in the predicted position of the source 3001 used for registration.

[0179] According to some embodiments of the present inventive concepts, instead of using projections of fixture fiducials and the geometric constraints of the pinhole camera model to determine the position of the x-ray source 3001, the currently monitored orientation of the C-arm 3009 is used to accurately predict how the C-arm 3009 flexes, and therefore accurately determine where the source 3001 is located. There are several different methods to monitor the orientation of the C-arm 3009, and several methods of using the discussed information to determine the position of the source 3001, which will be discussed below.

[0180] One way to monitor the orientation of the C-arm 3009 is to automatically or manually read the values ​​of the C-arm axes and provide these values ​​to an algorithm that uses them to determine the influence on the position of the source 3001. Typically, the C-arm 3009 has three rotational movements: orbit (Rz), tilt (Rx) and yaw (Ry), as shown in FIG. Fig. 22 3009. There are also some linear translation axes on the C-arm 3009. Since the yaw axis Ry is almost exactly aligned with gravity, it can be assumed that there will be no difference in the flexion of the C-arm 3009 due to changes in yaw Ry. In addition, the linear translation positions Tx and Ty should not affect how the C-arm 3009 flexes. Therefore, changes in the position of the source 3001 can be characterized relative to the position of the detector 3005, which mainly knows the changes in orbit Rz and tilt Rx.

[0181] Fig. 22 A configuration of a fluoroscopic system is depicted showing the different axes of movement including tilt Rx, in and out Tx, up and down Ty, rocking Ry and track Rz. Unless the C-arm 3009 is operated on a slope, the axis in which the rocking occurs (shown as Ry) is typically aligned with gravity and therefore only track Rz and tilt Rx significantly affect how the C-arm 3009 flexes under normal conditions.

[0182] Older C-arm fluoroscopy systems have locking and unlocking mechanisms to allow manual changes to track Rz and tilt Rx, and have scales indicating the position of each axis within + / - 1 degree. Therefore, the software user interface can allow the user to enter these values ​​each time a fluoroscopic snapshot is taken, and these values ​​will be used to calculate the offset of the source 3001 relative to the detector 3005. Alternatively, the axes of the C-arm 3009 can be equipped with optical, magnetic or other encoders to automatically detect the position of each axis and feed the information to the navigation system for processing.

[0183] Newer C-arm fluoroscopy systems have built-in encoders to digitally record the position of each axis. These systems store the axis coordinates into on-board memory and write the coordinates in the DICOM (Digital Imaging and Communications in Medicine) header of each fluoroscopic image taken. Therefore, the software can use the digital data stream via a serial cable or other means between the C-arm fluoroscopy system and the navigation system to communicate the current axis positions, specifically orbit Rz and tilt Rx during the fluoroscopic snapshot. Alternatively, the software can retrieve each image / snapshot as a DICOM file via Ethernet, serial communication, or wireless data transfer, and read the DICOM tags by standard software methods to retrieve information about the orbit Rz and tilt Rx positions.

[0184] Another way to monitor the orientation of the C-arm 3009 is to place an electronic tilt sensor or inertial measurement unit (IMU) somewhere on the C of the C-arm 3009 to monitor its angular orientation and feed that information to the navigation system when taking fluoroscopic snapshots.

[0185] Two tilt sensors will be used to monitor the excursions in the C-arm 3009 corresponding to the track Rz and tilt Rx. The IMUs typically monitor 3 or 6 degrees of freedom and when they are mounted on the C-arm 3009 will be oriented so that when the C-arm 3009 is in the reference position, the direction of gravity is initially aligned with the vertical axis of the IMU. The data from the tilt sensors and / or the IMU will be communicated to the navigation system for registration when the fluoroscopic snapshot is taken via Ethernet, WiFi, Bluetooth, serial port or any suitable means.

[0186] Another way to monitor the orientation of the C-arm 3009 during different snapshots is to utilize the optical tracking system that is already used to track the position of the detector 3005 (also called the collector). Doing so would require the system to first define the vertical orientation of the system, and then maintain that definition. For example, a software feature could prompt the user to position the C-arm 3009 with track Rz at 0° and tilt Rx at 0°, and then click a button to capture the upright position. Subsequent measurements of the optical tracker on the C-arm 3009 would then be able to determine the change in orientation, as long as the camera of the optical tracking system does not move. Alternatively, if the optical tracking camera does move, the vertical orientation can still be inferred if additional tracking reference fixtures that indicate the direction of gravity are also observed, such as tracking markers fixed to a table or other features that do not change their orientation relative to gravity in the described situation.

[0187] A different embodiment of monitoring the orientation of the C-arm 3009 during different snapshots is to utilize a gravity vector containing an optical tracking element such as a reflective sphere. Such a gravity vector can be located in close proximity to the fixture used to track the position of the detector 3005, and would only require two additional markers to be tracked as separate strays. If a reflective sphere is used, the two spheres forming the gravity vector can be connected to the base via a swivel feature, and a weight can be added on the far side of the second sphere to force both spheres to hang in the direction of gravity, such as Fig.23A , Fig. 23B , Fig.23C , Fig.24A and Fig. 24B . The described embodiments may be useful relative to using manual or automatic axis input and / or relying on tracked fixture orientation to determine the position of the C-arm 3009 because it is less susceptible to user error. That is, methods using gravity vectors may be automatic, where only a small amount of additional equipment is required, and such methods do not require user input or maintenance of communications of other data streams.

[0188] Fig.23A , Fig. 23B and Fig.23CA gravity vector is depicted containing two additional reflective tracking markers 4001A and 4001B adjacent to or within the space occupied by a 4-marker C-arm tracker (containing four reflective tracking markers 4005A, 4005B, 4005C, and 4005D) used to track the position of the detector 3005. Three angular positions of the C-arm 3009 are shown, and how the gravity vector changes its orientation relative to the fixture coordinate system at different positions is demonstrated. The gravity vector contains two reflective tracking spheres 4001A and 4001B, which are connected by a rigid or non-rigid link and connected to a base (in the example, the frame of the fixture tracker) by a freely rotating interface (such as a line or ball joint). The distal tracking sphere 4001B can be weighted, or an additional weight can be attached to the distal side of the tracking sphere 4001B to provide / ensure that it is suspended vertically aligned with gravity.

[0189] Fig.24A and Fig. 24B is a photograph of a detector fixture 4000 that includes a gravity vector with tracking markers 4001A and 4001B mounted on a frame 4021 with optical / reflective tracking markers 4005A, 4005B, 4005C, and 4005D. The rotating feature in the example is a chain 4007 similar to those used for light pull cords. This design may be preferred because the chain 4007 is free to rotate about its long axis while also flexing at the link points and therefore is less likely to become obstructed or snagged when moved to a new orientation. A fishing weight 4009 is also attached to the distal side of the gravity vector's distal tracking sphere 4001B. As shown below with reference to Fig.27A As discussed, the detector fixture can be permanently mounted to the detector 3005 and / or a portion of the C-arm 3009 adjacent to the detector 3005. For example, the detector fixture can be mounted to the detector 3005 and / or a portion of the C-arm 3009 in a manner that the detector fixture is out of the x-ray path between the source 3001 and the detector 3005.

[0190] One potential problem with the gravity vector as described above is that after the C-arm 3009 is moved to a new position, the two balls connected by the rotating joint may continue to swing for an undesirably long time, forcing the user to waste time before continuing the alignment process. According to some embodiments, damping of the movement of the gravity vector can be achieved by connecting the balls 4001A and 4001B to the base using a ball-and-socket joint filled with a fluid, the ball-and-socket joint having a small hole or channel in the feature of the joint that allows the joint to move, but only at the rate at which the fluid (e.g., water or oil) can flow through the hole. According to some other embodiments, a series of two vertical damped hinge joints, one corresponding to the track Rz and one corresponding to the tilt Rx, can be used to achieve the same behavior. Each hinge joint can be damped using known methods, such as installing a buffer on the joint, like the mechanism that controls the closing speed of a household door.

[0191] Based on the orientation of the C-arm 3009 detected by any of the techniques described above when taking a fluoroscopic snapshot, the system will use calibration to determine the location of the source 3001. Such calibration can be performed when the fluoroscopic tracker is mounted on the fluoroscopic unit. In some embodiments, as described below with reference to Fig.25 As discussed in the flowchart of FIG. , a series of operations will be followed to calibrate and then determine the precise positions of the source 3001 and the detector 3005.

[0192] Fig.25 3001 position to facilitate adjustment of the extrapolated source 3001 position. After calibration is complete, the temporary fixation device can be removed and the offset in the source 3001 position for any new fluoroscopic snapshot can be determined based on the stored relationship and the tracked gravity vector. Fig.25 In some embodiments, you can use Fig.27A , Fig.27B and Fig.27C The fixture can be omitted Fig.27A The gravity vector.

[0193] At box 25001, if Fig.27A As shown in FIG. 4 , a permanent fixture 4000 having a tracking array (including markers 4005A-D) is attached near an x-ray detector 3005 (also referred to as a collector), and temporary fixtures 8010 and 8050 having a radiopaque fiducial and tracking array are attached near an x-ray source 3001 (also referred to as an emitter) and the x-ray detector 3005, as shown in FIG. Fig.27B and Fig.27C As shown in .

[0194] At block 25003, the C-arm 3009 is positioned at a reference position and a tracked reference fluoroscopic snapshot is taken while capturing the orientation of the C-arm 3009. Thus, the position / orientation of the x-ray source 3001, x-ray detector 3005, and C-arm 3009 are determined based on the tracking arrays of the fixtures 4000, 8010, and 8050 for the reference position.

[0195] At box 25005, the C-arm 3009 is moved to a new orientation within the normal operating range.

[0196] At box 25007, tracking data from all tracker arrays (ie, the tracking arrays defined by markers 4005A-D, markers 8075A-D, and markers 8053A-D) and the angular orientation of the C-arm 3009 are captured.

[0197] At box 25009, the position of the x-ray source 3001 and the x-ray detector 3005 relative to the permanent fixture 4000 is calculated based on the data from the temporary tracking arrays of the fixtures 8010 and 8050.

[0198] At box 25011, the operations at boxes 25005, 25007 and 25009 are repeated until data is collected regarding the orientation of the C-arm 3009 over the entire operating range of the C-arm 3009.

[0199] At box 25015, the offset of the x-ray source 3001 relative to the reference x-ray source 3001 position (from box 25003) is correlated with the change in orientation of the C-arm 3009 relative to the reference orientation of the C-arm 3009 (from box 25003) and the relationship relative to the tracking array of the permanent fixture 4000 is stored for use in the position of the x-ray source 3001 when future fluoroscopic snapshots are taken.

[0200] Fig.26 is a flowchart for a specific use of an embodiment with a gravity vector (e.g., using a Fig.27A If the method for determining the orientation of the C-arm is the gravity vector, then Fig.25 The flowchart can be represented as Fig.26 Flowchart of the process.

[0201] At block 26001, if Fig.27AAs shown in FIG. 1 , a permanent fixture 4000 having a tracking array (including markers 4005A-D) and a gravity vector (including markers 4001A-B) is attached near an x-ray detector 3005 (also referred to as a collector), and temporary fixtures 8010 and 8050 having a radiopaque fiducial and tracking array are attached near an x-ray source 3001 (also referred to as an emitter) and the x-ray detector 3005, as shown in FIG. Fig.27B and Fig.27C As shown in .

[0202] At block 26003, the C-arm 3009 is positioned at a reference position and a tracked reference fluorescence snapshot is taken while capturing the orientation and gravity vector of the C-arm 3009. Thus, the position / orientation of the x-ray source 3001, x-ray detector 3005, C-arm 3009, and gravity vector are determined based on the tracking arrays of fixtures 4000, 8010, and 8050 for the reference position.

[0203] At box 26005, the C-arm 3009 is moved to a new orientation within the normal operating range.

[0204] At box 26007, tracking data and gravity vectors are captured from all tracker arrays (ie, the tracking arrays defined by markers 4005A-D, markers 8075A-D, and markers 8053A-D).

[0205] At box 26009, based on the data from the temporary tracking arrays of fixtures 8010 and 8050, the position of the x-ray source 3001 and the x-ray detector 3005 relative to the permanent fixture 4000 is calculated.

[0206] At box 26011, the operations at boxes 26005, 26007 and 26009 are repeated until data is collected regarding the orientation of the C-arm 3009 over the entire operating range of the C-arm 3009.

[0207] At box 26015, the offset of the x-ray source 3001 relative to the reference x-ray source 3001 position (from box 26003) is correlated to the directional change in the gravity vector relative to the reference gravity vector orientation (from box 26003), and the relationship relative to the tracking array of the permanent fixture 4000 is stored for use in the position of the x-ray source 3001 when taking future fluoroscopic snapshots.

[0208] Fig.27A , Fig.27B and Fig.27C Shown in Fig.25 and / or Fig.26Exemplary embodiments of permanent and temporary fixtures attached to the C-arm 3009, detector 3005 and / or source 3001 mentioned in the first operation.

[0209] Fig.27A Draw the basis Fig.24A and Fig. 24B , which is mounted to a portion of the C-arm 3009 adjacent to the upper surface of the detector 3005 so that the detector 3005 is between the detector fixture 4000 and the source 3001. Therefore, the permanent detector fixture 4000 does not block the path of x-rays from the source 3001 to the detector 3005.

[0210] Fig.27B A temporary detector fixture 8010 is depicted mounted on the face 3006 of the detector 3005 between the detector 3005 and the source 3001 .

[0211] Fig.27C A temporary source fixture 8050 is depicted mounted on the source 3001 between the source 3001 and the detector 3005. As shown, the source fixture 8050 includes a frame 8051 and a plurality of optical tracking marks 8053A, 8053B, 8053C, and 8053D.

[0212] therefore, Fig.27A , Fig.27B and Fig.27C Depicted is a fixture for calibrating a fluoroscopic imaging system based on optical tracking using optical tracking markers 4005A, 4005B, 4005C, 4005D, 8053A, 8053B, 8053C, 8053D, 8075A, 8075B, 8075C, and 8075D (e.g., provided as optically reflective spheres).

[0213] like Fig.27A , a permanent detector fixture 4000 is attached to / near the detector 3005. In the example, the permanent detector fixture 4000 also has a gravity vector (comprising optical tracking markers 4001A and 4001B and weight 4009) attached to its frame 4021, as well as optical tracking markers 4005A, 4005B, 4005C, and 4005D (e.g., provided as reflective spheres).

[0214] like Fig.27B, a temporary fixture 8010 is attached on / near the imaging plane of the detector 3005. The fixture 8010 has a tracking array comprising a portion 8073A of the frame 8073 extending perpendicularly from the face 3006 of the detector 3005 and optical tracking marks 8075A, 8075B, 8075C and 8075D on the frame portion 8073A. The fixture 8010 also comprises a BB plane in the portion 8073B of the frame 8073, the BB plane being flat against and parallel to the face 3006 of the detector 3005, such that the BB plane and the portion 8073B of the frame are between the detector 3005 and the source 3001. The optical tracking array (comprising optical tracking marks 8075A, 8075B, 8075C and 8075D) is therefore in a known / fixed position relative to the BB plane because both the tracking array and the BB of the plane are fixed to and / or disposed in / on the same rigid frame 8073, thereby allowing the position of the BB plane (and the position of the BB therein) to be accurately tracked using the optical tracking array.

[0215] like Fig.27C As shown in FIG. 1 , a temporary fixture 8050 is attached near the source 3001 (e.g., between the source 3001 and the detector 3005). The fixture 8050 has a tracking array (comprising optical tracking markers 8053A, 8053B, 8053C, and 8053D) and a BB plane 8055 (also referred to as an x-ray opaque fiducial pattern) embedded in a frame 8051 between the source 3001 and the detector 3005. The tracking array (comprising optical tracking markers 8053A, 8053B, 8053C, and 8053D) is in a known position relative to the BB plane 8055 because both are fixed to and / or disposed in / on the same rigid frame 8051, thereby allowing the position of the BB plane 8055 (and the position of the BB therein) to be accurately tracked using the optical tracking array. The BB plane 8055 is perpendicular to the path of the x-rays (from the source 3001 to the detector 3005) and is a few inches away from the x-ray source 3001.

[0216] Fig.28 Draws Fig.25 (Operation 25003) or Fig.26(Operation 26003) may be an embodiment of a reference fluoroscopic image / snapshot of a standard position of before and after snapshots, where the detector 3005 is above the operating table and the source 3001 is below the operating table. In a standard fluoroscopic C-arm system, the position is typically marked as 0° tilt Rx and 0° track Rz. The pinhole camera model constraints will be applied to the fluoroscopic image / snapshot taken at the reference position (operation 25003 or operation 26003) to accurately position the source 3001 based on the projection of the BB from the BB plane 8055 of the fixture 8050 on the source 3001 and based on the projection of the BB from the BB plane of the fixture 8010 on the detector 3005. The optical tracking markers (8075A, 8075B, 8075C, 8075D, 8053A, 8053B, 8053C, and 8053D) on the fixtures 8010 and 8050 will be used to track the distances between the BB planes required by the pinhole camera model. Methods for applying projection constraints to data sets are known in the art and are available through open source software such as OpenCV (Open Source Computer Vision, http: / / openCV.org). Since the reference fluoroscopic snapshot is only used for calibration purposes, the fluoroscopic snapshot can be taken without anything in the area between fixtures 8010 and 8050. By taking such an "air snapshot", it can be provided / ensured that good image contrast will be achieved and image processing will be able to effectively detect BB shadows, such as Fig.28 As shown in .

[0217] Fig.28 An example of a fluoroscopic x-ray air image / snapshot 28000 taken with temporary fixtures 8010 and 8050 attached to detector 3005 and source 3001, respectively, is depicted. The inner ring of BB shadow 28011 and the inner ring of BB shadow 28015 are from the BB of source fixture 8050, and the outer ring of BB shadow 28017 is from detector fixture 8010. The diameter of the BB near source 3001 (contained in fixture 8050) is 32% of the diameter of the BB near detector 3005 (contained in fixture 8010), but due to parallax during projection, the BB shadows appear to be approximately the same. The lighter shadow is from frame 8073 of fixture 8010.

[0218] Fig.27AThe orientation of the gravity vector (also referred to as the gravity vector value) of the C-arm 3009 (including optical tracking markers 4001A and 4001B) can be displayed as a unit vector with X, Y, and Z components, and the gravity vector value can be continuously updated by the software to monitor the current orientation of the C-arm 3009. The real-time gravity vector value can be used as a motion sensing control feature. If the gravity vector is not static, the C-arm is still moving after the last movement or the gravity vector is still swinging. Therefore, the sensed movement of the gravity vector can be used to reduce / prevent / limit the frame grabber from taking or acquiring unusable fluoroscopic snapshots through the software. The orientation of the gravity vector (i.e., the gravity vector value) can also be recorded as a reference, and the reference value can be used in the same or different surgical situations to help guide the user from a previous orientation of the C-arm to a specific angular orientation or angular increment. For example, if the gravity vector indicated a specific orientation for one fluoroscopic snapshot, but that orientation was later found to be invalid in the registration, and the user wishes to try a new C-arm orientation that is exactly 10° different from the previous orientation, the stored value of the gravity vector can help accurately find the new position by displaying and comparing the current and previous gravity vectors. The software can also display the calculated angular difference between the previous gravity vector value and the current gravity vector value as the arc cosine of the vector dot product of the two unit vectors.

[0219] exist Fig.25 Cycle or Fig.26 In the loop, when tracking Fig.27A , Fig.27B and Fig.27C The C-arm 3009 will swing through its range of standard operating positions when the marker array on the fixture and the gravity vector (or whatever C-arm orientation is used) are used. Importantly, if a gravity vector is used, the gravity vector will not move to any position where it is suspended from itself or another part of the fixture, or is not freely hanging for any reason within the range of study. Therefore, the attachment of the gravity vector can be designed in such a way that when it is in the most common orientation, it is hanging down, and in the least common orientation, it is inverted 180°. The least common orientation may be with the x-ray detector 3005 located below the operating table. Said orientation is not possible because the tracking array on the detector 3005 (i.e., comprising Fig.27A The optical tracking markers 4005A, 4005B, 4005C and 4005D) in the fixture may not be visible in such an orientation.

[0220] When collecting data for the range of available orientations of the C-arm 3009, sufficient samples should be collected so that the relationship between the orientation of the C-arm 3009 and the source 3001 offset (relative to the detector 3005) is well characterized. The primary data to be sampled are the source 3001 and detector 3005 positions, which are established relative to their tracking arrays in the reference snapshot. Additional fluoroscopic x-ray snapshots are not required, as the tracking array should locate the source 3001 and detector 3005 equally or more accurately than the projection data.

[0221] For sampling data at different orientations of the C-arm 3009 (also referred to as C-arm orientations), a relationship between the source 3001 offset and the C-arm orientation must be established. There are different ways to establish this relationship. One way is to use an interpolation lookup table in which different angular orientations corresponding to orbit Rz and tilt Rx are tabulated along with the offsets of the source 3001 in the X, Y, and Z dimensions. Then, when a new fluoroscopic x-ray snapshot is taken at a particular orientation of the C-arm 3009, the X, Y, and Z offsets are read from the lookup table. If there is no entry for the combination of orbit Rz and tilt Rx being used, the offset is interpolated linearly or nonlinearly from the nearest available entry.

[0222] A different way to set the relationship between the source 3001 offset and the C-arm orientation is to make a physical model of the C-arm 3009. The physical model can be calibrated with material and geometric properties that accurately predict how the C-arm 3009 will flex at different orientations. The data collected when sampling at different orientations will be used to adjust or optimize the model so that the model is able to generate accurate outputs when given a new orientation of the C-arm 3009 for a new x-ray fluoroscopic snapshot that is not part of the adjustment data set.

[0223] A third way to set up the relationship between source 3001 offset and C-arm orientation is to utilize machine learning, such as a neural network. The training data set will be data collected during a scan of the C-arm 3009 and the corresponding source 3001 offset. The accuracy of the neural network model predicting source 3009 offset based on C-arm orientation will depend on how much data is provided to the neural network in its training.

[0224] Another way to establish the relationship between the source 3001 offset and the C-arm orientation is to fit a general function of two input parameters to the output, which can be the X, Y or Z Cartesian coordinate shift of the source position. The two input parameters i1 and i2 can be the orbit Rz and tilt Rx angles or the two components of the gravity vector (because the unit orientation vector has two degrees of freedom). Therefore, the function can be written as:

[0225] X=f1(i1,i2)

[0226] Y=f2(i1,i2)

[0227] Z=f3(i1,i2)

[0228] Once calibrated, three such equations will fully define the source 3001 displacement as a function of the C-arm orientation. The sampled data can be fitted to each function by appropriate computational methods such as Levenberg-Marquardt Optimization. The functional form of fitting such a process can be a sine-cosine relationship, a polynomial model, a Gaussian mixture model, or a general curve with arbitrary degrees of freedom. In one example of an embodiment, based on i1 and i2, a function can be fitted to the X, Y, and Z Cartesian coordinates such that

[0229] X=p1*i1 2 *i2+p2*i2 2 *i1+p3*i1 2 +p4*i2 2 +p5*i1*i2+p6*i1+p7*i2+p8

[0230] Where p1 to p8 are parameters solved for by the optimization method, and each parameter represents the weight of the contribution of a component to the function. Similar functions can be written for Y and Z, each with its own set of weights solved separately.

[0231] After completing the calibration procedure and recording the deflection of the source 3001 over a range of C-arm orientations, Fig.27B and Fig.27C The temporary fixtures will be removed, leaving only Fig.27A Permanent fixation device for actual use during surgery / situation. References can be provided / secured during data collection Fig.27A The offset from the permanent fixture to the temporary fixture on the detector 3005 should be fixed so that it is easy to determine / calculate the position of the detector 3005 and the relationship between the permanent fixture and the temporary fixture. Fig.27A A permanent array of relationships. Fig.25 or Fig.26 In the second step of the flowchart, the Fig.27A The offset of the source 3001 from the permanent fixture to the upright position, then during data collection, the offset of the source 3001 will be Fig.27A The position in the coordinate system of the permanent fixture is related to the position in the permanent fixture to keep the relationship always according to the permanent fixture. Alternatively, it can be added after the calibration is completed Fig.27A permanent fixture, and can be transferred from the previously defined Fig.27B The relationship of the temporary fixture is transformed into Fig.27AThe temporary fixture is then removed.

[0232] Rather than attempting to associate the gravity vector with a specific tracking fixture, or designing tracking fixture attachments to ensure that the fixtures are always attached relative to each other in a reproducible manner, the gravity vector can be stored and accessed as a "normalized" value. For normalization, two specific orientations can correspond to reference positions. In one embodiment of the normalization concept, the upright position of the C-arm 3009 can correspond to (0,0,1), or exact alignment with the Z axis, and the lateral position can correspond to (1,0,0), or closest alignment with the X axis. To implement normalization of the gravity vector in practice, the C-arm 3009 would be placed in an upright position and the gravity vector captured relative to any reference tracking fixture ( Fig.23A , Fig. 23B , Fig.23C , Fig.24A , Fig. 24B or Fig.27A ). The C-arm 3009 is then placed in a lateral position and the gravity vector values ​​relative to that position are captured. At each position, a 3x3 rotation matrix from upright to actual orientation is recorded; that is, R0 may represent a 3x3 rotation of the gravity vector relative to the fixture in the upright position, R 90 It can represent a 3x3 rotation of the gravity vector relative to the fixture in the landscape position. If the unit vector g originally aligned with the Z axis g = (0,0,1) is rotated by R0, then the new vector g that is exactly aligned with gravity Z is Z would be described as:

[0233] g Z =R0*g.

[0234] Similarly, if the same vector is rotated R90, the new vector g that is approximately aligned with gravity X is Xa would be described as:

[0235] g Xa =R 90 *g.

[0236] The cross product of these vectors (after normalization to unit vectors) will be the unit vector g that is exactly aligned with the gravity Y Y :

[0237] g Y =g Z xg Xa .

[0238] G Y and g Z The intersection gives a unit vector that is exactly aligned with gravity X:

[0239] g X =g Y xg Z .

[0240] Therefore, using these vectors as columns of a 3x3 rotation matrix R gives us the rotation transformation from the identity matrix to the normalized gravity vector orientation:

[0241] R=[g X g Y g Z ].

[0242] When applied to any gravity vector g obtained relative to the tracking fixture F When the matrix R inv The inverse of gives the normalized gravity vector:

[0243] g N =R inv *g F .

[0244] Newer C-arm fluoroscopy systems utilize flat panel x-ray detectors that can form undistorted x-ray images. However, older C-arm fluoroscopy systems still in use in many hospitals utilize image intensifiers as x-ray detectors. Images created using older C-arm fluoroscopy systems may suffer from greater sigmoid and pincushion distortions than images created using flat panel x-ray detectors. The noise levels are large enough and unpredictable enough that the raw images may lack the accuracy required to detect the location of source 3001 using the pinhole camera constraints mentioned previously. Therefore, the raw images from the image intensifier-based detector may require an additional step to de-warp the images before they can be used for registration for any type of workflow, such as a fluorescence-only or preoperative CT workflow. In the registration method, where there are two BB planes on the fluoroscopy tracking fixture, as described above with respect to Fig. 20As discussed, the BB from plane 21009b closest to the detector 3005 is used for dewarping by applying a dewarping algorithm. One such dewarping algorithm is a deformation algorithm, in which the pixels within the triangle formed by the three detected BB shadows are stretched / shrunk in two dimensions so that the triangle matches a known shape. In the proposed new alignment method, the BB on the detector 3005 can also continue to be used for dewarping. Unlike the old fixture, only one plane is required for the BB (instead of two planes). The single plane will be used to support dewarping and also provide orientation landmarks to provide / ensure that each new image matches any screen orientation and offset that may vary from image to image. In addition, unlike the old method, since dewarping only occurs within the area bounded by the visible BB, it should be possible to collimate as much as desired / needed. At least 3 oriented BBs still need to be visible within the collimation area to provide / ensure the correct image orientation. Since during the calibration process, the BB on the detector 3005 Fig.27B The tracking fixture also uses / requires the BB plane, so the image intensifier based detector 3005 Fig.27B The temporary tracker can double as a permanent tracker.

[0245] Using two BB planes 21009a and 21009b Fig. 20 The fluoroscopic fixture has some features to account for user-controlled screen image rotation, image magnification, and image flipping. It is important to take into account these factors, which may vary from snapshot to snapshot depending on how the user adjusts the settings and the noise inherent to the system, to obtain an accurate mapping of the 3D coordinate system to the 2D image and vice versa. To account for magnification, some of the BBs can be arranged in a ring with a known diameter, and the spacing between the dewarping BBs can be known. To account for on-screen image rotation, the BBs in existing image intensifier fixtures are arranged in an asymmetric pattern. The asymmetric pattern is rotationally asymmetric and is also asymmetric from front to back, so that flipped and / or rotated images can be immediately identified.

[0246] When used with a flat panel device, the new fluoroscopic fixture with a single plane does not need to utilize any BBs for dewarping, but only needs to consider image orientation, flipping and magnification. The existing pattern on the far plane 21009a of the old fixture uses 6 directional BBs distributed at different rotational positions of a constant radius from the center of the fixture (not shown). Such an arrangement will meet the requirements of considering orientation, flipping and magnification, but may be somewhat restrictive for collimated snapshots. Another possible BB layout may utilize clusters of BBs in groups of 3 to allow collimation from top to bottom or left to right, such as Fig.29A , Fig.29B and Fig.29C As shown in Fig.29AAs shown in FIG. 2 , fixture 29001 may be disposed on face 3006 of detector 3005 between detector 3005 and source 3001. Fixture 29001 may include four groups of BBs (also referred to as references) 29005A, 29005B, 29005C, and 29005D, each group including three BBs, and wherein each group of three BBs is provided in a different pattern. Fig.29A In the embodiment, fixture 29001 includes an optical tracking array having optical tracking marks 29011A, 29011B, 29011C and 29011D provided in a fixed relationship relative to the BBs of groups 29005A, 29005B, 29005C and 29005D. Thus, fixture 29001 can be like Fig.27B The same fixture 8010 is used for calibration as in the example above, but because the fixture 29001 is intended to remain on the detector 3005 during surgery, the tracking array (comprising tracking markers 29011A, 29011B, 29011C, and 29011D) extends to one side of the detector 3005 and away from the source 3001, as shown in FIG. Fig.29A As shown in .

[0247] Fig.29B and Fig.29C Provide use Fig.29A An example of an image captured by the collimation device 29001. Fig.29B In , shadows 29005B' and 29005D' of BB groups 29005B and 29005D are present in the image, and the pattern of these BB groups can be used to determine the orientation of the image. Fig.29C , shadows 29005A' and 29005C' of BB groups 29005A and 29005C are present in the image, and the pattern of these BB groups can be used to determine the orientation of the image.

[0248] Another possible minimal fiducial layout could be a crosshair through the center of the screen, with additional crosshairs or spaces and asymmetric features, such as Fig.30 . Other possible features with useful / necessary properties of asymmetry and spacing may be provided to detect these factors. In addition to being useful for detecting image rotation, magnification, and / or flipping, these fiducials can be reprojected onto the image by software, and if calibrated correctly, these fiducials should overlap perfectly, giving the user confidence in a valid registration. Such software features are similar to how the BBs of two planes in old fluoroscopic fixtures were reprojected onto the image to check if the registration was good.

[0249] Fig.29AThe configuration of the BBs on fixture 29001 is depicted, with clusters of 3 BBs in the top (29005D), bottom (29005B), left (29005A), and right (29005C) regions. Positioning the BBs in this manner, 6 BBs can be easily detected even in a highly collimated snapshot, with Fig.29B is an x-ray fluorescence snapshot image with left-right collimation, and Fig.29C is an x-ray fluorescence snapshot image with upper and lower collimation.

[0250] Fig.30 Add to Fig.29A An example of a reference configuration of features of the detector fixture 29001 that may appear on a fluoroscopic x-ray snapshot to serve as a reference for image rotation, translation, magnification and / or flipping. The ring 30001 has a known diameter and the length of each line segment 30005A, 30005B and 30005C preceding the corresponding gap 30007A, 30007b and 30007C is also known. The gaps are placed asymmetrically so that it is known rotationally and with respect to flipping where the gaps should be on the image. Crosshairs may also be projected onto the image by software to provide feedback to the user as to what the relevant calibration considers to be the center of the image plane.

[0251] Because each new image captured after calibrating the system is related to the reference image through transformations (e.g., rotation, scaling, translation, and / or flipping), it is important that the fixture be securely attached to the x-ray detector 3005 (image plane). If the fixture slips, the position of the source relative to the image will no longer be accurate, even if adjusted for gravity. One approach is to collect a new, calibrated reference air snapshot at the beginning of the surgery / situation, along with gravity vector information for both upright and lateral positions. The reference air snapshot will be free of any anatomical structures, and there will be a second BB plane to allow the pinhole camera model to be used to accurately calculate the position of the source 3001. The second plane can be the tracked BB plane, such as Fig.27C The source fixture may be plane 8055 of the detector or a rigidly attached temporary plane offset by a known amount from the primary plane. Another approach is to design the detector fixture so that it always remains well attached, or if it is removed it can be reinstalled in the exact same position.

[0252] Because accuracy relies on the detector fixture 29001 being secured and / or accurately reattached to the detector 3005 in a reproducible and accurate manner, it may be desirable to provide a way to know if the calibration is still valid after a period of time. According to some embodiments, a method of assessing the effectiveness of the calibration is to attach one or more monitoring markers 8091 and 8093. In some embodiments, one monitoring marker 8091 is attached on / near the detector 3005 and one monitoring marker 8093 is attached on / near the source 3001, such as Fig.31 . For example, these monitoring markers can be reflective spheres, reflective tags, or activated markers that can be detected by the tracking system in the same manner as the permanent fixture. When performing calibration, the positions of these markers will be optically tracked relative to the permanent fixture. Later, when checking calibration, the monitoring markers 8091 near the detector 3005 should be in the same fixed position relative to the permanent tracking fixture 4000 and / or 8010 as during calibration. If the permanent tracking fixture or its tracking array is displaced or bent, the monitoring markers can be used to detect the offset of its predicted position, and this is obviously problematic. In addition, since the monitoring markers 8093 near the source 3001 should move in the same way as the source 3001 moves with the change of the C-arm and / or detector 3005 orientation, the system should be able to calculate and predict where the monitoring markers near the source 3001 will move on any C-arm and / or detector 3005 orientation. When checking calibration, the C-arm 3009 can be placed in several different orientations without taking x-rays. If the tracked monitor marker position is not in the predicted position (based on the recalled monitor marker upright position adjusted to the calibration offset) at all tested orientations of the C-arm 3009 and / or detector 3005, then the calibration is incorrect and needs to be rerun.

[0253] Fig.31 is a photograph of a C-arm 3009 with fixtures 8050, 8010 and 4000, and monitoring markers 8091 and 8093 drawn in two locations, one on / near source 3001 and one on / near detector 3005. The monitoring markers will allow the accuracy of the calibration to be checked before starting a procedure / case.

[0254] Fig.32 The flowchart of summarizes the overall workflow when using the fixture after calibration. Each new image coming in is aligned to the reference image, and the source 3001 has a known shift relative to the reference image, so if the image-to-fixture transformation of the reference image is known or has been found previously using two planes, a single plane fixture can be used according to the described workflow.

[0255] Fig.32is a flow chart of the overall workflow when utilizing a calibrated fixture on the day of a procedure / case (eg, surgery) according to some embodiments of the present inventive concepts.

[0256] At block 32001, the fixture 4000 is attached to the C-arm 3009, as shown in FIG. Fig.31 drawn.

[0257] At box 32005, if the attachment / alignment of the fixture 4000 on the C-arm 3009 is not reproducible, additional operations may be performed. If the attachment / alignment of the fixture 4000 on the C-arm 3009 is not reproducible at box 32005, anterior-posterior AP fluoroscopic images / snapshots are taken at box 32009 (where the detector 3005 is vertically aligned on the source 3001), and lateral fluoroscopic images / snapshots are taken at box 32009 (where the detector 3005 and source 3001 are horizontally aligned), and these fluoroscopic snapshots are used to normalize the gravity vector of the fixture 4000. If the attachment / alignment of the fixture 4000 at box 32009 is reproducible, the fluoroscopic snapshots at box 32009 may be omitted.

[0258] At box 32011, a fluoroscopic image is captured while tracking the fixation device 4000 and the gravity vector.

[0259] At box 32015, the image is transformed to match the reference image, taking into account rotation, translation, magnification and / or flipping.

[0260] At box 32019, the source 3001 offset is found for the detected gravity vector.

[0261] At box 32021, a transform of the image to the fixture is calculated by adjusting the existing transform of the reference image based on the source 3001 offset / shift.

[0262] According to some embodiments, a feature may be provided to detect and indicate the presence of x-rays. Such a feature is useful because the tracking information associated with the snapshot must be synchronized with the image data in order for the registration to be accurate. Currently, the system detects when the image through the video capture board changes, at which time the tracking data is polled. However, new image detections may be false negatives or false positives, or movement of the C-arm 3009 during snapshot capture may cause inaccuracies.

[0263] According to some embodiments, one mechanism for synchronizing imaging and tracking data is to transmit the state of the electronic x-ray detector to the robotic system (e.g., to a Fig.35The sync signal can be read via the DIO (digital input output) port and used by the software to ensure that tracking data is only collected when x-ray is active.

[0264] According to some other embodiments, another mechanism for synchronizing imaging and tracking data is to use information from the electronic x-ray presence sensor 33019 as a signal that causes one or more tracking markers (e.g., tracking markers of the activation marker array 33001) to flash only when x-rays are present / detected. If all markers of the array flash only when x-rays are detected, the system will only be able to track the fixture during an x-ray fluoroscopic snapshot. Alternatively, if only one marker on or adjacent to the array is turned on in response to detecting x-rays, the system can still determine whether the fixture is within the camera's field of view even if no x-ray snapshot has been collected. Fig.33 Such an active marker array 33001 is depicted in FIG.

[0265] According to some other embodiments, passive reflective markers (also referred to as removable markers) on the pivot or piston can be actuated to indicate when an x-ray fluoroscopic snapshot is taken. When x-rays are present / detected, a motor or other actuator will be turned on and the pivot or piston will be forced to a position. When x-rays are not present / detected, the motor will turn off and the pivot or piston will return the passive marker to its resting position under a spring-loaded mechanism. A tracking camera will detect the position of the marker and only record the position of the fixture when x-rays are activated. Such removable marker arrays can be used in place of Fig.33 An activation tag array of 33001.

[0266] According to some embodiments, Fig.333009A. The detector fixture may use an array of active markers 33001 that emit infrared light and can be tracked more accurately than passive reflective spheres. Additionally, the detector fixture may use an IMU 33005 instead of an optically tracked gravity vector to accurately detect the orientation of the C-arm 3009 and / or detector 3005 relative to gravity. The fixture may include onboard memory 33009A that may be loaded via a USB flash drive or other portable memory storage device, or via wireless communication, to hold the parameters of a lookup table or function that relates the IMU output to an offset (also called a shift) of the source 3001 relative to the detector 3005. A near field communication NFC tag, quick response QR and reader, barcode, radio frequency identifier RFID tag or other type of proximity device mounted on the C-arm detector 3005 near the fixture attachment can be used to automatically receive / retrieve the identification ID of the fluoroscopic C-arm system being used to provide / ensure that the correct parameters are loaded for the C-arm fluorescence system to which the detector fixture is attached, and allow the same detector fixture to be used for different fluoroscopic C-arm systems. The detector fixture can be wireless and battery powered, or wired A / C power. In either case, the active markers of the array 33001 can be electrically coupled to the rest of the fixture and / or powered using inductive coupling or other methods, allowing the markers to be provided as sterile and similar to how the end effector of the robot is currently attached to the robot arm above the drape, through the outside of the sterile drape 33015 surrounding the fixture and detector. The fixture can additionally have an x-ray presence sensor 33019 to synchronize tracking data with the fluoroscopic x-ray snapshot.

[0267] Fig.33 The detector fixture may also include a BB plane as discussed above, for example Fig.29A In general, Fig.33 The fixture can be similar to Fig.29A fixtures, adding / replacing active components such as IMU 33005, x-ray presence sensor 33019, and active marker array 33001. Also, Fig.33 The fixing device can include an array mount 33011 that allows the active marker array 33001 to be removably coupled via a sterile cover cloth 33015.

[0268] 1) As mentioned above Fig.29A , Fig.29B and Fig.29C As discussed, Fig.33The fixture can allow alignment without affecting the accuracy of the registration. Fig.33 As shown in FIG. 1 , the fixture may include a fiducial 29000 (also referred to as an x-ray opaque fiducial pattern, for example, provided using a metal BB or another x-ray opaque material), as described above with respect to FIG. Fig.29A , Fig.29B and Fig.29C As discussed. For example, each grouping (also referred to as a cluster) of 3 fiducials (e.g., provided using BB) may be provided in a different pattern / orientation to allow identification of each grouping / cluster in a fluoroscopic x-ray image. Furthermore, the x-ray opaque fiducial pattern may be provided in / on a layer of x-ray transparent material also referred to as a fiducial plane.

[0269] 2) Because multiple planes of BB are not needed, Fig.33 The fixation device in the apparatus may be less obtrusive, allowing it to be positioned closer to the patient.

[0270] 3) By reducing the number of BB shadows (also called base shadows), Fig.33 The fixation device may produce fewer artifacts in the x-ray images.

[0271] 4) Since image processing may be reduced according to some embodiments, the inability to distinguish fiducial x-rays may not be as much of a problem.

[0272] Fig.38A and Fig.38B Some embodiments of the present invention are shown in FIG. Fig.33 A plan view of the fixture 38000. Fig.38A and Fig.38B In the embodiment, the x-ray opaque fiducial pattern is provided in / on a fiducial plane 38011 (also referred to as a BB plane) provided using a layer of x-ray transparent material, such that all elements of the x-ray opaque fiducial pattern are in a single plane. As shown, the x-ray opaque fiducial pattern can be provided using a BB cluster (i.e., spherical metal balls, also referred to as ball bearings). According to some other embodiments, the x-ray opaque fiducial pattern can be provided using a crosshair pattern composed of fiducial segments, as described above with respect to Fig.30 According to other embodiments, circular x-ray opaque disks (e.g., metal disks) may be used, each of which has a flat upper and lower surface, or printed / painted circular patterns (using x-ray opaque material / paint) may be used to provide elements of the x-ray opaque reference pattern. For example, the x-ray opaque reference pattern may be painted / printed on the upper / lower surface of the reference plane 38011.

[0273] exist Fig.38A and Fig.38B Array mounting frame 33011a can be used to provide Fig.33 Array mounting members and can be provided using tracking marks 33001a, 33001b, 33001c and 33001d supported on array mounting frame 33011a Fig.33 The active marker array 33001 of FIG. 33001 can be removably coupled to the array mounting frame 33011a (e.g., using removable mechanical and / or magnetic couplings) to allow for example Fig.45A , Fig.45B and Fig.45C Attached on / through the sterile drape as shown in . In addition, a second array mounting frame 33011b (and corresponding tracking markers) may be provided on the opposite side of the fixture for a second active marker array. With two active marker arrays, a tracking sensor (e.g., a camera) can track the x-ray detector from either or both sides of the x-ray detector.

[0274] like Fig.40 , Fig.41 , Fig.43 , Fig.44 and Fig.45A As shown in FIG. 1 , fixture mounting pins 38001a, 38001b, 38001c and 38001d provide a removable mounting for the x-ray detector. Fig.38A and Fig.38B In some embodiments, each mounting pin may include a pair of snap-fit ​​projections, wherein each snap-fit ​​projection includes a flexible rod and an outwardly facing head, the outwardly facing head having an entry surface at an acute angle relative to the rod and a retaining surface orthogonal to the rod. Thus, the two snap-fit ​​projections of the mounting pin can be inserted into the corresponding mounting socket. The entry surfaces of the two heads facilitate the entry of the mounting pin into the mounting socket and the inward deflection of the corresponding rod. Once the mounting pin is fully inserted into the mounting socket and the two heads leave the mounting socket opening, the two snap-fit ​​projections will pop out so that the retaining surfaces of the two snap-fit ​​projections engage the outer surface of the mounting socket to securely attach the fixture to the x-ray detector. The mounting can be released by clamping the two heads of the mounting pin so that the retaining surface of the corresponding head is disengaged from the outer surface of the mounting socket, thereby allowing the mounting pin to be pulled out / pushed out of the mounting socket. Therefore, each of the mounting pins 38001a, 38001b, 38001c and 38001d can be configured to detachably cooperate with the corresponding mounting socket.

[0275] Additionally, the fixture may include mounting points 38015 to provide temporary mounting for a second radiopaque fiducial pattern in a second plane, such as reference Fig.41 and Fig.44 discussed.

[0276] therefore, Fig.39 and Fig.42 Mounting sockets 39001a, 39001b, 39001c, and 39001d are depicted on the x-ray detector 3005. Each of the mounting sockets is configured to receive a corresponding mounting pin, as described above with respect to Fig.38A and Fig.38B Therefore, if Fig.40 , Fig.41 , Fig.43 , Fig.44 and Fig.45A As shown in Fig.38A and Fig.38B The fixing device can thereby be detachably mounted on the x-ray detector 3005. For example, in Fig.43 In the embodiment, each mounting pin can be released from the corresponding socket by clamping the head and pushing it into the corresponding mounting socket.

[0277] like Fig.41 and Fig.44 As shown in FIG. 1 , a second x-ray opaque reference pattern 44001 (e.g., a BB pattern) may be provided in a second reference pattern plane 41001, which is temporarily mounted on a fixture using legs 44009 and mounting pins 44005, which are removably engaged with mounting points 38015 (described above with respect to FIG. 1 ). Fig.38A and Fig.38B ), and the second x-ray opaque fiducial pattern 44001 can be used to generate a correlation of the offset of the x-ray source 3001 relative to the x-ray detector 3005 as a function of the orientation of the x-ray detector 3005 relative to gravity. For example, a fluoroscopic x-ray imaging system can take an air snapshot image (i.e., an image without a patient present) at an orientation of a C-arm 3009 (e.g., a reference orientation of the C-arm, e.g., with the x-ray detector 3005 positioned vertically above the x-ray source 3001), where two fixtures contain a first fiducial pattern and a second fiducial pattern. Thus, the air snapshot image contains shadows corresponding to the two fiducial patterns in different planes, and the air snapshot image can be provided to a medical navigation system. For the air snapshot image, the fixture can also provide information about the orientation of the x-ray detector 3005 relative to gravity based on information from the IMU 33005, or the medical navigation system can determine the orientation of the x-ray detector 3005 relative to gravity based on the orientation of the gravity vector using information received via a tracking sensor (e.g., a camera system), as described above with respect to Fig.23A , Fig. 23B , Fig.23C , Fig.24A , Fig. 24B , Fig.27A and Fig.31 Therefore, the medical navigation system can generate a correlation of the offsets based on an air snapshot image containing shadows of the two reference patterns and using information about the orientation of the x-ray detector 3005 relative to gravity of the air snapshot image. According to some other embodiments, the second x-ray opaque reference pattern can be temporarily set on the x-ray source 3001, as described above with respect to Fig.27C According to some other embodiments, the optical tracking array can be coupled to the x-ray source 3001, as described above with respect to Fig.27C As discussed, the medical navigation system can use information received via the tracking sensor for the air snapshot image to determine the position / orientation of the x-ray source 3001, and in addition to the information discussed above, the position / orientation of the x-ray source 3001 can be used to determine the correlation of the offset.

[0278] According to some embodiments of the present inventive concept, the medical navigation system is configured to generate a correlation of the offset based on an air snapshot image from a fluoroscopic imaging system, wherein the air snapshot image corresponds to an orientation of the C-arm (e.g., a reference orientation of the C-arm, e.g., wherein the x-ray detector 3005 is vertically positioned above the x-ray source 3001). For example, the air snapshot image may be generated using the x-ray opaque fiducial pattern 29000 and the x-ray opaque fiducial pattern 44001 of the fixture 38000, as shown in FIG. Fig.41 and Fig.44 As shown in , the x-ray opaque fiducial pattern is temporarily coupled to the x-ray detector between the fixture and the x-ray source, so that the air snapshot image contains shadows corresponding to the x-ray opaque fiducial patterns 29000 and 44001. The medical navigation system can be configured to generate a correlation of the offset based on the air snapshot image containing shadows corresponding to the x-ray opaque fiducial patterns 29000 and 44001, and / or based on the positions of the x-ray source and / or the x-ray detector corresponding to the corresponding differently oriented x-ray sources of the C-arm, wherein the position of the x-ray source is determined based on an optical tracking array coupled to the x-ray source, and / or the position of the x-ray detector is determined based on the optical tracking array 33001.

[0279] According to some other embodiments of the present inventive concept, the medical navigation system is configured to generate a correlation of the offset based on an air snapshot image from a fluoroscopic imaging system, wherein the air snapshot image corresponds to an orientation of the C-arm (e.g., a reference orientation of the C-arm, e.g., wherein the x-ray detector 3005 is vertically positioned above the x-ray source 3001). For example, the air snapshot image may be generated using the x-ray opaque fiducial pattern 29000 of the fixture 38000 and the x-ray opaque fiducial pattern 8055, wherein the x-ray opaque fiducial pattern is temporarily coupled to the x-ray source between the fixture and the x-ray source, such as Fig.27Cand Fig.31 , so that the air snapshot image contains shadows corresponding to the x-ray opaque fiducial patterns 29000 and 8055. The medical navigation system can be configured to generate a correlation of the offsets based on the air snapshot image containing shadows corresponding to the first x-ray opaque fiducial patterns 29000 and 8055, and / or based on the positions of the x-ray sources and / or x-ray detectors corresponding to the respective different orientations of the C-arm, wherein the position of the x-ray source is determined based on an optical tracking array coupled to the x-ray source, and / or the position of the x-ray detector is determined based on the optical tracking array 33001.

[0280] Once the correlation of the offset is determined, the second x-ray opaque fiducial pattern (on the fixture or on the x-ray source 3001) and / or the tracking array on the x-ray source 3001 may be removed before performing the medical procedure. The correlation of the offset may be stored in the medical navigation system and used for registration, or the correlation of the offset may be saved in a memory at the fixture. If the correlation of the offset is saved at the fixture, the fixture may determine the orientation of the x-ray detector 3005 relative to gravity for each patient image during surgery (using the output from the IMU 33005), determine the offset of the patient image based on the correlation of the orientation relative to gravity and the offset for each patient image, and transmit an indication of the offset to the navigation system for each patient image. If the correlation of the offset is saved at the navigation system, the fixture may determine the orientation of the x-ray detector 3005 relative to gravity for each patient image during surgery (using the output from the IMU 33005), and transmit an indication of the orientation relative to gravity for each patient image to the medical navigation system, and the medical navigation system may determine the offset of the patient image based on the correlation of the orientation relative to gravity and the offset for each patient image. Either way, the medical navigation system can perform registration based on the airshow images and based on the corresponding offsets.

[0281] like Fig.45A , Fig.45B and Fig.45C, once the fixture is mounted on the x-ray detector 3005 and any temporary fiducial / tracking array is removed, a sterile drape 33015 may be provided over the fixture and x-ray detector 3005. For example, due to the electronic components contained in the fixture, it may be difficult to fully sterilize the fixture for surgery, so a sterile drape 33015 may be required. Once the sterile drape 33015 is provided, sterile tracking markers 33001a, 33001b, 33001c, and 33001d may be coupled to mounting points 44002a, 44002b, 44002c, and 44002d of the array mounting frame 33001a, either on or through the sterile drape. For example, the tracking markers may be magnetically mounted to the corresponding mounting points without penetrating the sterile drape, or the snap-fit ​​pins of each tracking marker may penetrate the sterile drape to provide a snap-fit ​​mount at the corresponding mounting point.

[0282] According to some embodiments, the optical monitoring marker 8091 may be provided on the C-arm 3009 and / or the x-ray detector 3005, as described above with respect to Fig.31 When the medical navigation system performs registration according to some embodiments, the optical monitoring marker 8091 can be used to verify the patient image.

[0283] Fig.34 The above is a picture of Fig.33 Block diagram of elements of a fixture controller 34000 for a detector fixture as discussed. As shown, the fixture controller 34000 may include a processor circuit 34007 coupled to an inertial measurement unit (IMU) circuit 33005 (also referred to as an inertial measurement unit or IMU), an x-ray presence sensor circuit 33019 (also referred to as an x-ray presence sensor / detector), a marker array interface circuit 33051 (also referred to as a marker array interface), a memory circuit 33009A (also referred to as a memory), and a communication interface circuit 33009B (also referred to as a communication interface). The memory circuit 33009A may include computer readable program code that, when executed by the processor circuit 34007, causes the processor circuit to perform operations in accordance with embodiments disclosed herein. According to other embodiments, the processor circuit 34007 may be limited to include a memory, thereby eliminating the need for a separate memory circuit.

[0284] Fig.35 The medical navigation system controller 35000 is shown (for example, in Figure 5408 of a robotic system). As shown, the medical navigation system controller 35000 may include a processor circuit 35007 (also referred to as a processor), which is coupled to an input interface circuit 35001 (also referred to as an input interface), an output interface circuit 35003 (also referred to as an output interface), a control interface circuit 35005 (also referred to as a control interface), a memory circuit 35009 (also referred to as a memory), and a communication interface circuit 35051 (also referred to as a communication interface). The memory circuit 35009 may include computer readable program code, which, when executed by the processor circuit 35007, causes the processor circuit to perform operations according to the embodiments disclosed herein. According to other embodiments, the processor circuit 35007 may be defined as including a memory, thereby eliminating the need for a separate memory circuit. The processor 35007 may use the communication interface 35051 to receive information about the offset of the x-ray source of the fluoroscopic x-ray imaging system relative to the x-ray detector of the fluoroscopic x-ray imaging system. For example, the communication interface 35051 may use wired or wireless (e.g., WiFi, Bluetooth, etc.) communication from Fig.34 and Fig.35 The fixture receives such information.

[0285] As discussed herein, operations for controlling a medical navigation system according to some embodiments of the present disclosure may be performed by a controller 35000, which includes a processor 35007, an input interface 35001, an output interface 35003, a control interface 35005, and / or a communication interface 35051. For example, the processor 35007 may receive user inputs through the input interface 35001, and such user inputs may include user inputs received through a foot pedal 544, a tablet 546, a touch-sensitive interface of a display 110 / 304, and the like. The processor 35007 may also receive position sensor inputs from a tracking subsystem 532 and / or a camera 200 through the input interface 35001. The processor 35007 may provide outputs through the output interface 35003, and such outputs may include information presented as graphical / visual information on the display 110 / 304 and / or audio outputs provided through the speaker 536. The processor 35007 may provide robot control information to the motion control subsystem 506 via the control interface 35005, and the robot control information may be used to control the operation of the robot actuators (e.g., the robot arms 104 / 306-308 / 604, also referred to as the robot's arms) and / or the end effector 112 / 602. The processor 35007 may also use the communication interface 35051 to receive information about the offset of the x-ray source of the fluoroscopic x-ray imaging system relative to the x-ray detector of the fluoroscopic x-ray imaging system. For example, the communication interface 35051 may use wired or wireless (e.g., WiFi, Bluetooth, etc.) communication from the processor 35007 to the motion control subsystem 506. Fig.33 and Fig.34 The fixture receives such information.

[0286] Reference will now be made to some embodiments according to the present inventive concept. Fig.36A and Fig.36B The operation of a fixture for a fluoroscopic x-ray imaging system including a C-arm is discussed in detail in the flowchart of FIG. Fig.34 The memory 33009A of the processor 34007 can provide instructions so that when the instructions of the modules are executed by the processor 34007, the processor 34007 executes Fig.36A and / or Fig.36B The corresponding operations of the flowchart.

[0287] exist Fig.36AAt block 36001 of the embodiment, correlations between different orientations of the x-ray detector 3005 (and / or the C-arm 3009) relative to gravity and corresponding offsets of the x-ray source 3001 relative to the x-ray detector 3005 may be provided in memory 33009A for the fluoroscopic x-ray imaging system. According to some embodiments, at block 36001, a single correlation may be provided in memory 33009A for a single fluoroscopic x-ray imaging system, and in such embodiments, the fixture may be used with the single fluoroscopic x-ray imaging system for which the correlation is applicable. For example, the correlation may be provided based on one or more calibrations as discussed above.

[0288] According to the following reference Fig.36BSome other embodiments discussed, multiple correlations may be provided in memory 33009A for corresponding different fluoroscopic x-ray imaging systems so that the same fixture can be used for different fluoroscopic x-ray imaging systems. According to such an embodiment at box 36001A, multiple correlations between different orientations of the C-arm 3009 and / or the x-ray detector 3005 of different fluoroscopic x-ray imaging systems and corresponding offsets of the x-ray source 3001 relative to the x-ray detector 3005 are provided in memory 33009A, and each of the multiple correlations is associated with a different identification of the different fluoroscopic x-ray imaging systems. Therefore, different correlations may be provided in memory 33009A for each of the fluoroscopic imaging systems in which the fixture will be used, thereby allowing one fixture to be used with different fluoroscopic x-ray imaging systems. At box 36001B, the processor 34007 receives (e.g., via the communication interface 33009B) an identification corresponding to the fluoroscopic imaging system in which the fixture will be used. For example, a smartphone (or other electronic device) may be used to read / capture an identification (e.g., a barcode, QR code, NFC tag, RFID tag, etc.) on a fluoroscopic x-ray imaging system with which the fixture is to be used, and the identification may be received by the processor 34007 via wireless communication from the smartphone or other electronic device via the communication interface 33009B, or the processor 34007 may receive the identification directly via the communication interface 33009B (e.g., by reading a near field communication NFC tag or a radio frequency identification RFID tag). At block 36001C, the processor 34007 may provide the association of block 36001 by selecting the association corresponding to the identification received at block 36001B. Thus, the fixture may include an attachment mount configured to provide a detachable mechanical coupling to each of the different fluoroscopic x-ray imaging systems, and when the fixture is mounted on one of the fluoroscopic x-ray imaging systems using the attachment mount, the identification of the x-ray imaging system may be used to select the association of the fluoroscopic x-ray imaging system. Furthermore, the fixture may include an x-ray opaque fiducial pattern in a single plane and a mounting structure (e.g., Fig.38A and Fig.38B The mounting pins 38001a-d) can be configured to provide a removable mechanical coupling to the x-ray detector 3005 such that when the fixture is coupled to the x-ray detector using the mounting structure, the reference pattern is on the face of the x-ray detector between the x-ray detector 3001 and the x-ray source 3005.

[0289] According to some embodiments, the correlation of box 36001 (or each correlation of box 36001A) may include a table in memory 33009A correlating different orientations of the x-ray detector 3005 (and / or C-arm 3009) with respect to gravity and corresponding offsets, a formula in memory 33009A correlating different orientations of the x-ray detector 3005 (and / or C-arm 3009) with respect to gravity and corresponding offsets, and / or a model in memory 33009A correlating different orientations of the x-ray detector 3005 (and / or C-arm 3009) with respect to gravity and corresponding offsets. If the fixture has correlations for different fluoroscopic x-ray imaging systems, a different table, formula, and / or model may be provided in memory 33009A for each of the fluoroscopic x-ray imaging systems.

[0290] At block 36005, the processor 34007 determines whether x-ray emissions from the x-ray source 3001 to the x-ray detector 3005 are present / detected based on information from the x-ray presence sensor 33019. By providing the x-ray presence sensor 33019 on the face 3006 of the detector 3005, the output from the x-ray presence sensor 33019 can be used to determine when to take a fluoroscopic image / snapshot, and therefore when offset information is needed.

[0291] According to some embodiments, at box 36009, the processor 34007 determines the orientation of the C-arm 3009 and / or the detector 3005 in response to detecting an x-ray emission from the x-ray source toward the x-ray detector at box 36005. For example, the orientation of the C-arm and / or the detector 3005 can be determined relative to gravity based on the output from the inertial measurement unit IMU 33005. By determining the orientation in response to detecting the x-ray emission, the orientation is determined at the snapshot fluoroscopic x-ray image / snapshot. Moreover, the orientation of the C-arm and / or the detector 3005 can be determined / defined relative to a reference orientation of the C-arm and / or the detector 3005, wherein the reference orientation of the C-arm and / or the detector 3005 is defined as the orientation of the C-arm and / or the detector 3005 with the x-ray detector vertically located above the x-ray source.

[0292] According to some embodiments, at block 36011, the processor 34007 determines an offset of the x-ray source 3001 relative to the x-ray detector 3005 in response to detecting x-ray emission from the x-ray source 3001 toward the x-ray detector 3005 at block 36005. For example, the offset of the x-ray source 3001 relative to the x-ray detector 3005 may be determined based on the orientation of the x-ray detector 3005 (and / or the C-arm 3009) from block 36009, based on the orientation of the x-ray detector 3005 (and / or the C-arm 3009) from block 36009 and a correlation in memory 33009A from block 36001, and / or based on the orientation of the x-ray detector 3005 (and / or the C-arm 3009) from block 36009 and based on a correlation in memory 33009A based on an identification associated with a fluoroscopic x-ray imaging system as discussed above with respect to blocks 36001A-C. The offset of the x-ray source relative to the x-ray detector may be determined / defined, for example, using a coordinate system of the x-ray detector, where the x-axis and y-axis are defined in the plane of the x-ray detector and the z-axis is defined as perpendicular to the plane of the x-ray detector.

[0293] According to some embodiments, at block 36015, the processor 34007 activates the tracking marker via the marker array interface 33051 in response to detecting x-ray emissions from the block 36005. For example, the tracking marker may be one of a plurality of light emitting diodes (LEDs) of the tracking marker array 33001, and activating the tracking marker may cause the plurality of LEDs to turn on, and / or the tracking marker may be a movable tracking marker, and activating the tracking marker may cause the movable tracking marker to move from a corresponding first position to a corresponding second position. Fig.33 As discussed, the marker array 33001 can be coupled to the array mount 33011 via a sterile drape 33015 covering at least a portion of the fixture and detector 3005. Thus, the marker array 33001 (or its tracking markers) can be activated (e.g., using electrical coupling, inductive coupling, mechanical coupling, radio coupling, electromagnetic coupling, magnetic coupling, etc.) by the processor 34007 using the interface 33051 (e.g., via the array mount 33011).

[0294] According to some embodiments, at block 36015, one or more tracking markers may be activated by turning on one or more light emitting diode (LED) tracking markers in response to detecting x-ray emissions at block 36005. According to some other embodiments, a fixture may be provided with a fixed / passive array of tracking markers and a movable tracking marker, and the movable tracking marker may be activated at block 36015 by moving the movable tracking marker relative to the array of tracking markers in response to detecting x-ray emissions at block 36005.

[0295] According to some embodiments, at block 36017, the processor 34007 provides an indication of the offset of the x-ray source from block 36011 to the medical navigation system via the communication interface 33009B. For example, the offset may be provided to the medical navigation system via a wired coupling or via a wireless coupling (e.g., via a wireless network, Bluetooth, etc.) via the communication interface 33009B. The medical navigation system may thus use the indication of the offset to provide registration, as discussed below.

[0296] According to some other embodiments, the correlation between the orientation and offset of the x-ray detector (and / or C-arm) relative to gravity may be provided at the medical navigation system (rather than the fixture). In such embodiments, the processor 34007 may provide an indication of the orientation of the x-ray detector 3005 (and / or C-arm) relative to gravity to the medical navigation (via a wired or wireless coupling, e.g., via WiFi, Bluetooth, etc.) via the communication interface 33009B in response to detecting an x-ray emission, and the medical navigation system may use the orientation of the x-ray detector 3005 (and / or C-arm) relative to gravity and the correlation between the orientation and offset of the x-ray detector 3005 (and / or C-arm) to determine / provide the offset of the x-ray source relative to the x-ray detector. In either embodiment, the processor 34007 may provide information (e.g., detector / C-arm orientation or offset) to the medical navigation system based on the x-ray detector 3005 (and / or C-arm orientation) relative to gravity for use by the medical navigation system to perform registration. Because the C-arm and the x-ray detector are connected and / or integrated, the orientation of the x-ray detector 3005 may refer to the orientation of the C-arm, and / or the orientation of the C-arm 3009 may refer to the orientation of the x-ray detector 3005 .

[0297] Reference will now be made to some embodiments according to the present inventive concept. Fig.37 The operation of the medical navigation system is discussed in the flowchart. For example, the modules can be stored in Fig.35 The memory 35009 of the present invention can provide instructions so that when the instructions of the modules are executed by the processor 35007, the processor 35007 executes Fig.35 The corresponding operations of the flowchart.

[0298] According to some embodiments, at block 37005, the processor 35007 receives information defining a first patient image from a fluoroscopic x-ray imaging system via a communication interface 35051, wherein the fluoroscopic x-ray imaging system includes a C-arm 3009, an x-ray source 3001 at a first end of the C-arm, and an x-ray detector 3005 at a second end of the C-arm. The first patient image may include a shadow corresponding to a radiopaque fiducial pattern of a fixture.

[0299] In accordance with some embodiments, at box 37009, the processor 35007 receives an indication of a first offset of the x-ray source 3001 relative to the x-ray detector 3005, wherein the first offset is associated with the first patient image, and wherein the indication of the first offset is received via the communication interface 35051.

[0300] According to some embodiments, receiving information defining the first patient image at block 37005 and receiving an indication of the first offset at block 37009 may include receiving a digital file (e.g., a Digital Imaging and Communications in Medicine (DOCOM) file) containing the information defining the first patient image and the indication of the first offset. According to some other embodiments, the first offset may be received separately from the information defining the first patient image. For example, the first offset may be received via a wireless coupling such as a WiFi coupling, a Bluetooth coupling, etc.

[0301] In accordance with some embodiments, at block 37011, the processor 35007 determines a first position (also referred to as a location) of the x-ray detector 3005 and / or the x-ray source 3001 based on a first position (also referred to as a location) of the tracking marker array 33001 on the x-ray detector 3005 determined using tracking information from a tracking sensor (e.g., tracking camera 200), wherein the first position is associated with the first patient image. For example, the first position of the x-ray detector may be determined based on a position of the tracking marker array 33001 determined using tracking information from the tracking sensor, and / or the first position of the x-ray detector may be determined in response to detecting a first activation of a tracking marker (e.g., one or more markers of the array 33001) using tracking information from the tracking sensor. For example, the activation may be detected based on detecting movement of the marker(s) and / or based on detecting active illumination of the marker(s).

[0302] According to some embodiments, at box 37012, the processor 35007 determines a first position (also referred to as a location) of the monitoring marker 8091 using tracking information from the tracking sensor, wherein the first position of the monitoring marker 8091 is associated with the first patient image. At box 37014, the processor 35007 may determine that the first position of the x-ray detector and / or the x-ray source is valid based on the first position of the monitoring marker 8091 relative to the tracking marker array at the first position of the tracking marker array. If the processor 35007 does not determine that the first position of the x-ray detector is valid, verification based on the first patient image, the first offset, and / or the first position of the x-ray detector may be prevented, and / or another first patient image (and associated information, such as the offset) may be requested.

[0303] According to some embodiments, at box 37015, the processor 35007 receives information defining a second patient image from the fluoroscopic x-ray imaging system via the communication interface 35051, wherein the first patient image and the second patient image are different.

[0304] In accordance with some embodiments, in box 37019, the processor 35007 receives an indication of a second offset of the x-ray source 3001 relative to the x-ray detector 3005, wherein the second offset is associated with a second patient image, wherein the first offset and the second offset are different, and wherein the indication of the second offset is received via the communication interface 35051.

[0305] According to some embodiments, receiving information defining the second patient image at block 37005 and receiving an indication of the second offset at block 37009 may include receiving a digital file (e.g., a Digital Imaging and Communications in Medicine (DOCOM) file) containing the information defining the second patient image and the indication of the second offset. According to some other embodiments, the second offset may be received separately from the information defining the second patient image. For example, the second offset may be received via a wireless coupling such as a WiFi coupling, a Bluetooth coupling, etc.

[0306] According to some embodiments, at block 37021, the processor 35007 determines a second position (also referred to as a position) of the x-ray detector 3005 and / or the x-ray source 3001 based on a second position (also referred to as a position) of the tracking marker array 33001 on the x-ray detector 3005 determined using tracking information from a tracking sensor (e.g., tracking camera 200), the second position being associated with the second patient image, and wherein the first patient image and the second patient image are different. For example, the second position of the x-ray detector may be determined based on the position of the tracking marker array 33001 determined using tracking information from the tracking sensor, and / or the second position of the x-ray detector may be determined in response to detecting a second activation of a tracking marker (e.g., one or more markers of the array 33001) using tracking information from the tracking sensor. For example, the activation may be detected based on detecting movement of the marker(s) and / or based on detecting active illumination of the marker(s).

[0307] According to some embodiments, at block 37022, the processor 35007 uses tracking information from the tracking sensor to determine a second position (also referred to as a location) of the monitoring marker 8091, wherein the second position of the monitoring marker 8091 is associated with the second patient image. At block 37014, the processor 35007 may determine that a second position of the x-ray detector and / or the x-ray source is valid based on the second position of the monitoring marker 8091 relative to the tracking marker array at the second position of the tracking marker array. If the processor 35007 does not determine that the second position of the x-ray detector is valid, verification based on the second patient image, the second offset, and / or the second position of the x-ray detector may be prevented, and / or another second patient image (and associated information, such as an offset) may be requested.

[0308] According to some embodiments, at block 37025, the processor 35007 provides a registration between a tracking coordinate system of a physical space monitored by a tracking sensor (e.g., tracking camera 200) and an image coordinate system of a first patient image and a second patient image from a fluoroscopic x-ray imaging system, wherein the registration is provided based on an indication of a first offset and a second offset. For example, the registration between the tracking coordinate system and the image coordinate system may be provided based on the first offset and the second offset, based on a first position and a second position of an x-ray detector, based on a correlation of an offset of an x-ray source relative to the x-ray detector as a function of an orientation of the x-ray detector relative to gravity, and / or based on shadows in the first patient image and the second patient image. According to some embodiments, the processor 35007 may determine a first position of the x-ray source based on a first position of the x-ray detector (e.g., determined using information about the marker array 33001 received through the optical tracking / camera system) and a first offset, the processor 35007 may determine a second position of the x-ray source based on a second position of the x-ray detector (e.g., determined using information about the marker array 33001 received through the optical tracking / camera system) and a second offset, and may provide registration based on a correlation of the offsets, based on the first offset, based on the second offset, based on the first shadow and the second shadow, based on the first position of the x-ray detector, based on the second position of the x-ray detector, based on the first position of the x-ray source, and / or based on one or more of the second position of the x-ray source. Furthermore, the registration may be provided in response to determining at blocks 37014 and 37024 that the first position and the second position of the x-ray detector are valid.

[0309] For example, the registration may be provided in response to detecting activation of a tracking marker (e.g., one or more markers of array 33001) using tracking information from a tracking sensor. For example, the activation may be detected based on detecting movement of the marker(s) and / or based on detecting active illumination of the marker(s). For example, the registration may be provided in response to detecting activation of a tracking marker (e.g., one or more markers of array 33001) using tracking information from a tracking sensor. For example, the activation may be detected based on detecting movement of the marker(s) and / or based on detecting active illumination of the marker(s).

[0310] According to some embodiments, at block 37029, the processor 35007 may control the robotic actuator to move the end effector to a target trajectory relative to the patient based on the first patient image and the second patient image from the fluoroscopic x-ray imaging system and based on the registration between the tracking coordinate system and the image coordinate system. For example, controlling the robotic actuator may include controlling the robotic actuator to move the end effector to the target trajectory relative to the patient based on the first patient image and the second patient image from the fluoroscopic x-ray imaging system, based on the registration between the tracking coordinate system and the image coordinate system, and based on the position of the end effector determined using tracking information from the tracking sensor. According to some other embodiments, the processor 35007 may present an image on a display based on the first patient image and the second patient image of blocks 37005 and 37015 and based on the registration. In such embodiments, the processor 35007 may present an image based on an image received through an optical tracking / camera system to also include a portion thereof, and / or based on a virtual element (e.g., a virtual screw or other medical device) that may be used for planning to also include a portion thereof.

[0311] As mentioned above about Fig.37As discussed, the medical navigation system may receive a corresponding offset from the fixture at boxes 37009 and 37019, such that a correlation between the orientation and offset of the x-ray detector 3005 (and / or C-arm) relative to gravity may be provided at the fixture, and the offset may be determined at the fixture. According to some other embodiments, a correlation between the orientation and offset of the x-ray detector 3005 (and / or C-arm) may be provided in a memory 35009 at the medical navigation system, and at boxes 37009 and 37019, the medical navigation system may alternatively receive (via the communication interface 35051) an indication of the orientation of the x-ray detector 3005 (and / or C-arm) relative to gravity. In such embodiments, the processor 35007 may determine a corresponding offset using the indication of the orientation of the x-ray detector 3005 (and / or C-arm) and the correlation between the orientation and offset of the x-ray detector 3005 (and / or C-arm) to determine / provide an offset of the x-ray source relative to the x-ray detector. In either embodiment, the processor 35007 may use information received from the fixation device (eg, C-arm / detector orientation or offset) to determine / provide offsets for performing the registration.

[0312] According to other embodiments, the fixture may not provide any information about the orientation or offset of the C-arm / detector. In such embodiments, the processor 35007 may use the same information as described above with respect to Fig.23A , Fig. 23B , Fig.23C , Fig.24A , Fig. 24B , Fig.27A and Fig.31 The gravity vector on the C-arm and / or x-ray detector discussed determines the orientation of the C-arm / detector for the corresponding image. As discussed above, the gravity vector includes at least two suspended tracking markers, and the processor 35007 can determine the orientation of the C-arm / detector relative to gravity by determining the position of the gravity vector tracking marker based on information received by the tracking sensor / camera at the time of the corresponding patient image. In such embodiments, the fixture can signal the timing of the image / snapshot by activating one or more markers of the active tracking marker array, and in response to detecting the activation of one or more markers of the active tracking marker array, the processor 35007 can use the gravity vector to determine the orientation of the C-arm / detector for the corresponding image. In the case where the gravity vector includes fixed tracking markers 4005A, 4005B, 4005C and 4005D and movable suspended tracking markers 4001A and 4001B, the processor 35007 can use the fixed tracking markers to determine the position of the x-ray detector, and the processor 35007 can use the movable suspended markers to determine the orientation relative to gravity.

[0313] In the above description of various embodiments of the inventive concept, it should be understood that the terms used herein are only used for the purpose of describing specific embodiments, and are not intended to limit the inventive concept. Unless otherwise specified, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the inventive concept belongs. It will be further understood that those terms defined in, for example, commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of this specification and in the relevant field, and unless explicitly defined herein, should not be interpreted in an idealized or overly formal sense.

[0314] When an element is referred to as "connected", "coupled", "responsive" (or its variant) to another element, it can be directly connected, coupled or responsive to another element, or there can be an intermediate element. On the contrary, when an element is referred to as "directly connected", "directly coupled", "directly responsive to" another element or its variant with another element, there is no intermediate element. Similar elements are referred to with similar numbers throughout the text. In addition, "coupling", "connection", "response" or its variants as used in this article may include wireless coupling, connection or response. Unless the context clearly indicates otherwise, as used herein, the singular forms "one / a kind (a, an)" and "the" are intended to include plural forms. For the sake of brevity and / or clarity, well-known functions or structures may not be described in detail. The term "and / or" includes any and all combinations of one or more items in the associated listed items.

[0315] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish an element / operation from another element / operation. Therefore, without departing from the teaching of the present invention concept, the first element / operation in some embodiments can be referred to as the second element / operation in other embodiments. Throughout this specification, the same reference numerals or the same figure designators represent the same or similar elements.

[0316] As used herein, the terms "comprise / comprising / comprises", "include / including / includes", "have / has / having" or variations thereof are open ended and include one or more stated features, integers, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or combinations thereof. In addition, as used herein, the general abbreviation "such as", derived from the Latin phrase "exempli gratia", may be used to introduce or specify one or more general examples of previously mentioned items and is not intended to limit such items. The general abbreviation "that is", derived from the Latin phrase "id est", may be used to specify a specific item from a more general statement.

[0317] Example embodiments are described herein with reference to block diagrams and / or flow diagrams of computer-implemented methods, devices (systems and / or apparatuses), and / or computer program products. It should be understood that the blocks of the block diagrams and / or flow diagrams and the combination of blocks in the block diagrams and / or flow diagrams can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to processor circuits of general-purpose computer circuits, special-purpose computer circuits, and / or other programmable data processing circuits to produce a machine so that instructions executed by processors of computers and / or other programmable data processing devices convert and control transistors, values ​​stored in memory locations, and other hardware components within such circuit systems to implement the functions / actions specified in the block diagrams and / or one or more flow diagram blocks, and thereby create a device (function) and / or structure for implementing the functions / actions specified in the block diagrams and / or one or more flow diagram blocks.

[0318] These tangible computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing device to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture containing instructions for implementing the functions / actions specified in the block diagram or one or more flowchart blocks. Thus, embodiments of the inventive concept may be embodied in hardware and / or software (including firmware, resident software, microcode, etc.) that runs on a processor such as a digital signal processor, which may be collectively referred to as a "circuitry system," "module," or variations thereof.

[0319] It should also be noted that, in some alternative implementations, the function / action marked in the frame may not occur in the order marked in the flow chart. For example, according to the function / action involved, the two frames shown continuously can actually be performed substantially simultaneously, or the frame can sometimes be performed in reverse order. In addition, the function of a given frame of a flow chart and / or block diagram can be divided into a plurality of frames, and / or the function of two or more frames of a flow chart and / or block diagram can be integrated at least in part. Finally, without departing from the scope of the inventive concept, other frames can be added / inserted between the frames shown, and / or frame / operation can be omitted. In addition, although some figures include arrows on the communication path to illustrate the main direction of communication, it should be understood that communication can occur in the direction opposite to the depicted arrow.

[0320] Although several embodiments of the inventive concept have been disclosed in the foregoing description, it should be understood that, with the benefit of the teachings presented in the foregoing description and the associated drawings, many modifications and other embodiments of the inventive concept to which the inventive concept relates will be conceivable. It should therefore be understood that the inventive concept is not limited to the specific embodiments disclosed above, and that many modifications and other embodiments are intended to be included within the scope of the appended claims. It is further contemplated that features from one embodiment may be combined or used together with features from different embodiments described herein. In addition, although specific terms are used herein and in the appended claims, they are used only in a general and descriptive sense, and not for the purpose of limiting the described inventive concept or the appended claims. The entire disclosure of each patent and patent disclosure cited herein is incorporated herein by reference in its entirety, as if each such patent or disclosure is individually incorporated herein by reference. Various features and / or potential advantages of the inventive concept are set forth in the appended claims.

Claims

1. A surgical imaging system configured for use with a fluoroscopic imaging system, wherein the fluoroscopic imaging system comprises a C-arm, an x-ray source at a first end of the C-arm, and an x-ray detector at a second end of the C-arm, the C-arm being mounted to a base of the system via a positioning mechanism, the positioning mechanism being configured to provide in-out Tx and up-down Ty translational movements, and to provide tilt Rx, yaw Ry, and orbit Rz rotational movements of the C-arm relative to the base, wherein the fluoroscopic imaging system is configured to generate an x-ray image based on x-rays received at the x-ray detector from the x-ray source, the surgical imaging system comprising: a fixture including an x-ray opaque fiducial pattern in a single plane, wherein the fixture is coupled to the x-ray detector such that the x-ray opaque fiducial pattern is located on a surface of the x-ray detector between the x-ray detector and an x-ray emitter; as well as A medical navigation system comprising a processor and a memory coupled to the processor, wherein the memory comprises instructions executable by the processor such that the processor is configured to: receiving, from the fluoroscopic imaging system, a first patient image corresponding to a first orientation of the C-arm, wherein the first patient image includes a first shading corresponding to the radiopaque fiducial pattern; receiving from the fluoroscopic imaging system a second patient image corresponding to a second orientation of the C-arm, wherein the second patient image includes a second shading corresponding to the radiopaque fiducial pattern; and Based on a correlation of an offset of the x-ray source relative to the x-ray detector as a function of the orientation of the x-ray detector relative to gravity, and based on the first shadow in the first patient image and the second shadow in the second patient image, an alignment between a tracking coordinate system of a physical space monitored by a tracking camera and an image coordinate system of the first patient image and the second patient image is provided.

2. The surgical imaging system of claim 1, wherein the radiopaque fiducial pattern comprises a plurality of BB fiducials located in the single plane.

3. A surgical imaging system according to claim 2, wherein the multiple BB references include a first BB reference cluster, a second BB reference cluster, a third BB reference cluster and a fourth BB reference cluster, wherein the first BB reference cluster is spaced apart from the third BB reference cluster along a first axis, wherein the second BB reference cluster is spaced apart from the fourth BB reference cluster along a second axis, and wherein the first axis is orthogonal to the second axis.

4. The surgical imaging system of claim 1 , wherein the x-ray opaque fiducial pattern comprises a first x-ray opaque fiducial segment, a second x-ray opaque fiducial segment, a third x-ray opaque fiducial segment, and a fourth x-ray opaque fiducial segment extending in different directions from a common point, wherein the first x-ray opaque fiducial segment has a first break at a first distance from the common point, wherein the second x-ray opaque fiducial segment has a second break at a second distance from the common point greater than the first distance, and wherein the third x-ray opaque fiducial segment has a third break at a third distance from the common point greater than the second distance.

5. A surgical imaging system according to claim 1, wherein providing the alignment based on the correlation of the offset of the x-ray source relative to the x-ray detector as a function of the orientation of the x-ray detector relative to gravity includes providing the alignment based on the correlation, based on a first orientation of the x-ray detector associated with the first orientation of the C-arm relative to gravity, and based on a second orientation of the x-ray detector associated with the second orientation of the C-arm relative to gravity.

6. A surgical imaging system according to claim 5, wherein the fixture includes an inertial measurement unit (IMU), the inertial measurement unit being configured to detect the orientation of the x-ray detector relative to gravity, and wherein the first orientation of the x-ray detector relative to gravity is determined based on a first output from the inertial measurement unit, wherein the C-arm is in the first orientation, and wherein the second orientation of the x-ray detector relative to gravity is determined based on a second output from the inertial measurement unit, wherein the C-arm is in the second orientation.

7. The surgical imaging system of claim 5, further comprising: a gravity vector coupled to the x-ray detector, wherein the gravity vector includes a pair of movably suspended optical tracking markers, wherein the first orientation of the x-ray detector relative to gravity is determined based on a first position of the pair of movably suspended optical tracking markers, wherein the C-arm is in the first orientation, and wherein the second orientation of the x-ray detector relative to gravity is determined based on a second position of the pair of movably suspended optical tracking markers, wherein the C-arm is in the second orientation.

8. The surgical imaging system of claim 1 , wherein the fluoroscopic imaging system comprises a plurality of mounting sockets coupled to the x-ray detector, wherein the fixture comprises a plurality of mounting pins, wherein each of the mounting pins is configured to removably engage with a corresponding one of the mounting sockets.

9. The surgical imaging system of claim 1, wherein the medical navigation system is configured to generate the correlation of the offset based on an air snapshot image from the fluoroscopic imaging system, wherein the air snapshot image corresponds to an orientation of the C-arm.

10. The surgical imaging system of claim 9, wherein the x-ray opaque fiducial pattern is a first x-ray opaque fiducial pattern, wherein a second x-ray opaque fiducial pattern is temporarily coupled to the x-ray detector between the fixture and the x-ray source such that the air snapshot image includes shadows corresponding to the first x-ray opaque fiducial pattern and the second x-ray opaque fiducial pattern, and wherein the medical navigation system is configured to generate the correlation of offsets based on the air snapshot image including the shadows corresponding to the first x-ray opaque fiducial pattern and the second x-ray opaque fiducial pattern.

11. The surgical imaging system of claim 9, wherein the x-ray opaque fiducial pattern is a first x-ray opaque fiducial pattern, wherein a second x-ray opaque fiducial pattern is temporarily coupled to the x-ray source between the fixture and the x-ray source such that the air snapshot image includes shadows corresponding to the first x-ray opaque fiducial pattern and the second x-ray opaque fiducial pattern, and wherein the medical navigation system is configured to generate the correlation of offsets based on the air snapshot image including the shadows corresponding to the first x-ray opaque fiducial pattern and the second x-ray opaque fiducial pattern.

12. A surgical imaging system according to claim 9, wherein the medical navigation system is configured to generate the correlation of the offset based on the air snapshot image and based on the position of the x-ray source corresponding to the corresponding different orientations of the C-arm, wherein the position of the x-ray source is determined based on an optical tracking array coupled to the x-ray source.

13. The surgical imaging system of claim 1 , further comprising: an optical tracking array coupled to the fixture; as well as an optical surveillance marker coupled to the x-ray detector and / or the x-ray source; wherein the medical navigation system is configured to: determine a first position of the x-ray detector and / or the x-ray source associated with the first patient image based on the optical tracking array; verifying the first position of the x-ray detector and / or the x-ray source based on a first position of the optical surveillance marker associated with the optical tracking array; determining a second position of the x-ray detector and / or the x-ray source associated with the second patient image based on the optical tracking array; verifying the second position of the x-ray detector and / or the x-ray source based on a second position of the optical surveillance marker associated with the optical tracking array; And providing the alignment based on the correlation of the offset, based on the first shadow and the second shadow, based on the first position of the x-ray detector and / or the x-ray source, based on verifying the first position of the x-ray detector and / or the x-ray source, based on the second position of the x-ray detector and / or the x-ray source, and based on verifying the second position of the x-ray detector and / or the x-ray source.

14. The surgical imaging system of claim 1, wherein the fixation device comprises: processor; as well as a memory coupled to the processor, wherein the memory contains instructions executable by the processor such that the fixture is configured to: detecting x-ray emissions from the x-ray source toward the x-ray detector, An offset of the x-ray source relative to the x-ray detector is determined in response to detecting the x-ray emission, and an indication of the offset of the x-ray source is provided to a medical navigation system.

15. The surgical imaging system of claim 14, wherein the memory contains instructions executable by the processor such that the fixation device is further configured to: In response to detecting the x-ray emission, an orientation of the x-ray detector is determined, wherein the offset of the x-ray source relative to the x-ray detector is determined based on the orientation of the x-ray detector.

16. The surgical imaging system of claim 15, wherein the memory contains instructions executable by the processor such that the fixation device is further configured to: A correlation between different orientations of the x-ray detector and corresponding offsets of the x-ray source relative to the x-ray detector is provided in a memory, wherein the offset of the x-ray source relative to the x-ray detector is determined based on the orientation of the x-ray detector and based on the correlation in the memory.

17. The surgical imaging system of claim 14, further comprising: a radiopaque fiducial pattern in a single plane; as well as A mounting structure is configured to provide a removable mechanical coupling between the fixture and the x-ray detector of the fluoroscopic imaging system such that when the fixture is coupled to the x-ray detector using the mounting structure, a reference is located on a face of the x-ray detector between the x-ray detector and the x-ray source.

18. The surgical imaging system of claim 1, wherein the medical navigation system comprises: processor; as well as a memory coupled to the processor, wherein the memory contains instructions executable by the processor such that the medical navigation system is configured to: receiving information defining a first image from a fluoroscopic x-ray imaging system, wherein the fluoroscopic x-ray imaging system comprises a C-shaped arm, an x-ray source at a first end of the C-shaped arm, and an x-ray detector at a second end of the C-shaped arm, providing an indication of a first offset of the x-ray source relative to the x-ray detector, wherein the first offset is associated with the first image, receiving information defining a second image from the fluoroscopic x-ray imaging system, wherein the first image and the second image are different, providing an indication of a second offset of the x-ray source relative to the x-ray detector, wherein the second offset is associated with the second image, and wherein the first offset and the second offset are different, and Providing registration between a tracking coordinate system of a physical space monitored by a tracking sensor and an image coordinate system of the first image and the second image from the fluoroscopic x-ray imaging system, wherein the registration is provided based on the indication of the first offset and the second offset.

19. The surgical imaging system of claim 18, wherein the memory contains instructions executable by the processor such that the medical navigation system is configured to: determining a first position of the x-ray detector using tracking information from the tracking sensor based on a first position of an array of tracking marks on the x-ray detector, wherein the first position is associated with the first image, determining a second position of the x-ray detector using tracking information from the tracking sensor based on a second position of the tracking mark array on the x-ray detector, wherein the second position is associated with the second image, Wherein the registration between the tracking coordinate system and the image coordinate system is provided based on the indication of the first offset and the second offset and based on the first position and the second position of the x-ray detector.

20. The surgical imaging system of claim 19, wherein the memory contains instructions executable by the processor such that the medical navigation system is further configured to: determining a first position of a surveillance marker using tracking information from the tracking sensor, wherein the first position of the surveillance marker is associated with the first image, and verifying the first position of the x-ray detector and / or the x-ray source based on the first position of the surveillance mark at the first position of the tracking mark array relative to the tracking mark array; determining a second position of the surveillance mark using tracking information from the tracking sensor, wherein the second position of the surveillance mark is associated with the second image; verifying the second position of the x-ray detector and / or the x-ray source based on the second position of the monitor mark at the second position of the tracking mark array relative to the tracking mark array; and Wherein the registration is provided in response to verifying the first position and the second position of the x-ray detector and / or the x-ray source.

Images