Robotic arm navigation using virtual bone mounts

Abstract

A system according to at least one embodiment of the present disclosure, the system comprising: a processor; and a memory having data stored thereon that, when processed by the processor, causes the processor to: determine a first registration between the navigation tracker and an anatomical element based on a first type of navigation element and a second type of navigation element both disposed on the navigation tracker; determining a second registration between the robotic arm and the navigational tracker based on a navigational element of a third type disposed on the navigational tracker; and navigating the robotic arm relative to the anatomical element based on the first registration and the second registration.

Description

Background Art

[0001] The present disclosure relates generally to surgical navigation, and more particularly to surgical navigation using a navigation tracker.

[0002] A surgical robot may assist a surgeon or other medical provider in performing a surgical procedure, or may autonomously complete one or more surgical procedures. Imaging may be used by a medical provider for diagnostic and / or therapeutic purposes. Patient anatomy may change over time, particularly after a medical implant has been placed in the patient's anatomy. Summary of the invention

[0003] Example aspects of the present disclosure include:

[0004] A system according to at least one embodiment of the present disclosure includes: a processor; and a memory having data stored thereon, which, when processed by the processor, causes the processor to: determine a first alignment between a navigation tracker and an anatomical element based on a first type of navigation element and a second type of navigation element both disposed on the navigation tracker; determine a second alignment between a robotic arm and the navigation tracker based on a third type of navigation element disposed on the navigation tracker; and navigate the robotic arm relative to the anatomical element based on the first alignment and the second alignment.

[0005] In any of the aspects herein, wherein the first type of navigational elements, the second type of navigational elements, and the third type of navigational elements are disposed on the navigation tracker in a predetermined configuration.

[0006] In any of the aspects herein, wherein the third type of navigation element comprises a first optical tracking marker tracked by a third imaging device, and wherein determining the second registration comprises the processor: receiving information about a first position of the first optical tracking marker and a first pose of the robotic arm from the third imaging device; and determining a position of the robotic arm relative to the first optical tracking marker based on the information about the first position of the first optical tracking marker and the first pose of the robotic arm.

[0007] In any of the aspects herein, wherein the first type of navigation element comprises a plurality of fluoroscopic markers detectable by a first imaging device, and wherein determining the first alignment comprises the processor: receiving a plurality of images depicting the plurality of fluoroscopic markers and the anatomical element from the first imaging device; and determining, based on the plurality of images, a position of each of the plurality of fluoroscopic markers relative to the anatomical element.

[0008] In any of the aspects of the present invention, wherein the second type of navigation element includes a plurality of navigation markers that can be detected by a second imaging device, and wherein determining the first alignment further includes the processor: receiving information about the position of each of the plurality of navigation markers from the second imaging device; and determining the position of each navigation marker relative to the anatomical element based on the information about the position of each navigation marker and the predetermined configuration.

[0009] In any of the aspects herein, wherein the robotic arm includes a second optical tracking marker, and wherein the third imaging device tracks the second optical tracking marker.

[0010] In any of the aspects herein, wherein the first imaging device comprises a fluoroscopic imaging device, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a laser tracker.

[0011] In any of the aspects herein, wherein the navigation tracker is disposed on a patient table, wherein the robotic arm is disposed on a second table a first distance from the patient table, and wherein the third imaging device is disposed a second distance from the robotic arm.

[0012] In any of the aspects herein, wherein the first imaging device comprises an O-arm, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a three-dimensional (3D) camera.

[0013] A system according to at least one embodiment of the present disclosure includes: a robotic arm, which is set to be a first distance away from a patient's bed; a processor; and a memory, on which data is stored, which, when processed by the processor, causes the processor to: determine a first alignment between the navigation tracker and an anatomical element based on a first type of navigation element and a second type of navigation element both set on the navigation tracker; determine a second alignment between the robotic arm and the navigation tracker based on a third type of navigation element set on the navigation tracker; and determine a first posture of the robotic arm relative to the anatomical element based on the first alignment and the second alignment.

[0014] In any of the aspects herein, wherein the first type of navigational elements, the second type of navigational elements, and the third type of navigational elements are disposed on the navigation tracker in a predetermined configuration.

[0015] In any of the aspects herein, wherein the third type of navigation element comprises an optical tracking target tracked by a third imaging device, and wherein determining the second alignment comprises the processor: receiving first information about a first pose of the optical tracking target from the third imaging device; and determining a first position of the robot arm relative to the navigation tracker based on the first information about the first pose of the optical tracking target and the predetermined configuration.

[0016] In any of the aspects herein, the first type of navigation element includes a plurality of fluoroscopic markers detectable by a fluoroscopic imaging device, and wherein the first registration includes causing the processor to: receive a plurality of images depicting the plurality of fluoroscopic markers and the anatomical element from the fluoroscopic imaging device; and determine, based on the plurality of images, a position of each of the plurality of fluoroscopic markers relative to the anatomical element.

[0017] In any of the aspects of the present invention, wherein the second type of navigation element includes a plurality of navigation markers that can be detected by a second imaging device, and wherein the first alignment further includes the processor: receiving information about the position of each of the plurality of navigation markers from the second imaging device; and determining the position of each navigation marker relative to the anatomical element based on the information about the position of each navigation marker and the predetermined configuration.

[0018] In any of the aspects herein, the third imaging device is disposed proximate to the end effector of the robotic arm.

[0019] In any of the aspects herein, the robotic arm is disposed on a second workbench that is movable relative to the patient bed.

[0020] In any of the aspects herein, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a laser tracker.

[0021] A method according to at least one embodiment of the present disclosure includes: determining a first alignment between a plurality of fluoroscopic markers disposed on a navigation tracker and an anatomical element based on a first plurality of images captured by a first imaging device; determining a second alignment between a plurality of navigation markers disposed on the navigation tracker and the anatomical element based on a second plurality of images captured by a second imaging device; determining a third alignment between the navigation tracker and a robotic arm based on an optical sensor tracker target disposed on the navigation tracker; and moving the robotic arm relative to the anatomical element based on the second alignment and the third alignment.

[0022] In any of the aspects herein, wherein the plurality of fluoroscopic markers, the plurality of navigation markers, and the optical sensor tracker target are disposed on the navigation tracker in a fixed orientation.

[0023] In any of the aspects herein, wherein the optical sensor tracker target is tracked by a third imaging device, and wherein determining the third alignment further comprises: receiving information describing a first position of the optical sensor tracker target relative to the robotic arm from the third imaging device; and determining a first pose of the robotic arm relative to the navigation tracker based on the information describing the first position of the optical sensor tracker target.

[0024] Any aspect may be combined with any one or more other aspects.

[0025] Any one or more of the features disclosed herein.

[0026] Any one or more of the features is generally disclosed herein.

[0027] Any one or more of the features generally disclosed herein may be combined with any one or more of the other features generally disclosed herein.

[0028] Any of the aspects / features / embodiments may be combined with any one or more of the other aspects / features / embodiments.

[0029] Use any one or more of the aspects or features disclosed herein.

[0030] It should be understood that any feature described herein may be claimed in combination with any other feature described herein, regardless of whether the features are from the same described embodiment.

[0031] The details of one or more aspects of the present disclosure are set forth in the following drawings and description. Other features, objectives, and advantages of the techniques described in the present disclosure will be apparent from the description and drawings, and from the claims.

[0032] The phrases "at least one", "one or more", and "and / or" are open expressions that have both connective and dissociative properties in operation. For example, the expressions "at least one of A, B, and C", "at least one of A, B, or C", "one or more of A, B, and C", "one or more of A, B, or C", and "A, B, and / or C" each mean only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together. When each of A, B, and C in the above expressions refers to an element such as X, Y, and Z or a class of elements such as X1-Xn, Y1-Ym, and Z1-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., X1 and X2), and a combination of elements selected from two or more classes (e.g., Y1 and Zo).

[0033] The term "an" entity refers to one or more of that entity. Therefore, the terms "a", "one or more", and "at least one" can be used interchangeably herein. It should also be noted that the terms "including", "comprising", and "having" can be used interchangeably.

[0034] The foregoing is a simplified overview of the present disclosure to provide an understanding of some aspects of the present disclosure. The present disclosure is neither a broad overview nor an exhaustive overview of the present disclosure and its various aspects, embodiments, and configurations. It is neither intended to determine the key or important elements of the present disclosure nor to delimit the scope of the present disclosure, but rather to present the selected concepts of the present disclosure in a simplified form as an introduction to the more detailed description presented below. As should be understood, other aspects, embodiments, and configurations of the present disclosure may utilize one or more of the features set forth above or described in detail below, either individually or in combination.

[0035] Numerous additional features and advantages of the present disclosure will become apparent to those skilled in the art after considering the description of the embodiments provided below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are incorporated into and form a part of this specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the present disclosure. The accompanying drawings illustrate only preferred and alternative examples of how to implement and use the present disclosure, and these examples should not be interpreted as limiting the present disclosure to only the illustrated and described examples. Additional features and advantages will become apparent from the following more detailed description of various aspects, embodiments and configurations of the present disclosure, as illustrated by the accompanying drawings referenced below.

[0037] Figure 1 is a block diagram of a system according to at least one embodiment of the present disclosure;

[0038] Figure 2is a block diagram of aspects of a system according to at least one embodiment of the present disclosure;

[0039] Figure 3A depicts a navigation tracker in accordance with at least one embodiment of the present disclosure;

[0040] Figure 3B depicts an optical tracking marker according to at least one embodiment of the present disclosure;

[0041] Figure 4 is a flow chart according to at least one embodiment of the present disclosure; and

[0042] Figure 5 is a flow chart according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] It should be understood that the various aspects disclosed herein may be combined in combinations different from those specifically presented in the specification and drawings. It should also be understood that, depending on the example or implementation, certain actions or events of any process or method described herein may be performed in a different order and / or may be added, combined, or omitted entirely (e.g., not all described actions or events may be required to practice the disclosed technology, according to different implementations of the present disclosure). In addition, although for clarity, certain aspects of the present disclosure are described as being performed by a single module or unit, it should be understood that the technology of the present disclosure may be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0044] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Alternatively or additionally, the functions may be implemented using a machine learning model, a neural network, an artificial neural network, or a combination thereof (alone or in combination with instructions). A computer-readable medium may include a non-transitory computer-readable medium corresponding to a tangible medium, such as a data storage medium (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer).

[0045] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general-purpose microprocessor), graphics processing units (e.g., Nvidia GeForce RTX 2000 series processors, Nvidia GeForce RTX 3000 series processors, AMD Radeon RX 5000 series processors, AMD Radeon RX 6000 series processors, or any other graphics processing unit), application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other physical structure suitable for implementing the described techniques. Additionally, these techniques may be fully implemented in one or more circuits or logic elements.

[0046] Before explaining any embodiment of the present disclosure in detail, it should be understood that the present disclosure is not limited to the construction details and component arrangements set forth in the following description or illustrated in the accompanying drawings in terms of its application. The present disclosure can have other embodiments and can be practiced or implemented in various ways. In addition, it should be understood that the wording and terminology used herein are for the purpose of description and should not be considered as restrictive. The use of "comprises", "including" or "having" and its variations herein is intended to cover the items listed thereafter and their equivalents, as well as additional items. In addition, the present disclosure can use examples to illustrate one or more aspects thereof. Unless otherwise expressly stated, the use or listing of one or more examples (which can be indicated by "for example", "by way of example", "such as" or similar language) is not intended to and does not limit the scope of the present disclosure.

[0047] The terms proximal and distal are used in this disclosure in their conventional medical sense, with proximal being closer to an operator or user of the system and farther away from the area of ​​surgical interest in or on the patient's body, and distal being closer to the area of ​​surgical interest in or on the patient's body and farther away from the operator or user of the system.

[0048] A floor-standing robot may be navigated to a patient's anatomy during a surgical procedure or surgical procedure. However, a floor-standing robot cannot be securely fixed to the patient's anatomy because the robotic arm may generate undesirable forces on the patient's anatomy. Alternatively, a floor-standing robot may be mounted to a patient's bed, thereby improving precision, but mounting the robot to the bed may limit the movement of the robotic arm, and the bed may be subject to a weight limit that may be exceeded by adding the robot to the bed.

[0049] In at least one embodiment of the present disclosure, a navigation tracker is provided that enables a navigation system (e.g., using a network digital interface (NDI) camera) to track navigation markers set on the navigation tracker. The navigation tracker may also include fluoroscopic markers that enable the use of a robot space with a fluorescent registration of anatomy. The navigation tracker may also include a high-precision optical tracker to allow tracking of the robot's position in space relative to the navigation tracker. In other words, the navigation tracker may act as a virtual bone mount, thereby enabling the robot to be virtually mounted to the patient's anatomical structure. The robot may include a tracking system to track the patient's anatomical structure, and the tracking system may be calibrated according to the robot's position. In some embodiments, the end effector of the robot may include tracking markers that can be tracked by the tracking system. In such embodiments, the tracking system may be positioned in another area of ​​the surgical environment and may track both the robot and the navigation tracker. The use of a virtual bone mount may enable the use of a floor-standing robot without compromising the accuracy obtained when the robot is mounted to the patient's anatomical structure.

[0050] A navigation tracker can act as a patient tracker by including navigation markers that enable the navigation system to move the instrument relative to the patient, metal markers that allow computed tomography-fluoroscopy alignment, and an optical sensor base that enables a floor-standing robot to determine its position relative to the patient's anatomy.

[0051] In some embodiments, the optical sensor may include a 3D camera target. The 3D camera target may include multiple faces that can be detected by a three-dimensional (3D) camera. The 3D camera may focus on a target on the robot arm and / or another 3D camera target and track the movement of these targets. Use of the 3D camera target may enable real-time tracking of the navigation tracker relative to the robot and vice versa. Use of the 3D camera target may also enhance fluoroscopic images captured when registering the tracker with the patient's anatomy, as the 3D camera target may not appear in the fluoroscopic image. In some embodiments, an alternative 3D camera (e.g., Navigation Camera).

[0052] In some embodiments, the optical sensor may include a laser target. The laser tracker may generate laser light and direct it to the laser target. The laser target may reflect the laser light back to the laser tracker, enabling the laser tracker to determine the position and / or orientation of the laser target. In some embodiments, the laser tracker may have two, three, four, five, or six or more degrees of freedom.

[0053] In some embodiments, the patient can be placed on a surgical table and additional preparations (e.g., surgical planning) can be performed. The robot can be placed in or near an operating room or other surgical environment. Two or more fluoroscopic images depicting the patient and the navigation tracker can be taken, and one or more machine learning or artificial intelligence models can determine the alignment between the navigation tracker and the patient's anatomy based on these fluoroscopic images. Additionally, the navigation system can detect the navigation tracker based on navigation markers set on the navigation tracker, and can use the recognition of these navigation markers and the alignment to navigate, for example, the robot. The robot can then be moved near the patient's examination table, and a measurement or optical sensor can be aimed at a measurement or optical tracker set on the navigation tracker. Alignment of the sensor and the tracker can enable the navigation system to determine the position of the robot relative to the patient's anatomy.

[0054] In some embodiments, an O-shaped arm can be used to register the patient with the navigation tracker and the navigation camera. The navigation system can similarly use the navigation markers on the navigation tracker and the registration of the O-shaped arm to navigate the robot. The robot can then be aligned near the bed, and the measurement or optical sensor can be aimed at the measurement or optical tracker so that the navigation system can know the position of the navigation tracker relative to the robot (based on the measurement or optical sensor) and the position of the navigation tracker relative to the anatomy (based on the navigation markers and registration). This can enable the navigation system to control the robot relative to the patient's anatomy.

[0055] Embodiments of the present disclosure provide technical solutions to one or more of the following problems: (1) inaccurate tracking of the robotic arm relative to the patient's anatomy when the robotic arm is not mounted to the bed, and (2) limitations / constraints on the movement of the robotic arm when the robotic arm is mounted to the bed.

[0056] First turn Figure 1, a block diagram of a system 100 according to at least one embodiment of the present disclosure is shown. The system 100 can be used to navigate a robotic arm relative to a patient anatomy, thereby determining one or more alignments of the patient anatomy with a navigation tracker and / or a robotic arm, and / or perform one or more other aspects of one or more methods discussed herein. The system 100 includes a computing device 102, one or more imaging devices 112, a robot 114, a navigation system 118, a database 130, and / or a cloud or other network 134. Systems according to other embodiments of the present disclosure may include more or fewer components than the system 100. For example, the system 100 may not include the imaging device 112, the robot 114, the navigation system 118, one or more components of the computing device 102, the database 130, and / or the cloud 134.

[0057] Computing device 102 includes processor 104, memory 106, communication interface 108, and user interface 110. Computing devices according to other embodiments of the present disclosure may include more or fewer components than computing device 102.

[0058] The processor 104 of the computing device 102 may be any processor described herein or any similar processor. The processor 104 may be configured to execute instructions stored in the memory 106, which may cause the processor 104 to perform one or more computing steps using or based on data received from the imaging device 112, the robot 114, the navigation system 118, the database 130, and / or the cloud 134.

[0059] The memory 106 may be or include RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible non-transitory memory for storing computer-readable data and / or instructions. The memory 106 may store information or data that can be used to complete any step of the method 400 and / or 500 or any other method described herein, for example. The memory 106 may store instructions and / or machine learning models that support one or more functions of the robot 114, for example. For example, the memory 106 may store content (e.g., instructions and / or machine learning models) that, when executed by the processor 104, enable image processing 120, segmentation 122, transformation 124, and / or registration 128. In some embodiments, if provided as instructions, such content may be organized into one or more applications, modules, packages, layers, or engines. Alternatively or additionally, the memory 106 may store other types of content or data (e.g., machine learning models, artificial neural networks, deep neural networks, etc.) that can be processed by the processor 104 to implement the various methods and features described herein. Thus, while the various contents of the memory 106 may be described as instructions, it should be understood that the functionality described herein may be implemented using instructions, algorithms, and / or machine learning models. The data, algorithms, and / or instructions may cause the processor 104 to manipulate data stored in the memory 106 and / or data received from or via the imaging device 112, the robot 114, the database 130, and / or the cloud 134.

[0060] The computing device 102 may also include a communication interface 108. The communication interface 108 may be used to receive image data or other information from an external source (such as an imaging device 112, a robot 114, a navigation system 118, a database 130, a cloud 134, and / or any other system or component that is not part of the system 100) and / or to transmit instructions, images, or other information to an external system or device (e.g., another computing device 102, an imaging device 112, a robot 114, a navigation system 118, a database 130, a cloud 134, and / or any other system or component that is not part of the system 100). The communication interface 108 may include one or more wired interfaces (e.g., a USB port, an Ethernet port, a FireWire port) and / or one or more wireless transceivers or interfaces (configured to transmit and / or receive information via one or more wireless communication protocols such as, for example, 802.11a / b / g / n, Bluetooth, NFC, ZigBee, etc.). In some embodiments, the communication interface 108 may be used to enable the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time required to complete computationally intensive tasks or for any other reason.

[0061] The computing device 102 may also include one or more user interfaces 110. The user interface 110 may be or include a keyboard, a mouse, a trackball, a monitor, a television, a screen, a touch screen and / or any other device for receiving information from a user and / or for providing information to a user. The user interface 110 may be used, for example, to receive a user selection or other user input for any step of any method described herein. Nevertheless, any required input for any step of any method described herein may be automatically generated by the system 100 (e.g., by another component of the processor 104 or the system 100) or received by the system 100 from a source external to the system 100. In some embodiments, the user interface 110 may be used to allow a surgeon or other user to modify the instructions to be executed by the processor 104 according to one or more embodiments of the present disclosure, and / or modify or adjust the settings of other information displayed on or corresponding to the user interface 110.

[0062] Although the user interface 110 is shown as part of the computing device 102, in some embodiments, the computing device 102 may utilize a user interface 110 that is housed separately from one or more remaining components of the computing device 102. In some embodiments, the user interface 110 may be located proximate to one or more other components of the computing device 102, while in other embodiments, the user interface 110 may be located remote from one or more other components of the computing device 102.

[0063] The imaging device 112 can be used to image anatomical features (e.g., bones, veins, tissues, etc.) and / or other aspects of the patient's anatomical structure to generate image data (e.g., image data depicting or corresponding to bones, veins, tissues, etc.). As used herein, "image data" refers to data generated or captured by the imaging device 112, including data in machine-readable form, graphical / visual form, and in any other form. In different examples, the image data may include data corresponding to the anatomical features of the patient or a portion thereof. The image data may be or include preoperative images, intraoperative images, postoperative images, or images taken independently of any surgical procedure. In some embodiments, the first imaging device 112 can be used to obtain first image data (e.g., a first image) at a first time, and the second imaging device 112 can be used to obtain second image data (e.g., a second image) at a second time after the first time. The imaging device 112 can be used to capture 2D images or 3D images to generate image data. The imaging device 112 may be or include, for example, an ultrasound scanner (which may include, for example, physically separate transducers and receivers, or a single ultrasound transceiver), an O-arm, C-arm, G-arm, or any other device utilizing X-ray based imaging (e.g., a fluoroscope, a CT scanner or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, a microscope, an optical camera, a thermal imaging camera (e.g., an infrared camera), a radar system (which may include, for example, a transmitter, a receiver, a processor, and one or more antennas), or any other imaging device 112 suitable for obtaining images of anatomical features of a patient. The imaging device 112 may be entirely contained within a single housing, or may include a transmitter / transmitter and a receiver / detector that are physically separated in a single housing or otherwise.

[0064] In some embodiments, the imaging device 112 may include more than one imaging device 112. For example, a first imaging device may provide first image data and / or a first image, and a second imaging device may provide second image data and / or a second image. In yet other embodiments, the same imaging device may be used to provide both the first image data and the second image data and / or any other image data described herein. The imaging device 112 may be used to generate an image data stream. For example, the imaging device 112 may be configured to operate with a shutter open, or to operate with a shutter continuously alternating between open and closed, so as to capture continuous images. For purposes of this disclosure, unless otherwise specified, if the image data represents two or more frames per second, the image data may be considered continuous and / or provided as an image data stream.

[0065] Robot 114 may be any surgical robot or surgical robotic system. Robot 114 may be or include, for example, a Mazor XTM Stealth Edition robotic guidance system. The robot 114 may be configured to position the imaging device 112 at one or more precise positions and orientations, and / or return the imaging device 112 to the same position and orientation at a later point in time. The robot 114 may additionally or alternatively be configured to manipulate surgical tools (whether or not based on guidance from the navigation system 118) to complete or assist in surgical tasks. In some embodiments, the robot 114 may be configured to hold and / or manipulate anatomical elements during or in conjunction with a surgical procedure. The robot 114 may include one or more robotic arms 116. In some embodiments, the robotic arm 116 may include a first robotic arm and a second robotic arm, but the robot 114 may include more than two robotic arms. In some embodiments, one or more of the robotic arms 116 may be used to hold and / or manipulate the imaging device 112. In embodiments where the imaging device 112 includes two or more physically separate components (e.g., a transmitter and a receiver), one robotic arm 116 may hold one such component, and another robotic arm 116 may hold another such component. Each robotic arm 116 is positionable independently of the other robotic arms.The robotic arms 116 may be controlled in a single shared coordinate space or in separate coordinate spaces.

[0066] The robot 114, together with the robotic arm 116, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Furthermore, the robotic arm 116 may be positioned or capable of being positioned in any pose, plane, and / or focus. The pose includes a position and an orientation. Thus, an imaging device 112, a surgical tool, or other object held by the robot 114 (or more specifically, by the robotic arm 116) may be capable of being precisely positioned in one or more desired and specific positions and orientations.

[0067] The robotic arm 116 may include one or more sensors that enable the processor 104 (or a processor of the robot 114) to determine the precise pose of the robotic arm (and any object or element held by or secured to the robotic arm) in space.

[0068] In some embodiments, reference markers (i.e., navigation markers) may be placed on the robot 114 (including, for example, on the robotic arm 116), the imaging device 112, or any other object in the surgical space. The reference markers may be tracked by the navigation system 118, and the results of the tracking may be used by the robot 114 and / or by an operator of the system 100 or any component thereof. In some embodiments, the navigation system 118 may be used to track other components of the system (e.g., the imaging device 112), and the system may be operated without the use of the robot 114 (e.g., the surgeon manually manipulates the imaging device 112 and / or one or more surgical tools, e.g., based on information and / or instructions generated by the navigation system 118).

[0069] During the operation, the navigation system 118 can provide navigation for the surgeon and / or the surgical robot. The navigation system 118 can be any navigation system now known or developed in the future, including, for example, the Medtronic Stealth Station TM S8 surgical navigation system or any successor thereof. The navigation system 118 may include one or more cameras or other sensors for tracking one or more reference markers, navigation trackers, or other objects in the operating room or other rooms where part or all of the system 100 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 118 may include one or more electromagnetic sensors. In various embodiments, the navigation system 118 may be used to track the position and orientation (e.g., pose) of the imaging device 112, the robot 114, and / or the robot arm 116, and / or one or more surgical tools (or more specifically, for tracking the pose of a navigation tracker attached directly or indirectly to one or more of the foregoing in a fixed relationship). The navigation system 118 may include a display for displaying one or more images from an external source (e.g., a computing device 102, an imaging device 112, or other source) or for displaying images and / or video streams from one or more cameras or other sensors of the navigation system 118. In some embodiments, the system 100 may operate without the use of the navigation system 118. The navigation system 118 can be configured to provide guidance to a surgeon or other user of the system 100 or its components, to the robot 114, or to any other element of the system 100 regarding, for example, the position of one or more anatomical elements, whether a tool is in an appropriate trajectory, and / or how to move a tool into an appropriate trajectory to perform a surgical task based on a preoperative or other surgical plan.

[0070] The database 130 may store information relating one coordinate system to another coordinate system (e.g., relating one or more robot coordinate systems to a patient coordinate system and / or a navigation coordinate system). The database 130 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about an anatomical structure of a patient at and / or near a surgical site for use by the robot 114, the navigation system 118, and / or a user of the computing device 102 or system 100); one or more images that may be used in conjunction with a surgical procedure performed by or with the assistance of one or more other components of the system 100; and / or any other useful information. The database 130 may be configured to provide any such information to the computing device 102 or any other device of the system 100 or any other device external to the system 100, whether directly or via the cloud 134. In some embodiments, database 130 can be or include a part of a hospital image storage system, such as a picture archiving and communication system (PACS), a health information system (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0071] Cloud 134 can be or represent the Internet or any other wide area network. Computing device 102 can connect to cloud 134 via communication interface 108 using a wired connection, a wireless connection, or both. In some embodiments, computing device 102 can communicate with database 130 and / or external devices (e.g., computing devices) via cloud 134.

[0072] System 100 or a similar system may be used, for example, to implement one or more aspects of any of methods 400 and / or 500 described herein. System 100 or a similar system may also be used for other purposes.

[0073] Figure 2 and FIG. 3A to FIG. 3B Aspects of a system 100 are depicted in accordance with at least one embodiment of the present disclosure. Figure 2A surgical environment, such as an operating room, is shown including a patient table 204 and a robotic table 208. The patient table 204 and the robotic table 208 may be positioned on a floor 212 of the surgical environment. In some embodiments, the patient table 204 and / or the robotic table 208 may be mobile and capable of moving around the surgical environment. For example, the robotic table 208 may be or include a cart that moves relative to the patient table 204, thereby allowing the robotic table 208 (and the robotic arm 116) to be brought into or otherwise introduced into the surgical environment after surgical preparation or surgical tasks have been performed. For example, the robotic table 208 may be brought into the surgical environment after preoperative imaging has been performed to help relieve congestion in the surgical environment.

[0074] Patient 216 may be positioned on patient table 204. Patient 216 may have anatomical elements 220A-220C that may be the subject of a surgical procedure or surgical procedure. For example, the surgical procedure may be spinal fusion, and anatomical elements 220A-220C may be vertebrae of the spine. In some embodiments, patient 216 may be securely positioned on patient table 204 so that patient 216 and / or anatomical elements 220A-220C cannot move relative to patient table 204. Although the discussion herein includes discussion of anatomical elements, it should be understood that more or fewer anatomical elements may exist and may be identified and registered using the methods discussed herein. In addition, it should be understood that the methods and embodiments discussed herein may alternatively be applied to a portion of an anatomical element (e.g., a spinous process of a vertebra).

[0075] The robotic table 208 includes a robotic arm 116 and an optical sensor 228. In some embodiments, the robotic table 208 may include additional or alternative components. For example, the optical sensor 228 may not be located on the robotic table 208, but may be set in another location in the surgical environment (e.g., on the floor 212, mounted on a wall, located on another surgical table, etc.). The robotic table 208 may include additional surgical components, such as surgical tools, and may include one or more cabinets, drawers, trays, etc. to accommodate surgical components. In some embodiments, the robotic table 208 may be mechanically disengaged or otherwise separated from the patient table 204, so that the robotic table 208 and / or surgical components thereon can be freely moved relative to the patient table 204, the patient 216, and / or the anatomical elements 220A-220C. In one embodiment, the robotic table 208 may be set to be a first distance (e.g., 0.5 meters (m), 1m, 1.5m, 2m, etc.) from the patient table 204.

[0076] The optical sensor 228 may be or include a sensor capable of detecting and / or tracking an optical device-based target (e.g., an illuminated object, a visual target, etc.). In some embodiments, the optical sensor 228 may be or include a laser tracker. The laser tracker may project or emit a laser that may reflect from one or more targets and return to the laser tracker. The reflected light may be received and processed by the computing device 102 and / or the navigation system 118, and may enable the computing device 102 and / or the navigation system 118 to determine the relative distance and / or pose of the target relative to the laser tracker based on, for example, the angle, intensity, frequency, etc. of the returning laser light. In one embodiment, the laser tracker may include a tracking system that tracks the target as the target moves, so that the laser tracker may continuously aim the laser at the target and receive the reflected laser light. The information of the reflected laser light may be processed (e.g., by the computing device 102, by the navigation system 118, etc.) to identify and determine the change in the pose of the target as the target moves relative to the optical sensor 228. Alternatively, the optical sensor 228 may be or include a 3D camera capable of identifying one or more 3D optical tracking targets. The 3D camera may be capable of identifying the 3D optical tracking target based on the number of faces, designs, or patterns displayed by the 3D optical tracking target. For example, the 3D camera may identify the optical tracking target based on a different QR code displayed on each surface of the 3D optical tracking target. The processor 104 may receive the identified faces and may determine (e.g., using the transformation 124) the pose of the optical tracking target within the surgical environment. For example, the identified faces may be compared to a predetermined (e.g., preoperative) pose of the surface, where the pose change of each face is used to determine the pose of the optical tracking target.

[0077] In some embodiments, the optical sensor 228 may be disposed proximate to the robotic arm 116 (e.g., disposed 0.1 m, 0.2 m, 0.5 m, 1 m, 1.5 m, 2 m, etc. away from the robotic arm 116) such that the optical sensor 228 is able to see and track the robotic arm 116 in addition to any optically based targets in the environment. Alternatively, the optical sensor 228 may be disposed within a portion of the robotic arm 116 (e.g., within the end effector 224). In such embodiments, the optical sensor 228 may be disposed in a predetermined configuration relative to the robotic arm 116 such that the pose of the robotic arm 116 may be determined based on internal readings generated by sensors (e.g., using gyroscopes, accelerometers, etc.).

[0078] The robotic arm 116 may include an end effector 224. The end effector 224 may be or include a socket, a mount, a clamp, or other mechanical interface for interacting with a surgical tool or instrument. For example, the end effector 224 may interface with a surgical tool to manipulate the surgical tool (whether or not based on guidance from the navigation system 118) to complete or assist in a surgical operation or surgical procedure or task. In some embodiments, the end effector 224 may have an interchangeable interface so that different surgical tools with different connection capabilities can be used with the end effector 224. For example, the end effector 224 may have a first interface and a second interface, the first interface allowing a surgical drill to be attached to the end effector, and the second interface retaining a surgical screw. In some embodiments, the end effector 224 may include a tracking mark 232 disposed thereon. The positioning of the tracking mark 232 on the end effector 224 may enable the imaging device 112 and / or the optical sensor 228 to track the pose of the end effector 224. For example, the tracking marker 232 may be a cube with a different QR code on each face, so that the optical sensor 228 tracks the tracking marker 232 and is able to recognize each face of the tracking marker 232, and the computing device 102 and / or the navigation system 118 may use the recognized face to determine the pose of the tracking marker 232. Based on the pose of the tracking marker 232 and the predetermined position of the tracking marker 232 on the end effector 224, the computing device 102 and / or the navigation system 118 may use the pose of the tracking marker 232 to determine the pose of the end effector 224, or more generally, the pose of the robotic arm 116.

[0079] The navigation tracker 236 can be disposed on or near the patient 216. In some embodiments, the navigation tracker 236 can be attached to the patient 216 using an attachment mechanism 240. The attachment mechanism 240 can be or include, for example, a clamp attached to the spinous process of a vertebra, a threaded rod that can be screwed into the patient table 204, etc. In some embodiments, the attachment mechanism 240 can be connected to the patient 216 so that any movement of the patient 216 and / or the anatomical elements 220A-220C can result in a corresponding movement of the navigation tracker 236. Movement of the first anatomical element 220A in a first direction by a first distance can, for example, cause the navigation tracker 236 attached to the first anatomical element 220A to also move in the first direction by a first distance.

[0080] Reference Figure 3A, the navigation tracker 236 may include different types of navigation elements for use with different navigation and registration processes. In one embodiment, the navigation tracker 236 may include a first type of navigation element, a second type of navigation element, and a third type of navigation element. Each of the different types of navigation elements is capable of being tracked, captured, or otherwise identified by a corresponding imaging device of the same type. Additionally or alternatively, the navigation element may not be presented or identified by a different type of imaging device. For example, a first type of navigation element may be detectable in fluoroscopic imaging, but may not appear in imaging associated with an optical sensor, or may not be identified by the optical sensor.

[0081] In some embodiments, the first type of navigation element can be or include one or more fluoroscopic markers 304A-304J. In one embodiment, the fluoroscopic markers 304A-304J can be or include metal spheres disposed on the navigation tracker 236. In other embodiments, the shape and size of the fluoroscopic markers 304A-304J can vary depending on, for example, the surgical procedure or surgical procedure, the accuracy requirements of the system 100, the type of navigation tracker 236, etc. The fluoroscopic markers 304A-304J can be or include radiopaque elements (e.g., elements that appear opaque during fluoroscopic imaging or other radiation-based imaging) such that the fluoroscopic markers 304A-304J can be identified in the fluoroscopic image. In other words, the fluoroscopic markers 304A-304J can appear in the fluoroscopic image along with the depicted patient anatomy such that the patient anatomy can be registered with the fluoroscopic markers 304A-304J (e.g., using registration 128), and / or vice versa. As discussed in more detail below, to accomplish registration of the fluoroscopic markers 304A-304J with the anatomical elements, the computing device 102 may receive image data associated with one or more fluoroscopic images and may use image processing 120 to identify the patient anatomy and each of the fluoroscopic markers 304A-304J. The computing device 102 may then use registration 128 to determine the position of each of the fluoroscopic markers 304A-304J relative to the patient anatomy. In some embodiments, the fluoroscopic markers 304A-304J may be arranged in a predetermined configuration or pattern. For example, Figure 3AAs depicted, the fluoroscopic markers 304A-304E are disposed on an upper portion of the navigation tracker 236, while the fluoroscopic markers 304F-304J are disposed on a lower portion of the navigation tracker 236. In some embodiments, the fluoroscopic markers may be evenly or uniformly spaced. In some embodiments, the pose of each of the fluoroscopic markers 304A-304J relative to one or more locations of the navigation tracker 236 (such as relative to other fluoroscopic markers, fluoroscopic markers 304A-304J, or relative to other navigation elements disposed on the navigation tracker 236) may be saved or stored in the database 130.

[0082] The navigation tracker 236 may also include a second type of navigation element, such as navigation markers 308A-308D. The navigation markers 308A-308D may be or include fiducial markers that can be recognized by the imaging device 112 of the navigation system 118. For example, the navigation markers 308A-308D may be or include spheres disposed at the corners or perimeter of the navigation tracker 236. In some embodiments, the navigation markers 308A-308D may include a reference marker that is recognizable by the imaging device 112 of the navigation system 118. Figure 3A 308A-308D may include more or fewer markers than those shown in the figure, and may be additionally or alternatively arranged in different orientations. For example, in some embodiments, the navigation markers 308A-308D may include a plurality of navigation markers arranged in a cross pattern. In embodiments where both the navigation markers 308A-308D and the fluoroscopic markers 304A-304J are similarly shaped (e.g., both sets of markers include spheres), the navigation markers 308A-308D may be distinguished from the fluoroscopic markers 304A-304J by the navigation system 118 based on the size of the spheres, the reflectivity of the spheres when illuminated with light, the position of the spheres on the navigation tracker 236, combinations thereof, and the like. In some embodiments, the navigation markers 308A-308D may be or include active markers (e.g., LEDs) that are illuminated when viewed by the imaging device 112 of the navigation system 118.

[0083] The navigation tracker 236 may also include a third type of navigation element, such as an optical sensor tracker target 312. The optical sensor tracker target 312 may be a target that can be tracked by the optical sensor 228. By tracking the position of the optical sensor tracker target 312, the navigation system 118 can determine the pose of the navigation tracker 236 relative to the optical sensor 228 and / or the robotic arm 116. In some embodiments, the optical sensor tracker target 312 may be disposed in a predetermined position on the navigation tracker 236. In one embodiment, the optical sensor tracker target 312 may be centered in the middle of the navigation tracker 236.

[0084] In some embodiments, the optical sensor tracker target 312 may be used to monitor the position of the patient's anatomy relative to the robotic arm 116. For example, a physician or other member of the surgical staff may accidentally bump into the patient table 204, or the anatomical elements 220A-220C may otherwise move relative to the robotic arm 116. The movement of the anatomical elements 220A-220C may cause the navigation tracker 236 and the optical sensor tracker target 312 to also move from a first pose to a second pose relative to the robotic arm 116. To measure the movement, the change in pose of the optical sensor tracker target 312 may be identified by the optical sensor 228, and a new pose of the optical sensor tracker target 312 may be determined based on the information generated by the optical sensor 228.

[0085] Steering Figure 3B , various optical sensor trackers according to embodiments of the present disclosure are shown. In some embodiments, the optical sensor tracker target 312 can be or include a laser target 316. The laser target 316 can be an optical target including a base 320 and a reflector 324. The base 320 can enable the laser target 316 to be attached or fixed to the navigation tracker 236. The reflector 324 can bend the laser or other light generated by the optical sensor 228 and reflect the light back to the optical sensor 228, thereby enabling the optical sensor 228 to determine the pose of the laser target 316. In some embodiments, when the light emitted by the optical sensor 228 hits the reflector 324, the laser target 316 can generate and emit one or more signals (e.g., electronic signals, RF signals, etc.). The one or more signals can be received by the computing device 102, the navigation system 118, and / or the optical sensor 228, and can be further processed to determine the pose of the laser target 316. For example, laser target 316 may emit a signal containing information related to the frequency, intensity, and / or direction of incidence of light impinging on laser target 316. This information may be used along with information about the initial light generated by optical sensor 228 to determine a pose of laser target 316 relative to optical sensor 228. In some embodiments, laser target 316 may be or include one or more metals or metal alloys such that laser target 316 may be radio-opaque.

[0086] In some embodiments, the optical sensor tracker target 312 may be or include a 3D target 328. The 3D target 328 may include a base 336 that can be used to attach the 3D target 328 to one or more surfaces, instruments, or trackers (e.g., to the navigation tracker 236, to the end effector 224 of the robotic arm 116, etc.). The 3D target 328 may also include faces 332A-332H that enable a 3D camera or other imaging device to recognize the 3D target 328. In some embodiments, each of the faces 332A-332H may include a unique pattern displayed so that each face can be recognized by the 3D camera. For example, each face may display a separate QR code, label, or other marking. When the 3D camera views or captures an image of the 3D target 328, each face seen by the camera or depicted in the image can be recognized. In some embodiments, the 3D target 328 may have a predetermined configuration so that any movement of the 3D target 328 can be recognized and captured by the 3D camera. In such embodiments, an additional 3D target 328 may be provided on the robotic arm 116 (eg, on the end effector 224 of the robotic arm 116) such that movement of the robotic arm 116 may be captured by the 3D camera.

[0087] In some embodiments, the fluoroscopic markers 304A-304J, the navigation markers 308A-308D, and the optical sensor tracker target 312 may be arranged in a predetermined configuration on the navigation tracker 236. The predetermined configuration may ensure that the navigation elements are arranged in a fixed orientation on the navigation tracker, which may allow the computing device 102 and / or the processor 104 to know or easily determine the pose of any of the fluoroscopic markers 304A-304J, any of the navigation markers 308A-308D, and / or the optical sensor tracker target 312 relative to any other navigation element on the navigation tracker 236 and / or relative to a fixed point (e.g., a point on the periphery, a center point, etc.) on the navigation tracker 236. In some embodiments, the predetermined configuration may be associated with a component or part number of the navigation tracker 236 (e.g., a first navigation tracker may have a different configuration of navigation elements than a second navigation tracker). In some embodiments, predetermined configuration information (eg, relative distances and orientations of any one navigation element to some or all of the other navigation elements) may be stored in database 130 and may be used to perform one or more registrations.

[0088] Figure 4 A method 400 is depicted that can be used, for example, to create a virtual bone mount and use the virtual bone mount to navigate a robot relative to a patient's anatomy.

[0089] Method 400 (and / or one or more steps thereof) may be performed or otherwise executed, for example, by at least one processor. The at least one processor may be the same or similar to processor 104 of computing device 102 described above. The at least one processor may be part of a robot (such as robot 114) or part of a navigation system (such as navigation system 118). Processors other than any processor described herein may also be used to execute method 400. The at least one processor may execute method 400 by executing elements stored in a memory (such as memory 106). Elements stored in the memory and executed by the processor may cause the processor to perform one or more steps of the functions shown in method 400. One or more portions of method 400 may be executed by a processor that executes any of the contents of the memory (such as image processing 120, segmentation 122, transformation 124, and / or registration 128).

[0090] Method 400 includes capturing a first plurality of images depicting an anatomical element and a plurality of fluoroscopic markers disposed on a navigation tracker (step 404). The anatomical element may be similar or identical to the first anatomical element 220A, the second anatomical element 220B, or the third anatomical element 220C. The navigation tracker may be similar or identical to the navigation tracker 236, and the plurality of fluoroscopic markers may be similar or identical to the fluoroscopic markers 304A-304J. In some embodiments, the fluoroscopic markers 304A-304J may be disposed on the navigation tracker 236 in a predetermined configuration. The first plurality of images may be or include a plurality of fluoroscopic images or fluoroscopic image data captured by a fluoroscopic imaging device (e.g., an X-ray source and an X-ray detector). In some embodiments, an O-arm or other imaging device 112 may be used to capture the images. When capturing each image, the fluoroscopic imaging device may be positioned at a predetermined position, and each captured image in the first plurality of images may depict the fluoroscopic markers 304A-304J and the navigation tracker 236 in a different pose (e.g., a different position and / or orientation). In some implementations, the first plurality of images may be captured preoperatively (eg, before a surgical operation or surgical procedure begins).

[0091] The method 400 also includes determining a first registration of the plurality of fluoroscopic markers with the anatomical element based on the first plurality of images (step 408). The first registration can be or include a mapping of coordinates associated with one or more of the fluoroscopic markers 304A-304J in the first coordinate system to a second coordinate system containing coordinates of the anatomical element, or vice versa. In some embodiments, the registration can transform both sets of coordinates to a third coordinate system, such as a coordinate system used by a robotic arm. In some embodiments, the plurality of images can depict additional anatomical elements (e.g., a plurality of vertebrae of a spine), and the first registration can include mapping coordinates associated with each of the additional anatomical elements into a common coordinate system.

[0092] In some embodiments, the first registration may be determined using image processing 120 and one or more registrations 128. Image processing 120 may be used to identify multiple fluoroscopic markers and anatomical elements in each image of the first plurality of images. In some embodiments, image processing 120 may be or include one or more machine learning and / or artificial intelligence models that receive each image as input coordinates and output coordinates associated with each identified fluoroscopic marker and anatomical element. Registration 128 may use the determined coordinates associated with each identified fluoroscopic marker and anatomical element to determine the pose of each fluoroscopic marker relative to the anatomical element, and may transform the coordinates associated with each fluoroscopic marker from a first coordinate system to a second coordinate system. For example, registration 128 may obtain the coordinates associated with each fluoroscopic marker in the identified fluoroscopic markers and map these coordinates to a coordinate system associated with the anatomical element (or vice versa). Additionally or alternatively, registration 128 may map the fluoroscopic marker coordinates and the anatomical element coordinates to a third coordinate system (e.g., a robot space coordinate system) shared by other surgical tools or components in the surgical environment.

[0093] Method 400 also includes capturing a second plurality of images depicting a plurality of navigation markers set on the navigation tracker (step 412). The plurality of navigation markers may be the same or similar to navigation markers 308A-308D. In some embodiments, navigation markers 308A-308D may be set on navigation tracker 236 in a predetermined configuration (e.g., along the periphery of navigation tracker 236, at the corners of navigation tracker 236, etc.). The second plurality of images may be images of a different type than those captured by the first plurality of images, so that the appearance of navigation markers and / or fluoroscopic markers is different in the second plurality of images. For example, while a plurality of navigation markers may not appear in the first plurality of images, a plurality of navigation markers may appear in the second plurality of images. In one embodiment, the second plurality of images may be images or image data captured by a navigation camera (e.g., a 3D camera, an ultrasound camera, or other camera). Each captured image in the second plurality of images may depict a navigation marker in a different pose, and the pose of the navigation camera of the captured image may be known (e.g., based on a predetermined image capture navigation path).

[0094] The method 400 also includes determining a second registration of the plurality of navigation markers with the anatomical element based on the second plurality of images and the first registration (step 416). The second registration can be or include a mapping of coordinates associated with each of the navigation markers 308A-308D to a coordinate system associated with the anatomical element, or vice versa. In some embodiments, the registration 128 can transform the coordinates associated with each of the navigation markers 308A-308D into a different coordinate system (e.g., a robot space coordinate system) shared by the fluoroscopic markers 304A-304J and the anatomical element.

[0095] In some embodiments, the image processing 120 and the one or more registrations 128 may be used to identify the navigation markers 308A-308D in the second plurality of images, thereby determining the coordinates associated with each of the navigation markers 308A-308D, and mapping these coordinates into a coordinate system shared by the fluoroscopic markers 304A-304J, the anatomical elements, and the robotic arm 116. In some embodiments, the image processing 120 may be or include one or more machine learning and / or artificial intelligence models that receive each image as input coordinates and output coordinates associated with each identified navigation fluoroscopic marker. A second registration may then be determined based on the coordinates of each of the navigation markers relative to the fluoroscopic markers 304A-304J and how the navigation markers 308A-308D and the fluoroscopic markers 304A-304J are arranged in a predetermined configuration on the navigation tracker 236. For example, the registration 128 may use information about the predetermined configuration to map the identified coordinates of the navigation markers 308A-308D into a coordinate system shared by the fluoroscopic markers 304A-304J and the anatomical elements.

[0096] Additionally or alternatively, the registration 128 may use a predetermined configuration of fluoroscopic markers 304A-304J disposed on the navigation tracker 236 to map coordinates associated with the navigation tracker 236 into a shared coordinate system. In other words, the registration 128 may be used to determine a pose of the navigation tracker 236 relative to the anatomical element. The registration 128 may, for example, receive coordinates of each of the fluoroscopic markers 304A-304J and one or more coordinates of the navigation tracker 236 (e.g., coordinates associated with the periphery of the navigation tracker 236, coordinates associated with a center point of the navigation tracker 236, etc.), and may map the coordinates of the navigation tracker 236 into a coordinate system that includes the fluoroscopic markers 304A-304J and / or into a shared coordinate system. The coordinates of the navigation tracker 236 in the shared coordinate system may be determined based on the determined poses of the navigation markers 308A- 308D and the predetermined configuration of the navigation markers 308A- 308D disposed on the navigation tracker 236 .

[0097] The method 400 also includes aligning an optical sensor disposed proximate to a robotic arm to track an optical sensor tracker target disposed on a navigation tracker (step 420). The robotic arm may be similar or identical to the robotic arm 116. The optical sensor may be similar or identical to the optical sensor 228, and the optical sensor tracker target may be similar or identical to the optical sensor tracker target 312. The optical sensor 228 may be pointed at the optical sensor tracker target 312 so that the optical sensor tracker target 312 can be identified and a pose of the optical sensor tracker target 312 can be determined. For example, the optical sensor 228 may be or include a laser tracker that emits a laser that is captured by the optical sensor tracker target 312 (e.g., the optical sensor tracker target 312 may be or include the laser target 316). Alternatively, the optical sensor 228 may be or include a 3D camera, and the optical sensor tracker target 312 may be or include the 3D target 328. In some embodiments, the optical sensor 228 may automatically identify the optical sensor tracker target 312 (e.g., the processor 104 may cause the optical sensor 228 to search the surrounding area until the optical sensor tracker target 312 is identified), or the optical sensor 228 may alternatively be manually aligned (e.g., by a physician, by a member of the surgical staff, etc.). In some embodiments, alignment of the optical sensor 228 with the optical sensor tracker target 312 may occur after capturing pre-operative images (e.g., the first plurality of images and / or the second plurality of images).

[0098] The method 400 also includes determining a third registration between the robot arm and the navigation tracker based on the information from the optical sensor (step 424). Step 424 may refer to Figure 5 The method 500 of FIG. 1 is described in more detail. The optical sensor 228 may capture information related to the pose of the optical sensor tracker target 312, and this information may be used to determine a third registration between the robotic arm 116 and the navigation tracker 236. In some embodiments, this information may be based on the pose, distance, position, etc. of the optical sensor tracker target 312 (and by extension the navigation tracker 236) relative to the optical sensor 228. In some embodiments, the optical sensor 228 may be disposed at a known distance from the robotic arm 116, so that information about the optical sensor tracker target 312 may be used by the registration 128 to generate the third registration.

[0099] The method 400 also includes navigating the robotic arm relative to the anatomical element based on the second registration and the third registration (step 428). The second registration and the third registration can be used to move the robotic arm 116 to perform one or more surgical tasks (e.g., drilling procedures, implantation procedures, screwing procedures, incision procedures, etc.) on the anatomical element, for example. The navigation system 118 can know the pose of the navigation tracker 236 relative to the anatomical element based on the second registration, and can also know the pose of the robotic arm relative to the navigation tracker 236 based on the third registration. The navigation system 118 can then adjust the pose of the robotic arm 116 relative to the anatomical element.

[0100] As the robotic arm 116 moves, for example, from a first pose to a second pose, the optical sensor 228 may continuously track the optical sensor tracker target 312 and generate data related thereto. The computing device 102 and / or the navigation system 118 may process the data regarding the change in pose of the optical sensor tracker target 312 as seen by the optical sensor 228. The computing device 102 and / or the navigation system 118 may alternatively process the data regarding the change in pose of the robotic arm 116 and / or the optical sensor tracker target 312 based on, for example, the tracking markers 232 disposed on the robotic arm 116 and the 3D target 328 disposed on the navigation tracker 236, respectively.

[0101] In some embodiments, the third registration may be updated based on unexpected or unplanned movement of the navigation tracker 236 (or a component thereof) relative to the robotic arm 116 (or a component thereof), or vice versa. For example, the patient table 204 may be accidentally bumped or moved, and the navigation tracker 236 may be unexpectedly moved relative to the robotic arm 116. The movement may be reflected in the data generated by the optical sensor 228 (e.g., a determination made by the computing device 102 indicates that the navigation tracker 236 is no longer in the planned or expected pose). The computing device 102 and / or the navigation system 118 may generate a warning that the third registration is no longer accurate (e.g., presenting a warning message to the user interface 110), and step 424 may be performed again to generate a new registration between the robotic arm 116 and the navigation tracker 236.

[0102] The present disclosure encompasses embodiments of method 400 that include more or fewer steps than those described above and / or one or more steps that are different than those described above.

[0103] Figure 5 Depicted is a method 500 that can be used to determine registration between a robotic arm and a navigation tracker, for example based on measurements made by sensors about the tracker. In some embodiments, the method 500 can be performed as part of step 424 of determining registration between a robotic arm and a navigation tracker.

[0104] Method 500 (and / or one or more steps thereof) may be performed or otherwise executed, for example, by at least one processor. The at least one processor may be the same or similar to processor 104 of computing device 102 described above. The at least one processor may be part of a robot (such as robot 114) or part of a navigation system (such as navigation system 118). Processors other than any processor described herein may also be used to perform method 500. The at least one processor may perform method 500 by executing elements stored in a memory (such as memory 106). The elements stored in the memory and executed by the processor may cause the processor to perform one or more steps of the functions shown in method 500. One or more portions of method 500 may be performed by a processor that executes any of the contents of the memory (such as image processing 120, segmentation 122, transformation 124, and / or registration 128).

[0105] The method 500 includes receiving information about the position of the optical sensor tracker target and the first pose of the robotic arm from the optical sensor (step 504). In some embodiments, the optical sensor 228 may generate information about the pose of the optical sensor tracker target 312 based on how light emitted from the optical sensor 228 interacts with the optical sensor tracker target 312 (e.g., when the optical sensor 228 is disposed on or within the robotic arm 116). For example, the optical sensor 228 may be or include a laser tracker that emits light that strikes the optical sensor tracker target 312 (which may be or include the laser target 316). Information about the intensity, frequency, angle, etc. of the light that contacts the laser target 316 may be generated and sent to the computing device 102 and / or the computing device 102. Similarly, information about the first pose of the robotic arm 116 may be generated by a sensor disposed within or proximate to the robotic arm 116 (e.g., a gyroscope, an accelerometer, a combination thereof, etc.). The measured movement may be used together with a known predetermined pose (eg, the pose before the robotic arm 116 begins moving) to determine a first pose of the robotic arm 116 .

[0106] In some embodiments, the optical sensor 228 may view both the optical sensor tracker target 312 and the tracking markers 232 disposed on the robotic arm 116. In such embodiments, the optical sensor 228 may include a 3D camera capable of identifying the optical sensor tracker target 312 (which may be a 3D target 328) and the tracking markers 232 disposed on the robotic arm 116. This information may be captured in the form of one or more images or a continuous data stream depicting both the 3D target 328 and the tracking markers 232 and their respective relative positions. In some embodiments, the optical sensor 228 may continuously generate information as the robotic arm 116 moves while performing a surgical task.

[0107] The method 500 also includes determining a pose of the robotic arm 116 relative to the navigation tracker based on the information received from the optical sensor (step 508). The registration 128 may be used to determine the pose of the robotic arm 116 relative to the navigation tracker 236. When the optical sensor 228 tracks only the laser target 316, the measured movement of the robotic arm (e.g., movement measured by the accelerometer) may be processed using one or more transformations 124 to determine coordinates associated with the robotic arm 116 in a first coordinate system. The registration 128 may then be used to map the coordinates of the robotic arm 116 into a coordinate system associated with the navigation tracker 236. In some embodiments, the registration 128 may map the coordinates associated with the navigation tracker 236 and the coordinates associated with the robotic arm 116 into a third coordinate system so that the computing device 102 and / or the navigation system 118 have information about the pose of the robotic arm 116 relative to the navigation tracker 236.

[0108] In embodiments where the optical sensor 228 tracks both the 3D target 328 and the robotic arm 116 (based on the relative movement of the tracking markers 232), image processing may be used to identify each of the 3D target 328 and the tracking markers 232 (e.g., using a QR code on each face of the optical sensor tracker target 312 and the tracking markers 232). The new pose of the optical sensor tracker target 312 and / or the tracking markers 232 may then be determined using one or more transformations 124 and information describing the initial pose of each of the 3D target 328 and the tracking markers 232 (e.g., from the database 130). The registration 128 may then map the coordinates of the tracking markers 232 into a coordinate system describing the 3D target 328, or vice versa. In some embodiments, the registration 128 may map the coordinates associated with the 3D target 328 and the coordinates associated with the tracking markers 232 into a common coordinate system so that the computing device 102 and / or the navigation system 118 have information about the pose of the robotic arm 116 relative to the navigation tracker 236.

[0109] In some embodiments, steps 504 and 508 can be repeated, as described above, each time the computing device 102 and / or the navigation system 118 determines that the third registration between the robotic arm 116 and the navigation tracker 236 is inaccurate or no longer valid. In some embodiments, when the third registration is no longer accurate, the third registration can be recalculated, while the first registration and / or the second registration can continue to be used (e.g., the anatomical element has not moved relative to the navigation tracker 236).

[0110] The present disclosure encompasses embodiments of method 500 that include more or fewer steps than those described above and / or one or more steps that are different than those described above.

[0111] As mentioned above, the present disclosure covers Figure 4 and Figure 5 (and the corresponding descriptions of methods 400 and 500) and including all steps less than the steps identified in Figure 4 and Figure 5 (and the corresponding descriptions of methods 400 and 500). The present disclosure also encompasses methods comprising one or more steps from one method described herein and one or more steps from another method described herein. Any correlation described herein may be or include registration or any other correlation.

[0112] The foregoing is not intended to limit the present disclosure to one or more forms disclosed herein. In the aforementioned specific embodiments, for example, for the purpose of simplifying the present disclosure, the various features of the present disclosure are grouped together in one or more aspects, embodiments and / or configurations. The features of the aspects, embodiments and / or configurations of the present disclosure may be combined in alternative aspects, embodiments and / or configurations other than those discussed above. The method of the present disclosure should not be interpreted as reflecting the following intention: the claims require more features than the features explicitly stated in each claim. On the contrary, as reflected in the following claims, aspects of the present invention are less than all the features of a single aforementioned disclosed aspect, embodiment and / or configuration. Therefore, the following claims are hereby incorporated into this specific embodiment, wherein each claim exists independently as a separate preferred embodiment of the present disclosure.

[0113] In addition, although the foregoing has included descriptions of one or more aspects, embodiments and / or configurations and certain variations and modifications, other variations, combinations and modifications are within the scope of the present disclosure, for example, within the skill and knowledge of those skilled in the art after understanding the present disclosure. It is intended to obtain rights to include alternative aspects, embodiments and / or configurations, including alternative, interchangeable and / or equivalent structures, functions, ranges or steps of those claimed, regardless of whether such alternative, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and are not intended to be disclosed for any patentable subject matter.

Claims

1. A system, comprising: processor; and a memory having data stored thereon, the data, when processed by the processor, causing the processor to: determining a first registration between the navigation tracker and the anatomical element based on a first type of navigation element and a second type of navigation element both disposed on the navigation tracker; determining a second registration between the robotic arm and the navigation tracker based on a third type of navigation element disposed on the navigation tracker; as well as The robotic arm is navigated relative to the anatomical element based on the first registration and the second registration. 2 . The system of claim 1 , wherein the navigation elements of the first type, the navigation elements of the second type, and the navigation elements of the third type are arranged on the navigation tracker in a predetermined configuration.

3. The system of claim 2, wherein the navigation element of the third type comprises a first optical tracking marker tracked by a third imaging device, and wherein determining the second registration comprises the processor: receiving information about a first position of the first optical tracking marker and a first pose of the robotic arm from the third imaging device; and Based on the information about the first position of the first optical tracking marker and the first pose of the robotic arm, a position of the robotic arm relative to the first optical tracking marker is determined.

4. The system of claim 3, wherein the navigation element of the first type comprises a plurality of fluoroscopic markers detectable by a first imaging device, and wherein determining the first registration comprises the processor: receiving a plurality of images depicting the plurality of fluoroscopic markers and the anatomical element from the first imaging device; and Based on the plurality of images, a position of each of the plurality of fluoroscopic markers relative to the anatomical element is determined.

5. The system of claim 4, wherein the navigational elements of the second type include a plurality of navigational markers detectable by a second imaging device, and wherein determining the first registration further comprises the processor: receiving information about the location of each of the plurality of navigation markers from the second imaging device; and Based on the information regarding the position of each navigation marker and the predetermined configuration, a position of each navigation marker relative to the anatomical element is determined.

6. The system of claim 5, wherein the robotic arm includes a second optical tracking marker, and wherein the third imaging device tracks the second optical tracking marker.

7. The system of claim 6, wherein the first imaging device comprises a fluoroscopic imaging device, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a laser tracker.

8. The system of claim 5, wherein the navigation tracker is disposed on a patient table, wherein the robotic arm is disposed on a second table a first distance from the patient table, and wherein the third imaging device is disposed a second distance from the robotic arm.

9. The system of claim 8, wherein the first imaging device comprises an O-arm, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a three-dimensional (3D) camera.

10. A system, comprising: a robotic arm, the robotic arm being arranged at a first distance from the patient bed; processor; and a memory having data stored thereon, the data, when processed by the processor, causing the processor to: determining a first registration between the navigation tracker and the anatomical element based on a first type of navigation element and a second type of navigation element both disposed on the navigation tracker; determining a second registration between the robotic arm and the navigation tracker based on a third type of navigation element disposed on the navigation tracker; as well as Based on the first registration and the second registration, a first pose of the robotic arm relative to the anatomical element is determined.

11. The system of claim 10, wherein the navigation elements of the first type, the navigation elements of the second type, and the navigation elements of the third type are arranged on the navigation tracker in a predetermined configuration.

12. The system of claim 11, wherein the navigation element of the third type comprises an optically tracked target tracked by a third imaging device, and wherein determining the second registration comprises the processor: receiving first information about a first pose of the optical tracking target from the third imaging device; and Based on the first information about the first pose of the optically tracked target and the predetermined configuration, a first position of the robotic arm relative to the navigation tracker is determined.

13. The system of claim 12, wherein the navigation element of the first type comprises a plurality of fluoroscopic markers detectable by a fluoroscopic imaging device, and wherein the first registration comprises the processor: receiving a plurality of images depicting the plurality of fluoroscopic markers and the anatomical element from the fluoroscopic imaging device; and Based on the plurality of images, a position of each of the plurality of fluoroscopic markers relative to the anatomical element is determined.

14. The system of claim 13, wherein the navigational elements of the second type include a plurality of navigational markers detectable by a second imaging device, and wherein the first registering further comprises the processor: receiving information about the location of each of the plurality of navigation markers from the second imaging device; and Based on the information regarding the position of each navigation marker and the predetermined configuration, a position of each navigation marker relative to the anatomical element is determined.

15. The system of claim 14, wherein the third imaging device is disposed proximate to an end effector of the robotic arm.

16. The system of claim 15, wherein the robotic arm is disposed on a second workbench that is movable relative to the patient bed.

17. The system of claim 15, wherein the second imaging device comprises a navigation camera, and wherein the third imaging device comprises a laser tracker.

18. A method comprising: determining a first registration between a plurality of fluoroscopic markers disposed on the navigation tracker and the anatomical element based on a first plurality of images captured by the first imaging device; determining a second registration between a plurality of navigation markers disposed on the navigation tracker and the anatomical element based on a second plurality of images captured by a second imaging device; determining a third registration between the navigation tracker and the robotic arm based on an optical sensor tracker target disposed on the navigation tracker; as well as The robotic arm is moved relative to the anatomical element based on the second registration and the third registration.

19. The method of claim 18, wherein the plurality of fluoroscopic markers, the plurality of navigation markers, and the optical sensor tracker target are disposed on the navigation tracker in a fixed orientation.

20. The method of claim 19, wherein the optical sensor tracker target is tracked by a third imaging device, and wherein determining the third registration further comprises: receiving information describing a first position of the optical sensor tracker target relative to the robotic arm from the third imaging device; as well as A first pose of the robotic arm relative to the navigation tracker is determined based on the information describing the first position of the optical sensor tracker target.

Images