Systems and methods for registration using one or more fiducial markers

Abstract

Systems and methods for registration using one or more fiducial markers are provided. First image data depicting at least a portion of an anatomical element and a fiducial marker can be received, and a fiducial marker-based registration process can be performed. Second image data depicting at least a portion of the anatomical element and the fiducial marker can be received. Movement of the anatomical element beyond a threshold distance can be determined based on the fiducial marker, and the registration of the anatomical element can be updated in response to determining that the movement is beyond the threshold distance.

Description

Background Technology

[0001] This disclosure relates in general to registration, and more specifically to registration using one or more reference markers.

[0002] Surgical robots can assist surgeons or other healthcare providers in performing surgical procedures, or can autonomously complete one or more surgical procedures. Imaging can be used by healthcare providers for diagnostic and / or therapeutic purposes. Patient anatomy can change over time, especially after medical implants are placed within the patient's anatomy. Summary of the Invention

[0003] Examples of aspects of this disclosure include: A system according to at least one embodiment of the present disclosure includes: an imaging device operable to acquire image data; a reference marker, which is biocompatible and visible in the image data, wherein the reference marker is implanted near an anatomical element in a patient; a processor; and a memory storing data for processing by the processor, the data causing the processor, when processed, to: receive first image data depicting at least a portion of the patient and the reference marker; perform a registration process based on the reference marker; receive second image data depicting at least a portion of the anatomical element and the reference marker; determine, based on the reference marker, whether movement of the anatomical element exceeds a threshold distance; and, in response to determining that movement exceeds the threshold distance, update the registration of the anatomical element with a navigation system based on the reference marker.

[0004] Any aspect of this article, wherein the reference marker includes at least one of a liquid or a gel.

[0005] Any aspect of this article, wherein the imaging device includes an ultrasound imaging device.

[0006] Any aspect of this article, wherein the reference markers include at least three reference markers.

[0007] In any aspect of this paper, the system also includes a robotic arm configured to locate and orient at least one reference marker in the vicinity of the anatomical element.

[0008] In any aspect of this article, the anatomical element is one or more vertebrae.

[0009] In any aspect of this article, a benchmark marker is permanently implanted in the patient.

[0010] Any aspect of this document, wherein the imaging apparatus includes a first imaging apparatus using ionizing radiation and a second imaging apparatus using non-ionizing radiation, and wherein first image data is received from the first imaging apparatus and second image data is received from the second imaging apparatus.

[0011] Any aspect of this article, wherein the imaging device uses non-ionizing radiation, and wherein first image data and second image data are received from the imaging device.

[0012] In any aspect of this document, the memory stores additional instructions for execution by at least one processor, which, when executed, further cause the at least one processor to: receive third image data depicting at least a portion of an anatomical element and a reference marker; determine, based on the reference marker, whether the movement of the anatomical element exceeds a threshold distance; and, in response to determining that the movement exceeds the threshold distance, update the registration of the anatomical element with the navigation system based on the reference marker.

[0013] In any aspect of this article, the third image data is received from the second imaging device.

[0014] In any aspect of this document, the memory stores additional instructions for execution by at least one processor, which, when executed, further cause the at least one processor to: use image processing to determine the pose of a reference marker, wherein registration is based on the pose of the reference marker.

[0015] A system according to at least one embodiment of the present disclosure includes: an imaging device operable to acquire image data, wherein the imaging device uses non-ionizing radiation; a reference marker, which is biocompatible and visible in the image data, wherein the reference marker is implanted near a target anatomical element in a patient; a processor; and a memory storing data for processing by the processor, the data causing the processor, when processed, to: receive first image data depicting at least a portion of the target anatomical element and the reference marker; determine the pose of the reference marker using image processing; perform a registration process based on the pose of the reference marker; receive second image data depicting at least a portion of the target anatomical element and the reference marker; determine, based on the reference marker, whether movement of the target anatomical element exceeds a threshold distance; and update the registration of the target anatomical element with a navigation system based on the reference marker in response to determining that movement exceeds the threshold distance.

[0016] In any aspect of this article, a benchmark marker is permanently implanted in the patient.

[0017] Any aspect of this article, wherein the reference marker includes at least one of a liquid or a gel.

[0018] Any aspect of this article, wherein the imaging device includes an ultrasound imaging device.

[0019] A method according to at least one embodiment of the present disclosure, the method comprising: receiving from an imaging device first image data depicting at least a portion of a target anatomical element and a reference marker implanted near the target anatomical element; using image processing to determine the pose of the reference marker; performing a registration process based on the pose of the reference marker; receiving second image data depicting at least a portion of the target anatomical element and the reference marker; determining, based on the reference marker, whether movement of the target anatomical element exceeds a threshold distance; and updating the registration of the target anatomical element with a navigation system based on the reference marker in response to determining that movement exceeds the threshold distance.

[0020] In any of the aspects of this paper, the method further includes: receiving third image data depicting at least a portion of the target anatomical element and a reference marker; determining, based on the reference marker, whether the movement of the anatomical element exceeds a threshold distance; and updating the registration of the anatomical element with the navigation system based on the reference marker in response to determining that the movement exceeds the threshold distance.

[0021] Any aspect of this article, wherein the reference marker includes at least one of a liquid or a gel.

[0022] Any aspect of this article, wherein the imaging device includes an ultrasound imaging device.

[0023] Any one aspect can be combined with any one or more other aspects.

[0024] Any one or more of the features disclosed in this article.

[0025] This article generally discloses one or more of the features.

[0026] Any one or more of the features generally disclosed in this article are combined with any one or more other features generally disclosed in this article.

[0027] Any aspect / feature / implementation may be combined with any one or more other aspects / features / implementations.

[0028] Use any one or more of the aspects or features disclosed herein.

[0029] It should be understood that any feature described herein may be combined with any other feature as described herein to claim protection, regardless of whether the feature comes from an implementation of the same description.

[0030] Details of one or more aspects of this disclosure are set forth in the following drawings and description. Other features, objects, and advantages of the technology described in this disclosure will be apparent from the description, drawings, and claims.

[0031] The phrases “at least one,” “one or more,” and “and / or” are open-ended expressions that possess both connective and disjoint qualities in operation. For example, each of the expressions “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and / or C” means only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together. When each of A, B, and C in the above expressions refers to elements such as X, Y, and Z or element classes such as X1-Xn, Y1-Ym, and Z1-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., X1 and X2), and a combination of elements selected from two or more classes (e.g., Y1 and Zo).

[0032] The term "a" refers to one or more of the same entity. Therefore, the terms "a," "one or more," and "at least one" are used interchangeably herein. It should also be noted that the terms "comprising," "including," and "having" are used interchangeably.

[0033] The foregoing is a simplified overview of this disclosure to provide an understanding of some aspects thereof. This summary is neither a broad nor an exhaustive overview of this disclosure and its various aspects, embodiments, and configurations. It is not intended to identify key or essential elements of this disclosure, nor to define its scope, but rather to present the concepts of this disclosure in a simplified form as an introduction to the more detailed description presented below. It should be understood that other aspects, embodiments, and configurations of this disclosure may utilize one or more of the features set forth above or described in detail below, individually or in combination.

[0034] Many additional features and advantages of this disclosure will become apparent to those skilled in the art upon consideration of the embodiments described below. Attached Figure Description

[0035] The accompanying drawings are incorporated in and form a part of this specification to illustrate several examples of this disclosure. These drawings, together with the description, explain the principles of this disclosure. The drawings illustrate only preferred and alternative examples of how to implement and use this disclosure, and these examples should not be construed as limiting this disclosure solely to the illustrated and described examples. Further features and advantages will become apparent from the following more detailed description of various aspects, embodiments, and configurations of this disclosure, as illustrated by the accompanying drawings referenced below.

[0036] Figure 1 It is a block diagram of a system according to at least one embodiment of the present disclosure; Figure 2 It is a flowchart of at least one embodiment according to this disclosure; Figure 3 It is a flowchart of at least one embodiment according to this disclosure; Figure 4 This is a schematic diagram of anatomical elements and one or more reference markers according to at least one embodiment of this disclosure; and Figure 5 It is a flowchart of at least one embodiment according to this disclosure. Detailed Implementation

[0037] It should be understood that the various aspects disclosed herein can be combined with combinations different from those specifically presented in the specification and drawings. It should also be understood that, depending on the example or embodiment, certain actions or events of any process or method described herein may be performed in a different order, and / or may be added, combined, or omitted entirely (e.g., implementing the disclosed technology may not require all described actions or events depending on the different embodiments of this disclosure). Furthermore, although some aspects of this disclosure are described for clarity as being performed by a single module or unit, it should be understood that the technology of this disclosure can be performed by a combination of units or modules associated with, for example, computing devices and / or medical devices.

[0038] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the function may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Alternatively or additionally, the function may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (instructions alone or in combination). The computer-readable medium may include a non-transitory computer-readable medium that corresponds to a tangible medium, such as a data storage medium (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer).

[0039] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general-purpose microprocessor), graphics processing units (e.g., Nvidia GeForce RTX 2000 series processors, Nvidia GeForce RTX 3000 series processors, AMD Radeon RX 5000 series processors, AMD Radeon RX 6000 series processors, or any other graphics processing units), application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the foregoing structures or any other physical structures suitable for implementing the described techniques. Furthermore, these techniques may be fully implemented in one or more circuit or logic elements.

[0040] Before explaining any embodiment of this disclosure in detail, it should be understood that this disclosure is not limited in its application to the construction details and component arrangements set forth in the following description or illustrated in the accompanying drawings. This disclosure can have other embodiments and can be practiced or implemented in various ways. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising,” “including,” or “having,” and variations thereof, is intended to cover items listed thereafter and their equivalents, as well as additional items. In addition, this disclosure may use examples to illustrate one or more aspects thereof. Unless otherwise expressly stated, the use or listing of one or more examples (which may be indicated by “for example,” “by way of example,” “such as,” or similar language) is not intended to, and does not limit, the scope of this disclosure.

[0041] The terms proximal and distal are used in this disclosure in their usual medical sense, with the proximal being closer to the operator or user of the system and further away from the patient's body or surgical area of ​​concern, and the distal being closer to the patient's body or surgical area of ​​concern and further away from the operator or user of the system.

[0042] In surgical procedures such as those used for robotic surgery or robot-assisted surgery (e.g., spinal surgery), at least one vertebra is initially registered using two X-ray images registered to a three-dimensional (3D) computed tomography (CT) or magnetic resonance imaging (MRI) scan. During the surgical procedure, the patient may move. These movements necessitate re-registration of the vertebrae to address the movement. Routinely, this re-registration is achieved by obtaining two additional X-ray images. Another routine procedure is to use 3D markers drilled into the vertebrae and visible to the navigation system. The first option is time-consuming and involves exposing the patient to unwanted ionizing radiation. The second routine procedure may interfere with the robotic system.

[0043] Therefore, according to at least one embodiment of this disclosure, a biocompatible reference marker can be manually inserted by a user or by a robot at a specific location near a relevant or target anatomical element (e.g., a vertebra). A re-registration process can then be applied by using an ultrasound probe in the small area where the reference marker was placed. The re-registration process can be applied to various scenarios, including but not limited to non-moving verification of anatomical elements; and / or registration of anatomical elements after movement.

[0044] The embodiments disclosed herein provide technical solutions to one or more of the following problems: (1) determining the movement of anatomical elements without the use of ionizing radiation, (2) achieving registration or re-registration of anatomical elements using imaging without ionizing radiation, and (3) improving the safety of patients and surgical teams by reducing exposure to ionizing radiation.

[0045] First go to Figure 1 This diagram illustrates a block diagram of a system 100 according to at least one embodiment of the present disclosure. System 100 can be used to perform registration and / or implement one or more other aspects of one or more methods disclosed herein using at least three reference markers 126 implanted in a patient. System 100 includes a computing device 102, one or more imaging devices 112, a robot 114, a navigation system 118, one or more reference markers 126, a database 130, and / or a cloud or other network 134. Systems according to other embodiments of the present disclosure may include more or fewer components than system 100. For example, system 100 may not include imaging devices 112, robot 114, navigation system 118, one or more components of computing device 102, database 130, and / or cloud 134.

[0046] The computing device 102 includes a processor 104, a memory 106, a communication interface 108, and a user interface 110. Other embodiments of the computing device according to this disclosure may include more or fewer components than computing device 102.

[0047] The processor 104 of computing device 102 may be any processor described herein or any similar processor. Processor 104 may be configured to execute instructions stored in memory 106 that enable processor 104 to perform one or more computational steps using or based on data received from imaging device 112, robot 114, navigation system 118, database 130 and / or cloud 134.

[0048] Memory 106 may be or include RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible non-transitory memory used to store computer-readable data and / or instructions. Memory 106 may store information or data useful for performing any step of, for example, method 500 described herein or any step of any other method. Memory 106 may store, for example, instructions and / or machine learning models that support one or more functions of robot 114. For example, memory 106 may store content (e.g., instructions and / or machine learning models) that enables image processing 120, segmentation 122, and / or registration 124 when executed by processor 104.

[0049] Image processing 120 enables processor 104 to process image data (received from, for example, imaging device 112, the imaging device of navigation system 118, or any imaging device) of an image to, for example, identify one or more anatomical elements and / or reference markers 126 depicted in the image data. Reference markers 126 may be biocompatible and located near a target anatomical element implanted in the patient's body. The target anatomical element may be, for example, one or more vertebrae, but it should be understood that the target anatomical element may be any soft and / or hard tissue. Information may include, for example, identification of the anatomical element and / or reference marker 126, boundaries between anatomical elements, boundaries of hard and / or soft tissue, etc. Image processing 120 may identify anatomical elements, for example, based on reference marker 126, and / or by determining the difference or contrast between the colors or grayscale values ​​of image pixels. For example, the boundary of an anatomical element may be identified as the contrast between brighter and darker pixels. Imaging processing 120 may also use segmentation 122, as described below.

[0050] Segmentation 122 enables processor 104 to process image data (received from, for example, the imaging device of imaging device 112, the imaging device of navigation system 118, or any imaging device) to, for example, identify individual objects and / or anatomical elements in the image data. In some embodiments, image processing 120 may use segmentation 122. Segmentation 122 may enable processor 104 to identify the boundaries of reference marker 126 or anatomical elements by using, for example, feature recognition. For example, segmentation 122 may enable processor 104 to identify vertebrae in the image data. In other cases, segmentation 122 may enable processor 104 to identify the boundaries of reference marker 126 or anatomical elements by determining the differences or contrasts between the colors or grayscale values ​​of image pixels.

[0051] The reference markers 126 and / or anatomical elements identified from image processing 120 and / or segmentation 122 enable registration 124 to identify target anatomical elements based on the reference markers 126, as will be described in more detail below.

[0052] Registration 124 enables processor 104 to process the identified reference markers 126 and / or anatomical elements obtained from image processing 120 and / or segmentation 122 to register the anatomical elements depicted in the image data to, for example, a preliminary image of a patient based on the identified reference markers 126. It should be understood that although image processing 120, segmentation 122, and registration 124 are described separately, image processing 120 and / or segmentation 122 may be a part or step of registration 124. For example, registering one or more anatomical elements may include using image processing 120 and / or segmentation 122 to identify one or more reference markers 126 and / or one or more anatomical elements depicted in the image data.

[0053] In some implementations, such content may be organized into one or more applications, modules, packages, layers, or engines, if provided as instructions. Alternatively or additionally, memory 106 may store other types of content or data (e.g., machine learning models, artificial neural networks, deep neural networks, etc.) that can be processed by processor 104 to implement the various methods and features described herein. Therefore, although the various contents of memory 106 may be described as instructions, it should be understood that the functions described herein can be implemented using instructions, algorithms, and / or machine learning models. This data, algorithms, and / or instructions enable processor 104 to manipulate data stored in memory 106 and / or data received from or via imaging device 112, robot 114, database 130, and / or cloud 134.

[0054] The computing device 102 may also include a communication interface 108. The communication interface 108 can be used to receive image data or other information from external sources (such as imaging device 112, robot 114, navigation system 118, database 130, cloud 134, and / or any other system or component not part of system 100), and / or to send instructions, images, or other information to external systems or devices (e.g., another computing device 102, imaging device 112, robot 114, navigation system 118, database 130, cloud 134, and / or any other system or component not part of system 100). The communication interface 108 may include one or more wired interfaces (e.g., USB port, Ethernet port, FireWire port) and / or one or more wireless transceivers or interfaces (configured to send and / or receive information, for example, via one or more wireless communication protocols such as 802.11a / b / g / n, Bluetooth, NFC, ZigBee, etc.). In some implementations, the communication interface 108 can be used to enable the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time required to complete computationally intensive tasks or for any other reason.

[0055] The computing device 102 may also include one or more user interfaces 110. User interface 110 may be or include a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or providing information to a user. User interface 110 may be used, for example, to receive user selections or other user input regarding any step of any method described herein. Nevertheless, any required input for any step of any method described herein may be automatically generated by system 100 (e.g., by processor 104 or another component of system 100) or received by system 100 from a source external to system 100. In some embodiments, user interface 110 may be used to allow a surgeon or other user to modify instructions to be executed by processor 104, and / or modify or adjust settings displayed on user interface 110 or corresponding to other information on user interface 110, according to one or more embodiments of this disclosure.

[0056] Although user interface 110 is shown as part of computing device 102, in some embodiments, computing device 102 may utilize user interface 110 which is housed separately from one or more other components of computing device 102. In some embodiments, user interface 110 may be located near one or more other components of computing device 102, while in other embodiments, user interface 110 may be located away from one or more other components of computing device 102.

[0057] Imaging device 112 may be operable to image reference marker 126, anatomical features (e.g., bones, veins, tissues, etc.) and / or other aspects of the patient's anatomy to produce image data (e.g., image data depicting or corresponding to bones, veins, tissues, etc.). As used herein, "image data" refers to data generated or captured by imaging device 112, including data in machine-readable form, graphical / visual form, and any other form. In various examples, image data may include data corresponding to the patient's reference marker 126, anatomical features, or a portion thereof. The image data may be or include preoperative images, intraoperative images, postoperative images, or images taken independently of any surgical procedure. In some embodiments, first imaging device 112 may be used to acquire first image data (e.g., a first image) at a first time, and second imaging device 112 may be used to acquire second image data (e.g., a second image) at a second time after the first time. In such embodiments, the first imaging device may use ionizing radiation (e.g., X-ray scanning) and the second imaging device may be radiation-free (e.g., ultrasound scanning). In other embodiments, imaging device 112 may acquire both first and second image data.

[0058] Imaging device 112 may be capable of capturing 2D or 3D images to generate image data. Imaging device 112 may be or include, for example, an ultrasound scanner (which may include, for example, physically separate transducers and receivers, or a single ultrasound transceiver), an O-arm, C-arm, G-arm, or any other device utilizing X-ray-based imaging (e.g., a fluorescence microscope, CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, a microscope, an optical camera, a thermal imaging camera (e.g., an infrared camera), a radar system (which may include, for example, a transmitter, a receiver, a processor, and one or more antennas), or any other imaging device 112 adapted to obtain images of a patient's anatomical features. Imaging device 112 may be entirely contained within a single housing, or may include a transmitter / transmitter and a receiver / detector located in a separate housing or otherwise physically separated.

[0059] In some embodiments, imaging device 112 may include more than one imaging device 112. For example, a first imaging device may provide first image data and / or a first image, and a second imaging device may provide second image data and / or a second image. In yet other embodiments, the same imaging device may be used to provide both first image data and second image data and / or any other image data described herein. Imaging device 112 may be used to generate an image data stream. For example, imaging device 112 may be configured to use an open shutter operation or to use shutter operations that alternate continuously between open and closed to capture a series of images. For the purposes of this disclosure, unless otherwise specified, if the image data represents two or more frames per second, the image data may be considered continuous and / or provided as an image data stream.

[0060] Robot 114 can be any surgical robot or surgical robot system. Robot 114 can be, or includes, for example, Mazor X. ™ The Stealth Edition robotic guidance system. Robot 114 can be configured to position imaging device 112 at one or more precise locations and orientations, and / or return imaging device 112 to the same location and orientation at a later time point. Robot 114 may additionally or alternatively be configured to manipulate surgical instruments (whether or not based on guidance from navigation system 118) to perform or assist surgical tasks. Robot 114 may also be configured to position and / or insert one or more reference markers 126 into the patient and in proximity to target anatomical elements. In some embodiments, robot 114 may be configured to hold and / or manipulate anatomical elements during or in conjunction with surgical procedures. Robot 114 may include one or more robotic arms 116. In some embodiments, robotic arms 116 may include a first robotic arm and a second robotic arm, but robot 114 may include more than two robotic arms. In some embodiments, one or more of the robotic arms 116 may be used to hold and / or manipulate imaging device 112. In embodiments where the imaging device 112 includes two or more physically separate components (e.g., transmitters and receivers), one robotic arm 116 may hold one such component, and another robotic arm 116 may hold another such component. Each robotic arm 116 may be able to be positioned independently of the other robotic arms. The robotic arms 116 may be controlled in a single shared coordinate space or in separate coordinate spaces.

[0061] The robot 114, together with the robotic arm 116, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Furthermore, the robotic arm 116 can be or is capable of being positioned in any pose, plane, and / or focal position. Pose includes position and orientation. Therefore, an imaging device 112, surgical instrument, or other object held by the robot 114 (or more specifically, by the robotic arm 116) can be precisely positioned in one or more desired and specific locations and orientations.

[0062] The robotic arm 116 may include one or more sensors that enable the processor 104 (or the processor of the robot 114) to determine the precise pose of the robotic arm (and any object or element held or fixed to the robotic arm) in space.

[0063] In some embodiments, reference markers (i.e., navigation markers) may be placed on robot 114 (including, for example, on robotic arm 116), imaging device 112, or any other object in the surgical space. The reference markers may be tracked by navigation system 118, and the results of the tracking may be used by the operator of robot 114 and / or system 100 or any of its components. In some embodiments, navigation system 118 may be used to track other components of the system (e.g., imaging device 112), and the system may operate without using robot 114 (e.g., a surgeon manually manipulating imaging device 112 and / or one or more surgical instruments, for example, based on information and / or instructions generated by navigation system 118).

[0064] During operation, navigation system 118 can provide navigation for the surgeon and / or surgical robot. Navigation system 118 can be any navigation system currently known or developed in the future, including, for example, Medtronic StealthStation. ™The S8 surgical navigation system or any successor thereof. Navigation system 118 may include one or more cameras or other sensors for tracking one or more reference markers, navigation trackers, or other objects within the operating room or other room where part or all of system 100 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, navigation system 118 may include one or more electromagnetic sensors. In various embodiments, navigation system 118 may be used to track the positioning and orientation (e.g., pose) of imaging device 112, robot 114 and / or robotic arm 116 and / or one or more surgical instruments (or more specifically, for tracking the pose of navigation trackers directly or indirectly attached in a fixed relationship to one or more of the foregoing). Navigation system 118 may include a display for displaying one or more images from an external source (e.g., computing device 102, imaging device 112, or other sources), or for displaying images and / or video streams from one or more cameras or other sensors of navigation system 118. In some embodiments, system 100 may operate without using navigation system 118. The navigation system 118 may be configured to provide guidance to the surgeon or other users of the system 100 or its components, to the robot 114, or to any other element of the system 100 regarding, for example, the pose of one or more anatomical elements, whether the tool is in the appropriate trajectory, and / or how to move the tool into the appropriate trajectory to perform the surgical task in accordance with the preoperative or other surgical plan.

[0065] The reference marker 126 may be implanted near a target anatomical element within the patient's body to achieve or assist in registration of the target anatomical element. The reference marker 126 may be biocompatible, allowing it to be permanently or temporarily inserted into the patient's body. The reference marker 126 is visible in image data from any imaging device 112, such as, for example, an ultrasound imaging device, an X-ray imaging device, etc. The reference marker 126 may be, for example, a liquid or gel; however, it should be understood that in other embodiments, the reference marker 126 may be a solid material. The reference marker 126 may be used in registration processes using, for example, registration 124.

[0066] In cases where the target anatomical element may be difficult to identify in image data, the reference marker 126 can be used to identify the target anatomical element. For example, vertebrae may be difficult to identify in ultrasound imaging, but the reference marker 126 can be easily identified in ultrasound imaging. Furthermore, at least three reference markers 126 can be implanted near the target anatomical element, such that the at least three reference markers 126 form a unique pattern, and can help identify the target anatomical element as well as its orientation and / or location.

[0067] The reference marker 126 can also be used to detect movement of a target anatomical element using imaging, for example, without ionizing radiation. For example, first image data depicting at least a portion of the target anatomical element and the reference marker, obtained from, for example, an ultrasound probe, can be compared with second image data depicting at least a portion of the target anatomical element and the reference marker, obtained after the first image data. Differences between the first and second image data (and, in particular, differences in the pose of the reference marker 126 in the first image data compared to the second image data) can indicate that the target anatomical element has moved. In such cases, registration can be updated to account for movement of the target anatomical element. This process can be repeated throughout the surgical procedure as needed. For example, third image data can be obtained and compared with the second image data to determine whether movement of the anatomical element has occurred.

[0068] Database 130 may store information relating one coordinate system to another (e.g., relating one or more robot coordinate systems to a patient coordinate system and / or a navigation coordinate system). Database 130 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about the target and / or image information about the anatomical structures of the patient at and / or proximal to the surgical site, for use by the user of robot 114, navigation system 118, and / or computing device 102 or system 100); one or more images of surgical procedures performed by or with the assistance of one or more other components of system 100; and / or any other useful information. Database 130 may be configured to provide any such information to computing device 102 or any other device of system 100 or any other device outside system 100, whether directly or via cloud 134. In some implementations, database 130 may be or include part of a hospital image storage system, such as a Picture Archiving and Communication System (PACS), a Health Information System (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records that include image data.

[0069] Cloud 134 can be or represents the Internet or any other wide area network. Computing device 102 can connect to cloud 134 via communication interface 108 using a wired connection, a wireless connection, or both. In some embodiments, computing device 102 can communicate with database 130 and / or external devices (e.g., computing devices) via cloud 134.

[0070] System 100 or a similar system may be used, for example, to implement one or more aspects of any of the methods described herein, such as method 500. System 100 or a similar system may also be used for other purposes.

[0071] Turning Figure 2An example of a model architecture 200 is shown that supports methods and systems (e.g., artificial intelligence (AI) based methods and / or systems) for processing image data and registering one or more anatomical elements.

[0072] Image data 206 may be used as input to image processing 120 by a processor (such as processor 104). Image processing 120 may output the identified target anatomical element 210 and / or one or more identified reference markers 126. In some embodiments, image data 206 may be received from an imaging device (such as imaging 112), an imaging device of a navigation system (such as navigation system 118), or any other imaging device or component of a system (such as system 100). It should be understood that image data 206 may depict one or more reference markers 126, and image processing 120 may process image data 206 to output pose information of reference markers 126 (which can then be used, for example, to determine pose information of reference markers 126 and / or associated target anatomical element 210). The pose information may correspond to computer-coded data describing the pose of reference markers 126. For example, in some embodiments, pose information may include the coordinates and / or orientation of reference markers 126. In other examples, pose information may include, for example, a matrix describing the pose of reference markers 126. It should be understood that pose information can be encoded in any number of ways and may include, for example, a description of the location of reference marker 126 in reference space, a vector (e.g., a ternary vector), or a matrix.

[0073] As previously described, image processing 120 may use segmentation 122 to identify reference markers 126 and / or anatomical elements 210. Segmentation 122 may be configured to segment reference markers 126 and / or anatomical elements 210 from image data 206 to produce one or more identified anatomical elements 210 and / or identified reference markers 126. Segmentation of reference markers 126 and / or anatomical elements 210 from image data when image data 206 includes a three-dimensional representation of a patient's anatomy may include: identifying the boundaries of one or more reference markers 126 and / or anatomical elements 210, and forming separate three-dimensional representations of one or more reference markers 126 and / or anatomical elements 210. In some embodiments, identifying boundaries may include identifying adjacent groups of pixels with sufficiently large contrast to represent the boundaries of the anatomical element 210 depicted therein. In other embodiments, feature recognition may be used to identify the boundaries of anatomical elements 210 and / or reference markers 126. For example, feature recognition may be used to identify the contours of vertebrae.

[0074] Image processing 120 can be trained using historical image data. In other embodiments, image processing 120 can be trained using image data 206. In such embodiments, image processing 120 can be trained before image data 206 is input into image processing 120 or in parallel with inputting image data 206 into image processing 120.

[0075] As previously described, image processing 120 may output identified anatomical elements 210 and / or identified reference markers 126. The identified anatomical elements 210 and / or reference markers 126 may be used by processor 104 as input to registration 124. Registration 124 may output one or more registered anatomical elements 216. Registration 124 may register anatomical elements based on reference markers 126 identified in image data 206. More specifically, in some embodiments, registration 124 may use the identified reference markers 126, and the pose information of the identified reference markers 126 may be used to register anatomical elements 210. Registration model 214 may be configured to register one or more anatomical elements 210 to, for example, preoperative images or any image.

[0076] Registration 124 can be trained using historical or analog image data depicting one or more anatomical elements 210 and reference markers 126, historical identified anatomical elements, and / or historical identified reference markers. In other embodiments, registration 124 can be trained using identified anatomical elements 210 and reference markers 126. In such embodiments, registration model 124 can be trained before or in parallel with inputting identified anatomical elements 210 and reference markers 126 into registration 124.

[0077] Figure 3 Method 300, which can be used, for example, to generate models, is described.

[0078] Method 300 includes generating a model (step 304). The model may be image processing 120, segmentation 122, and / or registration 124. A processor (such as processor 104) may generate the model. The model may be generated to facilitate and achieve, for example, the identification of one or more anatomical elements and / or objects depicted in image data, and the registration of one or more anatomical elements.

[0079] Method 300 also includes training the model (step 308). In embodiments where the model is trained prior to surgical procedures, historical data from multiple patients can be used to train the model. In some embodiments, historical data can be obtained from patients similar to those to whom the surgical procedures will be performed. In other embodiments, historical data can be obtained from any patient.

[0080] In other implementations, the model can be trained in parallel with the use of another model. In some implementations, parallel training may include training the model using input received, for example, during or prior to a surgical procedure, while also using a separate model to receive and operate on the same input. This input may be specific to the patient undergoing the surgical procedure. In some cases, the trained model may replace the model in use when the trained model outperforms the model in use (whether in terms of efficiency, accuracy, or otherwise). Such parallel training may be useful, for example, when the model is used continuously (e.g., when inputs, such as images, are continuously updated), and the corresponding model can be trained in parallel for further improvement.

[0081] In some implementations, it should be understood that a model trained using historical data may initially be used as the primary model at the start of the surgical procedure. The training model may also be trained in parallel with the primary model using patient-specific input until the training model is sufficiently trained. The primary model can then be replaced by the training model.

[0082] Method 300 also includes storing the model (step 312). The model may be stored in memory (such as memory 106) and / or a database (such as database 130) for later use. In some embodiments, the model is stored in memory when it has been fully trained. The model is considered fully trained when it produces output that satisfies a predetermined threshold, which may be determined, for example, by a user, or automatically by a processor (such as processor 104).

[0083] This disclosure covers embodiments of method 300 that include more or fewer steps than those described above and / or one or more steps that are different from those described above.

[0084] Turning Figure 4This diagram illustrates a view of an example target anatomical element 210 having one or more reference markers 126. As previously described, reference markers 126 may be implanted in the vicinity of the target anatomical element 210 within the patient to achieve or assist in registration of the target anatomical element 210. As shown, three reference markers 126 are implanted in the patient and near the target anatomical element 210. It should be understood that reference markers 126 may include one, two, or more than two reference markers 126. Reference markers 126 are visible in image data from any imaging device 112 (such as, for example, an ultrasound imaging device, an X-ray imaging device, etc.). Reference markers 126 can be used to identify the target anatomical element 210 where it may be difficult to identify in imaging. For example, vertebrae may be difficult to identify in ultrasound imaging, but reference markers 126 can be easily identified in ultrasound imaging.

[0085] As previously described, the reference marker 126 can also be used to detect movement of the target anatomical element 210. For example, first image data depicting at least a portion of the target anatomical element 210 and the reference marker 126 can be compared with second image data depicting at least a portion of the target anatomical element 210 and the reference marker 126, acquired after the first image data. The difference between the first and second image data (and in particular, the difference in pose of the reference marker 126 in the first image data compared to the second image data) can indicate that the target anatomical element 210 has moved. In such cases, the registration can be updated to account for the movement of the target anatomical element.

[0086] Figure 5 Method 500 is described, which can be used, for example, in a registration process to identify a corresponding target anatomical element (such as target anatomical element 210) using one or more reference markers (such as one or more reference markers 126).

[0087] Method 500 (and / or one or more steps thereof) may be implemented by, for example, at least one processor or otherwise performed. The at least one processor may be the same as or similar to processor 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as robot 114) or a navigation system (such as navigation system 118). Method 500 may also be performed using processors other than any processor described herein. The at least one processor may perform method 500 by executing elements stored in memory (such as memory 106). Elements stored in memory and executed by the processor may cause the processor to perform one or more steps of the functions shown in method 500. One or more portions of method 500 may be performed by the processor executing any content of the memory (such as image processing 120, segmentation 122, and / or registration 124).

[0088] Method 500 includes receiving first image data (step 504). The first image data may be received or obtained from an imaging device (such as imaging device 112), which may be any imaging device, such as an MRI scanner, a CT scanner, any other X-ray-based imaging device, or an ultrasound imaging device. In other embodiments, the first image data may be received via a user interface (such as user interface 110), a database such as database 130, and / or a communication interface (such as communication interface 108 of a computing device such as computing device 102), and may be stored in memory (such as memory 106 of the computing device). The first image data may also be received from an external database or image repository (e.g., a hospital image storage system, such as a Picture Archiving and Communication System (PACS), a Health Information System (HIS), and / or another system for collecting, storing, managing, and / or sending electronic medical records), and / or via the Internet or another network. In some embodiments, the first image data may be received indirectly via any other component of the system or a node of a network to which the system is connected.

[0089] The first image data may be a two-dimensional image or a three-dimensional image (e.g., a three-dimensional representation) or a collection of two-dimensional images and / or three-dimensional representations. The first image data may depict at least a portion of a target anatomical element (such as target anatomical element 210) and one or more reference markers (such as one or more reference markers 126). In some embodiments, the first image data may be captured preoperatively (e.g., before surgery) and may be stored in a system (e.g., system 100) and / or one or more components of the system (e.g., database 130). The stored images may then be received preoperatively (e.g., before surgery) and / or intraoperatively (e.g., during surgery) as described above (e.g., by processor 104). In other embodiments, the first image data may be obtained during or before the surgical procedure. For example, the first image data may be used to establish the initial positions of reference markers and target anatomical elements.

[0090] In some implementations, the initial image data can be obtained using imaging devices that do not emit ionizing radiation, such as, for example, ultrasound probes. Reference markers can help register anatomical elements as depicted in ultrasound imaging, since reference markers are readily visible in ultrasound imaging, while anatomical elements may be difficult to identify. This advantageously reduces the exposure of patients and the surgical team to ionizing radiation, thereby increasing their safety.

[0091] Method 500 further includes determining the pose of a reference marker (step 508). As previously described, the reference marker may be implanted near a target anatomical element within the patient to achieve or assist in registration of the target anatomical element. Reference marker 126 may be used to identify the target anatomical element in cases where it may be difficult to identify in imaging. The reference marker may also be used to detect movement of the target anatomical element. The pose information of the reference marker may be obtained from first image data depicting at least a portion of the target anatomical element and the reference marker, processed by a processor such as processor 104 (or a processor of a navigation system) using image processing (such as image processing 120). In some embodiments, image processing may also be used, for example, to determine the pose information of the target anatomical element.

[0092] Method 500 further includes performing a registration process (step 512). Registration may be the same as or similar to registration 124. As previously described, the first image data may depict reference markers and target anatomical elements. Reference markers and / or target anatomical elements identified in the first image data enable processor 104 to register the target anatomical elements depicted in the first image data based on the identified reference markers. More specifically, registration may transform, map, or create correlations between the first image data and / or its components and initial or preliminary image data, which may then be used by a system (e.g., system 100) and / or one or more of its components (e.g., navigation system 118) to convert one or more coordinates in the patient coordinate space to one or more coordinates in the robot (e.g., robot 114) coordinate space and / or vice versa. It should be understood that registration may include registration between 3D images (e.g., CT scans) and one or more 2D images (e.g., fluoroscopic images) and / or vice versa, and / or registration between one 2D image and another 2D image and / or vice versa.

[0093] Method 500 also includes receiving registered anatomical elements from the registration model (step 524). The registered anatomical elements may be the same as or similar to registered anatomical element 216. The registered anatomical elements may be registered or associated with, for example, patient coordinate space and / or robot coordinate space. The registered anatomical elements enable the navigation system to provide navigation during surgical procedures.

[0094] Method 500 further includes receiving second image data (step 516). Step 516 may be the same as or similar to step 504 described above. The second image data may depict at least a portion of the target anatomical element and reference markers. In some embodiments, the second image data is acquired at a time period following the first image data. For example, the first image data may be acquired near the start or initiation of the surgical procedure, and the second image data may be acquired during the surgical procedure.

[0095] Method 500 further includes determining whether the movement of the anatomical element exceeds a threshold distance (step 520). Determining whether the movement of the anatomical element exceeds the threshold distance includes determining a distance difference between a reference marker and / or the target anatomical element in the first image data and the second image data. The distance difference can then be compared with a threshold distance. This distance difference can be determined automatically, for example, by a processor. In some embodiments, the distance difference can be determined by a user (such as, for example, a surgeon or other medical provider). Similarly, the threshold distance can be determined automatically by a processor or can be received as user input via, for example, a user interface.

[0096] Method 500 further includes updating the registration (step 524). Registration can be updated when the movement of the anatomical element exceeds a threshold distance as determined in step 520. Step 524 can be the same as or similar to step 512, except that second image data is received as input to the registration. Alternatively or additionally, the distance difference determined in step 520 can be received as input to the registration (whether or not second image data is available).

[0097] Method 500 further includes receiving third image data (step 528). Step 528 may be the same as or similar to step 504 described above. The third image data may depict at least a portion of the target anatomical element and reference markers. In some embodiments, the third image data is acquired at a time interval following the first and second image data. For example, the first image data may be acquired near the start or initiation of the surgical procedure, the second image data may be acquired before the first surgical step, and the third image data may be acquired before the second surgical step. It should be understood that the first, second, and third image data may be acquired at any time.

[0098] Method 500 further includes determining whether the movement of the anatomical element exceeds a threshold distance (step 532). Step 532 may be the same as or similar to step 520 described above.

[0099] Method 500 also includes updating the registration (step 536). Step 536 may be the same as or similar to step 524 described above.

[0100] It should be understood that method 500 may not include some of the steps described above, or these steps may be performed in any order. For example, in some embodiments, method 500 may not include steps 528, 532, and 536. In other words, in some embodiments, registration may be updated once during surgical procedures based on movement detected in the first and second image data. In yet other embodiments, method 500 may include steps 524, 528, 532, and 536. In other words, it may be determined that there is no movement, and therefore registration may not be updated.

[0101] This disclosure covers embodiments of method 500 that include more or fewer steps than those described above and / or one or more steps that are different from those described above.

[0102] As noted above, this disclosure covers those with less than Figure 5 The method includes all steps identified in the diagram (and the corresponding description of method 500), as well as those exceeding... Figure 5The method includes additional steps of those steps identified in (and the corresponding description of method 500). This disclosure also covers methods that include one or more steps from one method described herein and one or more steps from another method described herein. Any correlation described herein may be or includes registration or any other correlation.

[0103] The following provides embodiments and alternatives disclosed herein.

[0104] Example 1. A system comprising: an imaging device operable to acquire image data; at least one reference marker, the at least one reference marker being biocompatible and visible in the image data, wherein the at least one reference marker is implanted near an anatomical element in a patient; a processor; and a memory storing data for processing by the processor, the data, when processed, causing the processor to: receive first image data depicting at least a portion of the anatomical element and the at least one reference marker; perform a registration process based on the at least one reference marker; receive second image data depicting the at least a portion of the anatomical element and the at least one reference marker; determine, based on the at least one reference marker, whether movement of the anatomical element exceeds a threshold distance; and, in response to determining that the movement exceeds the threshold distance, update the registration of the anatomical element with a navigation system based on the at least one reference marker.

[0105] Example 2. The system according to Example 1, wherein the at least one reference marker comprises at least one of a liquid or a gel.

[0106] Example 3. The system according to Example 1 or 2, wherein the imaging device includes an ultrasound imaging device.

[0107] Example 4. The system according to any one of Examples 1 to 3, wherein the at least one reference marker comprises at least three reference markers.

[0108] Example 5. The system according to any one of Examples 1 to 4, the system further includes a robotic arm configured to locate and orient the at least one reference marker in the vicinity of the anatomical element.

[0109] Example 6. The system according to any one of Examples 1 to 5, wherein the anatomical element comprises one or more vertebrae.

[0110] Example 7. The system according to any one of Examples 1 to 6, wherein the at least one reference marker is permanently implanted in the patient.

[0111] Example 8. The system according to any one of Examples 1 to 7, wherein the imaging device includes a first imaging device using ionizing radiation and a second imaging device using non-ionizing radiation, and wherein the first image data is received from the first imaging device and the second image data is received from the second imaging device.

[0112] Example 9. The system according to Example 8, wherein the imaging device uses non-ionizing radiation, and wherein the first image data and the second image data are received from the imaging device.

[0113] Example 10. The system according to Example 9, wherein the memory stores additional instructions for execution by the at least one processor, the additional instructions, when executed, further causing the at least one processor to: receive third image data depicting at least a portion of the anatomical element and the at least one reference marker; determine, based on the at least one reference marker, whether the movement of the anatomical element exceeds the threshold distance; and, in response to determining that the movement exceeds the threshold distance, update the registration of the anatomical element with the navigation system based on the at least one reference marker.

[0114] Example 11. The system according to Example 10, wherein the third image data is received from the second imaging device.

[0115] Example 12. The system according to any one of Examples 1 to 11, wherein the memory stores additional instructions for execution by the at least one processor, the additional instructions, when executed, further causing the at least one processor to: use image processing to determine the pose of the at least one reference marker, wherein the registration is based on the pose of the at least one reference marker.

[0116] Example 13. A system comprising: an imaging device operable to acquire image data, wherein the imaging device uses non-ionizing radiation; at least one reference marker, the at least one reference marker being biocompatible and visible in the image data, wherein the at least one reference marker is implanted near a target anatomical element in a patient; a processor; and a memory storing data for processing by the processor, the data causing the processor, when processed, to: receive first image data depicting at least a portion of the target anatomical element and the at least one reference marker; determine the pose of the at least one reference marker using image processing; perform a registration process based on the pose of the at least one reference marker; receive second image data depicting the at least a portion of the target anatomical element and the at least one reference marker; determine, based on the at least one reference marker, whether movement of the target anatomical element exceeds a threshold distance; and, in response to determining that the movement exceeds the threshold distance, update the registration of the target anatomical element with a navigation system based on the at least one reference marker.

[0117] Example 14. The system according to Example 13, wherein the at least one reference marker is permanently implanted in the patient.

[0118] Example 15. The system according to Example 13 or 14, wherein the at least one reference marker comprises at least one of a liquid or a gel.

[0119] Example 16. The system according to any one of Examples 13 to 15, wherein the imaging device includes an ultrasound imaging device.

[0120] Example 17. A method comprising: receiving from an imaging device first image data depicting at least a portion of a target anatomical element and at least one reference marker, the at least one reference marker being implanted near the target anatomical element; using image processing to determine the pose of the at least one reference marker; performing a registration process based on the pose of the at least one reference marker; receiving second image data depicting the at least a portion of the target anatomical element and the at least one reference marker; determining, based on the at least one reference marker, whether movement of the target anatomical element exceeds a threshold distance; and in response to determining that the movement exceeds the threshold distance, updating the registration of the target anatomical element with a navigation system based on the at least one reference marker.

[0121] Example 18. The method according to Example 17, the method further comprising: receiving third image data depicting at least a portion of the target anatomical element and the at least one reference marker; determining, based on the at least one reference marker, whether the movement of the anatomical element exceeds the threshold distance; and, in response to determining that the movement exceeds the threshold distance, updating the registration of the anatomical element with the navigation system based on the at least one reference marker.

[0122] Example 19. The method according to Example 17 or 18, wherein the at least one reference marker comprises at least one of a liquid or a gel.

[0123] Example 20. The method according to any one of Examples 17 to 19, wherein the imaging device includes an ultrasound imaging device.

[0124] The foregoing is not intended to limit this disclosure to the one or more forms disclosed herein. In the foregoing detailed description, for example, for the purpose of simplification, various features of this disclosure are grouped together in one or more aspects, embodiments, and / or configurations. Features of aspects, embodiments, and / or configurations of this disclosure may be combined in alternative aspects, embodiments, and / or configurations other than those discussed above. The approach of this disclosure should not be construed as reflecting an intention that the claims require more features than expressly recited in each claim. Rather, as reflected in the following claims, aspects of the invention lie in fewer than all the features of a single foregoing aspect, embodiment, and / or configuration. Therefore, the following claims are hereby incorporated into this detailed description, wherein each claim exists independently as a separate preferred embodiment of this disclosure.

[0125] Furthermore, while the foregoing has already included descriptions of one or more aspects, embodiments, and / or configurations, as well as certain variations and modifications, other variations, combinations, and modifications may be made within the scope of this disclosure, for example, within the skill and knowledge of those skilled in the art, upon understanding of this disclosure. It is intended to obtain, to the permissible extent, rights including alternative aspects, embodiments, and / or configurations, including alternative, interchangeable, and / or equivalent structures, functions, scopes, or steps of those claimed, regardless of whether such alternative, interchangeable, and / or equivalent structures, functions, scopes, or steps are disclosed herein, and not to disclose for use in any patentable subject matter.

Claims

1. A system comprising: An imaging device (112) is operable to acquire image data (206). At least one reference marker (126) is biocompatible and visible in the image data, wherein the at least one reference marker is implanted near an anatomical element (210) in the patient's body; Processor (104); and Memory (106), the memory storing data for processing by the processor, the data causing the processor to: Receive first image data depicting at least a portion of the anatomical element and the at least one reference marker; The registration process (124) is performed based on the at least one reference marker. Receive second image data depicting at least a portion of the anatomical element and at least one reference marker; Based on the at least one reference marker, determine whether the movement of the anatomical element exceeds a threshold distance; as well as In response to determining that the movement exceeds the threshold distance, the registration of the anatomical element with the navigation system is updated based on the at least one reference marker.

2. The system of claim 1, wherein the at least one reference marker comprises at least one of a liquid or a gel.

3. The system according to claim 1 or 2, wherein the imaging device includes an ultrasound imaging device.

4. The system according to any one of the preceding claims, wherein the at least one reference marker comprises at least three reference markers.

5. The system according to any one of the preceding claims, further comprising a robotic arm (116) configured to locate and orient the at least one reference marker in the vicinity of the anatomical element.

6. The system according to any one of the preceding claims, wherein the anatomical element comprises one or more vertebrae.

7. The system according to any one of the preceding claims, wherein the at least one reference marker is permanently implanted in the patient.

8. The system according to any one of the preceding claims, wherein the imaging device comprises a first imaging device using ionizing radiation and a second imaging device using non-ionizing radiation, and wherein the first image data is received from the first imaging device and the second image data is received from the second imaging device.

9. The system of claim 8, wherein the imaging device uses non-ionizing radiation, and wherein the first image data and the second image data are received from the imaging device.

10. The system of claim 9, wherein the memory stores additional instructions for execution by the at least one processor, the additional instructions further causing the at least one processor, when executed, to: Receive third image data depicting at least a portion of the anatomical element and at least one reference marker; Based on the at least one reference marker, determine whether the movement of the anatomical element exceeds the threshold distance; as well as In response to determining that the movement exceeds the threshold distance, the registration of the anatomical element with the navigation system is updated based on the at least one reference marker.

11. The system of claim 10, wherein the third image data is received from the second imaging device.

12. The system according to any one of the preceding claims, wherein the memory stores additional instructions for execution by the at least one processor, the additional instructions further causing the at least one processor, when executed, to: Image processing is used to determine the pose of the at least one reference marker. The registration is based on the pose of the at least one reference marker.

13. A method, the method comprising: First image data is received from the imaging device (112) depicting at least a portion of a target anatomical element (210) and at least one reference marker (126) implanted near the target anatomical element; Image processing is used to determine the pose of the at least one reference marker; The registration process (124) is performed based on the pose of the at least one reference marker. Receive second image data depicting at least a portion of the target anatomical element and at least one reference marker; Based on the at least one reference marker, determine whether the movement of the target anatomical element exceeds a threshold distance; as well as In response to determining that the movement exceeds the threshold distance, the registration of the target anatomical element with the navigation system is updated based on the at least one reference marker.

14. The method according to claim 13, further comprising: Receive third image data depicting at least a portion of the target anatomical element and at least one reference marker; Based on the at least one reference marker, determine whether the movement of the anatomical element exceeds the threshold distance; as well as In response to determining that the movement exceeds the threshold distance, the registration of the anatomical element with the navigation system is updated based on the at least one reference marker.

15. The method of claim 13 or 14, wherein the at least one reference marker comprises at least one of a liquid or a gel.

Images