System and method for single image registration update

Abstract

A method includes receiving information regarding a pose of each of a plurality of fiducials positioned on or within a patient, causing an imaging device to generate a single image of a portion of the patient, the single image depicting at least a portion of each of the plurality of fiducials, determining a pose of one or more anatomical components represented in the image based on the information and the image, and comparing the determined pose of the one or more anatomical components to a predetermined pose of the one or more anatomical components.

Description

Technical Field

[0001] This technique generally involves surgical registration, and more specifically involves capturing a single image for registration. Background Technology

[0002] Images can be used for registration during surgical procedures. Images can be acquired before and / or during surgery. Images can be taken throughout the surgical procedure and may require the use of imaging devices during the procedure. Summary of the Invention

[0003] Examples of this disclosure include:

[0004] A method according to at least one embodiment of the present disclosure includes receiving information about the posture of each of a plurality of references positioned on or within a patient; causing an imaging device to generate a single image of a portion of the patient, the single image depicting at least a portion of each of the plurality of references; determining, based on the information and the image, the posture of one or more anatomical components represented in the image; and comparing the determined posture of the one or more anatomical components with a predetermined posture of the one or more anatomical components.

[0005] Any aspect of this document further includes updating a pre-existing registration using the determined pose.

[0006] Any aspect of this document further includes comparing the determined posture with the target posture of one or more anatomical components in the surgical plan.

[0007] Any aspect of this document further includes determining the position of the imaging device relative to the plurality of references based on the use of a phantom.

[0008] In any aspect of this document, the plurality of references includes one or more screws.

[0009] In any aspect of this article, the multiple benchmarks mentioned include multiple different types of implants.

[0010] Any aspect of this document further includes quantifying the change in posture of at least one of the one or more anatomical components from the predetermined posture to the determined posture.

[0011] In any aspect of this document, the image further depicts a device held by a robot, and the method further includes locating the determined pose in a coordinate space corresponding to the robot based on the image.

[0012] In any aspect of this document, the information described herein includes a computer-aided design (CAD) model of at least one of the plurality of benchmarks.

[0013] A system according to at least one embodiment of the present disclosure includes: a processor; and a memory storing instructions for execution by the processor, the instructions, when executed by the processor, causing the processor to receive information regarding the shape and orientation of a plurality of tracking devices placed on or within a patient; causing an imaging device to generate a single image of a portion of the patient, the single image including at least a portion of each of the plurality of tracking devices; identifying the portion of each of the plurality of tracking devices in the single image; calculating the orientation of one or more anatomical components represented in the image based on the information and the identified portion of each of the plurality of tracking devices; and quantifying the difference between the calculated orientation of at least one of the one or more anatomical components and a predetermined orientation.

[0014] In any aspect of this document, the imaging device is fixed in a first position relative to the patient.

[0015] In any aspect of this document, the first surface of each of the portions of the plurality of tracking devices in the single image appears at an angle greater than ten degrees and less than eighty degrees relative to the imaging device.

[0016] In any aspect of this article, the single image described is generated using fluorescence perspectration.

[0017] In any aspect of this document, the plurality of tracking devices comprise a plurality of different types of implants.

[0018] In any aspect of this document, at least one of the plurality of tracking devices is a screw implanted in the patient.

[0019] In any aspect of this document, the instructions further include instructions that cause the processor to compare the calculated pose of the one or more anatomical components with a predetermined pose of the one or more anatomical components.

[0020] In any aspect of this document, the plurality of tracking devices includes one or more screws.

[0021] In any aspect of this article, each of the one or more anatomical components mentioned is a vertebra.

[0022] A system according to at least one embodiment of the present disclosure includes: a processor; an imaging device; a plurality of references disposed on or within a patient; and a memory storing instructions for execution by the processor, the instructions, when executed by the processor, causing the processor to receive information about the pose of the plurality of references; causing the imaging device to generate an image of a portion of the patient, the image including the plurality of references; determining, based on the information and the image, the pose of one or more anatomical tissues represented in the image; and causing the determined pose to be displayed on a user interface.

[0023] In any aspect of this document, the determined pose is displayed on the user interface as an overlay of the target pose.

[0024] Details of one or more aspects of this disclosure are set forth in the accompanying drawings and description below. Other features, objectives, and advantages of the technology described in this disclosure will be apparent from the description and drawings and from the claims.

[0025] The phrases “at least one,” “one or more,” and “and / or” are open-ended expressions that possess both connective and disjoint qualities in operation. For example, the expressions “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” and “A, B, and / or C” mean only A, only B, only C, A and B together, A and C together, B and C together, or A, B, and C together. When each of A, B, and C in the above expressions refers to an element such as X, Y, and Z, or such as X1-X… n Y1-Y m and Z1-Z o When referring to a single class of elements, the phrase means a single element selected from X, Y, and Z; a combination of elements selected from the same class (e.g., X1 and X2); or elements selected from two or more classes (e.g., Y1 and Z). o () combination.

[0026] The term "a / an" refers to one or more of the entities mentioned. Thus, the terms "a / an," "one or more," and "at least one" are used interchangeably herein. It should also be noted that the terms "comprising," "including," and "having" are used interchangeably.

[0027] The foregoing is a simplified overview of this disclosure to provide an understanding of some aspects thereof. This summary is neither a broad nor an exhaustive overview of this disclosure and its various aspects, embodiments, and configurations. It is not intended to identify key or essential elements of this disclosure, nor to depict its scope, but rather to present selected concepts in a simplified form as an introduction to the more detailed description presented below. It should be understood that other aspects, embodiments, and configurations of this disclosure may utilize one or more of the features set forth above or described in detail below, individually or in combination.

[0028] Many additional features and advantages of the invention will become apparent to those skilled in the art upon consideration of the embodiments described below. Attached Figure Description

[0029] The accompanying drawings are incorporated in and form part of this specification to illustrate several examples of this disclosure. These drawings, together with the description, explain the principles of this disclosure. The drawings simply illustrate preferred and alternative examples of how to carry out and use this disclosure, and should not be construed as limiting this disclosure to only the examples shown and described. Further features and advantages will become apparent from the following more detailed description of various aspects, embodiments, and configurations of this disclosure, as illustrated by the figures referenced below.

[0030] Figure 1 A block diagram of a system according to an embodiment of the present disclosure is shown;

[0031] Figure 2 A flowchart of a method according to an embodiment of the present disclosure is shown;

[0032] Figure 3 Additional flowcharts are provided for methods according to embodiments of this disclosure; and

[0033] Figure 4 This is another flowchart of a method according to an embodiment of the present disclosure. Detailed Implementation

[0034] It should be understood that the various aspects disclosed herein can be combined in combinations different from those specifically presented in the description and figures. It should also be understood that, depending on the instance or embodiment, certain actions or events of any of the processes or methods described herein may be performed in a different order, and / or may be added, combined, or completely omitted (e.g., according to different embodiments of this disclosure, performing the disclosed technology may not require all the described actions or events). Furthermore, for clarity, although some aspects of this disclosure are described as being performed by a single module or unit, it should be understood that the technology of this disclosure can be performed by a combination of units or modules associated with, for example, computing devices and / or medical devices.

[0035] In one or more instances, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, functionality may be stored on a computer-readable medium in the form of one or more instructions or code and may be executed by a hardware-based processing unit. The computer-readable medium may include non-transitory computer-readable media, as opposed to tangible media, such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0036] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general-purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000 series processors, Nvidia RTX 3000 series processors, AMD Radeon RX 5000 series processors, AMD Radeon RX 6000 series processors, or any other graphics processing units), application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the term "processor" can refer to any of the foregoing structures or any other physical structures suitable for embodiments of the described techniques. Furthermore, the techniques can be fully implemented within one or more circuit or logic elements.

[0037] Before explaining any embodiment of this disclosure in detail, it should be understood that this disclosure is not limited in its application to the construction details and component arrangements set forth in the following description or shown in the accompanying drawings. This disclosure is capable of having other embodiments and can be practiced or carried out in various ways. Likewise, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising,” “including,” or “having,” and variations thereof, is intended to cover items listed thereafter and their equivalents, as well as additional items. Furthermore, examples may be used to illustrate one or more aspects of this disclosure. Unless expressly stated otherwise, the use or listing of one or more examples (which may be indicated by “for example,” “by means of an example,” “e.g.,” “such as,” or similar language) is not intended to, and does not limit, the scope of this disclosure.

[0038] Intraoperative fluoroscopy registration with preoperative computed tomography (CT) typically involves capturing two fluoroscopic images. Each image exposes the patient—and possibly one or more attending physicians—to known harmful ionizing radiation. Furthermore, capturing two fluoroscopic images can be time-consuming and may introduce additional error margins, as the amount of error increases with the number of images captured. Surgeons may benefit from a reduction in the number of movements and execution time associated with capturing two images, while patients may benefit from a reduction in anesthesia time.

[0039] According to embodiments of this disclosure, a reference marker having a known item size is captured in an image. The reference marker can comprise any surgical accessory of known size and reflectivity. Embodiments of this disclosure also include a single, relevant angled C-arm lens capturing (e.g., photographing) the reference marker. In other words, the C-arm lens is angled, allowing the reference marker to be used to represent missing depth indications. Embodiments of this disclosure can be used to update existing, previously acquired registrations or to create new registrations.

[0040] Embodiments of this disclosure advantageously enable the generation or updating of registration using only a single image. Embodiments of this disclosure also reduce the workload required to capture additional images for registration, while simultaneously reducing the radiation dose experienced by the patient during surgery or surgical procedures. Embodiments of this disclosure also advantageously enable improved registration accuracy in surgeries or surgical procedures involving robots and / or robotic arms.

[0041] First turn Figure 1 This diagram illustrates a block diagram of a system 100 according to at least one embodiment of the present disclosure. System 100 can be used, for example, to acquire and process image data; to perform one or more methods described herein; to perform one or more algorithms described herein; and / or to facilitate surgery or surgical procedures. System 100 includes a computing device 102, one or more imaging devices 108, a robot 130, a navigation system 136, a database 144, and a cloud or other network 148. Nevertheless, systems according to other embodiments of the present disclosure may omit one or more components of system 100. For example, in some embodiments, system 100 may omit one or more components of computing device 102. Additionally, systems according to other embodiments of the present disclosure may arrange one or more components of system 100 differently (e.g., imaging device 108, robot 130 and / or components thereof, and / or navigation system 136 may include one or more components of computing device 102, and / or vice versa), and / or include additional components not shown.

[0042] The computing device 102 includes at least one processor 104, at least one communication interface 106, at least one user interface 112, and at least one memory 116. In other embodiments of this disclosure, the computing device may omit one or both of the communication interface 106 and / or the user interface 112.

[0043] At least one processor 104 of computing device 102 may be any processor identified or described herein or any similar processor. The at least one processor 104 may be configured to execute instructions 140 stored in the at least one memory 116, the instructions 140 causing the at least one processor 104 to perform one or more computational steps using or based on data received, for example, from imaging device 108, memory 116, robotic arm 130 and / or components thereof, navigation system 136, database 144 and / or cloud 148. Instructions 140 may also cause the at least one processor 104 to utilize one or more algorithms stored in memory 116. In some embodiments, the at least one processor 104 may be used to control one or more imaging devices 108, robotic arm 130 and / or components thereof (e.g., one or more robotic arms 132) and / or navigation system 136 during surgical procedures, including during imaging processes or other processes performed autonomously or semi-autonomously by robot 130 using navigation system 136.

[0044] The computing device 102 may also include at least one communication interface 106. The at least one communication interface 106 may be used to receive sensor data (e.g., from one or more imaging devices 108, robots 130 and / or navigation systems 136), surgical plans (e.g., surgical plan 134) or other planning data, or other information from external sources (e.g., database 144, cloud 148 and / or portable storage media (e.g., USB drives, DVDs, CDs)), and / or to more broadly transmit instructions, images or other information from at least one processor 104 and / or the computing device 102 to external systems or devices (e.g., another computing device 102, one or more imaging devices 108, robots 130, navigation systems 136, database 144, cloud 148 and / or portable storage media (e.g., USB drives, DVDs, CDs)). At least one communication interface 106 may include one or more wired interfaces (e.g., a USB port, an Ethernet port, a FireWire port) and / or one or more wireless interfaces (e.g., configured to transmit information via one or more wireless communication protocols such as 802.11a / b / g / n, Bluetooth, Bluetooth Low Energy, NFC, ZigBee, etc.). In some embodiments, at least one communication interface 106 may be used to enable device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time required to complete computationally intensive tasks or for any other reason.

[0045] At least one user interface 112 may be or include a keyboard, mouse, trackball, monitor, television, touchscreen, button, joystick, switch, lever, and / or any other means for receiving information from a user and / or providing information to a user of computing device 102. At least one user interface 112 may be used, for example, to receive user selections or other user input related to any step of any method described herein; to receive user selections or other user input regarding one or more configurable settings of computing device 102, one or more imaging devices 108, robot 130, navigation system 136, and / or any other component of system 100; to receive user selections or other user input regarding how and / or where data received, modified, and / or generated by computing device 102 is stored and / or transmitted; and / or to display information (e.g., text, images) and / or play sound to a user based on data received, modified, and / or generated by computing device 102. Although system 100 includes at least one user interface 112, system 100 may automatically (e.g., without any input through at least one user interface 112 or otherwise) perform one or more or all of the steps of any method described herein.

[0046] Although at least one user interface 112 is shown as part of computing device 102, in some embodiments, computing device 102 may utilize a user interface 112 that is separately housed from one or more other components of computing device 102. In some embodiments, user interface 112 may be located close to one or more other components of computing device 102, while in other embodiments, user interface 112 may be located away from one or more other components of computing device 102.

[0047] At least one memory 116 may be or include RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. At least one memory 116 may store information or data for performing any steps of, for example, methods 200, 300, and / or 400 described herein. At least one memory 116 may store, for example, instructions 140 and / or one or more algorithms. In some embodiments, memory 116 may also store one or more preoperative plans and / or other surgical plans (e.g., surgical plan 134); one or more images of one or more patients, specifically including anatomical features / components of one or more patients to whom one or more surgical procedures are to be performed; images and / or other data received from one or more imaging devices 108 (or any of the foregoing), robot 130 and / or navigation system 136 (including any components thereof), or other locations; and / or other useful information relating to this disclosure.

[0048] As described above, instruction 140 may be or include any instructions executed by at least one processor 104, which cause the at least one processor to perform one or more steps of any of the methods described herein. Instruction 140 may be or include instructions for determining the pose of one or more anatomical components (e.g., one or more vertebrae) and / or multiple references (e.g., tracking markers); instructions for comparing the determined pose of one or more anatomical components with a predetermined pose of one or more anatomical components; instructions for quantifying the difference between the determined pose of the one or more anatomical components and the predetermined pose; instructions for locating the determined pose in coordinate space; instructions for registering an image space to a patient space; instructions for manipulating a robot and / or a robotic arm, such as robot 130 and / or one or more robotic arms 132, to perform any or more of the methods described herein; or otherwise. Instruction 140 may additionally or alternatively enable at least one processor 104 and / or more generally computing device 102 to operate as a machine learning engine that receives data and outputs one or more thresholds, criteria, algorithms and / or other parameters that may be used during the intervertebral implantation procedure and / or during any other surgical procedure in which information obtained from intervertebral tools as described herein may be relevant, to increase the likelihood of a positive surgical outcome.

[0049] The previously mentioned one or more algorithms may be or include any algorithm for converting sensor data received from sensors (including imaging sensors comprising one or more imaging devices 108) and / or from meters into meaningful information (e.g., image space registration to patient space, spatial position information relative to a given coordinate system, calculated force values, pressure values, distance measurements). One or more algorithms may also be or include one or more image processing algorithms 120, one or more pose algorithms 122, one or more registration algorithms 124, one or more image update or comparison algorithms 126, and one or more registration update or comparison algorithms 128. One or more algorithms may be used to control one or more imaging devices 108, robot 130, and / or navigation system 136. One or more algorithms may also be or include algorithms for generating one or more recommendations for the surgeon or other users of system 100 based on information received from sensors and / or meters, and / or for modifying preoperative plans or other surgical plans (e.g., surgical plan 134) based on such information and / or evaluation of such information. In some embodiments, one or more algorithms may be or include machine learning algorithms for analyzing historical data (e.g., historical data stored in database 144).

[0050] Database 144 may store any information described herein as stored in memory 116, including instructions such as instruction 140 and / or algorithms such as one or more algorithms (e.g., one or more posture algorithms 122, etc.). In some embodiments, database 144 stores one or more preoperative plans or other surgical plans (e.g., surgical plan 134). Database 144 may additionally or alternatively store information such as information about or corresponding to one or more characteristics of one or more of one or more imaging devices 108, robot 130, and navigation system 136; and / or other information about available tools and / or equipment used in relation to surgical procedures. Database 144 may be configured to provide any such information to one or more imaging devices 108, robot 130, computing device 102, navigation system 136, or to any other device of or outside system 100, whether directly or via cloud 148. In some embodiments, database 144 may be or include part of a hospital image storage system, such as a Picture Archiving and Communication System (PACS), a Health Information System (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records containing image data. Similarly, in some embodiments, memory 116 may store any of the information described above.

[0051] Cloud 148 can be or represents the Internet or any other wide area network. Computing device 102 can be connected to cloud 148 via communication interface 106 using a wired connection, a wireless connection, or both. In some embodiments, computing device 102 can communicate with database 144 and / or external devices (e.g., computing devices) via cloud 148.

[0052] Navigation system 136 can provide navigation for the surgeon and / or robot 130 and / or its components (e.g., one or more robotic arms 132) during surgery or surgical procedures. Navigation system 136 can be any navigation system now known or developed in the future, including, for example, Medtronic StealthStation. TMS8 Surgical Navigation System. Navigation system 136 may include a camera or other sensors for detecting and / or tracking one or more reference markers, navigation trackers, or other objects in the operating room or other rooms where surgery is performed. In some embodiments, navigation system 136 may include multiple sensors. In various embodiments, navigation system 136 may be used to track the position of one or more imaging devices 108, robot 130, and / or one or more other objects for which navigation system 136 has a line of sight (where the navigation system is an optical system) or which objects can be otherwise detected by navigation system 136. Navigation system 136 may be used to track the position of one or more reference markers or arrays or other structures that can be detected by the camera or other sensors of navigation system 136. Navigation system 136 may include a display for displaying one or more images from an external source (e.g., computing device 102, cloud 148, or other source) or video streams from a navigation camera or from one or more imaging devices 108 or from another sensor. In some embodiments, system 100 may operate without using navigation system 136.

[0053] Imaging device 108 is configured to capture, store, and / or transmit images and / or image data (e.g., image metadata, pixel data, etc.) between various components of system 100 (e.g., to robot 130, navigation system 136, computing device 102, any combination thereof, etc.). Imaging device 108 may include one or more sensors that assist system 100 in determining the position and orientation (e.g., pose) of imaging device 108. In some embodiments, system 100 may determine the position and orientation of imaging device 108 relative to one or more other components in system 100 (e.g., robot 130). Determining the position and orientation of imaging device 108 can assist system 100 when processing data associated with images captured by imaging device 108. In various instances, the image data captured by imaging device 108 may include data corresponding to anatomical features of a patient or a portion thereof. Imaging device 108 may be or include, for example, an ultrasound scanner, an O-arm, a C-arm, a G-arm, or any other device utilizing X-ray-based imaging (e.g., a fluorescence microscope, a CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, a thermal imaging camera (e.g., an infrared camera), or any other imaging device capable of acquiring images of a patient's anatomical features. In some embodiments, imaging device 108 may include additional or alternative connection components (e.g., a phantom) that facilitate the capture and / or processing of images captured by imaging device 108. The connection components may filter and / or process received light that is not orthogonal to imaging device 108 (e.g., the sides and / or boundaries of an image associated with a non-planar image) to provide image data consistent with a planar image (e.g., all portions of the image are orthogonal to imaging device 108).

[0054] In some embodiments, one or more images captured by imaging device 108 can be used to verify registration for a surgery or surgical procedure (e.g., transforming different datasets of data associated with the captured images into a single coordinate system, or the correlation between one coordinate system or space and another). For example, a surgery or surgical procedure may include registering the coordinate system of a robot and / or robotic arm (e.g., robotic arm 132) to the patient's coordinate system. In some embodiments, the coordinate system or space of a navigation system may be additionally or alternatively registered to the robot coordinate system and / or the patient coordinate system. Registration thereafter allows the robot to move to a specific position relative to the patient. However, if the position of one or more of the patient, robot, and / or navigation system changes relative to any other one or more of the patient, robot, and / or navigation system, the registration may become invalid. Therefore, images from imaging device 108 can be used to determine whether the registered entities are still in the same position relative to each other.

[0055] Images captured by imaging device 108 can also be used to update registration or perform additional registration, whether due to patient movement relative to the robot or vice versa, or for any other reason. System 100 and / or its components (e.g., computing device 102) can then use the updated or additional forward registration.

[0056] Robot 130 can be any surgical robot or surgical robot system. Robot 130 can be, or includes, for example, Mazor X. TM Stealth robot guidance system. Robot 130 may include one or more robotic arms 132. In some embodiments, the robotic arms 132 may assist in surgical procedures (e.g., by holding the tool in a desired trajectory or posture, by supporting the weight of the tool while the surgeon or other user operates the tool, by moving the tool to a specific posture under the control of the surgeon or other user, and / or otherwise) and / or perform surgical procedures automatically.

[0057] The robotic arm 132 may have one, two, three, four, five, six, seven, or more degrees of freedom. The robotic arm 132 may include one or more segments. Each segment may be attached to at least one adjacent member via a joint, such that the robotic arm 132 is hinged. The joint may be any type of joint capable of selectively moving the member relative to the structure to which the joint is attached (e.g., another segment of the robotic arm). For example, the joint may be a pivot joint, a hinged joint, a saddle joint, or a ball joint. The joint may allow the member to move in one or more dimensions and / or along one or more axes. While the proximal end of the robotic arm 132 may be attached to a base (whether via a joint or otherwise), the distal end of the robotic arm 132 may support an end effector. The end effector may be, for example, a tool (e.g., a drill, saw, imaging device) or a tool guide (e.g., for guiding a biopsy needle, ablation probe, or other tool along a desired trajectory).

[0058] The robot arm 132 may include one or more posture sensors. The posture sensors may be configured to detect the posture of the robot arm or portions thereof, and may be or include one or more rotary encoders, linear encoders, incremental encoders, or other sensors. Data from the posture sensors may be provided to the processor of the robot arm 132, the processor 104 of the computing device 102, and / or the navigation system 136. The data may be used to calculate the spatial position of the robot arm 132 relative to a predetermined coordinate system. This calculated position may be used, for example, to determine the spatial position of one or more of a plurality of sensors attached to the robot arm 132. Additionally and / or alternatively, one or more tracking markers may be fixed or otherwise attached to the robot arm 132, and the navigation system 136 may utilize one or more tracking markers to determine the spatial position (e.g., relative to a navigation coordinate system) of the robot arm 132 and / or the end effector supported thereby.

[0059] Embodiments of this disclosure may include a system 100 having more than one robotic arm 132. For example, one or more robotic arms may be used to support an imaging device 108. As another example, multiple robotic arms may be used to hold different tools or medical devices, each of which may need to be used simultaneously to successfully perform a surgical procedure. For example, in some embodiments, the robot 130 and / or one or more robotic arms 132 may hold a device (e.g., a tracking device, a reference, etc.) in a fixed position relative to the robot 130 and / or the patient. An image of the held device may be captured by the imaging device 108, and one or more components of the system 100 (e.g., processor 104) may use the image to determine a pose in coordinate space. For example, during registration, the system 100 may use the captured image and the device displayed therein to register an image space (e.g., the coordinates of the image) to a coordinate space (e.g., the space in which the device is held). Additionally or alternatively, the system 100 may determine the pose of the device, which may be used for registration or for defining the coordinate space.

[0060] Now go to Figure 2This document illustrates a method 200 for determining the pose of a single image of one or more anatomical components according to embodiments of the present disclosure. Method 200 may be performed partially or entirely by, for example, a computing device 102 or similar means, and may utilize system 100 and / or one or more components thereof (e.g., processor 104, one or more imaging devices 108, navigation system 136, and / or combinations thereof). At least one processor for performing method 200 and / or one or more steps thereof may be the same as or similar to processor 104 of the aforementioned computing device 102. At least one processor may be part of a robot (e.g., robot 130 including one or more robotic arms 132) or a navigation system (e.g., navigation system 136). Processors other than any processors described herein may also be used to perform method 200. At least one processor may perform method 200 by executing instructions (e.g., instruction 140) stored in a memory (e.g., memory 116). One or more aspects of method 200 may be performed by a robot and / or a surgical robotic arm (e.g., robotic arm 132 attached to robot 130) and / or components thereof, or together with a robot and / or a surgical robotic arm and / or components thereof. Method 200 may be used to update registration, for example, based on the movement of one or more components of the patient and / or system 100 during, for example, surgery or surgical procedure. For example, the patient may have been displaced during surgery and may require new registration.

[0061] Method 200 includes receiving information regarding the size, shape, configuration, posture, and / or relative position of each of a plurality of references (step 204). The plurality of references may be positioned on or within the patient. The plurality of references may be or include one or more screws, rods, intervertebral bodies, or other medical devices implanted within the patient. The information may be received via a user interface such as user interface 112 and / or a communication interface (e.g., communication interface 106) and may be stored in memory (e.g., memory 116) by a computing device (e.g., computing device 102). In some embodiments, information may be received from one or more components of system 100, such as from database 144 and / or cloud 148 and / or from external components of system 100 (e.g., from other computing devices and / or databases). The information may include CAD (computer-aided design) models of one or more of the plurality of references. The information may correspond to the reflectivity of one or more references. The information may correspond to surgical planning and / or data from a robot used to perform the surgical planning and may include, for example, information regarding the posture (e.g., position and orientation) or location or orientation of a given reference relative to the patient's anatomy when inserted into the patient. The information may include additional data regarding multiple references (e.g., the number of references inserted into the patient, the relative timing of the placement of multiple references, etc.) and / or surgical procedures (e.g., the type of surgical procedure, the duration of the surgical procedure, etc.).

[0062] The received information may at least describe the size, shape, configuration, pose, and / or relative position of the references. The size of each of the plurality of references may be or include the dimensions of each reference (e.g., length, width, height, etc.). The shape of each of the plurality of references may be information about the outline, geometry, contour, surface, edges, etc., of the plurality of references. The pose of each of the plurality of references may be the position and orientation of each reference relative to the patient or in another specified coordinate space. The relative position of each reference may be a measurement of the relative distance of each reference to the remaining references of the plurality of references. In some embodiments, the information may be input into one or more algorithms (e.g., pose algorithm 122) to extract or determine additional information and / or transform the information into information useful to the system (e.g., information useful for or related to registration). In some embodiments, information about the plurality of references may be further stored / saved (e.g., in database 144, memory 116, or elsewhere) within system 100.

[0063] Method 200 further includes generating a single image of a portion of the patient (step 208). For example, computing device 102 (and more specifically, processor 104) may cause imaging device (e.g., imaging device 108) to generate (e.g., capture) an image of a portion of the patient, the image including one or more of a plurality of references. The single image may be, for example, a fluoroscopic image. In some embodiments, the processor may cause the imaging device to move relative to the patient in a first pose (e.g., position and orientation) such that the imaging device can observe each of the plurality of references. In some embodiments, the first pose may be a position and orientation such that the imaging device sees each of the plurality of references and that the relative spacing and depth information of each of the plurality of references (e.g., the position of each reference in a coordinate system and how far each reference is from the camera relative to the coordinate system) can be determined based on the captured image and the information received in step 204. In some embodiments, the image may capture only portions of each of the plurality of references. For example, only a percentage of each of the plurality of references may be present in the image (e.g., 1%, 2%, 5%, 10%, 15%, 25%, 50%, 75%, 80%, 90%, etc.). In such embodiments, the percentage may be different for each of the multiple references. In some embodiments, the image may depict only a percentage of the total number of references (e.g., only half of the total number of references). In some embodiments, method 200 generates only a single image of the portion of the patient. The imaging device may transmit the captured image to one or more components within system 100 (e.g., to computing device 102, database 144, etc.) and / or to components outside system 100. Each of the multiple references may be present in the image and may be distinguishable from other artifacts or structures in the image. For example, in an embodiment where the multiple references include screws, imaging device 108 may emit radiation (e.g., X-rays), with a detector positioned such that the multiple references are between imaging device 108 and the detector. The resulting image may highlight (e.g., displayed with different colors, textures, intensities, and / or shapes) the multiple references located within the patient. As previously noted, the multiple references may be attached to or positioned near one or more anatomical components (e.g., internal organs, bones, ligaments, tendons, muscle structures, other soft tissues and / or hard tissues, etc.). In such embodiments, the capture of images depicting said portions of the patient can also depict the relative positions of one or more anatomical components to which the reference is attached or located proximally.

[0064] In some embodiments, one or more of a plurality of reference points may be placed on or within the patient prior to surgery or surgical procedures. For example, one or more of the plurality of reference points may be secured to the patient's skin (or even to the patient's skeletal anatomy, whether through a small incision or otherwise) before surgery begins. Also in some embodiments, one or more of the plurality of reference points may have been previously implanted in the patient (e.g., during initial surgery), allowing system 100 to capture (e.g., via imaging device 108) the relative layout of the plurality of reference points prior to subsequent procedures (e.g., remedial surgery). System 100 may use the captured information during surgery to compare the layout of the plurality of reference points with new images of the plurality of reference points captured during or after surgery or surgical procedures.

[0065] Furthermore, as described above, in some embodiments, one or more of a plurality of reference points may be implanted in the patient. For example, a surgery or surgical procedure may be performed within the patient. In such embodiments, one or more of the plurality of reference points may be implanted at specific locations on the patient (e.g., anatomical features, structures, etc.). This allows system 100 and / or its components to determine the patient's relative anatomy. The plurality of reference points can then be removed from the patient during and / or after the surgery or surgical procedure. The plurality of reference points is by no means limiting, and examples may include screws, tracking devices, radio frequency identification (RFID) tags, etc. Screws may be, for example, pedicle screws, cortical screws, cancellous bone screws, combinations thereof, and / or the like.

[0066] In some embodiments, multiple references can be positioned such that depth measurements can be obtained from the image. The depth measurement can be determined based on a first surface of the references being observed. For example, the first surface of the references may be angled relative to a first pose of the imaging device, such that the imaging device can see only an isometric (e.g., perspective) view of the references. In other words, the references are neither orthogonal (e.g., such that the imaging device only observes the top or bottom of the references and not the sides) nor lateral (e.g., such that the imaging device only observes the sides of the references and not the top or bottom). Instead, the view of the references seen by the imaging device (and subsequently appearing in the image) allows the first surface to be observed in addition to at least one of the top, bottom, and sides of the references. The appearance of at least two surfaces allows method 200 to determine the depth of the references (e.g., how far each point of the reference is from the imaging device relative to the coordinate system) using the information received in step 204 and by determining the angle of the references in the captured image.

[0067] Method 200 further includes determining the pose of one or more anatomical components in a single captured image (step 212). The determination may be based on information received in step 204 and the image generated in step 208. The determination may use one or more algorithms, such as pose algorithm 122. The pose determination may involve using coordinates and / or orientations of multiple references attached to or near one or more anatomical components. In some embodiments, the pose algorithm may be configured to calculate one or more poses based on multiple references captured in the image and information received in step 204. For example, the pose algorithm may determine that references are attached to or near an anatomical component and may use received information about the size, shape, configuration, pose, and / or relative position of the references, along with the image generated in step 208, to define the pose of the anatomical component. In some embodiments, the pose algorithm may output the defined pose relative to a predetermined coordinate system, the robot, and / or other components (e.g., other components of system 100). In some embodiments, the pose algorithm may be, uses, or is used by artificial intelligence (AI), machine learning (e.g., deep learning), and / or other solutions to identify, detect, or otherwise recognize the pose of a baseline and / or determine anatomical components. The determined pose may be sent to and / or stored in one or more components of system 100. Method 200 may additionally or alternatively utilize image processing algorithms (e.g., image processing algorithm 120). The image processing algorithm may receive captured images (e.g., from an imaging device) and may filter, process, or otherwise digitally manipulate individual images. Such image post-processing may improve image storage efficiency and / or improve image quality to more easily facilitate pose determination (e.g., by improving the quality of the images passed to the pose algorithm).

[0068] In some embodiments, the depth of each of a plurality of references can be determined. In embodiments where the imaging device captures views of at least two surfaces of a reference, relative depth can be determined by angle measurement in conjunction with the information received in step 204. For example, a pose algorithm can determine the relative angle of a reference in an image of a portion of the patient, and the pose of the reference can be calculated using angle and position information (e.g., information received about the relative position of the reference). A first surface of each reference can be the surface on which the pose algorithm measures the relative angle. The selection of the first surface is not limited, and the selection of the first surface can be arbitrary and can be additionally or alternatively based on the appearance of the reference in the image, based on predetermined instructions for specifying the first surface, etc. The angle of the first surface can appear in a single image at angles greater than 0 degrees and less than 90 degrees, greater than 10 degrees and less than 80 degrees, greater than 20 degrees and less than 70 degrees, etc. However, it should be understood that the above-described angle ranges are by no means limited to the embodiments of this disclosure, and additional or alternative angle ranges between 0 and 90 degrees are possible. Method 200 can use the determined reference positions to further determine the pose of one or more anatomical components. For example, in embodiments where multiple references are fixed to one or more anatomical components, the pose of the references can allow the processor to determine the relative pose of one or more anatomical components. In some embodiments, one or more of the multiple references may be attached to or near one or more anatomical components, which can allow the pose of one or more anatomical components to be determined based on the relative poses of the multiple references and the known distances between the multiple references and one or more anatomical components. In some embodiments, the pose of one or more anatomical components is compared to a target pose. For example, the target pose may be a predicted or expected pose of each of the one or more anatomical components. A comparison algorithm may compare the determined pose to the target pose to determine an accuracy level (e.g., how close the determined pose is to the expected pose). In some embodiments, the method may include sending a warning to a user interface (e.g., user interface 112) when the determined pose is outside a threshold accuracy level of the target pose.

[0069] Method 200 further includes comparing the determined pose of one or more anatomical components with a predetermined pose of one or more anatomical components (step 216). The predetermined pose of one or more anatomical components may correspond to a previous registration and / or may be provided by one or more components of, for example, system 100. In some embodiments, the predetermined pose may have been determined prior to surgery or surgical procedures. For example, the predetermined pose may include information (e.g., coordinates, images, combinations thereof) about where one or more anatomical components and / or multiple references should be located based on previously captured data (e.g., images of one or more anatomical components and / or multiple references acquired prior to surgery or surgical procedures or during previous registration). The predetermined pose may correspond to preoperative CT or MRI images. Method 200 may implement a comparison algorithm (e.g., image update or comparison algorithm 126) to compare the determined pose of one or more anatomical components with a predetermined pose determined at an earlier time. In some embodiments, the comparison algorithm may take into account a change in position of the patient from a first position where the predetermined pose was determined and a second position where the determined pose can be calculated. For example, the predetermined pose may be determined when the patient is in a supine position, and the determined pose may be based on the patient being in a prone position. The comparison algorithm can compare the determined posture with the predetermined posture in part based on the switch in the patient's position.

[0070] Method 200 further includes quantifying the difference between a determined pose of one or more anatomical components and a predetermined pose of one or more anatomical components (step 220). Quantification may utilize one or more algorithms (e.g., image update or comparison algorithm 126) to provide a quantity reflecting the difference between the determined pose and the predetermined pose. For example, the algorithm may receive the results of a comparison of the relative positions of one or more anatomical components in the determined pose and one or more anatomical components in the predetermined pose, and may quantify based on the compared positions. The quantified difference may be based on a percentage (e.g., 0.1% difference, 0.2% difference, 1% difference, 5% difference, 10% difference, etc.). In some embodiments, the percentage-based difference may be based on the percentage of the determined pose that matches (e.g., overlaps, aligns, etc.) with the predetermined pose. If a percentage-based difference is reported for all anatomical components, the percentage-based difference may use a weighted average, and the percentage-based difference for each of the one or more anatomical components may be reported additionally or alternatively. In some embodiments, a 100% difference may indicate that there is no pose overlap between the determined pose and the predetermined pose. In some embodiments, the quantified difference can be a positional or angular difference (e.g., 0.1 mm difference, 0.2 mm difference, 1 mm difference, 0.1 degree difference, 0.2 degree difference, 1 degree difference, etc.). Positional differences can be based on the relative coordinate changes between the determined pose and the predetermined pose in one or more directions (e.g., positional changes along the length, width, and / or height of one or more anatomical components). In some embodiments, the algorithm can be based on positional differences on captured images of one or more anatomical components. For example, the algorithm can receive captured images of one or more anatomical components, which can be designated as a predetermined pose, prior to the commencement of surgery or surgical procedure. Such images can be, for example, CT scans, MRI images, or other preoperative images. The images can be two-dimensional or three-dimensional. The algorithm can additionally receive a single image captured by an imaging device that indicates the determined pose. The single image can be, for example, a fluoroscopic image or an ultrasound image. The single image can be two-dimensional, but aspects of this disclosure can be used to infer and / or compute third-dimensional information (including, for example, computed depth information using an imaging reference). The algorithm can compare two images of one or more anatomical components and determine the percentage or positional change from the predetermined pose to the determined pose.

[0071] Method 200 further includes locating the determined pose in a coordinate system (step 224). Method 200 may utilize one or more algorithms (e.g., pose algorithm 122) to locate the determined pose in a coordinate system. For example, the algorithm may receive the determined pose of one or more anatomical components in a single image and may map the pose to a coordinate system in a coordinate space. In some embodiments, the coordinate space may be similar to or the same as the patient space. The mapping may be based on the position of the imaging device (e.g., imaging device 108) when the image is captured and may additionally or alternatively be based on the positioning of multiple references relative to one or more anatomical components. The pose of each of the one or more anatomical components in the coordinate space may be stored / saved (e.g., in database 144) and / or used by a robot (e.g., robot 130 and / or one or more robotic arms 132) for surgery or surgical procedures.

[0072] As can be understood based on the foregoing disclosure, this disclosure covers those with less than Figure 2 The methods for all steps identified in the code (and the corresponding description of method 200), and including all steps except... Figure 2 The method for additional steps beyond those steps identified in the diagram (and the corresponding description of method 200).

[0073] Go to Figure 3This document illustrates a method 300 for displaying a determined pose to a user interface according to embodiments of the present disclosure. Method 300 may be performed partially or entirely by, for example, computing device 102 or similar means, and may utilize system 100 and / or components thereof (e.g., processor 104, one or more imaging devices 108, navigation system 136, and / or combinations thereof). At least one processor for performing method 300 and / or one or more steps thereof may be the same as or similar to processor 104 of the aforementioned computing device 102. At least one processor may be part of a robot (e.g., robot 130 including one or more robotic arms 132), part of a navigation system (e.g., navigation system 136), or part of a user interface (e.g., user interface 112). Processors other than any processor described herein may also be used to perform method 300. At least one processor may perform method 300 by executing instructions (e.g., instruction 140) stored in a memory such as memory 116. One or more aspects of method 300 may be performed by or in conjunction with a robot and / or a surgical robotic arm (e.g., robotic arm 132 attached to robot 130) and / or its components. Method 300 may be used, for example, to update registration (whether due to movement of the patient and / or one or more components of system 100 during, for example, surgery or surgical procedure, or for other reasons). For example, the patient may have shifted during surgery, and new registration may be required to ensure that surgical devices and / or instruments (e.g., robot 130) are correctly positioned relative to the patient. Method 300 may additionally or alternatively be used to display the determined posture of one or more anatomical components on a user interface, for example, by a surgeon during surgery or surgical procedure.

[0074] Method 300 includes receiving information regarding the size, shape, configuration, posture, and / or relative position of each of a plurality of references (positioned on or within the patient) (step 304). The plurality of references may be or include one or more screws, rods, intervertebral bodies, or other medical devices implanted within the patient. The information may be received via a user interface such as user interface 112 and / or a communication interface (e.g., communication interface 106) and may be stored in memory (e.g., memory 116) by a computing device (e.g., computing device 102). In some embodiments, the information may be received from one or more components of system 100, such as from database 144 and / or cloud 148 and / or from external components of system 100 (e.g., from other computing devices and / or databases). The information may include CAD (computer-aided design) models of one or more of the plurality of references. The information may correspond to the reflectivity of one or more references. The information may correspond to surgical planning and / or data from a robot used to perform the surgical planning and may include, for example, information regarding the posture (e.g., position and orientation) or location or orientation of a given reference relative to the patient's anatomy when inserted into the patient. The information may include additional data regarding multiple references (e.g., the number of references inserted into the patient, the relative timing of the placement of the multiple references, etc.) and / or surgical procedures (e.g., the type of surgical procedure, the duration of the surgical procedure, etc.). In some embodiments, the information may include CT, MRI, or other preoperative images depicting one or more of the multiple references.

[0075] The received information may at least describe the size, shape, configuration, pose, and / or relative position of the references. The size of each of the plurality of references may be or include the dimensions of each reference (e.g., length, width, height, etc.). The shape of each of the plurality of references may be information about the outline, geometry, contour, surface, edges, etc., of the plurality of references. The pose of each of the plurality of references may be the position and orientation of each reference relative to the patient or in another specified coordinate space. The relative position of each reference may be a measurement of the relative distance of each reference to the remaining references of the plurality of references. In some embodiments, the information may be input into one or more algorithms (e.g., pose algorithm 122) to extract or determine additional information and / or transform the information into information useful to the system (e.g., information useful for or related to registration). In some embodiments, information about the plurality of references may be further stored / saved (e.g., in database 144, memory 116, or elsewhere) within system 100.

[0076] Method 300 further includes generating a single image of a portion of the patient (step 308). For example, computing device 102 (and more specifically, processor 104) may cause imaging device (e.g., imaging device 108) to generate (e.g., capture) an image of a portion of the patient, the image including one or more of a plurality of references. The single image may be, for example, a fluorescence fluoroscopic image. In some embodiments, the processor may cause the imaging device to move relative to the patient in a first pose (e.g., position and orientation) such that the imaging device can observe each of the plurality of references. In some embodiments, the first pose may be a position and orientation such that the imaging device sees each of the plurality of references, and that the relative spacing and depth information of each of the plurality of references (e.g., the position of each reference in a coordinate system and how far each reference is from the camera relative to the coordinate system) can be determined based on the captured image and the information received in step 304. In some embodiments, the image may capture only portions of each of the plurality of references. For example, only a percentage of each of the plurality of references may be present in the image (e.g., 1%, 2%, 5%, 10%, 15%, 25%, 50%, 75%, 80%, 90%, etc.). In such embodiments, the percentage may be different for each of the multiple references. In some embodiments, the image may depict only a percentage of the total number of references (e.g., only half of the total number of references). In some embodiments, method 300 generates only a single image of the portion of the patient. The imaging device may transmit the captured image to one or more components within system 100 (e.g., to computing device 102, database 144, etc.) and / or to components outside system 100. Each of the multiple references may be present in the image and may be distinguishable from other artifacts or structures in the image. For example, in an embodiment where the multiple references include screws, imaging device 108 may emit radiation (e.g., X-rays), wherein a detector is positioned such that the multiple references are between imaging device 108 and the detector. The resulting image may highlight (e.g., displayed with different colors, textures, intensities, and / or shapes) the multiple references located within the patient. As previously noted, the multiple references may be attached to or positioned near one or more anatomical components (e.g., internal organs, bones, ligaments, tendons, muscle structures, other soft tissues and / or hard tissues, etc.). In such embodiments, the capture of images depicting said portions of the patient can also depict the relative positions of one or more anatomical components to which the reference is attached or located proximally.

[0077] In some embodiments, one or more of a plurality of reference points may be placed on or within the patient prior to surgery or surgical procedures. For example, one or more of the plurality of reference points may be secured to the patient's skin (or even to the patient's skeletal anatomy, whether through a small incision or otherwise) before surgery begins. Also in some embodiments, one or more of the plurality of reference points may have been previously implanted in the patient (e.g., during initial surgery), allowing system 100 to capture (e.g., via imaging device 108) the relative layout of the plurality of reference points prior to subsequent procedures (e.g., remedial surgery). System 100 may use the captured information during surgery to compare the layout of the plurality of reference points with new images of the plurality of reference points captured during or after surgery or surgical procedures.

[0078] Furthermore, as described above, in some embodiments, one or more of a plurality of reference points may be implanted in the patient. For example, a surgery or surgical procedure may be performed within the patient. In such embodiments, one or more of the plurality of reference points may be implanted at specific locations on the patient (e.g., anatomical features, structures, etc.). This allows system 100 and / or its components to determine the patient's relative anatomy. The plurality of reference points can then be removed from the patient during and / or after the surgery or surgical procedure. The plurality of reference points is by no means limiting, and examples may include screws, tracking devices, radio frequency identification (RFID) tags, etc. Screws may be, for example, pedicle screws, cortical screws, cancellous bone screws, combinations thereof, and / or the like.

[0079] In some embodiments, multiple references can be positioned such that depth measurements can be obtained from the image. The depth measurement can be determined based on a first surface of the references being observed. For example, the first surface of the references may be angled relative to a first pose of the imaging device, such that the imaging device can see only an isometric (e.g., perspective) view of the references. In other words, the references are neither orthogonal (e.g., such that the imaging device only observes the top or bottom of the references and not the sides) nor lateral (e.g., such that the imaging device only observes the sides of the references and not the top or bottom). Instead, the view of the references seen by the imaging device (and subsequently appearing in the image) allows the first surface to be observed in addition to at least one of the top, bottom, and sides of the references. The appearance of at least two surfaces allows method 300 to determine the depth of the references (e.g., how far each point of the reference is from the imaging device relative to the coordinate system) using the information received in step 304 and by determining the angle of the references in the captured image.

[0080] Method 300 further includes determining the pose of one or more anatomical components in a single captured image (step 312). The determination may be based on information received in step 304 and the image generated in step 308. The determination may use one or more algorithms, such as pose algorithm 122. The pose determination may involve using coordinates and / or orientations of multiple references attached to or near one or more anatomical components. In some embodiments, the pose algorithm may be configured to calculate one or more poses based on multiple references captured in the image and information received in step 304. For example, the pose algorithm may determine that references are attached to or near an anatomical component and may use received information about the size, shape, configuration, pose, and / or relative position of the references, along with the image generated in step 308, to define the pose of the anatomical component. In some embodiments, the pose algorithm may output the defined pose relative to a predetermined coordinate system, robot, and / or other components (e.g., other components of system 100). In some embodiments, the pose algorithm may be, or use, artificial intelligence (AI), machine learning (e.g., deep learning), and / or other solutions to identify, detect, or otherwise recognize the references. The determined pose may be sent to and / or stored in one or more components of system 100. Method 300 may additionally or alternatively utilize an image processing algorithm (e.g., image processing algorithm 120). The image processing algorithm may receive captured images (e.g., from an imaging device) and may filter, process, or otherwise digitally manipulate individual images. Such image post-processing may improve image storage efficiency and / or may improve image quality to more easily facilitate pose determination (e.g., by improving the quality of the image passed to the pose algorithm).

[0081] In some embodiments, the depth of each of a plurality of references can be determined. In embodiments where the imaging device captures views of at least two surfaces of a reference, relative depth can be determined by angle measurement in conjunction with the information received in step 304. For example, a pose algorithm can determine the relative angle of a reference in an image of a portion of the patient, and the pose of the reference can be calculated using angle and position information (e.g., information received about the relative position of the reference). A first surface of each reference can be the surface on which the pose algorithm measures the relative angle. The selection of the first surface is not limited, and the selection of the first surface can be arbitrary and can be additionally or alternatively based on the appearance of the reference in the image, based on predetermined instructions for specifying the first surface, etc. The angle of the first surface can appear in a single image at angles greater than 0 degrees and less than 90 degrees, greater than 10 degrees and less than 80 degrees, greater than 20 degrees and less than 70 degrees, etc. However, it should be understood that the above angle ranges are by no means limited to the embodiments of this disclosure, and additional or alternative angle ranges between 0 and 90 degrees are possible. Method 300 can use the determined reference positions to further determine the pose of one or more anatomical components. For example, in embodiments where multiple references are fixed to one or more anatomical components, the pose of the references can allow the processor to determine the relative pose of one or more anatomical components. In some embodiments, one or more of the multiple references may be attached to or near one or more anatomical components, which can allow the pose of one or more anatomical components to be determined based on the relative poses of the multiple references and the known distances between the multiple references and one or more anatomical components. In some embodiments, the pose of one or more anatomical components is compared to a target pose. For example, the target pose may be a predicted or expected pose of each of the one or more anatomical components. A comparison algorithm may compare the determined pose to the target pose to determine an accuracy level (e.g., how close the determined pose is to the expected pose). In some embodiments, the method may include sending a warning to a user interface (e.g., user interface 112) when the determined pose is outside a threshold accuracy level of the target pose.

[0082] Method 300 further includes displaying the determined pose on a user interface (step 316). The user interface (e.g., user interface 112) allows the surgeon to view the pose determined based on a single image and received information. The determined pose of one or more anatomical components can be referenced by the surgeon or technician to check the accuracy of the determined pose to aid in performing surgery, and / or to determine if updated registration is needed for the surgery or surgical procedure. In some embodiments, method 300 may use the user interface to present a single image and / or the determined pose of one or more anatomical components. In some embodiments, the determined pose may be presented as metadata and / or an image depicting the determined pose. The surgeon may use the display to adjust or verify the surgery or surgical procedure.

[0083] Based on the type of tissue (e.g., soft or hard tissue), the relative movement of one or more anatomical components when compared to a baseline posture value (e.g., based on a predetermined posture), and combinations thereof, the presentation of the determined posture can display one or more anatomical components with different visual markers. For example, a sensitive anatomical component that has moved out of position between the calculation of the predetermined posture and the calculation of the determined posture may be displayed more prominently (e.g., with greater intensity, contrast, etc.) than an anatomical component that has moved less (e.g., below a predetermined threshold). In some embodiments, the determined posture can be displayed using metadata associated with each of the one or more anatomical components, which can help surgeons differentiate between the one or more anatomical components, thereby better viewing the posture of the one or more anatomical components on the user interface. In some embodiments, the determined posture can be overlaid on the predetermined posture of one or more anatomical components. Overlay maps can allow users (e.g., surgeons) to better visually determine the posture difference between the determined posture and the predetermined posture. Information about the amount or degree of difference (whether absolute or relative) between the determined posture of one or more anatomical components and the predetermined posture of one or more anatomical components can also be displayed.

[0084] As can be understood based on the foregoing disclosure, this disclosure covers those with less than Figure 3 The methods for all steps identified in the code (and the corresponding description of method 300), and those including but not limited to... Figure 3 The method for additional steps beyond those steps identified in the diagram (and the corresponding description of method 300).

[0085] Turning Figure 4This document illustrates a method 400 for updating registration according to embodiments of the present disclosure. Method 400 may be performed partially or entirely by, for example, computing device 102 or similar means, and may utilize system 100 and / or components thereof (e.g., processor 104, one or more imaging devices 108, navigation system 136, and / or combinations thereof). At least one processor for performing method 400 and / or one or more steps thereof may be the same as or similar to processor 104 of the computing device 102 described above. At least one processor may be part of a robot (e.g., robot 130 including one or more robotic arms 132), part of a navigation system (e.g., navigation system 136), or part of a user interface (e.g., user interface 112). Processors other than any processors described herein may also be used to perform method 400. At least one processor may perform method 400 by executing instructions (e.g., instruction 140) stored in a memory such as memory 116. One or more aspects of method 400 may be performed by a robot and / or a surgical robotic arm (e.g., robotic arm 132 attached to robot 130) and / or components thereof, or together with a robot and / or a surgical robotic arm and / or components thereof. Method 400 may be used, for example, to update registration based on the movement of one or more components of the patient and / or system 100 during, for example, surgery or surgical procedure. For example, the patient may have shifted during surgery, and a new registration may be required to ensure that surgical devices and / or instruments (e.g., robot 130) are correctly positioned relative to the patient. Method 400 may additionally or alternatively be used to display the determined posture of one or more anatomical components on a user interface, for example, by a surgeon during surgery or surgical procedure.

[0086] Method 400 further includes receiving information corresponding to a pre-existing registration between the image space and the patient space (step 404). The information may include, for example, CT images, MRI images, or other preoperative images of the patient, and / or one or more intraoperative images of the patient. Any such images can be acquired using an imaging device such as imaging device 108 or any other imaging device. The information may additionally or alternatively include a function or algorithm for transforming coordinates in one of the image space and the patient space to the other. The information may include information regarding the pose of one or more reference points and / or one or more anatomical components (in the case of pre-existing registration).

[0087] Method 400 further includes generating a single image of a portion of the patient (step 408). For example, computing device 102 (and more specifically, processor 104) may cause imaging device (e.g., imaging device 108) to generate (e.g., capture) an image of a portion of the patient, the image including one or more of a plurality of references. The single image may be, for example, a fluoroscopic image. In some embodiments, the processor may cause the imaging device to move relative to the patient in a first pose (e.g., position and orientation) such that the imaging device can observe each of the plurality of references. In some embodiments, the first pose may be a position and orientation such that the imaging device sees each of the plurality of references and that the relative spacing and depth information of each of the plurality of references (e.g., the position of each reference in a coordinate system and how far each reference is from the camera relative to the coordinate system) can be determined based on the captured image and the information received in step 404. In some embodiments, the image may capture only portions of each of the plurality of references. For example, only a percentage of each of the plurality of references may be present in the image (e.g., 1%, 2%, 5%, 10%, 15%, 25%, 50%, 75%, 80%, 90%, etc.). In such embodiments, the percentage may be different for each of the multiple references. In some embodiments, the image may depict only a percentage of the total number of references (e.g., only half of the total number of references). In some embodiments, method 400 generates only a single image of the portion of the patient. The imaging device may transmit the captured image to one or more components within system 100 (e.g., to computing device 102, database 144, etc.) and / or to components outside system 100. Each of the multiple references may be present in the image and may be distinguishable from other artifacts or structures in the image. For example, in an embodiment where the multiple references include screws, imaging device 108 may emit radiation (e.g., X-rays), with a detector positioned such that the multiple references are between imaging device 108 and the detector. The resulting image may highlight (e.g., displayed with different colors, textures, intensities, and / or shapes) the multiple references located within the patient. As previously noted, the multiple references may be attached to or positioned near one or more anatomical components (e.g., internal organs, bones, ligaments, tendons, muscle structures, other soft tissues and / or hard tissues, etc.). In such embodiments, the capture of images depicting said portions of the patient can also depict the relative positions of one or more anatomical components to which the reference is attached or located proximally.

[0088] In some embodiments, one or more of a plurality of reference points may be placed on or within the patient prior to surgery or surgical procedures. For example, one or more of the plurality of reference points may be secured to the patient's skin (or even to the patient's skeletal anatomy, whether through a small incision or otherwise) before surgery begins. Also in some embodiments, one or more of the plurality of reference points may have been previously implanted in the patient (e.g., during initial surgery), allowing system 100 to capture (e.g., via imaging device 108) the relative layout of the plurality of reference points prior to subsequent procedures (e.g., remedial surgery). System 100 may use the captured information during surgery to compare the layout of the plurality of reference points with new images of the plurality of reference points captured during or after surgery or surgical procedures.

[0089] Furthermore, as described above, in some embodiments, one or more of a plurality of reference points may be implanted in the patient. For example, a surgery or surgical procedure may be performed within the patient. In such embodiments, one or more of the plurality of reference points may be implanted at specific locations on the patient (e.g., anatomical features, structures, etc.). This allows system 100 and / or its components to determine the patient's relative anatomy. The plurality of reference points can then be removed from the patient during and / or after the surgery or surgical procedure. The plurality of reference points is by no means limiting, and examples may include screws, tracking devices, radio frequency identification (RFID) tags, etc. Screws may be, for example, pedicle screws, cortical screws, cancellous bone screws, combinations thereof, and / or the like.

[0090] In some embodiments, multiple references can be positioned such that depth measurements can be obtained from the image. The depth measurement can be determined based on a first surface of the references being observed. For example, the first surface of the references may be angled relative to a first pose of the imaging device, such that the imaging device can see only an isometric (e.g., perspective) view of the references. In other words, the references are neither orthogonal (e.g., such that the imaging device only observes the top or bottom of the references and not the sides) nor lateral (e.g., such that the imaging device only observes the sides of the references and not the top or bottom). Instead, the view of the references seen by the imaging device (and subsequently appearing in the image) allows the first surface to be observed in addition to at least one of the top, bottom, and sides of the references. The appearance of at least two surfaces allows method 400 to determine the depth of the references (e.g., how far each point of the reference is from the imaging device relative to the coordinate system) using the information received in step 404 and by determining the angle of the references in the captured image.

[0091] Method 400 includes receiving information (step 412) regarding the size, shape, configuration, posture, and / or relative position of each of a plurality of references (positioned on or within the patient). The plurality of references may be or include one or more screws, rods, intervertebral bodies, or other medical devices implanted within the patient. The information may be received via a user interface such as user interface 112 and / or a communication interface (e.g., communication interface 106) and may be stored in memory (e.g., memory 116) by a computing device (e.g., computing device 102). In some embodiments, the information may be received from one or more components of system 100, such as from database 144 and / or cloud 148 and / or from external components of system 100 (e.g., from other computing devices and / or databases). The information may include CAD (computer-aided design) models of one or more of the plurality of references. The information may correspond to the reflectivity of one or more references. The information may correspond to surgical planning and / or data from a robot used to perform the surgical planning and may include, for example, information regarding the posture (e.g., position and orientation) or location or orientation of a given reference relative to the patient's anatomy when inserted into the patient. The information may include additional data regarding multiple references (e.g., the number of references inserted into the patient, the relative timing of the placement of the multiple references, etc.) and / or surgical procedures (e.g., the type of surgical procedure, the duration of the surgical procedure, etc.). In some embodiments, the information may include CT images, MRI images, or another image depicting the multiple references.

[0092] The received information may at least describe the size, shape, configuration, pose, and / or relative position of the references. The size of each of the plurality of references may be or include the dimensions of each reference (e.g., length, width, height, etc.). The shape of each of the plurality of references may be information about the outline, geometry, contour, surface, edges, etc., of the plurality of references. The pose of each of the plurality of references may be the position and orientation of each reference relative to the patient or in another specified coordinate space. The relative position of each reference may be a measurement of the relative distance of each reference to the remaining references of the plurality of references. In some embodiments, the information may be input into one or more algorithms (e.g., pose algorithm 122) to extract or determine additional information and / or transform the information into information useful to the system (e.g., information useful for or related to registration). In some embodiments, information about the plurality of references may be further stored / saved (e.g., in database 144, memory 116, or elsewhere) within system 100.

[0093] Method 400 further includes determining the pose of one or more anatomical components in a single captured image (step 416). The determination may be based on information received in steps 404 and / or 412 and the image generated in step 408. The determination may use one or more algorithms, such as pose algorithm 122. The pose determination may involve using coordinates and / or orientations of multiple references attached to or near one or more anatomical components. In some embodiments, the pose algorithm may be configured to compute one or more poses based on multiple references captured in the image and information received in step 412. For example, the pose algorithm may determine that references are attached to or near an anatomical component and may use received information about the size, shape, configuration, pose, and / or relative position of the references, along with the image generated in step 308, to define the pose of the anatomical component. In some embodiments, the pose algorithm may output the defined pose relative to a predetermined coordinate system, the robot, and / or other components (e.g., other components of system 100). In some embodiments, the pose algorithm may be, or use, artificial intelligence (AI), machine learning (e.g., deep learning), and / or other solutions to identify, detect, or otherwise recognize the references. The determined pose can be sent to and / or stored in one or more components of system 100. Method 400 may additionally or alternatively utilize an image processing algorithm (e.g., image processing algorithm 120). The image processing algorithm may receive captured images (e.g., from an imaging device) and may filter, process, or otherwise digitally manipulate individual images. Such image post-processing may improve image storage efficiency and / or improve image quality to more easily facilitate pose determination (e.g., by improving the quality of the image passed to the pose algorithm).

[0094] In some embodiments, the depth of each of a plurality of references can be determined. In embodiments where the imaging device captures views of at least two surfaces of a reference, relative depth can be determined by angle measurement in conjunction with the information received in step 412. For example, a pose algorithm can determine the relative angle of a reference in an image of a portion of the patient, and the pose of the reference can be calculated using angle and position information (e.g., information received about the relative position of the reference). A first surface of each reference can be the surface on which the pose algorithm measures the relative angle. The selection of the first surface is not limited, and the selection of the first surface can be arbitrary and can be additionally or alternatively based on the appearance of the reference in the image, based on predetermined instructions for specifying the first surface, etc. The angle of the first surface can appear in a single image at angles greater than 0 degrees and less than 90 degrees, greater than 10 degrees and less than 80 degrees, greater than 20 degrees and less than 70 degrees, etc. However, it should be understood that the above-described angle ranges are by no means limited to the embodiments of this disclosure, and additional or alternative angle ranges between 0 and 90 degrees are possible. Method 300 can use the determined reference positions to further determine the pose of one or more anatomical components. For example, in embodiments where multiple references are fixed to one or more anatomical components, the pose of the references can allow the processor to determine the relative pose of one or more anatomical components. In some embodiments, one or more of the multiple references may be attached to or near one or more anatomical components, which can allow the pose of one or more anatomical components to be determined based on the relative poses of the multiple references and the known distances between the multiple references and one or more anatomical components. In some embodiments, the pose of one or more anatomical components is compared to a target pose. For example, the target pose may be a predicted or expected pose of each of the one or more anatomical components. A comparison algorithm may compare the determined pose to the target pose to determine an accuracy level (e.g., how close the determined pose is to the expected pose). In some embodiments, the method may include sending a warning to a user interface (e.g., user interface 112) when the determined pose is outside a threshold accuracy level of the target pose.

[0095] Method 400 further includes updating the pre-existing registration between the image space and the patient space based on the determined pose of one or more anatomical components (step 420). Method 400 may utilize one or more algorithms (e.g., registration update or comparison algorithm 128). One or more algorithms may compare the determined pose of one or more anatomical components with a predetermined pose of one or more anatomical components. For example, one or more algorithms may receive the determined pose of one or more anatomical components and compare the determined pose with a predetermined pose. In such embodiments, one or more algorithms may determine that the values ​​of the determined pose and the predetermined pose do not match and / or are outside tolerance values. As a result, one or more algorithms may update the registration between the image space and the patient space. The updated registration may map the determined pose to the patient space (e.g., using position and orientation information associated with the determined pose) and may update the surgical plan (e.g., surgical plan 134) to reflect the changed pose of one or more anatomical components relative to the patient space. In some embodiments, the system and / or its components (e.g., robot 130 and / or one or more robotic arms 132) may then operate autonomously or semi-autonomously based on the updated registration. In some embodiments, updating a registration may include deleting a previous registration or replacing a previous registration with a new registration.

[0096] The new registration can be based at least in part on the received information and the determined pose of one or more anatomical components in the image. For example, information about multiple references (and images of the multiple references) can facilitate updating the registration with acceptable accuracy, regardless of whether only one image of the multiple references is generated. One or more algorithms can output the updated registration, which can be passed to one or more of a robot (e.g., robot 130 and / or one or more robotic arms 132), a surgical plan (e.g., surgical plan 134), and / or a database (e.g., database 144). In some embodiments, the updated registration can then be used by one or more components of a system (e.g., system 100) to perform surgery or surgical procedures using the new registration.

[0097] As can be understood based on the foregoing disclosure, this disclosure covers those with less than Figure 4 The methods for all steps identified in the code (and the corresponding description of method 400), and including all steps except... Figure 4 The method for additional steps beyond those steps identified in the diagram (and the corresponding description of method 400).

[0098] The foregoing is not intended to limit this disclosure to the one or more forms disclosed herein. In the foregoing specific embodiments, for the purpose of simplifying this disclosure, various features of this disclosure are grouped together in one or more aspects, embodiments, and / or configurations. Features of aspects, embodiments, and / or configurations of this disclosure may be combined in alternative aspects, embodiments, and / or configurations other than those discussed above. The approach of this disclosure should not be construed as reflecting an intention that the claims require more features than expressly recited in each claim. Rather, as reflected in the following claims, aspects of the invention lie in fewer than all the features of a single foregoing aspect, embodiment, and / or configuration. Therefore, the following claims are hereby incorporated into this specific embodiment, wherein each claim exists independently as a separate preferred embodiment of this disclosure.

[0099] Furthermore, although the description has already included descriptions of one or more aspects, embodiments, and / or configurations, as well as certain variations and modifications, other variations, combinations, and modifications are also within the scope of this disclosure upon understanding it, for example, as may be within the skill and knowledge of one skilled in the art. It is intended to obtain the right to include alternative aspects, embodiments, and / or configurations within the permitted scope, including claimed alternatives, replacements, and / or equivalent structures, functions, scopes, or steps, regardless of whether such alternatives, replacements, and / or equivalent structures, functions, scopes, or steps are disclosed herein, and it is not intended for use with any patentable subject matter.

Claims

1. A method for surgical registration, the method comprising: Receive information corresponding to a pre-existing registration between the image space and the patient space; Receive information about the orientation of each of a plurality of references positioned on or within the patient; The imaging device generates a single image of a portion of the patient, the single image depicting at least a portion of each of the plurality of references; The pose of one or more anatomical components represented in the single image is determined based on the information regarding the pose of each of the multiple references positioned on or within the patient and the single image; as well as Based on comparing the determined pose of one or more anatomical components with the predetermined pose of one or more anatomical components, when the values ​​of the determined pose and the predetermined pose do not match and / or are outside the tolerance value, the determined pose is mapped to the patient space to update the pre-existing registration between the image space and the patient space; as well as The surgical plan is updated to reflect the changing posture of the one or more anatomical components relative to the patient space.

2. The method of claim 1, wherein comparing the determined posture further comprises comparing the determined posture with a target posture of the one or more anatomical components in the surgical plan.

3. The method according to claim 1, further comprising: The position of the imaging device relative to the plurality of references is determined based on the use of the phantom.

4. The method of claim 1, wherein the plurality of references comprises one or more screws.

5. The method of claim 1, wherein the plurality of benchmarks comprises a plurality of different types of implants.

6. The method of claim 1, further comprising: Quantify the change in posture of at least one of the one or more anatomical components from the predetermined posture to the determined posture.

7. The method of claim 1, wherein the image further depicts a device held by a robot, and wherein the method further comprises: The determined posture is located in the coordinate space corresponding to the robot based on the image.

8. The method of claim 1, wherein the information includes a computer-aided design (CAD) model of at least one of the plurality of references.

9. A system for surgical registration, the system comprising: processor; as well as A memory that stores instructions for execution by the processor, which, when executed by the processor, cause the processor to: Receive information corresponding to a pre-existing registration between the image space and the patient space; Receive information about the shape and orientation of multiple tracking devices placed on or inside the patient; The imaging device generates a single image of a portion of the patient, the single image including at least a portion of each of the plurality of tracking devices; The portion of each of the plurality of tracking devices is identified in the single image; The pose of one or more anatomical components represented in the single image is calculated based on the information regarding the shape and orientation of the plurality of tracking devices placed on or within the patient and the identified portion of each of the plurality of tracking devices; as well as Based on comparing the determined pose of one or more anatomical components with the predetermined pose of one or more anatomical components, when the values ​​of the determined pose and the predetermined pose do not match and / or are outside the tolerance value, the determined pose is mapped to the patient space to update the pre-existing registration between the image space and the patient space; as well as The surgical plan is updated to reflect the changing posture of the one or more anatomical components relative to the patient space.

10. The system of claim 9, wherein the imaging device is fixed relative to the patient in a first position.

11. The system of claim 10, wherein the first surface of each of the portions of the plurality of tracking devices in the single image appears at an angle greater than ten degrees and less than eighty degrees relative to the imaging device.

12. The system of claim 9, wherein the single image is generated using fluorescence perspectration.

13. The system of claim 9, wherein the plurality of tracking devices comprises a plurality of different types of implants.

14. The system of claim 9, wherein at least one of the plurality of tracking devices is a screw implanted in the patient.

15. The system of claim 9, wherein the plurality of tracking devices comprises one or more screws.

16. The system of claim 9, wherein each of the one or more anatomical components is a vertebra.

17. A system for surgical registration, the system comprising: processor; Imaging devices; Multiple reference points are placed on or within the patient; as well as A memory that stores instructions for execution by the processor, which, when executed by the processor, cause the processor to: Receive information corresponding to a pre-existing registration between the image space and the patient space; Receive information about the pose of the plurality of references; The imaging device generates a single image of a portion of the patient, the single image including the plurality of references; The pose of one or more anatomical tissues represented in the single image is determined based on the pose information of the multiple references and the single image; Based on comparing the determined pose of the one or more anatomical tissues with the predetermined pose of the one or more anatomical tissues, when the values ​​of the determined pose and the predetermined pose do not match and / or are outside the tolerance value, the determined pose is mapped to the patient space to update the pre-existing registration between the image space and the patient space; as well as Update the surgical plan to reflect the changing posture of the one or more anatomical tissues relative to the patient space; as well as Display the determined pose on the user interface.

18. The system of claim 17, wherein the determined pose is displayed on the user interface as an overlay of the target pose.

Images