Systems and methods for aligning localizers with imaging fields of view

Abstract

A system according to at least one embodiment of the present disclosure includes: a processor; and a memory storing data thereon that, when processed by the processor, enable the processor to: receive a video feed from an imaging device depicting an anatomy of a patient; overlay the video feed with a virtual depiction of a localizer frame; detect, in the video feed, the localizer frame; detect when a difference between the virtual depiction of the localizer frame overlaid on the localizer frame depicted in the video feed falls below a threshold value; and generate, when the difference falls below the threshold value, an alert.

Description

A0013282SYSTEMS AND METHODS FOR ALIGNING LOCALIZERS WITH IMAGING FIELDS OF VIEW

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No.63 / 777,485, filed 25 March 2025, the entire content of which is incorporated herein by reference.BACKGROUND

[0002] The present disclosure is generally directed to imaging, and relates more particularly to surgical imaging.

[0003] Images may be used during a surgical operation for a variety of surgical tasks. The images may be obtained prior to and / or during a surgical operation. The imaging may be used by a medical provider for diagnostic and / or therapeutic purposes. Patient anatomy can change over time, particularly following placement of a medical implant in the patient anatomy.BRIEF SUMMARY

[0004] Example aspects of the present disclosure include:

[0005] A system according to at least one embodiment of the present disclosure comprises: a processor; and a memory storing data thereon that, when processed by the processor, enable the processor to: receive a video feed from an imaging device depicting an anatomy of a patient; overlay the video feed with a virtual depiction of a localizer frame; detect, in the video feed, the localizer frame; detect when a difference between the virtual depiction of the localizer frame overlaid on the localizer frame depicted in the video feed falls below a threshold value; and generate, when the difference falls below the threshold value, an alert.

[0006] A system according to at least one embodiment of the present disclosure comprises: a depth camera; a processor; and a memory storing data thereon that, when executed by the processor, enable the processor to: receive, from the depth camera, a video feed of a patient and a stereotactic frame connected to the patient; overlay, on the video feed, a virtual depiction of the stereotactic frame and a virtual depiction of a field of view of a radiation source; detect, in the video feed, the stereotactic frame; determine a difference between a position of the virtual depiction of the stereotactic frame and the stereotactic frame; and generate, when the difference falls below a threshold value, an alert.

[0007] A system according to at least one embodiment of the present disclosure comprises: a processor; and a memory storing data thereon that, when processed by the processor, enable the processor to: receive, from a depth camera, a video feed of a patient and a stereotactic frame connected to the patient; overlay, on the video feed, a virtual depiction of the stereotactic frame and a virtual depiction of a field of view of a radiation source; detect, in the video feed, the stereotactic frame; determine a difference between a position of the virtualA0013282 depiction of the stereotactic frame and the stereotactic frame; and generate, when the difference falls below a threshold value, a motion trajectory of a radiation source to align the radiation source with the stereotactic frame.

[0008] Any aspect in combination with any one or more other aspects.

[0009] Any one or more of the features disclosed herein.

[0010] Any one or more of the features as substantially disclosed herein.

[0011] Any one or more of the features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein.

[0012] Any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments.

[0013] Use of any one or more of the aspects or features as disclosed herein.

[0014] It is to be appreciated that any feature described herein can be claimed in combination with any other feature(s) as described herein, regardless of whether the features come from the same described embodiment.

[0015] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

[0016] The phrases “at least one”, “one or more”, and “and / or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and / or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together. When each one of A, B, and C in the above expressions refers to an element, such as X, Y, and Z, or class of elements, such as XI -Xn, Yl-Ym, and Zl-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., XI and X2) as well as a combination of elements selected from two or more classes (e.g., Y1 and Zo).

[0017] The term “a” or “an” entity refers to one or more of that entity. As such, the terms “a” (or “an”), “one or more” and “at least one” can be used interchangeably herein. It is also to be noted that the terms “comprising”, “including”, and “having” can be used interchangeably.

[0018] The preceding is a simplified summary of the disclosure to provide an understanding of some aspects of the disclosure. This summary is neither an extensive nor exhaustive overview of the disclosure and its various aspects, embodiments, and configurations. It is intended neither to identify key or critical elements of the disclosure nor to delineate the scope of theA0013282 disclosure but to present selected concepts of the disclosure in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other aspects, embodiments, and configurations of the disclosure are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.

[0019] Numerous additional features and advantages of the present disclosure will become apparent to those skilled in the art upon consideration of the embodiment descriptions provided hereinbelow.BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are incorporated into and form a part of the specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the disclosure. The drawings simply illustrate preferred and alternative examples of how the disclosure can be made and used and are not to be construed as limiting the disclosure to only the illustrated and described examples. Further features and advantages will become apparent from the following, more detailed, description of the various aspects, embodiments, and configurations of the disclosure, as illustrated by the drawings referenced below.

[0021] Fig. 1 A is a block diagram of aspects of a system according to at least one embodiment of the present disclosure;

[0022] Fig. IB is a block diagram of aspects of a memory according to at least one embodiment of the present disclosure;

[0023] Fig. 1C shows aspects of an imaging platform according to at least one embodiment of the present disclosure;

[0024] Fig. ID shows additional aspects of the imaging platform according to at least one embodiment of the present disclosure;

[0025] Fig. 2A is a depiction of a live video feed of a patient with an imaging field of view and stereotactic frame overlay according to at least one embodiment of the present disclosure;

[0026] Fig. 2B is another depiction of a live video feed of a patient with an imaging field of view and stereotactic frame overlay according to at least one embodiment of the present disclosure;

[0027] Fig. 2C is a depiction of a live video feed of a patient with a stereotactic frame and the imaging field of view and stereotactic frame overlay according to at least one embodiment of the present disclosure;

[0028] Fig. 2D is another depiction of a live video feed of a patient with a stereotactic frame and an imaging field of view and stereotactic frame overlay according to at least one embodiment of the present disclosure; andA0013282

[0029] Fig. 3 is a flowchart according to at least one embodiment of the present disclosure.DETAILED DESCRIPTION

[0030] It should be understood that various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and / or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0031] Before any embodiments of the disclosure are explained in detail, it is to be understood that the disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Further, the present disclosure may use examples to illustrate one or more aspects thereof. Unless explicitly stated otherwise, the use or listing of one or more examples (which may be denoted by “for example,” “by way of example,” “e.g.,” “such as,” or similar language) is not intended to and does not limit the scope of the present disclosure.

[0032] Positioning an intraoperative imaging system during surgical procedures, such as cranial Deep Brain Stimulation procedures, may present challenges due to the patient’s position and equipment connected to the patient. Further difficulties can arise when trying to capture images of a localizer frame, which frame is often used to register intraoperative three- dimensional (3D) volumes to preoperative Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) images. Images of portions of the frame may be captured to perform the registration. The imaging devices used to capture the images for registration may have limited fields of view, further complicating the registration process. Related art methods implement multiple iterations of two-dimensional (2D) scout shots and positioning to alignA0013282 the imaging device with the frame. Such methods may increase surgical procedure time and expose the patient to additional doses of radiation.

[0033] According to at least one embodiment of the present disclosure, cameras located on an intraoperative imaging system may be used to facilitate alignment between the imaging devices and the localizer frame. The live video feed of the cameras a may be displayed to aid a user (e.g., a surgeon) in aligning the imaging field of view with the localizer frame. In one example, an overlay may be displayed on the camera feed that depicts an outline of the imaging field of view and the rods of the localize frame. The user can then align the overlay with the localizer attached to the patient, resulting in faster positioning for intraoperative registration. Additionally, the patient is exposed to fewer scout shots and thus fewer radiation doses.

[0034] According to at least one embodiment of the present disclosure, cameras, depth sensors, and an infrared (IR) sensor are provided that facilitate positioning of the intraoperative imaging system relative to the patient. Data from the cameras and sensors may be input into one or more algorithms that generate an output display. For example, the data may be input into a display algorithm that processes the input data and causes an output video feed of the camera to be displayed on a display in the surgical environment. The display algorithm may also display the outline of the stereotactic frame overlaid on the video feed. The data may also be provided to a feature detection algorithm that detects the presence of stereotactic rods in the input data and generates an alert when the stereotactic rods have achieved target alignment positions (e.g., the position of the stereotactic rods in the video feed matches the position of the stereotactic rods as depicted by the overlay).

[0035] When the stereotactic rods have achieved the target alignment position, the imaging system may use a motion trajectory algorithm to determine a required movement of the imaging platform to position the imaging source (e.g., an x-ray source) at the position of the camera to capture an image of the patient with the stereotactic rods. The image of the patient with the stereotactic rods may then be used to register preoperative and intraoperative patient images.

[0036] Embodiments of the present disclosure provide technical solutions to one or more of the problems of (1) excessive radiation exposure to patient anatomy and (2) slow and inaccurate registration.

[0037] Turning first to Fig. 1 A, a block diagram of aspects of a system 100 according to at least one embodiment of the present disclosure is shown. The system 100 may be used to display one or more features overlaid on a video feed of an imaging device; to detect one or more features of a stereotactic frame or other object in the video feed; to determine one orA0013282 more motion trajectories to move the imaging device to align the imaging device with the stereotactic frame; to determine when the imaging device is aligned with the stereotactic frame; and / or to carry out one or more other aspects of one or more of the methods disclosed herein. The system 100 comprises a computing device 102, a robot 114, a navigation system 118, an imaging platform 126 with one or more imaging devices 112, a database 130, and / or a cloud or other network 134. Systems according to other embodiments of the present disclosure may comprise more or fewer components than the system 100. For example, the system 100 may not include the robot 114, one or more components of the computing device 102, the database 130, and / or the cloud 134.

[0038] The computing device 102 comprises a processor 104, a memory 106, a communication interface 108, and a user interface 110. Computing devices according to other embodiments of the present disclosure may comprise more or fewer components than the computing device 102.

[0039] The processor 104 of the computing device 102 may be any processor described herein or any similar processor. Non-limiting examples of the processor 104 comprise one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple All, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000-series processors, Nvidia GeForce RTX 3000-series processors, AMD Radeon RX 5000-series processors, AMD Radeon RX 6000-series processors, or any other graphics processing units), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0040] The processor 104 may be configured to execute instructions stored in the memory 106, which instructions may cause the processor 104 to carry out one or more computing steps utilizing or based on data received from the robot 114, the navigation system 118, the imaging platform 126, the database 130, and / or the cloud 134.

[0041] The memory 106 may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. The memory 106 may store information or data useful for completing, for example, any step of the method 300 described herein, or ofA0013282 any other methods. The memory 106 may be configured to store a variety of parameters, weights, algorithms, and / or the like. In the example shown in Fig. IB, the memory 106 may store or comprise image processing 120, segmentation 122, transformation 124, registration 128, one or more display algorithms 132, stereotactic frame information 142, one or more feature detection algorithms 146, and one or more motion trajectory algorithms 154. In other examples, the memory 106 may act as a temporary buffer for storing data until such data can be uploaded to the database 130 and / or other data repository.

[0042] The content of memory 106, if provided as in instruction, may, in some embodiments, be organized into one or more applications, modules, packages, layers, or engines. Although various contents of the memory 106 may be described as instructions, it should be appreciated that functionality described herein can be achieved through use of instructions, algorithms, and / or artificial intelligence or machine learning data models. The data, algorithms, and / or instructions may cause the processor 104 to manipulate data stored in the memory 106 and / or received from the imaging devices 112, the robot 114, the database 130, and / or the cloud 134.

[0043] The image processing 120 enables the processor 104 to process image data of an image received, for example, from the imaging platform 126 and / or components thereof (e.g., the depth camera 152) for the purpose of, for example, identifying information about a patient and / or an object such as an implanted device depicted in the image data. “Image data” as used herein refers to the data generated or captured by one or more components of the imaging platform 126 such as the depth camera 152, the radiation source 138 and radiation detector 140, etc., including in a machine-readable form, a graphical / visual form, and in any other form. The image data may be or comprise a preoperative image, an intraoperative image, a postoperative image, or an image taken independently of any surgical procedure.

[0044] The segmentation 122 enables the processor 104 to segment image data so as to identify the patient and / or one or more objects such as, for example, an implanted device in the image data. The segmentation 122 may enable the processor 104 to identify patient features using, for example, feature recognition. For example, the segmentation 122 may enable the processor 104 to identify one or more anatomical elements (e.g., vertebrae, ribs, etc.) of the patient and / or one or more devices (e.g., surgical screws) implanted into the anatomical elements. In other examples, the segmentation 122 may enable the processor 104 to identify a boundary of an object (e.g., a boundary of a rib, a boundary of a vertebra, etc.) by determining a difference in or contrast between colors or grayscales of image pixels. In some cases, the segmentation 122 may enable the processor 104 to identify the boundary of one orA0013282 more objects, such as one or more features of the stereotactic frame 148 depicted in the image data.

[0045] The transformation 124 enables the processor 104 to transform one coordinate system into another coordinate system. In other words, the transformation 124 enables the processor 104 to transform a first coordinate system (e.g., a patient coordinate system) into a second coordinate system (e.g., a reference frame coordinate system) based on, for example, the registration between the first coordinate system and a third coordinate system and the registration between the second coordinate system and the third coordinate system.

[0046] The registration 128 enables the processor 104 to correlate one coordinate system with another coordinate system. For example, the registration 128 may enable the processor 104 to correlate or map a first coordinate system (e.g., a patient coordinate system) with a third coordinate system (e.g., an imaging device coordinate system) and a second coordinate system (e.g., a radiation source coordinate system) with the third coordinate system (e.g., the imaging device coordinate system).

[0047] The display algorithm 132 enables the processor 104 to display stereotactic frame information 142 to the user interface 110 or other display. In one example, the display algorithm 132 may use image processing 120 to receive image data from the depth camera 152 as well as the stereotactic frame information 142 as an input and output a live video feed to the user interface 110 that shows the field of view of the depth camera 152 as well as an outline of the stereotactic frame information 142 that is overlaid on the field of view video feed. The stereotactic frame information 142 may comprise virtual depictions of the stereotactic frame 148. For example, the stereotactic frame information 142 may comprise the overall shape of the stereotactic frame 148, one or more features of the stereotactic frame 148 (e.g., a Z-shaped pattern of crossbars), an outline of one or more portions of the stereotactic frame 148, combinations thereof, and / or the like. In some cases, and depending on the position of the depth camera 152, the display algorithm 132 may generate in a lateral view and / or an anterior-posterior (AP) view of the stereotactic frame information 142. In some examples, the display algorithm 132 may be configured to additionally or alternatively display a field of view of the radiation source 138 that is overlaid on the video feed from the depth camera 152.

[0048] The feature detection algorithm 146 enables the processor 104 to detect one or more features of the stereotactic frame 148 when the stereotactic frame 148 is in view of the depth camera 152. In one example, the feature detection algorithm 146 may use segmentation 122 to identify the features of the stereotactic frame 148 (e.g., the rods of the stereotactic frame 148 that form a Z-shaped pattern in a lateral view). In some cases, the feature detectionA0013282 algorithm 146 may receive the output from the display algorithm 132 (e.g., the live video feed with stereotactic frame information 142 and / or a field of view of the radiation source 138) as an input, identify the stereotactic frame 148 in the live video feed, compare the position of the stereotactic frame 148 to the stereotactic frame information 142 that is overlaid on the video feed from the depth camera 152, and generate an output indicative of the position comparison. When the difference in position of the stereotactic frame 148 and the stereotactic frame information 142 falls below a predetermined threshold value, the feature detection algorithm 146 may determine that the depth camera 152 is aligned with the patient 170, and may generate an alert. The alert may be visual (e.g., rendered to the user interface 110) and / or audio (e.g., an alarm, a beeping sound, etc.) that indicates to the user of the system 100 (e.g., a surgeon, a member of surgical staff, etc.) that the depth camera 152 is aligned with the stereotactic frame 148.

[0049] The motion trajectory algorithm 154 may determine one or more trajectories of one or more components in the surgical environment to avoid collisions. In one example, the motion trajectory algorithm 154 generates trajectories that can be used by the processor 104 to position one or more of the imaging devices 112 (e.g., the depth camera 152, the radiation source 138, the radiation detector 140, etc.) relative to the patient 170. For example, once the feature detection algorithm 146 has determined that the depth camera 152 and the patient 170 are aligned, the motion trajectory algorithm 154 may determine a motion trajectory of the radiation source 138 to move the radiation source 138 to the location of the depth camera 152 (such that, after movement, the radiation source 138 is aligned with the patient 170). The motion trajectory algorithm 154 may take the position of one or more of the imaging devices 112 as an input, and output one or more motion trajectories that can be used by the processor 104 to change the position of the one or more imaging devices 112. In one case, the depth camera 152, the radiation source 138, and the radiation detector 140 may be part of the imaging platform 126, such that moving one of the imaging devices may result in complementary movement of the other imaging devices (e.g., the imaging devices are fixed relative to each other on a gantry). In such cases, the motion trajectory algorithm 154 may receive the position of the depth camera 152 and information about the position of the radiation source 138 relative to the depth camera 152 as an input, and may output a motion trajectory that moves the radiation source 138 into the position of the depth camera 152. In this example, the movement of the radiation source 138 may result in simultaneous movement of the depth camera 152, such that the depth camera 152 and the radiation source 138 do not collide with one another.A0013282

[0050] The communication interface 108 may be used for receiving image data or other information from an external source (such as the robot 114, the navigation system 118, the imaging platform 126, the database 130, the cloud 134, and / or any other system or component not part of the system 100), and / or for transmitting instructions, images, or other information to an external system or device (e.g., another computing device 102, the robot 114, the navigation system 118, the imaging platform 126, the database 130, the cloud 134, and / or any other system or component not part of the system 100). The communication interface 108 may comprise one or more wired interfaces (e.g., a USB port, an Ethernet port, a Firewire port) and / or one or more wireless transceivers or interfaces (configured, for example, to transmit and / or receive information via one or more wireless communication protocols such as 802.11a / b / g / n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface 108 may be useful for enabling the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

[0051] The computing device 102 may also comprise one or more user interfaces 110. The user interface 110 may be or comprise a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or for providing information to a user. The user interface 110 may be used, for example, to receive a user selection or other user input regarding any step of any method described herein.Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the system 100 (e.g., by the processor 104 or another component of the system 100) or received by the system 100 from a source external to the system 100. In some embodiments, the user interface 110 may be useful to allow a surgeon or other user to modify instructions to be executed by the processor 104 according to one or more embodiments of the present disclosure, and / or to modify or adjust a setting of other information displayed on the user interface 110 or corresponding thereto.

[0052] Although the user interface 110 is shown as part of the computing device 102, in some embodiments, the computing device 102 may utilize a user interface 110 that is housed separately from one or more remaining components of the computing device 102. In some embodiments, the user interface 110 may be located proximate one or more other components of the computing device 102, while in other embodiments, the user interface 110 may be located remotely from one or more other components of the computing device 102.

[0053] The one or more imaging devices 112 may be operable to image anatomical feature(s) (e.g., a bone, veins, tissue, etc.) and / or other aspects of patient anatomy to yield image data (e.g., image data depicting or corresponding to a bone, veins, tissue, etc.). In someA0013282 embodiments, a first imaging device 112 may be used to obtain first image data (e.g., a first image) at a first time, and a second imaging device 112 may be used to obtain second image data (e.g., a second image) at a second time after the first time. The one or more imaging devices 112 may be capable of taking a 2D image or a 3D image to yield the image data. The one or more imaging devices 112 may be or comprise, for example, an ultrasound scanner (which may comprise, for example, a physically separate transducer and receiver, or a single ultrasound transceiver), an O-arm, a C-arm, a G-arm, or any other device utilizing X-raybased imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, a microscope, an optical camera, a thermographic camera (e.g., an infrared camera), a radar system (which may comprise, for example, a transmitter, a receiver, a processor, and one or more antennae), or any other imaging device 112 suitable for obtaining images of an anatomical feature of a patient. The imaging devices 112 may be contained entirely within a single housing, or may comprise a transmitter / emitter and a receiver / detector that are in separate housings or are otherwise physically separated.

[0054] One or more of the imaging devices 112 may be operable to generate a stream of image data. For example, the depth camera 152 of the imaging devices 112 may be configured to operate with an open shutter, or with a shutter that continuously alternates between open and shut so as to capture successive images. For purposes of the present disclosure, unless specified otherwise, image data may be considered to be continuous and / or provided as an image data stream if the image data represents two or more frames per second.

[0055] The robot 114 may be any surgical robot or surgical robotic system. The robot 114 may be or comprise, for example, a robotic guidance system. The robot 114 may be configured to manipulate a surgical tool (whether based on guidance from the navigation system 118 or not) to accomplish or to assist with a surgical task. In some embodiments, the robot 114 may be configured to hold and / or manipulate an anatomical element during or in connection with a surgical procedure. The robot 114 may comprise one or more robotic arms 116. In some embodiments, the robotic arm 116 may comprise a first robotic arm and a second robotic arm, though the robot 114 may comprise more than two robotic arms. Each robotic arm 116 may be positionable independently of the other robotic arm. The robotic arms 116 may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

[0056] The robot 114, together with the robotic arm 116, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm 116 may be positioned or positionable in any pose, plane, and / or focal point. The pose includes a position and an orientation. As a result, one or more of the imaging devices 112, surgical tool, or otherA0013282 object held by the robot 114 (or, more specifically, by the robotic arm 116) may be precisely positionable in one or more needed and specific positions and orientations.

[0057] The robotic arm(s) 116 may comprise one or more sensors that enable the processor 104 (or a processor of the robot 114) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm).

[0058] In some embodiments, reference markers (e.g., navigation markers) may be placed on the robot 114 (including, e.g., on the robotic arm 116), one or more components of the imaging platform 126 (e.g., the radiation source 138, the radiation detector 140, the depth camera 152, etc.) or any other object in the surgical space. The reference markers may be tracked by the navigation system 118, and the results of the tracking may be used by the robot 114 and / or by an operator of the system 100 or any component thereof.

[0059] The navigation system 118 may provide navigation for a surgeon and / or a surgical robot during an operation. The navigation system 118 may be any now-known or future-developed navigation system, including, for example, the Medtronic Stealth Station™ S8 surgical navigation system or any successor thereof. The navigation system 118 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigated trackers, or other objects within the operating room or other room in which some or all of the system 100 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 118 may comprise one or more electromagnetic sensors. In various embodiments, the navigation system 118 may be used to track a position and orientation (e.g., a pose) of the robot 114 and / or robotic arm 116, and / or one or more surgical tools (or, more particularly, to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the foregoing). The navigation system 118 may include a display for displaying one or more images from an external source (e.g., the computing device 102, one or more imaging devices 112, or other source) or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 118. In some embodiments, the system 100 can operate without the use of the navigation system 118. The navigation system 118 may be configured to provide guidance to a surgeon or other user of the system 100 or a component thereof, to the robot 114, or to any other element of the system 100 regarding, for example, a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

[0060] The database 130 may store information that correlates one coordinate system to another (e.g., one or more robotic coordinate systems to a patient coordinate system and / or toA0013282 a navigation coordinate system). The database 130 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about a patient’s anatomy at and / or proximate the surgical site, for use by the robot 114, the navigation system 118, and / or a user of the computing device 102 or of the system 100); one or more images useful in connection with a surgery to be completed by or with the assistance of one or more other components of the system 100; sensor data (e.g., data generated by one or more IR sensors); threshold values; information about the stereotactic frame 148 (e.g., a type of stereotactic frame used in the surgery or surgical procedure, dimensions of the stereotactic frame, etc.); and / or any other useful information. The database 130 may be configured to provide any such information to the computing device 102 or to any other device of the system 100 or external to the system 100, whether directly or via the cloud 134. In some embodiments, the database 130 may be or comprise part of a hospital image storage system, such as a picture archiving and communication system (PACS), a health information system (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0061] The cloud 134 may be or represent the Internet or any other wide area network. The computing device 102 may be connected to the cloud 134 via the communication interface 108, using a wired connection, a wireless connection, or both. In some embodiments, the computing device 102 may communicate with the database 130 and / or an external device (e.g., a computing device) via the cloud 134.

[0062] The stereotactic frame 148 (also referred to herein as a localizer frame) may be or comprise a structure with features that can be recognized in image data for the purposes of localizing the patient during image registration. The stereotactic frame 148 may comprise crossbars positioned near and / or around a portion of patient anatomy that is the subject of the surgery or surgical procedure. For example, in Deep Brain Stimulation procedures, the stereotactic frame 148 may be mounted to the patient’s head and imaged to localize the patient’s head in a known coordinate system. Such localization may enable additional registration of, for example, preoperative patient images to intraoperative patient images. In some cases, the stereotactic frame 148 may be or comprise a head ring, an arc frame system, and / or any known stereotactic frame.

[0063] In some embodiments, the imaging devices 112 comprises a radiation source 138, a radiation detector 140, and a depth camera 152. The depth camera 152 may comprise one or more imaging features discussed herein with respect to the imaging devices 112. In other words, the depth camera 152 may operate to capture image data, which may be used by the processor 104 (e.g., using image processing 120) to generate an image from the image data.A0013282 As discussed further below, the depth camera 152 may be used in identifying patient anatomy and the stereotactic frame 148, rendering a virtual depiction of the field of view of the depth camera 152 with a depiction of the stereotactic frame 148, and determining when the depth camera 152 is aligned with the stereotactic frame 148.

[0064] In one example, the depth camera 152 may generate lateral images and / or API images of the patient 170. The depth camera 152 may generate the lateral images and / or the AP images by moving along a gantry of an imaging platform. For example, the depth camera 152 may be initially positioned on a side of the patient to generate the lateral image, and then may be positioned above the patient by moving along the gantry (e.g., using one or more motors in the gantry) to generate the AP image. In other examples, the depth camera 152 may comprise two cameras: a first camera positioned to generate a lateral image view of the patient 170 and a second camera positioned to generate an AP image view of the patient 170.

[0065] The radiation source 138 generates or otherwise emits radiation, waves, or other signals that are received or captured by the radiation detector 140 to generate an image of the anatomical elements (e.g., patient anatomy) positioned therebetween.

[0066] The radiation source 138 and / or the radiation detector 140 may each comprise a collimator 144. The collimator 144 aligns the X-rays or other signals passing therethrough (e.g., X-rays generated by the imaging source, X-rays captured by the imaging detector, and so forth) to, for example, improve the resulting image. In some embodiments, the collimator 144 may comprise an open portion and a closed portion. The open portion may be or comprise a portion of the collimator 144 through which X-rays or other signals may pass, and through which the passing X-rays are focused or aligned. The closed portion may be or comprise a portion of the collimator 144 through which X-rays or other signals are blocked or prevented from passing.

[0067] In some embodiments, the collimator 144 may comprise one, two, three, or more degrees of freedom. For instance, the collimator 144 may comprise three degrees of freedom, with the collimator 144 capable of opening or closing one or more shutters in a first direction (e.g., an X-axis direction) and a second direction (e.g., a Y-axis direction), while also capable of rotating the shutters independently of one another such that an open portion of the collimator 144 (e.g., the portion through which the X-rays are focused) is capable of rotating in a first plane (e.g., in an XY-plane). In some embodiments, the shutters may be controlled by the one or more motors disposed in the radiation source 138 or the radiation detector 140.

[0068] Figs. 1C-1D show aspects of the imaging platform 126 according to at least one embodiment of the present disclosure. The imaging platform 126 includes a support structure 156 and a table 172 on which a patient 170 can be positioned. The support structure 156A0013282 comprises an upper member or wall 160, a lower member or wall 164, and a pair of members or sidewalls 168 A, 168B. In some embodiments, the table 172 is positioned orthogonally to the support structure 156 (e.g., along a direction of the arrow 178 in Fig. ID), such that the table 172 may extend in a first direction from the support structure 156. In some embodiments, the table 172 may be mounted to the support structure 156. In other embodiments, the table 172 may not be mounted to the support structure 156 and may be mounted elsewhere.

[0069] In some embodiments, the support structure 156 is fixed securable to an operating room wall 162 (such as, for example, a ground surface of an operating room or other room). In other embodiments, the support structure 156 may be releasably securable to the operating room wall 162 or may be a standalone component that is simply supported by the operating room wall 162. In some embodiments, the table 172 may be mounted to the support structure 156. In other embodiments, the table 172 may be releasably mounted to the support structure 156. In still other embodiments, the table 172 may not be attached to the support structure 156. In such embodiments, the table 172 may be supported and / or mounted to an operating room wall, for example. In embodiments where the table 172 is mounted to the support structure 156 (whether detachably mounted or permanently mounted), the table 172 may be mounted to the support structure 156 such that a pose of the table 172 relative to the support structure 156 is selectively adjustable, such as to change a distance between the patient 170 and the radiation source 138.

[0070] The table 172 may be any operating table configured to support the patient 170 during a surgical procedure. The table 172 may include any accessories mounted to or otherwise coupled to the table 172 such as, for example, a bed rail, a bed rail adaptor, an arm rest, an extender, or the like. The table 172 may be stationary or may be operable to maneuver a patient (e.g., the table 172 may be able to move). In some embodiments, the table 172 has two positioning degrees of freedom and one rotational degree of freedom, which allows positioning of the specific anatomy of the patient 170 anywhere in space (within a volume defined by the limits of movement of the table 172). For example, the table 172 can slide forward and backward and from side to side, and can tilt (e.g., around an axis positioned between the head and foot of the table 172 and extending from one side of the table 172 to the other) and / or roll (e.g., around an axis positioned between the two sides of the table 172 and extending from the head of the table 172 to the foot thereof). In other embodiments, the table 172 can bend at one or more areas (which bending may be possible due to, for example, the use of a flexible surface for the table 172, or by physically separating one portion of the table 172 from another portion of the table 172 and moving the two portions independently).A0013282 In at least some embodiments, the table 172 may be manually moved or manipulated by, for example, a surgeon or other user, or the table 172 may comprise one or more motors, actuators, and / or other mechanisms configured to enable movement and / or manipulation of the table 172 by a processor such as the processor 104.

[0071] The imaging platform 126 comprises a gantry. The gantry may be or comprise a substantially circular, or “O-shaped,” housing that enables imaging of objects placed into the isocenter of the housing. In other words, the gantry may be positioned around the object (e.g., patient 170) being imaged. In some cases, the gantry may be disposed at least partially within the upper wall 160, the sidewalls 168 A, 168B, and the lower wall 164 of the support structure 156.

[0072] The imaging platform 126 also comprises the radiation source 138 and the radiation detector 140. The radiation source 138 may be positioned a distance 176 away from the patient 170. In some embodiments, both the radiation source 138 and the radiation detector 140 may be moveable via the imaging platform 126. For example, the support structure 156 may be rotatable, and may rotate to change the position of the radiation source 138 and the radiation detector 140. In some embodiments, the radiation source 138 and the radiation detector 140 may be movable by a gantry, such that the radiation source 138 and the radiation detector 140 can be rotated 360 degrees around the table 172 (and around any patient on the table 172). Additionally or alternatively, the imaging platform 126 and / or components thereof, such as the gantry that positions the radiation source 138 and the radiation detector 140, can be moved laterally along the table 172 (e.g., in a direction of the arrow 178 in Fig. ID), such that a full body scan of the patient can be captured.

[0073] In some embodiments, the depth camera 152 may be disposed in a known position relative to the radiation source 138 and / or the radiation detector 140. In such embodiments, coordinates associated with the depth camera 152, the radiation source 138, and / or the radiation detector 140 may be registered into a common coordinate system. Additionally or alternatively, coordinates associated with the depth camera 152 may be registered into a coordinate system associated with the radiation source 138 and / or the radiation detector 140, or vice versa. Based on the known distances between the depth camera 152, the radiation source 138, and the radiation detector 140, the navigation system 118 may be able to maneuver the depth camera 152, the radiation source 138, and the radiation detector 140 relative to one another while avoiding collisions therebetween (e.g., based on motion trajectories generated by the motion trajectory algorithm 154). In some embodiments, the depth camera 152 may capture a first image in a first pose, while the radiation source 138 is disposed in a second pose different from the first pose. In such cases, the depth camera 152A0013282 may be moved out of the first pose after capturing the first image, and the radiation source 138 may be moved into the first pose.

[0074] The system 100 or similar systems may be used, for example, to carry out one or more aspects of the method 300 described herein. The system 100 or similar systems may also be used for other purposes.

[0075] With reference to Figs. 2A-2B, depictions of live video feeds of a patient with an imaging field of view and stereotactic frame overlay are shown in accordance with embodiments of the present disclosure. Fig. 2A depicts a lateral view 204 of a patient 270 while Fig. 2B depicts an AP view 208 of the patient 270. In some cases, the lateral view 204 and / or the AP view 208 may correspond to live video feeds of the patient 270 generated using a camera (e.g., depth camera 152) that are rendered to a display (e.g., user interface 110).

[0076] In some examples, a field of view 210 and a localizer pattern overlay 220 may be overlaid on the lateral view 204. The field of view 210 may correspond to the field of view of the radiation source 138, and the localizer pattern overlay 220 may be similar to or the same as the stereotactic frame information 142. In other words, the live video feed provided by the depth camera 152 may be modified to include a virtual depiction of the field of view of the radiation source 138 and an outline of the stereotactic frame 148 from a lateral view (e.g., a Z-shaped pattern is displayed). The field of view 210 may be a visual representation of the portion of the patient 270 that will receive radiation from the radiation source 138, while the localizer pattern overlay 220 may be a visual representation of the desired or ideal location of the stereotactic frame 148 relative to the field of view 210. In one example, the localizer pattern overlay 220 may be positioned and / or confined within the field of view 210 (e.g., the field of view 210 encompasses a portion or the entirety of the localizer pattern overlay 220). In some cases, the display algorithm 132 may be used by the processor 104 to generate and overlay the field of view 210 and the localizer pattern overlay 220 on the live feed of the depth camera 152.

[0077] The AP view 208 may additionally or alternatively be modified with a field of view 214 and the localizer pattern overlay 220. The field of view 214 may provide a visual representation of the portion of the patient 270 that will receive radiation from the radiation source 138 when the radiation source 138 is placed above the patient 270 to generate an AP image, and the localizer pattern overlay 220 may depict a desired or ideal position of the stereotactic frame relative to the field of view 214 of the radiation source 138. The display algorithm 132 may in some examples be used by the processor 104 to generate the AP view 208 (e.g., the display algorithm 132 is used to overlay the field of view 214 and the localizer pattern overlay 220 on the live feed of the depth camera 152).A0013282

[0078] With reference to Figs. 2C-2D, depictions of live video feeds of the patient 270 with a stereotactic frame and an imaging field of view and stereotactic frame overlay are shown in accordance with embodiments of the present disclosure. Fig. 2C depicts a lateral view 204 of the patient 270 while Fig. 2D depicts an AP view 208 of the patient 270. In some cases, the lateral view 204 and / or the AP view 208 may correspond to live video feeds of the patient 270 generated using a camera (e.g., depth camera 152) that are rendered to a display (e.g., user interface 110). In some cases, the lateral view 204 and the AP view 208 may be similar to those depicted in Figs. 2A-2B, but with the addition of a stereotactic frame 248 that is attached to the patient 270. The stereotactic frame 248 may be similar to or the same as the stereotactic frame 148 in some examples.

[0079] In some examples, the stereotactic frame 248 may be positioned on the patient 270 and the patient 270 may be imaged to facilitate registration. For example, images of the patient 270 and the stereotactic frame 248 may be captured by the radiation source 138 and radiation detector 140, and the known structure of the stereotactic frame 248 may be used to register the patient 270 to a known coordinate system and / or to register preoperative images of the patient 270 to intraoperative images of the patient 270.

[0080] The lateral view 204 depicts a live video feed of the patient 270 with the stereotactic frame 248 as seen by a camera (e.g., depth camera 152). The live video feed is also modified with the localizer pattern overlay 220 and the field of view 210. The modified live video feed may be rendered to a display (e.g., user interface 110) to enable a user to visualize the positioning of the depth camera 152 relative to the patient 270, as well as an ideal position of the depth camera 152 based on the position of the localizer pattern overlay 220 relative to the actual position of the stereotactic frame 248. In some examples, the depth camera 152 may initially be unaligned with the patient 270, as depicted by a difference in position of the stereotactic frame 248 seen by the depth camera 152 and the virtual depiction of the localizer pattern overlay 220. In this unaligned state, if the radiation source 138 were moved into the position occupied by the depth camera 152, an image captured by the radiation source 138 would not be sufficient for registration purposes, since the image would not capture enough features of the stereotactic frame 248.

[0081] To compensate for the misalignment, the depth camera 152 may be moved relative to the patient 270 to change the view of the depth camera 152 until the depth camera 152 is considered aligned with the patient 270. The change in view may result in the localizer pattern overlay 220 moving relative to the stereotactic frame 248. In some examples, the depth camera 152 may be moved by the user, while in other examples the depth camera 152 may be automatically moved by the processor 104 (e.g., using motors in a gantry). In someA0013282 cases, the processor 104 may use the feature detection algorithm 146 to detect the stereotactic frame 248 (e.g., by detecting the Z-shaped pattern created by the crossbars of the stereotactic frame 248) and determine a movement of the depth camera 152 that reduces the difference between the position of the stereotactic frame 248 and the depiction of the localizer pattern overlay 220.

[0082] Once the depth camera 152 and the stereotactic frame 248 are aligned, which may be determined by the feature detection algorithm 146 based on the difference between the features of the stereotactic frame 248 and the features of the localizer pattern overlay 220 falling below a threshold value, the feature detection algorithm 146 may generate an alert that indicates the same. In some cases, the pose of the depth camera 152 may be tracked by the navigation system 118. The tracked pose of the depth camera 152 may be provided as an input to the motion trajectory algorithm 154, which may use the pose of the depth camera 152 in the aligned position to determine a corresponding movement of the radiation source 138. In one example, the trajectory may correspond to a movement of the radiation source 138 into the same pose as the depth camera 152. In other words, the motion trajectory algorithm 154 may determine a movement of the radiation source 138 such that the field of view 210 corresponds to the actual field of view of the radiation source 138 after the movement has occurred. In such cases, the motion trajectory algorithm 154 may generate another movement trajectory for the depth camera 152 to move the depth camera 152 to avoid collision with the radiation source 138.

[0083] In some cases, the alignment process may be repeated for the AP view 208. In other words, the depth camera 152 that views the patient 270 from above may be moved until the localizer pattern overlay 220 is overlaid with the stereotactic frame 248, and the motion trajectory algorithm 154 may generate a motion trajectory that moves the radiation source 138 with a field of view matching the field of view 214 into the corresponding pose of the depth camera 152 to capture an AP image of the patient 270 that can be used in registration.

[0084] Fig. 3 depicts a method 300 that may be used, for example, to align a radiation source with a patient and a localizer to generate images of the patient and the localizer that can be used for registration.

[0085] The method 300 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as a robot 114) or part of a navigation system (such as a navigation system 118). A processor other than any processor described herein may also be used to execute the method 300. The at least one processor may performA0013282 the method 300 by executing elements stored in a memory such as the memory 106. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 300. One or more portions of a method 300 may be performed by the processor executing any of the contents of memory, such as image processing 120, segmentation 122, transformation 124, registration 128, one or more display algorithms 132, one or more feature detection algorithms 146, and / or one or more motion trajectory algorithms 154.

[0086] The method 300 comprises receiving image data depicting an anatomy of a patient (step 304). The image data may correspond to camera images (e.g., a live video feed) generated by the depth camera 152. The anatomy of the patient may correspond to anatomy that is to receive radiation from a radiation source such as the radiation source 138. In some cases, the image data may be or comprise lateral and anterior-posterior images captured by the depth camera 152 while the patient is positioned in an imaging platform such as the imaging platform 126. The image data may also comprise information about a stereotactic frame (e.g., stereotactic frame 148, stereotactic frame 248, etc.) attached to the patient. The image data may be stored in the memory 106 and / or the database 130.

[0087] The method 300 also comprises rendering, to a display, the image data, field of view information, and stereotactic frame information (step 308). The image data may be rendered to a display (e.g., user interface 110). The field of view information (e.g., field of view 210, field of view 214, etc.) and stereotactic frame information (e.g., stereotactic frame information 142, localizer pattern overlay 220, etc.) may also be overlaid on the image data. In one example, the display algorithm 132 may be used to render the image data, the field of view information, and the stereotactic frame information to the display. The field of view information may correspond to a field of view of the radiation source 138 when the radiation source 138 is positioned in the same pose as the depth camera that generates the live video feed. The stereotactic frame information may correspond to an optimal or desired position of the stereotactic frame relative to the field of view when the depth camera is aligned with the patient.

[0088] The method 300 also comprises identifying, based on the image data, a stereotactic frame (step 312). The identifying may be performed by the feature detection algorithm 146, which may use segmentation 122 to identify the stereotactic frame within the image data. The feature detection algorithm 146 may identify an outline of the stereotactic frame and / or identify one or more features (e.g., crossbars forming a Z-shaped pattern) of the stereotactic frame.A0013282

[0089] The method 300 also comprises generating, when the stereotactic frame depicted in the image data is aligned with a pattern overlay of the stereotactic frame, an alert (step 316). In some cases, the feature detection algorithm 146 may be used to detect when the difference between the stereotactic frame in the image data and the pattern overlay (e.g., localizer pattern overlay 220) of the stereotactic frame falls below a threshold value. The camera generating the image data may be moved (e.g., by a user or automatically by a processor actuating one or more motors of a gantry) to change the position of the pattern overlay relative to the depiction of the stereotactic frame. Once the difference falls below the threshold value, the feature detection algorithm 146 may generate an audio and / or visual alert (e.g., an indicator rendered to the user interface 110) that indicates the depth camera 152 is aligned relative to the stereotactic frame.

[0090] The method 300 also comprises determining, based on the pattern overlay and the image data, a motion trajectory of a radiation source to align the stereotactic frame within a field of view of a radiation source (step 320). Based on the generating of the alert in the step 316, the motion trajectory algorithm 154 may determine the motion trajectory of the radiation source (e.g., radiation source 138) to align the stereotactic frame with the field of view of the radiation source. In one example where the radiation source is configured to capture a lateral view of the patient and the stereotactic frame, the motion trajectory algorithm 154 may determine a movement of the radiation source to position the radiation source in the same pose as the camera (which was aligned with the patient and stereotactic frame in the step 316). In this example, the motion trajectory algorithm 154 may determine the pose of the camera and the radiation source, determine a difference between the poses, and generate an output that actuates one or more motors in the gantry to move the camera out of the aligned pose and move the radiation source into the aligned pose previously occupied by the camera. In this way, the radiation source can be automatically aligned with the stereotactic frame to capture one or more images of the patient and stereotactic frame.

[0091] The method 300 also comprises imaging the stereotactic frame and registering, based on the image, the patient to the radiation source (step 324). Once the radiation source is aligned with the stereotactic frame, the radiation source may capture one or more images of the stereotactic frame and the patient anatomy. Since the radiation source is aligned with the stereotactic frame, the image generated by the radiation source may include sufficient detail of the stereotactic frame (e.g., the image may depict a sufficient shape or number of crossbars or other features of the stereotactic frame) to enable the patient anatomy to be localized. The patient anatomy may be registered to one or more known coordinate systems. Additionally orA0013282 alternatively, the localized patient anatomy may enable preoperative patient images to be registered to intraoperative patient images (e.g., using registration 128).

[0092] In some examples, the step 324 may comprise controlling a robot based on the registration. For example, after the patient has been registered to the radiation source, the navigation system 118 may navigate the robot (e.g., robot 114) and / or a tool attached thereto to carry out a surgical procedure based on the registration. In some cases, navigation markers attached to the robot may be tracked by the navigation system 118 and used, along with the registration between the robot and the patient anatomy, to navigate the robot and / or the tool attached thereto relative to the patient.

[0093] The present disclosure encompasses embodiments of the method 300 that comprise more or fewer steps than those described above, and / or one or more steps that are different than the steps described above.

[0094] As noted above, the present disclosure encompasses methods with fewer than all of the steps identified in Fig. 3 (and the corresponding description of the method 300), as well as methods that include additional steps beyond those identified in Fig. 3 (and the corresponding description of the method 300). The present disclosure also encompasses methods that comprise one or more steps from one method described herein, and one or more steps from another method described herein. Any correlation described herein may be or comprise a registration or any other correlation.

[0095] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer- readable medium and executed by a hardware-based processing unit. Alternatively or additionally, functions may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (alone or in combination with instructions). Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0096] The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and / or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and / or configurations of the disclosure may be combined in alternate aspects, embodiments, and / orA0013282 configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and / or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

[0097] Moreover, though the foregoing has included description of one or more aspects, embodiments, and / or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, embodiments, and / or configurations to the extent permitted, including alternate, interchangeable and / or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.

[0098] The techniques of this disclosure may also be described in the following examples.

[0099] Example 1. A system, comprising:a processor (104); and a memory (106) storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive a video feed from an imaging device (112) depicting an anatomy of a patient (170); overlay the video feed with a virtual depiction of a localizer frame (148); detect, in the video feed, the localizer frame (148); detect when a difference between the virtual depiction of the localizer frame (148) overlaid on the localizer frame (148) depicted in the video feed falls below a threshold value; and generate, when the difference falls below the threshold value, an alert.Example 2. The system according to example 1, wherein the data, when processed by the processor (104), further enable the processor (104) to: overlay, on the video feed, a virtual depiction of a field of view of a radiation source (138).

[0100] Example 3. The system according to example 2, wherein the imaging device (112) comprises a depth camera.

[0101] Example 4. The system according to any of examples 2-3, wherein a movement of the imaging device (112) results in a change in position of the virtual depiction of the localizer frame (148) and the virtual depiction of the field of view of the radiation source (138).

[0102] Example 5. The system according to any of examples 2-4, wherein the virtual depiction of the localizer frame (148) is positioned within the virtual depiction of the field of view of the radiation source (138).A0013282

[0103] Example 6. The system according to any of examples 1-5, wherein the data, when processed by the processor (104), further enable the processor (104) to: determine, when the difference falls below a threshold value, a pose of the imaging device (112); generate, based on the pose of the imaging device (112), a motion trajectory of a radiation source (138) to move the radiation source (138) into the pose; and cause the radiation source (138) to move along the motion trajectory and into the pose.

[0104] Example 7. The system according to example 6, wherein the data, when processed by the processor (104), further enable the processor (104) to: capture, when the radiation source (138) is in the pose, an image of the localizer frame (148); and register, using the image of the localizer frame (148), the patient to the radiation source (138).

[0105] Example 8. The system according to any of examples 1-7, wherein detecting the localizer frame (148) comprises identifying at least one of an outline of the localizer frame (148) and a feature of the localizer frame (148).

[0106] Example 9. A system, comprising: a depth camera (152); a processor (104); and a memory (106) storing data thereon that, when executed by the processor (104), enable the processor (104) to: receive, from the depth camera (152), a video feed of a patient (170) and a stereotactic frame (148) connected to the patient (170); overlay, on the video feed, a virtual depiction of the stereotactic frame (148) and a virtual depiction of a field of view of a radiation source (138); detect, in the video feed, the stereotactic frame (148); determine a difference between a position of the virtual depiction of the stereotactic frame (148) and the stereotactic frame (148); and generate, when the difference falls below a threshold value, an alert.

[0107] Example 10. The system according to example 9, wherein a movement of the depth camera (152) results in a change in position of the virtual depiction of the stereotactic frame (148) and the virtual depiction of the field of view.

[0108] Example 11. The system according to any of examples 9-10, wherein the virtual depiction of the stereotactic frame (148) is positioned within the virtual depiction of the field of view of the radiation source (138).

[0109] Example 12. The system according to any of examples 9-11, wherein the data, when processed by the processor (104), further enable the processor (104) to: determine, when the difference falls below a threshold value, a pose of the depth camera (152); generate, based on the pose of the depth camera (152), a motion trajectory of the radiation source (138) to move the radiation source (138) into the pose; and cause the radiation source (138) to move along the motion trajectory and into the pose.A0013282

[0110] Example 13. The system according to example 12, wherein the depth camera (152) is moved out of the pose as the radiation source (138) is moved into the pose.

[0111] Example 14. The system according to example 12, wherein the data, when processed by the processor (104), further enable the processor (104) to: capture, using the radiation source (138), an image of the stereotactic frame (148); and register, using the image of the stereotactic frame (148), the patient to the radiation source (138).

[0112] Example 15. The system according to any of examples 9-14, wherein detecting the stereotactic frame (148) comprises identifying at least one of an outline of the stereotactic frame (148) and a feature of the stereotactic frame (148).

[0113] Example 16. A system, comprising: a processor (104); and a memory (106) storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive, from a depth camera (152), a video feed of a patient (170) and a stereotactic frame (148) connected to the patient (170); overlay, on the video feed, a virtual depiction of the stereotactic frame (148) and a virtual depiction of a field of view of a radiation source (138); detect, in the video feed, the stereotactic frame (148); determine a difference between a position of the virtual depiction of the stereotactic frame (148) and the stereotactic frame (148); and generate, when the difference falls below a threshold value, a motion trajectory of the radiation source (138) to align the radiation source (138) with the stereotactic frame.

[0114] Example 17. The system according to example 16, wherein generating the motion trajectory of the radiation source (138) comprises determining a pose of the depth camera (152), and wherein the data, when processed by the processor (104), further enable the processor (104) to: cause the radiation source (138) to move into the pose.

[0115] Example 18. The system according to example 17, wherein the data, when processed by the processor (104), further enable the processor (104) to: capture, using the radiation source (138), an image of the stereotactic frame (148); and register, using the image of the stereotactic frame (148), the patient (170) to the radiation source (138).

[0116] Example 19. The system according to any of examples 16-18, wherein detecting the stereotactic frame (148) comprises identifying at least one of an outline of the stereotactic frame (148) and a feature of the stereotactic frame (148).

[0117] Example 20. The system according to example 19, wherein the virtual depiction of the Stereotactic frame (148) is positioned at least partially within the virtual depiction of the field of view of the radiation source (138).

Claims

A0013282CLAIMSWhat is claimed is:

1. A system, comprising:a processor (104); anda memory (106) storing data thereon that, when processed by the processor (104), enable the processor (104) to:receive a video feed from an imaging device (112) depicting an anatomy of a patient (170);overlay the video feed with a virtual depiction of a localizer frame (148); detect, in the video feed, the localizer frame (148);detect when a difference between the virtual depiction of the localizer frame (148) overlaid on the localizer frame (148) depicted in the video feed falls below a threshold value; and generate, when the difference falls below the threshold value, an alert.

2. The system according to claim 1, wherein the data, when processed by the processor (104), further enable the processor (104) to:overlay, on the video feed, a virtual depiction of a field of view of a radiation source (138).

3. The system according to claim 2, wherein the imaging device (112) comprises a depth camera.

4. The system according to any of claims 2-3, wherein a movement of the imaging device (112) results in a change in position of the virtual depiction of the localizer frame (148) and the virtual depiction of the field of view of the radiation source (138).

5. The system according to any of claims 2-4, wherein the virtual depiction of the localizer frame (148) is positioned within the virtual depiction of the field of view of the radiation source (138).

6. The system according to any of claims 1-5, wherein the data, when processed by the processor (104), further enable the processor (104) to:determine, when the difference falls below a threshold value, a pose of the imaging device (112);A0013282 generate, based on the pose of the imaging device (112), a motion trajectory of a radiation source (138) to move the radiation source (138) into the pose; andcause the radiation source (138) to move along the motion trajectory and into the pose.

7. The system according to claim 6, wherein the data, when processed by the processor (104), further enable the processor (104) to:capture, when the radiation source (138) is in the pose, an image of the localizer frame (148); andregister, using the image of the localizer frame (148), the patient to the radiation source (138).

8. The system according to any of claims 1-7, wherein detecting the localizer frame (148) comprises identifying at least one of an outline of the localizer frame (148) and a feature of the localizer frame (148).

9. A system, comprising:a depth camera (152);a processor (104); anda memory (106) storing data thereon that, when executed by the processor (104), enable the processor (104) to:receive, from the depth camera (152), a video feed of a patient (170) and a stereotactic frame (148) connected to the patient (170);overlay, on the video feed, a virtual depiction of the stereotactic frame (148) and a virtual depiction of a field of view of a radiation source (138);detect, in the video feed, the stereotactic frame (148);determine a difference between a position of the virtual depiction of the stereotactic frame (148) and the stereotactic frame (148); andgenerate, when the difference falls below a threshold value, an alert.

10. The system according to claim 9, wherein a movement of the depth camera (152) results in a change in position of the virtual depiction of the stereotactic frame (148) and the virtual depiction of the field of view.A0013282 11. The system according to any of claims 9-10, wherein the virtual depiction of the stereotactic frame (148) is positioned within the virtual depiction of the field of view of the radiation source (138).

12. The system according to any of claims 9-11, wherein the data, when processed by the processor (104), further enable the processor (104) to:determine, when the difference falls below a threshold value, a pose of the depth camera (152);generate, based on the pose of the depth camera (152), a motion trajectory of the radiation source (138) to move the radiation source (138) into the pose; andcause the radiation source (138) to move along the motion trajectory and into the pose.

13. The system according to claim 12, wherein the depth camera (152) is moved out of the pose as the radiation source (138) is moved into the pose.

14. The system according to claim 12, wherein the data, when processed by the processor (104), further enable the processor (104) to:capture, using the radiation source (138), an image of the stereotactic frame (148); and register, using the image of the stereotactic frame (148), the patient to the radiation source (138).

15. The system according to any of claims 9-14, wherein detecting the stereotactic frame (148) comprises identifying at least one of an outline of the stereotactic frame (148) and a feature of the stereotactic frame (148).

Images