Surgical robotic and spatial measurement system

Abstract

Surgical guidance systems, platforms, robots, and methods for spatial measurement are provided. The method includes receiving information associated with a surgical task. The method also includes determining operating parameters to perform the surgical task. The method further includes transferring a control signal to control a surgical robot to perform the surgical task. The method includes receiving image data of a surgical environment during the surgical task. The method also includes tracking a location of each surgical component within the surgical environment during the surgical task. The method further includes monitoring performance of the surgical task to ensure operation is according to the determined operating parameters.

Description

SURGICAL ROBOTIC AND SPATIAL MEASUREMENT SYSTEMCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 669,458, filed 10 July 2024, the entire content of which is incorporated herein by reference.BACKGROUND

[0002] The present disclosure is generally directed to surgical systems, and relates more particularly to robotic surgical systems.

[0003] Surgical robots may assist a surgeon or other medical provider in carrying out a surgical procedure, or may complete one or more surgical procedures autonomously. Imaging may be used by a medical provider for diagnostic and / or therapeutic purposes. Patient anatomy can change over time, particularly following placement of a medical implant in the patient anatomy.BRIEF SUMMARY

[0004] Example aspects of the present disclosure include:

[0005] A system for spatial measurement, including a processor; and a memory storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

[0006] A system including a robot; a processor; and a memory storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

[0007] A method including receiving information associated with a surgical task; determining operating parameters to perform the surgical task; transferring a control signal to control a surgicalrobot to perform the surgical task; receiving image data of a surgical environment during the surgical task; tracking a location of each surgical component within the surgical environment during the surgical task; and monitoring performance of the surgical task to ensure operation is according to the determined operating parameters.

[0008] Any of the aspects herein, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

[0009] Any of the aspects herein, wherein the location of each surgical component within the surgical environment is tracked by determining the location of each surgical component within the surgical environment relative to other surgical components within the surgical environment and / or a registration marker.

[0010] Any of the aspects herein, wherein the surgical task comprises real-time imaging and the information associated with the surgical task comprises position information for one or more imagers.

[0011] Any of the aspects herein, wherein the surgical task comprises decompression and the information associated with the surgical task comprises neurological output indicating impingement, a location of the decompression, and a type of decompression to be performed, and wherein the data, when processed by the processor, further enable the processor to: monitor the neurological output indicating impingement.

[0012] Any of the aspects herein, wherein the surgical task comprises cutting using at least one of: a mechanical burr, a laser, and an ultrasonic cutting tool.

[0013] Any of the aspects herein, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type and a control signal to hold lock on the retractor.

[0014] Any of the aspects herein, wherein the surgical task comprises tissue removal and the information associated with the surgical task comprises a location of the tissue, a type of tissue to be removed, and a preoperative plan.

[0015] Any of the aspects herein, further comprising displaying in real-time the location of each surgical component within the surgical environment.

[0016] Any aspect in combination with any one or more other aspects.

[0017] Any one or more of the features disclosed herein.

[0018] Any one or more of the features as substantially disclosed herein.

[0019] Any one or more of the features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein.

[0020] Any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments.

[0021] Use of any one or more of the aspects or features as disclosed herein.

[0022] It is to be appreciated that any feature described herein can be claimed in combination with any other feature(s) as described herein, regardless of whether the features come from the same described embodiment.

[0023] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

[0024] The phrases “at least one”, “one or more”, and “and / or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and / or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together. When each one of A, B, and C in the above expressions refers to an element, such as X, Y, and Z, or class of elements, such as XI -Xn, Yl-Ym, and Zl- Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., XI and X2) as well as a combination of elements selected from two or more classes (e.g., Y 1 and Zo).

[0025] The term “a” or “an” entity refers to one or more of that entity. As such, the terms “a” (or “an”), “one or more” and “at least one” can be used interchangeably herein. It is also to be noted that the terms “comprising”, “including”, and “having” can be used interchangeably.

[0026] The preceding is a simplified summary of the disclosure to provide an understanding of some aspects of the disclosure. This summary is neither an extensive nor exhaustive overview of the disclosure and its various aspects, embodiments, and configurations. It is intended neither to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure but to present selected concepts of the disclosure in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other aspects, embodiments, andconfigurations of the disclosure are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.

[0027] Numerous additional features and advantages of the present disclosure will become apparent to those skilled in the art upon consideration of the embodiment descriptions provided hereinbelow.BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are incorporated into and form a part of the specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the disclosure. The drawings simply illustrate preferred and alternative examples of how the disclosure can be made and used and are not to be construed as limiting the disclosure to only the illustrated and described examples. Further features and advantages will become apparent from the following, more detailed, description of the various aspects, embodiments, and configurations of the disclosure, as illustrated by the drawings referenced below.

[0029] Fig. 1 is a block diagram illustrating aspects of a system for performing a surgery or surgical procedure according to at least one embodiment of the present disclosure;

[0030] Fig. 2A is a conceptual diagram of aspects of an imaging device according to at least one embodiment of the present disclosure;

[0031] Fig. 2B is a conceptual diagram of additional aspects of the imaging device according to at least one embodiment of the present disclosure;

[0032] Fig. 3 is a conceptual diagram of aspects of a robot and a navigation system according to at least one embodiment of the present disclosure;

[0033] Fig. 4 is a block diagram of aspects of a computing system according to at least one embodiment of the present disclosure;

[0034] Fig. 5 is a flowchart according to at least one embodiment of the present disclosure;

[0035] Fig. 6 is a flowchart according to at least one embodiment of the present disclosure;

[0036] Fig. 7 is a flowchart according to at least one embodiment of the present disclosure; and

[0037] Fig. 8 is a flowchart according to at least one embodiment of the present disclosure.

[0038] Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of examples, aspects, and features illustrated.

[0039] In some instances, the apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the of various implementations, examples, aspects, and features so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.DETAILED DESCRIPTION

[0040] It should be understood that various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and / or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0041] Before any embodiments of the disclosure are explained in detail, it is to be understood that the disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Further, the present disclosure may use examples to illustrate one or more aspects thereof. Unless explicitly stated otherwise, the use or listing of one or more examples (which may be denoted by “for example,” “by way of example,” “e.g.,” “such as,” or similar language) is not intended to and does not limit the scope of the present disclosure.

[0042] The terms proximal and distal are used in this disclosure with their conventional medical meanings, proximal being closer to the operator or user of the system, and further from the region of surgical interest in or on the patient, and distal being closer to the region of surgical interest in or on the patient, and further from the operator or user of the system.

[0043] The use of robotic surgical systems may benefit or help surgeons achieve clinical goals and objectives. According to at least one embodiment of the present disclosure, an autonomous bone cutting surgical robotic system is provided that implements tissue cutting devices, navigation positioning, robotic control, and advanced imaging. The robotic system may be beneficial in spine procedures as tissue removal is an important step in most spine workflows.

[0044] According to at least one embodiment of the present disclosure, the surgical robotic system may be controlled by one or more inputs. For example, the surgical robotic system may comprise a fully autonomous mechanism driven by closed-loop feedback acquired intraoperatively.

[0045] A surgical robotic system and method that consists of a robotic element, a spatial measurement system, computer processing and storage, and a plurality of instruments that enables a robot to deliver and / or assist in the delivery of medical therapy including but not limited to performing incisions, decompression, and imaging; providing assistance during a surgical task (e.g., holding retractors); inserting implants; etc. The surgical robotic system and method takes various inputs (e.g., information on a surgical tool, preoperative plan, procedure data, image data, sensor information (e.g., force measurements, capacitance measurements, etc.)) to perform a surgical task (e.g., incision, decompression, retraction, implantation, cutting, imaging, etc.)

[0046] For the surgical robot to autonomously perform the surgical tasks, the surgical robot can use spatial measurement methods to visualize the surgical environment. Spatial measurement methods enable the surgical robot to identify, locate, measure, classify, and even reproduce a 3D model of objects. The spatial measurement methods may use data from various sources (e.g., cameras, sensors, etc.) to monitor the surgical environment and performance of the surgical tasks.

[0047] Accordingly to at least one embodiment of the present disclosure, the surgical robotic system may be able to restart or resume with a surgical task in the surgical plan after an automatic or manual stop has been initiated. In other words, the surgical robotic system may begin from where the surgical robotic system “left off.” In some cases, the robotic surgical system may be rewound to prior known positions (e.g., after a robotic arm moves from a first pose to a second pose, the user may be able to cause the robotic arm to move back to the first pose). The robotic surgical system may enable the user to verify the status of the surgery or surgical procedure, visualize the next step in the surgery or surgical procedure (e.g., via a laser pointer showing a cut path), and provide an input that enables the robotic surgical system to proceed with the next step of the surgery or surgical procedure.

[0048] Embodiments of the present disclosure provide technical solutions to one or more of the problems of a surgical robot performing autonomous surgery and / or assisting a user delivering medical therapy.

[0049] With reference to Fig. 1, an example system 150 that supports aspects of the present disclosure is shown according to at least one embodiment of the present disclosure. The system depicted in Fig. 1 includes a computing system 100 that may be in communication with an imaging device 104, a navigation system 108, a robot 112, a user device 116, a database 120, and / or other components. Systems according to other implementations of the present disclosure may include more or fewer components than illustrated in Fig. 1. For example, systems and methods described herein may be implemented while omitting and / or including additional instances of one or more of the computing system 100, the imaging device 104, the robot 112, and / or the user device 116. In another example, the system may omit features such as the imaging device 104. The computing system 100 or similar systems may be used, for example, in conjunction with the imaging device 104, the navigation system 108, the robot 112, the user device 116, the database 120, and / or other components to carry out one or more aspects of any of the methods 500 and / or 600 described herein. The computing system 100 or similar systems may also be used for other purposes.

[0050] In some examples, the computing system 100 may take the form of a computer workstation, handheld computing device, server or other network computing device, or external program that includes a user interface 416 for presenting information to and receiving input from a user. The user (e.g., a physician, technician, clinician, member of surgical staff, etc.) may interact with the computing system 100 via the user device 116 to process images of patients (e.g., images from the imaging device 104) for the purposes of performing or assisting with a surgery or surgical procedure. For instance, the user may interact with the computing system 100 to facilitate surgical ablation(s), implant surgical screws, insert Deep Brain Stimulation (DBS) leads, perform surgical drilling and / or cutting procedures, perform soft tissue removal, combinations thereof, and / or the like.

[0051] The computing system 100 may communicate with the imaging device 104, the navigation system 108, the robot 112, the user device 116, and / or the database 120 via a cloud or other network. The cloud or other network may comprise one or more computing device (not shown), such as one or more non-edge switches, routers, hubs, gateways, security devices such as firewalls, computer terminals, wireless mobile devices (e.g., cellular phones), wireless accesspoints, or other network devices. The cloud or other network may provide computing devices, the computing system 100, the imaging device 104, the navigation system 108, the robot 112, the user device 116, the database 120, and / or other components access to the Internet, and may provide a communication framework that allows the computing devices to communicate with one another. In some cases, the cloud or other network may be a private network that allows the computing system 100, the imaging device 104, the navigation system 108, the robot 112, the user device 116, the database 120, and / or other components to communicate with one another via a wired connection, a wireless connection, or both. In such cases, the communications between the foregoing components may be encrypted.

[0052] The user device 116 may be or comprise a keyboard, mouse, trackball, joystick, monitor, television, screen, touchscreen, and / or any other device for enabling a user to interact with the computing system 100, the imaging device 104, the navigation system 108, the robot 112, the database 120, and / or other components. For example, the user device 116 may enable the user to input information (e.g., instructions) into and / or receive information from the computing system 100, the imaging device 104, the navigation system 108, the robot 112, the database 120, and / or other components. The user may use the user device 116, for example, to select or provide other input regarding any step of any method described herein. Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the computing system 100 (e.g., by the processor 404 or another component of the computing system 100) or received by the computing system 100 from a source external to the computing system 100. In some cases, the user device 116 may be useful to allow a surgeon or other user to modify instructions to be executed by the processor 404 according to one or more embodiments of the present disclosure, and / or to modify or adjust a setting of other information displayed on the user device 116 or corresponding thereto.

[0053] The imaging device 104 may comprise one or more components capable of capturing X- ray, fluoroscopy, Computed Tomography (CT), and / or other images of a patient 102. The captured images may comprise anatomical feature(s) (e.g., bones, veins, nerves, soft tissues, etc.) of the patient 102 and / or other aspects of the anatomy of the patient 102. The imaging device 104 may capture a single, still image of the patient 102, and may additionally or alternatively capture multiple images of the patient 102 continuously to generate a data stream of images that depict movement of the anatomical feature(s) of the patient over time. The imaging device 104 maygenerate two-dimensional (2D) or three-dimensional (3D) images. In some examples, the imaging device 104 may be used to capture one or more preoperative images, one or more intraoperative images, one or more postoperative images, and / or one or more images taken independently of any surgery or surgical procedure.

[0054] The database 120 may store information that correlates one coordinate system to another (e.g., one or more robotic coordinate systems to a patient coordinate system and / or to a navigation coordinate system). The database 120 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about a patient’ s anatomy at and / or proximate the surgical site, for use by the robot 112, the navigation system 108, and / or a user of the computing system 100); one or more images useful in connection with a surgery to be completed by or with the assistance of one or more other components of the computing system 100; and / or any other useful information. The database 120 may be configured to provide any such information to the computing system 100, the imaging device 104, the navigation system 108, the robot 112, and / or to any other device. In some embodiments, the database 120 may be or comprise part of a hospital image storage system, such as a picture archiving and communication system (PACS), a health information system (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0055] The imaging device 104 may be or comprise one or more components capable of performing various imaging modalities. For example, the imaging device 104 may be or comprise an 0-arm, C-arm, G-arm, or any other device capable of utilizing X-ray-based imaging (e.g., a fluoroscope, a CT scanner, an X-ray machine, etc.). As another example, the imaging device 104 may be or comprise an ultrasound scanner (which may comprise a physically separate transducer and receiver or a single ultrasound transceiver) capable of generating information that can be processed to produce an ultrasound image. In other examples, the imaging device 104 may be or comprise a magnetic resonance imaging (MRI) scanner, an optical camera, a thermographic (e.g., infrared) camera, or any other imaging device suitable for obtaining images of anatomical feature(s) of the patient 102. In some cases, the components of the imaging device 104 may be contained entirely wi thing a singular housing, such as when the imaging device 104 comprises an optical camera or other imaging device whose components can be positioned together. Additionally or alternatively, the imaging device 104 may comprise components housed in separate housings,such as when the imaging device 104 comprises a transmitter / emitter positioning in a first housing and a receiver / detector positioned in a second housing such that the two components are physically separated.

[0056] As previously mentioned, the imaging device 104 may comprise more than one imaging device. For example, the imaging device 104 may comprise a first imaging device that provides first image data (e.g., data that can be manipulated by hardware and / or software components of the computing system 100 to generate an image) and / or a first image, and a second imaging device that provides second image data and / or second image. In such examples, the first imaging device and the second imaging device may correspond to components that implement different imaging modalities (e.g., the first imaging device implements X-ray fluoroscopy, while the second imaging device implements ultrasonic waves), which may offer the user of the computing system 100 additional flexibility in imaging the patient 102.

[0057] With reference to Figs. 2A-2B, aspects of an example imaging device 104 are shown in accordance with at least one embodiment of the present disclosure. In the example shown in Figs. 2A-2B, the imaging device 104 may be or comprise an 0-arm or other imaging apparatus capable of imaging anatomical features or other features of the patient 102. The imaging device 104 depicted in Figs. 2A-2B may comprise an upper portion 208 and a lower portion 212 connected by a pair of sidewalls. In some cases, the imaging device 104 may be secured to the ground surface or floor 216 of an operating room or other surgical environment. In other cases, the imaging device 104 may be releasably securable to the floor 216 or may be a standalone component that is supported by the floor 216.

[0058] The patient 102 may be positioned on a table 204 that is positioned orthogonally to and extends at least partially through the isocenter of the imaging device 104, such that the imaging device 104 can image one or more portions of the patient 102. In some cases, the table 204 may be mounted to the imaging device 104. In other cases, table 204 may be releasably mounted to the imaging device 104. In cases where the table 204 is mounted to the imaging device 104 (whether detachably mounted or permanently mounted), the table 204 may be mounted to the imaging device 104 such that a pose of the table 204 relative to the imaging device 104 is selectively adjustable. In still other cases, the table 204 may not be attached to the imaging device 104. In such cases, the table 204 may be supported and / or mounted to an operating room wall, for example.

[0059] The table 204 may be any operating table configured to support the patient 102 during a surgical procedure. The table 204 may include any accessories mounted to or otherwise coupled to the table 204 such as, for example, a bed rail, a bed rail adaptor, an arm rest, an extender, or the like. The table 204 may be stationary or may be operable to maneuver the patient 102 (e.g., the table 204 may be able to move). In some cases, the table 204 has two positioning degrees of freedom and one rotational degree of freedom, which allows positioning of the specific anatomy of the patient anywhere in space (within a volume defined by the limits of movement of the table 204). For example, the table 204 can slide forward and backward and from side to side, and can tilt (e.g., around an axis positioned between the head and foot of the table 204 and extending from one side of the table 204 to the other) and / or roll (e.g., around an axis positioned between the two sides of the table 204 and extending from the head of the table 204 to the foot thereof). In other cases, the table 204 can bend at one or more areas (which bending may be possible due to, for example, the use of a flexible surface for the table 204, or by physically separating one portion of the table 204 from another portion of the table 204 and moving the two portions independently). In at least some examples, the table 204 may be manually moved or manipulated by, for example, a surgeon or other user, or the table 204 may comprise one or more motors, actuators, and / or other mechanisms configured to enable movement and / or manipulation of the table 204 by a processor such as the processor 404.

[0060] The imaging device 104 may also comprise a gantry. The gantry may be or comprise a substantially circular, or “O-shaped,” housing that enables imaging of objects placed into an isocenter of the housing. In other words, the gantry may be positioned around the object being imaged (e.g., the patient 102). In some examples, the gantry may be disposed at least partially within the upper portion 208, the sidewalls, and the lower portion 212 of the imaging device 104.

[0061] The imaging device 104 comprises a source 224 and a detector 228. The source 224 may be or comprise a device configured to generate and emit radiation, and the detector 228 may be or comprise a device configured to detect the emitted radiation. In some examples, the source 224 and the detector 228 may be or comprise an imaging source and an imaging detector (e.g., the source 224 and the detector 228 are used to generate data useful for producing images). The source 224 may be positioned in a first position and the detector 228 may be positioned in a second position opposite the source 224. In some examples, the source 224 may comprise an X-ray source such as, for example, a thermionic emission tube, a cold emission X-ray tube, and / or the like. The source224 may project a radiation beam that passes through the patient 102 and onto the detector 228 located on the opposite side of the patient 102. The detector 228 may be or comprise one or more sensors that receive the radiation beam (e.g., once the radiation beam has passed through the patient 102) and transmit information related to the radiation beam to one or more other components of the computing system 100 for processing, such as to the processor 404. In some examples, the detector 228 may comprise an array. For example, the detector 228 may comprise three 2D flat panel solid-state detectors arranged side-by-side, and angled to approximate the curvature of the imaging device 104. It will be understood, however, that various detectors and detector arrays can be used with the imaging device 104, including any detector configurations used in typical diagnostic fan-beam or cone-beam CT scanners. The source 224 and / or the detector 228 may comprise a collimator 232. The collimator 232 may be configured to confine or shape the radiation beam as the radiation beam is emitted from the source 224 and / or as the radiation beam is received by the detector 228.

[0062] The source 224 and the detector 228 may be attached to the gantry and configured to rotate 360 degrees around the patient 102 in a continuous or step-wise manner so that the radiation beam can be projected through the patient 102 at various angles. In other words, the source 224 and the detector 228 may rotate, spin, or otherwise revolve about an axis that passes through the top and bottom of the patient, with the volume of interest positioned at the isocenter of the imaging device 104. The imaging device 104 comprises a drive mechanism capable of causing the gantry to move such that the source 224 and the detector 228 encircle the patient 102 on the table 204. Additionally or alternatively, the source 224 and the detector 228 may move along a length of the patient 102. For example, the table 204 holding the patient 102 may move in a lateral direction while the source 224 and detector 228 remain in a fixed location, such that the length of the patient can be scanned. At each projection angle in the revolution, the radiation beam passes through and is attenuated by the patient 102. The attenuated radiation is then detected by the detector 228. The detected radiation from each of the projection angles can then be processed, using various reconstruction techniques such as image processing 424, to produce a 2D or 3D reconstruction image of one or more anatomical features of the patient 102. In one example, the processor 404 may use the image processing 424 to generate a 3D cone beam computed tomography (CBCT) reconstruction image.

[0063] In some examples, a switch 220 may be provided to a user (e.g., a physician, a member of surgical staff, etc.) that can be used to control over one or more components of the imaging device 104, the navigation system 108, and / or the robot 112. For example, the switch 220 may comprise a manual override that automatically stops movement of the imaging device 104 and / or the robot 112 when the user actuates the switch 220. In some cases, the switch 220 may be configured to stop movement of the imaging device 104 and / or the robot 112 when the user actuates the switch 220, and to resume movement of the imaging device 104 and / or the robot 112 when the user stops the actuation of the switch 220.

[0064] With reference to Fig. 3, aspects of a robot and navigation system are shown in accordance with at least one embodiment of the present disclosure. In the example shown in Fig. 3, the patient 102 and the robot 112 are both positioned on the table 204 such that the robot 112 and / or the robotic arms 114 can interact with the patient 102 with assistance or guidance from the navigation system 108. While the example depicts the robot 112 positioned on the table 204, the robot 112 and / or the robotic arms 114 may alternatively be positioned on a cart or other surface near the patient 102. In some cases, the robot 112 and / or the robotic arms 114 are mounted or connected to other portions of the table 204 such that movement of the robot 112 and / or the robotic arms 114 relative to the table 204 is reduced or minimized.

[0065] The robot 112 may be any surgical robot or surgical robotic system. The robot 112 may be or comprise, for example, the Mazor X™ Stealth Edition robotic guidance system. The robot 112 may be configured to manipulate a surgical tool (whether based on guidance from the navigation system 108 or not) to accomplish or to assist with a surgical task. In some examples, the robot 112 may be configured to hold and / or manipulate an anatomical element during or in connection with a surgical procedure. The robot 112 may comprise one or more robotic arms 114. In some examples, the robotic arm 114 may comprise a first robotic arm and a second robotic arm, though the robot 112 may comprise more than two robotic arms. Each robotic arm 114 may be positionable independently of the other robotic arm 114. The robotic arms 114 may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

[0066] In some examples, one or more of the robotic arms 114 may be used to hold and / or maneuver the imaging device 104. In examples where the imaging device 104 comprises two or more physically separate components (e.g., a transmitter and receiver), one robotic arm 114 may hold one such component, and another robotic arm 114 may hold another such component. Therobot 112 may be configured to position the imaging device 104 at one or more precise position(s) and orientation(s), and / or to return the imaging device 104 to the same position(s) and orientation / s) at a later point in time.

[0067] The robot 112, together with the robotic arm 114, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm 114 may be positioned or positionable in any pose, plane, and / or focal point. The pose includes a position and an orientation. As a result, an imaging device 104, surgical tool, or other object held by the robot 112 (or, more specifically, by the robotic arm 114) may be precisely positionable in one or more needed and specific positions and orientations. The robotic arm(s) 114 may comprise one or more sensors, markers, and / or the like (e.g., navigation markers 320A-320N) that enable a processor 404 (or a processor of the robot 112) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm).

[0068] In various embodiments, the navigation system 108 may be used to track a position and orientation (e.g., a pose) of the imaging device 104, the robot 112 and / or robotic arm 114, and / or one or more surgical tools (or, more particularly, to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the foregoing). The navigation system 108 may include a display for displaying one or more images from an external source (e.g., the computing system 100, imaging device 104, or other source) or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 108. The navigation system 108 may be configured to provide guidance to a surgeon or other user of the computing system 100 or a component thereof, to the robot 112, and / or the like regarding, for example, a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

[0069] In some cases and as discussed in further detail below, reference markers (e.g., navigation markers) may be placed on the robot 112 (including, e.g., on the robotic arm 114), the imaging device 104, or any other object in the surgical space. The reference markers may be tracked by the navigation system 108, and the results of the tracking may be used by the robot 112 and / or by an operator of the computing system 100 or any component thereof. In some embodiments, the navigation system 108 can be used to track other components of the system (e.g., imaging device 104) and the system can operate without the use of the robot 112 (e.g., with the surgeon manuallymanipulating the imaging device 104 and / or one or more surgical tools, based on information and / or instructions generated by the navigation system 108, for example).

[0070] The navigation system 108 may provide navigation for a surgeon and / or a surgical robot during an operation. The navigation system 108 may be any now-known or future -developed navigation system, including, for example, the Medtronic StealthStation™ S8 surgical navigation system or any successor thereof. The navigation system 108 may include one or more cameras (e.g., a navigation camera 324) or other sensor(s) for tracking one or more reference markers, navigated trackers (e.g., navigation markers 320A-320N), or other objects (e.g., optical localizer(s) 308, electromagnetic localizer(s) 316, etc.) within the operating room or other room in which some or all of the navigation system 108 is located. The one or more navigation cameras 324 may be optical cameras, infrared cameras, or other cameras.

[0071] In some cases, the navigation system 108 may comprise a depth sensor (e.g., a sensor positioned within the navigation camera 324) that can identify one or more gestures associated with the physician and / or other individuals. The depth sensor may provide a continuous or live feed of captured gestures to the computing system 100, where the processor 404 may interpret or process the captured gestures (e.g., using image processing 424) for the purposes of controlling the imaging device 104, the robot 112, and / or other components. In some cases, the processor 404 may use gesture recognition software (e.g., OpenPose or other software capable of capturing human hand gestures or other movements) to process the information captured by the depth sensor to determine one or more gestures. The gesture recognition software may in some examples use pose estimation of various portions of the bodies of multiple individuals to determine the relative movements or gestures of the individuals.

[0072] In one example, the navigation system 108 implements optical tracking to facilitate navigation. The optical tracking may comprise the navigation system 108 using an optical marker 304 and navigation markers 320A-320N to track objects within the surgical environment. The optical marker 304 may be positioned relative to the patient (e.g., on the table 204) and the navigation markers 320A-320N may be positioned relative to one or more objects (e.g., the robot 112 and / or the robotic arms 114) within and tracked by the navigation camera 324. The optical marker 304 and / or the navigation markers 320A-320N may be or comprise optical fiducials, reflective surfaces, and / or the like capable of being detected in images generated by the imaging device 104. Additionally or alternatively, the optical marker 304 and / or the navigation markers320A-320N may be identifiable real-time by the imaging device 104, such as in examples where the imaging device 104 provides a live feed of components within the view of the imaging device 104. Based on the information captured by the imaging device 104, the navigation system 108 may identify the optical markers and use the marker location to determine the pose of the optical markers in an optical coordinate system. The navigation system 108 may then navigate one or more surgical tools (e.g., relative to the optical localizer 308 whose position is known or determined in one or more coordinate systems).

[0073] In another example, the navigation system 108 implements electromagnetic tracking to facilitate navigation. In some cases, the navigation system 108 may implement both electromagnetic and optical tracking (e.g., via co-registration of optical and electromagnetic localizers). The electromagnetic tracking may include the navigation system 108 using an electromagnetic field emitter 312 and an electromagnetic localizer 316 to track objects within the surgical environment. The electromagnetic field emitter 312 generates an electromagnetic field in which one or more components are positioned. The electromagnetic field emitter 312 may generate a constant electromagnetic field, or may alternatively emit a time-variant electromagnetic field. In some examples, the electromagnetic field emitter 312 may comprise a plurality of emitters each configured to generate slightly different electromagnetic fields. The presence of multiple electromagnetic fields may enable multiplexed sensing of the electromagnetic fields to determine the location of objects in the surgical environment.

[0074] In some cases, the electromagnetic field emitter 312 may be positioned proximate to the patient 102 (e.g., positioned underneath the patient, positioned next to the patient, positioned within the table 204, etc.). For example, the patient 102 may lie on a pad, pillow, or other support containing the electromagnetic field emitter 312. In some examples, the known pose of the electromagnetic field emitter 312 may enable the computing system 100 to register the electromagnetic field emitter 312 to one or more other coordinate systems using, for example, registration 436.

[0075] The electromagnetic field generated and emitted by the electromagnetic field emitter 312 may be detected by one or more sensors positioned within the surgical environment such that the navigation system 108 can perform electromagnetic tracking of the sensors. For instance, the electromagnetic localizer 316 may be positioned within the electromagnetic field generated by the electromagnetic field emitter 312. In some cases, the electromagnetic localizer 316 may bepositioned near the anatomical elements 302A-302N of the patient 102. It is to be understood that, while a single electromagnetic localizer 316 is depicted in Fig. 3, an additional number of localizers or electromagnetic sensors may be present in the surgical environment. The electromagnetic localizer 316 may comprise one or more electromagnetic sensors or other similar devices capable of measuring aspects of the electromagnetic field (e.g., magnitude of the electromagnetic field, direction of the electromagnetic field, etc.). The sensor measurements may be sent to the computing system 100 (e.g., stored in a memory 408 as sensor information 440) and / or to the database 120. The measurements may be processed by the processor 404 to determine the pose of the electromagnetic localizer 316 relative to an electromagnetic coordinate system. The pose of the electromagnetic localizer 316 may also be known in one or more other coordinate systems (e.g., a patient coordinate system), such that the processor 404 can use registration 436 to register the electromagnetic coordinate system with the patient coordinate system. Based on the registration, the navigation system 108 may use the pose of the electromagnetic localizer 316 and subsequent readings therefrom to perform tracking of one or more anatomical elements, to help navigate one or more surgical tools relative to the patient 102, combinations thereof, and / or the like.

[0076] In some cases, the optical tracking can include using optical markers (e.g., reference markers) to calibrate and register a location of each surgical component within the surgical environment. The optical markers may be placed within the surgical environment (e.g., on stationary parts, surgical tools, etc.). The optical markers may be tracked by the navigation system 108 and / or navigation module 452, and the results of the tracking may be used by the system 100 and / or by an operator of the system 100 or any component thereof. In some embodiments, the navigation system 108 and / or navigation module 452 can be used to monitor the markers / surgical hardware components, or other components of the system.

[0077] The navigation system 108 and / or navigation module 452 may provide navigation for a surgeon and / or a surgical robot during an operation. The navigation system 108 and / or navigation module 452 may be any now-known or future-developed navigation system, including, for example, the Medtronic StealthStationTM S8 surgical navigation system or any successor thereof. The navigation system 108 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigation trackers, or other objects within the environment in which some or all of the system 100 is located.Y1

[0078] The navigation system 108 may include one or more cameras, which may be optical cameras, infrared cameras, or other cameras RF. In various embodiments, the navigation system 108 may be used to track a position and orientation of surgical components (the robot, position and location of surgical tools, portions of the patient anatomy, markers, etc.). In embodiments, the navigation module 452 may receive data from one or more cameras or other sensor(s).

[0079] The navigation system 108 may include a display for displaying one or more images from an external source (e.g., the user device 116, imaging device 104, or other source) or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 108. The navigation system 108 may be configured to provide guidance to a surgeon or other user of the system 100 or a component thereof, to the robot 114, or to any other element of the system 100 regarding, for example, location of the robot, surgical tools, portions of the patient anatomy, markers, pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

[0080] With reference to Fig. 4, aspects of a computing system are shown in accordance with embodiments of the present disclosure. The computing system 100 is illustrated in Fig. 4 to comprise the processor 404, a memory 408, a communication interface 412, and a user interface 416. In some cases, the computing system 100 may omit and / or include additional features. For example, the user interface 416 may be omitted in cases where the computing system 100 communicates with the user device 116.

[0081] The processor 404 may be configured to execute instructions stored in the memory 408, which instructions may cause the processor 404 to carry out one or more computing steps utilizing or based on data received from the imaging device 104, the navigation system 108, the robot 112, the user device 116, the database 120, and / or any other component. The processor 404 may be or comprise one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple Al l, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000-series processors, Nvidia GeForce RTX 3000-series processors, AMD Radeon RX 5000- series processors, AMD Radeon RX 6000-series processors, or any other graphics processingunits), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0082] The communication interface 412 may be used for receiving image data or other information from an external source (such as the imaging device 104, the robot 112, the navigation system 108, the user device 116, the database 120, and / or any other system or component), and / or for transmitting instructions, images, or other information to an external system or device (e.g., the imaging device 104, the robot 112, the navigation system 108, the database 120, and / or any other system or component). The communication interface 412 may comprise one or more wired interfaces (e.g., a USB port, an Ethernet port, a Firewire port) and / or one or more wireless transceivers or interfaces (configured, for example, to transmit and / or receive information via one or more wireless communication protocols such as 802.1 la / b / g / n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface 412 may be useful for enabling the computing system 100 to communicate with one or more other processors or computing devices not a part of the computing system 100, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

[0083] The user interface 416 may be or comprise a keyboard, mouse, trackball, joystick, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or for providing information to a user. Although the user interface 416 is shown as part of the computing system 100, in some embodiments, the computing system 100 may utilize a user interface 416 that is housed separately from one or more remaining components of the computing system 100. In some examples, the user interface 416 may be located proximate one or more other components of the computing system 100, while in other embodiments, the user interface 416 may be located remotely from one or more other components of the computing system 100. For example, the user interface 416 may be associated with or a part of the user device 116, such as when a surgeon, physician, or other user uses the user device 116 to interact with the computing system 100.

[0084] The memory 408 may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. The memory 408 may comprise a surgical plan 420, image processing 424, segmentation 428, transformation 432, registration 436, sensor information 440, and / or one or more control algorithms 444. The memory 408 may store information or data useful for completing, for example, any step of the methods 500 and / or 600 described herein, or of any other methods. The memory 408 may store, for example, instructions and / or machine learning models that support one or more functions of the robot 112. For instance, the memory 408 may store content (e.g., instructions and / or machine learning models) that, when executed by the processor 404, enable the image processing 424, the segmentation 428, the transformation 432, and / or the registration 436. Such content, if provided as in instruction, may, in some embodiments, be organized into one or more applications, modules, packages, layers, or engines. Alternatively or additionally, the memory 408 may store other types of content or data (e.g., machine learning models, artificial neural networks, deep neural networks, etc.) that can be processed by the processor 404 to carry out the various methods and features described herein. Thus, although various contents of memory 408 may be described as instructions, it should be appreciated that functionality described herein can be achieved through use of instructions, algorithms, and / or machine learning models. The data, algorithms, and / or instructions may cause the processor 404 to manipulate data stored in the memory 408 and / or received from or via the imaging device 104, the robot 112, the database 120, and / or other components.

[0085] The surgical plan 420 may comprise information about one or more surgeries or surgical procedures. For example, the surgical plan 420 may be or comprise information about one or more trajectories (e.g., an implant trajectory for one or more surgical implants), information about the various steps in the surgery or surgical procedure, information about the types of surgical tools to be used in the surgery or surgical procedure, combinations thereof, and / or the like. In some cases, the user may be able to modify the content of the surgical plan 420, such as through user inputs to the user device 116 and / or the user interface 416.

[0086] The image processing 424 enables the processor 404 to process image data of an image (received from, for example, the imaging device 104 or any other imaging device) for the purpose of, for example, identifying information about the patient 102 and / or an object (e.g., a surgical tool, an implant, etc.) depicted in the image. “Image data” as used herein refers to the data generated or captured by the imaging device 104, including in a machine -readable form, a graphical / visual form, and in any other form. The information about the patient 102 and / or the object may comprise, forexample, a pose of the patient, a pose of an object (e.g., a surgical tool, an implant, etc.), a boundary of reference marks(s) proximate the patient 102, combinations thereof, and / or the like. In some cases, the image processing 424 may use one or more algorithms to enhance the appearance of the initial image data captured by the imaging device 104 (e.g., artifact removal algorithms, contrast enhancement algorithms, etc.).

[0087] The information obtained from the image processing 424 may enable, for example, determining of the pose of the patient 102, the pose of reference markers (e.g., optical marker 304, navigation markers 320A-320N, etc.) proximate the patient 102, combinations thereof, and / or the like. The information may also enable registration of the patient 102 to a common coordinate frame of the imaging device 104, and / or registration of the elements depicted in the image data to the common coordinate frame of the imaging device 104. The image processing 424 may be used in conjunction with segmentation 428 to identify anatomical features of the patient 102 and / or of one or more objects, as discussed below.

[0088] The segmentation 428 enables the processor 404 to segment image data so as to identify anatomical features of the patient 102 (and / or anatomical features thereof) and / or one or more objects depicted in the image data such as, for example, a surgical tool, implanted medical device, combinations thereof, and / or the like. The segmentation 428 may enable the processor 404 to identify a boundary of an object or an anatomical feature of the patient 102 using, for example, feature recognition. For example, the segmentation 428 may enable the processor 404 to identify, in the image data, one or more vertebrae of the patient 102 and / or one or more objects (e.g., surgical screws) implanted in the vertebrae of the patient 102. In other examples, the segmentation 428 may enable the processor 404 to identify a boundary of an object (e.g., a boundary of a vertebra, a boundary of a surgical screw, etc.) by determining a difference in or contrast between colors or grayscale values of image pixels.

[0089] The transformation 432 enables the processor 404 to generate transformations that map coordinates in one coordinate system into another coordinate system. In other words, the transformation 432 enables the processor 404 to transform coordinates associated with an object in the surgical environment (e.g., the robot 112, a portion of patient anatomy, etc.) from a first coordinate system (e.g., a patient coordinate system) into a second coordinate system (e.g., a reference frame coordinate system) based on, for example, the registration of the first coordinatesystem and a third coordinate system (e.g., an imaging device coordinate system) and the registration of the third coordinate system and the first coordinate system.

[0090] The registration 436 enables the processor 404 to correlate one coordinate system with another coordinate system. For example, the registration 436 may enable the processor 404 to correlate or map a first coordinate system (e.g., a patient coordinate system) with a second coordinate system (e.g., an imaging device coordinate system) using localization (e.g., optical localization, electromagnetic localization, etc.). The registration 436 may, for example, comprise an algorithm that receives a set of 3D points of optical and / or electromagnetic markers in the second coordinate system and information about the position of one or more localizers in the first coordinate system and the second coordinate system to generate a correlation or map between the two coordinate systems.

[0091] The sensor information 440 may be or comprise a collection of measurements generated by one or more sensors positioned within the surgical environment. For example, the sensor information 440 may be or comprise measurements from force sensors, such as from force and / or torque sensors positioned on the robot 112 to measure forces generated as a result of the robot 112 interacting with the patient 102. As another example, the sensor information 440 may comprise information recorded by a depth sensor that captures a user gesture, which gesture may be used by the computing system 100 to control the robot 112 based on the user gesture. As yet another example, the sensor information 440 may comprise capacitance measurements of various tissues of the patient 102. In some examples, the sensor information 440 may be used as an input into the control algorithm 444, which may provide outputs that can be used to control the imaging device 104, the navigation system 108, the robot 112, and / or the like.

[0092] The control algorithm 444 may be or comprise one or more algorithms that generate outputs that can be used by the processor 404 to control the imaging device 104, the navigation system 108, the robot 112, and / or the like. The control algorithm 444 may receive one or more inputs, such as the sensor information 440, one or more images generated by the imaging device 104, combinations thereof, and / or the like. The control algorithm 444 may then process the inputs and generate one or more outputs. The outputs may comprise signals or other data that, when processed by the processor 404, enable the processor 404 to control the robot 112 in accordance with the outputs. In one example, such as when the robot 112 is being used to drill a hole for a pedicle screw, force sensors positioned on the robot 112 may generate measurements that are inputinto the control algorithm 444. The control algorithm 444 may then determine an overall force generated by the robot 112 on the patient 102 and compare the overall force to a threshold value. When the overall force meets or exceeds the threshold value, the control algorithm 444 may generate an output that, when processed by the processor 404, enables the processor 404 to change one or more operation parameters of the surgical drill to reduce the overall force generated by the robot 112 on the patient 102.

[0093] The procedure library 450 may be or comprise one or more algorithms for performing various surgical procedures (e.g., incision, decompression, drilling, tissue removal, imaging, etc.). For example, to perform an incision procedure, the input may be a location of the incision, a preoperative plan and a list of surgical tools to be used. The procedure library 450 can further calculate a reachability for each listed surgical tool.

[0094] The navigation module 452 enables the processor 404 to determine and display a realtime location of surgical components (e.g., robotic elements, portion of anatomy, surgical tools, the surgical environment, etc.).

[0095] The spatial measurement module 454 enables the processor 404 to determine the relative location of surgical elements (e.g., robotic components, surgical tools, etc.). For example, optical markers may be used to register a location of components in a surgical environment at the start of a procedure. The optical markers may be used to track / monitor the registered components during performance of the procedure. A real-time location of each registered component may also be displayed.

[0096] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Alternatively or additionally, functions may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (alone or in combination with instructions). Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0097] Fig. 5 depicts a method 500 that may be used, for example, to determine an implant trajectory.

[0098] The method 500 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 404 of the computing system 100 described above. The at least one processor may be part of a robot (such as a robot 112) or part of a navigation system (such as a navigation system 108). A processor other than any processor described herein may also be used to execute the method 500. The at least one processor may perform the method 500 by executing elements stored in a memory such as the memory 408. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 500. One or more portions of a method 500 may be performed by the processor executing any of the contents of memory, such as image processing 424, segmentation 428, transformation 432, registration 436, and / or one or more control algorithm 444.

[0099] The method 500 comprises receiving an image of a patient’s anatomy (step 504). The image of the patient anatomy may be captured by the imaging device 104 and sent to the computing system 100. The image may in some cases be displayed to the user via the user device 116, the user interface 416, and / or the like. In some embodiments, the image data captured by the imaging device 104 may be processed by the processor 404 using image processing 424 to, for example, remove object artifacts (e.g., metal artifact noise), improve contrast between soft tissue and bones, and / or the like. In some embodiments, the image of the patient anatomy may be saved in the memory 408 and / or the database 120.

[0100] The method 500 also comprises detecting a depiction of at least one anatomical element and / or at least one medical device in the image (step 508). The processor 404 may use segmentation 428 to identify at least one anatomical element (e.g., a vertebrae) as well as the at least one medical device (e.g., an optical marker) that appear in the image. The segmentation 428 may use feature detection algorithms to segment the image into a plurality of segments, where one segment contains the depiction of the at least one anatomical element, and a second, different segment contains a depiction of the at least one medical device. In some cases, the medical device may be positioned proximate the anatomical element.

[0101] The method 500 also comprises determining a pose of the at least one anatomical element (step 512). Based on the depiction of the medical device relative to the anatomical element, theprocessor 404 may determine the pose of the anatomical element. For example, the pose of the medical device may be known or determined by the processor 404. Based on the pose of the medical device, the processor 404 may determine the pose of the anatomical element relative to the medical device, and then map (e.g., using transformation 432) the coordinates associated with the anatomical element to a known coordinate system based on a registration between the medical device and a known coordinate system (e.g., an imaging device coordinate system). In some embodiments, the processor 404 may transform coordinates associated with the anatomical element into one or more other coordinate systems (e.g., the medical device coordinate system) and / or into a common coordinate system shared by other objects in the surgical environment. In some cases, the processor 404 may capture multiple images and use image -to-image registration to determine the pose of the anatomical element.

[0102] The method 500 also comprises determining a trajectory based on the determined pose of the at least one anatomical element and a surgical plan (step 516). The processor 404 may use the surgical plan 420 and the determined pose of the anatomical element to determine a trajectory. The trajectory may be associated with, for example, the direction along which a surgical screw is to be implanted. In other examples, the trajectory may be associated with the alignment of the robot 112 (e.g., the robotic arms 114 may hold a tool guide along a certain trajectory for a physician to perform a drilling or cutting procedure). In yet another example, the trajectory may be associated with a direction along which a surgical tool is to move when interacting with the patient 102, such as when the robotic arms 114 holds a surgical burr for resecting anatomical tissue. The processor 404 may determine the trajectory by mapping a planned surgical trajectory in the surgical plan 420 to the current pose of the anatomical element based on the difference in pose between the anatomical element depicted in the surgical plan 420 and the determined pose of the anatomical element.

[0103] The method 500 also comprises controlling a robotic arm based on the determined trajectory (step 520). After the trajectory is determined, the processor 404 may cause the robot 112 (or more specifically the robotic arms 114) to move based on the determined trajectory. For example, when a robotic arm 114 holds a surgical tool, the processor 404 may cause the robotic arm 114 to move such that the surgical tool aligns with the trajectory. In cases where there are a plurality of robotic arms, the processor 404 may cause the robotic arm(s) not holding the surgicaltool to move out of a working volume to mitigate the likelihood of collision between the surgical tool and the robotic arm(s).

[0104] The method 500 also comprises causing the determined trajectory to be displayed on a user interface (step 524). The determined trajectory may be displayed to the user via the user device 116, the user interface 416, combinations thereof, and / or the like. In some cases, the display may include a visual depiction of the trajectory relative to patient anatomy (e.g., relative to one or more vertebrae). For example, during an implant step where a surgical screw (e.g., a pedicle screw) is implanted into a vertebra, the display may depict the vertebra (e.g., based on one or more images captured by the imaging device 104) as well as the trajectory along which the surgical screw is to be implanted.

[0105] The method 500 also comprises receiving a user input regarding the determined trajectory (step 528). Once the determined trajectory is displayed to the user, the computing system 100 may require that the user approve the trajectory prior to starting the implant step. Once the user accepts or approves the trajectory, the processor 404 may control the robotic arm 114 to begin implanting the surgical screw. In some cases, the user may choose to adjust the determined trajectory. When the user adjusts the trajectory, the processor 404 may perform the steps 520 and 524 again to adjust the robotic arm to conform with the trajectory, and render the new trajectory to the user interface.

[0106] The present disclosure encompasses embodiments of the method 500 that comprise more or fewer steps than those described above, and / or one or more steps that are different than the steps described above.

[0107] Fig. 6 depicts a method 600 that may be used, for example, to control a robotic arm based on sensor information.

[0108] The method 600 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 404 of the computing system 100 described above. The at least one processor may be part of a robot (such as a robot 112) or part of a navigation system (such as a navigation system 108). A processor other than any processor described herein may also be used to execute the method 600. The at least one processor may perform the method 600 by executing elements stored in a memory such as the memory 408. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 600. One or more portions of a method 600 may be performed by the processor executingany of the contents of memory, such as an image processing 424, a segmentation 428, a transformation 432, a registration 436, and / or one or more control algorithms 444.

[0109] The method 600 comprises receiving sensor information from one or more sensors positioned within a surgical environment (step 604). The sensor information may be similar to or the same as the sensor information 440, which may comprise information generated by force sensors, capacitance sensors, etc. that are positioned on or near the patient 102, the robot 112 (e.g., the robotic arms 114), one or more surgical tools held by the robot 112, combinations thereof, and / or the like. In some examples, the sensor information may be received from the memory 408 and / or from the database 120.

[0110] In some implementations, the step 604 may optionally follow from the step 520 of the method 500, where the robotic arm was controlled based on the determined trajectory. For example, once the robotic arm has been moved such that a surgical tool is aligned with the determined trajectory, the robotic arm 114 may proceed with drilling into the vertebra of the patient 102 using a surgical tool aligned with the determined trajectory. As the surgical tool drills into the patient 102, force sensors on or near the surgical tool (e.g., on the robotic arm 114, within the surgical tool, etc.) may generate force measurements that are stored as sensor information in the memory 408 and / or the database 120.

[0111] The method 600 also comprises providing an input associated with the sensor information into one or more control algorithms (step 608). The sensor information may be provided by the processor 404 as an input into the one or more control algorithms, which may be similar to or the same as the control algorithm(s) 444. In some embodiments, the sensor information may be input into the control algorithm(s), while in other embodiments the processor 404 may perform one or more processing steps on the sensor information before inputting into the control algorithm, such as normalization of data, conversion of force measurement vectors into a common coordinate system, and / or the like. Continuing with the vertebra drilling step example, the force measurements generated by the force sensors on the robot may be provided as input into the control algorithm.

[0112] The method 600 also comprises receiving, from the one or more control algorithms, one or more outputs associated with controlling the robotic arm (step 612). The control algorithm 444 may process the input data and generate one or more outputs. For example, when force measurements are provided to the control algorithm, the control algorithm may determine theoverall force being applied to the patient 102 by the surgical drill and then compare the overall force to a threshold value (e.g., a value stored in the database 120 and / or the memory 408).

[0113] The method 600 also comprises controlling the robotic arm in accordance with the one or more outputs (step 616). When the force meets or exceeds the threshold value, the control algorithm may output a control signal that, when processed by the processor 404, causes the processor 404 to change the operation of the robotic arm 114 to reduce the overall force applied to the patient 102. For example, the processor 404 may cause the robotic arm 114 to move such that the operative end of the surgical tool advances more slowly. In some cases, the control signal may cause additional or alternative changes to the robotic arms 114 and / or the tools attached thereto. As an example, the control signal may cause the processor 404 to reduce the rotations per minute (rpm) of the surgical drill bit to reduce the force applied by the surgical drill. In other examples, the control signal may cause a change in one or more other operation parameters of the surgical drill. In some cases, the force measurements may be continuously provided as inputs into the control algorithm 444 to verify that force generated by the surgical drill does not meet or exceed the threshold value.

[0114] The present disclosure encompasses embodiments of the method 600 that comprise more or fewer steps than those described above, and / or one or more steps that are different than the steps described above.

[0115] Fig. 7 depicts a method 700 that may be used, for example, to control a robotic arm when an unexpected condition is encountered during a surgical task.

[0116] The method 700 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 404 of the computing system 100 described above. The at least one processor may be part of a robot (such as a robot 112) or part of a navigation system (such as a navigation system 108). A processor other than any processor described herein may also be used to execute the method 700. The at least one processor may perform the method 700 by executing elements stored in a memory such as the memory 408. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 700. One or more portions of a method 700 may be performed by the processor executing any of the contents of memory, such as an image processing 424, a segmentation 428, a transformation 432, a registration 436, and / or one or more control algorithms 444.

[0117] The method 700 comprises preparing for and performing a surgical task. The method 700 starts with receiving input indicating a selected surgical task (step 704). For example, a surgical task may comprise an incision, decompression, cutting, drilling, implantation, retraction, imaging, etc.

[0118] The method 700 receives information associated with the surgical task to be performed (step 708). For example, the information associated with the surgical task may include a location on the patient’s anatomy, surgical tools to be used (e.g., scalpel, drill, bone cutter, etc.) parameters for the surgical tools (e.g., how deep to make the incision, drill speed, etc.), preoperative plan data, preoperative image data, etc.

[0119] The method 700 proceeds to determine operating parameters to perform the surgical task (step 712). For example, the system may determine positioning and motion of the robotic arms 114 to perform the surgical tasks. The operating parameters may be translated into control signals that are transferred to the robot (e.g., the robot 112) to perform the surgical task (step 716).

[0120] Fig. 8 depicts a method 800 that may be used, for example, to monitor performance of a surgical task. The surgical task may be performed autonomously by a surgical robot (e.g., the robot 112) or performed by a surgeon assisted by a surgical robot (e.g., the robot 112).

[0121] The method 800 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 404 of the computing system 100 described above. The at least one processor may be part of a robot (such as a robot 112) or part of a navigation system (such as a navigation system 108). A processor other than any processor described herein may also be used to execute the method 800. The at least one processor may perform the method 800 by executing elements stored in a memory such as the memory 408. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 800. One or more portions of a method 800 may be performed by the processor executing any of the contents of memory, such as an image processing 424, a segmentation 428, a transformation 432, a registration 436, one or more control algorithms 444, navigation module 452, and / or spatial measurement module 454.

[0122] The method 800 starts with receiving image data of a surgical environment during the surgical task (step 804). For example, a camera may record the surgical environment during the surgical task. In another example, the image data may comprise multiple still images. The surgicalenvironment may include optical markers included in the image data. Using the received image data, the system registers a location the surgical component in the surgical environment. For example, using spatial measurement module 454, a location of a surgical component may be determined relative to other surgical components within the surgical environment and / or a registration marker. For example, the robot 112 may operate to create an incision on a portion of the patient’s anatomy (e.g., the patient’s back spine area). A location and position of a scalpel and the robotic arms 114 handling the scalpel may be registered 808 in an initial position before the surgical procedure begins. During the surgical procedure, the location of each registered component is tracked and used to monitor the performance of the surgical task (step 812). The location of each registered component may also be displayed in real-time (step 816). For example, the location and position of the scalpel and the robotic arms 114 handling the scalpel may be monitored to ensure that the position / depth / length of the incisions is being performed according to the preoperative plan, operating parameters, and / or surgeon specifications. The surgical area may also be monitored to determine when a surgical task is completed.

[0123] Continuing the example, once the incision is completed, the area may need to be retracted. Surgical retractors help hold an incision or wound open during surgical procedures. The retractors aid in holding back underlying organs or tissues, allowing better visibility and access to the exposed area. In embodiments, the robot 112 may assist with holding the surgical retractors or the robotic arms 114 may include a retractor component. During retraction, the system can send a control signal to hold lock on the retractor(s), monitor and display (via the device 116) a position of the robotic arms 114, a position and status (e.g., locked / unlocked) of the retractors either held by the robotic arms 114 or part of the robotic arms 114, and the surgical area including the portion of the patient’s anatomy being retracted. The exposed area can also be displayed. The robot 112 may perform several tasks simultaneously. For example, while assisting with retraction, the robot 112 may perform drilling, cutting, shaving, etc. in the exposed area. The surgeon operating the robot may also manually control the robot 112 to make changes on the fly.

[0124] In another example, the robot 112 may perform decompression. Decompression is used to treat pain (e.g., neck or back pain) and / or neuropathies due to nerve compression caused by a herniated disc, spinal stenosis, spondylolisthesis, etc. The system may receive neurological output indicating impingement as part of the information associated with the surgical task and also monitor the neurological output to determine when the decompression is complete.

[0125] As noted above, the present disclosure encompasses methods with fewer than all of the steps identified in Figs. 5-7 (and the corresponding description of the methods 500, 600, and 700), as well as methods that include additional steps beyond those identified in Figs. 5-7 (and the corresponding description of the methods 500, 600, and 700). The present disclosure also encompasses methods that comprise one or more steps from one method described herein, and one or more steps from another method described herein. Any correlation described herein may be or comprise a registration or any other correlation.

[0126] The following examples provide various embodiments disclosed herein.

[0127] Example 1. A system comprising: a processor; and a memory storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

[0128] Example 2. The system of Example 1, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

[0129] Example 3. The system of any one of Examples 1-2, wherein the location of each surgical component within the surgical environment is tracked by determining the location of each surgical component within the surgical environment relative to other surgical components within the surgical environment and / or a registration marker.

[0130] Example 4. The system of any one of Examples 1-3, wherein the surgical task comprises real-time imaging and the information associated with the surgical task comprises position information for one or more imagers.

[0131] Example 5. The system of any one of Examples 1-4, wherein the surgical task comprises decompression and the information associated with the surgical task comprises neurological output indicating impingement, a location of the decompression, and a type of decompression to be performed, and wherein the data, when processed by the processor, further enable the processor to monitor the neurological output indicating impingement.

[0132] Example 6. The system of any one of Examples 1-5, wherein the surgical task comprises cutting using at least one of: a mechanical burr, a laser, and an ultrasonic cutting tool.

[0133] Example 7. The system of any one of Examples 1-6, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type of a retractor and the control signal to hold lock on the retractor.

[0134] Example 8. The system of any one of Examples 1-7, wherein the surgical task comprises removal of tissue and the information associated with the surgical task comprises a location of the tissue, a type of tissue to be removed, and a preoperative plan.

[0135] Example 9. A system comprising: a robot; a processor; and a memory storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

[0136] Example 10. The system of Example 9, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

[0137] Example 11. The system of any one of Examples 9-10, wherein the location of each surgical component within the surgical environment is determined relative to other surgical components within the surgical environment and / or a registration marker.

[0138] Example 12. The system of any one of Examples 9-11, wherein the surgical task comprises real-time imaging and the information associated with the surgical task comprises position information for one or more imagers.

[0139] Example 13. The system of any one of Examples 9-12, wherein the surgical task comprises decompression and the information associated with the surgical task comprises a location of the decompression and a type of decompression to be performed, and wherein the data, when processed by the processor, further enable the processor to receive neurological output indicating impingement.

[0140] Example 14. The system of any one of Examples 9-13, wherein the surgical task comprises cutting using at least one of: a mechanical burr, a laser, and an ultrasonic cutting tool.

[0141] Example 15. The system of any one of Examples 9-14, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type of a retractor and the control signal to hold lock on the retractor.

[0142] Example 16. The system of any one of Examples 9-15, wherein the surgical task comprises removal of tissue and the information associated with the surgical task comprises a location of the tissue, a type of tissue to be removed, and a preoperative plan.

[0143] Example 17. A method, comprising: receiving information associated with a surgical task; determining operating parameters to perform the surgical task; transferring a control signal to control a surgical robot to perform the surgical task; receiving image data of a surgical environment during the surgical task; tracking a location of each surgical component within the surgical environment during the surgical task; and monitoring performance of the surgical task to ensure operation is according to the determined operating parameters.

[0144] Example 18. The method of Example 17 further comprising displaying in real-time the location of each surgical component within the surgical environment.

[0145] Example 19. The method of any one of Examples 17-18, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

[0146] Example 20. The method of any one of Examples 17-19, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type of a retractor and a control signal to hold lock on the retractor.

[0147] The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and / or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and / or configurations of the disclosure may be combined in alternate aspects, embodiments, and / or configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and / or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

[0148] Moreover, though the foregoing has included description of one or more aspects, embodiments, and / or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, embodiments, and / or configurations to the extent permitted, including alternate, interchangeable and / or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.

Claims

CLAIMSWhat is claimed is:

1. A system (100) comprising: a processor (404); and a memory (408) storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

2. The system (100) of claim 1, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

3. The system (100) of claim 1 or 2, wherein the location of each surgical component within the surgical environment is tracked by determining the location of each surgical component within the surgical environment relative to other surgical components within the surgical environment and / or a registration marker.

4. The system (100) of any one of claims 1-3, wherein the surgical task comprises realtime imaging and the information associated with the surgical task comprises position information for one or more imagers.

5. The system (100) of any one of claims 1-4, wherein the surgical task comprises decompression and the information associated with the surgical task comprises neurological output indicating impingement, a location of the decompression, and a type of decompression to be performed, and wherein the data, when processed by the processor, further enable the processor to: monitor the neurological output indicating impingement.

6. The system (100) of any one of claims 1-5, wherein the surgical task comprises cutting using at least one of: a mechanical burr, a laser, and an ultrasonic cutting tool.

7. The system (100) of any one of claims 1-6, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type and a control signal to hold lock on the retractor.

8. The system (100) of any one of claims 1-7, wherein the surgical task comprises tissue removal and the information associated with the surgical task comprises a location of the tissue, a type of tissue to be removed, and a preoperative plan.

9. A system (150) comprising: a robot (112); a processor (404); and a memory (408) storing data thereon that, when executed by the processor, enable the processor to: receive information associated with a surgical task; determine operating parameters to perform the surgical task; transfer a control signal to control a surgical robot to perform the surgical task; receive image data of a surgical environment during the surgical task; track a location of each surgical component within the surgical environment during the surgical task; and monitor performance of the surgical task to ensure operation is according to the determined operating parameters.

10. The system (150) of claim 9, wherein the surgical task comprises an incision and the information associated with the surgical task comprises a location of the incision, a preoperative plan, and surgical tool information.

11. The system (150) of claim 9 or 10, wherein the location of each surgical component within the surgical environment is determined relative to other surgical components within the surgical area and / or a registration marker.

12. The system (150) of any one of claims 9-11, wherein the surgical task comprises real-time imaging and the information associated with the surgical task comprises position information for one or more imagers.

13. The system (150) of any one of claims 9-12, wherein the surgical task comprises decompression and the information associated with the surgical task comprises a location of the decompression and a type of decompression to be performed, and wherein the data, when processed by the processor, further enable the processor to receive neurological output indicating impingement.

14. The system (150) of any one of claims 9-13, wherein the surgical task comprises retraction and the information associated with the surgical task comprises a retractor type and a control signal to hold lock on the retractor.

15. A method (500, 600, and 700) comprising: receiving information associated with a surgical task; determining operating parameters to perform the surgical task; transferring a control signal to control a surgical robot to perform the surgical task; receiving image data of a surgical environment during the surgical task; tracking a location of each surgical component within the surgical environment during the surgical task; and monitoring performance of the surgical task to ensure operation is according to the determined operating parameters.

Images