Systems and methods for spinal anatomical structure registration frameworks - Patents.com

Abstract

Systems and methods are disclosed that provide a computerized framework for performing decision intelligence (DI)-based evaluation and / or surgery of a patient's spine. The disclosed spinal evaluation framework provides a robotically actuated screw placement and evaluation system. The disclosed framework can be implemented for performing pre-operative, intra-operative, and / or post-operative spinal evaluations / procedures. The disclosed spinal framework can be utilized for robotically actuated pedicle screw placement, robotically actuated bone removal, and / or spinal configuration optimization via spinal flexibility and alignment assessment.

Description

[Technical Field]

[0001] This application contains subject matter that is subject to copyright protection. As such subject matter is disclosed in the files and records of the United States Patent and Trademark Office, the copyright owner has no objection to the copying by anyone of the patent disclosure, but otherwise reserves all copyright rights whatsoever.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 434,295, filed December 21, 2022, the contents of which are incorporated herein by reference in their entirety.

[0003] The present disclosure relates generally to pre-operative, intra-operative and / or post-operative spinal assessment, and more particularly to a computer-linked framework for performing decision intelligence (DI)-based assessment and / or surgery of a patient's spine. [Background technology]

[0004] Currently, there are many different types of spinal surgery procedures that patients may require, such as spinal fusion, microdiscectomy, artificial disc replacement, laminectomy, vertebroplasty, foraminotomy, and interlaminar implants.

[0005] Typically, one spine surgeon is employed to perform such procedures. However, in complex situations, additional spine surgeons may be required. In most cases, two spine surgeons versus one spine surgeon demonstrate lower blood loss (e.g., 763 ml vs. 1524 ml), fewer transfusions (e.g., 0.5 vs. 2.3), and fewer 90-day readmissions (0% vs. 15.8%).

[0006] However, procedures requiring pedicle screws and osteotomies (such as spinal fusions) have demonstrated numerous problems / failures under traditional surgical methods and techniques, despite the presence of two surgeons. For example, pedicle screw malapposition can occur, which can necessitate a revision procedure and / or lead to malpractice lawsuits. In fact, of the 400,000+ lumbar spine fusion procedures performed in the United States in the past year, 1.5% of procedures involved pedicle screw malapposition. Summary of the Invention

[0007] To address these concerns and issues in the medical field, practices include utilizing robots. However, many conventional robots have limited capabilities and / or functions. For example, many current robots have limited mechanisms used for screw placement, and they do not perform with an accuracy level that eliminates the possibility of a revision procedure being required. For example, many current robots utilize nodal geometries associated with a predetermined path, which allows the screw to be rotated a certain number of times. However, this does not account for variations caused by, among other things, insertion speed, driving force, and / or both, which, when not accounted for, can lead to misplacement of the screw, among other variables. Furthermore, current robots can have driving forces across multiple axes, which can lead to improper positioning and / or improper insertion into the patient's spine (e.g., which can cause the robotic tool to unintentionally shift position and / or angle, thereby leading to improper use of pedicle screws).

[0008] Additionally, current application of robotics in spine medical technology effectively requires surgeons to become robotics experts: rather than simply focusing on the patient care and procedure at hand, spine surgeons must now often become robotics engineers to solve the robotic problems they rely on.

[0009] Thus, according to some embodiments, the disclosed systems and methods provide a novel spinal assessment framework that addresses deficiencies in the art, among other things, by providing a robotically actuated screw placement and assessment system. As discussed herein, according to some embodiments, the disclosed framework can be implemented to perform pre-operative, intra-operative, and / or post-operative spinal assessments / procedures. In some embodiments, the spinal framework discussed herein can be utilized for robotically actuated pedicle screw placement, robotically actuated bone removal, and / or spinal flexibility and alignment assessment.

[0010] According to some embodiments, the framework can achieve various methodologies specifically configured to ensure improved efficiency, accuracy, and safety of spinal procedures compared to existing mechanisms. According to some embodiments, the disclosed framework can perform and / or enable intra-operative registration of a patient's spinal anatomy to navigation space during spinal surgery. According to some embodiments, as discussed in more detail below, the disclosed framework can utilize various sensors and / or detectors to determine a pose estimate of the spinal anatomy within navigation space. In some embodiments, as discussed in more detail below, the framework can invalidate previously assumed pose estimates of the spinal anatomy within navigation space during spinal surgery.

[0011] According to some embodiments, the disclosed framework can be configured to determine the anatomical system and / or the accuracy of the anatomical system, and the disclosed framework can implement mechanisms for determining the anatomical system and / or the accuracy of the anatomical system. In some embodiments, as discussed below, the framework can perform determinations regarding the accuracy of the anatomical system from multiple mechanical touch points on the patient's bones / anatomical structures.

[0012] According to some embodiments, the disclosed framework can be configured to autonomously place pedicle screws via robotics, and the disclosed framework can implement mechanisms for autonomously placing pedicle screws via robotics. In some embodiments, as discussed below, the framework can determine and / or utilize screw trajectories, skive likelihoods (or probabilities), optimal pilot hole sizes, etc., and can implement a robot (e.g., a two-arm robot) to implant pedicle screws.

[0013] According to some embodiments, the disclosed framework can be configured to implement a linear actuator end effector, and the disclosed framework can implement mechanisms for implementing the linear actuator end effector. In some embodiments, as discussed below, the framework can achieve pedicle screw placement via the linear actuator end effector, which can improve workflow efficiency and safety (e.g., by reducing the number and range of robotic movements and steps compared to traditional robots / robotics).

[0014] According to some embodiments, the disclosed framework can be configured to determine a tracking array shift from a camera element, and the disclosed framework can implement a mechanism for determining the tracking array shift from a camera element. In some embodiments, the framework can determine gross patient tracking array movement based on camera-centric tracking, as discussed below. In some embodiments, after determining the gross movement, the framework can perform realignment of the patient tracking array, as discussed in more detail below.

[0015] According to some embodiments, the disclosed framework can be configured to avoid line-of-sight obstructions during spinal procedures (e.g., pedicle screw insertion / placement), and the disclosed framework can implement mechanisms for avoiding line-of-sight obstructions during spinal procedures (e.g., pedicle screw insertion / placement). In some embodiments, as discussed in more detail below, the framework can operate to determine optimal camera placement for simultaneously viewing a dynamic reference base (DRB) and an instrument tracking array, which can be based on preoperatively planned screw trajectories and / or intraoperative alignment. As used herein, the term "optimal" and similar adjectives include both an absolutely optimal solution and a solution that provides results within 5 percent of the absolutely optimal solution.

[0016] Thus, according to some embodiments, the disclosed framework can be configured to perform, and can be operative to perform, each of the disclosed embodiments as an integral part of a pre-operative, intra-operative, and / or post-operative procedure, thereby enabling analysis performed pre-operatively to be utilized intra-operatively and post-operatively, as discussed herein.

[0017] Thus, according to some embodiments, control of a surgical robot (e.g., FIG. 9A , etc.) in this framework can provide optimization of spinal alignment dependent on optimization of pedicle screw fixation (e.g., screw precision) and optimization of spinal flexibility (e.g., planning). Thus, no validated or known method currently exists for determining initial pedicle screw fixation strength, and as described herein, it allows for the first time to quantify the surgeon's "feel," which may lead to more refined intraoperative decisions that can be made regarding cement augmentation, interbody placement, and osteotomies.

[0018] According to some embodiments, a method for performing DI-based assessment and / or surgery of a patient's spine is disclosed. According to some embodiments, the present disclosure provides a non-transitory computer-readable storage medium for performing the above-described technical steps of the functionality of this framework. The non-transitory computer-readable storage medium has tangibly stored thereon or tangibly encoded thereon computer-readable instructions that, when executed by a device, cause at least one processor to perform the method for performing DI-based assessment and / or surgery of a patient's spine.

[0019] According to one or more embodiments, apparatuses and / or systems are provided that include one or more processors and / or computing devices configured to provide functionality according to such embodiments. According to one or more embodiments, the functionality is implemented in method steps performed by at least one computing device. According to one or more embodiments, program code (or program logic) executed by a processor of a computing device to perform functionality according to such one or more embodiments is embodied in, by, and / or on a non-transitory computer-readable medium.

[0020] Features and advantages of the present disclosure will become apparent from the following description of the embodiments illustrated in the accompanying drawings, in which like reference characters refer to like parts throughout the various views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a block diagram of an exemplary configuration in which the systems and methods disclosed herein may be implemented in accordance with some embodiments of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating components of an exemplary system according to some embodiments of the present disclosure. [Figure 3] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 4A] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4B] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4C]1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4D] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4E] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4F] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4G] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 4H] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 5] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 6] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 7] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 8] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 9A] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 9B] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 9C] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 10] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 11A] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 11B] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 11C] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 11D] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 11E] 1A-1C illustrate non-limiting exemplary embodiments for implementing the disclosed systems and methods according to some embodiments of the present disclosure. [Figure 12] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 13] FIG. 2 illustrates an exemplary data flow according to some embodiments of the present disclosure. [Figure 14] FIG. 1 is a block diagram illustrating a computing device that illustrates an example of a client or server device for use in various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0022] The present disclosure will now be described in more detail with reference to the accompanying drawings. The accompanying drawings, which form a part of this disclosure, illustrate, by way of non-limiting example, certain exemplary embodiments. However, the subject matter can be embodied in a variety of different forms, and therefore, the subject matter covered or claimed is not intended to be construed as being limited to only the exemplary embodiments set forth herein, which exemplary embodiments are presented by way of example only. Likewise, a reasonably broad scope of the subject matter claimed or covered is intended. Among other things, for example, the subject matter can be embodied as a method, device, component, or system. Thus, embodiments can take the form of, for example, hardware, software, firmware, or any combination thereof (other than software itself). Therefore, the following detailed description is not intended to be construed in a limiting sense.

[0023] Throughout this specification and the claims, terms may have nuanced meanings that are suggested or implied in context beyond their stated meaning. Similarly, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to different embodiments. For example, claimed subject matter is intended to include any and all combinations of the example embodiments.

[0024] Generally, terms can be understood, at least in part, from their use in context. For example, terms such as "and," "or," or "and / or" as used herein may include various meanings that may be determined, at least in part, by the context in which such terms are used. Typically, when used to relate a list such as A, B, or C, "or" is intended to mean A, B, and C, and is used here in an inclusive sense, and is also intended to mean A, B, or C, and is used here in an exclusive sense. Additionally, as used herein, the term "one or more" may be used to describe any feature, structure, or characteristic in a singular sense, or may be used to describe a combination of features, structures, or characteristics in a plural sense, depending at least in part on the context. Similarly, terms such as "a," "an," and "the" may be understood to convey singular or plural use, depending at least in part on the context. Additionally, the term "based on" may be understood as not necessarily intended to convey an exclusive set of factors, but instead may allow for the presence of additional factors not necessarily expressly recited, again depending at least in part on the context.

[0025] The present disclosure will be described below with reference to block diagrams and operational diagrams of methods and devices. It will be understood that each block of the block diagrams or operational diagrams, and combinations of blocks in the block diagrams or operational diagrams, can be implemented by analog or digital hardware and computer program instructions. These computer program instructions can be provided to a general-purpose computer processor or a special-purpose computer, ASIC, or other programmable data processing device to modify the functions detailed herein, such that the instructions, executed via the processor of the computer or other programmable data processing device, perform the functions / operations specified in the block diagrams or one or more operational blocks. In some alternative implementations, the functions / operations noted in the blocks can be performed in an order different from that noted in the operational diagrams. For example, two blocks shown in succession can in fact be executed substantially concurrently, or the blocks can sometimes be executed in reverse order, depending on the functions / operations involved.

[0026] For purposes of this disclosure, a non-transitory computer-readable medium (or one or more computer-readable storage media) stores computer data, which can include computer program code (or computer-executable instructions) in machine-readable form that can be executed by a computer. By way of example, and not limitation, a computer-readable medium can include a computer-readable storage medium for tangible or fixed data storage of data or a communication medium for transitory interpretation of signals containing code. As used herein, a computer-readable storage medium refers to a physical or tangible storage device (as opposed to a signal) and includes, but is not limited to, volatile and non-volatile removable and non-removable media implemented in any method or technology for the tangible storage of information, such as computer-readable instructions, data structures, program modules, or other data. A computer-readable storage medium includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, optical storage, cloud storage, magnetic storage devices, or any other physical or material medium that can be used to tangibly store desired information or data or instructions and that can be accessed by a computer or processor.

[0027] For purposes of this disclosure, the term "server" should be understood to refer to a service point that provides processing, database, and communication capabilities. By way of example, and without limitation, the term "server" can refer to a single physical processor with associated communication and data storage and database capabilities, or the term "server" can refer to a networked or clustered complex of processors and associated network and storage devices, as well as operating software, one or more database systems, and application software that support the services provided by the server. A cloud server is an example.

[0028] For purposes of this disclosure, a "network" should be understood to refer to a network that can couple devices together in such a manner that communications can be exchanged, such as between a server and a client device or between a server and other types of devices, including, for example, between wireless devices coupled via a wireless network. A network can also include mass storage devices, such as, for example, network-attached storage (NAS), a storage area network (SAN), a content delivery network (CDN), or other forms of computer- or machine-readable media. A network can include the Internet, one or more local area networks (LANs), one or more wide area networks (WANs), wireline-type connections, wireless-type connections, cellular, or any combination thereof. Similarly, subnetworks that can use different architectures or conform to or are consistent with different protocols may interoperate within a larger network.

[0029] For purposes of this disclosure, a "wireless network" should be understood to couple client devices together through a network. The wireless network may use a standalone ad-hoc network, a mesh network, a wireless LAN (WLAN) network, a cellular network, etc. The wireless network may further use multiple network access technologies, including Wi-Fi, Long Term Evolution (LTE), WLAN, wireless router mesh, or second, third, fourth, or fifth generation (2G, 3G, 4G, or 5G) cellular technology, mobile edge computing (MEC), Bluetooth, 802.11b / g / n, etc. The network access technology may enable wide-area coverage for devices, such as client devices, with varying degrees of mobility.

[0030] In summary, a wireless network may include substantially any type of wireless communication mechanism capable of conveying signals between devices, such as client devices or computing devices, or between or within networks.

[0031] Certain embodiments will be described in more detail with reference to the figures. For purposes of this disclosure, the disclosure focuses generally on the spine; however, as will be apparent from the discussion herein, those skilled in the art will understand that the disclosure should not be construed as limiting, and that the disclosed systems and methods can be achieved and / or implemented in accordance with particular spinal sections or portions, including, but not limited to, vertebrae, discs, cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacral vertebrae, etc., or some combination thereof, without departing from the scope of the disclosure.

[0032] Referring to FIG. 1 , a system 100 is shown that provides the functionality for providing the robotically actuated screw placement and evaluation system discussed herein. According to some embodiments, system 100 can include, but is not limited to, a user equipment (UE) 102, a network 104, an imaging device 106, a cloud system 108, and a spinal evaluation engine 200. While system 100 is shown as including such components, it should be understood that system 100 should not be construed as limiting, as one skilled in the art would readily appreciate that a variety of numbers of UEs, imaging devices, cloud systems, databases, and networks can be utilized. However, for purposes of explanation, system 100 will be discussed with reference to the exemplary depiction of FIG. 1 .

[0033] According to some embodiments, the UE 102 may be any type of electronic device that can be used to perform, be part of, rely on, and / or assist in a spinal procedure. In some embodiments, the UE 102 may be, but is not limited to, a mobile phone, a tablet, a laptop, an Internet of Things (IoT) device, a wearable device, a surgical robot, an autonomous machine, and any other type of modern device. According to some embodiments, a non-limiting example of the UE 102 is a robotic computer connected to the network 104 that can analyze medical images captured by the imaging device 106 and utilize those medical images to perform the implantation of pedicle screws into a patient's spine, as discussed herein.

[0034] Non-limiting exemplary embodiments of the UE 102 are provided below with respect to FIGS. 9A, 11A-11E, and / or 14, according to some embodiments.

[0035] According to some embodiments, imaging device 106 refers to a device used to acquire medical images. For example, imaging device 106 may achieve image capture by any type of known or future known mechanism, such as, but not limited to, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, positron emission tomography (PET), ultrasound arthrography, angiography, fluoroscopy, myelography, etc. Imaging device 106 may acquire images in real time and / or may be used to generate composite images or models, which may also be generated in real time or (substantially similar) near-real time.

[0036] According to some embodiments, imaging device 106 may include any device capable of detecting acoustic or electromagnetic waves and assembling a visual representation of the detected waves. Imaging device 106 may collect waves from any frequency range or any portion of the electromagnetic spectrum or sound, often as a matrix of independently acquired measurements, each measurement representing one pixel of a three-dimensional (3D) image. These measurements may be acquired simultaneously or sequentially via a scanning process or combination of methods. To increase the resolution of the resulting image, some pixels of the image generated by imaging device 106 may be interpolated from direct measurements representing neighboring pixels.

[0037] In some embodiments, imaging device 106 may include, correspond to, and / or be related to medical imaging equipment, such as, but not limited to, MRI, CT, ultrasound, etc. Thus, imaging device 106 may be any type of device that produces images, such as any of a variety of machines used to produce diagnostic images of a patient's body, for example, an MRI machine, compressed sensing (CS) technology, a CT scanner, an X-ray machine, etc.

[0038] In some embodiments, an imaging device 106 may receive imaging data from or generate imaging data from multiple imaging devices 106. For example, in some embodiments, an imaging device 106 may include a camera mounted to a ceiling or other structure above the surgical room, a camera that may be mounted on a tripod or other separate wearable device, a camera that may be worn by a surgeon or other surgical staff, a camera that may be incorporated into a wearable device (e.g., UE 102) such as an augmented reality device like Google® Glass, Microsoft® HoloLens, etc., a camera that may be integrated into an endoscope, microscope, laparoscope, or any camera or other imaging device 106 (e.g., ultrasound) that may be in the surgical room.

[0039] According to some embodiments, the imaging device 106 may include and / or execute any type of known or later known ML and / or AI algorithm and / or software module capable of determining qualitative or quantitative data from medical images, where the ML and / or AI algorithm and / or software module may be, for example, a deep learning algorithm trained on a dataset of medical images.

[0040] According to some embodiments, the imaging device 106 may be connected to and / or configured to be in electronic communication with the UE 102. In some embodiments, the imaging device 106 (and / or the UE 102) may include an inertial measurement unit (IMU), which is an electronic device that measures and reports, among other variables, body-specific forces, angular velocity, and orientation of movement. Thus, as discussed below, an IMU implemented in association with the device 102 and / or 106 may be utilized to track the movement of the device 102 / 106.

[0041] The network 104 can be any type of network, such as, but not limited to, a wireless network, a cellular network, the Internet, etc. The network 104 facilitates connectivity of the components of the system 100, as shown in FIG.

[0042] Cloud system 108 may provide a distributed network of computers including servers and databases. In some embodiments, cloud system 108 may be any type of cloud operating platform and / or network-based system on which applications, operations, and / or other forms of network resources may reside. For example, system 108 may be a service provider and / or network provider from which applications may be accessed, served, or executed. In some embodiments, cloud system 108 may include servers and / or databases of information accessible via network 104. In some embodiments, databases of cloud system 108 may store data and metadata datasets related to patients, users of UEs 102 and UEs 102, imaging devices 106, and local and / or network information related to services, applications, and content rendered and / or executed by UEs 102 and imaging devices 106.

[0043] In some embodiments, the cloud system 108 may be a private cloud and / or network with limited access by isolating the network, such as preventing external access, or by using encryption to limit access to only authorized users, for example, a secure local network associated with a hospital. In some embodiments, the cloud system 108 may be a public cloud 108 with access widely available via the Internet.

[0044] Spinal assessment engine 200 (also referred to interchangeably as assessment engine 200 and engine 200) includes components for executing the robotically actuated screw placement and assessment system discussed in more detail below with respect to at least FIGS. 3-13. According to some embodiments, spinal assessment engine 200 may be a special purpose machine or processor and may be hosted on UE 102. In some embodiments, engine 200 may be hosted on a peripheral device connected to UE 102, imaging device 106, and / or any other device connected to and / or present on network 104.

[0045] In some embodiments, for example, the UE 102 may be a computer connected to another UE 102, such as a surgical robot (e.g., FIG. 9A), whereby the movements, manipulations and / or procedures performed by the robot are directed by the computer.

[0046] According to some embodiments, as discussed above, the assessment engine 200 may function as an application installed on the UE 102 via the cloud system 108 and / or the imaging device 106. In some embodiments, such an application may be a web-based application from the cloud system 108 that the UE 102 accesses via the network 104 (e.g., as indicated by the connection between the network 104 and the engine 200 and / or the dashed line between the UE 102 and the engine 200 in FIG. 1 ). In some embodiments, the engine 200 may be configured and / or installed via the cloud system 108 and / or the imaging device 106 as an extension script, program, or application (e.g., a plug-in or extension) to another application or program running on the UE 102.

[0047] As shown in FIG. 2 , according to some embodiments, the evaluation engine 200 includes a navigation space module 202, an anatomical accuracy module 204, a screw placement module 206, a robotic arm module 208, an end effector module 210, a tracking module 214, and an obstacle module 214.

[0048] According to some embodiments, the navigation space module 202 can be configured to perform and / or carry out the steps of process 300 of FIG. 3 , discussed below. In some embodiments, the anatomical accuracy module 204 can be configured to perform and / or carry out the steps of process 500 of FIG. 5 , discussed below. In some embodiments, the screw placement module 206 can be configured to perform and / or carry out the steps of process 700 of FIG. 7 , discussed below. In some embodiments, the robotic arm module 208 can be configured to perform and / or carry out the steps of process 800 of FIG. 8 , discussed below. In some embodiments, the end effector module 210 can be configured to perform and / or carry out the steps of process 1000 of FIG. 10 , discussed below. In some embodiments, the tracking module 214 can be configured to perform and / or carry out the steps of process 1200 of FIG. 12 , discussed below. Additionally, in some embodiments, the obstacle module 214 can be configured to perform and / or carry out the steps of process 1300 of FIG. 13 , discussed below.

[0049] According to some embodiments, it should be understood that the engines and modules discussed herein are non-exhaustive, as additional or fewer engines and / or modules (or sub-modules) may be applicable to the system and method embodiments discussed.

[0050] 1, the rating engine 200 and / or the cloud system 108 may be associated with a database 110. According to some embodiments, the database 110, which may be, for example, a patient database, may include data and / or metadata regarding multiple patients, locations (e.g., hospitals), physicians, practice areas, etc., or some combination thereof, where the data / metadata is stored as an electronic health record (EHR).

[0051] According to some embodiments, an EHR refers to a digital record of a patient's health information, which may be systematically collected and stored over time. According to some embodiments, an EHR for a patient may be a comprehensive patient record, including, but not limited to, patient identification information, captured patient images, demographics, medical history, history of present illness (HPI), progress notes, problems, medications, vital signs, immunizations, lab data, and radiology reports. In some embodiments, computer software is used to capture, store, and share patient data in a structured manner. An EHR may be generated and managed by authorized providers, allowing authorized providers instant access to health information across clinical and health organizations, such as laboratories, specialists, medical professionals, medical imaging facilities, pharmacies, and emergency facilities. Thus, an EHR may be utilized via the disclosed framework discussed herein.

[0052] In some embodiments, the functionality of engine 200 can include database 110. That is, for example, engine 200 can include a memory or memory stack that allows it to host and / or remotely identify data structures associated with database 110 via a set of pointers or resource identifiers (e.g., uniform resource identifiers (URIs)). In some embodiments, database 110 can be located on a network location, and engine 200 can access database 110. For example, in some embodiments, database 110 can be associated with cloud system 108. In some embodiments, database 110 can be configured as a lookup table (LUT), a blockchain (e.g., a distributed ledger), and / or any other type of secure data repository.

[0053] The operation, configuration and functionality of engine 200 and each module of engine 200, and their role within embodiments of the present disclosure, are discussed in more detail below.

[0054] FIG. 3 illustrates a process 300 detailing an embodiment for performing intraoperative registration of spinal anatomy to navigation space during spinal surgery.

[0055] According to some embodiments, as discussed herein, the engine 200 can execute the process 300 via a system configuration (e.g., shown in FIG. 1 ), and the UE 102 can be configured with a navigation instrument and a transducer. For example, the UE 102 can be provided as a surgical robot shown in FIG. 9A , which will be discussed in more detail later. In some embodiments, the UE 102 can determine the surface topography of an object with which the UE 102 is in contact and communicate the captured data to an external computer processor (e.g., another UE 102) using the transducer. According to some embodiments, processing by the UE 102 in contact with the object using a navigated pointer can indicate that the UE 102 is touching the surface of the object, and a camera (e.g., the imaging device 106; in some embodiments, the imaging device 106 may be configured with and / or connected to the UE 102) can be used to capture the navigation position of the surface and translate it in navigation (or navigational; these are used interchangeably) space. In some embodiments, such navigation space may correspond to, for example, but not limited to, sagittal, coronal, axial space, x, y, z space, 3D space, and the like.

[0056] As discussed below, in some embodiments, the navigation space can be utilized to analyze pre-operative images and determine a pose estimate of the patient's spinal anatomy during the examination. According to some embodiments, the images can be three-dimensional (3D) images (e.g., CT, MRI, etc.), two-dimensional (2D) images (e.g., intraoperative fluoroscopy, etc.), and / or pre-operative calibrated bi-directional AP / lateral images (e.g., EOS imaging, images capturing vertebral body surfaces in a point cloud via surface normals and / or intraoperative surface touch points from a navigated pointer).

[0057] According to some embodiments, as discussed below, a user interface (UI) can be provided and displayed on the UE 102. In some embodiments, the UI can receive user annotations solicited from the UI regarding pre-operative and / or intra-operative images, as well as algorithmically determine and output recommended intra-operative orientation parameters for the UE 102 (e.g., a robotic C-arm in a manner that can optimize subsequent generated fluoroscopic images to improve the convergence likelihood of the 2D / 3D merging algorithm). According to some embodiments, as discussed herein, this can improve initialization parameters for the execution of the 2D / 3D merging algorithm, thereby improving the signal-to-noise (S2N) ratio of individual vertebrae and their components.

[0058] According to some embodiments, process 300 begins at step 302, where engine 200 identifies and analyzes medical images of a patient. According to some embodiments, step 302 may include capturing and processing medical images (e.g., 2D / 3D images such as MRI or CT). In some embodiments, the medical images may be previously captured (e.g., pre-operative) medical images that are stored and retrieved.

[0059] It should be understood that while the discussion herein focuses on a single captured medical image, this should not be construed as limiting, as one skilled in the art would understand that the disclosed functionality of process 300 (and the remainder of this disclosure) can be implemented for any type of image capture without departing from the scope of this disclosure, which may include a set of medical images, video, live-streamed content, augmented reality (AR) / virtual reality (VR) content, etc.

[0060] In some embodiments, step 302 may include engine 200 performing automatic segmentation (or auto-segmentation, which are used interchangeably) of the medical image. That is, according to some embodiments, engine 200 may computationally analyze the medical image and segment portions of the image corresponding to each bone / component of the spine and / or anatomical landmarks of the spine. For example, engine 200 may determine image segments (or slices, regions, or objects) of a CT image that correspond to particular regions of the spine (e.g., cervical, thoracic, lumbar, and sacral vertebrae, respectively).

[0061] According to some embodiments, engine 200 may execute a U-Net (or UNet, which is a convolutional neural network (CNN)) application or model trained using cross-entropy loss to perform segmentation. In some embodiments, segmentation may be performed according to specific criteria (which may be included in the request), which may correspond to, but are not limited to, specific regions of the spine, regions and / or types of bones of the spine, and / or other characteristics (e.g., gray level, color, texture, brightness, contrast, etc.), or some combination thereof.

[0062] In some embodiments, engine 200 may implement and execute any type of image segmentation algorithm, technique, or mechanism known or hereafter known, including, but not limited to, thresholding, edge-based, region-based (e.g., active contours, equipotential sets, graph cuts, and watershed algorithms), watershed, clustering-based, neural network-based (e.g., FCN or CNN), transfer learning, heuristic edge detection, probability-based (e.g., Gaussian mixture models, clustering, k-nearest neighbors, Bayesian classifiers, and shallow artificial neural networks), etc., or some combination thereof.

[0063] According to some embodiments, engine 200 can perform automatic segmentation of medical images to generate a surface mesh and / or a point cloud of the spinal anatomy. According to some embodiments, engine 200 can run different algorithms on the medical images, whereby the output can be analyzed to determine matches and / or differences between the images. For example, but not limited to, the medical images may be analyzed by a region growing algorithm, an atlas-based algorithm, and a CNN, whereby an output can be provided through a merging algorithm, where the merging algorithm determines matches with the output image, which allows for the generation of a surface mesh and / or a surface point cloud.

[0064] According to some embodiments, the automatic segmentation performed by engine 200 can enable exposure of the patient's anatomy, allowing engine 200 to utilize a Tactile Elastomer to generate a vertebral point cloud.

[0065] In some embodiments, engine 200 can utilize an iterative closest point algorithm to register the intraoperative point cloud to the preoperative CT point cloud. According to some embodiments, the iterative closest point algorithm can be initialized based on software prompting of the anatomical structure of interest and subsequent user placement of a haptic elastomer tool over the anatomical structure of interest. For example, the software prompting can enable identification of a left or right facet, which can be enabled via user placement, software determination and placement, etc., or some combination thereof.

[0066] According to some embodiments, the automatic segmentation may include 2D / 3D registration, which may be provided via engine 200 executing a reference 2D / 3D merging algorithm for display within a provided UI, as discussed above.

[0067] Thus, with reference to Figure 4A, an example of an anterior-posterior (AP) or lateral fluoroscopic image is shown. In some embodiments, with reference to Figures 4A and 9A, the C-arm (or C-arm attachment) of the robot shown (Figure 9A) can utilize a trackable array via a grid pattern on the film, with radiopaque beads at known distances from the augmentation device associated with the C-arm.

[0068] Thus, according to some embodiments, step 302 may include an initial alignment of the patient tracking array to the patient's anatomy. Thus, in some embodiments, alignment of the patient's spine may be performed.

[0069] In step 304, engine 200 may determine an optimal position for surface-based registration. According to some embodiments, step 304 may include automatic segmentation of the medical image. In some embodiments, the image segmentation from step 302 may be utilized.

[0070] In some embodiments, the engine 200 can algorithmically identify non-matching anatomical areas (e.g., areas with a large number of non-matching points visible within the field of view of the tactile elastomer) within the expected exposure site (e.g., derived from automatically segmented medical images) that have the highest convergence potential through the execution of an iterative closest point algorithm.

[0071] In some embodiments, step 304 may further include displaying the area of ​​interest to the user in a UI, thereby providing a prompt to place the haptic elastomer over the identified site.

[0072] 4B, in some embodiments, a display on the UI displaying a previously generated AP or lateral fluoroscopic image is shown, and the UI can receive user labeling of the anatomical features of the endplates and centroids of each vertebral body. Thus, the information under "Lateral Shot" can include requested information and / or information determined from the segmented provided medical image.

[0073] In some embodiments, Figure 4C provides a non-limiting example of processing performed on the image of Figure 4B. In some embodiments, engine 200 can receive the image of Figure 4B and then output the image of Figure 4C, whereby engine 200 can receive the user-generated line and output the optimal lateral orientation (wag angle) of the C-arm in the lateral C-arm position from the AP fluoroscopy shot, whereby engine 200 can further determine the optimal orientation (e.g., Ferguson angle) of the C-arm in the AP position (from the lateral fluoroscopy shot).

[0074] According to some embodiments, the UI may display a separate UI for the radiologist to interact with, thereby calculating the optimal orientation in a manner similar to that discussed above for generating fluoroscopic shots. According to some embodiments, optimal AP and / or lateral shots may be determined and / or taken for each vertebral body.

[0075] According to some embodiments, distortions can be removed from C-arm images to account for the effects of magnetic fields on images generated by fluoroscopy. For example, the image of FIG. 4A can be distorted with non-distorted features associated with the C-arm.

[0076] According to some embodiments, the engine 200 can perform and / or implement a dynamic re-registration and reconfiguration (DRR) search initialization, which can perform a trajectory parallel to the spinous processes and superior and inferior endplates placed at the center of rotation to find the appropriate APX fluoroscopy shot.

[0077] According to some embodiments, in both the AP and lateral directions, DRR initialization can be based on a parallel trajectory with the superior endplate of each vertebral body. The difference in orientation between the tracking arrays attached to the C-arm distortion relief fixture during each image generated can determine the rotation angle between the AP and lateral DRRs.

[0078] Referring to Figure 4D, an example of a rotationally centered CT image with consistent fluoroscopic alignment is shown, which can provide alignment via DRR search, and consistent fluoroscopic alignment reduces convergence time and improves 2D / 3D algorithm reliability. Figure 4E shows an example of an A-P fluoroscopy with rotational alignment, with the spinous process centered between the pedicles.

[0079] According to some embodiments, the engine 200 can generate iterative DRRs to determine / identify DRRs with acceptable similarity scores (e.g., pixel intensity match between images compared to a threshold / amount), which can be converted / generated as intraoperative fluoroscopic shots (or images).

[0080] According to some embodiments, after an acceptable DRR is identified, engine 200 can perform a 2D / 3D merging algorithm to align the anatomical space in the medical image (e.g., a CT scan) to the navigation space. In some embodiments, such an analysis may be performed for each vertebral level.

[0081] According to some embodiments, the engine 200 can generate an initial alignment of the vertebral posture using keypoint detection of fluoroscopic images mapped to images (e.g., CT scans). Using the initial alignment and simultaneous tracking of the patient via the patient tracking array and the C-arm via the C-arm cap tracking array, the engine 200 can generate a live DRR simulation that outputs a simulated DRR on the UI, which corresponds to a theoretical fluoroscopic image that would be generated based on the instantaneous posture of the C-arm relative to the patient at any time after the initial alignment. According to some embodiments, the DRR simulator provides the surgeon or radiologist with the ability to visualize live (e.g., in real time) what an X-ray would look like in that C-arm posture in order to more quickly and accurately orient the C-arm to take additional X-rays, thereby reducing radiation exposure to the patient and OR staff and reducing the time and resource consumption for acquiring additional images, among other benefits.

[0082] In some embodiments, it may be desirable to optimize the 2D / 3D merging process to determine pedicle screw or interbody positioning within post-implantation 3D CT images. To this end, in some embodiments, engine 200 can store the locations of navigated screw placements or navigated interbody placements, thereby initializing an automatic segmentation algorithm that segments all metal artifacts within intraoperative fluoroscopic images using the stored locations of the aforementioned items. In some embodiments, lines (e.g., user-drawn lines) can be provided on intraoperative fluoroscopic shots that can identify metal artifacts, and engine 200 can utilize these lines as initialization points to perform automatic segmentation of the metal artifacts. In some embodiments, a computer-generated mask can be provided that can be automatically and / or user-adjustable. According to some embodiments, masking can eliminate areas of known discontinuities between the composite DRR and the intraoperative fluoroscopic shots in order to refine the 2D / 3D merging algorithm. Thus, in some embodiments, after the 2D / 3D merge is complete, the UI can display the digitally reconstructed implant within the 3D image generated from the preoperative CT to analyze the safety of the implant placement.

[0083] According to some embodiments, as discussed above, engine 200 may utilize a haptic elastomer. According to some embodiments, the haptic elastomer may be an innovatively configured haptic elastomer that can be a navigated instrument capable of determining the surface topography of an object with which it is in contact.

[0084] According to some embodiments, the tactile elastomeric device can comprise a distal elastomer that can deform so that the device covers at least a portion of the surface it is contacting. In some embodiments, the device can further include a reflective layer at the distal end of the elastomer that reflects the contours of the surface it is contacting. In some embodiments, the device can further include a transparent rigid block that allows light to pass through and also provides a backstop for the elastomer. According to some embodiments, the device can be functionalized such that light passes through the transparent block and reflects off the reflective layer.

[0085] In some embodiments, the instrument may further include a camera to capture the environment inside the elastomer to read the contours of the sensed surface, hi some embodiments, the instrument may further include a printed circuit board (PCB) controller to control the camera and light and send the captured data to an external processor (e.g., another UE 102).

[0086] In some embodiments, an external processor (e.g., additional UE 102) can process the surface topology data from the instrument to generate a 3D surface, which can be compared to an existing three-dimensional model from another medical image (e.g., CT / MRI, etc.). Thus, the processor, through execution of engine 200, can determine / merge the two data sets together, thereby mounting a passive tracking array on the proximal end of the instrument in such a manner that a camera can track the passive tracking array in space.

[0087] Referring to Figure 4F, a tactile elastomeric instrument is shown that reads the surface topology of a vertebral body. In this non-limiting example, an instrument with an elastomer at its distal end can be inserted through a small incision in the skin. In some embodiments, the user moves the instrument through the incision and up to the vertebral body to feel the bony anatomy.

[0088] In Figure 4G, a detailed view of the haptic elastomer touch-sensing instrument of Figure 4F, a tracking array 402 (shown in Figure 4F) tracks the instrument pose within the navigation system. The distal tip of the elastomer 404 deforms to conform to the anatomical surface it is touching (e.g., the elastomer 404 is touching and deforming relative to the anatomical structure of a vertebral body while a camera 408 captures the deformation using light 412 (e.g., via a provided light source and / or associated light source). A reflective layer within the elastomer reflects the light 406 transmitted through a transparent backstop 406. The camera 408 captures the environment, and the resulting data is transferred to an external processor by a PCB board 410.

[0089] According to some embodiments, the instrument can be configured with and / or associated with an external display / monitor capable of displaying live images captured by the instrument. Using this display, the user / surgeon can use the instrument until they identify the surface anatomical structures that they believe will lead to successful registration / merging, at which point they can trigger the system to save the images to a point cloud for processing and compare the pose estimate to an existing three-dimensional model. An example of such a captured image is shown in FIG. 4H.

[0090] Thus, in step 304, application of the haptic elastomer device can provide data of the areas related to the spine, which can be analyzed as discussed above via an iterative closest point algorithm. Thus, step 304 can include alignment of the spine based on the above analysis.

[0091] In step 306, engine 200 may perform alignment of a patient tracking array. In some embodiments, step 306 may include realigning a patient tracking array (e.g., from step 302 discussed above), which may correspond to, for example, skin markers, spinous process clamps, pedicle screws, posterior superior iliac spine (PSIS) pins, etc.

[0092] According to some embodiments, step 306 may include engine 200 registering the patient tracking array from medical images via 2D / 3D merging or intraoperative CT (iCT) performed in a manner similar to that discussed above. In some embodiments, open exposure of the spine may be required. In some embodiments, performance of osteotomies, placement of pedicle screws, and / or surgical manipulation may be required.

[0093] In some embodiments, the use of haptic elastomers to realign vertebral bodies can be performed using pose initialization of an iterative closest point algorithm based on patient registration from a patient tracking array. Such use of haptic elastomers can be performed in a manner similar to that discussed above.

[0094] Thus, in some embodiments, a surgical operation (e.g., pedicle screw placement, osteotomy, etc.) can be performed, which in some embodiments can be performed with or without a surgical robot (e.g., the robot of FIG. 9A).

[0095] In step 308, the engine 200 may determine the accuracy of the surgical procedure (e.g., pedicle screw placement). According to some embodiments, such accuracy determination may include, but is not limited to, the engine 200 registering the PSIS and / or spinous process tracking array with the medical image and / or the 2D / 3D merge. Thus, the pedicle screws may be identified and / or placed.

[0096] In some embodiments, a tracking array can be attached to the pedicle screw so that the tracking array is aligned with the screw shank and the pedicle screw position can be confirmed.

[0097] In a manner similar to that discussed above, engine 200 can utilize the use of haptic elastomers to realign vertebral bodies using pose initialization based on PSIS and / or spinous process tracking array pose estimates from previous alignments. Thus, in some embodiments, engine 200 can determine pedicle screw placement and thereby placement accuracy based on updated vertebral body pose estimates and the positions of tracking arrays attached to the pedicle screws.

[0098] In step 310, engine 200 may generate vertebral body specific registrations. According to some embodiments, step 310 may include generating vertebral body specific registrations with pedicle screws and / or spinous process tracking arrays, as well as real-time tracking alignment of the spine via segment tracking arrays.

[0099] According to some embodiments, step 310 may include, but is not limited to, automatically segmenting the vertebrae from the medical image, as discussed above. In some embodiments, a previously automatically segmented model may be identified. The engine 200 may register a patient tracking array, as discussed above. In some embodiments, performing osteotomies, placing pedicle screws, and / or surgical manipulations may be required.

[0100] In some embodiments, engine 200 can utilize the use of haptic elastomers to align a specific vertebral body to a tracking array (e.g., pedicle screw or spinous process clamp) attached to the vertebral body, using a pose initialization determined via an iterative closest point algorithm based on the initial patient tracking array. According to some embodiments, the specific vertebral body tracking can be an automatically segmented vertebral body tracking rather than a tracking of the entire spine.

[0101] Thus, in some embodiments, navigated surgical procedures (e.g., pedicle screw placement) can be performed, which may be performed with or without an autonomous robot in a manner similar to that discussed above. In some embodiments, multiple individual segments may be tracked to determine the intraoperative alignment of the spine in real time.

[0102] In step 312, screw and / or vertebra tracking, registration and navigation spatial information may be output via the UI as discussed above.

[0103] Referring to FIG. 5, a process 500 is shown in detail that provides a non-limiting exemplary embodiment for determining the anatomical accuracy of an anatomical system from multiple touch points on a patient's bony anatomy.

[0104] According to some embodiments, process 500 begins at step 502, where engine 200 may align the tracking array. According to some embodiments, this alignment may be performed via steps at least similar to those discussed above with respect to process 300 (e.g., via 2D / 3D merging and / or iCT).

[0105] In step 504, engine 200 can determine, generate, produce, extract, etc., a surface topography from a medical image. In some embodiments, the medical image can be related to a pre-operative image, and in some embodiments, the medical image can be an image captured in real time. For example, the medical image can be a pre-operative CT scan of a patient.

[0106] According to some embodiments, a neural network algorithm may be implemented to train a keypoint optimizer to determine 3D keypoints visible in multiple perspectives of the C-arm projection fluoroscopic image. Automatic segmentation of the vertebrae generates a point cloud or surface model of the spine. Additionally, a surgical instrument tracked by an external camera is inserted into the patient, and the surgical instrument is visible in intraoperative images such as fluoroscopy when it touches a vertebra. In some embodiments, this collision may be confirmed via a 6DOF force sensor on the robot end effector, while in other embodiments, the collision is determined via a tactile motor sensor. As a result of these operations, the solver can utilize the following data: negligible distance from the instrument tip to the vertebral surface, an initial pose estimate from a previous alignment before instrument insertion, and the segmented projection geometry of the metal tip. The following alignment algorithm is then implemented: First, keypoints relative to the vertebrae and instrument tip are identified. Then, to address the known z-axis inaccuracy of attempting single-shot keypoint registration, a solver that minimizes the loss of the point cloud from the instrument tip to the vertebrae is run in combination with a solver that minimizes the loss of the instrument and vertebra keypoint detectors. Finally, a patch-based registration is performed on the area of ​​the fluoroscopic image containing the metal instrument and vertebrae, and a similarity index is run to fine-tune this registration. The similarity index compares the reprojected digital reconstruction radiograph with the known tool position and the actual fluoroscopic image, and the resulting similarity measure is converted into an estimated measurement of the accuracy of the relationship between the instrument position and the vertebra position.

[0107] According to some embodiments, the known tool geometry of the surgical instrument may be used in combination with a coordinate grid, and both of these are tracked in the external camera frame to determine the x-ray camera intrinsic when both geometries are visible in the x-ray frame.

[0108] To establish registration, a novel single-shot registration method is proposed in which 10 or more 3D keypoints and their corresponding 2D projections are acquired via digitally reconstructed radiographs. The 3D points are acquired from preoperative or intraoperative 3D modalities such as CT, MRI, or bone MRI. These points can be selected randomly or using an intelligent optimization method for selecting these points.

[0109] According to some embodiments, the anatomical keypoints identified in the keypoint optimizer may be used in combination with the coordinate grid of the surgical instrument or known tool geometry to determine the X-ray camera intrinsics when both geometries are visible in the X-ray frame. For this solver, both the intrinsics and the poses are solved simultaneously. Additionally, a multi-level segment registration may be performed, which assumes that the poses of a group of anatomical keypoints belonging to a non-deformable object, such as a vertebra, are fixed relative to each other. Multiple registrations may also be performed simultaneously for multiple vertebrae whose poses are not assumed to be fixed relative to each other but are assumed to be constrained to a specific range of motion determined by kinematic population data. Although the group of anatomical keypoints and their poses may differ from the preoperative CT, the camera intrinsic solutions for a set of solutions are assumed to be the same, resulting in both a set of solutions for the vertebral poses and a single solution for the X-ray camera intrinsics.

[0110] According to some embodiments, surface topography can be generated according to mechanisms similar to those discussed above. For example, engine 200 can utilize a haptic elastomer configured to be a navigated instrument capable of determining the surface topography of an object with which it is in contact.

[0111] In step 506, engine 200 may determine a set of points associated with the touch point (eg, from a haptic elastomer contact, etc.).

[0112] According to some embodiments, step 506 may include algorithmically determining and identifying two or more points from the medical image that can separate the convergence point between the planned pedicle screw and the vertebra, which may invalidate the registration based on required registration accuracy acceptance criteria. In some embodiments, the invalidation points may be ranked by absolute distance from the convergence point.

[0113] In some embodiments, the engine 200 can algorithmically determine and identify anatomical touch points from the medical image that will invalidate the registration based on required registration accuracy acceptance criteria from a single point of contact with the tactile elastomer.

[0114] According to some embodiments, the algorithmic determination performed via step 506 may include engine 200 executing any type of known or hereafter known ML and / or AI algorithm or technique, such as, but not limited to, computer vision, neural network analysis, feature vector analysis, logistic regression, hidden Markov modeling, Bayes' theorem, etc. (in addition to other algorithms discussed herein / above) to perform such computational image analysis.

[0115] In step 508, engine 200 may accomplish navigation of the setpoint within the navigation space.

[0116] According to some embodiments, the navigated mechanical element can be used to move along the line required to touch a pre-specified point, which can be recorded, and the position in the navigation space where the navigated mechanical element encounters mechanical resistance indicating an encounter with bone can be recorded.

[0117] In some embodiments, the mechanical element may include an end effector of independently tracked concentric elements (e.g., multiple concentric dilators, or a burr surrounded by concentric dilators, etc.). In some embodiments, the concentric elements may have different inner and outer diameters. In some embodiments, the touch point may be registered as a surrounding area in the navigation space. An example of such an embodiment is shown in FIG. 6.

[0118] According to some embodiments, the mechanical element can be a bar connected to a navigated drill end effector, and in some embodiments, the occurrence of a collision can be determined via a six degree of freedom (6DOF) force sensor attached to the end effector, or via torque, or some combination thereof.

[0119] According to some embodiments, a vibration motor is attached to the fixed end of an anisotropic structure, such as a rod, which then vibrates in a circular motion. Additionally, a monitor, such as a three-axis accelerometer, is attached to the anisotropic structure. The resulting motion is then electronically mapped for analysis. When no force is applied, circular motion is achieved. When a net force is applied to the freely vibrating end of the rod, the tracked circular pattern distorts, e.g., gradually flattens into an ellipse in a reproducible manner that is directly proportional to the applied force. The axis of the applied force can be identified by the direction the ellipse forms. In this way, it can be determined that the tool is touching bone or soft tissue based on the change in the tracked pattern.

[0120] In some embodiments, the mechanical element can be a burr attached to a navigated drill that can be threaded through a cylindrical drill guide end effector. In some embodiments, an indication can be utilized that identifies when the drill is at the surface of the vertebra. In some embodiments, this indication can be computationally determined via engine 200, in some embodiments, this indication can be user-provided, or some combination thereof.

[0121] In some embodiments, the mechanical element may be a navigated pointer, whereby in some embodiments, an indication may identify that the drill is on the surface of the vertebra, in some embodiments, this indication may be computationally determined via engine 200, in some embodiments, this indication may be user-provided, or some combination thereof.

[0122] According to some embodiments, step 508 can be performed via touch navigation with a haptic elastomer on the surface, thereby storing a position in the navigation space along the point cloud generated by the haptic elastomer.

[0123] At step 510, engine 200 may determine the registration accuracy. In some embodiments, engine 200 may computationally determine, from the recorded positions of the navigated elements, whether the registration accuracy meets acceptance criteria (e.g., accuracy or distance thresholds). In some embodiments, engine 200 may execute ML / AI algorithms (discussed herein) to determine whether / when the point cloud aligns with an expected surface topography point cloud generated from a medical image (e.g., a CT scan). For example, engine 200 may execute any type of known or hereafter known ML and / or AI algorithm or technique, such as, but not limited to, computer vision, neural network analysis, feature vector analysis, logistic regression, hidden Markov modeling, Bayes' theorem, etc. (in addition to other algorithms discussed herein / above) to perform such computational image analysis.

[0124] In step 512, engine 200 can determine and provide an output (e.g., a display on a UI or an audible output) instructing the surgeon whether to proceed with realigning the patient (e.g., realigning the tracking array, etc.). In some embodiments, this instruction can alert the surgeon to realign when the accuracy determined in step 510 is at or above a threshold level (or outside an accuracy range). If it is within the threshold (or range), the surgeon can be alerted to proceed.

[0125] 7, a process 700 provides a non-limiting exemplary embodiment for automated pedicle screw placement via an autonomous robot (e.g., a surgical robot). In some embodiments, the process 700 enables placing the screw based on a determined instrument skiveability based on a planned pedicle screw trajectory. In some embodiments, the process 700 enables determining an optimal pilot hole size for implanting the pedicle screw, which can be based on skiveability.

[0126] According to some embodiments, process 700 begins at step 702, where engine 200 auto-segments the medical image. This may be performed in a manner similar to that discussed above. In some embodiments, a previously auto-segmented model may be retrieved, and in some embodiments, the auto-segmentation may be performed on a captured medical image.

[0127] In step 704, engine 200 may determine information regarding the planned pedicle screw implantation. According to some embodiments, the determined information may include, but is not limited to, the angle of incidence of the planned pedicle screw trajectory onto the surface topography of the pedicle screw. In some embodiments, the surface topography information may be retrieved and / or determined in a manner similar to that discussed above. This surface topography information may be compared to the screw implantation angle from the planned trajectory, and a determined angle of incidence may be determined. The analysis performed for step 704 may be performed via any of the ML / AI algorithms described above, among others.

[0128] In step 706, skive models that may correspond to the patient's spine and / or other information regarding the spinal surgery, angle of incidence and / or planned trajectory may be identified and utilized to determine skive feasibility.

[0129] In step 708, engine 200 can determine optimal pilot holes for screw implantation. In some embodiments, engine 200 can analyze skiveability according to the angle of incidence to determine optimal holes for pedicle screw implantation. In some embodiments, the optimal pilot holes can be based on additional or alternative inputs, which can include, but are not limited to, bone density, surface convexity, etc., or some combination thereof.

[0130] At step 710, an output may be provided to the surgeon. In some embodiments, the output may be provided on a UI, whereby the output may be based on information determined from step 706 and / or step 708. Thus, in some embodiments, the output may be a displayed interface that provides digital / virtual information regarding screw tip placement, which may be based on the skive model and / or the posture override algorithm (discussed above).

[0131] In FIG. 8, process 800 provides a non-limiting example embodiment for the workflow and sequence for placing an autonomous robotically placed screw using a two-arm robot (e.g., the robot shown in FIG. 9A).

[0132] According to some embodiments, process 800 begins at step 802, where each robotic arm of engine 200 is moved to a position above the patient along a screw trajectory. This position and screw trajectory can be based on pre-operative planning and any of the information derived from the processing described above.

[0133] According to some embodiments, a dovetail feature on the robotic end effector guides a skin knife to the surface of the vertebral body from both sides. In some embodiments, the end effector can point a laser to mark the location on the skin where the surgeon should create the skin incision. On each side, the surgeon creates, cuts, and extends the skin incision down to the spine. Modern dilators feature a funnel-shaped opening to guide the dilator to mate with the robotic end effector, which has a tracking array that is captured by the navigation system.

[0134] In step 804, the engine 200 may perform the process 500 discussed above to determine the anatomical system accuracy.

[0135] In step 806, the engine 200 causes the robot to leave both drills on the bone surface to stabilize the segment.

[0136] In step 808, after stabilization, pilot holes can be drilled according to the number of planned screw insertions, and in step 810, the robot can then insert screws into the drilled pilot holes.

[0137] According to some embodiments, the robotic end effector may drill the bone in a manner that creates a uniform hole concentric with the orbital axis. In some embodiments, the burr may instead be moved in a concentric sweeping motion to create a larger hole of any size centered about the orbital axis on the surface of the vertebra to prevent screw skiving, and then the burr may be driven straight into the bone in a manner that creates a hole concentric with the orbital axis. In some embodiments, the pilot hole may be preoperatively determined and / or user determined, as discussed above with respect to process 700.

[0138] In some embodiments, after the first pilot hole is created, the end effector drill remains in the bone as a fixation point while the second arm repeats the process on the contralateral side.

[0139] After creating both pilot holes one by one, each robotic arm switches its end effector tool to a tap screw driver to place a screw along the trajectory while the other arm remains anchored in the bone.

[0140] Thus, in some embodiments, a robotic arm advances an end effector drill attached to a screw along a trajectory to the surface of the vertebral body, and when the end effector drill reaches a predetermined point above the pilot hole, the end effector begins to rotate and advance the screw at a constant speed toward the hole.

[0141] According to some embodiments, the arm can sense that the screw has entered the bone when it senses a reaction force (above a threshold level) at the screw tip equipped with a force sensor, when a 6DOF force sensor coupled to the end effector drill experiences a reaction force, or by joint torque sensors at each individual robotic joint experiencing a joint torque. In some embodiments, the arm can then utilize a control feedback loop to advance along the axis of the planned screw trajectory at an advancement rate that maintains a constant pressure determined by the reaction force on the end effector drill.

[0142] According to some embodiments, one arm may remain fixed to the screw while drilling the upper or lower contralateral pedicle.

[0143] In some embodiments, the arm can stop advancing when the screw reaches its pre-planned position, when the reaction force exceeds a certain threshold, and / or when the reaction force drops below a certain threshold. After the screw reaches its final position, the arm can maintain connection to the screw, utilizing the newly placed screw as a new fixation point, while a second arm repeats the same workflow on the contralateral side.

[0144] According to some embodiments, after both screws are placed, the arms disengage from the screws, referring to step 812. At step 814, engine 200 may determine whether to insert additional screws according to the preoperative plan. If additional screws are to be inserted, processing proceeds to step 818, where engine 200 loops back to step 802 to repeat the processing for the additional screws. If additional screws are not to be inserted, processing ends at step 816.

[0145] According to some embodiments, the process 800 can accomplish the determination of spinal flexibility via the engine 200 controlling the surgical robot shown in Figure 9A. According to some embodiments, this data can provide intraoperative feedback to the surgeon to aid in decision-making regarding correction, procedure / maneuver, magnitude of correction needed, and the effectiveness of corrective maneuvers already performed, among other benefits.

[0146] According to some embodiments, to determine the vertebral range of motion in different postural alignments (e.g., standing and supine), engine 200 can determine the range of motion pre-operatively by acquiring pre-operative medical images (e.g., CT, MRI, and / or upright EOS films, etc.). In some embodiments, the range of motion may also be determined intra-operatively by determining postural changes between the pre-operative CT and intra-operative positioning (e.g., prone or lateral).

[0147] According to some embodiments, the robot may manipulate the spine while it is fixed to the spine either by bur insertion or pedicle screw placement as described above. In some embodiments, moving the end effectors of the arms to a new position while still fixed to the spine may create a reaction force against each robotic arm by the spine, which can be sensed via force sensors in the arm joints or end effectors.

[0148] According to some embodiments, the robot may manipulate the spine through a predetermined range of motion to calculate translational / rotational force curves, which may be determined by pre-operative or intra-operative scans in different postural patient alignments (e.g., prone, supine, standing, lateral).

[0149] In some embodiments, the surgeon may use this translation / rotation force curve, or a curve comparison with a previously generated curve, to determine the effect of their surgical manipulations (e.g., osteotomy, soft tissue release, discectomy) throughout the procedure. The translation / rotation force curve may also be used to determine optimal rod curvature for confirmation of optimal post-operative spinal alignment.

[0150] In some embodiments, important / useful surgical decisions that may be augmented by this intraoperative information may be, but are not limited to, intraoperative rod bending, determination of sufficiency of soft / hard tissue release, comparison of spinal stiffness before and after intervention, achievable segmental and global alignment, potential for adjacent segment pathology via calculation of expected surgical forces encountered by the spine during rod reduction operations, etc.

[0151] According to some embodiments, the process 800 can accomplish the determination of initial fixation strength via the engine 200 controlling the surgical robot shown in Figure 9A. This data provides intraoperative feedback to the surgeon that drives decision-making regarding correction, procedure / maneuver, magnitude of correction needed, and the effectiveness of corrective maneuvers already performed, among other benefits.

[0152] According to some embodiments, a surgical robot uses a vibration motor attached to the fixed end of an anisotropic structure, such as a rod, screwdriver, tap, knife, burr, or drill, which then vibrates in a circular motion. This vibration may be applied via a typical motor motion or a burr or drill. The anisotropic structure is further equipped with a monitor, such as a three-axis accelerometer. The resulting motion is then electronically mapped for analysis. Circular motion is achieved when no force is applied. When a net force is applied to the free-vibrating end of the rod, the tracked circular pattern distorts in a reproducible manner, e.g., gradually flattening into an ellipse. The distortion is directly proportional to the applied force. The axis of the applied force can be identified by the direction in which the ellipse forms.

[0153] According to some embodiments, after the burr pilot holes are created, taps or pedicle screws smaller in size than the planned pedicle screws (e.g., tap = 4.5 mm, pedicle screw = 5.5 mm, etc.) can be robotically screwed into the vertebrae from both sides. In some embodiments, the robotic end effector can store a torque-time graph as the pedicle screws are inserted. An example of such a graph is shown in FIG. 9B.

[0154] In some embodiments, the robotic end effector and / or vibration motor can then apply a force to the implanted tap or screw such that the vertebrae remain stationary but the tool may undergo micro-movements of <1 mm movement within the vertebra. According to some embodiments, a displacement curve can be generated and stored from this applied force that results in the tool moving within the stationary vertebra.

[0155] In some embodiments, the force-displacement curve may be combined with the insertion torque-time curve and preoperative patient characteristics, which may include, but are not limited to, bone mineral density, age, sex, frailty, hormonal status, and other relevant medical characteristics, to determine the pullout strength and cyclic loading strength of the pedicle screw.

[0156] Additionally, in some embodiments, the model may include quantifiable geometric characteristics of the pedicle screw geometry, the patient vertebral geometry generated from previous automated segmentation, the pedicle screw trajectory, and the screw material type (e.g., stainless steel, titanium, carbon fiber, etc.).

[0157] In some embodiments, the expected forces from a planned or intraoperatively perceived corrective maneuver can be compared to the modeled bone-implant structural integrity, and a warning may be generated to the user that the expected forces of the corrective maneuver exceed the structural integrity of the implant interface.

[0158] In some embodiments, this model outperforms the simple insertion torque model, which has unacceptably high variance, an example of which is shown in Figure 9C.

[0159] According to some embodiments, the process 800 can achieve sagittal, coronal, detorsion correction operations, and / or combinations of correction operations via the engine 200 controlling the surgical robot shown in FIG. 9A to maintain the correction.

[0160] According to some embodiments, after all pedicle screws have been placed, the robot can have the ability to reconnect to the pre-operatively planned screws by returning to its trajectory within the reference coordinate system. While fixating the spine through either the aforementioned burr insertion or pedicle screw placement, the surgeon has the ability to use the robot to manipulate and correct the spine by unlocking and moving each arm. In some embodiments, this can be performed by autonomously correcting the spine using the corrective maneuvers described in the pre-operative plan.

[0161] In some embodiments, based on pre-operative planning, the robotic arm can be moved to a new desired position for better alignment while being fixed at a different level of the spine.

[0162] In some embodiments, force sensors in the end effector and / or joint torque sensors in the arm joints can sense the spinal reaction force, and the sensors can terminate the operation when the sensed force exceeds or falls below a predetermined threshold. In some embodiments, the correction operation can be performed in any plane (e.g., coronal, sagittal, transverse) or combination of planes. In some embodiments, the robot can maintain the correction while the surgeon places a rod on the contralateral side and locks the screw to the rod.

[0163] According to some embodiments, the process 800 can achieve interbody placement assistance via the engine 200 controlling the surgical robot shown in FIG. 9A, which can increase the reliability of interbody placement and reduce the likelihood of endplate failure.

[0164] According to some embodiments, after placing screws at adjacent levels, the robot can reconnect the screws such that one robotic arm is fixed to each of the adjacent levels. In some embodiments, the robot can then autonomously manipulate the vertebral bodies and / or the surgeon can unlock the robot, manipulate it, and relock it to hold the position. In some embodiments, the correction can be tracked by intraoperative planning software to provide real-time visualization of vertebral body posture by tracking the end effector pose / robot arm via a camera vision navigation system.

[0165] In some embodiments, force sensors in the end effectors and / or joints of the robotic arms can sense spinal reaction forces and have the ability to unlock when the sensed force falls below or exceeds a certain threshold. In some embodiments, after the robot uses navigated instruments to position the vertebral bodies in the desired locations, the surgeon can perform the discectomy and place the interbody device. In some embodiments, sensors on the end effectors or in the robotic arms can measure the force on the endplates during insertion to determine the possibility of endplate failure.

[0166] Referring to FIG. 10, a process 1000 is provided detailing a non-limiting exemplary embodiment for placing pedicle screws using a linear actuator end effector.

[0167] Accordingly, with reference to Figures 11A-11E, different perspective views of the end effector are shown. As discussed above, the end effector is capable of connecting to and manipulating an autonomous surgical tool.

[0168] According to some embodiments, the end effector can include a mounting interface for attaching the end effector to a robot arm such that the mounting interface securely connects the robot arm to a tool. The end effector can further include a central bore containing two main components: a rotor and a stator. The rotor can be securely attached to the mounting interface and thus to the robot arm. The rotor can be concentric with the stator and electromagnetically coupled to the stator such that varying current to the stator rotates the rotor, thereby rotating the engagement tool.

[0169] According to some embodiments, various known or future known surgical tools (e.g., scalpels, drills, etc.) can be inserted into (or otherwise coupled to) the distal portion of the central lumen, and the surgical tool securely engages with the rotor such that its rotation can be driven by the rotation of the rotor. In some embodiments, the tool can be autonomously advanced toward the surgical site by a robotic arm, which simultaneously controls the rotational speed, direction, and position of the tool engaged with the end effector.

[0170] In some embodiments, the disclosed end effector can include a third component such that the interface between the stator and the mounting interface is not rigid, but instead is a linear actuator designed to translate the rotor, stator, and tool along a linear trajectory. Thus, the robot arm can remain stationary in space while the end effector, including the linear actuator, translates the rotary drive mechanism, including the stator, rotor, and tool, while simultaneously controlling the linear and rotational speed, direction, and position of the tool engaged with the end effector.

[0171] As shown in FIGS. 11A-11E, in some embodiments, a housing 1102 connects the robotic arm to the surgical tool. Inside the housing is a rotary actuator consisting of a stator 1104 and a rotor 1106. The stator can be fixed to the housing and can include a wire coil. The rotor can be a magnet electromagnetically coupled to the stator such that current supplied to the stator from the robotic arm causes the rotor to rotate. The rotor can be fixed to a central tube 1108. An inductive encoder 1110 can measure the relative speed and position of the rotor with respect to the stator. The central tube is held within the housing by bearings 1112, which stabilize and allow smooth rotation relative to the housing. The central tube 1108 features a female hexagon to allow the tool to rotate with the central tube and is controlled by the robot controller. The central tube locks the tool in place relative to the rotor. The cover 1114 seals the internal structure making it waterproof and dustproof.

[0172] Referring again to FIG. 10 , according to some embodiments, the process 1000 can include an end effector of a spine surgical robotic arm. In some embodiments, the end effector has an internal lumen capable of interfacing with various surgical tools, as discussed above. The end effector has the ability to engage and disengage tools and advance and retract the tools along a trajectory. The end effector is designed to allow a tool, such as a burr, awl, tap, or screw, to be inserted into the proximal end of the lumen and advanced linearly along the axis of the lumen through the distal end toward the spine.

[0173] According to some embodiments, process 1000 begins at step 1002, where the end effector aligns its medial axis along a trajectory. The trajectory may be determined according to the processes and / or pre-operative planning discussed above.

[0174] In step 1004, an operator loads a tool (e.g., a scalpel) having a geometry that allows the tool (e.g., a scalpel) to interface with the inner lumen into the proximal end of the end effector. The end effector has the ability to lock / engage the scalpel or passively guide the scalpel along a predetermined trajectory and allow the user to move the scalpel linearly along an axis.

[0175] According to some embodiments, when the end effector locks / engages the scalpel, the end effector has the ability to advance the scalpel linearly along its axis into the skin to create a skin incision. In some embodiments, after the incision is created, the tool (e.g., scalpel) can be retracted and withdrawn from the body. In some embodiments, after being fully withdrawn, the scalpel can be unlocked and removed from the proximal end of the end effector.

[0176] In step 1006, the operator then loads the bur. The bur has a geometry that allows it to interface with the inner lumen. In some embodiments, the bur can be locked, advanced, turned on, retracted, turned off, and detached by the robot or the user. In some embodiments, the end effector has the ability to linearly advance the tool along its axis, but the robot has the ability to simultaneously or non-simultaneously move its axis along different trajectories determined by the robot controller to remove bone as dictated by the surgical plan. In some embodiments, after bone removal is complete, the user can retract the bur, detach it from the end effector, and remove it from the proximal end of the end effector.

[0177] In step 1008, the operator then loads the tap with mating geometry into the end effector. In some embodiments, the end effector has the ability to engage the tap, advance, rotate, and retract the tap, and separate the tap. In some embodiments, the tap can be advanced at a set linear and rotational velocity until a force on the tap is sensed by a torque in the robot end effector or joint. From there, the tap can be advanced at a constant velocity, where the rotational velocity is proportional to the applied axial force. In some embodiments, the robot can use a control loop to maintain a constant axial force as it advances the tap.

[0178] In some embodiments, the tap can be rotated at a constant rate and advanced at a rate proportional to the sensed force. In some embodiments, the robot can use a control feedback loop to maintain a constant force. In some embodiments, the tap can be withdrawn by simultaneously rotating and retracting the tool until the tap is fully withdrawn from the patient, unlocked, and removed through the proximal end of the end effector.

[0179] In some embodiments, a multi-axis force sensor can be used in the end effector or on the tool, which can determine biometric or proprioceptive data by sensing vibrations and changes in vibrations from the motor end effector. In some embodiments, the motor vibrations generate a movement profile at the end of the tool, which can be sensed by an accelerometer located at the end of the tool. In some embodiments, changes to the movement profile can be used to algorithmically determine forces on the tool. Just as the robot can sense whether the tool is touching bony anatomical structures, forces such as, but not limited to, contact forces can be sensed and checked against alignment to determine whether alignment is effective. The multi-axis force sensor has the ability to replicate the sensation of touch commonly used by spine surgeons when performing spinal procedures.

[0180] In some embodiments, multi-axis force sensors can be used to sense forces during burring, drilling, tapping, or screwing of the pedicle. These forces can be used to algorithmically determine bone integrity, screw purchase quality, bone pull-out strength, and the likelihood or likelihood that the screw trajectory will exit the bony anatomy, among other biometric data. In some embodiments, force-torque sensors on the robotic arm or in the end effector can be used to sense forces that are used to interpret that data.

[0181] Without navigation or robotics, a common surgeon practice may be to use landmark checking by placing a tool tip and verifying its relative pose and position relative to the vertebrae using fluoroscopic images. In some embodiments, such a workflow can be replicated by an autonomous robot as a validity check for aligned vertebral body poses or to reduce errors in tool tip position relative to the anatomy. In some embodiments, the robot has n arms (e.g., two arms), each arm equipped with an end effector and tool. Each arm can utilize the previously described multi-axis force or force-torque sensors on the robotic arms to contact the vertebrae from both sides without compromising the arm's position in space. In some embodiments, fluoroscopic images can be captured with both tool tips in known positions, tracked by cameras. In some embodiments, automatic segmentation and keypoint detection of the tool and vertebrae in the images can be utilized to determine their pose relative to each other to enable, disable, or refine alignment.

[0182] In step 1010, the operator then loads a screw and screwdriver with mating geometries through the proximal end of the end effector. In some embodiments, the end effector inserts the screw in a manner similar to a tap. In some embodiments, after insertion, the robot can maintain engagement with, and therefore fixation to, the spine, or can separate the screwdriver from the screw and retract the screwdriver through the proximal end of the end effector.

[0183] According to some embodiments, the process 100 can be repeated on the contralateral side using a second robotic arm while maintaining fixation to the screw. Fixation to the implanted screw maintains navigation fidelity, provides counter torque for contralateral insertion, and allows robotic assistance in corrective maneuvers.

[0184] 12, a process 1200 is provided detailing a non-limiting exemplary embodiment for determining tracking array shift from a camera element. In some embodiments, the framework can determine the total movement of the patient tracking array based on camera-centered tracking, as discussed herein. In some embodiments, the framework can perform realignment of the patient tracking array after determining the total movement as instructed from the table-frame-based redundant tracking array and / or camera-centered movement detection, as discussed herein.

[0185] According to some embodiments, process 1200 begins at step 1202, where the patient tracking array is aligned, which can be performed in a manner similar to that discussed above.

[0186] In step 1204, engine 200 may determine whether the camera has moved during a time span (or a predetermined / dynamically determined time period). In some embodiments, this time span may correspond to a predetermined time period, which may be based on time to align, time to perform a surgical step / procedure, time between captured images, etc., or some combination thereof. In some embodiments, engine 200 may cause an IMU associated with the camera (e.g., imaging device 106) to perform such analysis and determination.

[0187] In some embodiments, sensors on the imaging device 106 may be utilized to determine whether a threshold amount of movement has been detected (e.g., a gyroscope, accelerometer, GPS, etc.). In some embodiments, images captured at or near a timespan may be compared, and movement may be detected if there is a pixel difference beyond a threshold amount (e.g., via computer vision, etc.).

[0188] In step 1206, the engine 200 may determine whether the patient tracking array has a gross deformation. Also, in step 1208, after determining the gross deformation, the surgeon may be notified via an output that may be displayed visually on the UI and / or an audible output in a manner similar to that discussed above. Thus, according to some embodiments, the surgeon may be notified to realign the patient-based tracking array if / when a gross deformation of the patient tracking array is observed based on the camera-centered reference frame during a time span in which the IMU signals are not instructing camera movement.

[0189] According to some embodiments, the operations of process 1200 can be utilized to realign the patient tracking array from table-frame and / or detected camera-centered movement based on redundant tracking arrays.

[0190] According to some embodiments, the patient tracking array can be registered (e.g., 2D / 3D merge and / or iCT, as discussed above), thereby, in some embodiments, simultaneously merging the patient tracking array with the redundant table-based tracking array.

[0191] In some embodiments, the table-based tracking array can be used to determine whether there is a total deformation of the patient tracking array, and only one element of the table-based tracking array needs to be visible at any one time to determine the relative movement of the patient tracking array.

[0192] In some embodiments, camera movement can additionally be determined based on referencing a table-based tracking array, and if no movement is detected, total deformation of the patient tracking array can be detected via camera-frame-based movement (e.g., based on a morphology acceptance criterion that indicates that the patient tracking array has moved along the tracking array tool axis).

[0193] In some embodiments, as discussed above, the surgeon is notified to realign the patient-based tracking array when a gross deformation of the patient-based tracking array is observed.

[0194] In some embodiments, the engine 200 may utilize haptic elastomers to realign the vertebral bodies using pose initialization via an iterative closest point algorithm based on a table-based tracking array.

[0195] 13, process 1300 provides a non-limiting example embodiment for avoiding line-of-sight obstructions during a spinal procedure (e.g., pedicle screw insertion / placement). In some embodiments, as discussed herein, engine 200 may operate to determine optimal camera placement for simultaneously viewing the DRB and the instrument tracking array, which may be based on preoperatively planned screw trajectories and / or intraoperative alignment.

[0196] According to some embodiments, process 1300 begins at step 1302, where engine 200 may align the patient tracking array. As discussed above, this may be performed via the above-described processing via iCT and / or 2D / 3D merging, among other mechanisms.

[0197] In step 1304, engine 200 may determine whether the positioned camera captures both the DRB and the planned screw trajectories. According to some embodiments, engine 200 may execute ML / AI algorithms, such as computer vision, neural network analysis, feature vector analysis, logistic regression, hidden Markov modeling, Bayes' theorem, etc. (in addition to other algorithms discussed herein / above). Such ML / AI modeling may be performed based on computational analysis of the pre-operative plan, information regarding the screw trajectories (and / or angles), and the determined camera positions. Thus, in some embodiments, engine 200 may parse a data file containing information regarding, but not limited to, the pre-operative plan, the intra-operative procedure, the angle of incidence of the planned pedicle screw trajectories onto the surface topography, the surface topography, and / or any other information derivable regarding any of the above processes, spinal information, etc., or some combination thereof. To perform the determination of step 1304, such parsed information may be extracted and fed to the ML / AI algorithm executed by engine 200.

[0198] In step 1306, engine 200 may compile as output information regarding the determination of step 1304. In some embodiments, step 1306 may include transmitting information regarding whether the camera placement has a clear view (e.g., without obstructions) of the navigated instrument throughout the axial translation of the screw driver within the planned screw trajectory (e.g., each of the associated operations for implantation and pedicle screw implantation, etc.), for display on a UI, for example.

[0199] In some embodiments, the displayed output may provide suggested moves that may allow for clearance of the obstacle, whereby such suggestions / recommendations may be compiled and displayed when it is determined (from step 1306) that an obstacle is present. In such embodiments, recommendations may be compiled based on calculations, ML / AI analysis, and determinations regarding camera positioning based on the trajectory of each screw.

[0200] According to some embodiments, the disclosed systems and methods can be provided as an overall system that interacts as a process, thereby interacting as processes 300, 500, 700, 800, 1000, 1200, and 1300. Thus, according to some embodiments, the disclosed processing by engine 200, including the control and operation of haptic elastomeric instruments and surgical robots (e.g., FIG. 9A ), can include pre-operative planning and intra-operative procedures (and ultimately post-operative procedures, whereby, among other things, accuracy measures can be used to further train the modeling techniques utilized for each process).

[0201] By way of background, advanced surgical systems, as discussed herein, include many different types of instruments to assist surgeons in performing surgical tasks.

[0202] For example, a medical visualization system refers to a visualization system used to visualize and analyze objects, preferably three-dimensional (3D) objects. Medical visualization systems include surface point selection, region of interest selection, and object selection. Medical visualization systems can be used for diagnostic applications, treatment planning, intraoperative assistance, documentation, and educational purposes. Medical visualization systems can consist of microscopes, endoscopes / arthroscopes / laparoscopes, fiber optics, surgical lights, high-resolution monitors, operating room cameras, etc. 3D visualization software provides a visual representation of the scanned body part via a virtual model, adding significant depth and nuance to static two-dimensional medical images. This software facilitates improved diagnosis, a narrower surgical learning curve, reduced surgical costs, and reduced image acquisition times. According to some embodiments, medical visualization systems can be integrated and / or utilized via the disclosed framework discussed above.

[0203] X-ray may refer to a medical imaging device that uses X-ray radiation (e.g., the X-ray range of the electromagnetic radiation spectrum) to generate images of the inside of the human body for diagnostic and therapeutic purposes. X-ray devices may also be referred to as X-ray generators. X-ray devices are non-invasive devices based on the differential absorption of X-rays by tissues based on their radiological density (radiological density is different for bone and soft tissue). To generate an image with an X-ray device, X-rays generated by an X-ray tube can be sent through a positioned patient to a detector. As the X-rays pass through the body, a black and white shading image may appear, and the shading may depend on the type and density of the tissue through which the X-rays pass. Some applications for which X-rays are used may be fractures, infections, calcifications, tumors, arthritis, vascular blockages, digestive problems, and cardiac problems. X-ray devices may consist of components such as an x-ray tube, an operating console, a collimator, a grid, a detector, and radiographic film. According to some embodiments, X-rays can be utilized via the disclosed framework discussed above.

[0204] MRI can refer to a medical imaging device that uses magnets to generate images of the inside of the human body for diagnostic and therapeutic purposes. Some applications for which MRI may be used include brain / spinal cord abnormalities, internal tumors, breast cancer screening, joint injuries, uterine / pelvic pain detection, and cardiac issues. To generate images with an MRI device, magnetic resonance may be generated by a magnet that generates a magnetic field that induces protons within the body to align with the field. Then, when a radiofrequency current is pulsed through the patient, the protons are stimulated, spinning out of equilibrium and distorting against the pull of the magnetic field. Turning off the radiofrequency field allows the MRI sensor to detect the energy released by the protons' realignment with the magnetic field. The time it takes for the protons to realign with the magnetic field and the energy release may depend on environmental factors and the chemical properties of the molecule. MRI may be suitable for imaging non-bony parts of the body or soft tissues. MRI may be relatively harmless because it does not use harmful ionizing radiation, as is the case with X-ray devices. MRI devices may consist of a magnet, gradient, radiofrequency system, and computer control system. Some areas that should not be imaged by MRI may be people with implants. According to some embodiments, MRI can be utilized via the disclosed framework discussed above.

[0205] CT may refer to a medical imaging tool that uses x-ray radiation (e.g., in the x-ray range of the electromagnetic radiation spectrum) to produce cross-sectional images of the interior of, for example, the human body, for diagnostic and therapeutic purposes. CT may be a computerized x-ray imaging procedure that aims a narrow beam of x-rays at a patient and rapidly rotates the beam around the body to produce signals that are processed by the machine's computer to produce cross-sectional images, or "slices," of the body. CT tools may produce cross-sectional images of the body. A computed tomography tool may differ from an x-ray tool because a computed tomography tool produces a three-dimensional cross-sectional image of the body, while x-rays produce a two-dimensional image of the body. In such cases, the three-dimensional cross-sectional image may be produced by taking images from different angles, which may be performed by taking a series of tomographic images from different angles. The different images may be collected by a computer and digitally stacked to form a three-dimensional image of the patient. To generate images with a CT tool, for example, a CT scanner may use a motorized x-ray source that rotates around a circular opening in a donut-shaped structure called a gantry; during this rotation, an x-ray tube rotates around the patient, capturing a narrow beam of x-rays that pass through the body. Some applications for which CT may be used include blood clots, fractures, including microfractures that are not visible on x-rays, and organ damage. According to some embodiments, CT can be utilized via the disclosed framework discussed above.

[0206] Ultrasound imaging, also known as sonography or ultrasonography, refers to a medical imaging tool that uses ultrasound or sound waves (also called acoustic waves) to create cross-sectional images of the inside of the human body for diagnostic and therapeutic purposes. The ultrasound waves in the tool may be generated by a piezoelectric transducer, which generates sound waves and sends them into the body. The reflected sound waves are converted into electrical signals and sent to an ultrasound scanner. Ultrasound tools may be used for diagnostic and functional imaging. They may be used for therapeutic or interventional procedures. Some applications for ultrasound are diagnosis / treatment / guidance during medical procedures of soft tissue, muscles, blood vessels, tendons, and joints, for example, during biopsies, internal organs such as the liver, kidneys, and pancreas, and during fetal monitoring. Ultrasound may be used internally (transducers are placed within organs, e.g., the vagina) and externally (transducers are placed on the chest for cardiac monitoring or the abdomen for fetuses). An ultrasound machine may consist of a monitor, keyboard, processor, data storage, probe, and transducer. According to some embodiments, ultrasound may be utilized via the disclosed framework discussed above.

[0207] According to some embodiments, a device refers to an item, tool, or set of objects that aids in performing or accomplishing an action or activity. Medical equipment (or medical imaging equipment) refers to an item, instrument, device, or machine used for some health purpose: diagnosing, preventing, or treating a medical condition or disease, or detecting, measuring, restoring, correcting, or altering a body structure / function. Medical equipment may perform functions invasively or non-invasively. Medical equipment may consist of components such as sensors / transducers, signal conditioners, displays, and data storage units. Medical equipment functions by receiving signals from a subject / patient, a transducer for converting one form of energy into electrical energy, an amplifier for converting the output from the transducer into an electrical value, a signal conditioner such as a filter, a display for providing a visual representation of the measured parameter or quantity, and a storage system for storing data that can be used for later reference. Medical equipment may perform diagnostic functions or provide therapy; for example, a device may deliver air / breath into the lungs and move it out of the lungs to a patient who is physically unable to breathe or cannot breathe adequately. According to some embodiments, medical devices can be utilized via the disclosed framework discussed above.

[0208] A robotic system is a system that interacts with its environment, including humans, to provide intelligent services and information through the use of various sensors, actuators, and human interfaces. Robotic systems are used to automate processes in a wide range of applications, including industrial (manufacturing), domestic, medical, service, military, entertainment, and space. The adoption of robotic systems offers several benefits, including increased efficiency and speed, lower cost, and greater precision. Performing medical procedures with the aid of robotic technology is called a medical robotic system. The medical robotic systems market can be segmented by product type into surgical robotic systems, rehabilitation robotic systems, non-invasive radiosurgery robots, and hospital and pharmacy robotic systems. Robotic technology has provided beneficial enhancements to medical or surgical processes through improved precision, stability, and dexterity. In medicine, robots are useful by freeing medical personnel from routine tasks and making medical procedures safer and less costly for patients. Robots can also perform precise surgical procedures in tight spaces and transport hazardous materials. Robotic surgery is performed using a telemanipulator, which uses the surgeon's movements on one hand and controls "effectors" on the other. Medical robotic systems ensure precision and may be used for remotely controlled minimally invasive procedures. The systems include computer-controlled electromechanical devices that function in response to controls operated by the surgeon. According to some embodiments, robotic systems can be utilized via the disclosed framework discussed above.

[0209] 14 is a schematic diagram illustrating an exemplary embodiment of a client device that may be used within the present disclosure. Client device 1400 may include more or fewer components than those shown in FIG. 14. However, the components shown are sufficient to disclose an exemplary embodiment for practicing the present disclosure. Client device 1400 may represent, for example, UE 102 discussed above with respect to at least FIG. 1.

[0210] As shown in this figure, in some embodiments, client device 1400 includes a processing unit (CPU) 1422 in communication with mass memory 1430 via a bus 1424. Client device 1400 further includes a power supply 1426, one or more network interfaces 1450, an audio interface 1452, a display 1454, a keypad 1456, an illuminator 1458, an input / output interface 1460, a tactile interface 1462, an optional global positioning system (GPS) receiver 1464, and a camera or other light, heat, or electromagnetic sensor 1466. As will be appreciated by those skilled in the art, device 1400 can include one camera / sensor 1466 or multiple cameras / sensors 1466. Power supply 1426 provides power to client device 1400.

[0211] The client device 1400 may optionally communicate with a base station (not shown) or may communicate directly with another computing device. In some embodiments, the network interface 1450 is sometimes known as a transceiver, a transceiver device, or a network interface card (NIC).

[0212] In some embodiments, audio interface 1452 is arranged to generate and receive audio signals, such as the sound of a human voice. Display 1454 may be a liquid crystal display (LCD), gas plasma, light emitting diode (LED), or any other type of display used with a computing device. Display 1454 may further include a touch-sensitive screen arranged to receive input from an object, such as a stylus or a human finger.

[0213] Keypad 1456 may include any input device arranged to receive input from a user. Illuminator 1458 may provide status indicators and / or light.

[0214] Client device 1400 further includes an input / output interface 1460 for communicating with the outside world. In some embodiments, input / output interface 1460 may utilize one or more communication technologies, such as USB, infrared, Bluetooth™, etc. Haptic interface 1462 is arranged to provide tactile feedback to a user of the client device.

[0215] The optional GPS receiver 1464 can determine the physical coordinates of the client device 1400 on the surface of the Earth, which typically outputs a location as a latitude and longitude value. The GPS receiver 1464 can also use other global positioning mechanisms, such as, but not limited to, trigonometry, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSS, etc., to further determine the physical location of the client device 1400 on the surface of the Earth. However, in one embodiment, the client device may provide other information, including, for example, a MAC address, an Internet Protocol (IP) address, etc., through other components, that may be used to determine the device's physical location.

[0216] Mass memory 1430 includes RAM 1432, ROM 1434 and other storage means. Mass memory 1430 represents another example of a computer storage medium for storing information such as computer-readable instructions, data structures, program modules, etc. Mass memory 1430 stores basic input / output system ("BIOS") 1440 for controlling low-level operation of client device 1400. Mass memory also stores operating system 1441 for controlling operation of client device 1400.

[0217] Memory 1430 further includes one or more data stores that can be utilized by client device 1400 to store, among other things, applications 1442 and / or other information or data. For example, a data store may be used to store information describing various capabilities of client device 1400. This information may then be provided to another device based on any of a variety of events, including being sent as part of a header during a communication (e.g., an index file for an HLS stream), being sent in response to a request, etc. At least a portion of the capability information may also be stored on a disk drive or other storage medium (not shown) within client device 1400.

[0218] Applications 1442 may include computer-executable instructions that, when executed by client device 1400, send, receive, and / or otherwise process audio, video, images, and enable electronic communication with a server and / or another user of another client device. Applications 1442 may also include clients configured to send, receive, and / or otherwise process games, goods / services, and / or other forms of data, messages, and content hosted and offered by the platform associated with engine 200 and its affiliates.

[0219] As used herein, the terms "computer engine" and "engine" identify at least one software component and / or a combination of at least one software component and at least one hardware component that is designed / programmed / configured to manage / control other software and / or hardware components (such as libraries, software development kits (SDKs), objects, etc.).

[0220] Examples of hardware elements may include a processor, a microprocessor, a circuit, a circuit element (e.g., a transistor, a resistor, a capacitor, an inductor, etc.), an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a digital signal processor (DSP), a field programmable gate array (FPGA), a logic gate, a register, a semiconductor device, a chip, a microchip, a chipset, etc. In some embodiments, one or more processors may be implemented as a complex instruction set computer (CISC) or reduced instruction set computer (RISC) processor, an x86 instruction set compatible processor, a multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, one or more processors may be a dual-core processor, a dual-core mobile processor, etc.

[0221] As used herein, computer-related system, computer system, and system include any combination of hardware and software. Examples of software may include software components, programs, applications, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computer code, computer code segments, words, values, symbols, or any combination thereof. The decision whether an embodiment is implemented using hardware and / or software elements may depend on any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budgets, input data rates, output data rates, memory resources, data bus speeds, and other design or performance constraints.

[0222] For purposes of this disclosure, a module is a software, hardware, or firmware (or combination thereof) system, process, or function, or component thereof, that performs or facilitates the processes, features, and / or functions described herein (with or without human interaction or augmentation). A module may include sub-modules. The software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integral with one or more servers or may be loaded or executed by one or more servers. One or more modules may be grouped into an engine or application.

[0223] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium representing various logic within a processor, which, when read by a machine, causes the machine to execute logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on tangible machine-readable media and supplied to various customers or manufacturing facilities for loading into manufacturing machines that produce the logic or processors. Of course, the various embodiments described herein may be implemented using any suitable hardware and / or computing software language (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc.).

[0224] For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may be downloadable from a network, e.g., a website, as a stand-alone product or as an add-in package for installation into an existing software application. For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may also be usable as a client-server software application or as a web-enabled software application. For example, exemplary software specially programmed in accordance with one or more principles of the present disclosure may also be embodied as a software package installed on a hardware device.

[0225] For purposes of this disclosure, the terms “user,” “subscriber,” “consumer,” or “customer” should be understood to refer to a user of one or more applications described herein and / or a consumer of data provided by a data provider. By way of example, and not limitation, the terms “user” or “subscriber” can refer to a person receiving data or data provided by a service provider over the Internet in a browser session, or to an automated software application that receives, stores, or processes the data. Those skilled in the art will recognize that the methods and systems of the present disclosure can be implemented in many ways and, therefore, are not limited by the exemplary embodiments and examples described above. In other words, functional elements performed by single or multiple components in various combinations of hardware and software or firmware, and individual functions can be distributed among software applications, at the client level, the server level, or both. In this regard, any number of features of different embodiments described herein may be combined into a single or multiple embodiments, and alternative embodiments having fewer or more than all of the features described herein are also possible.

[0226] Furthermore, functionality may be distributed, in whole or in part, among multiple components in ways now known or hereafter known. Thus, countless software / hardware / firmware combinations are possible to achieve the functions, features, interfaces, and preferences described herein. Moreover, the scope of the present disclosure covers conventionally known ways for implementing the described features and functions and interfaces, as well as variations and modifications thereof that may be made to the hardware, software, or firmware components described herein, as would be understood by one of ordinary skill in the art, now or in the future.

[0227] Furthermore, the method embodiments shown and described in this disclosure as flow charts are provided as examples to provide a more complete understanding of the present technology. The disclosed methods are not limited to the operations and logical flow presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.

[0228] For purposes of this disclosure, various embodiments have been described, but such embodiments should not be construed as limiting the teachings of this disclosure to those embodiments. Various changes and modifications to the elements and operations described above can be made to achieve results that fall within the scope of the systems and processes described in this disclosure.

Claims

1. identifying, by the device, a medical image of the patient; analyzing, by the device, a medical image, the analysis including automatic segmentation of the medical image; determining, by the device, an optimal position for surface-based registration based on the analysis; and performing, by said device, alignment of a patient tracking array; determining the accuracy of pedicle screw placement with the device; generating a vertebral body specific alignment with said device; outputting, by the device, tracking information based at least on the generated vertebral body specific registration and / or patient tracking array, the output being displayed within a user interface (UI); A method comprising:

2. generating a surface topography based on the patient tracking array; determining a set of points associated with the set of touch points; navigating the set of points within a navigation space; determining a registration accuracy based on the navigation; determining and outputting instructions to realign or continue based on the alignment accuracy; The method of claim 1 further comprising:

3. determining information regarding planned pedicle screw implantation based on the automated segmentation of the medical image; determining a skive probability based on the applied skive model; determining an optimal pilot hole based on the skive probability; outputting information about the optimal pilot hole for display within the UI; The method of claim 1 further comprising:

4. determining whether a camera has moved during a time span based at least in part on the patient tracking array; determining that the patient tracking array has a total deformation; outputting a notification via the UI indicating the need to realign the patient tracking array; The method of claim 1 further comprising:

5. determining whether a positioned camera captures both a dynamic reference base (DRB) and a planned screw trajectory based at least in part on the patient tracking array; and outputting information regarding the determination of the camera positioning via the UI, wherein a continue message is provided when both the DRB and planned screw trajectory are captured, and a reposition message is provided when both are not captured. The method of claim 1 further comprising:

6. determining a dynamic re-registration and reconstruction (DRR) simulation, the DDR simulation corresponding to a theoretical fluoroscopic image based on an instantaneous position of the C-arm of the device and the patient relative to each other at any time after initial registration; outputting the DDR simulation to a user interface (UI); The method of claim 1 further comprising:

7. A surgical robot, a first robotic arm configured to grasp and control a medical instrument; a second robotic arm configured to grasp and control the medical instrument; and Equipped with Each robotic arm is configured to execute computer-executable instructions to accomplish the performance of a surgical procedure, the surgical robot comprising: Rotary Actuator Effector and wherein the rotary actuator effector comprises: a mounting interface for attaching end effectors to the first and second robotic arms such that the mounting interface securely connects the robotic arms to the medical instrument; and a central bore containing two main components, a rotor and a stator, the stator being rigidly attached to the mounting interface and thus to the robot arm, the rotor being concentric with the stator, and the rotor being electromagnetically coupled to the stator such that varying current to the stator causes the rotor to rotate, thereby rotating the engagement tool; and A surgical robot comprising:

8. The surgical robot of claim 7 , wherein the surgical procedure is implantation of at least one pedicle screw.

9. 8. The surgical robot of claim 7, wherein each end effector of the rotary actuator effector is coupled to at least one of a force sensor and a torque sensor such that the surgical robot is configured to advance a screw into a patient by utilizing at least one of a force feedback control loop and a torque feedback control loop, and the advancement of the screw is controlled via an associated motor that controls a rotational speed and a rate of the advancement.

10. 8. The surgical robot of claim 7, wherein each end effector of the rotary actuator effector is coupled to a multi-axis force sensor, the multi-axis force sensor configured to read motor vibrations and algorithmically determine a multi-axis force corresponding to at least one of the rotary actuator effector and an end effector engaged by the rotary actuator effector.

11. 10. The surgical robot of claim 7, wherein the force-torque sensor control loop derives bone quality, screw-bone interface heuristics, and other biomechanical data based on data collected during burring, drilling, and / or screw and tap insertion.

12. manipulating the patient's spine through a predetermined range of motion; calculating translational and / or rotational force curves; determining an optimal rod curvature for verification of spinal alignment; performing the alignment of the spine based on the determined optimal rod curvature; and The surgical robot of claim 7 further comprising:

13. The surgical robot of claim 7 , wherein the engagement tool includes a pedicle screw for securely attaching the surgical robot to a patient.

14. A circuit board; A camera and A light source and a distal tip that conforms to an anatomical surface that it contacts; and a reflective layer within the elastomer that reflects light through the transparent backstop; Equipped with The tactile elastomer device, wherein the circuit board captures the resulting data from the camera and transmits the data to an external processor.

15. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, which when executed by a device: identifying, by the device, a medical image of the patient; analyzing, by the device, a medical image, the analysis including automatic segmentation of the medical image; determining, by the device, an optimal position for surface-based registration based on the analysis; and performing, by said device, alignment of a patient tracking array; determining the accuracy of pedicle screw placement with the device; generating a vertebral body specific alignment with said device; outputting, by the device, tracking information based at least on the generated vertebral body specific registration and / or patient tracking array, the output being displayed within a user interface (UI); A non-transitory computer-readable storage medium for performing a method comprising:

16. generating a surface topography based on the patient tracking array; determining a set of points associated with the set of touch points; navigating the set of points within a navigation space; determining a registration accuracy based on the navigation; determining and outputting instructions to realign or continue based on the alignment accuracy; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

17. determining information regarding planned pedicle screw implantation based on the automated segmentation of the medical image; determining a skive probability based on the applied skive model; determining an optimal pilot hole based on the skive probability; outputting information about the optimal pilot hole for display within the UI; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

18. determining whether a camera has moved during a time span based at least in part on the patient tracking array; determining that the patient tracking array has a total deformation; outputting a notification via the UI indicating the need to realign the patient tracking array; 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

19. determining whether a positioned camera captures both a dynamic reference base (DRB) and a planned screw trajectory based at least in part on the patient tracking array; and outputting information regarding the determination of the camera positioning via the UI, wherein a continue message is provided when both the DRB and planned screw trajectory are captured, and a reposition message is provided when both are not captured.

16. The non-transitory computer-readable storage medium of claim 15, further comprising:

20. determining a dynamic re-registration and reconstruction (DRR) simulation, the DDR simulation corresponding to a theoretical fluoroscopic image based on an instantaneous position of the C-arm of the device and the patient relative to each other at any time after initial registration; outputting the DDR simulation to a user interface (UI); 16. The non-transitory computer-readable storage medium of claim 15, further comprising:

Images