Method and system for alignment parameters of a surgical object such as the spine - Patents.com

Abstract

Disclosed herein are methods and systems for intra-operatively determining alignment parameters of a spine during a surgical procedure of the spine. In some embodiments, the method for intra-operatively determining alignment parameters of a spine during a surgical procedure includes receiving initial image data of a spine including a plurality of vertebrae and identifying geometric features associated with each vertebra in the initial image data. The geometric features each have a pose in the initial image data and characterize a three-dimensional (3D) shape of the associated vertebra. The method further includes receiving intra-operative image data of the spine and registering the initial image data to the intra-operative image data. The method may further include updating the pose of each geometric feature based on the registration and the intra-operative image data, and determining alignment parameters based on the updated poses of the geometric features associated with the two or more vertebrae.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. patent application Ser. No. 17 / 735,945, entitled "METHOD AND SYSTEM FOR DETERMINING ALIGNMENT PARAMETERS OF A SURGICAL OBJECT, SUCH AS THE SPINAL COLUMN," filed May 3, 2022, the entire contents of which are incorporated herein by reference.

[0002] (Technical field) The present technology relates generally to methods and systems for intra-operatively determining spinal alignment parameters of a patient undergoing spinal surgery. [Background technology]

[0003] Spinal deformities include pathological curvatures of the spine and may occur naturally or as a result of spinal disease or injury. Spinal deformities may occur along the coronal, sagittal, and / or axial planes and may include scoliosis, lordosis, hypolordosis, kyphosis, hypokyphosis, etc. In some instances, spinal deformities require surgical correction. Spinal surgery may be done by exposing a portion of the spine and / or via minimally invasive techniques that physically manipulate the bones of the spine into a corrected configuration, temporarily fixing them with surgical hardware (e.g., pedicle screws, rod and tower devices, interbody devices) until bone growth creates a permanent fixation.

[0004] Existing navigation systems for spinal deformity surgery are limited in guiding hardware placement. Thus, surgeons must rely on manual interpretation of either (i) exposed physical anatomy or (ii) two-dimensional (2D) or three-dimensional (3D) radiographic (e.g., x-ray) or computed tomography (CT) images acquired during surgery to assess progress of surgery, such as correction to spinal deformity. Subjective mental assessment of exposed physical anatomy and the operative field by the surgeon is challenging, potentially inaccurate, and / or non-repeatable, and introduces more than just "skill" into the surgical procedure. While interpretation of existing 2D or 3D x-ray images can reduce the degree of subjectivity in assessing correction, the image acquisition process disrupts the surgical workflow and exposes both the patient and the surgical team to harmful amounts of partially ionizing radiation. Many aspects of the present disclosure can be better understood with reference to the following drawings, in which components are not necessarily drawn to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 is a schematic diagram of an imaging system in accordance with an embodiment of the present technology. [Diagram 2] FIG. 2 is a perspective view of a surgical environment using the imaging system of FIG. 1 for surgical applications in accordance with an embodiment of the present technology. [Diagram 3] FIG. 3 is an isometric view of a portion of the imaging system of FIG. 1 illustrating four cameras of the imaging system in accordance with an embodiment of the present technology. [Figure 4A] FIG. 4A illustrates several different types of spinal deformities that can occur along the coronal and sagittal planes in accordance with an embodiment of the present technology. [Figure 4B] FIG. 4B illustrates several different types of spinal deformities that can occur along the coronal and sagittal planes in accordance with an embodiment of the present technology. [Diagram 5] FIG. 5 is a flow diagram of a process or method for intra-operatively determining spinal alignment parameters during a spinal surgical procedure, in accordance with an embodiment of the present technology. [Figure 6A] FIG. 6A is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6B] FIG. 6B is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6C] FIG. 6C is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6D] FIG. 6D is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6E] FIG. 6E is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6F] FIG. 6F is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6G] FIG. 6G is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 6H] FIG. 6H is an isometric view of a vertebra of a spinal column including various features of the vertebra's three-dimensional (3D) profile, in accordance with an embodiment of the present technology. [Figure 7A] 7A-7D are anterior and lateral views, respectively, of a spinal column including multiple vertebrae and corresponding superior endplates characterized in initial image data of the spine, in accordance with an embodiment of the present technology. [Figure 7B] 7A and 7B are anterior and lateral views, respectively, of a spinal column including multiple vertebrae and corresponding superior endplates characterized in initial image data of the spine, in accordance with an embodiment of the present technology. [Figure 8A]8A-8D are anterior and lateral views, respectively, of a spinal column including multiple vertebrae and corresponding underlying endplates characterized in initial image data of the spine, in accordance with an embodiment of the present technology. [Figure 8B] 8A and 8B are anterior and lateral views, respectively, of a spinal column including multiple vertebrae and corresponding underlying endplates characterized in initial image data of the spine, in accordance with an embodiment of the present technology. [Figure 9A] FIG. 9A illustrates the progression of output / data capture of the system of FIG. 1 throughout a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, in accordance with an embodiment of the present technology. [Figure 9B] FIG. 9B illustrates the progression of output / data capture of the system of FIG. 1 throughout a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, in accordance with an embodiment of the present technology. [Figure 9C] FIG. 9C illustrates the progression of output / data capture of the system of FIG. 1 throughout a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, in accordance with an embodiment of the present technology. [Figure 9D] FIG. 9D illustrates the progression of output / data capture of the system of FIG. 1 throughout a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, in accordance with an embodiment of the present technology. [Figure 9E] FIG. 9E illustrates the progression of output / data capture of the system of FIG. 1 throughout a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, in accordance with an embodiment of the present technology. [Figure 10] FIG. 10 is a flow diagram of a process or method for registering initial image data to intra-operative image data based on one or more surgical devices, in accordance with an embodiment of the present technology. [Figure 11]FIG. 11 is a flow diagram of a process or method for registering initial image data to intra-operative image data by tracking markers affixed to one or more spinal vertebrae, in accordance with an embodiment of the present technology. [Figure 12] FIG. 12 is a flow diagram of a process or method for non-invasively determining alignment parameters of a patient's spine in accordance with an embodiment of the present technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Aspects of the present technology are generally directed to methods and systems for intra-operatively determining alignment parameters of a patient's spine, for example, during a spinal surgical procedure. In some embodiments described below, a method of intra-operatively determining alignment parameters of a spine during a surgical procedure on the spine includes receiving initial image data (e.g., pre-operative computed tomography (CT) data) of a spine including a plurality of vertebrae and individually recognizing geometric features associated with each vertebra in the initial image data. The geometric features each have a pose in the initial image data and characterize a three-dimensional (3D) profile of the associated vertebra. The method further includes receiving intra-operative image data of the spine, for example, from a camera array positioned on the surgical field. The method further includes registering the initial image data to the intra-operative image data and updating the pose of each geometric feature based on the registration and the intra-operative image data. The method further includes determining alignment parameters of the spine based on the updated poses of the geometric features associated with two or more vertebrae.

[0007] In some embodiments, the alignment parameters characterize the curvature of the spine, such as the Cobb angle or other angles. In some aspects of the present technology, the methods can determine the alignment parameters continuously, substantially in real time, throughout the surgical procedure, such that the alignment parameters continually reflect the true physical alignment of the spine. A surgeon or other user can use the determined spinal alignment parameters to track their progress through the surgical procedure and / or ascertain the effects thereon.

[0008] Specific details of some embodiments of the present technology are described herein with reference to Figures 1-12. The present technology, however, can be practiced without some of these specific details. In some examples, well-known structures and techniques often associated with camera arrays, light field cameras, image reconstruction, registration processes, spine analysis, spinal deformation, etc., have not been shown in detail so as not to obscure the present technology.

[0009] The terms used in the description provided below are intended to be interpreted in their broadest reasonable manner, even though they are used in conjunction with a detailed description of certain specific embodiments of the present disclosure. Certain terms may even be highlighted below, however, any terms intended to be interpreted in any restrictive manner will be expressly and specifically defined as such in this detailed description section.

[0010] The accompanying drawings represent embodiments of the present technology and are not intended to limit its scope. Represented elements are not necessarily drawn to scale, and various elements may be optionally expanded to improve readability. Details of components may be abstracted in the drawings to exclude details, since such details are not necessary for a complete understanding of how to make and use the technology. Many of the details, dimensions, angles, and other features shown in the figures are merely illustrative of certain embodiments of the present disclosure. Thus, other embodiments may have other dimensions, angles, and features without departing from the spirit or scope of the technology.

[0011] The headings provided herein are for convenience only and should not be construed as limiting the disclosed subject matter. To the extent that any material incorporated by reference herein conflicts with the present disclosure, the present disclosure controls.

[0012] Selected Embodiments of the Imaging System 1 is a schematic diagram of an imaging system 100 ("system 100") in accordance with an embodiment of the present technology. In some embodiments, system 100 can be an artificial augmented reality system, a virtual reality imaging system, an augmented reality imaging system, a mediated reality imaging system, and / or a non-immersive computational imaging system. In the illustrated embodiment, system 100 includes a processing device 102 communicatively coupled to one or more display devices 104, one or more input controls 106, and a camera array 110. In other embodiments, system 100 can include additional, fewer, or different components. In some embodiments, system 100 includes some features that are generally similar or identical to the features of the mediated reality imaging systems disclosed in (i) U.S. patent application Ser. No. 16 / 586,375, filed September 27, 2019, entitled "Camera Array for Mediated Reality Systems," and / or (ii) U.S. patent application Ser. No. 15 / 930,305, filed May 12, 2020, entitled "Method and System for Photographing a Scene, Such as a Medical Scene, and Tracking Objects Within the Scene," each of which is incorporated by reference herein in its entirety.

[0013] In the illustrated embodiment, the camera array 110 includes multiple cameras 112 (individually identified as cameras 112a-112n, which may also be referred to as first cameras) that can each capture an image of the scene 108 from a different perspective (e.g., first image data). The scene 108 may include, for example, a patient undergoing a surgical procedure (e.g., spinal surgery) and / or another medical procedure. In other embodiments, the scene 108 may be another type of scene. The camera array 110 may further include dedicated object tracking hardware 113 (e.g., including individually identified trackers 113a-113n) that captures position data of one or more objects, such as an instrument 101 (e.g., a surgical instrument or tool) having a tip 111, to track the movement and / or orientation of the object through / in the scene 108. In some embodiments, the cameras 112 and trackers 113 are positioned at fixed positions and orientations (e.g., poses) relative to one another. For example, the cameras 112 and trackers 113 may be structurally fixed by / to a mounting structure (e.g., a frame) at predefined and fixed positions and orientations. In some embodiments, the cameras 112 are positioned such that adjacent cameras 112 share overlapping fields of view of the scene 108. Typically, the positions of the cameras 112 may be selected to maximize clarity and accuracy of capture of all or a selected portion of the scene 108. Similarly, the trackers 113 may be positioned such that adjacent trackers 113 share overlapping fields of view of the scene 108. Thus, all or a subset of the cameras 112 and trackers 113 may have different unrelated parameters, such as positions and orientations.

[0014] In some embodiments, the cameras 112 in the camera array 110 are synchronized (within a threshold temporal error) to capture images of the scene 108 simultaneously. In some embodiments, all or a subset of the cameras 112 are light field / plenoptic / RGB cameras that capture information about the light field emanating from the scene 108 (e.g., information about the intensity of light rays at the scene 108 and information about the direction the light rays are traveling through space). Thus, in some embodiments, the images captured by the cameras 112 encode depth information that represents the surface geometry of the scene 108. In some embodiments, the cameras 112 are substantially identical. In other embodiments, the cameras 112 include multiple cameras of different types. For example, different subsets of the cameras 112 can have different intrinsic parameters such as focal length, sensor type, optics, etc. The cameras 112 can have charge-coupled device (CCD) and / or complementary metal-oxide semiconductor (CMOS) image sensors and associated optics. Such optical elements can include a variety of configurations, including individual image sensors, lensed or unlensed, in combination with larger microlenses, microlens arrays, prisms, and / or negative lenses. For example, the cameras 112 can be separate light field cameras, each with their own image sensor and optical elements. In other embodiments, some or all of the cameras 112 can include separate microlenslets (e.g., lenslets, lenses, microlenses) of a microlens array (MLA) that share a common image sensor.

[0015] In some embodiments, the tracker 113 is an imaging device such as an infrared (IR) camera that can capture images of the scene 108 from a different perspective compared to one of the other trackers 113. Thus, the tracker 113 and the camera 112 can have different spectral sensitivities (e.g., infrared vs. visible wavelengths). In some embodiments, the tracker 113 captures image data of multiple optical markers (e.g., fiducial markers, marker balls) in the scene 108, such as a marker 115 coupled to the instrument 101.

[0016] In the illustrated embodiment, the camera array 110 further includes a depth sensor 114. In some embodiments, the depth sensor 114 includes (i) one or more projectors 116 that project a structured light pattern onto / into the scene 108, and (ii) one or more depth cameras 118 (which may also be referred to as second cameras) that capture second image data of the scene 108 including the structured light projected onto the scene 108 by the projectors 116. The projectors 116 and the depth cameras 118 may operate at the same wavelength, or in some embodiments, at a different wavelength than the cameras 112. For example, the cameras 112 may capture first image data in the visible spectrum, while the depth cameras 118 capture second image data in the infrared spectrum. In some embodiments, the depth cameras 118 have a lower resolution than the resolution of the cameras 112. For example, the depth camera 118 may have a resolution that is 70%, 60%, 50%, 40%, 30%, or 20% lower than the resolution of the camera 112. In other embodiments, the depth sensor 114 may include other types of dedicated depth sensing hardware (e.g., a LiDAR detector) for determining the surface geometry of the scene 108. In other embodiments, the camera array 110 may omit the projector 116 and / or the depth camera 118.

[0017] In the illustrated embodiment, the processing device 102 includes an image processing device 103 (e.g., an image processor, an image processing module, an image processing unit), a registration processing device 105 (e.g., a registration processor, a registration processing module, a registration processing unit), a tracking processing device 107 (e.g., a tracking processor, a tracking processing module, a tracking processing unit), and an alignment processing device 109. The image processing device 103 can (i) receive first image data (e.g., a light field image, a hyperspectral image, a light field image data, an RGB image) captured by the camera 112 and depth information (e.g., a second image data captured by the depth camera 118) from the depth sensor 114, and (ii) process the image data and the depth information to synthesize (e.g., generate, reconstruct, render) a three-dimensional (3D) output image of the scene 108 that corresponds to a virtual camera viewpoint. The output image can correspond to an approximation of an image of the scene 108 that would be captured by a camera positioned at any position and orientation that corresponds to the virtual camera viewpoint. In some embodiments, the image processing device 103 can further receive and / or store calibration data for the camera 112 and / or the depth camera 118 and synthesize an output image based on the image data, depth information, and / or calibration data. More specifically, the depth information and calibration data can be used / combined with images from the camera 112 to synthesize an output image as a 3D (or stereoscopic 2D) rendering of the scene 108 as seen from the virtual camera viewpoint. In some embodiments, the image processing device 103 can synthesize the output image using any of the methods disclosed in U.S. Patent Application No. 16 / 457,780, filed June 28, 2019, entitled "Synthesizing Images from a Virtual Viewpoint Using Pixels from a Physical Imaging Element Array Weighted Based on Depth Error Sensitivity," which is incorporated herein by reference in its entirety. In other embodiments, the image processing device 103 can generate a virtual camera viewpoint based solely on images captured by the camera 112 without utilizing depth information from the depth sensor 114.For example, the image processing device 103 can generate a virtual camera viewpoint by interpolating between different images captured by one or more cameras 112 .

[0018] The image processing device 103 may synthesize an output image from images captured by a subset (e.g., two or more) of the cameras 112 in the camera array 110, and need not necessarily utilize images from all the cameras 112. For example, for a given virtual camera viewpoint, the processing device 102 may select a pair of stereoscopic images from two of the cameras 112. In some embodiments, such a stereoscopic pair may be selected to be positioned and oriented to most closely match the virtual camera viewpoint. In some embodiments, the image processing device 103 (and / or the depth sensor 114) evaluates depth for each surface point of the scene 108 relative to a common origin to generate a point cloud and / or a 3D mesh that represents the surface geometry of the scene 108. Such a representation of the surface geometry may be referred to as a depth map, an N35 surface, a depth surface, and / or the like. In some embodiments, the depth camera 118 of the depth sensor 114 detects structured light projected onto the scene 108 by the projector 116 to evaluate depth information of the scene 108. In some embodiments, the image processing device 103 estimates depth from multi-view image data from the cameras 112 using techniques such as light field correspondence, stereo block matching, photometric symmetry, correspondence, defocus, block matching, texture-assisted block matching, structured light, etc., with or without utilizing information collected by the depth sensor 114. In other embodiments, depth can be obtained by a specialized set of cameras 112 performing the aforementioned methods at another wavelength.

[0019] In some embodiments, the registration processing device 105 receives and / or stores pre-captured or initial image data, such as three-dimensional volume image data (3D image data) of a patient. The image data may include, for example, computed tomography (CT) scan data, magnetic resonance imaging (MRI) scan data, ultrasound images, fluoroscope images, and / or other medical or other image data. The registration processing device 105 may register the initial image data to real-time images captured by the camera 112 and / or depth sensor 114, for example, by determining one or more transformations / transformations / mappings between the two. The processing device 102 (e.g., image processing device 103) may further apply one or more transformations to the initial image data such that the initial image data may be matched (e.g., overlaid on) an output image of the scene 108 in real-time or near real-time, on a frame-by-frame basis, even as the virtual viewpoint changes. That is, the image processing device 103 can fuse the initial image data with a real-time output image of the scene 108 to provide a mediated reality view that allows the surgeon to simultaneously view the surgical site and the underlying 3D anatomical structures of the patient undergoing surgery at the scene 108, for example. In some embodiments, the registration processing device 105 can register the pre-captured image data to the real-time image using any of the methods disclosed in U.S. patent application Ser. No. 17 / 140,885, filed Jan. 4, 2021, entitled "Method and System for Registering Pre-Operative Image Data to Intra-Operative Image Data at a Scene Such as a Surgical Scene."

[0020] In some embodiments, the tracking processing device 107 processes position data captured by the trackers 113 to track objects (e.g., instruments 101) within the perimeter of the scene 108. For example, the tracking processing device 107 can determine the position of the markers 115 in 2D images captured by two or more trackers 113 and determine the 3D position of the markers 115 through triangulation of the 2D position data. More specifically, in some embodiments, the trackers 113 include dedicated processing hardware for determining position data from the captured images, such as the centroid of the markers 115 in the captured images. The trackers 113 can then communicate the position data to the tracking processing device 107 to determine the 3D position of the markers 115. In other embodiments, the tracking processing device 107 can receive raw image data from the trackers 113. In surgical applications, for example, the tracked objects can include surgical instruments, implants, a doctor's or assistant's hands or arms, and / or another object to which the markers 115 are attached. In some embodiments, the processing device 102 can recognize that a tracked object has left the scene 108 and can apply visual effects to the 3D output image to distinguish the tracked object, for example, by highlighting the object, labeling the object, and / or applying transparency to the object.

[0021] In some embodiments, the alignment processing device 109 determines (e.g., measures, calculates, computes) various alignment parameters (e.g., geometric parameters) for a surgical procedure, such as one or more angles, areas, volumes, distances, and / or the like. For example, for a spinal surgical procedure, the alignment processing device 109 may determine ..., such as Cobb angles (first, second, and / or third), lumbar lordosis measurements, thoracic lordosis measurements, cervical lordosis measurements, sagittal vertical axis, pelvic tilt angle, pelvic angle, sacral tilt angle, pelvic tilt angle, vertebral rotation angles, segmental lordosis / kyphosis angle measurements, anterior disc height, posterior disc height, foraminal height, foraminal area, disc volume, spondylolisthesis grading (e.g., At least one of the following may be determined: vertebral height measurements (anterior, posterior, left, and / or right), C7 plumb line (C7PL), central sacral vertical line (CSVL), lateral spondylolisthesis grading, sagittal vertical axis (SVA), T1 tilt angle, T1 pelvic angle, L1 tilt angle, L1 pelvic angle, etc. As described in detail below with reference to FIGS. 4-12, the alignment processing device 109 may determine alignment parameters of the spine in real-time or near real-time (e.g., substantially real-time) based on (i) initial image data of the spine, (ii) intraoperative image data of the spine captured by the camera array 110, and (iii) registration between the initial image data and the intraoperative image data of the spine. Additionally, the system 100 may calculate a formula and / or score based on the alignment parameters, such as a global alignment and percentage (GAP) score.

[0022] In some embodiments, the functionality ascribed to the processing device 102, image processing device 103, registration processing device 105, tracking processing device 107, and / or alignment processing device 109 may be practically implemented by two or more physical devices. For example, in some embodiments, a synchronization controller (not shown) controls the images displayed by the projector 116 and transmits synchronization signals to the cameras 112 to enable high-speed, multi-frame, multi-camera structured light scanning and ensure synchronization between the cameras 112 and the projector 116. In addition, such a synchronization controller may act as a parameter server that stores hardware-specific configurations such as parameters for structured light scanning, camera settings, and camera calibration data specific to the camera configuration of the camera array 110. The synchronization controller may be implemented in a separate physical device from the display controller that controls the display device 104, and the devices may be integrated together.

[0023] The processing device 102 includes a processor and may include a non-transitory computer-readable storage medium that stores instructions that, when executed by the processor, perform functions ascribed to the processing device 102 as described herein. Although not required, aspects and embodiments of the technology may be described in the general context of computer-executable instructions, such as routines executed by a general purpose computer, e.g., a server or a personal computer. Those skilled in the relevant art will appreciate that the technology may be practiced in other computer system configurations, including Internet appliances, handheld devices, wearable computers, cellular or mobile phones, multi-processing systems, microprocessor-based or programmable consumer electronics, set-top boxes, network PCs, minicomputers, mainframe computers, and the like. The technology may be embodied in a special purpose computer or data processor specifically programmed, configured, or constructed to execute one or more computer-executable instructions described in detail below. Indeed, the term "computer" (and similar terms), as used generally herein, refers to any data processor or any device capable of network communication, including any of the above devices, as well as consumer electronics such as gaming devices, cameras, or other electronic devices having processors and other components, e.g., network communication circuitry.

[0024] The present technology can also be practiced in a distributed computing environment where tasks or modules are executed by remote processing devices linked through a communications network such as a local area network ("LAN"), a wide area network ("WAN"), or the Internet. In a distributed computing environment, program modules or subroutines can be located in both local and remote memory storage devices. Aspects of the present technology described below can be stored and distributed on computer-readable media, including magnetically and optically readable and removable computer disks, stored in chips (e.g., EEPROM or flash memory chips). Alternatively, aspects of the present technology can be distributed electronically over the Internet or other networks (including wireless networks). Those skilled in the relevant art will recognize that parts of the present technology can reside on a server computer while corresponding parts can reside on a client computer. Data structures and data transfer specific to aspects of the present technology are also included within the scope of the present technology.

[0025] The virtual camera viewpoint is controlled by input controllers 106, which can update the virtual camera viewpoint based on user-initiated changes to the camera position and rotation. An output image corresponding to the virtual camera viewpoint can be output to display device 104. In some embodiments, image processing device 103 can vary the virtual camera viewpoint, depth of field (e.g., aperture), focal plane, and / or another parameter (e.g., based on input from input controllers) to generate different 3D output images without physically moving camera array 110. Display device 104 can receive the output image (e.g., a composited 3D rendering of scene 108) and display the output image for viewing by one or more observers. In some embodiments, processing device 102 receives and processes input from input controllers 106 and processes images captured from camera array 110 to generate an output image corresponding to the virtual viewpoint in substantially real-time or near real-time as perceived by an observer of display device 104 (e.g., at least as fast as the frame rate of camera array 110).

[0026] In addition, the display device 104 can display a graphical representation of any (i) tracked objects (e.g., surgical tools) in the scene 108 and / or (ii) on / in the image of the virtual perspective of the initial image data, registered or unregistered. That is, the system 100 (e.g., via the display device 104) can fuse augmented data into the scene 108 by overlaying and aligning information on top of a "pass-through" image of the scene 108 captured by the camera 112. Furthermore, the system 100 can create a mediated reality experience in which the scene 108 is reconstructed using light field image data of the scene 108 captured by the camera 112, and the tools are virtually represented in the reconstructed scene via information from the tracker 113. Additionally or alternatively, the system 100 can remove the initial scene 108 and replace it entirely with a registered, representative arrangement of the initially captured image data, removing information in the scene 108 that is not relevant to the user's task in that regard.

[0027] The display device 104 may include, for example, a head mounted display device, a monitor, a computer display, and / or another display device. In some embodiments, the input controller 106 and the display device 104 are integrated into a head mounted display device, and the input controller 106 includes a motion sensor that detects the position and orientation of the head mounted display device. In some embodiments, the system 100 may further include a separate tracking system (not shown), such as an optical tracking system, for tracking the display device 104, the instrument 101, and / or other components in the scene 108. Such a tracking system may detect the position of the head mounted display device 104 and may input the position to the input controller 106. A virtual camera viewpoint may then be derived to correspond to the position and orientation of the head mounted display device 104, in the same frame of reference and at a calculated depth (e.g., as calculated by the depth sensor 114) such that the virtual viewpoint corresponds to a viewpoint seen by an observer wearing the head mounted display device 104. Thus, in such embodiments, the head mounted display device 104 can provide a real-time rendering of the scene 108 for viewing by an observer without the head mounted display device 104. Alternatively, the input control device 106 can include a user-controlled control device (e.g., a mouse, a pointing device, a handheld controller, a gesture recognition controller) that allows the observer to manually control the virtual viewpoint displayed by the display device 104.

[0028] 2 is a perspective view of a surgical environment using system 100 for a surgical application, in accordance with an embodiment of the present technology. In the illustrated embodiment, camera array 110 is positioned above scene 108 (e.g., surgical site) and is supported / positioned via a mobile arm 222 operably coupled to a workstation 224. In some embodiments, arm 222 is manually movable to position camera array 110, while in other embodiments, arm 222 is robotically controlled in response to input controller 106 (FIG. 1) and / or another controller. In the illustrated embodiment, workstation 224 is mounted on wheels or casters 225 that allow system 100 to be moved smoothly. In some embodiments, system 100 can be moved smoothly on casters 225 and / or arm 222 can be moved to scan a portion of scene 108, such as a portion of a patient's spine.

[0029] In the illustrated embodiment, the display device 104 is a head-mounted display device (e.g., a virtual reality headset, an augmented reality headset). The workstation 224 may include a computer for controlling various functions of the processing device 102, the display device 104, the input controller 106, the camera array 110, and / or other components of the system 100 shown in FIG. 1. Thus, in some embodiments, the processing device 102 and the input controller 106 are each integrated within the workstation 224. In some embodiments, the workstation 224 includes a second display 226 that may display a user interface for performing various configuration functions, a mirror image of the display on the display device 104, and / or other useful visual images / data. In other embodiments, the system 100 may include more or fewer display devices. For example, in addition to the display device 104 and the second display 226, the system 100 may include another display (e.g., a medical computer monitor) visible to a user wearing the display device 104.

[0030] FIG. 3 is an isometric view of a portion of the system 100 illustrating four cameras 112, according to an embodiment of the present technology. Other components of the system 100 (e.g., other portions of the camera array 110, the processing device 102, etc.) are not shown in FIG. 3 for clarity. In the illustrated embodiment, each camera 112 has a field of view 327 and a focal axis 329. Similarly, the depth sensor 114 can have a field of view 328 aligned with a portion of the scene 108. The cameras 112 together define an imaging volume and can be oriented such that the fields of view 327 are aligned with the portion of the scene 108 and at least partially overlap each other. In some embodiments, some or all of the fields of view 327, 328 at least partially overlap. For example, in the illustrated embodiment, the fields of view 327, 328 converge towards a common measurement volume that includes a portion of the spine 309 of a patient (e.g., a human patient) positioned in / at the scene 108. In some embodiments, the cameras 112 are further oriented such that the focal axes 329 converge to a common point within the scene 108. In some aspects of the present technology, the convergence / alignment of the focal axes 329 can generally maximize disparity measurements between the cameras 112. In some embodiments, the cameras 112 and the depth sensor 114 are fixedly positioned relative to one another (e.g., rigidly fixed to a common frame) such that the positions of the cameras 112 and the depth sensor 114 relative to one another are known and / or can be easily determined via a calibration process. In other embodiments, the system 100 can include a different number of cameras 112 and / or the cameras 112 can be positioned differently relative to one another. In some embodiments, the camera array 110 can be moved (e.g., via the arm 222 in FIG. 2 ) to, for example, scan the spine 309 and move the fields of view 327, 328.

[0031] 1-3 together, in some aspects of the present technology, the system 100 can generate a digitized view of the scene 108 that provides the user (e.g., surgeon) with increased “volumetric intelligence” of the scene 108. For example, the digitized scene 108 can be presented to the user from the user's perspective, orientation, and / or eye perspective such that the user effectively sees the scene 108 as if the user were not looking at the digitized image (e.g., as if the user were not wearing the head mounted display 104). However, the digitized scene 108 allows the user to digitally rotate, zoom, crop, or otherwise enhance the user's view, for example, to facilitate the surgical workflow. Similarly, images of the scene 108 can be registered and overlaid onto initial image data, such as a CT scan, to allow the surgeon to view these data sets together. Such a fused view can allow the surgeon to see aspects of the surgical site that may be obscured in the physical scene 108, such as areas of bone and / or tissue that were not surgically exposed. 4-11, as described in further detail below, system 100 can determine in real-time or near real-time one or more alignment parameters of site 108 (e.g., of spine 309), which can be provided to a user via display device 104 for use in a surgical procedure. Such alignment parameters can further provide a user with "volumetric intelligence" for different volumes within site 108, including regional, local, spinal, and global parameters.

[0032] Selected Embodiments of Methods and Systems for Determining Intra-Operative Alignment Parameters of a Surgical Object, Such as the Spine Spinal deformities include pathological curvatures of the spine and can occur naturally or as a result of disease or injury to the spine. For example, Figures 4A and 4B illustrate several different types of spinal deformities that can occur along the coronal and sagittal planes, respectively, as per an embodiment of the present technology. Spinal deformities can also exist in the axial plane. In some instances, spinal deformities can require surgical correction. Spinal surgery can be done by exposing a portion of the spine and / or by minimally invasive techniques.

[0033] Existing navigation systems for spinal deformity surgery are limited to guiding hardware placement. Thus, surgeons must rely on manual interpretation of either (i) the surgically exposed spine or (ii) intraoperative images of the spine (e.g., x-rays, CT images, fluoroscopic images) to assess the surgical process, such as correction to the spinal deformity. The surgeon's subjective assessment of the exposed physical anatomy and the field of surgery is challenging, potentially inaccurate, and non-reproducible, leading to additional "techniques" in surgery. While interpretation of existing 2D or 3D x-ray images can reduce the degree of subjectivity in assessing correction, the image acquisition process disrupts the surgical workflow and exposes both the patient and the surgical team to potentially harmful amounts of ionizing radiation. Also, when spinal alignment is a 3D problem, interpretation of 2D dimensional x-rays can be added to the technique to generate spinal alignment.

[0034] 1-3 together, in some embodiments, system 100 can determine alignment parameters intra-operatively without interfering with the surgical procedure and in substantially real-time without the use of radiation-dependent imaging modalities. System 100 can provide the determined alignment parameters to the surgeon for use in evaluating the surgical procedure and / or can use the determined alignment parameters to automatically update or suggest updates to the surgical plan, thereby improving the accuracy and effectiveness of the surgical procedure.

[0035] More specifically, Figure 5 is a flow diagram of a process or method 530 for intra-operatively determining spinal alignment parameters during a spinal surgical procedure in accordance with an embodiment of the present technology. Although certain features of the method 530 are described in the context of the system 100 shown in Figures 1-3 for illustrative purposes, one of ordinary skill in the art will readily appreciate that the method 530 may be implemented using other suitable systems and / or devices described herein. Furthermore, while primarily referred to determining spinal alignment parameters of a spinal surgical patient, in other embodiments, the method 530 may be implemented to determine alignment parameters for other types of surgical objects (e.g., bone, flesh, ligaments) during other surgical procedures, such as orthopedic joint replacement, cranial-based surgery, orthopedic trauma surgery, ear, nose and throat surgery, etc., as described in further detail below. Similarly, while reference is made herein to initial image data, intra-operative image data, and surgical sites, method 530 can be used to determine alignment parameters for / within other types of sites, For example, method 530 can be used more generally to register any pre-captured image data with corresponding real-time or near real-time image data of the site to determine alignment parameters of interest within the site.

[0036] At block 531, the method 530 may include receiving initial image data of a spinal column (e.g., the spinal column 309 of a human patient) including a number of vertebrae. In some embodiments, the initial image data is pre-operative image data. As described in detail above, the pre-operative image data may be medical scan data representing a 3D volume of a patient, such as, for example, computed tomography (CT) scan data, magnetic resonance imaging (MRI) scan data, ultrasound images, fluoroscopic images, and / or the like. In some embodiments, the initial image data may be captured intraoperatively. For example, the initial image data may include captured 2D or 3D X-ray images, fluoroscopic images, CT images, MRI images, and the like, and combinations thereof, of the patient in an operating room. In some embodiments, the initial image data may include a point cloud, a three-dimensional (3D) mesh, and / or another 3D data set. In some embodiments, the initial image data may include segmented 3D CT scan data of some or all of the spinal column 309 (segmented on a vertebral-by-vertebral basis).

[0037] At block 532, the method 530 includes characterizing a 3D shape of a plurality of vertebrae of the spine 309 in the initial image data. In some embodiments, the characterizing includes identifying one or more geometric features of each vertebra or a subset of vertebrae. For example, FIGS. 6A-6H are isometric views of a vertebra 640 of the spine 309 including various characterizations of the vertebra's 3D shape, in accordance with embodiments of the present technology. FIG. 6A illustrates a plane of symmetry 641 of the vertebra 640. FIG. 6B illustrates a superior endplate 642 and an inferior endplate 643 of the vertebra 640. FIG. 6C illustrates a regional coordinate frame of the vertebra 640 including (i) an origin 644 generally centered relative to the vertebra 640 and (ii) first through third orthogonal axes 644a-c, respectively, intersecting at the origin 644. 6D-6F are side, top, and bottom views, respectively, of a vertebra 640 illustrating first and second straight lines 646a-b, respectively. The straight lines 646a-b can be used to calculate, for example, sagittal curvature (e.g., lumbar lordosis), as described in more detail below. FIGS. 6G and 6H are top and bottom views, respectively, of a vertebra 640 illustrating first and second straight lines 647a-b, respectively. The straight lines 647a-b can be used to calculate, for example, coronal curvature, as described in more detail below. The alignment processing device 109 can determine and store one or more of the geometric features of each vertebra, including those shown in FIGS. 6A-6H, and / or other geometric features, such as lines, planes, points, coordinates, and the like, that can be useful in determining the degree of spinal curvature. The geometric features each have an initial pose (e.g., position and / or orientation) relative to one another in the initial image data.

[0038] At block 533, the method 530 may include receiving intra-operative image data of the surgical scene 108 including the spine 309 from the camera array 110. The intra-operative image data may include real-time or near real-time images of the spine 309 at the scene 108 captured by the camera 112 and / or the depth camera 118. In some embodiments, the intra-operative image data includes (i) a light field image from the camera 112 and (ii) an image from the depth camera 118 that includes depth information encoded about the scene 108. Some spinal deformities may be large enough within the scene 108 that they are not completely visible to the camera 112. Thus, in some embodiments, receiving the intra-operative image data may include receiving intra-operative image data from the camera array 110 from a different perspective relative to the scene 108 that captures the entire spinal deformity. For example, the camera array 110 may be moved (e.g., scanned) relative to the spine 309 to capture image data of the entire spine 309.

[0039] The initial image data and the intra-operative image data are initially in different coordinate systems such that the same features are represented differently in both data sets. Thus, at block 534, the method 530 can include registering the initial image data to the intra-operative image data, for example, to establish a transformation / mapping / transformation between the intra-operative image data and the initial image data such that these data sets can be represented in the same coordinate system. In some embodiments, the registration process (i) matches 3D points in a point cloud or 3D mesh representing the initial image data with 3D points in a point cloud or 3D mesh representing the intra-operative image data. In some embodiments, the system 100 (e.g., the registration processing device 105) generates a 3D point cloud or mesh from the intra-operative image data from the depth camera 118 of the depth sensor 114, and registers the point cloud or mesh to the initial image data by detecting the location of fiducial markers and / or visible features in both data sets. For example, where the initial image data includes CT scan data, rigid bodies of bone surfaces calculated from the CT scan can be registered to corresponding points / surfaces of the point cloud or mesh.

[0040] In some embodiments, registration is based on / initialized by the surgeon or other user identifying corresponding points in both data sets. For example, the surgeon can identify points in the intra-operative image data that correspond to the same points in the initial image data, such as screw entry points identified by the pre-operative plan. In some embodiments, the surgeon can identify the points by touching a tracked instrument to the spine 309. In other embodiments, the system 100 can use other registration processes based on other methods of shape matching and / or registration processes that do not rely on fiducial markers (e.g., markerless registration processes). In some embodiments, the registration / alignment process may include features generally similar or identical to the registration / alignment process disclosed in (i) U.S. patent application Ser. No. 16 / 749,963, filed Jan. 22, 2020, entitled "Aligning Pre-Operative Scan Images to Real-Time Surgical Images for Mediated Reality View of Surgical Site," and / or (ii) U.S. patent application Ser. No. 17 / 140,885, filed Jan. 4, 2021, entitled "Method and System for Registering Pre-Operative Image Data to Intra-Operative Image Data at a Site, Such as a Surgical Site," each of which is incorporated herein by reference in its entirety. In some embodiments, registration may be performed using any feature or surface matching registration method, such as iterative closest point (ICP), coherent point drift (CPD), or algorithms based on probability density estimation, such as Gaussian mixture models (GMM).

[0041] At block 535, the method 530 may include updating the pose (e.g., position and / or orientation) of each of the characterized 3D shapes of the multiple vertebrae relative to one another based on the registration and the captured intra-operative image data. For example, the poses of any or all of the geometric features of the vertebrae (e.g., plane of symmetry 641, superior endplate 642, inferior endplate 643, region coordinate frame, lines 646a-b, lines 647a-b, etc.) may be updated relative to one another based on the registration and the current captured intra-operative image data that reflects the current physical position and alignment of the spine 309. Thus, after updating, the geometric features are aligned with one another such that measurements between the geometric features reflect the physical geometry of the spine 309.

[0042] At block 536, the method 530 may include determining one or more alignment parameters of the spine 309 based on the updated pose of the characterized 3D shape. In some embodiments, the alignment parameters are measurements between the identified geometric features (block 532) of one or more vertebrae. The alignment parameters can be regional measurements of a single bone of the spine 309 (e.g., indicating the angle between the superior and inferior endplates of a single vertebra), local measurements of a portion of the spine 309 (e.g., indicating the curvature of a subset of the vertebrae of the spine), or measurements of the spine (e.g., relating to the entire spine). The alignment parameters can include at least one of the following: Cobb angle (first, second, and / or third), lumbar lordosis measurement, thoracic lordosis measurement, cervical lordosis measurement, sagittal vertical axis, pelvic tilt angle, pelvic angle, sacral tilt angle, pelvic tilt angle, vertebral rotation angle, segmental lordosis / kyphosis angle measurement, anterior disc height, posterior disc height, foraminal height, foraminal area, disc volume, spondylolisthesis grading (e.g., by millimeter measurement and / or grades I, II, III, IV, and V), vertebral height measurements (anterior, posterior, left, and / or right), C7 plumb line (C7PL), central sacral vertical line (CSVL), lateral spondylolisthesis grading, sagittal vertical axis (SVA), T1 tilt angle, T1 pelvic angle, L1 tilt angle, L1 pelvic angle, etc.

[0043] In some embodiments, the alignment parameters can include 3D skeletal correction parameters, such as vertebral centroids, vertebral body lines, vertical lines, apical vertebra / apical disc, and vertebrae, vertebral axis system, trihedral axis system, spinal axis system, global axis system, vertebral planes, best fit planes, planes of maximum curvature, planes of minimum curvature, apical vertebral lateral planes, apical vertebral frontal planes, apical vertebral planes, vertebral lateral deviations, offsets of regions (balance), vertebral length, lateral deviations of curves, slenderness, vertebral lateral deviations, frontal plane offsets (frontal plane balance), sagittal plane offsets (sagittal plane balance), maximum lateral deviations, inclination angles of vertebral transverse planes (vertebral axial rotation), vertebral These include bone frontal plane inclination angle (vertebral lateral rotation), vertebral sagittal (medial) plane inclination angle (vertebral flexion / extension), apical vertebra axial rotation, best fit plane angle, maximum curvature plane angle, minimum curvature plane angle, apex angle, frontal plane offset angle, sagittal plane offset angle, frontal plane angle balance, geometric curvature, curvature angle, Cobb method, Ferguson method, analytical Cobb method, analytical Ferguson method, constrained curvature angle method, local geometric curvature, local curvature direction, regional geometric curvature, curvature angle, local geometric twist, local twist direction, local mechanical twist, regional geometric twist, regional mechanical twist, etc.

[0044] As one example, Figures 7A and 7B are anterior and lateral views, respectively, of a spine 309 including a plurality of vertebrae 740 (individually identified as first through sixth vertebrae 740a-f, respectively) and corresponding superior endplates 742 (individually identified as first through sixth endplates 742a-f, respectively) that have been characterized / identified in the initial image data of the spine 309, in accordance with an embodiment of the present technology. With joint reference to Figures 7A and 7B, the first vertebra 740a may include the sacrum of the spine 309. The superior endplates 742 are geometric features of the vertebrae 740 (block 532) that have been updated with the pose based registration (block 536). With joint reference to Figures 7A and 7B, in some embodiments, angles (e.g., Cobb angles) may be calculated between any pair of endplates 742. For example, angle A shown in FIG. 7A between the first endplate 742a and the sixth endplate 742f along the coronal plane can provide a coronal Cobb angle measurement that is representative of the spinal curvature in the coronal plane. Similarly, angle B shown in FIG. 7B between the first endplate 742a and the sixth endplate 742f along the sagittal plane can provide a lumbar lordosis angle (which may also be referred to as a sagittal Cobb angle) measurement that is representative of the spinal curvature in the sagittal plane. Other angles can be calculated between endplates 742 of adjacent ones of the vertebrae 740 and / or between any pair of endplates 742 (e.g., non-adjacent ones), depending on, for example, the region of interest of the spine 709. That is, the system 100 can calculate angles between any and all endplates 742 for adjacent vertebrae (e.g., between a first vertebra 740a and a second vertebra 740b, between a second vertebra 740b and a third vertebra 740c, etc.), and / or for various pairs of non-adjacent vertebrae (e.g., between a first vertebra 740a and a fourth vertebra 740d, between a second vertebra 740b and a fifth vertebra 740e, etc.) In contrast, traditional alignment angle measurements are generally calculated using information from the beginning and ending vertebrae of the curve using only initial image data, and therefore do not reflect the current alignment of the spine during the surgical procedure.

[0045] As another example, Figures 8A and 8B are respectively anterior and lateral views of spine 309 including inferior endplates 843 (individually identified as second through sixth endplates 843b-f, respectively) corresponding to vertebrae 740 and characterized / identified in the initial image data of spine 309, in accordance with an embodiment of the present technology. Referring together to Figures 8A and 8B, in some embodiments, inferior endplates are not generated in sacrum 740a. In some embodiments, the system 100 can determine the angle (eg, the Cobb angle) between the inferior endplates 843, as discussed in detail above with reference to Figures 7A and 7B.

[0046] In some embodiments, the system 100 can determine different alignment parameters by measuring between different identified geometric parameters associated with the vertebrae and / or by combining different calculated alignment parameters (e.g., to generate 3D skeletal correction parameters). For example, with joint reference to FIGS. 7A-8B, the vertebrae 740 can be separated by intervertebral discs (not shown), and the system 100 can calculate distances, spacing, angles, and / or other measurements between a superior endplate 742 and an inferior endplate 843 to assess disc size, spacing, volume, angle, and / or other measurements. More specifically, for example, the system 100 can measure distances and / or angles between a second superior endplate 742b and a third superior endplate 843c to assess disc size, volume, angle, and / or the like between a second vertebra 740b and a third vertebra 740c.

[0047] 7A-8B illustrate the use of superior endplate 742 and inferior endplate 743 to measure the angle and disc size between vertebrae 740, system 100 may use any of the various geometric features as shown in FIGS. 6A-6H and / or additional geometric features of any vertebrae 740 to measure the spinal alignment parameters listed above (or otherwise known in the art) between any or all of vertebrae 740. For example, system 100 may automatically determine any one or more alignment parameters described in "Radiographic Measurement Manual" 2008 Edition by the Spinal Deformity Research Group, edited by Michael F. O'Brien and published by Medtronic Sofamor Danek on January 1, 2005, which is incorporated herein by reference in its entirety.

[0048] Returning to FIG. 5, at block 537, the method 530 can include displaying and / or storing the alignment parameters. For example, the system 100 can display the alignment parameters in real-time or near real-time on the head mounted display 104 and / or on the second display 226 during the surgical procedure so that the alignment parameters are available to the surgeon and / or another user. In some aspects of the present technology, this can allow the surgeon to know in real-time or near real-time how their surgical actions (e.g., implantation of pedicle screws or other implants) have affected the alignment of the spine 309. In contrast, conventional navigation systems cannot provide such feedback to the surgeon during the surgical procedure to effect the impact of their actions. Similarly, the system 100 can store the alignment parameters as a function of time so that the impact of the surgical procedure can be analyzed after the surgical procedure.

[0049] At block 538, the method 530 can optionally include updating the surgical plan based on the determined alignment parameters. The system 100 can automatically generate and implement updates to the surgical plan and / or suggest updates to the user. For example, if the surgeon expected a certain correction in alignment after operating on a particular vertebra of the spine 309, but the alignment parameters indicate that such correction was not achieved, the system 100 can update the surgical plan to note the discrepancy between the expected correction and the actual correction. For example, the system 100 can use the determined alignment features of the vertebrae and associated relative pose information to suggest specific adjustments to each vertebra such that they are optimal for some specific measurement (e.g., minimum average Cobb angle between adjacent vertebrae). In some embodiments, this is done by optimizing over possible updates to the alignment of the spine 309 with an appropriate cost function (e.g., in a manner similar to inverse kinematics in robotic control). For example, in some embodiments, system 100 can use intraoperatively determined alignment parameters related to the disc space to assist the surgeon in selecting an interbody device size and / or choosing an appropriate amount of interbody injection material (e.g., bone graft).

[0050] At block 539, the method 530 can optionally include calculating a score, formula, or the like based on the alignment parameters. For example, the score can be a Global Alignment and Proportion (GAP) score, which analyzes the sagittal plane based on pelvic incidence-based proportionality parameters to predict mechanical complications in patients undergoing surgery for adult spinal deformity. Further details for calculating such a GAP score are described in "Global Alignment and Proportion (GAP) Score: Development and Validation of a Novel Method of Analyzing Spinopelvic Alignment to Predict Mechanical Complications After Adult Spinal Deformity Surgery" by C. Yilgor et al., published in Journal of Bone and Joint Surgery, Vol. 99, No. 19, pp. 1661-1672, on October 4, 2017, which is incorporated herein by reference in its entirety.

[0051] Further, in some embodiments, the method 530 can return to block 535 after any of blocks 536-539 to again update the pose of the characterized 3D shape of the plurality of vertebrae. In this manner, the system 100 can continuously update the alignment parameters in real-time or near real-time to reflect the current alignment of the spine 309 and track the progress of the alignment of the spine 309 during the spinal surgical procedure. For example, Figures 9A-9E illustrate the output / data capture progression of the system 100 during a spinal surgical procedure on the spine to correct a spinal deformity including updated alignment parameters, according to an embodiment of the present technology.

[0052] 9A illustrates an initial coronal CT slice of the spine, an initial sagittal CT slice of the spine, a coronal Cobb angle determined from the initial coronal CT slice, and a lumbar lordosis angle determined from the initial sagittal CT slice. The initial coronal and sagittal CT slices can be captured pre-operatively or intra-operatively, such as at the beginning of a spinal surgical procedure (e.g., after surgically exposing the spine). The coronal Cobb angle and lumbar lordosis angle can be determined manually by a user using conventional methods or automatically by the system 100 by calculating spinal alignment parameters (block 536) based on the initial pose of the spinal vertebrae in the initial image data. In the illustrated embodiment, the spine has a deformation including a curvature having a coronal Cobb angle of 26.5° and a lumbar lordosis angle of 18.3°.

[0053] 9B illustrates intraoperatively captured image data 950 of the spine, initial image data of the spine superimposed on the intraoperative image data 950 after registration, a coronal Cobb angle determined from the registered initial image data 952, and a lumbar lordosis angle determined from the registered initial image data 952 on the spine during a surgical procedure for the first time. The first time can be after a surgeon operates on a first vertebra of the spine to correct a deformity. The system 100 can update the pose of the vertebrae of the spine relative to one another based on the registration (e.g., based on the physical positioning of the spine captured in the intraoperative image data 950) (block 535). As described in detail above, the intraoperative image data 950 can include real-time or near real-time images of the spine 309 at the scene 108 captured by the cameras 112 of the camera array 110 and / or the depth camera 118. In some embodiments, the intra-operative image data includes (i) light field images from camera 112 and (ii) images from depth camera 118 that contain depth information encoded about scene 108 .

[0054] After registration, the initial image data 952 and the intraoperative image data 950 can be aligned to the same coordinate system and superimposed on each other. Based on the updated pose of the characterized 3D shape of the vertebrae of the spine (block 535), the system 100 can determine an updated coronal Cobb angle (e.g., 25.8°) and an updated lumbar lordosis angle (e.g., 21.1°) for the first time during the surgical procedure along with any one or more different alignment parameters of the spine (block 536). For example, the system 100 can calculate an angle between one or more endplates identified in the initial image data characterizing the 3D shape of the vertebrae (e.g., as described in detail above with reference to FIGS. 7A-8B). In the illustrated embodiment, the determined coronal Cobb angle and lumbar lordosis angle indicate that the surgical procedure has decreased the coronal Cobb angle and increased the lumbar lordosis angle compared to the initial alignment of the spine (FIG. 9A). Similarly, the determined coronal Cobb angle and lumbar lordosis angle can indicate that the surgical procedure has changed (e.g., improved) the angles toward the surgical goal or plan. In some embodiments, the determined coronal Cobb angle and lumbar lordosis angle are displayed to the surgeon to provide real-time or near real-time feedback (e.g., provided at first glance) on the effectiveness of the surgical procedure in correcting the deformity.

[0055] 9C illustrates spine image data 950 captured intraoperatively, initial image data 952 of the spine superimposed on the intraoperative image data 950 after registration (block 534) and updating the pose of the characterized shape of the vertebrae (block 535) at a second time during a surgical procedure on the spine, the coronal Cobb angle determined from the registered initial image data 952, and the lumbar lordosis angle determined from the registered initial image data 952. The second time can be later than the first time, such as after the surgeon has performed surgery on a second vertebra of the spine after the first vertebra to correct a deformity. Based on the updated pose of the characterized 3D shape of the vertebrae of the spine (block 535), the system 100 can determine an updated coronal Cobb angle (e.g., 22.4°) and an updated lumbar lordosis angle (e.g., 25.5°) at a second time during a surgical procedure with any one or more different alignment parameters of the spine (block 536). In the illustrated embodiment, the determined coronal Cobb angle and lumbar lordosis angle indicate that the surgical procedure has decreased the coronal Cobb angle and increased the lumbar lordosis angle compared to the spinal alignment the first time during the surgical procedure (FIG. 9B). Similarly, the determined coronal Cobb angle and lumbar lordosis angle can indicate that the surgical procedure has changed (e.g., improved) the angles toward the surgical goal or plan. In some embodiments, the determined coronal Cobb angle and lumbar lordosis angle are displayed to the surgeon to provide real-time or near real-time (e.g., provided the second time) feedback on the effectiveness of the surgical procedure in correcting the deformity.

[0056] 9D illustrates spine image data 950 captured intraoperatively during a surgical procedure on the spine at a third time, initial image data 952 of the spine superimposed on the intraoperative image data 950 after registration (block 534) and updating the pose of the characterized shape of the vertebrae (block 535), the coronal Cobb angle determined from the registered initial image data 952, and the lumbar lordosis angle determined from the registered initial image data 952. The third time can be later than the second time, such as after the surgeon operates on a third vertebra of the spine after the first and second vertebrae to correct a deformity and / or after placing one or more intervertebral body devices. Based on the updated poses of the characterized 3D shapes of the vertebrae of the spine (block 535), the system 100 can determine an updated coronal Cobb angle (e.g., 17.4°) and an updated lumbar lordosis angle (e.g., 30.7°) at a third time point during the surgical procedure along with any one or more different alignment parameters of the spine (block 536). In the illustrated embodiment, the determined coronal Cobb angle and lumbar lordosis angle indicate that the surgical procedure has decreased the coronal Cobb angle and increased the lumbar lordosis angle relative to the alignment of the spine at a second time point during the surgical procedure (FIG. 9C). Similarly, the determined coronal Cobb angle and lumbar lordosis angle can indicate that the surgical procedure has changed (e.g., improved) the angles toward the surgical goal or plan. In some embodiments, the determined coronal Cobb angle and lumbar lordosis angle are displayed to the surgeon to provide real-time or near real-time (e.g., provided at the third time point) feedback on the effectiveness of the surgical procedure in correcting the deformity.

[0057] 9E illustrates spine image data 950 captured intra-operatively during a surgical procedure on the spine at a fourth time, initial image data 952 of the spine superimposed on the intra-operative image data 950 after registration (block 534) and updating the pose of the characterized shape of the vertebrae (block 535), the coronal Cobb angle determined from the registered initial image data 952, and the lumbar lordosis angle determined from the registered initial image data 952. The fourth time can be later than the third time, such as after the surgeon operates on the fourth vertebra of the spine after the first, second, and third vertebrae to correct a deformity. Based on the updated poses of the characterized 3D shapes of the vertebrae of the spine (block 535), the system 100 can determine an updated coronal Cobb angle (e.g., 6.8°) and an updated lumbar lordosis angle (e.g., 32.4) at the fourth time point during the surgical procedure along with any one or more different alignment parameters of the spine (block 536). In the illustrated embodiment, the determined coronal Cobb angle and lumbar lordosis angle indicate that the surgical procedure has decreased the coronal Cobb angle and increased the lumbar lordosis angle for the alignment of the spine at the third time point during the surgical procedure (FIG. 9D). Similarly, the determined coronal Cobb angle and lumbar lordosis angle can indicate that the surgical procedure has changed (e.g., improved) the angles toward the surgical goal or plan. In some embodiments, the determined coronal Cobb angle and lumbar lordosis angle are displayed to the surgeon to provide real-time or near real-time (e.g., provided at time point 4) feedback on the effectiveness of the surgical procedure in correcting the deformity. In some embodiments, the determined coronal Cobb angle and lumbar lordosis angle at time point 4 illustrated in FIG. 9E can indicate to the surgeon that they have sufficiently straightened the curve (e.g., relative to the surgical plan or goal) such that no further surgical steps are required.

[0058] Thus, in some aspects of the present technology, the system 100 can determine spinal alignment parameters intraoperatively in situ, in real-time or near real-time, throughout the spinal surgical procedure that reflect the true physical alignment of the spine. The surgeon or other user can use the determined spinal alignment parameters to track their progress through the surgical procedure and / or to validate its effectiveness. In addition, in some embodiments, the surgeon can use the determined spinal alignment parameters to guide further steps of the spinal surgical procedure. For example, if the reduction in the coronal Cobb angle is different (e.g., more or less) than expected at the third time point shown in FIG. 9C, the surgeon can change or modify the next surgical step (e.g., implant size, implant location, etc.) to account for and / or try and adjust for the difference. Similarly, the system 100 can automatically analyze the determined alignment parameters to update / modify the surgical plan, as described in detail above with reference to block 538.

[0059] Some spinal surgical procedures may include the introduction of rod and tower devices and / or other surgical devices used to correct deformities to the spine. Referring again to FIGS. 1-5 together, such surgical devices may clutter the intraoperative field 108 and occlude anatomical structures at the field 108, including one or more vertebrae of the spine 309. Some occlusions may reduce the accuracy of registration of the initial image data of the spine 309 to the intraoperative image data of the spine 309. In some embodiments, the system 100 may register the initial image data to the intraoperative image data based on a known spatial relationship between one or more surgical devices at the field 108 and the spine 309.

[0060] For example, Figure 10 is a flow diagram of a process or method 1060 (e.g., block 534 of method 530) for registering initial image data to intra-operative image data based on one or more surgical devices, in accordance with an embodiment of the present technology. Although certain features of method 1060 are described in the context of system 100 shown in Figures 1-3 for illustrative purposes, one of ordinary skill in the art will readily appreciate that method 1060 can be performed using other suitable systems and / or devices described herein.

[0061] At block 1061, the method 1060 may include receiving a 3D model of a surgical device configured to be secured to a vertebra of the spinal column 309. In some embodiments, the surgical device may be a rod and tower device configured to be rigidly attached to the vertebra throughout the spinal surgical procedure. The 3D model may fully characterize / specify the shape and size of the surgical device.

[0062] At block 1062, the method 1060 may include determining a spatial relationship between the surgical device and the vertebrae to which it is secured. For example, the surgical device may be secured to the vertebrae via one or more implants (e.g., screws) having a pose relative to the vertebrae known from pre-operative planning and / or determined intra-operatively via the camera array 110. Based on the known relationship of the one or more implants to the vertebrae and the surgical device, the system 100 may determine (e.g., restore) the spatial relationship (e.g., rigid relationship) of the surgical device relative to the vertebrae to which it is secured.

[0063] At block 1063, the method 1060 may include receiving intra-operative image data of a surgical device. The intra-operative image data may include real-time or near real-time images of the surgical device at the scene 108 captured by the camera 112 and / or the depth camera 118. In some embodiments, the intra-operative image data includes (i) a light field image from the camera 112 and (ii) an image from the depth camera 118 that includes encoded depth information about the scene 108 including the surgical device.

[0064] At block 1064, the method 1060 includes registering the initial image data of the spine 309 to the intra-operative image data based on the spatial relationship. That is, the image data of the surgical device can be used to register (e.g., align in pose and position) the initial image data to the patient's physical spine 309 based on a rigid spatial relationship between the surgical device visible to the camera array 110 and the associated vertebrae that may be occluded from the camera array 110 by the surgical device. Referring to block 534 of the method 530 of FIG. 5, the registration can be performed using any of the registration methods described in detail above. Thus, in some aspects of the present technology, the system 100 can still determine and track alignment parameters even when the patient's spine 309 and / or other physical anatomical structures are occluded by surgical devices such as rods and tower devices. In some embodiments, the method 1060 can be used to register multiple vertebrae, each having a surgical device attached thereto.

[0065] As described above with reference to block 534, some spinal deformations can be large enough that they are completely invisible to the camera 112 in the scene 108. In some aspects of the present technology, the system 100 can move the camera array 110 (e.g., via the arm 222) relative to the spine 309 to capture image data of the full range of deformation. However, some spinal surgical workflows may require the camera array 110 to remain in a fixed position relative to the scene 108 such that some intra-operative image data of the spine 309 is unavailable throughout some or all of the surgical procedure, thereby potentially reducing the accuracy of the registration of the initial image data of the spine 309 to the intra-operative image data of the spine 309. In some embodiments, the system 100 can register the initial image data to the intra-operative image data by tracking (e.g., optically tracking) markers affixed to one or more vertebrae that are invisible to the camera array 110 during the procedure.

[0066] 11 is a flow diagram of a process or method 1170 for registering initial image data to intra-operative image data (e.g., block 534 of method 530) by tracking markers affixed to one or more vertebrae of the spinal column 309, as an embodiment of the present technology. Although certain features of method 1170 are described in the context of system 100 shown in FIGS. 1-3 for illustrative purposes, one of ordinary skill in the art will readily appreciate that method 1170 can be performed using other suitable systems and / or devices described herein.

[0067] In block 1171, the method 1170 can include attaching markers to vertebrae of the spine 309 for which intraoperative image data will not be available during the surgical procedure. In some embodiments, the vertebrae are the top vertebrae (e.g., the C7 vertebra). The markers can be optical markers, such as a collection of marker balls that can be rigidly attached to the vertebrae for which intraoperative image data will not be available. In some embodiments, the markers can include generally similar or identical features to any markers disclosed in U.S. Patent Application No. 16 / 749,963, filed January 22, 2020, entitled "Aligning Pre-Operative Scan Images to Real-Time Surgical Images for Mediated Reality View of Surgical Site," which is incorporated herein by reference in its entirety. In some embodiments, the markers are attached after surgically exposing the spine 309 but prior to performing corrective adjustments to the spine 309. In some embodiments, the markers are affixed to vertebrae that will not be surgically exposed, e.g., not visible to the camera array 110, throughout the surgical procedure. For example, a marker can be fastened, affixed, or otherwise attached to the patient's skin over a vertebra, which can be a vertebra at one extreme of the curve being measured. With joint reference to Figures 7A and 7B, for example, a marker can be attached to the sixth vertebra 740f to track angle A during a surgical procedure when the sixth vertebra 740f is the vertebra for which intraoperative image data cannot be acquired.

[0068] At block 1172, the method 1170 may include registering the initial image data to the vertebrae for which intra-operative image data will be unavailable / cannot be acquired. The initial image data can be registered / aligned to the vertebrae for which intra-operative image data will be unavailable based on the known pose of the markers relative to the vertebrae. In some embodiments, if the vertebrae for which intra-operative image data will be unavailable are not surgically exposed, the initial pose of the vertebrae can be determined using another imaging modality, such as by taking CT or fluoroscopic images of the vertebrae. The system can then perform a 2D or 3D rigid registration between the captured image data of the vertebrae and the rigid positions of the attached markers.

[0069] At block 1173, the method 1170 may include registering the initial image data to the intra-operative image data of one or more visible vertebrae (i.e., for which intra-operative image data is available). The initial image data may be registered to the intra-operative image data of the one or more visible vertebrae (e.g., by locating fiducial markers and / or feature points visible in both data sets) using any of the registration techniques discussed in detail herein.

[0070] At block 1174, the method 1170 may include intra-operatively tracking markers for use in updating the pose of the 3D characterization of the vertebrae for which intra-operative image data is unavailable. The system 100 may track the markers via the tracker 113 or via an auxiliary tracking unit if the markers are not within the field of view of the tracker 113. In some embodiments, the system 100 tracks the markers via optical tracking, radio frequency identification (RFID) tracking, electromagnetic tracking, and / or the like. Thus, by tracking (via the markers) the vertebrae for which intra-operative image data is unavailable, the system 100 may update the pose of the vertebrae, thereby allowing the determination of alignment parameters of the spine 309 based on the vertebrae for which intra-operative image data is unavailable (block 536). In some embodiments, rather than tracking the marker itself, the system 100 can track a rod (e.g., an extension rod) or other reference device that is coupled to the marker and visible to the camera array 110 throughout the surgical procedure (e.g., via images from the camera 112 and / or tracker 113).

[0071] In some embodiments, intra-operative image data is unavailable for one or more vertebrae. For example, referring to FIGS. 7A and 7B jointly, intra-operative image data may be unavailable for the fourth through sixth vertebrae 740d-f. To recover the pose of the vertebrae 740d-f, markers can be attached to each vertebra 740d-f and used to track the vertebrae 740d-f. Alternatively, markers can be attached only to the sixth vertebra 740f, and the system 100 can interpolate the pose of the fourth and fifth vertebrae 740d-e based on the known tracked pose of the sixth vertebra 740f (via tracking markers attached thereto) and the known poses of the first through third vertebrae 740a-c (via intra-operative image data captured therefrom). Thus, in this manner, the system 100 can calculate alignment parameters between any two or more vertebrae 740.

[0072] In some embodiments, the system 100 can be used to non-invasively track and calculate alignment parameters of a patient's spine pre-operatively, post-operatively, and / or diagnostically without surgical exposure of the spine. Figure 12 is a flow diagram of a process or method 1280 for non-invasively determining alignment parameters of a patient's spine, for example, in accordance with an embodiment of the present technology. Although certain features of the method 1280 are described in the context of the system 100 shown in Figures 1-3 for illustrative purposes, one of ordinary skill in the art will readily appreciate that the method 1280 can be performed using other suitable systems and / or devices described herein.

[0073] Method 1280 may include several processing steps that are generally similar or identical to those of method 530, described in detail above with reference to Figure 5. For example, blocks 1281 and 1282 of method 1280 may be generally identical to blocks 531 and 532, respectively, of method 530 and may include receiving initial image data of a patient's spine and characterizing the 3D shape of a plurality of vertebrae in the initial image data.

[0074] At block 1283, method 1280 may include receiving image data of the patient's back from camera array 110. The image data may include real-time or near real-time images of the back captured by camera 112 and / or depth camera 118. In some embodiments, the image data may include (i) a light field image from camera 112 and (ii) an image from depth camera 118 that includes encoded depth information about the patient's back. In some embodiments, receiving the image data may include capturing images of the patient's back from different perspectives, for example, by moving (e.g., scanning) camera array 110 relative to the patient's back.

[0075] At block 1284, the method 1280 can include registering the initial image data to the captured image data of the patient's back, for example to establish a transformation / mapping / transformation between the captured image data and the initial image data such that these data sets can be represented in the same coordinate system. In some embodiments, the registration process includes analyzing the captured image data to identify points / regions of the patient's spine that are visible as bumps in the skin along the patient's back. These points / regions can then be matched to corresponding points / regions in the initial image data.

[0076] In block 1285, the method 1280 can include updating the pose (e.g., position and / or orientation) of each of the characterized 3D shapes of the plurality of vertebrae relative to one another based on the registration and the captured image data of the patient's back. For example, the pose of any or all of the geometric features of the vertebrae (e.g., symmetry plane 641, superior endplate 642, inferior endplate 643, region coordinate frame, lines 646a-b, lines 647a-b, etc., as shown in FIG. 6 ) can be updated relative to one another based on the registration and the current captured image data that reflects the current physical position and alignment of the patient's spine. Thus, after updating, the geometric features are aligned with one another such that measurements between the geometric features reflect the physical geometry of the patient's spine, even when the spine is not exposed.

[0077] Blocks 1286-1288 of method 1280 can be generally identical to blocks 536, 537, and 539, respectively, of method 530. In some aspects of the present technology, by registering the patient's spine to initial image data when the spine is not surgically exposed, method 1280 can determine alignment parameters pre-operatively, post-operatively, and / or diagnostically. For example, method 1280 can detect how spinal alignment changes as the patient moves (e.g., at a desk or while standing). Furthermore, method 1280 can determine alignment parameters without exposing the patient to additional radiation, as required by many other imaging modalities (e.g., CT, MRI, X-ray).

[0078] Although reference is primarily made herein to determining alignment parameters of a patient's spine, in other embodiments, any of the systems or methods described herein (e.g., methods 530, 1060, 1170, and / or 1280) can be implemented to pre-operatively, post-operatively, and / or diagnostically determine alignment parameters for other types of patient surgical / anatomical targets. More specifically, the methods and systems of the present technology can be used to calculate and track alignment between different bone structures, cartilage, artificial implants, ligaments, etc. throughout the associated surgical procedure.

[0079] For example, during total hip arthroplasty (THA), the primary measurements for cup placement include anteversion and inclination. Typically, there is a tolerance range for anteversion and inclination that has been shown to result in less dislocation. The methods and systems of the present technology can track the patient's prosthetic implant and anatomy to calculate anteversion and inclination in real time or near real time. Similarly, parameters for the femoral component (e.g., stem) can be used to calculate alignment parameters, which in combination with other alignment parameters can be used to calculate limb length and / or other measurements.

[0080] For example, the methods and systems of the present technology can be used to track portions of bone (e.g., bone fragments) and / or cartilage during hip osteotomy or traumatic surgical repair of the hip joint, thereby calculating alignment parameters (e.g., biomechanical indices) such as lateral medial edge angle, anterior medial edge angle, toenis angle, joint contact pressure, etc. In particular, the system 100 can intraoperatively register bone fragments to a pre-acquired hip joint model to calculate alignment parameters. In some aspects of the present technology, such tracking and calculation of alignment parameters can be done without attaching optical tracking markers (e.g., dynamic reference frame markers) or the like to the bone fragments for tracking, which can be difficult and cumbersome to accomplish.

[0081] For example, the methods and systems of the present technology can be used to track exposed bone portions during a craniofacial procedure, such as a Le Fort osteotomy, to calculate alignment parameters related to functional goals (e.g., the patient's ability to chew) and / or aesthetics.

[0082] For example, the methods and systems of the present technology can be used to calculate alignment parameters during knee replacement or other joint replacement surgery. Knee replacement surgery involves positioning the femur and tibia along the mechanical axis of the leg (e.g., from the hip joint down through the middle of the knee to the ankle) and precisely placing multiple implants on the knee in the correct orientation to generate the proper mechanical axis. The methods and systems of the present technology can track portions of bones, such as the femur, tibia, etc., to calculate the mechanical axis, femoral and tibial alignment, etc.

[0083] For example, the methods and systems of the present technology can be used to calculate alignment parameters during an angle osteotomy (e.g., bone resection) that affects the alignment of the bone. For example, a wedge osteotomy allows the surgeon to change the alignment of a bone from one position to another, and current technology can track the changes in real time or near real time as well as aid in planning the actual osteotomy wedge needed to make the change.

[0084] (More examples) The following examples are illustrative of some embodiments of the present technology. (1) A method for intra-operatively determining alignment parameters of a spinal column during a surgical procedure of the spine, comprising: receiving initial image data of the spine including a plurality of vertebrae; identifying geometric features associated with each vertebra in the initial image data, each having a pose in the initial image data; receiving intraoperative image data of the spine; registering the initial image data with the intraoperative image data; updating a pose of each of the geometric features based on the registration and the intra-operative image data; determining the alignment parameters of the spine based on the updated poses of the geometric features associated with two or more of the vertebrae; A method comprising: (2) The method of claim 1, wherein the alignment parameter is an angle indicating the curvature of the spine. (3) The method described in example 1 or example 2, wherein the alignment parameter is a Cobb angle indicating the curvature of the spine. (4) The method of any one of Examples 1 or 2, wherein the alignment parameters are at least one of Cobb angle, lumbar lordosis measurement, thoracic lordosis measurement, cervical lordosis measurement, sagittal vertical axis, pelvic tilt angle, pelvic angle, sacral tilt angle, pelvic tilt angle, vertebral rotation angle, segmental lordosis / kyphosis angle measurement, anterior disc height, posterior disc height, foraminal height, foraminal area, disc volume, spondylolisthesis grading, vertebral height measurement, C7 plumb line, central sacral vertical line, lateral spondylolisthesis grading, sagittal vertical axis, T1 tilt angle, T1 pelvic angle, L1 tilt angle, and L1 pelvic angle. (5) The method according to any one of Examples 1 to 4, wherein the alignment parameters are three-dimensional (3D) skeletal correction parameters. (6) The method of any one of Examples 1 to 5, wherein the geometric feature associated with each vertebra is an endplate. (7) The method described in any one of Examples 1 to 5, wherein the geometric features associated with each vertebra are at least one of a plane of symmetry, an upper end plate, a lower end plate, a vertebra-specific coordinate plane, and a straight line. (8) The method according to any one of Examples 1 to 7, wherein the initial image data is pre-operative three-dimensional (3D) image data. (9) The method of any one of Examples 1 to 8, wherein determining the alignment parameters of the spine includes continuously determining the alignment parameters substantially in real time. (10) A method as described in any one of Examples 1 to 9, wherein the method further includes at least one of displaying the alignment parameters on a display and updating a surgical plan based on the determined alignment parameters. (11) The method according to any one of Examples 1 to 10, further comprising calculating a score based on the determined alignment parameters. (12) The imaging system is a camera array including a plurality of cameras configured to capture intra-operative image data of a spine of a patient undergoing a spinal surgical procedure; a processing device communicatively coupled to the camera array; The processing device comprises: receiving initial image data of the spine including a plurality of vertebrae; identifying geometric features associated with each vertebra in the initial image data, each having a pose in the initial image data; receiving the intraoperative image data of the spine from the camera array; registering the initial image data to the intraoperative image data; updating a pose of each of the geometric features based on the registration and the intra-operative image data; determining alignment parameters of the spine based on the updated poses of the geometric features associated with two or more of the vertebrae. Imaging system. (13) The imaging system of embodiment 12, wherein the alignment parameter characterizes a curvature of the spine. (14) The imaging system of any one of Examples 12 and 13, wherein the geometric feature associated with each vertebra is an endplate and the alignment parameter is a Cobb angle. (15) The imaging system of any one of Examples 12 to 14, wherein the processing device is configured to continuously determine the alignment parameters substantially in real time. (16) The imaging system of any one of Examples 12 to 15, further comprising a display device, wherein the processing device is configured to display the intraoperative image data and the alignment parameters on the display device. (17) A method for determining alignment parameters of an anatomical target in a patient, comprising: The method comprises: receiving initial three-dimensional (3D) image data of the anatomical target including a plurality of portions; identifying geometric features associated with each portion of the anatomical target in the initial image data, the geometric features characterizing a 3D shape of the associated portion of the anatomical target, each having a pose relative to one another; receiving light field image data of the anatomical target; registering the 3D preliminary image data to the light field image data; updating a pose of each of the geometric features based on the registration and the light field image data; determining the alignment parameters of the anatomical target based on the updated poses of the geometric features associated with two or more of the portions of the anatomical target; The method includes: (18) The method of example 17, wherein the anatomical target is the spinal column. (19) The method of example 17 or example 18, wherein the anatomical target is a bone structure. (20) The method of any one of Examples 17 to 19, wherein determining the alignment parameters of the anatomical target includes continuously determining the alignment parameters substantially in real time. (21) A method for intra-operatively determining alignment parameters of a spinal column during a surgical procedure on the spine, comprising: receiving pre-operative image data of the spinal column including a plurality of vertebrae; identifying geometric features associated with each vertebra in the pre-operative image data, each having a pose in the pre-operative image data; receiving intraoperative image data of the spine; registering the pre-operative image data with the intra-operative image data; updating the pose of each geometric feature based on the registration and the intra-operative image data; and determining the alignment parameters of the spine based on the updated poses of the geometric features associated with two or more of the vertebrae; The method includes: (22) The method of embodiment 21, wherein the alignment parameter is an angle indicating the curvature of the spine. (23) The method described in example 21 or example 22, wherein the alignment parameter is a Cobb angle indicating the curvature of the spine. (24) The method of embodiment 21 or embodiment 22, wherein the alignment parameter is at least one of Cobb angle, lumbar lordosis measurement, thoracic lordosis measurement, cervical lordosis measurement, sagittal vertical axis, pelvic tilt angle, pelvic angle, sacral tilt angle, pelvic tilt angle, vertebral rotation angle, segmental lordosis / kyphosis angle measurement, anterior disc height, posterior disc height, foraminal height, foraminal area, disc volume, spondylolisthesis grading, vertebral height measurement, C7 plumb line, central sacral vertical line, lateral spondylolisthesis grading, sagittal vertical axis, T1 tilt angle, T1 pelvic angle, L1 tilt angle, and L1 pelvic angle. (25) The method according to any one of Examples 21 to 24, wherein the alignment parameters are three-dimensional (3D) skeletal correction parameters. (26) The method of any one of Examples 21 to 25, wherein the geometric feature associated with each vertebra is an endplate. (27) The method described in any one of Examples 21 to 25, wherein the geometric features associated with each vertebra are at least one of a plane of symmetry, an upper endplate, an inferior endplate, a vertebra-specific coordinate plane, and a straight line. (28) The method according to any one of Examples 21 to 27, wherein the preoperative image data is three-dimensional (3D) computed tomography (CT) data. (29) The method of any one of Examples 21 to 28, wherein determining the alignment parameters of the spine includes continuously determining the alignment parameters substantially in real time. (30) The method of any one of Examples 21 to 29, wherein the method further comprises displaying the alignment parameters on a display. (31) The method according to any one of Examples 21 to 30, wherein the method further comprises updating a surgical plan based on the determined alignment parameters. (32) The imaging system includes: a camera array including a plurality of cameras configured to capture intra-operative image data of a spine of a patient undergoing a spinal surgical procedure; a processing device communicatively coupled to the camera array; The processing device comprises: receiving pre-operative image data of the spinal column including a plurality of vertebrae; Identifying geometric features associated with each vertebra in the pre-operative image data, each having a pose in the pre-operative image data; receiving the intraoperative image data of the spine from the camera array; registering the preoperative image data to the intraoperative image data; updating a pose of each of the geometric features based on the registration and the intra-operative image data; determining alignment parameters of the spine based on the updated poses of the geometric features associated with two or more of the vertebrae. Imaging system. (33) The imaging system of Example 32, wherein the alignment parameter characterizes a curvature of the spine. (34) The imaging system of embodiment 32 or embodiment 33, wherein the geometric feature associated with each vertebra is an endplate and the alignment parameter is a Cobb angle. (35) The imaging system of any one of Examples 32 to 34, wherein the processing device is configured to continuously determine the alignment parameters substantially in real time. (36) The imaging system of any one of Examples 32 to 34, further comprising a display device, wherein the processing device is configured to display the intraoperative image data and the alignment parameters on the display device. (37) A method for intra-operatively determining alignment parameters of a spinal column during a surgical procedure, comprising: Receiving three-dimensional (3D) pre-operative image data of a spinal column including a plurality of vertebrae; identifying geometric features associated with each vertebra in the pre-operative image data that characterize a 3D shape of the associated vertebra and have a pose for each; receiving intraoperative light field image data of the spine; registering the 3D pre-operative image data to the intra-operative light field image data; updating a pose of each of the geometric features based on the registration and the intra-operative light field image data; determining the alignment parameters of the spinal column based on the updated poses of the geometric features associated with two or more of the vertebrae; The method includes: (38) The method of example 37, wherein the alignment parameter characterizes the curvature of the spine. (39) The method of embodiment 37 or embodiment 38, wherein the alignment parameters are three-dimensional skeletal correction parameters. (40) The method of any one of Examples 37 to 39, wherein determining the alignment parameters of the spine includes continuously determining the alignment parameters substantially in real time. (41) A method for determining alignment parameters of an anatomical target, comprising: receiving initial image data of the anatomical target including a plurality of portions; identifying geometric features associated with each portion of the anatomical target in the initial image data, each having a pose in the initial image data; receiving current image data of the anatomical target; registering the initial image data with the current image data; updating a pose of each of the geometric features based on the registration and the current image data; and determining the alignment parameters of the anatomical features based on the updated poses of the geometric features associated with two or more of the portions of the anatomical target; The method includes: (42) The method described in Example 41, wherein the alignment parameter is an angle indicating the curvature of the anatomical target. (43) The method described in any one of Examples 41 or 42, wherein the alignment parameters are three-dimensional (3D) parameters. (44) The method of any one of Examples 41 to 43, wherein the geometric feature associated with each portion of the anatomical target is an endplate. (45) The method described in any one of Examples 41 to 43, wherein the geometric features associated with each portion of the anatomical target are at least one of a plane of symmetry, an upper end plate, an inferior end plate, a vertebral body-specific coordinate plane, and a straight line. (46) The method according to any one of Examples 41 to 45, wherein the initial image data is preoperative three-dimensional (3D) image data. (47) The method of any one of Examples 41 to 46, wherein determining the alignment parameters of the anatomical target includes continuously determining the alignment parameters substantially in real time. (48) The method described in any one of Examples 41 to 47, wherein the method further includes at least one of displaying the alignment parameters on a display and updating a surgical plan based on the determined alignment parameters. (49) The method according to any one of Examples 41 to 48, wherein the method further comprises calculating a score based on the determined alignment parameters. (50) The imaging system is a camera array including a plurality of cameras configured to capture current image data of an anatomical target on a patient; a processing device communicatively coupled to the camera array; The processing device comprises: receiving initial image data of the anatomical target including a plurality of portions; identifying geometric features associated with each portion of the anatomical target in the initial image data, each having a pose in the initial image data; receiving current image data of the anatomical target from the camera array; registering the initial image data with the current image data; updating a pose of each of the geometric features based on the registration and the current image data; determining alignment parameters of the anatomical target based on the updated poses of the geometric features associated with two or more of the portions of the anatomical target. Imaging system. (51) The imaging system of Example 50, wherein the alignment parameter characterizes the curvature of the anatomical target. (52) An imaging system as described in Example 50 or Example 51, wherein the geometric feature associated with each portion of the anatomical target is an endplate and the alignment parameter is an angle. (53) The imaging system of any one of Examples 50 to 52, wherein the processing device is configured to continuously determine the alignment parameters substantially in real time. (54) The imaging system of any one of Examples 50 to 53, further comprising a display device, wherein the processing device is configured to display the current image data and the alignment parameters on the display device.

[0085] The above detailed description of the embodiments of the present technology is not intended to be exhaustive or to limit the present technology to the precise form disclosed above. Although specific embodiments and examples of the present technology are described above for illustrative purposes, various equivalent modifications are possible within the scope of the present technology as recognized by those skilled in the relevant art. For example, steps are provided in a given order, alternatively, an embodiment may perform steps in a different order. Various embodiments described herein may also be combined to provide further embodiments.

[0086] From the foregoing, it will be understood that, although specific embodiments of the present technology have been described herein for illustrative purposes, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the present technology. Where the context permits, singular or plural terms may also include the plural or singular terms respectively.

[0087] Further, unless expressly limited to the use of the word "or" in relation to a list of two or more items to mean only a single item exclusively from the other items, the use of "or" in such a list is to be interpreted as including (a) any single item in the list, (b) all items in the list, or (c) any combination of items in the list. In addition, the term "comprising" is used throughout to mean the inclusion of at least the recited feature(s), without excluding any more similar features, and / or additional types of other features. Specific embodiments have also been described herein for illustrative purposes, but it will be understood that various modifications can be made without departing from the technology. Moreover, while advantages associated with some embodiments of the technology have been described in the context of these embodiments, other embodiments may also exhibit such advantages, and not all embodiments necessarily exhibit such advantages to fall within the scope of the technology. Thus, the present disclosure and associated technology may encompass other embodiments not expressly shown or described herein.

Claims

1. A method for determining the alignment parameters of the spine during a surgical procedure on the spine, Receiving initial image data of the spinal column including multiple vertebrae, Each of the geometric features associated with each vertebra in the initial image data has a pose in the initial image data, and, In virtually real-time, Receiving data of the spine during surgery on the spine, wherein the data received during surgery includes depth data of the spine from a depth sensor and image data of the spine from multiple cameras. The initial image data is registered in the data from the surgery, The pose of each geometric feature is updated based on the registration and the data obtained during the surgery. Determining the alignment parameters of the spine based on the updated pose of the geometric features associated with two or more vertebrae, Methods that include...

2. The method according to claim 1, wherein the alignment parameter is an angle indicating the curvature of the spine.

3. The method according to claim 1, wherein the alignment parameter is the Cobb angle representing the curvature of the spine.

4. The method according to claim 1, wherein the alignment parameter is at least one of the following: Cobb angle, lumbar lordosis measurement, thoracic lordosis measurement, cervical lordosis measurement, sagittal plane vertical axis, pelvic tilt angle, pelvic angle, sacral tilt angle, pelvic tilt angle, vertebral rotation angle, segmental lordosis / kyphosis measurement, anterior disc height, posterior disc height, foramen height, foramen area, disc volume, spondylolisthesis grading, vertebral height measurement, C7 plum line, central sacral vertical line, lateral spondylolisthesis grading, sagittal plane vertical axis, T1 tilt, T1 pelvic angle, L1 tilt, L1 pelvic angle.

5. The method according to claim 1, wherein the alignment parameter is a three-dimensional (3D) scoliosis parameter.

6. The method according to claim 1, wherein the geometric feature associated with each vertebra is the endplate.

7. The method according to claim 1, wherein the geometric feature associated with each vertebra is at least one of a plane of symmetry, an upper endplate, an lower endplate, a coordinate plane specific to the vertebra, and a straight line.

8. The method according to claim 1, wherein the initial image data is three-dimensional (3D) image data.

9. The method according to claim 1, wherein the initial image data is computed tomography (CT) image data.

10. The method according to claim 1, wherein the registration of the initial image data into the data during the surgery is at least partially based on depth data.

11. The method according to claim 1, further comprising displaying the alignment parameters on a display and updating the surgical plan based on the determined alignment parameters.

12. The method according to claim 1, wherein the camera comprises an RGB camera.

13. The method according to claim 1, wherein the cameras are firmly fixed to a common frame, each camera has a focal axis, and the focal axes of the cameras converge.

14. The method according to claim 1, wherein the image data is light field image data.

15. The image system is A sensor array including a plurality of cameras and a depth sensor, configured to capture intraoperative data of the spine of a patient undergoing spinal surgery, including image data of the spine from the cameras and depth data of the spine from the depth sensors, Includes a processing device that is communicably coupled to the camera array, The processing device is Initial image data of the spinal column, including multiple vertebrae, is received. In the initial image data, each geometric feature associated with each vertebra in the initial image data is identified, and In virtually real-time and continuously, The sensor array receives data from the spinal column during the surgery. The initial image data is registered in the data from the surgery, Based on the registration and the data obtained during the surgery, the pose of each of the geometric features is updated. The system is configured to determine the alignment parameters of the spine based on the updated pose of the geometric features associated with two or more vertebrae. Image system.

16. The imaging system according to claim 15, wherein the alignment parameter characterizes the curvature of the spine.

17. The imaging system according to claim 15, wherein the geometric feature associated with each vertebra is the endplate, and the alignment parameter is the Cobb angle.

18. The image system according to claim 15, wherein the image system further includes a display device, and the processing device is further configured to display the image data and the alignment parameters on the display device.

19. The image system according to claim 15, wherein the image data is light field image data.

20. The image system according to claim 15, wherein the camera is firmly fixed to a common frame, the depth sensor is firmly fixed to a common frame, each camera has a focal axis, and the focal axes of the cameras converge.

21. A method for determining the alignment parameters of anatomical structures during a medical procedure during surgery, The aforementioned method, Receiving initial image data of the anatomical structure, which includes multiple anatomical parts, To identify geometric features associated with each anatomical part in the initial image data, which have poses relative to each other in the initial image data, and, In virtually real-time and continuously, Receiving data of the anatomical structure during surgery, wherein the data includes depth data of the anatomical structure from a depth sensor and image data of the anatomical structure from multiple cameras. The initial image data is registered in the data from the surgery, The pose of each geometric feature is updated based on the registration and the data obtained during the surgery. Determining the alignment parameters of the anatomical structure based on the updated pose of the geometric features associated with two or more of the anatomical parts, A method that includes this.

22. The method according to claim 21, wherein the alignment parameter is an angle representing the curvature of the anatomical structure.

23. The method according to claim 21, wherein the anatomical structure includes a bone structure.

24. The method according to claim 21, wherein at least one of the anatomical parts includes bone.

25. The method according to claim 21, wherein at least one of the anatomical parts includes a ligament.

26. The method according to claim 21, wherein the medical procedure is artificial joint replacement surgery.

27. ​​The method according to claim 21, wherein the anatomical structure includes the spine.

28. The method according to claim 21, wherein the alignment parameter is a three-dimensional (3D) parameter.

29. The method according to claim 21, wherein the geometric features associated with each anatomical part include a plane.

30. The method according to claim 21, wherein the initial image data is three-dimensional (3D) image data.

31. The method according to claim 21, wherein the initial image data is computed tomography (CT) image data.

32. The method according to claim 21, wherein registering the initial image data into the data during the surgery is at least partially based on depth data.

33. The method according to claim 21, further comprising displaying the alignment parameters on a display and updating the surgical plan based on the determined alignment parameters.

34. The method according to claim 21, wherein the camera includes an RGB camera.

35. The cameras are firmly fixed to a common frame, each camera has a focal axis, and the focal axes of the cameras converge. The method according to claim 21.

36. The image system is A sensor array comprising multiple cameras and depth sensors configured to capture intraoperative data of a patient's anatomical structures during a medical procedure, wherein the intraoperative data includes image data of the anatomical structures from the cameras and depth data of the anatomical structures from the depth sensors. A processing device that is communicatively coupled to a camera array, wherein the processing device is Initial image data of the anatomical structure, which includes multiple anatomical parts, is received. In the initial image data, identify the geometric features associated with each anatomical part, each of which has a pose in the initial image data. In virtually real-time and continuously, The sensor array receives data of the anatomical structure during the surgery. The initial image data is registered in the data from the surgery, Based on the registration and the data obtained during the surgery, the pose of each geometric feature is updated. An imaging system comprising a processing device configured to determine alignment parameters of an anatomical structure based on the updated pose of the geometric features associated with the two or more anatomical parts.

37. The image system according to claim 36, wherein the image system further includes a display device, and the processing device is further configured to display the image data and the alignment parameters on the display device.

38. The imaging system according to claim 36, wherein the sensor array further includes a rigid frame, and the camera and the depth sensor are fixedly connected to the rigid frame so that they have known poses and distances from one another.

39. The image system according to claim 38, wherein the camera includes an RGB camera.

40. The imaging system according to claim 38, wherein at least one of the anatomical parts includes bone or ligament.

Images