Segmented body image data interpolation

Abstract

In some implementations, a device may obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element. The device may obtain, during a procedure, tracking information indicative of pose of the anatomical element during the procedure. The device may generate, based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information. The device may interpolate, based on a navigational viewpoint, the second image data to generate third image data that includes the contextual data. The device may provide, during the procedure, the third image data for display.

Description

Docket No. A0011887W001SEGMENTED BODY IMAGE DATA INTERPOLATION

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 709,245, filed 18 October 2024, the entire content of which is incorporated herein by reference.BACKGROUND

[0002] Medical imaging data includes various imaging techniques that can be used to visualize internal body structures for diagnostic and therapeutic purposes. These images are crucial for assessing health conditions, guiding treatment decisions, and / or monitoring disease progression, among other examples. During a procedure (e.g., surgery), real-time imaging can assist surgeons by providing information associated with the anatomy of a patient, allowing for more precise interventions and improved patient outcomes.SUMMARY

[0003] In some implementations, a device includes one or more memories, and one or more processors, communicatively coupled to the one or more memories, configured to: obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtain second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies of the anatomical element that are registered to a common coordinate system in the second space, and wherein the second image data depicts relative poses of the set of segmented bodies; identify a navigational viewpoint on the anatomical element; interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and provide the third image data for display.

[0004] In some implementations, a method includes obtaining, by a device, first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtaining, by the device and during a procedure, tracking information indicative of pose of the anatomical element during the procedure; generating, by the device and based on the tracking information, second image data depicting the anatomical element, wherein theDocket No. A0011887W001 second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information; identifying, by the device, a navigational viewpoint on the anatomical element; interpolating, by the device and based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and providing, by the device and during the procedure, the third image data for display.

[0005] In some implementations, a non-transitory computer-readable medium storing a set of instructions includes one or more instructions that, when executed by one or more processors of a device, cause the device to: obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtain, during a procedure, tracking information indicative of pose of the anatomical element during the procedure; generate, based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information; identify a navigational viewpoint on the anatomical element; interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and provide, during the procedure, the third image data for display.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figs. 1A-1C are diagrams of an example implementation associated with segmented body image data interpolation.

[0007] Fig. 2 is a diagram of an example associated with segmented body image data interpolation.

[0008] Fig. 3 is a diagram of an example associated with segmented body image data interpolation.

[0009] Fig. 4 is a diagram of an example environment in which systems and / or methods described herein may be implemented.Docket No. A0011887W001

[0010] Fig. 5 is a diagram of example components of a device associated with segmented body image data interpolation.

[0011] Fig. 6 is a flowchart of an example process associated with segmented body image data interpolation.DETAILED DESCRIPTION

[0012] The following detailed description of example aspects refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements.

[0013] Medical imaging data may be used as part of a preoperative assessment of a patient. For example, one or more imaging devices may be used to capture medical imaging data of a patient. The medical imaging data may depict one or more anatomical elements of the patient, such as bones, organs, tissues, nerves, blood vessels, muscles, cartilage, ligaments, and / or lymphatic vessels, among other examples. The medical imaging data may include visual information that aids in diagnosis and surgical planning for the patient. For example, the medical imaging data may include X-ray data, computed tomography (CT) data, magnetic resonance imaging (MRI) data, and / or ultrasound data, among other examples, each offering unique insights into anatomical structures and potential pathologies of the patient. The medical imaging data may enable a clinician to evaluate the condition of one or more anatomical elements of the patient, identify abnormalities, and / or determine the most effective surgical approach. By analyzing the medical imaging data, the clinician can also assess risks and improve patient outcomes through more precise interventions.

[0014] Pre-operative medical imaging data (e.g., medical imaging data captured prior to a procedure or surgery, such as in an exam space) may provide a comprehensive view of anatomical elements pertinent to surgical intervention (e.g., anatomical elements of interest that are a target of a procedure), enabling a clinician to visualize both one or more target structures (e.g., anatomical elements of interest that are a target of a procedure) and contextual data (e.g., the surrounding context of the target structure(s)). For example, medical imaging data obtained via high-resolution MRI scans can delineate specific organs and tissues, revealing pathological conditions, such as herniated or degenerated intervertebral discs, while providing insights into adjacent anatomical features, such as spinal nerves and / or blood vessels, among other examples. As another example, medical imaging data obtained via CT imaging can provide detailed cross-sectional views, allowing for precise assessmentDocket No. A0011887W001 of landmarks and potential deformities in one or more anatomical elements. The medical imaging data may provide insight regarding target structure(s), and may contextualize the target structure(s) within the broader landscape of the anatomy of the patient. This enables a thorough understanding of spatial relationships and potential complications for a procedure associated with the target structure(s). For example, a clinician and / or a device may determine a tailored surgical plan for the target structure(s) based on the medical imaging data, ensuring that the procedure can be executed with increased precision and reduced risk to structures or anatomical elements surrounding the target structure(s).

[0015] During the procedure, one or more imaging devices and / or systems may perform segmental tracking of the target structure(s). “Segmental tracking” refers to the tracking of individual anatomical elements, such as one or more vertebrae of a spine (e.g., in the example of a spinal procedure). Segmental tracking may be used, for example, to enable adjustment of any preoperative plan based on movement (whether planned or unexpected) of the tracked anatomical element(s) during surgery. The terms “object tracking,” “element tracking,” and “anatomical tracking” may be used interchangeably with “segmental tracking.” For example, during the procedure, one or more imaging devices and / or systems may perform segmental tracking to determine a current pose or orientation of the target structure(s) (e.g., during the procedure), which may be different than the pose or orientation of the target structure(s) as depicted in the medical imaging data captured prior to the procedure. This enables the clinician and / or a device (e.g., a surgical robot or navigation system) to identify the pose or orientation of the target structure(s) during the surgery for improved navigation of surgical tools and / or more accurate surgical intervention associated with the target structure(s). In some examples, one or more images may be displayed depicting the target structure(s) in a current orientation or pose based on the performed segmental tracking. However, because of the complexity associated with registering and tracking the target structure(s), only segmented bodies of the target structure(s) may be tracked during the procedure. As a result, the information provided via displayed and / or generated image data during the procedure may depict the target structure(s) in the current pose or orientation, but may not depict or indicate the contextual data that may be indicated or included in the medical imaging data (e.g., that is obtained prior to the start of the procedure). This may result in the displayed and / or generated image data having reduced utility because the image data does not depict the contextual data. In some examples, this may result in increased risk of damaging one or more anatomical elements associated with the contextual data, because the contextual data is not depicted in the displayed and / or generated image data (for example) and a device (e.g., aDocket No. A0011887W001 robot or navigation system) or clinician may be unable to identify spatial relationships between the target structure(s) in the current orientation or pose and the one or anatomical elements associated with the contextual data. However, tracking and generating imaging data showing the contextual data during the procedure may consume significant processing resources and / or computing resources because of the complexity of the anatomy of the patient.

[0016] Some aspects described herein enable segmented body image data interpolation. For example, a device (e.g., a display device) may obtain first image data depicting an anatomical element. The first image data may be a first space (e.g., a first imaging modality), and may depict contextual data associated with the anatomical element. For example, the first image data may be medical image data captured prior to a procedure for a patient (e.g., during a medical exam). The device may obtain second image data depicting the anatomical element (e.g., as part of performing the procedure). The second image data may be associated with a second space (e.g., a second imaging modality). The second image data may depict a set of segmented bodies of the anatomical element that are registered to a common coordinate system in the second space (e.g., based on performing segmental tracking of the set of segmented bodies). The device may identify a navigational viewpoint on the anatomical element. The navigational viewpoint may be a selected or identified viewpoint from which another device (e.g., the navigational system or a robot) or the clinician is viewing (or intends to view) the anatomical element. The device may interpolate, based on the navigational viewpoint, the second image data to generate third image data. The third image data may include the second image data transformed from the second space to the first space. Additionally, the third image data may depict the contextual data. The device may provide the third image data for display.

[0017] For example, the device may perform, based on the navigational viewpoint, a rigid body transformation to transform the second image data from the second space to the first space. For example, the third image data may be in the first space (e.g., in a coordinate system and / or imaging modality) in which the first image data is captured. The device may generate the third image data depicting the set of segmented bodies of the anatomical element (e.g., in a tracked or current orientation or pose) and the contextual data included in, or depicted in, the first image data.

[0018] As a result, the device may display image data (e.g., the third image data) that depicts one or more target structures (e.g., the set of segmented bodies of the anatomical element) in a tracked or current orientation or pose during a procedure and that depicts theDocket No. A0011887W001 contextual data obtained via the medical imaging data that is obtained prior to the procedure (e.g., via more complex or intricate imaging devices, such as an MRI device or a CT device). By the device interpolating or transforming the second image data (e.g., that is obtained based on performing segmental tracking of the anatomical element during the procedure), the device may generate the third image data in the first space. This reduces the complexity associated with generating the third image data that includes the contextual data. For example, this may conserve processing resources and / or computing resources that would have otherwise been associated with registering and / or tracking structures or elements depicted by, or indicated by, the contextual data during the procedure.

[0019] Additionally, by the device generating the third image data to include both the second image data (e.g., the set of segmented bodies of the anatomical element in a tracked or current orientation or pose) and the contextual data, the utility of the image data generated and / or displayed during the procedure may be improved because the image data may depict the current spatial relationships among the set of segmented bodies with the contextual data, enabling the clinician, the device, a navigation system, and / or a robot, among other examples, to make improved determinations for navigation and / or intervention associated with the anatomical element. Further, by the device generating the third image data based on the navigational viewpoint, the third image data may depict the set of segmented bodies and the contextual data from a more useful viewpoint, thereby increasing the utility of the third image data.

[0020] Figs. 1 A-1C are diagrams of an example implementation 100 associated with segmented body image data interpolation. As shown in Figs. 1 A-1C, example implementation 100 includes a display device, an imaging device, an image database, and a navigation system. These devices are described in more detail below in connection with Fig. 4 and Fig. 5.

[0021] Some examples are described herein using a spinal procedure as an example procedure. For example, an anatomical element described herein may be described and / or depicted as the spine and / or a vertebra of the spine as an example. However, the implementations, aspects, processes, and / or techniques described herein may be similarly applied for other anatomical elements and / or other types of procedures. For example, the anatomical element described herein may be any anatomical element that includes one or more bodies (e.g., that are tracked during the procedure, as described in more detail elsewhere herein). For example, the anatomical element may be one or more bones, tissues, nerves, organs, and / or other anatomical elements of a patient.Docket No. A0011887W001

[0022] As shown in Fig. 1A, and by reference number 105, one or more imaging devices may capture examination (exam) images of a patient. For example, the one or more imaging devices may capture or obtain medical imaging data of the patient. The exam images may be included in first image data (referred to herein as exam image data), as described elsewhere herein. For example, the exam image data may include one or more exam images captured during one or more exams of the patient. As an example, the imaging device(s) may be used to capture medical imaging data of the patient prior to a procedure associated with an anatomical element.

[0023] As described elsewhere herein, the exam image data may be pre-operative medical imaging data (e.g., medical imaging data captured prior to a procedure or surgery, such as in an exam space) that provides a comprehensive view of anatomical element(s) pertinent to a surgical intervention or procedure (e.g., anatomical elements of interest that are a target of a procedure). For example, the exam image data may depict one or more bodies (e.g., of one or more anatomical elements) and one or more contextual elements. For example, the exam image data may include anatomical element data (e.g., configured to depict the one or more bodies) and contextual data (e.g., configured to depict the one or more contextual elements).

[0024] The one or more contextual elements may include any anatomical elements or anatomical characteristics in an area around one or more anatomical elements that are the target of a procedure. For example, the one or more contextual elements may indicate one or more vein locations, one or more nerve locations, one or more blood vessel locations, and / or one or more sensitive structure locations, among other examples. The one or more sensitive structures may be structures or elements in the anatomy of the patient that are vulnerable to damage or disruption and / or which may result in functional impairment or negative outcomes for the patient if damaged. For example, sensitive structures may have a delicate structure, a high degree of innervation, and / or critical physiological functions, among other examples. The one or more sensitive structures may include nerves, blood vessels, organs, tissues, and / or sensory receptors, among other examples. As an example, the one or more bodies may include one or more spinal vertebrae, and the one or more contextual elements may include one or more nerves, one or more blood vessels, and / or one or more intervertebral discs, among other examples. As another example, the one or more bodies may include one or more metacarpals or phalanges in the hand of the patient, and the one or more contextual elements may include one or more tendons, one or more ligaments, one or more nerves, and / or one or more blood vessels, among other examples, in the hand of the patient.Docket No. A0011887W001

[0025] The exam image data may be associated with a first space (e.g., an exam space). The first space may be associated with a first imaging modality (e.g., an exam imaging modality). For example, the first space may include one or more coordinate systems to represent the anatomical structures being imaged. The first space may include one or more imaging data parameters that indicate or define how image data is to depict one or more images. For example, the one or more imaging data parameters may include a coordinate system, spatial dimensions (e.g., width, height, and / or depth), and / or color channels, among other examples.

[0026] As shown by reference number 110, the exam image data may be stored via the image database. For example, the one or more imaging devices and / or another device may provide the exam image data to be stored in the image database. The image database may be configured to store medical imaging data for analysis and / or use, such as for pre-operative planning, navigation planning, and / or real-time navigation during the procedure, among other examples.

[0027] As shown in Fig. IB, and by reference number 115, the navigation system may obtain intra-operative image data. The intra-operative image data may be image data captured during a procedure associated with the patient. The intra-operative image data may be referred to herein as second image data. The intra-operative image data may be associated with a second space (e.g., a second imaging modality). The second space may be different than the first space that is associated with the exam image data.

[0028] The intra-operative image data may be received in real-time (e.g., with a delay of 500 milliseconds or less, or a delay of 250 milliseconds or less) or in near real-time (e.g., within one minute or less, thirty seconds or less, ten seconds or less, five seconds or less, or one second or less, after the second image data is generated) during the procedure. The intraoperative image data may be obtained after the exam image data is obtained, and generally corresponds to the same anatomical area (e.g., the same anatomical element(s)) as the exam image data. For example, if the exam image data corresponds to (e.g., depicts) a spine or segment thereof of the patient, then the intra-operative image data may also correspond to (e.g., depict) the spine or segment thereof. As another example, if the exam image data corresponds to a knee or portion thereof of the patient, then the intra-operative image data may also correspond to the knee or a portion thereof. As a result, the intra-operative image data may depict a representation of the same anatomical element(s) as the exam image data.

[0029] The intra-operative image data may be received, directly or indirectly, from an imaging device. The imaging device used to capture the intra-operative image data may beDocket No. A0011887W001 an ultrasound probe including a transducer and a receiver in a common housing, or an ultrasound device comprising a transducer and a receiver that are physically separate. Alternatively, the imaging device may be a radar system that includes, for example, a transmitter (including a transmitting antenna) and receiver (including a receiving antenna). The transmitter and the receiver may be contained within a common housing, or may be provided in physically separate housings.

[0030] The navigation system may obtain the exam image data, such as via the image database. For example, the navigation system (or a device or system in communication with the navigation system) may obtain or retrieve the exam image data from the image database.

[0031] As shown by reference number 120, the navigation system may track one or more bodies of an anatomical element during the procedure. The navigation system may use the exam image data and the intra-operative image data to track the one or more bodies (e.g., rigid bodies or bony structures) during the procedure. For example, the navigational system may perform a segmental tracking operation to track the one or more bodies.

[0032] For example, the navigation system may identify the one or more bodies in the intra-operative image data using the exam image data. For example, the navigation system may correlate a representation of an anatomical element (e.g., a body) in the intra-operative image data with a representation of the anatomical element in the exam image data. The correlating may include segmenting the one or more bodies in the intra-operative image data and registering respective bodies of the one or more bodies. “Registering” may refer to the navigation system identifying (e.g., using the exam image data) a body as depicted in the second space (e.g., to register the body in the coordinate system of the intra-operative image data). Any image registration method or technique may be used to accomplish the correlating and / or registering. The correlating and / or registering may include the navigation system utilizing one or more algorithms, such as an image processing algorithm, a feature recognition algorithm, and / or a segmentation algorithm, among other examples, to identify one or more objects in the exam image data and / or in the intra-operative image data, including the one or more bodies. In some examples, the one or more algorithms may be configured to optimize mutual information in the exam image data and the intra-operative image data (e.g., to identify a “best fit” between the two sets of image data that causes features in each set of image data to overlap). In some examples, a feature recognition algorithm may utilize edge detection techniques to detect edges between adjacent objects, and thus to define the boundaries of one or more objects. In some examples, a segmentation algorithm may utilize the exam image data and / or the intra-operative image data to detectDocket No. A0011887W001 boundaries between one or more bodies represented in the image data. Where the exam image data and / or the intra-operative image data includes topographic and / or tomographic image data, one or more algorithms may be used to analyze the topographic and / or tomographic image data to detect boundaries between, and / or to otherwise segment, one or more bodies represented by or within the image data.

[0033] The result of the correlating and / or registering is that a representation of an anatomical element (e.g., one or more bodies) in the intra-operative image data is matched with a representation of the same anatomical element in the exam image data. A body may be, for example, a vertebra of a spine. Because the vertebra may have moved between a first time when the exam image data was generated and a second time when the intra-operative image data was generated, the correlating and / or registering ensures that the same vertebra is identified in both the exam image data and the intra-operative image data.

[0034] The navigation system may generate, modify, or update image data and / or one or more models to depict the one or bodies in a current orientation or pose based on the intraoperative image data. For example, a three-dimensional model or other image data may be generated (e.g., prior to performing the procedure) using the exam image data. The navigational system (and / or the display device) may update the model to depict the one or more bodies in the position, orientation, and / or pose indicated by the intra-operative image data.

[0035] As shown by reference number 125, the navigation system may provide, and the display device may obtain, the intra-operative image data in the second space (e.g., a common space). For example, as described above, the navigation system may register one or more bodies of an anatomical element to a coordinate system (e.g., a common coordinate system) of the common space (e.g., using the intra-operative image data and the exam image data). For example, the intra-operative image data may depict relative poses of a set of segmented bodies of the anatomical element. The set of segmented bodies may be the one or more bodies that have been segmented and registered by the navigation system to enable the navigation system to track the position, pose, and / or orientation of the one or more bodies during the procedure, as described elsewhere herein.

[0036] For example, the intra-operative image data may be based on tracking data of the anatomical element during the procedure. The tracking data may be data obtained by the navigation system via an imaging device and / or a tracking device during the procedure, as described in more detail elsewhere herein. For example, the navigation system may obtain positional information of the anatomical element indicative of a position of the anatomicalDocket No. A0011887W001 element during the procedure. The navigation system may generate, based on the positional information, the intra-operative image data to depict the registered set of segmented bodies in the position, pose, and / or orientation as indicated by the positional information and / or the tracking data.

[0037] As described elsewhere herein, the contextual data of the exam image data may not be tracked during the procedure. For example, imaging devices capable of capturing the contextual data (e.g., MRI device or a CT device) may not be used during the procedure because of the complexity and / or operational conditions present during the procedure. The exam image data may be used by the navigation system to register the one or more bodies that are depicted by the intra-operative image data in the coordinate system of the intraoperative image data. The contextual data may not be registered to the coordinate system of the intra-operative image data.

[0038] As shown by reference number 130, the display device may obtain the exam image data. For example, the display device may obtain or retrieve, from the image database, the exam image data. In other examples, the display device may obtain the exam image data from another device, such as the navigation system.

[0039] As shown by reference number 135, the display device may identify a navigational viewpoint for generating third image data. The navigational viewpoint may be a viewpoint from which the anatomical element is to be viewed or depicted (e.g., in the third image data). For example, the display device may identify the navigational viewpoint on the anatomical element. In some examples, the navigational viewpoint may be a body from the set of segmented bodies of the anatomical element. In other examples, the navigational viewpoint may be a point or location on the anatomical element.

[0040] In some examples, the display device may obtain a user input indicating a position of the anatomical element. The display device may identify the navigational viewpoint based on the user input. For example, a clinician may interact with the display device to select the navigational viewpoint desired by the clinician. The clinician may interact with a user interface associated with the display device to provide the user input. As an example, the user interface may include a display element that is configured to enable a user to select different navigational viewpoints for the anatomical element. For example, the display element may enable the user to provide a user input (e.g., a click, a touch, a drag, or a slide) to cause the display device to generate and / or display the third image data from one or more navigational viewpoints, as described in more detail elsewhere herein.Docket No. A0011887W001

[0041] As another example, the display device may identify a center position on the anatomical element. The display device may identify the navigational viewpoint based on the center position. For example, the navigational viewpoint may be the center position (e.g., a center or middle) of the anatomical element. In some examples, the display device may identify the navigational viewpoint based on the center position for a default navigational viewpoint. For example, the center position may be the default navigational viewpoint if the display device does not have other information described herein (e.g., a user input, a surgical tool position, or other information) to identify the navigational viewpoint.

[0042] As another example, the display device may obtain, during the procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element. For example, the navigation system may be configured to track the position of the surgical tool during the procedure. The navigation system may generate the positional information for the surgical tool. For example, the navigation system may obtain information (e.g., image data or sensor data) indicative of the position of the surgical tool in a surgical field. The navigation system may map the information (e.g., image data or sensor data) indicative of the position of the surgical tool to the coordinate system of the intraoperative image data to generate the positional information. The navigation system may transmit, and the display device may receive, the positional information. The display device may identify the navigational viewpoint based on the positional information. For example, the navigational viewpoint may be a point (or body) of the anatomical element that is closest to the position of the surgical tool.

[0043] This enables the display device to generate the third image data from a viewpoint based on where the clinician (or a device, such as a robot) is currently interacting with anatomical element using the surgical tool. This enables the display device to generate the third image data from an optimized and / or most relevant viewpoint without user input or user intervention. As a result, the display device can generate (and / or update) the third image data from the navigational viewpoint based on the position of the surgical tool as the clinician performs the procedure, thereby improving the utility of the third image data (e.g., because the third image data depicts the anatomical element from the position at which the surgical tool is currently near, or interacting with, the anatomical element).

[0044] As another example, the display device may obtain, during the procedure, movement information of one or more segmented bodies of the set of segmented bodies. The movement information may be indicative of a movement of the one or more segmented bodies. For example, as described elsewhere herein, the navigation system may beDocket No. A0011887W001 configured to track the position(s) of respective bodies from the set of segmented bodies. The navigation system (or the display device) may determine the movement information based on tracking relative position(s) of a given body over time. In some examples, the navigation system may transmit, and the display device may receive, the movement information. The display device may identify the navigational viewpoint based on the movement information. For example, the movement information may indicate one or more bodies that are associated with a largest movement among the set of segmented bodies. This may be indicative of the one or more bodies being interacted with during the procedure. Therefore, the display device may identify that the navigational viewpoint is to be from, or at, the one or more bodies. This enables the display device to generate the third image data that depicts the anatomical element from the viewpoint of the one or more bodies that are currently being interacted with during the procedure. This improves the relevancy and utility of the third image data.

[0045] In some examples, the display device may identify the navigational viewpoint using one or more of the techniques described herein based on detecting one or more triggers or conditions. For example, the display device may initially identify the navigational viewpoint based on the center location, as described above (e.g., before any navigation or activity occurs as part of the procedure). If the display device detects that a surgical tool is actively being used and / or is being navigated as part of the procedure, then the display device may identify the navigational viewpoint based on positional information of the surgical tool. If the display device detects motion in one or more bodies of the anatomical element (e.g., and that a surgical tool is not being actively used and / or is not being navigated), then the display device may identify the navigational viewpoint based on movement information of the one or more bodies. Additionally, the display device may obtain user input to select (or override) the navigational viewpoint at one or more times during the procedure. For example, a user (e.g., a clinician) can override the identified navigational viewpoint at any time during the procedure.

[0046] As shown in Fig. 1C, the display device may interpolate or transform the intraoperative image data based on the navigational viewpoint. For example, as shown by reference number 140, the display device may determine an interpolant based on the intraoperative image data. For example, the interpolant may be a function or algorithm to estimate intermediate values between known data points or states depicted by the intraoperative image data. As an example, the interpolant may estimate or indicate a path or lineDocket No. A0011887W001 between the set of segmented bodies to represent the relative poses or positions between the set of segmented bodies as indicated by the intra-operative image data.

[0047] For example, the interpolant may be based on a three-dimensional line (e.g., E) that is based on the position(s) of the set of segmented bodies as indicated by the intra-operative image data. The three-dimensional line L may be a curve in a three-dimensional space. The three-dimensional line Z is depicted and described in more detail in connection with Fig. 2. The display device may determine the interpolant based on the three-dimensional line (e.g., Z) such that the interpolant is based on the relative poses of the set of segmented bodies identified by the navigation system as part of tracking the position of the set of segmented bodies during the procedure. The display device may parameterize the three-dimensional line using a point on the line (e.g., and one or more directions). The parameterization of the three-dimensional line enables the display device to smoothly interpolate between different positions along the three-dimensional line (e.g., to enable smooth transitions for motion representations as the identified navigational viewpoint changes over time). The parameter used for the parameterization of the three-dimensional line may be a position along the three- dimensional line.

[0048] For example, as shown by reference number 145, the display device may interpolate, based on the navigational viewpoint, the intra-operative image data to generate third image data (e.g., navigational image data). For example, the interpolation of the intraoperative image data may transform the intra-operative image data from the second space (e.g., associated with the intra-operative image data) to the first space (e.g., associated with the exam image data). For example, the display device may use the interpolant to perform a rigid body transformation to transform the intra-operative image data from the second space to the first space based on the navigational viewpoint.

[0049] For example, the display device may generate, based on the intra-operative image data, pose information representative of the relative poses of the set of segmented bodies. The pose information may include the parameterized three-dimensional line that represents the relative poses of the set of segmented bodies. The display device may determine the interpolant as a function of the pose information for a coordinate transformation from the second space to the first space.

[0050] For example,examTCOm may represent an overall rigid transformation from the common space (e.g., the second space) to the exam space (e.g., the first space).examEr v)Com may represent a rigid transformation from the common space (e.g., the second space) to the exam space (e.g., the first space) for a given body v from the set of segmented bodies. ADocket No. A0011887W001 rigid transformation may include a rotation and / or a translation of image data from the common space (e.g., the second space) to the exam space (e.g., the first space). The display device may determine thatexamTCOm is equal toexamTCOm(s), where .s is the parameter used to parameterize the three-dimensional line. Therefore,examTCOm(s) may be the interpolant that includes a function that varies based on a value of the parameter 5. As the value of the parameter 5 changes, the interpolantexamTCOm(s) may transition between various transformations ofexamT(v)com- By determining the interpolant in this manner, the display device may enable the interpolation of transformations for the anatomical element as the navigational viewpoint moves or changes position, enabling more fluid and realistic simulations or analyses of the anatomical element.

[0051] For example, the display device may evaluate the interpolant (e.g.,examTCom(s)')' at a value of the parameter 5. The value of the parameter 5 may be based on the navigational viewpoint. For example, the value of the parameter 5 may be indicative of a position on the three-dimensional line at, or indicated by, the navigational viewpoint. This enables the display device to obtain a transformation for the intra-operative image data from the common space to the exam space from the navigational viewpoint. For example, the display device may use the result of evaluating the interpolant (e.g.,ex“"lTc„ll}) at the value of the parameter 5 (e.g., referred to herein as an interpolated value) as the overall transformation from the common space to the exam space. The display device may interpolate or transform the intraoperative image data using the result of evaluating the interpolant (e.g.,examTCOm(s ) at the value of the parameter 5 as the overall transformation. The interpolation or transformation of the intra-operative image data may include a spherical linear interpolation, a linear interpolation, and / or other types of interpolation.

[0052] As shown by reference number 150, the display device may generate the navigational image data (e.g., the third image data described elsewhere herein). For example, after interpolating or transforming the intra-operative image data to the exam space (e.g., based on the navigational viewpoint), the display device may generate the navigational image data to include the interpolated intra-operative image data and the contextual data obtain via the exam image data. For example, the navigational image data may include the contextual data (or a portion of the contextual data). For example, by transforming or interpolating the intra-operative image data from the common space to the exam space, the display device may add the contextual data to the transformed or interpolated intra-operative image data in the same space (or coordinate system) in which the contextual data was obtained or captured. This reduces the complexity associated with depicting the contextual data with the intraDocket No. A0011887W001 operative image data because the intra-operative image data is in the same space (e.g., the same coordinate system) as the contextual data. For example, the display device may combine the transformed or interpolated intra-operative image data with the contextual data to generate the navigational image data.

[0053] As shown by reference number 155, the display device may provide, for display, the navigational image data (e.g., the third image data). For example, the display device may display the navigational image data during the procedure. As shown in Fig. 1C, the navigational image data may include one or more bodies of the anatomical element (e.g., shown in Fig. 1C as a body A and a body B) with the relative pose between the one or more bodies (e.g., as indicated by the intra-operative image data) and may include the contextual element(s) indicated by the contextual data. In the example shown in Fig. 1C, the navigational viewpoint may be from, or at, the position of the body A. As a result, the navigational image data may depict the current or tracked position and / or pose(s) of the one or more bodies in context of the contextual data obtained via the exam image data.

[0054] In some examples, the display device may use the navigational image data to generate a model (e.g., a three-dimensional model) of the anatomical element with the one or more contextual elements. For example, the display device may provide the three- dimensional model for display during the procedure to provide enhanced data for the clinician (e.g., showing the relative poses of the bodies of the anatomical element with the one or more contextual elements).

[0055] As indicated above, Figs. 1A-1C are provided as an example. Other examples may differ from what is described with regard to Figs. 1 A-1C.

[0056] Fig. 2 is a diagram of an example 200 associated with segmented body image data interpolation. As shown in Fig. 2, intra-operative image data 205 may depict one or more bodies (e.g., segmented bodies) of an anatomical element in a common space. The common space may be a reference space or coordinate system used to track the position(s), orientation(s), and / or pose(s) of the one or more bodies during a procedure, as described in more detail elsewhere herein.

[0057] As shown in Fig. 2, the display device may generate a three-dimensional (3D) line 210 that represents or indicates the relative poses or orientations between the one or more bodies (e.g., segmented bodies) of the anatomical element. For example, the 3D line 210 may pass through centroids of respective bodies of the one or more bodies. Additionally, the 3D line 210 may pass through centers of respective end faces (or end plates) of the one or more bodies. By generating the 3D line 210, the display device may represent the relativeDocket No. A0011887W001 poses or orientations for evaluating an interpolant (e.g., configured for transforming the intraoperative image data 205 to an exam space, as described elsewhere herein) at different navigational viewpoints. For example, the display device may parameterize the 3D line 210 using a position parameter (e.g., 5). The display device may determine a value of the position parameter based on the identified navigational viewpoint. The display device may evaluate the interpolant based on the value of the position parameter to obtain one or more interpolant values for the transformation of the intra-operative image data 205 to an exam space. This enables the relative poses, positions, and / or orientations of the one or more bodies (e.g., as indicated by the intra-operative image data 205) to be maintained during the transformation.

[0058] As indicated above, Fig. 2 is provided as an example. Other examples may differ from what is described with regard to Fig. 2.

[0059] Fig. 3 is a diagram of an example 300 associated with segmented body image data interpolation. Fig. 3 depicts navigational image data of one or more segmented bodies of an anatomical element from different navigational viewpoints.

[0060] For example, a first navigational image 305 may be associated with the navigational viewpoint being based on the body A as shown in Fig. 3. For example, by setting the position parameter (e.g., 5) to a value that is based on the position of the body A (e.g., along the 3D line 210), the display device may interpolate the intra-operative image data 205 to show the intra-operative image data 205 from the viewpoint of the body A, as shown in the first navigational image 305. In some examples, interpolant values may be fixed for value(s) of the position parameter (e.g., 5) that are along the 3D line 210 and inside (or within) a given body, such as the body A. This results in the same viewpoint for all positions within the given body.

[0061] A second navigational image 310 may be associated with the navigational viewpoint being based on the body B as shown in Fig. 3. For example, by setting the position parameter (e.g., 5) to a value that is based on the position of the body B (e.g., along the 3D line 210), the display device may interpolate the intra-operative image data 205 to show the intra-operative image data 205 from the viewpoint of the body B, as shown in the second navigational image 310.

[0062] A third navigational image 315 may be associated with the navigational viewpoint being between the body A and the body B as shown in Fig. 3. For example, by setting the position parameter (e.g., 5) to a value that is based on the position between the body A and the body B (e.g., along the 3D line 210), the display device may interpolate the intraDocket No. A0011887W001 operative image data 205 to show the intra-operative image data 205 from the viewpoint between the body A and the body B, as shown in the third navigational image 315.

[0063] By the display device identifying the navigational viewpoint as described in more detail elsewhere herein, the display device may generate and / or display navigational image data (e.g., one or more navigational images) from a viewpoint that is more relevant and / or useful for a user (e.g., a clinician), thereby improving the utility of the generated and / or displayed navigational image data.

[0064] As indicated above, Fig. 3 is provided as an example. Other examples may differ from what is described with regard to Fig. 3.

[0065] Fig. 4 is a diagram of an example environment 400 in which systems and / or methods described herein may be implemented. As shown in Fig. 4, environment 400 may include a display device 410, an image database 420, an imaging device 430, a navigation system 440, and a network 450. Devices of environment 400 may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

[0066] The display device 410 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with generating protocols for segmented body image data interpolation, as described elsewhere herein. The display device 410 may include a communication device and / or a computing device. The display device 410 may be any device capable of, or configured to, provide, display, or otherwise output (e.g., via a visual output and / or an audio output) information. For example, the display device 410 may include a wireless communication device, a mobile phone, a user equipment, a laptop computer, a tablet computer, a desktop computer, a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, a head mounted display, or a virtual reality headset), a keyboard, a mouse, a trackball, a monitor, a television, a screen, a touch screen, a speaker, and / or any other device for receiving information from a user and / or for providing information to a user. In some examples, the display device 410 may be included in one or more devices or components of the environment 400. For example, the display device 410 may be a component of the navigation system 440, among other examples. In some examples, multiple devices or components of the environment 400 may include display devices 410. For example, a first device may include a first display device 410 and the navigation system 440 may include a second display device 410.

[0067] The image database 420 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with generating protocols for segmented body image data interpolation, as described elsewhere herein. TheDocket No. A0011887W001 image database 420 may include a communication device and / or a computing device. For example, the image database 420 may include a data structure, a database, a data source, a server, a database server, an application server, a client server, a web server, a host server, a proxy server, a virtual server (e.g., executing on computing hardware), a server in a cloud computing system, a device that includes computing hardware used in a cloud computing environment, or a similar type of device. As an example, the image database 420 may store medical imaging data, as described elsewhere herein.

[0068] In some implementations, the medical imaging data may be generated by, captured by, and / or provided by (e.g., to the image database 420) one or more imaging devices 430. The imaging device 430 may be operable to image anatomical element(s) (e.g., a bone, veins, tissue, and / or other anatomical elements) and / or other aspects of patient anatomy to output medical imaging data (e.g., image data depicting or corresponding to a bone, veins, and / or tissue). “Image data” or “imaging data” as used herein refers to the data generated or captured by an imaging device 430, including in a machine-readable form, a graphical / visual form, and in any other form. In some examples, medical imaging data may include data corresponding to an anatomical feature of a patient, or to a portion thereof. The image data may be or include a preoperative image, an intraoperative image, a postoperative image, or an image taken independently of any surgical procedure. In some examples, a first imaging device 430 may be used to obtain first image data (e.g., first image data in a first space (e.g., an exam space) in a similar manner as described in more detail elsewhere herein) at a first time, and a second imaging device 430 may be used to obtain second image data (e.g., second image data in a second space (e.g., an intra-operative space) in a similar manner as described in more detail elsewhere herein) at a second time after the first time. The imaging device 430 may be capable of taking a 2D image or a 3D image to yield the image data. The imaging device 430 may be or comprise, for example, an ultrasound scanner (which may comprise, for example, a physically separate transducer and receiver, or a single ultrasound transceiver), an O-arm, a C-arm, a G-arm, or any other device utilizing X-ray-based imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), an MRI scanner, an OCT scanner, an endoscope, a microscope, an optical camera, a thermographic camera (e.g., an infrared camera), a radar system (which may include a transmitter, a receiver, a processor, and one or more antennas), or any other imaging device suitable for obtaining images of an anatomical feature of a patient. In some examples, medical imaging data, as described herein, may be considered to be continuous and / or provided as an image data stream (e.g., to the imageDocket No. A0011887W001 database 420 via one or more imaging devices 430), such as if the medical image data represents two or more frames per second.

[0069] The navigation system 440 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with generating protocols for segmented body image data interpolation, as described elsewhere herein. The navigation system 440 may include a communication device and / or a computing device. The navigation system 440 may provide navigation for a surgeon and / or a surgical robot during a surgical procedure. The navigation system 440 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigated trackers, or other objects within the operating room or other room in which some or all of the environment 400 is located. The one or more cameras may be optical cameras, endoscopic cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 440 may include one or more electromagnetic sensors. In some examples, the navigation system 440 may be used to track a position and orientation (e.g., a pose) of an imaging device, a retraction assembly, an end unit, a robot, a robotic arm, and / or one or more surgical tools (e.g., to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the examples described above). The navigation system 440 may include a display for displaying one or more images from an external source or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 440. In some examples, the display may be, or may include, the display device 410. The navigation system 440 may be configured to provide guidance to a surgeon or other user in the environment 400, to a robot, or to any other device or component of the environment 400. The guidance provided by the navigation system 440 may include a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan, among other examples.

[0070] The network 450 may include one or more wired and / or wireless networks. For example, the network 450 may include a wireless wide area network (e.g., a cellular network or a public land mobile network), a local area network (e.g., a wired local area network or a wireless local area network (WLAN), such as a Wi-Fi network), a personal area network (e.g., a Bluetooth network), a near-field communication network, a telephone network, a private network, the Internet, and / or a combination of these or other types of networks. The network 450 enables communication among the devices of environment 400.Docket No. A0011887W001

[0071] The number and arrangement of devices and networks shown in Fig. 4 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown in Fig. 4. Furthermore, two or more devices shown in Fig. 4 may be implemented within a single device, or a single device shown in Fig. 4 may be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) of environment 400 may perform one or more functions described as being performed by another set of devices of environment 400.

[0072] Fig. 5 is a diagram of example components of a device 500 associated with segmented body image data interpolation. The device 500 may correspond to the display device 410, the image database 420, the imaging device 430, and / or the navigation system 440. In some implementations, the display device 410, the image database 420, the imaging device 430, and / or the navigation system 440 may include one or more devices 500 and / or one or more components of the device 500. As shown in Fig. 5, the device 500 may include a bus 510, a processor 520, a memory 530, an input component 540, an output component 550, and / or a communication component 560.

[0073] The bus 510 may include one or more components that enable wired and / or wireless communication among the components of the device 500. The bus 510 may couple together two or more components of Fig. 5, such as via operative coupling, communicative coupling, electronic coupling, and / or electric coupling. For example, the bus 510 may include an electrical connection (e.g., a wire, a trace, and / or a lead) and / or a wireless bus. The processor 520 may include a central processing unit, a graphics processing unit, a microprocessor, a controller, a microcontroller, a digital signal processor, a field-programmable gate array, an application-specific integrated circuit, and / or another type of processing component. The processor 520 may be implemented in hardware, firmware, or a combination of hardware and software. In some implementations, the processor 520 may include one or more processors capable of being programmed to perform one or more operations or processes described elsewhere herein.

[0074] The memory 530 may include volatile and / or nonvolatile memory. For example, the memory 530 may include random access memory (RAM), read only memory (ROM), a hard disk drive, and / or another type of memory (e.g., a flash memory, a magnetic memory, and / or an optical memory). The memory 530 may include internal memory (e.g., RAM, ROM, or a hard disk drive) and / or removable memory (e.g., removable via a universal serial bus connection). The memory 530 may be a non-transitory computer-readable medium. TheDocket No. A0011887W001 memory 530 may store information, one or more instructions, and / or software (e.g., one or more software applications) related to the operation of the device 500. In some implementations, the memory 530 may include one or more memories that are coupled (e.g., communicatively coupled) to one or more processors (e.g., processor 520), such as via the bus 510. Communicative coupling between a processor 520 and a memory 530 may enable the processor 520 to read and / or process information stored in the memory 530 and / or to store information in the memory 530.

[0075] The input component 540 may enable the device 500 to receive input, such as user input and / or sensed input. For example, the input component 540 may include a touch screen, a keyboard, a keypad, a mouse, a button, a microphone, a switch, a sensor, a global positioning system sensor, a global navigation satellite system sensor, an accelerometer, a gyroscope, and / or an actuator. The output component 550 may enable the device 500 to provide output, such as via a display, a speaker, and / or a light-emitting diode. The communication component 560 may enable the device 500 to communicate with other devices via a wired connection and / or a wireless connection. For example, the communication component 560 may include a receiver, a transmitter, a transceiver, a modem, a network interface card, and / or an antenna.

[0076] The device 500 may perform one or more operations or processes described herein. For example, a non-transitory computer-readable medium (e.g., memory 530) may store a set of instructions (e.g., one or more instructions or code) for execution by the processor 520. The processor 520 may execute the set of instructions to perform one or more operations or processes described herein. In some implementations, execution of the set of instructions, by one or more processors 520, causes the one or more processors 520 and / or the device 500 to perform one or more operations or processes described herein. In some implementations, hardwired circuitry may be used instead of or in combination with the instructions to perform one or more operations or processes described herein. Additionally, or alternatively, the processor 520 may be configured to perform one or more operations or processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0077] The number and arrangement of components shown in Fig. 5 are provided as an example. The device 500 may include additional components, fewer components, different components, or differently arranged components than those shown in Fig. 5. Additionally, or alternatively, a set of components (e.g., one or more components) of the device 500 mayDocket No. A0011887W001 perform one or more functions described as being performed by another set of components of the device 500.

[0078] Fig. 6 is a flowchart of an example process 600 associated with segmented body image data interpolation. In some aspects, one or more process blocks of Fig. 6 are performed by a display device (e.g., display device 410). In some aspects, one or more process blocks of Fig. 6 are performed by another device or a group of devices separate from or including the display device, such as the image database 420, the imaging device 430, and / or the navigation system 440, among other examples. Additionally, or alternatively, one or more process blocks of Fig. 6 may be performed by one or more components of device 500, such as processor 520, memory 530, input component 540, output component 550, and / or communication component 560.

[0079] As shown in Fig. 6, process 600 may include obtaining first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element (block 610). For example, the display device may obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element, as described above.

[0080] As further shown in Fig. 6, process 600 may include obtaining, during a procedure, tracking information indicative of pose of the anatomical element during the procedure (block 620). For example, the display device may obtain, during a procedure, tracking information indicative of pose of the anatomical element during the procedure, as described above.

[0081] As further shown in Fig. 6, process 600 may include generating, based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information (block 630). For example, the display device may generate, based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information, as described above.

[0082] As further shown in Fig. 6, process 600 may include identifying a navigational viewpoint on the anatomical element (block 640). For example, the display device may identify a navigational viewpoint on the anatomical element, as described above.Docket No. A0011887W001

[0083] As further shown in Fig. 6, process 600 may include interpolating, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data (block 650). For example, the display device may interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data, as described above.

[0084] As further shown in Fig. 6, process 600 may include providing, during the procedure, the third image data for display (block 660). For example, the display device may provide, during the procedure, the third image data for display, as described above.

[0085] Process 600 may include additional aspects, such as any single aspect or any combination of aspects described below and / or in connection with one or more other processes described elsewhere herein.

[0086] In a first aspect, interpolating the second image data to generate the third image data includes generating, based on the second image data, pose information representative of the relative poses of the set of segmented bodies; determining an interpolant as a function of the pose information for a coordinate transformation from the second space to the first space; determining, using a value and the interpolant, an interpolated value, wherein the value is based on the navigational viewpoint; and generating the third image data using the interpolated value, the second image data, and the first image data.

[0087] In a second aspect, alone or in combination with the first aspect, the navigational viewpoint is based on a point on the anatomical element, and the third image data depicts the anatomical element relative to the point.

[0088] In a third aspect, alone or in combination with one or more of the first and second aspects, identifying the navigational viewpoint includes receiving a user input indicating a position on the anatomical element, and the navigational viewpoint is based on the position.

[0089] In a fourth aspect, alone or in combination with one or more of the first through third aspects, identifying the navigational viewpoint includes identifying a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

[0090] In a fifth aspect, alone or in combination with one or more of the first through fourth aspects, identifying the navigational viewpoint comprises obtaining, during the procedure, positional information of a surgical tool indicative of a position of the surgical tool relative toDocket No. A0011887W001 the anatomical element, and identifying the navigational viewpoint based on the positional information.

[0091] In a sixth aspect, alone or in combination with one or more of the first through fifth aspects, identifying the navigational viewpoint includes obtaining, during the procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body, and identifying the navigational viewpoint based on the movement information.

[0092] Although Fig. 6 shows example blocks of process 600, in some aspects, process 600 includes additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in Fig. 6. Additionally, or alternatively, two or more of the blocks of process 600 may be performed in parallel.

[0093] The techniques of this disclosure may also be described in the following examples:

[0094] Example 1 : A device, comprising: one or more memories; and one or more processors, communicatively coupled to the one or more memories, configured to: obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtain second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies of the anatomical element that are registered to a common coordinate system in the second space, and wherein the second image data depicts relative poses of the set of segmented bodies; identify a navigational viewpoint on the anatomical element; interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and provide the third image data for display.

[0095] Example 2: The device of Example 1, wherein the one or more processors, to interpolate the second image data to generate the third image data, are configured to: perform, based on the navigational viewpoint, a rigid body transformation to transform the second image data from the second space to the first space.

[0096] Example 3: The device of any of Examples 1-2, wherein the one or more processors, to interpolate the second image data to generate the third image data, are configured to: generate, based on the second image data, pose information representative of the relative poses of the set of segmented bodies; determine an interpolant as a function of the pose information for a coordinate transformation from the second space to the first space;Docket No. A0011887W001 determine, using a value and the interpolant, an interpolated value, wherein the value is based on the navigational viewpoint; and generate the third image data using the interpolated value, the second image data, and the first image data.

[0097] Example 4: The device of any of Examples 1-3, wherein the navigational viewpoint is based on a point on the anatomical element, and wherein the third image data depicts the anatomical element relative to the point.

[0098] Example 5: The device of any of Examples 1-4, wherein the one or more processors, to identify the navigational viewpoint, are configured to: receive a user input indicating a position on the anatomical element, and wherein the navigational viewpoint is based on the position.

[0099] Example 6: The device of any of Examples 1-5, wherein the one or more processors, to identify the navigational viewpoint, are configured to: identify a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

[0100] Example 7: The device of any of Examples 1-6, wherein the one or more processors, to identify the navigational viewpoint, are configured to: obtain, during a procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element; and identify the navigational viewpoint based on the positional information.

[0101] Example 8: The device of any of Examples 1-7, wherein the one or more processors, to identify the navigational viewpoint, are configured to: obtain, during a procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body; and identify the navigational viewpoint based on the movement information.

[0102] Example 9: The device of Example 8, wherein the movement information indicates that the at least one segmented body is associated with a largest movement among the set of segmented bodies.

[0103] Example 10: The device of any of Examples 1-9, wherein the second image data is based on tracking data of the anatomical element during a procedure, and wherein the contextual data is not tracked during the procedure.

[0104] Example 11 : The device of Example 10, wherein the one or more processors, to provide the third image data for display, are configured to: provide the third image data for display during the procedure.Docket No. A0011887W001

[0105] Example 12: The device of any of Examples 1-11, wherein the contextual data indicates at least one of: one or more vein locations, one or more nerve locations, or one or more sensitive structure locations.

[0106] Example 13: The device of any of Examples 1-12, wherein the one or more processors, to obtain the second image data, are configured to: obtain, for a procedure, positional information of the anatomical element indicative of a position of the anatomical element during the procedure; and generate, based on the positional information, the second image data.

[0107] Example 14: A method, comprising: obtaining, by a device, first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtaining, by the device and during a procedure, tracking information indicative of pose of the anatomical element during the procedure; generating, by the device and based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information; identifying, by the device, a navigational viewpoint on the anatomical element; interpolating, by the device and based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and providing, by the device and during the procedure, the third image data for display.

[0108] Example 15: The method of Example 14, wherein interpolating the second image data to generate the third image data comprises: generating, based on the second image data, pose information representative of the relative poses of the set of segmented bodies; determining an interpolant as a function of the pose information for a coordinate transformation from the second space to the first space; determining, using a value and the interpolant, an interpolated value, wherein the value is based on the navigational viewpoint; and generating the third image data using the interpolated value, the second image data, and the first image data.

[0109] Example 16: The method of any of Examples 14-15, wherein the navigational viewpoint is based on a point on the anatomical element, and wherein the third image data depicts the anatomical element relative to the point.Docket No. A0011887W001

[0110] Example 17: The method of any of Examples 14-16, wherein identifying the navigational viewpoint comprises: receiving a user input indicating a position on the anatomical element, and wherein the navigational viewpoint is based on the position.

[0111] Example 18: The method of any of Examples 14-17, wherein identifying the navigational viewpoint comprises: identifying a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

[0112] Example 19: The method of any of Examples 14-18, wherein identifying the navigational viewpoint comprises: obtaining, during the procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element; and identifying the navigational viewpoint based on the positional information.

[0113] Example 20: The method of any of Examples 14-19, wherein identifying the navigational viewpoint comprises: obtaining, during the procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body; and identifying the navigational viewpoint based on the movement information.

[0114] Example 21 : A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to: obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtain, during a procedure, tracking information indicative of pose of the anatomical element during the procedure; generate, based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information; identify a navigational viewpoint on the anatomical element; interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and provide, during the procedure, the third image data for display.

[0115] Example 22: The non-transitory computer-readable medium of Example 21, wherein the one or more instructions, that cause the device to identify the navigational viewpoint, cause the device to: receive a user input indicating a position on the anatomical element, and wherein the navigational viewpoint is based on the position.Docket No. A0011887W001

[0116] Example 23: The non-transitory computer-readable medium of any of Examples 21-22, wherein the one or more instructions, that cause the device to identify the navigational viewpoint, cause the device to: identify a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

[0117] Example 24: The non-transitory computer-readable medium of any of Examples 21-23, wherein the one or more instructions, that cause the device to identify the navigational viewpoint, cause the device to: obtain, during the procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element; and identify the navigational viewpoint based on the positional information.

[0118] Example 25: The non-transitory computer-readable medium of any of Examples 21-24, wherein the one or more instructions, that cause the device to identify the navigational viewpoint, cause the device to: obtain, during the procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body; and identify the navigational viewpoint based on the movement information.

[0119] Example 26: A system configured to perform one or more operations recited in one or more of Examples 1-25.

[0120] Example 27: An apparatus comprising means for performing one or more operations recited in one or more of Examples 1-25.

[0121] Example 28: A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising one or more instructions that, when executed by a device, cause the device to perform one or more operations recited in one or more of Examples 1-25.

[0122] Example 29: A computer program product comprising instructions or code for executing one or more operations recited in one or more of Examples 1-25.

[0123] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the aspects described herein to the precise forms that are described. Modifications and variations may be made in light of the above description or may be acquired from practice of the aspects described herein.

[0124] As used herein, the term “component” is intended to be broadly construed as hardware, firmware, and / or a combination of hardware and software. It will be apparent that systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is notDocket No. A0011887W001 limiting of the aspects described herein. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code — it being understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0125] As used herein, satisfying a threshold may, depending on the context, refer to a value being greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, or the like.

[0126] Even though particular combinations of features are recited in the claims and / or described in the specification, these combinations are not intended to limit the aspects described herein. In fact, many of these features may be combined in ways not specifically recited in the claims and / or described in the specification. Although each dependent claim listed below may directly depend on only one claim, the description includes each dependent claim in combination with every other claim in the claim set. As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiple of the same item.

[0127] When “a component” or “one or more components” (or another element, such as “a controller” or “one or more controllers”) is described or claimed (within a single claim or across multiple claims) as performing multiple operations or being configured to perform multiple operations, this language is intended to broadly cover a variety of architectures and environments. For example, unless explicitly claimed otherwise (e.g., via the use of “first component” and “second component” or other language that differentiates components in the claims), this language is intended to cover a single component performing or being configured to perform all of the operations, a group of components collectively performing or being configured to perform all of the operations, a first component performing or being configured to perform a first operation and a second component performing or being configured to perform a second operation, or any combination of components performing or being configured to perform the operations. For example, when a claim has the form “one or more components configured to: perform X; perform Y; and perform Z,” that claim should be interpreted to mean “one or more components configured to perform X; one or more (possibly different) components configured to perform Y; and one or more (also possibly different) components configured to perform Z.”Docket No. A0011887W001

[0128] No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items, and may be used interchangeably with “one or more.” Further, as used herein, the article “the” is intended to include one or more items referenced in connection with the article “the” and may be used interchangeably with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, or a combination of related and unrelated items,), and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. Also, as used herein, the term “or” is intended to be inclusive when used in a series and may be used interchangeably with “and / or,” unless explicitly stated otherwise (e.g., if used in combination with “either” or “only one of’).

Claims

Docket No. A0011887W001WHAT IS CLAIMED IS1. A device (410), comprising: one or more memories (530); and one or more processors (520), communicatively coupled to the one or more memories (530), configured to: obtain first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtain second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies of the anatomical element that are registered to a common coordinate system in the second space, and wherein the second image data depicts relative poses of the set of segmented bodies; identify a navigational viewpoint on the anatomical element; interpolate, based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and provide the third image data for display.

2. The device ( 10) of claim 1, wherein the one or more processors (520), to interpolate the second image data to generate the third image data, are configured to: perform, based on the navigational viewpoint, a rigid body transformation to transform the second image data from the second space to the first space.

3. The device (410) of any of claims 1-2, wherein the one or more processors (520), to interpolate the second image data to generate the third image data, are configured to: generate, based on the second image data, pose information representative of the relative poses of the set of segmented bodies; determine an interpolant as a function of the pose information for a coordinate transformation from the second space to the first space; determine, using a value and the interpolant, an interpolated value, wherein the value is based on the navigational viewpoint; andDocket No. A0011887W001 generate the third image data using the interpolated value, the second image data, and the first image data.

4. The device (410) of any of claims 1-3, wherein the navigational viewpoint is based on a point on the anatomical element, and wherein the third image data depicts the anatomical element relative to the point.

5. The device (410) of any of claims 1-4, wherein the one or more processors (520), to identify the navigational viewpoint, are configured to: receive a user input indicating a position on the anatomical element, and wherein the navigational viewpoint is based on the position.

6. The device (410) of any of claims 1-5, wherein the one or more processors (520), to identify the navigational viewpoint, are configured to: identify a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

7. The device (410) of any of claims 1-6, wherein the one or more processors (520), to identify the navigational viewpoint, are configured to: obtain, during a procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element; and identify the navigational viewpoint based on the positional information.

8. The device (410) of any of claims 1-7, wherein the one or more processors (520), to identify the navigational viewpoint, are configured to: obtain, during a procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body; and identify the navigational viewpoint based on the movement information.

9. The device (410) of any of claims 1-8, wherein the second image data is based on tracking data of the anatomical element during a procedure, and wherein the contextual data is not tracked during the procedure.Docket No. A0011887W00110. The device (410) of any of claims 1-9, wherein the contextual data indicates at least one of: one or more blood vessel locations, one or more nerve locations, or one or more sensitive structure locations.

11. A method, comprising: obtaining, by a device ( 10), first image data depicting an anatomical element, wherein the first image data is associated with a first space, and wherein the first image data includes contextual data associated with the anatomical element; obtaining, by the device (410) and during a procedure, tracking information indicative of pose of the anatomical element during the procedure; generating, by the device (410) and based on the tracking information, second image data depicting the anatomical element, wherein the second image data is associated with a second space, wherein the second image data depicts a set of segmented bodies in relative poses that are based on the tracking information; identifying, by the device (410), a navigational viewpoint on the anatomical element; interpolating, by the device (410) and based on the navigational viewpoint, the second image data to generate third image data, wherein the third image data includes the second image data transformed from the second space to the first space, and wherein the third image data includes the contextual data; and providing, by the device (410) and during the procedure, the third image data for display.

12. The method of claim 11, wherein interpolating the second image data to generate the third image data comprises: generating, based on the second image data, pose information representative of the relative poses of the set of segmented bodies; determining an interpolant as a function of the pose information for a coordinate transformation from the second space to the first space; determining, using a value and the interpolant, an interpolated value, wherein the value is based on the navigational viewpoint; and generating the third image data using the interpolated value, the second image data, and the first image data.Docket No. A0011887W00113. The method of any of claims 11-12, wherein identifying the navigational viewpoint comprises: identifying a center position on the anatomical element, wherein the navigational viewpoint is based on the center position.

14. The method of any of claims 11-13, wherein identifying the navigational viewpoint comprises: obtaining, during the procedure, positional information of a surgical tool indicative of a position of the surgical tool relative to the anatomical element; and identifying the navigational viewpoint based on the positional information.

15. The method of any of claims 11-14, wherein identifying the navigational viewpoint comprises: obtaining, during the procedure, movement information of at least one segmented body of the set of segmented bodies, wherein the movement information is indicative of a movement of the at least one segmented body; and identifying the navigational viewpoint based on the movement information.

Images