Systems and methods for adaptive visualization of medical data

Abstract

A system according to at least one embodiment of the present disclosure includes: a processor; and a memory coupled with the processor and storing data thereon that, when processed by the processor, enable the processor to: receive medical data and a rendering instruction indicative of an appearance of the medical data when the medical data is displayed on a display; determine, based on the rendering instruction, that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction; and render, to the display, the medical data.

Description

SYSTEMS AND METHODS FOR ADAPTIVE VISUALIZATION OF MEDICALDATACROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of and priority to U.S. Provisional Application No. 63 / 607,034 filed on December 6, 2023, entitled “SYSTEMS AND METHODS FOR ADAPTIVE VISUALIZATION OF MEDICAL DATA”, the entirety of which is hereby incorporated herein by reference.FIELD OF INVENTION

[0002] The present disclosure is generally directed to visual depictions of data rendered to a display, and relates more particularly to visual depictions of medical data rendered to a display.BACKGROUND

[0003] Surgical robots may assist a surgeon or other medical provider in carrying out a surgical procedure, or may complete one or more surgical procedures autonomously. Imaging may be used by a medical provider for diagnostic and / or therapeutic purposes. Patient anatomy can change over time, particularly following placement of a medical implant in the patient anatomy.BRIEF SUMMARY

[0004] Rendering large or complete portions of medical data (e.g., medical images) can be computationally expensive. Moreover, often surgical teams do not require such complete renderings, as they may be focused solely on small or specific portions of the medical data. According to embodiments of the present disclosure, adaptive visualization is implemented that allocates increased computational resources to rendering portions of medical data that are deemed critical services, while minimizing or implementing reduced computational resources used to render non-critical portions of the medical data. Such an approach beneficially optimizes computational resources, provides surgical teams with increased flexibility, and enables remote or resource scarce computing systems to render medical data.

[0005] Example aspects of the present disclosure include:

[0006] A system according to at least one embodiment of the present disclosure comprises: a processor; and a memory coupled with the processor and storing data thereon that, when processed by the processor, enable the processor to: receive medical data and a rendering instruction indicative of an appearance of the medical data when the medicaldata is displayed on a display; determine, based on the rendering instruction, that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction; and render, to the display, the medical data.

[0007] Any of the aspects herein, wherein the rendering instruction is based on at least one of a user input and information stored in a database.

[0008] Any of the aspects herein, wherein the first portion of the medical data is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

[0009] Any of the aspects herein, wherein the first portion of the medical data is associated with an anatomical element, and wherein the second portion of the medical data is associated with at least one surgical tool.

[0010] Any of the aspects herein, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

[0011] Any of the aspects herein, wherein the rendering instruction is based on an amount of bandwidth associated with the display.

[0012] Any of the aspects herein, wherein the rendering instruction is based on at least one of a step in a surgical procedure and a physician preference.

[0013] Any of the aspects herein, wherein the first appearance comprises a border rendered around a perimeter of the first portion.

[0014] A method according to at least one embodiment of the present disclosure comprises: receiving medical data and a rendering instruction indicative of an appearance of the medical data when the medical data is displayed on a display; determining, based on the rendering instruction, that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transforming at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction; and rendering, to the display, the medical data.

[0015] Any of the aspects herein, wherein the medical data is rendered in a plurality of views, and wherein the rendering instruction comprises disabling at least one view of theplurality of views.

[0016] Any of the aspects herein, further comprising: determining that a processing bandwidth of a processor has fallen below a threshold value; and generating, when the processing bandwidth is below the threshold value, a second rendering instruction.

[0017] Any of the aspects herein, wherein the transforming of the at least one of the first portion and the second portion comprises at least one of upsampling and downsampling, using a data model, data associated with the at least one of the first portion and the second portion.

[0018] Any of the aspects herein, wherein the first portion of the medical data is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

[0019] Any of the aspects herein, wherein the first portion of the medical data is associated with an anatomical element, and wherein the second portion of the medical data is associated with at least one surgical tool.

[0020] Any of the aspects herein, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

[0021] Any of the aspects herein, wherein the rendering instruction is based on an amount of bandwidth associated with the display.

[0022] Any of the aspects herein, wherein the rendering instruction is based on at least one of a step in a surgical procedure and a physician preference.

[0023] Any of the aspects herein, wherein the first appearance comprises a border rendered around a perimeter of the first portion.

[0024] A system according to at least one embodiment of the present disclosure comprises: a display; a processor; and a memory coupled with the processor and storing data thereon that, when processed by the processor, enable the processor to: receive medical data and a rendering instruction indicative of an appearance of the medical data when the medical data is displayed on the display; determine, based on the rendering instruction, that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction; and render, to the display, the medical data.

[0025] Any of the aspects herein, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

[0026] Any aspect in combination with any one or more other aspects.

[0027] Any one or more of the features disclosed herein.

[0028] Any one or more of the features as substantially disclosed herein.

[0029] Any one or more of the features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein.

[0030] Any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments.

[0031] Use of any one or more of the aspects or features as disclosed herein.

[0032] It is to be appreciated that any feature described herein can be claimed in combination with any other feature(s) as described herein, regardless of whether the features come from the same described embodiment.

[0033] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

[0034] The phrases “at least one”, “one or more”, and “and / or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at least one of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and / or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together. When each one of A, B, and C in the above expressions refers to an element, such as X, Y, and Z, or class of elements, such as Xl-Xn, Yl-Ym, and Zl-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., XI and X2) as well as a combination of elements selected from two or more classes (e.g., Y1 and Zo).

[0035] The term “a” or “an” entity refers to one or more of that entity. As such, the terms “a” (or “an”), “one or more” and “at least one” can be used interchangeably herein. It is also to be noted that the terms “comprising”, “including”, and “having” can be used interchangeably.

[0036] The preceding is a simplified summary of the disclosure to provide anunderstanding of some aspects of the disclosure. This summary is neither an extensive nor exhaustive overview of the disclosure and its various aspects, embodiments, and configurations. It is intended neither to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure but to present selected concepts of the disclosure in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other aspects, embodiments, and configurations of the disclosure are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.

[0037] Numerous additional features and advantages of the present disclosure will become apparent to those skilled in the art upon consideration of the embodiment descriptions provided hereinbelow.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0038] The accompanying drawings are incorporated into and form a part of the specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the disclosure. The drawings simply illustrate preferred and alternative examples of how the disclosure can be made and used and are not to be construed as limiting the disclosure to only the illustrated and described examples. Further features and advantages will become apparent from the following, more detailed, description of the various aspects, embodiments, and configurations of the disclosure, as illustrated by the drawings referenced below.

[0039] Fig. l is a block diagram of a system according to at least one embodiment of the present disclosure;

[0040] Fig. 2A shows aspects of a display according to at least one embodiment of the present disclosure;

[0041] Fig. 2B shows the display with adaptive display features applied according to at least one embodiment of the present disclosure;

[0042] Fig. 3 A shows aspects of a second display according to at least one embodiment of the present disclosure;

[0043] Fig. 3B shows of the second display with adaptive display features applied according to at least one embodiment of the present disclosure; and

[0044] Fig. 4 is a flowchart according to at least one embodiment of the present disclosure.DETAILED DESCRIPTION

[0045] It should be understood that various aspects disclosed herein may be combined indifferent combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and / or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0046] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Alternatively or additionally, functions may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (alone or in combination with instructions). Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0047] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple Al l, A12, A12X, A12Z, or A13 Bionic processors; Advanced RISC Machine (ARM) processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000-series processors, Nvidia GeForce RTX 3000-series processors, AMD Radeon RX 5000-series processors, AMD Radeon RX 6000-series processors, or any other graphics processing units), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, thetechniques could be fully implemented in one or more circuits or logic elements.

[0048] Before any embodiments of the disclosure are explained in detail, it is to be understood that the disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Further, the present disclosure may use examples to illustrate one or more aspects thereof. Unless explicitly stated otherwise, the use or listing of one or more examples (which may be denoted by “for example,” “by way of example,” “e.g.,” “such as,” or similar language) is not intended to and does not limit the scope of the present disclosure.

[0049] High quality visualization in surgical navigation and robotics systems is beneficial since the visualization is relied on by surgical teams to guide and deliver therapy and interventions. Such visualization may include the entirety of medical data being rendered for visualization. While all the medical data may be rendered, surgeons and other members of the surgical team may be focused on certain portions (e.g., portions deemed useful or important by the surgical team, the surgical plan, etc.) of the anatomy for procedure, such that a full rendering of the entire anatomy (e.g., including far field data) may be unnecessary and may present visual distractions. Such full rendering may also take away from the limited screen real estate available to display the certain portions desired by the surgical team. Additionally, the rendering may be a computationally expensive task requiring software and hardware resources that may burden the system, which may result in unpleasant lags in the display for users and / or may deny resources for other computationally-heavy tasks, such as Artificial Intelligence (Al) or Machine Learning (ML) methods.

[0050] According to at least one embodiment of the present disclosure, systems and methods may adaptively focus on providing high quality visualizations that are deemed critical microservices (e.g., instrument location, any other task or item identified as a critical microservice by the surgical team, the surgical plan, etc.) while reducing visualization needs in less critical tasks. Such focus may reduce visual distractions, such as when bandwidth or computational resources are limited, and / or improve utilization of thescreen real estate for critical microservices (e.g., anatomy, software features, etc.), which may in turn enhance the overall visualization experience of the users as well as reduce computational and / or rendering burdens on the system.

[0051] According to at least one embodiment of the present disclosure, adaptive visualization may provide an enhanced view of the regions or parts of the medical data (e.g., Magnetic Resonance Imaging (MRI) images, Computed Tomography (CT) images, etc.), such as by providing a zoomed view, an integrated or fused view of multiple data sources, and / or a view that includes AI / ML enhanced data in the region. The adaptive visualization may also contain customizable configurations (e.g., based on the preferences of the surgeon and surgical team) that allow definitions of critical services (e.g., instrument locations, biopsy trajectory, etc.). In other words, the surgeon and the surgical team can pick and choose which portions of medical data are labeled as a critical microservice (also referred to herein as a critical service), and the adaptive visualization system may adjust the display of medical data based on such labeling.

[0052] According to at least one embodiment of the present disclosure, adaptive visualization may reduce, when bandwidth and / or computational resources are limited, the display quality for microservices that are defined or deemed as less critical to free up resources for others features or functions that are computationally expensive (e.g., AI / ML methods). Such an approach may enable the use of visualization with greater software and hardware flexibility (e.g., on remote computers or the like, where computational specifications may not be as powerful). Such a solution may provide a unique, adaptive, and enhanced display that better enables surgeons and surgical teams to provide high quality care in regions of interest, maximize utilization of equipment, and enable users from remote applications to better interact with the medical data. The adaptive visualization may be used in any type of surgery or surgical procedure, such as a cranial procedure, a spine procedure, combinations thereof, and / or the like.

[0053] Embodiments of the present disclosure provide technical solutions to one or more of the problems of (1) rendering medical data to a display using systems with low bandwidth or computational resources, (2) latency issues associated with displaying medical data, and (3) rendering medical data on remote user computers.

[0054] Turning first to Fig. 1, a block diagram of an adaptive visualization system 100 (referred to herein as the system 100) according to at least one embodiment of the present disclosure is shown. The system 100 may be used to render medical data to a display; to control, pose, and / or otherwise manipulate a surgical mount system, a surgical arm, and / orsurgical tools attached thereto; and / or carry out one or more other aspects of one or more of the methods disclosed herein. The system 100 comprises a computing device 102, one or more imaging devices 112, a robot 114, a navigation system 118, a database 130, cloud or other network 134, and / or a display 136. Systems according to other embodiments of the present disclosure may comprise more or fewer components than the system 100. For example, the system 100 may not include the imaging device 112, the robot 114, the navigation system 118, one or more components of the computing device 102, the database 130, and / or the cloud 134.

[0055] The computing device 102 comprises a processor 104, a memory 106, a communication interface 108, and a user interface 110. Computing devices according to other embodiments of the present disclosure may comprise more or fewer components than the computing device 102.

[0056] The processor 104 of the computing device 102 may be any processor described herein or any similar processor. The processor 104 may be configured to execute instructions stored in the memory 106, which instructions may cause the processor 104 to carry out one or more computing steps utilizing or based on data received from the imaging device 112, the robot 114, the navigation system 118, the database 130, and / or the cloud 134. The processor 104 may be or comprise one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple Al l, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000-series processors, Nvidia GeForce RTX 3000-series processors, AMD Radeon RX 5000-series processors, AMD Radeon RX 6000-series processors, or any other graphics processing units), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.

[0057] The memory 106 may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. The memory 106 may store information or data useful for completing, for example, any step of the method 400 described herein, or of any other methods. The memory 106 may store, for example, instructions and / or machine learning models that support one or more functions of the robot 114. For instance, the memory 106 may store content (e.g., instructions and / ormachine learning models) that, when executed by the processor 104, enable image processing 120, segmentation 122, transformation 124, and / or registration 128. Such content, if provided as in instruction, may, in some embodiments, be organized into one or more applications, modules, packages, layers, or engines. Alternatively or additionally, the memory 106 may store other types of content or data (e.g., machine learning models, artificial neural networks, deep neural networks, etc.) that can be processed by the processor 104 to carry out the various method and features described herein. Thus, although various contents of memory 106 may be described as instructions, it should be appreciated that functionality described herein can be achieved through use of instructions, algorithms, and / or machine learning models. The data, algorithms, and / or instructions may cause the processor 104 to manipulate data stored in the memory 106 and / or received from or via the imaging device 112, the robot 114, the database 130, and / or the cloud 134 to, for example, render medical data to the display 136 in accordance with the rendering instructions 140, as discussed in further detail below.

[0058] The communication interface 108 may be used for receiving image data or other information (e.g., medical data) from an external source (such as the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, and / or any other system or component not part of the system 100), and / or for transmitting instructions, images, or other information to an external system or device (e.g., another computing device 102, the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, the display 136, and / or any other system or component not part of the system 100). The communication interface 108 may comprise one or more wired interfaces (e.g., a USB port, an Ethernet port, a Firewire port) and / or one or more wireless transceivers or interfaces (configured, for example, to transmit and / or receive information via one or more wireless communication protocols such as 802.1 la / b / g / n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface 108 may be useful for enabling the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

[0059] The computing device 102 may also comprise one or more user interfaces 110. The user interface 110 may be or comprise a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or for providing information to a user. The user interface 110 may be used, for example, to receive a user selection or other user input regarding any step of any methoddescribed herein. Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the system 100 (e.g., by the processor 104 or another component of the system 100) or received by the system 100 from a source external to the system 100. In some embodiments, the user interface 110 may be useful to allow a surgeon or other user to modify instructions to be executed by the processor 104 according to one or more embodiments of the present disclosure, and / or to modify or adjust a setting of other information displayed on the user interface 110 or corresponding thereto, such as the rendering instructions 140.

[0060] Although the user interface 110 is shown as part of the computing device 102, in some embodiments, the computing device 102 may utilize a user interface 110 that is housed separately from one or more remaining components of the computing device 102. In some embodiments, the user interface 110 may be located proximate one or more other components of the computing device 102, while in other embodiments, the user interface 110 may be located remotely from one or more other components of the computing device 102.

[0061] The imaging device 112 may be operable to image anatomical feature(s) (e.g., a bone, veins, tissue, etc.) and / or other aspects of patient anatomy to yield image data (e.g., image data depicting or corresponding to a bone, veins, tissue, etc.). “Image data” as used herein refers to the data generated or captured by an imaging device 112, including in a machine-readable form, a graphical / visual form, and in any other form. In various examples, the image data may comprise data corresponding to an anatomical feature of a patient, or to a portion thereof. The image data may be or comprise a preoperative image, an intraoperative image, a postoperative image, or an image taken independently of any surgical procedure. In some embodiments, a first imaging device 112 may be used to obtain first image data (e.g., a first image) at a first time, and a second imaging device 112 may be used to obtain second image data (e.g., a second image) at a second time after the first time. The imaging device 112 may be capable of taking a two-dimensional (2D) image or a three-dimensional (3D) image to yield the image data. The imaging device 112 may be or comprise, for example, an ultrasound scanner (which may comprise, for example, a physically separate transducer and receiver, or a single ultrasound transceiver), an O-arm, a C-arm, a G-arm, or any other device utilizing X-ray -based imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, a microscope, an optical camera, a thermographic camera (e.g., an infrared camera), a radar system (whichmay comprise, for example, a transmitter, a receiver, a processor, and one or more antennae), or any other imaging device 112 suitable for obtaining images of an anatomical feature of a patient. The imaging device 112 may be contained entirely within a single housing, or may comprise a transmitter / emitter and a receiver / detector that are in separate housings or are otherwise physically separated.

[0062] In some embodiments, the imaging device 112 may comprise more than one imaging device 112. For example, a first imaging device may provide first image data and / or a first image, and a second imaging device may provide second image data and / or a second image. In still other embodiments, the same imaging device may be used to provide both the first image data and the second image data, and / or any other image data described herein. The imaging device 112 may be operable to generate a stream of image data. For example, the imaging device 112 may be configured to operate with an open shutter, or with a shutter that continuously alternates between open and shut so as to capture successive images. For purposes of the present disclosure, unless specified otherwise, image data may be considered to be continuous and / or provided as an image data stream if the image data represents two or more frames per second.

[0063] The robot 114 may be any surgical robot or surgical robotic system. The robot 114 may be or comprise, for example, the Mazor X™ Stealth Edition robotic guidance system. The robot 114 may be configured to position the imaging device 112 at one or more precise position(s) and orientation(s), and / or to return the imaging device 112 to the same position(s) and orientation(s) at a later point in time. The robot 114 may additionally or alternatively be configured to manipulate a surgical tool (whether based on guidance from the navigation system 118 or not) to accomplish or to assist with a surgical task. In some embodiments, the robot 114 may be configured to hold and / or manipulate an anatomical element during or in connection with a surgical procedure. The robot 114 may comprise one or more robotic arms 116. In some embodiments, the robotic arm 116 may comprise a first robotic arm and a second robotic arm, though the robot 114 may comprise more than two robotic arms. In some embodiments, one or more of the robotic arms 116 may be used to hold and / or maneuver the imaging device 112. In embodiments where the imaging device 112 comprises two or more physically separate components (e.g., a transmitter and receiver), one robotic arm 116 may hold one such component, and another robotic arm 116 may hold another such component. Each robotic arm 116 may be positionable independently of the other robotic arm. The robotic arms 116 may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

[0064] The robot 114, together with the robotic arm 116, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm 116 may be positioned or positionable in any pose, plane, and / or focal point. The pose includes a position and an orientation. As a result, an imaging device 112, surgical tool, or other object held by the robot 114 (or, more specifically, by the robotic arm 116) may be precisely positionable in one or more needed and specific positions and orientations. The robotic arm(s) 116 may comprise one or more sensors that enable the processor 104 (or a processor of the robot 114) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm). In some embodiments, reference markers (e.g., navigation markers) may be placed on the robot 114 (including, e.g., on the robotic arm 116), the imaging device 112, or any other object in the surgical space. The reference markers may be tracked by the navigation system 118, and the results of the tracking may be used by the robot 114 and / or by an operator of the system 100 or any component thereof.

[0065] The navigation system 118 may provide navigation for a surgeon and / or a surgical robot during an operation. The navigation system 118 may be any now-known or future-developed navigation system, including, for example, the MedtronicStealth Station™ S8 surgical navigation system or any successor thereof. The navigation system 118 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigated trackers, or other objects within the operating room or other room in which some or all of the system 100 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 118 may comprise one or more electromagnetic sensors. In various embodiments, the navigation system 118 may be used to track a position and orientation (e.g., a pose) of the imaging device 112, the robot 114 and / or robotic arm 116, and / or one or more surgical tools (or, more particularly, to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the foregoing). The navigation system 118 may include a display for displaying one or more images from an external source (e.g., the computing device 102, imaging device 112, or other source) or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 118. In some embodiments, the system 100 can operate without the use of the navigation system 118. The navigation system 118 may be configured to provide guidance to a surgeon or other user of the system 100 or a component thereof, to the robot 114, or to any other element of the system 100 regarding, for example, a pose of one ormore anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

[0066] The database 130 may store information that correlates one coordinate system to another (e.g., one or more robotic coordinate systems to a patient coordinate system and / or to a navigation coordinate system). The database 130 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about a patient’s anatomy at and / or proximate the surgical site, for use by the robot 114, the navigation system 118, and / or a user of the computing device 102 or of the system 100); one or more images useful in connection with a surgery to be completed by or with the assistance of one or more other components of the system 100; rendering or computational information (e.g., threshold values for processing bandwidths or computational resources); critical service information (e.g., which elements or features rendered to the display 136 are deemed critical or less critical for each step of surgery or surgical procedure); and / or any other useful information. The database 130 may be configured to provide any such information to the computing device 102 or to any other device of the system 100 or external to the system 100, whether directly or via the cloud 134. In some embodiments, the database 130 may be or comprise part of a hospital image storage system, such as a picture archiving and communication system (PACS), a health information system (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0067] The cloud 134 may be or represent the Internet or any other wide area network. The computing device 102 and / or the display 136 may be connected to the cloud 134 via the communication interface 108, using a wired connection, a wireless connection, or both. In some embodiments, the computing device 102 may communicate with the database 130, the display 136, and / or an external device (e.g., a computing device) via the cloud 134. Examples of the cloud 134 include Amazon Web Services® (AWS) clouds, Microsoft Azure® clouds, and the like.

[0068] The display 136 may be connected to or included in one or more components of the system 100. The display 136 is configured to display medical data (e.g., various views of one or more images, surgical information, surgical plans, etc.) from an external source (e.g., the computing device 102, the imaging device 112, or other source) or to display an image and / or video stream from one or more cameras or other sensors of the imaging device 112, the navigation system 118, and / or the like. In some embodiments, thetransformation 124 may be similar to or the same as the user interface 110, and may communicate with one or more other components of the system 100 (e.g., via the communication interface 108, via the cloud 134, etc.). The display 136 may be configured to display additional medical data, such as a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan, visual depictions of one or more objects in the surgical environment, combinations thereof, and / or the like.

[0069] The rendering instructions 140 may be or comprise computer-readable and executable data stored in the memory 106 that, when executed by the processor 104, cause the processor 104 to render medical data in accordance with the specifications of the rendering instructions 140. The rendering instructions 140 may specify parameters or requirements for rendering medical data to the display 136, and may be based on data stored in the database 130, user input, and / or the like. The rendering instructions 140 may be adjustable based on user input (e.g., a user such as a surgeon providing data to the user interface 110). For example, the user of the system 100 may input one or more commands (e.g., via the user interface 110, via the display 136, etc.) that specify how the medical data is to be rendered. Such commands may be converted and stored as the rendering instructions 140 such that, when the rendering instructions 140 are executed, the processor 104 renders the medical data in accordance with the user’s one or more commands.Additionally or alternatively, the rendering instructions 140 may be determined based on user preference. For example, the surgeon’s preferences for the rendering of the medical data (e.g., the surgeon prefers a zoomed in view of the pedicle screw at a drilling step of surgery or surgical procedure) may be stored in the database 130, and the rendering instructions 140 may be updated with such preferences when the surgeon is part of the surgical team. In some cases, such preferences may be adjustable by the user and the may be saved for later use. For example, the rendering instructions 140 may initially be or comprise a general set of rendering instructions that the user can then modify (e.g., using inputs into the user interface 110).

[0070] The rendering instructions 140 may comprise information about critical services (e.g., surgical instruments, anatomical elements, etc.) as well as services that are less critical (e.g., background or far field data; anatomical elements that are not subject to the surgery, surgical procedure, and / or the current surgical task or step in the surgical plan; etc.). The critical services and the less critical services may be distinguished using, forexample, a binary system (e.g., data associated with critical services comprises metadata with a “1” flag, while data associated with less critical services comprises metadata with a “0” flag) or other data flagging system. Additionally or alternatively, the critical services and less critical services may be specified based on user preference. For example, the surgeon may deem the surgical instrument to be a critical service regardless of the step of the surgical procedure, and may deem the anatomical element to be a critical service during a step of drilling into the anatomical element using the surgical instrument but not a critical service for the other steps in the surgery or surgical procedure. As a result, the rendering instructions 140 may instruct the surgical instrument to be rendered with a set appearance throughout the course of the surgical procedure, and may specify that the anatomical element be rendered with different appearances depending on the step of the surgical procedure (e.g., with a first appearance when the anatomical element is deemed a critical service and with a second, different appearance when the anatomical element is deemed less critical). In some examples, the rendering instructions 140 may be rendered to the user interface 110 before implementation, such that the user can adjust the rendering instructions 140 before beginning the surgery or surgical procedure (or a step thereof).

[0071] The rendering instructions 140 may specify the appearance of one or more of the critical services and one or more of the less critical services when rendered to the display 136. In some cases, the one or more critical services may be rendered differently than the less critical services. For example, data associated with the critical services may be rendered with a higher resolution than data associated with less critical services (e.g., data associated with an anatomical element may be rendered in higher resolution than data associated with the background behind the anatomical element). In another example, the data may be omitted, such that data associated with the critical services is rendered to the display 136, while data associated with less critical services is not rendered or cropped out from the display 136. Additional or alternative non-limiting examples of differences in appearance between critical services and less critical services comprise rendering a critical service with a different window shape, a different amount of blur, a different amount of coarseness or resolution (e.g., voxel spacing of voxels representative of the medical image data), a different amount of granularity than the less critical services, upsampling data associated with the critical services using one or more Al or ML methods, downsampling data associated with less critical services using one or more Al or ML methods, combinations thereof, and / or the like.

[0072] The system 100 or similar systems may be used, for example, to carry out one ormore aspects of the method 400 described herein. The system 100 or similar systems may also be used for other purposes.

[0073] With reference to Figs. 2A-2B, aspects of a display 202 are shown in accordance with embodiments of the present disclosure. The display 202 may be similar to or the same as the display 136. The display 202 may be divided into a plurality of sections (e.g., quadrants) that can render data independently of the other sections. As an example, the display 202 depicted in Fig. 2A comprises a first quadrant 204, a second quadrant 208, a third quadrant 212, and a fourth quadrant 216. In some embodiments, the display 202 may comprise an additional or alternative number of sections (e.g., two sections, three sections, five sections, six sections, etc.).

[0074] Each of the first quadrant 204, the second quadrant 208, the third quadrant 212, and the fourth quadrant 216 may depict a portion or view of medical data, such as an MRI image of patient’s cranium. For example, the first quadrant 204 may depict a coronal view, the second quadrant 208 may depict a sagittal view, the third quadrant 212 may depict an axial view, and the fourth quadrant 216 may depict a 3D view. In some embodiments, the display 202 may be adjustable (e.g., by a user inputting commands into the user interface 110) to change which view is displayed in each quadrant of the display 202. In some embodiments, one or more portions of the display 202 may depict a first set of medical data, while the one or more other portions of the display 202 may depict a second different set of medical data. For example, the first quadrant 204 and the second quadrant 208 may depict medical images of the patient, the third quadrant 212 may depict information about the surgical plan, and the fourth quadrant 216 may depict a surgical instrument.

[0075] The display 202 may display data in accordance with the rendering instructions 140. The rendering instructions 140 may specify the appearance of the data to be rendered to each quadrant of the display 202, and the processor 104 may, when generating the data for display, use image processing 120 to transform the data such that, when displayed, the data conforms with the rendering instructions 140. For example, the rendering instructions 140 may specify that a surgical instrument 220 (e.g., a probe with navigation markers attached thereto) and an anatomical element 222 proximate the surgical instrument 220 are critical services, and that anatomical tissue 224 outside a threshold distance from the surgical instrument 220 is a less critical service. The processor 104 may, after receiving executing the rendering instructions, identify the portions of the data associated with the critical service and the less critical service (e.g., based on segmentation 122 of a medical image depicting the surgical instrument 220, the anatomical element 222, and theanatomical tissue 224). Then, the processor 104 may transform, using transformation 124, the medical data such that the surgical instrument 220, the anatomical element 222, and the anatomical tissue 224 can be rendered in accordance with the rendering instructions 140. For example, the rendering instructions 140 may specify that the anatomical tissue 224 should be blocked, obfuscated, or otherwise obscured while the surgical instrument 220 and the anatomical element 222 should be zoomed in. The processor 104 may then use image processing 120 to minimize the appearance of the pixels associated with the anatomical tissue 224 (e.g., darkening the pixels) and / or to blur other far field data (e.g., other objects in the medical data proximate the anatomical tissue 224 and / or other objects identified as less critical services) while also adjusting pixels values associated with the surgical instrument 220 and the anatomical element 222 to appear zoomed in. The processor 104 may then render the medical data associated with the surgical instrument 220, the anatomical element 222, and the anatomical tissue 224 to the display 202.

[0076] With reference to Figs. 3A-3B, aspects of a display 302 are shown in accordance with embodiments of the present disclosure. The display 302 may be similar to or the same as the display 136 or the display 202. In other words, the display 302 may be divided into a plurality of sections (e.g., quadrants) that can render data independently of the other sections. As an example, the display 302 depicted in Fig. 3A comprises a first section 304, a second section 308, and a third section 312. In some embodiments, the display 302 may comprise an additional or alternative number of sections (e.g., two sections, four sections, five sections, six sections, etc.).

[0077] Each of the first section 304, the second section 308, and the third section 312 may depict a portion or view of medical data, such as fluoroscopic images of the patient’s spine. For example, the first section 304 may depict an anterior-posterior view of the spine, the second section 308 may depict a lateral view of the spine, the third section 312 may depict an isometric or 3D view of the spine. In some embodiments, the display 302 may be adjustable (e.g., by a user inputting commands into the user interface 110) to change which view is displayed in each section of the display 302. The images may also include additional elements, such as a first element 320 and a second element 324. The first element 320 and the second element 324 may correspond to additional objects present in the surgical scene when the fluoroscopic images were taken (or, in the case when the display 136 renders a live feed, objects currently present in the surgical scene). For example, the first element 320 may correspond to a surgical instrument (e.g., the end effector of the robotic arm 116), while the second element 324 may correspond to asurgical implant (e.g., a personalized rod spanning across multiple vertebrae used in a spinal fusion surgery).

[0078] The display 302 may display medical data in accordance with the rendering instructions 140, which may be different for one or more steps of a surgery or surgical procedure. For example, as depicted in Fig. 3B, the current step of a surgery or surgical procedure may be associated with an anatomical element 316 (e.g., a vertebra of the patient’s spine), such as when the first element 320 and the second element 324 interact with the anatomical element 316 (e.g., the second element 324 comprises a rod that is inserted by the first element 320, which may comprise one or more surgical tools). During this step, the rendering instructions 140 may specify that the anatomical element 316, the first element 320, and the second element 324 are critical services (e.g., elements that should be rendered to the display 302), while the remaining medical data (e.g., data associated with other anatomical elements, data associated with navigation markers, etc.) are less critical. The rendering instructions 140 for this step may be based on user input (e.g., the physician specifies that the anatomical element 316, the first element 320, and the second element 324 are critical services), predetermined user preference (e.g., a surgical plan saved in the database 130 specifies that the user prefers that the anatomical element 316, the first element 320, and the second element 324 be designated critical services at this particular step of the surgery or surgical procedure), the traditional workflow associated with surgery or surgical procedure (e.g., default workflows or workflows in the industry usually specify the anatomical element 316, the first element 320, and the second element 324 are critical services), combinations thereof, and / or the like.

[0079] Once the rendering instructions 140 are specified, the processor 104 may process the rendering instructions 140 and use image processing 120 to adjust one or more sections of the display 302 such that the data complies with the rendering instructions 140. Continuing from the above example, the processor 104 may use image processing 120 to crop out, blur, or otherwise obfuscate data not associated with the anatomical element 316, the first element 320, and the second element 324. For example, the processor 104 may place windows 328 over the left and right hand sides of the lateral view shown in the second section 308, such that vertebrae other than the anatomical element 316 are obfuscated, blocked, blurred, or otherwise not shown. Additionally or alternatively, the other vertebrae other than the anatomical element 316 may be rendered with reduced quality (e.g., are blurrier, are coarser, have decreased granularity, etc.) relative to theanatomical element 316. As another example, the processor 104 may use transformation 124 to zoom in on the first element 320 and the second element 324 in the third section 312. The transformation 124 may include adjusting the values of the pixels associated with the first element 320 and / or the second element 324 such that a zoomed in view of the first element 320 and / or the second element 324 is displayed in third section 312. In some embodiments, one or more sections of the display 302 may remain constant or the same, even when other sections of the display 302 are changed. For example and as depicted in Fig. 3B, even when data rendered in the second section 308 and the third section 312 is changed, the first section 304 may remain the same. In some embodiments, the section may not change because the rendering instructions 140 do not require any change to the section, because all critical structures are already depicted, because the rendering of the section requires little or minimal computational resources (e.g., the bandwidth required to render the section falls below a threshold value stored in the database 130), combinations thereof, and / or the like. In some embodiments, the rendering instructions 140 may specify that one or more views of display 302 should be disabled. For example, the rendering instructions 140 may specify that the first section 304 on the display 302 should be disabled. In other words, the first section 304 may be turned off, or alternatively the anterior-posterior view may no longer be rendered, and instead the lateral view may be rendered across both the first section 304 and the second section 308, such that the depiction of the lateral view uses half the screen real estate (with the other half used by the isometric view depicted in the third section 312). In some embodiments, a visual indicator may be rendered to the display 136 that indicates that the first section 304 has been disabled in accordance with the rendering instructions 140.

[0080] Fig. 4 depicts a method 400 that may be used, for example, to adaptively adjust visualization of medical data on a display.

[0081] The method 400 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 104 of the computing device 102 described above. The at least one processor may be part of a robot (such as a robot 114) or part of a navigation system (such as a navigation system 118). A processor other than any processor described herein may also be used to execute the method 400. The at least one processor may perform the method 400 by executing elements stored in a memory such as the memory 106. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 400.One or more portions of a method 400 may be performed by the processor executing any of the contents of memory, such as an image processing 120, a segmentation 122, a transformation 124, and / or a registration 128.

[0082] The method 400 comprises determining that a processing bandwidth of a processor falls below a threshold value (step 404). The processor may be similar to or the same as the processor 104. The computational resource usage of the processor 104 may be monitored by, for example, the computing device 102 as the processor 104 performs rendering tasks (e.g., rendering data to the display 136, the display 202, the display 302, or any other display) based on rendering instructions. When the bandwidth of the processor falls below the threshold value, the computational resource usage of the processor 104 exceeds a threshold value, and / or the like, the computing device 102 may determine that the processor cannot render in accordance with the rendering instructions. Such determination may mean that processor 104 cannot render the data in an appropriate amount of time (e.g., a time at which the user does not detect latency in the rendering), the computational resources required to perform the rendering exceed those available, and / or the like. In some cases, the processor 104 may generate computational resource or bandwidth requirements associated with rendering the medical data in accordance with the rendering instructions. Such bandwidth requirement may be used by the computing device 102 to determine if the processor 104 can render in accordance with the rendering instructions by comparing the bandwidth requirement to the threshold value. The threshold value may be based on a predetermined value stored in the database 130 and accessed by the computing device 102, or may be accessed from a database or storage device external to the system 100.

[0083] The method 400 also comprises generating, when the processing bandwidth is below the threshold value, a rendering instruction (step 408). When the processing bandwidth of the processor 104 fails to meet the threshold value, a new rendering instruction (e.g., rendering instructions 140) may be generated that changes or alters the rendering of services deemed less critical in the original rendering instruction. For example, if the original rendering instruction specified that the background was a less critical structure that should be rendered with the same resolution as a critical service, the new rendering instruction may specify that the background is a less critical structure that is to be rendered with a reduced resolution. In some embodiments, the new rendering instruction may be generated based on information accessed in the database 130 (e.g., the rendering instruction is a predetermined instruction that complies with the bandwidthlimitations of the processor 104), while in other embodiments the new rendering instruction may be based on queries sent a user through the display 136 and the user’s responses thereto (e.g., the user chooses a view to omit from the rendering). Such specification may reduce the computational resources needed by the processor 104 to render the medical data, which may in turn ensure that the processing bandwidth of the processor 104 exceeds the threshold value.

[0084] The method 400 also comprises receiving medical data and the rendering instruction indicative of an appearance of the medical data when the medical data is displayed on a display (step 412). The medical data may be or comprise information associated with medical images (e.g., MRI images, CT images, etc.) to be rendered to the display (e.g., the display 136, the display 202, the display 302, etc.). The rendering instructions may be similar to or the same as the rendering instructions 140, and may specify the appearance of the medical data when rendered to the display. The rendering instructions 140 may be or comprise the new rendering instructions in the event the bandwidth of the processor 104 fell below the threshold value, or may be or comprise the original rendering instructions in the event the processor 104 had sufficient bandwidth to handle the rendering of the original rendering instructions.

[0085] The method 400 also comprises determining, based on the rendering instruction, that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance (step 416). The first portion of the medical data may correspond to a critical service (e.g., a surgical instrument such as the surgical instrument 220) while the second portion of medical data may correspond to a less critical service (e.g., an anatomical element that does not interact with the surgical instrument during the current step of the surgery or surgical procedure). The difference between the first appearance and the second appearance may be or comprise a different window shape. For example, the first portion may be rendered across two quadrants of the screen while the second portion may be rendered in a third quadrant. As another example, the first appearance may include a window or line that outlines an outer surface, boundary, border, perimeter, etc. of the first portion (e.g., the outline of the surgical instrument), while the second section may comprise a window that covers the second portion, such that the second portion is obfuscated or blocked. In some embodiments, the window shape may be rendered in various colors (e.g., the first section is outlined in a bright color to allow the user to visually distinguish the first section from the second section or other portions of themedical data).

[0086] The difference between the first appearance and the second appearance may be or comprise a different amount of blur, coarseness or resolution, and / or granularity. For example, the second appearance may be or comprise the second section being rendered with a greater amount of blur, a greater amount of coarseness or a less amount of resolution (e.g., by changes to voxel spacing), and / or a reduced amount of granularity compared to the first section. In some embodiments, the second section may be rendered with a lower resolution than the first section. Such differences in resolution may enable the processor 104 to use greater amounts of computational resources on rendering of the first section (which corresponds to the critical service) than on the second section (which corresponds to a less critical structure).

[0087] The method 400 also comprises transforming at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction (step 420). The processor 104 may use image processing 120 and / or transformation 124 to transform the first portion and / or the second portion. For example, the processor 104 may use image processing 120 to increase the resolution of data associated with the first portion and / or decrease the resolution of data associated with the second portion. As another example, the processor 104 may use transformation 124 to zoom in on the first section.

[0088] The method 400 also comprises rendering, to the display, the medical data (step 424). Once the first and second portions of the medical data are compliant with the rendering instructions 140, the processor 104 may cause the medical data to be rendered to the display (e.g., the display 136, the display 202, the display 302, etc.). In some embodiments, the method 400 may repeat for each step of the surgery or surgical procedure, or whenever the user (e.g., a surgeon) specifies adjustments to the rendering instructions. For example, the method 400 may occur for a first step of the surgery or surgical procedure, and then may repeat once the surgery or surgical procedure advances to a second step (which may specify different critical services and less critical services).

[0089] The present disclosure encompasses embodiments of the method 400 that comprise more or fewer steps than those described above, and / or one or more steps that are different than the steps described above.

[0090] As noted above, the present disclosure encompasses methods with fewer than all of the steps identified in Fig. 4 (and the corresponding description of the method 400), as well as methods that include additional steps beyond those identified in Fig. 4 (and thecorresponding description of the method 400). The present disclosure also encompasses methods that comprise one or more steps from one method described herein, and one or more steps from another method described herein. Any correlation described herein may be or comprise a registration or any other correlation.

[0091] The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and / or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and / or configurations of the disclosure may be combined in alternate aspects, embodiments, and / or configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and / or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

[0092] Moreover, though the foregoing has included description of one or more aspects, embodiments, and / or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, embodiments, and / or configurations to the extent permitted, including alternate, interchangeable and / or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.

[0093] A set of example statements are provided below:

[0094] Statement 1 : A system, comprising: a processor (104); and a memory (106) coupled with the processor (104) and storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on a display (136, 202); determine, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portionsuch that, when rendered, the medical data is compliant with the rendering instruction (140); and render, to the display (136, 202), the medical data.

[0095] Statement 2: The system of Statement 1, wherein the rendering instruction (140) is based on at least one of a user input and information stored in a database.

[0096] Statement 3: The system of any of Statements 1-2, wherein the first portion of the medical data is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

[0097] Statement 4: The system of any of Statements 1-3, wherein the first portion of the medical data is associated with an anatomical element, and wherein the second portion of the medical data is associated with at least one surgical tool.

[0098] Statement 5: The system of any of Statements 1-4, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

[0099] Statement 6: The system of any of Statements 1-5, wherein the rendering instruction (140) is based on an amount of bandwidth associated with the display (136, 202).

[0100] Statement 7: The system of any of Statements 1-6, wherein the rendering instruction (140) is based on at least one of a step in a surgical procedure and a physician preference.

[0101] Statement 8: The system of any of Statements 1-7, wherein the first appearance comprises a border rendered around a perimeter of the first portion.

[0102] Statement 9: A method, comprising: receiving medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on a display (136, 202); determining, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transforming at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction (140); and rendering, to the display (136, 202), the medical data.

[0103] Statement 10: The method of Statement 9, wherein the medical data is rendered in a plurality of views, and wherein the rendering instruction (140) comprises disabling at least one view of the plurality of views.

[0104] Statement 11 : The method of any of Statements 9-10, further comprising:determining that a processing bandwidth of a processor (104) has fallen below a threshold value; and generating, when the processing bandwidth is below the threshold value, a second rendering instruction (1 0).

[0105] Statement 12: The method of any of Statements 9-11, wherein the transforming of the at least one of the first portion and the second portion comprises at least one of upsampling and downsampling, using a data model, data associated with the at least one of the first portion and the second portion.

[0106] Statement 13: The method of any of Statements 9-12, wherein the first portion of the medical data is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

[0107] Statement 14: The method of any of Statements 9-13, wherein the first portion of the medical data is associated with an anatomical element, and wherein the second portion of the medical data is associated with at least one surgical tool.

[0108] Statement 15: The method of any of Statements 9-14, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

[0109] Statement 16: The method of any of Statements 9-15, wherein the rendering instruction (140) is based on an amount of bandwidth associated with the display (136, 202).

[0110] Statement 17: The method of any of Statements 9-16, wherein the rendering instruction (140) is based on at least one of a step in a surgical procedure and a physician preference.

[0111] Statement 18: The method of any of Statements 9-17, wherein the first appearance comprises a border rendered around a perimeter of the first portion.

[0112] Statement 19: A system, comprising: a display (136, 202); a processor (104); and a memory (106) coupled with the processor (104) and storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on the display (136, 202); determine, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with therendering instruction (140); and render, to the display (136, 202), the medical data.

[0113] Statement 20: The system of Statement 19, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

Claims

CLAIMSWhat is claimed is:

1. A system, comprising: a processor (104); and a memory (106) coupled with the processor (104) and storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on a display (136); determine, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction (140); and render, to the display (136, 202), the medical data.

2. The system of claim 1, wherein the rendering instruction (140) is based on at least one of a user input and information stored in a database.

3. The system of any of claims 1-2, wherein the first portion of the medical data is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

4. The system of any of claims 1-3, wherein the first portion of the medical data is associated with an anatomical element, and wherein the second portion of the medical data is associated with at least one surgical tool.

5. The system of any of claims 1-4, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

6. The system of any of claims 1-5, wherein the rendering instruction (140) isbased on an amount of bandwidth associated with the display (136, 202).

7. The system of any of claims 1-6, wherein the rendering instruction (140) is based on at least one of a step in a surgical procedure and a physician preference, and wherein the first appearance comprises a border rendered around a perimeter of the first portion.

8. A method, comprising: receiving medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on a display (136, 202); determining, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transforming at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction (140); and rendering, to the display (136, 202), the medical data.

9. The method of claim 8, wherein the medical data is rendered in a plurality of views, and wherein the rendering instruction (140) comprises disabling at least one view of the plurality of views.

10. The method of any of claims 8-9, further comprising: determining that a processing bandwidth of a processor (104) has fallen below a threshold value; and generating, when the processing bandwidth is below the threshold value, a second rendering instruction (140).

11. The method of any of claims 8-10, wherein the transforming of the at least one of the first portion and the second portion comprises at least one of upsampling and downsampling, using a data model, data associated with the at least one of the first portion and the second portion.

12. The method of any of claims 8-11, wherein the first portion of the medicaldata is associated with at least one surgical tool, and wherein the second portion of the medical data is associated with an anatomical element.

13. The method of any of claims 8-12, wherein a difference between the first appearance and the second appearance comprises at least one of a different window shape, a different amount of blur, a different amount of coarseness or resolution, and a different amount of granularity.

14. The method of any of claims 8-13, wherein the rendering instruction (140) is based on at least one of an amount of bandwidth associated with the display (136, 202), a step in a surgical procedure, and a physician preference.

15. A system, comprising: a display (136, 202); a processor (104); and a memory (106) coupled with the processor (104) and storing data thereon that, when processed by the processor (104), enable the processor (104) to: receive medical data and a rendering instruction (140) indicative of an appearance of the medical data when the medical data is displayed on the display (136); determine, based on the rendering instruction (140), that a first portion of the medical data is to be rendered with a first appearance and that a second portion of the medical data is to be rendered with a second appearance different from the first appearance; transform at least one of the first portion and the second portion such that, when rendered, the medical data is compliant with the rendering instruction (140); and render, to the display (136, 202), the medical data.

Images