System and method for displaying medical imaging data

Abstract

A system for displaying medical imaging data comprises one or more data inputs, one or more processors, and one or more displays, wherein the one or more data inputs are configured to receive first image data generated by a first medical imaging device, wherein the first image data comprises a field of view (FOV) portion and a non-FOV portion, and the one or more processors are configured to identify the non-FOV portion of the first image data, generate cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and transmit the cropped first image data for display in a first portion of the display and transmit additional information for display in a second portion of the display.

Description

[0001] References to Related Applications

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 775,622, filed on December 5, 2018, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to medical imaging and, more particularly, to medical imaging processing for tissue visualization. Background Art

[0004] With the advent of high-definition (HD) and ultra-HD / 4K resolutions in surgical visualization, surgical displays with a 16:9 aspect ratio have become increasingly common. However, many minimally invasive surgical procedures still rely on optical scopes that result in a field of view (FOV) at the image sensor that is smaller than the image sensor's sensing area. This results in images and videos with a circular FOV area dominated by black pixels. For example, in many cases, utilization is only 44% of the available imager area, which can result in only 44% utilization of the available display area. This is especially true for smaller diameter scopes, typically the 4mm scopes used in arthroscopy and ENT / neuro procedures.

[0005] When a surgeon needs to view multiple information sources, such as multiple low-utilization optical microscope images, the surgeon may have to switch inputs on their primary surgical display between the various imaging feeds, use picture-in-picture or picture-by-picture modes on the surgical display, or look at two different monitors that may be in two different locations in the operating room. All of these options may result in suboptimal use of the surgical display's viewable area or cause the surgeon to context-switch between focusing on one display and another. Summary of the Invention

[0006] According to some embodiments, a medical imaging processing system is configured to process and combine medical imaging data to generate a display feed that provides an enhanced display of medical imaging. According to some embodiments, the medical imaging processing system can combine multiple imaging data streams into one or more display streams for displaying data from multiple imaging sources and other imaging session related information sources together in a single display layout. According to some embodiments, the utilization of the display layout can be optimized by removing unused portions of the imaging data, such as data generated by the imager outside of the captured field of view. In some embodiments, the display feed can be generated based on imaging session specific preferences customized for a particular type of imaging session and / or a particular imaging system user. In some embodiments, the reconfigurable hardware processor of the medical imaging processing system can be reconfigured from one imaging session to another to provide imaging data processing customized for the next imaging session. Through one or more of these capabilities, the imaging processing system can provide enhanced medical imaging visualization customized to user preferences.

[0007] According to some embodiments, a method of configuring a medical imaging processing system includes: configuring a reconfigurable hardware processor of the medical imaging processing system to a first configuration for a first medical imaging session based on first configuration data stored in a memory, wherein the first configuration implements at least a first medical imaging processing algorithm; receiving first medical imaging data generated during the first medical imaging session; generating enhanced first medical imaging data at least in part by processing the first medical imaging data using the first medical imaging processing algorithm implemented in the first configuration; displaying the enhanced first medical imaging data for observation during the first medical imaging session; reconfiguring the reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the second configuration implements at least a second medical imaging processing algorithm not implemented in the first configuration; receiving second medical imaging data generated during the second medical imaging session; generating enhanced second medical imaging data at least in part by processing the second medical imaging data using the second medical imaging processing algorithm implemented in the second configuration; and displaying the enhanced second medical imaging data on a display for observation during the second medical imaging session.

[0008] In any of these embodiments, the method may include receiving input indicative of a second medical imaging session, and in response to receiving the input, automatically reconfiguring the reconfigurable hardware processor to the second configuration.

[0009] In any of these embodiments, the input may include a selection of a type of medical procedure.

[0010] In any of these embodiments, the input may include selection of a user profile.

[0011] In any of these embodiments, the input may include selection of a default configuration profile.

[0012] In any of these embodiments, the default configuration profile may be based on one or more connections from one or more external devices to the medical imaging processing system.

[0013] In any of these embodiments, the default configuration profile may be based on the field of view of the connected external device.

[0014] In any of these embodiments, the first configuration may be associated with a first type of medical procedure and the second configuration may be associated with a second type of medical procedure.

[0015] In any of these embodiments, the first medical imaging session may include performing a first type of medical procedure on the patient, and the second medical imaging session may include performing a second type of medical procedure on the patient.

[0016] In any of these embodiments, the first configuration may be associated with a first user profile and the second configuration may be associated with a second user profile.

[0017] In any of these embodiments, the first medical imaging session may include imaging the patient and the second medical imaging session may include imaging the patient.

[0018] In any of these embodiments, the first configuration data and the second configuration data may be associated with the same type of medical procedure.

[0019] In any of these embodiments, the first medical imaging session may be a first surgical session and the second medical imaging session may be a second surgical session.

[0020] In any of these embodiments, the at least one medical imaging processing algorithm implemented in the second configuration may include a smoke detection algorithm, and generating the enhanced second medical imaging data may include enhancing the sharpness of one or more portions of the one or more images associated with smoke.

[0021] In any of these embodiments, the first medical imaging processing algorithm may be configured to detect features of the imaged tissue.

[0022] In any of these embodiments, the characteristic of the imaged tissue can be tissue perfusion, location of blood vessels, blood flow, size of the imaged tissue, or a combination thereof.

[0023] In any of these embodiments, the enhanced second medical imaging data may include an overlay on at least a portion of the second medical imaging data.

[0024] In any of these embodiments, the reconfigurable hardware processor may be reconfigured before imaging begins.

[0025] In any of these embodiments, one or more medical imaging processing algorithms may be implemented in both the first configuration and the second configuration.

[0026] In any of these embodiments, the second medical imaging data may include at least one of a video frame and an image.

[0027] In any of these embodiments, the second medical imaging data may be received from an endoscopic imaging system.

[0028] In any of these embodiments, the second medical imaging data may be received from a camera control unit.

[0029] In any of these embodiments, the reconfigurable hardware processor may be an FPGA or a GPU.

[0030] In any of these embodiments, the method may include receiving second medical imaging data from the first device, receiving data from the second medical device, and outputting a display feed to a display, the display feed including the enhanced second medical imaging data and at least a portion of the data from the second medical device.

[0031] In any of these embodiments, the method may include receiving, at a first processor, second medical imaging data and data from a second medical device, transmitting the second medical imaging data from the first processor to a reconfigurable hardware processor, receiving, at the first processor, enhanced second medical imaging data from the reconfigurable hardware processor, and generating, by the first processor, a display feed by combining the enhanced second medical imaging data with at least a portion of the data associated with the second medical device.

[0032] In any of these embodiments, the first configuration data may be stored in a remote memory and received via a network connection.

[0033] In accordance with some embodiments, a method for displaying medical imaging data includes receiving first image data generated by a first medical imaging device, wherein the first image data includes a field of view (FOV) portion and a non-FOV portion; identifying the non-FOV portion of the first image data; generating cropped first image data by removing at least a portion of the non-FOV portion of the first image data; and displaying the cropped first image data in a first portion of a display and displaying additional information in a second portion of the display.

[0034] In any of these embodiments, edge detection may be used to identify non-FOV portions.

[0035] In any of these embodiments, the first image data may include a series of video frames, and edge detection may be performed on more than one frame.

[0036] In any of these embodiments, the non-FOV portion may be identified using one or more of a center location of the FOV portion and a measurement associated with a size of the FOV portion.

[0037] In any of these embodiments, the center location of the FOV portion and measurements associated with the size of the FOV portion may be determined during an imaging session initialization process.

[0038] In any of these embodiments, the imaging session initialization process may be a white balancing process.

[0039] In any of these embodiments, the first image data may include a rectangular image or video frame, and the FOV portion may be a circular portion of the rectangular image or video frame.

[0040] In any of these embodiments, the first image data may include a video frame.

[0041] In any of these embodiments, the first image data may be received on a first input of the medical imaging processing system, and the additional information may be based on data received on a second input of the medical imaging processing system.

[0042] In any of these embodiments, the method may include transmitting a display feed from the medical imaging processing system to a display, the display feed including a combination of the cropped first image data and the additional information.

[0043] In any of these embodiments, the method may include receiving second image data generated by a second medical imaging device; identifying a non-FOV portion of the second image data; generating cropped second image data by removing at least a portion of the non-FOV portion of the second image data; and displaying the cropped second image data in a second portion of the display.

[0044] In any of these embodiments, the first image data may be received on a first input of the medical imaging processing system and the second image data may be received on a second input of the medical imaging processing system.

[0045] In any of these embodiments, the method may include transmitting a display feed from the medical imaging processing system to a display, the display feed including a combination of the cropped first image data and the cropped second image data.

[0046] In any of these embodiments, the cropped first image data and the additional information may be located on the display based on configuration data stored in the memory.

[0047] In any of these embodiments, the configuration data may include user-specified configuration data.

[0048] In any of these embodiments, the configuration data may be received via a network connection.

[0049] In any of these embodiments, the first image data may be received from an endoscopic imaging system, an intraoperative C-arm imaging system, or an ultrasound system.

[0050] In any of these embodiments, the first image data may be received from a camera control unit.

[0051] In any of these embodiments, the additional information may include one or more of patient data, measurements, charts, images, device status, and video feeds.

[0052] According to some embodiments, a reconfigurable medical imaging processing system includes a display; a memory; a reconfigurable hardware processor; and a second processor configured to: configure the reconfigurable hardware processor to a first configuration for a first medical imaging session based on first configuration data stored in the memory, wherein the reconfigurable hardware processor in the first configuration is configured to implement at least a first medical imaging processing algorithm and generate enhanced first medical imaging data for display on the display at least in part by processing the first medical imaging data using the first medical imaging processing algorithm; and reconfigure the reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the reconfigurable hardware processor in the second configuration is configured to implement at least a second medical imaging processing algorithm and generate enhanced second medical imaging data for display on the display at least in part by processing the second medical imaging data using the second medical imaging processing algorithm.

[0053] In any of these embodiments, the second processor may be configured to receive input indicative of a second medical imaging session and, in response to receiving the input, automatically reconfigure the reconfigurable hardware processor to the second configuration.

[0054] In any of these embodiments, the input may include a selection of a type of medical procedure.

[0055] In any of these embodiments, the input may include selection of a user profile.

[0056] In any of these embodiments, the input may include selection of a default configuration profile.

[0057] In any of these embodiments, the default configuration profile may be based on one or more connections from one or more external devices to the medical imaging processing system.

[0058] In any of these embodiments, the default configuration profile may be based on the field of view of the connected external device.

[0059] In any of these embodiments, the first configuration may be associated with a first type of medical procedure and the second configuration may be associated with a second type of medical procedure.

[0060] In any of these embodiments, the first medical imaging session may include performing a first type of medical procedure on the patient, and the second medical imaging session may include performing a second type of medical procedure on the patient.

[0061] In any of these embodiments, the first configuration may be associated with a first user profile and the second configuration may be associated with a second user profile.

[0062] In any of these embodiments, the first medical imaging session may include imaging the patient and the second medical imaging session may include imaging the patient.

[0063] In any of these embodiments, the first configuration data and the second configuration data may be associated with the same type of medical procedure.

[0064] In any of these embodiments, the first medical imaging session may be a first surgical session and the second medical imaging session may be a second surgical session.

[0065] In any of these embodiments, the at least one medical imaging processing algorithm implemented in the second configuration may include a smoke detection algorithm, and generating the enhanced second medical imaging data may include enhancing the sharpness of one or more portions of the one or more images associated with smoke.

[0066] In any of these embodiments, the first medical imaging processing algorithm may be configured to detect features of the imaged tissue.

[0067] In any of these embodiments, the characteristic of the imaged tissue can be tissue perfusion, location of blood vessels, blood flow, size of the imaged tissue, or a combination thereof.

[0068] In any of these embodiments, the enhanced second medical imaging data may include an overlay on at least a portion of the second medical imaging data.

[0069] In any of these embodiments, the system can be configured to reconfigure the reconfigurable hardware processor before imaging begins.

[0070] In any of these embodiments, one or more medical imaging processing algorithms may be implemented in both the first configuration and the second configuration.

[0071] In any of these embodiments, the second medical imaging data may include at least one of a video frame and an image.

[0072] In any of these embodiments, the system may be configured to receive second medical imaging data from an endoscopic imaging system.

[0073] In any of these embodiments, the system may be configured to receive second medical imaging data from a camera control unit.

[0074] In any of these embodiments, the reconfigurable hardware processor may be an FPGA or a GPU.

[0075] In any of these embodiments, the system may be configured to receive the second medical imaging data from the first device, receive the data from the second medical device, and display at least a portion of the enhanced second medical imaging data and the data from the second medical device.

[0076] In any of these embodiments, the system can be configured to receive the second medical imaging data and data from the second medical device at the second processor, transmit the second medical imaging data from the second processor to the reconfigurable hardware processor, receive the enhanced second medical imaging data from the reconfigurable hardware processor at the second processor, and generate a display feed for display by the second processor by combining the enhanced second medical imaging data with at least a portion of the data associated with the second medical device.

[0077] In any of these embodiments, the first configuration data may be stored in a remote memory and received via a network connection.

[0078] In accordance with some embodiments, a system for displaying medical imaging data includes one or more data inputs; one or more processors; and one or more displays, wherein the one or more data inputs are configured to receive first image data generated by a first medical imaging device, wherein the first image data includes a field of view (FOV) portion and a non-FOV portion, and the one or more processors are configured to identify the non-FOV portion of the first image data, generate cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and transmit the cropped first image data for display in a first portion of the display and transmit additional information for display in a second portion of the one or more displays.

[0079] In any of these embodiments, the one or more processors may be configured to identify the non-FOV portion using edge detection.

[0080] In any of these embodiments, the first image data may include a series of video frames, and the one or more processors may be configured to identify the non-FOV portion using edge detection performed on more than one frame.

[0081] In any of these embodiments, the one or more processors may be configured to identify the non-FOV portion using one or more of a center location of the FOV portion and a measurement associated with a size of the FOV portion.

[0082] In any of these embodiments, the one or more processors may be configured to determine a center location of the FOV portion and measurements associated with a size of the FOV portion during an imaging session initialization process.

[0083] In any of these embodiments, the imaging session initialization process may be a white balancing process.

[0084] In any of these embodiments, the first image data may include a rectangular image or video frame, and the FOV portion may be a circular portion of the rectangular image or video frame.

[0085] In any of these embodiments, the first image data may include a video frame.

[0086] In any of these embodiments, the one or more data inputs may be configured to receive first image data on a first input of the medical imaging processing system, and the additional medical imaging data may be based on data received on a second input of the medical imaging processing system.

[0087] In any of these embodiments, the medical imaging processing system may be configured to transmit a display feed from the medical imaging processing system to a display, which display feed may include a combination of the cropped first image data and the additional medical imaging data.

[0088] In any of these embodiments, the one or more data inputs may be configured to receive second image data generated by a second medical imaging device; and the one or more processors may be configured to: identify a non-FOV portion of the second image data, generate cropped second image data by removing at least a portion of the non-FOV portion of the second image data, and transmit the cropped second image data for display in a second portion of the one or more displays.

[0089] In any of these embodiments, the one or more data inputs may be configured to receive first image data on a first input of the medical imaging processing system and to receive second image data on a second input of the medical imaging processing system.

[0090] In any of these embodiments, the medical imaging processing system may be configured to transmit a display feed from the medical imaging processing system to a display, the display feed comprising a combination of the cropped first image data and the cropped second image data.

[0091] In any of these embodiments, the cropped first image data and the additional medical imaging data may be located on the display based on configuration data stored in the memory.

[0092] In any of these embodiments, the configuration data may include user-specified configuration data.

[0093] In any of these embodiments, the system is configured to receive configuration data via a network connection.

[0094] In any of these embodiments, the one or more data inputs may be configured to receive first image data from an endoscopic imaging system, an intraoperative C-arm imaging system, or an ultrasound system.

[0095] In any of these embodiments, the one or more data inputs may be configured to receive first image data from a camera control unit.

[0096] In any of these embodiments, the additional information may include one or more of patient data, measurements, charts, images, device status, and video feeds.

[0097] According to some embodiments, a non-transitory tangible computer-readable medium includes computer-executable program code embedded thereon for performing any one of the above methods.

[0098] According to some embodiments, a kit for processing a time series of fluorescence images of tissue of a subject includes any one of the above systems and / or any one of the above non-transitory tangible computer-readable media and a fluorescent imaging agent.

[0099] According to some embodiments, there is provided a fluorescent imaging agent for use in any of the above methods, in any of the above systems, or in any of the above kits for imaging a subject.

[0100] In any of these embodiments, imaging the subject may include imaging the subject during blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof.

[0101] In any of these embodiments, blood flow imaging, tissue perfusion imaging, and / or lymphatic imaging may include blood flow imaging, tissue perfusion imaging, and / or lymphatic imaging during an invasive surgical procedure, a minimally invasive surgical procedure, or during a non-invasive surgical procedure.

[0102] In any of these embodiments, the invasive surgical procedure may include a heart-related surgical procedure or a reconstructive surgical procedure.

[0103] In any of these embodiments, the heart-related surgical procedure may include a coronary artery bypass graft (CABG) procedure.

[0104] In any of these embodiments, the CABG procedure can be on-pump or off-pump.

[0105] In any of these embodiments, the non-invasive surgical procedure may include a wound care procedure.

[0106] In any of these embodiments, lymphatic imaging can include identification of lymph nodes, lymph node drainage, lymphatic mapping, or a combination thereof.

[0107] In any of these embodiments, the lymphatic imaging may relate to the female reproductive system.

[0108] Some embodiments include use of any of the above methods in any of the above systems or in any of the above kits for imaging a subject for lymphatic imaging.

[0109] Some embodiments include use of any of the above methods in any of the above systems or in any of the above kits for imaging a subject for blood flow imaging, tissue perfusion imaging, or a combination thereof.

[0110] It should be appreciated that any of the variations disclosed herein in connection with the methods, systems, kits, and other aspects of the present disclosure may be combined. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] The present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0112] Figure 1 is a block diagram of a system for generating and displaying medical imaging data during a medical imaging session, according to some embodiments;

[0113] Figure 2 illustrates a method for displaying medical imaging data according to some embodiments;

[0114] Figure 3A illustrates an exemplary image generated by an endoscopic imager, and Figure 3B illustrates two endoscopic images displayed side-by-side on an exemplary display;

[0115] Figure 3C illustrates an exemplary display showing a cropped endoscopic image according to some embodiments;

[0116] Figure 3D illustrates an exemplary display according to some embodiments showing a cropped endoscopic image and additional imaging session related data according to some embodiments;

[0117] Figure 4 is a block diagram of a medical imaging data processing hub according to some embodiments;

[0118] Figure 5A The diagram shows that Figure 4 an example of a first predefined display layout generated by the hub, and Figure 5B The diagram shows that Figure 4 An example of a second predefined display layout generated by the hub;

[0119] Figure 6 illustrates an example of a medical imaging processing hub configured for a first imaging session according to some embodiments;

[0120] Figure 7 illustrates a method for configuring a medical imaging processing system according to some embodiments;

[0121] Figure 8A and 8B According to an embodiment of the present invention Figure 7 A block diagram of a medical imaging processing system according to the method;

[0122] Figure 9A and 9Billustrates a graphical user interface for configuring a medical imaging processing system for a new imaging session according to some embodiments;

[0123] Figure 10 is an illustrative depiction of an exemplary fluorescence imaging system according to some embodiments;

[0124] Figure 11 is an illustrative depiction of an exemplary illumination module of a fluorescence imaging system according to some embodiments;

[0125] Figure 12 is an exemplary camera module for a fluorescence imaging system according to some embodiments; and

[0126] Figure 13 is an exemplary endoscopic imaging cart in accordance with some embodiments. DETAILED DESCRIPTION

[0127] Reference will now be made in detail to implementations and embodiments of various aspects and variations of the systems and methods described herein. Although several exemplary variations of the systems and methods are described herein, other variations of the systems and methods may include aspects of the systems and methods described herein combined in any suitable manner, having combinations of all or some of the described aspects. Described herein are systems and methods for generating enhanced medical imaging for display in conjunction with a medical imaging session (e.g., during a medical imaging session). The systems and methods can process data from multiple imaging systems to generate enhanced imaging, and can stitch multiple imaging data sets together into a single display feed for displaying information from multiple sources on a single display. The imaging data can be processed to maximize utilization of the display, thereby enabling more relevant information to be presented to the practitioner during the imaging session.

[0128] According to some embodiments, the systems and methods may process and combine imaging data differently based on the needs of each imaging session. Practitioners may be able to define the information displayed during their imaging sessions, thereby ensuring that the data is presented in a manner that is appropriate for the practitioner, which may reduce the amount of time required for the practitioner to adjust the display of the data.

[0129] In some embodiments, one or more reconfigurable hardware processors are reconfigured for each imaging session to provide imaging processing customized for each imaging session. Reconfigurable hardware processors, such as field programmable gate arrays (FPGAs), provide the low latency and high bandwidth required for real-time video processing and also provide the ability to implement different algorithms or different combinations of algorithms on different data inputs or combinations of data inputs as required in different imaging sessions, thereby providing imaging processing customized to meet the different needs of different imaging sessions. According to the embodiments described herein, this configurability and flexibility in the ability to process and combine different input data enables a single imaging processing system to support a wide variety of imaging sessions, including a wide variety of surgical procedures.

[0130] In the following description of various embodiments, reference is made to the accompanying drawings, in which are shown by way of illustration specific embodiments that can be practiced. It is to be understood that other embodiments and examples can be practiced, and changes can be made, without departing from the scope of the present disclosure.

[0131] Furthermore, it is to be understood that the singular forms "a," "an," and "the" as used in the following description are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is to be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It is to be further understood that when used herein, the terms "comprises," "comprising," "includes," and / or "comprising" specify the presence of stated features, integers, steps, operations, elements, parts, and / or units, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, units, and / or groups thereof.

[0132] Certain aspects of the present disclosure include process steps and instructions described herein in algorithmic form. It should be noted that the process steps and instructions of the present disclosure can be embodied in software, firmware, or hardware, and when embodied in software, can be downloaded to reside on and operate from different platforms used by various operating systems. Unless specifically stated otherwise as is apparent from the following discussion, it should be understood that throughout the description, discussions utilizing terms such as "process," "calculate," "calculate," "determine," "display," "generate," etc. refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within a computer system memory or register or other such information storage, transmission, or display device.

[0133] In some embodiments, the present disclosure also relates to an apparatus for performing the operations herein. The apparatus may be specially constructed for the required purpose, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory computer-readable storage medium, such as, but not limited to, any type of disk, including a floppy disk, a USB flash drive, an external hard drive, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application-specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor, or may be architectures employing a multi-processor design for increased computing power.

[0134] The methods, apparatus, and systems described herein are not inherently related to any particular computer or other device. Various general-purpose systems may also be used in conjunction with programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the required method steps. The required structures for various such systems will become apparent from the following description. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that various programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0135] Figure 1 A system 100 for generating and displaying medical imaging data during a medical imaging session is illustrated. The system 100 includes a medical data processing hub 102 that processes data received from one or more imaging modalities 104 to generate one or more display feeds for displaying enhanced medical imaging on one or more displays 106. The one or more imaging modalities 104 can generate image data associated with a patient's treatment. The image data can be images or videos generated during the patient's treatment to support one or more medical procedures, such as video captured by an endoscopic camera during an endoscopic procedure on a patient. Examples of medical imaging modalities include, but are not limited to, endoscopic systems, open field imaging systems, x-ray systems (such as intraoperative C-arm systems), computed tomography systems, ultrasound systems, magnetic resonance imaging systems, and nuclear medicine systems.

[0136] In some embodiments, the hub 102 can receive data from one or more non-imaging devices 120 that can be used in conjunction with (e.g., during) a medical imaging session and can provide information that may be relevant for display during the medical imaging session. Non-limiting examples of non-imaging devices include insufflators, irradiation controllers, and voice control systems.

[0137] The hub 102 can receive image data from one or more imaging modalities 104 via one or more input ports 108. The hub 102 generates one or more display feeds using the received imaging data and transmits the one or more display feeds to one or more displays 106 via one or more output ports 110. For example, the hub 102 can generate a display feed that includes enhanced imaging of patient tissue based on imaging generated by the one or more imaging modalities 104, and the enhanced imaging can be displayed on the one or more displays 106 to assist the practitioner during patient treatment. The hub 102 can also transmit the display feed to one or more recording devices 112 for recording the enhanced imaging for later retrieval. The input port 108 and the output port 110 can be any suitable type of data transmission port, such as a DVI port, an HDMI port, an RS232 port, an IP port, etc.

[0138] The hub 102 can be connected to one or more networks 116 via one or more network connections 118. The one or more networks can be a local area network, such as a hospital information system, or can be a wider network, such as a wide area network or the Internet. The network connection 118 can be a wired connection, such as an Ethernet connection, or a wireless network connection, such as a Wi-Fi connection. In some embodiments, the hub 102 can access one or more networks 116 to retrieve configuration data stored at a network location for configuring the hub for an imaging session, and / or can access one or more networks to receive updated software and / or updated hardware files for processing imaging data.

[0139] One or more user interfaces 114 can be connected to the hub 102 to facilitate user input to the hub 102. The user can enter data related to configuring the hub 102 for the imaging session. The user input can include, for example, selection of a practitioner profile associated with the upcoming imaging session, selection of an imaging session type or a type of procedure to be performed during the imaging session, or any other relevant information. The one or more user interfaces 114 can include a tablet computer, a keyboard, a mouse, a voice control system, a keypad, a touch screen, or any combination thereof.

[0140] As described in detail below, the hub 102 processes the received medical imaging data and any other relevant data and generates an enhanced display feed for display during an imaging session on one or more displays 106. According to some embodiments, the hub 102 may combine multiple imaging sources into a single display feed, may process the received imaging data to generate richer imaging data, may modify the imaging data for better utilization of display space, and / or may reconfigure the processing of the imaging data depending on the needs and preferences of the user in different imaging sessions.

[0141] Figure 2 A method 200 for displaying medical imaging data according to some embodiments is illustrated. The method 200 may be performed by a medical imaging data processing hub, such as the medical imaging data processing hub 102 of the system 100. The method 200 is performed to remove unutilized portions of received imaging data to better utilize display space, which may provide the ability to display more relevant information to a user during an imaging session.

[0142] In many conventional imaging systems (such as endoscope-based imaging systems, including, for example, endoscopic imaging systems), light from a roughly circular area of ​​a scene is projected onto the photosensitive portion of one or more imaging sensors. This is because the imager's sensor(s) are generally larger in area than the area of ​​light provided by the mirror optics. Consequently, the imagery captured by the sensor(s) includes a field of view (FOV) portion representing light received from the field of view and a non-FOV portion generated by portions (i.e., pixels) of the sensor(s) that do not receive light from the scene. This typically results in a rectangular image with a circular FOV portion in the middle, depicting the imaged scene surrounded by a black non-FOV portion (or nearly black due to sensor noise). When endoscopic imaging is displayed in a conventional manner, a large portion of the display is occupied by the non-FOV portion, which displays black pixels that do not provide any useful information.

[0143] To illustrate this concept, Figure 3A , an imaging system such as an exemplary endoscopic image 300 is shown in FIG. The image 300 includes a FOV portion 302 generated by a portion of the sensor that receives light from the imaged scene and a non-FOV portion 304 generated by a portion of the sensor that does not receive light from the scene. Figure 2 B illustrates two images 300 displayed side by side on an exemplary display 350. As illustrated, a relatively large amount of display space is wasted due to the non-FOV portions of the two images. In some embodiments, the hub 102 may crop some or all of the non-FOV portions of the received image data.

[0144] return Figure 2 At step 202, a medical imaging data processing hub receives first image data generated by a first medical imaging device. The first image data, which may be an image or video frame, includes a FOV portion and a non-FOV portion. For example, the first image data may be a video frame generated by an endoscopic camera, such as Figure 3A The frame may include an FOV portion generated by pixels of one or more camera sensors that receive light incident on the one or more sensors from the imaged scene, and may include a non-FOV portion generated by pixels of the one or more camera sensors that do not receive light from the imaged scene.

[0145] At step 204, a non-FOV portion of the first image data is identified. According to some embodiments, the non-FOV portion may be identified based on one or more predetermined parameters associated with the FOV portion. Examples of predetermined parameters include the center of the FOV portion, the radius or diameter of the FOV portion, and pixel locations associated with the FOV portion or the non-FOV portion. Pixels outside the area defined by the predetermined parameters may be identified as being in the non-FOV portion.

[0146] In some embodiments, parameters associated with the FOV portion of data received from a connected device can be determined once and reused when new image data is received from the device to identify the non-FOV portion of the data received from the connected device. For example, the center and diameter (or radius) of the FOV portion can be determined based on image data received from the connected device, and this center and diameter (or radius) can be used to identify the non-FOV portion in future image data received from the device. In some embodiments, one or more edge detection algorithms are used to detect the edges of the FOV portion, and this edge data can be used to identify the non-FOV portion of the image, or can be used to determine the center and diameter (or radius) of the FOV portion, which in turn can be used to identify the non-FOV portion of the image.

[0147] In some embodiments, one or more parameters associated with a FOV portion of image data received from a connected device are determined during an initialization phase of an imaging session with the connected device, wherein an image having a clear boundary between the FOV portion and a non-FOV portion is captured. This initialization phase can be, for example, a white balance phase, in which an imager is directed toward a white surface to allow the imager and / or associated light source to adjust one or more imaging parameters, such as gain and light intensity, based on the amount of light received from the white surface. During the white balance phase, the FOV portion of the image data generated by the imager (which is directed toward the white background) is relatively bright and, as such, has a high contrast with the black non-FOV portion, thereby providing a clear edge that can be easily detected using one or more edge detection algorithms.

[0148] In some embodiments, the medical imaging data processing hub receives an indication from a connected device that the connected device is in an initialization phase (such as a white balance phase). In response to receiving this indication, the medical imaging data processing hub performs an edge detection process to determine the location of a FOV portion of image data received from the connected device. The determined location (e.g., center, diameter, pixel location, etc.) of the FOV portion can be used to identify a non-FOV portion of subsequently received image data.

[0149] In some embodiments, the non-FOV portion may be identified by detecting the location of the perimeter of the FOV portion of each received image or frame. In some embodiments, the perimeter of the FOV portion in the first image data may be detected using, for example, one or more edge detection algorithms.

[0150] At step 206, cropped first image data is generated by removing at least a portion of the non-FOV portion of the first image data. The one or more non-FOV portions that are removed can be selected based on any suitable cropping criteria, including a desired aspect ratio of the cropped image or a predefined size of the cropped image. For example, the cropping criteria can specify that the cropped image should be square, and based on this criterion, non-FOV portions outside of the square encompassing the FOV portion can be removed, resulting in a square cropped image. Alternatively, the cropping criteria can specify an aspect ratio, and based on this criterion, rectangular non-FOV portions encompassing the FOV portion can be removed, resulting in a cropped image having the specified aspect ratio.

[0151] In some embodiments, one or more cropping criteria used in step 206 may be based on one or more properties of the connected display. For example, the size of the display may be used to determine the boundaries of the cropped image. The display size may be divided into display segments, and the size of the display segments may determine the boundaries of the cropped image. For example, in Figure 2 In the exemplary display of D, the first display segment 220 can be sized so that the image for display in the segment 220 can be cropped to the width of the FOV portion of the image for display in the segment 220, and the image for display in the second segment 222 can be cropped to the height of the FOV portion of the image for display in the segment 222.

[0152] In some embodiments, the medical imaging data processing hub may receive information about the display area (ie, pixel dimensions, spatial dimensions, etc.) from a connected display. In other embodiments, one or more display area parameters are user defined.

[0153] At step 208, a display feed is generated based on the cropped first image data. The display feed is transmitted to one or more connected displays, such as display 106 of system 100, via one or more display connections, and the cropped first image data is displayed on the display. In some embodiments, the cropped first image may be displayed in a first portion of the display, and additional information may be displayed in a second portion of the display. Examples of additional information that may be displayed include one or more images, videos, patient data and / or patient metadata, connected device status, metrics associated with imaging or any other connected device-related information, one or more charts, and the like. According to some embodiments, by cropping the first image data, the first portion of the display may occupy less space on the display, thereby increasing the amount of display space available for displaying additional information. According to some embodiments, the cropped image data may be shown in a portion of the screen having the same height and / or width as the portion that would have shown the uncropped image data, but the cropping of the image data allows the FOV portion to be larger on the display.

[0154] In some embodiments, image data may be received from multiple connected devices, and the image data from each connected device may be cropped according to method 200 discussed above. A display feed may be generated for displaying multiple cropped images on one or more connected displays. In some embodiments, additional information may be displayed along with the one or more cropped images. The additional information may be based on data received from the one or more connected devices. For example, an insufflator system connected to a medical imaging data processing hub may transmit an insufflation pressure reading to the system, and the pressure reading may be combined with a cropped endoscopic image in the display feed for display on a display alongside the cropped endoscopic image.

[0155] Figure 3C 2. The diagram shows a diagram with two cropped images 310 generated according to the method 200. Figure 3B An exemplary display 350 of FIG. The cropped image 310 includes Figure 3B FOV portion 302 of image 300, wherein part of the non-FOV portion is removed. As illustrated, cropping of the image allows the image to be displayed much larger. Image cropping can also provide space for displaying additional information. For example, in Figure 3C 3 , cropped image 310 occupies first portion 320 of display 350, second cropped image 326 occupies second portion 322 of display 350, and exemplary chart 328 occupies third portion 324 of the display screen. Thus, a medical imaging processing system such as hub 102 can maximize the utilization of a display screen for displaying medical imaging data and other information.

[0156] Figure 4is a block diagram of a medical imaging data processing hub 400 according to one embodiment, which may be used in a medical imaging system such as Figure 1 The system 100 is configured to process multiple data streams from connected medical devices (such as imaging devices) and generate an optimized display layout for displaying useful information to a user (such as a surgeon) during a medical procedure. The hub 400 includes one or more input connections 402 for receiving data from connected devices. The hub 400 includes one or more outputs 404 for connecting to one or more display devices. The hub 400 includes a main processing unit 406 that processes at least a portion of the data received from the connected devices and generates a display feed for output to one or more connected displays.

[0157] Hub 400 includes a main processing unit 406 for managing the processing of imaging data and generating display feeds using the processed data, a reconfigurable hardware processor 408 for processing imaging data streams, and an auxiliary processing unit 410 for providing software-based processing of imaging data and other data.

[0158] The reconfigurable hardware processor 408 can be a field-programmable gate array (FPGA) that can be reconfigured by loading a hardware logic file that defines the circuit connections in the FPGA. The reconfigurable hardware processor 408 provides low-latency and high-bandwidth processing of imaging data and can be repeatedly reconfigured to provide different processing of imaging data for different imaging sessions. By utilizing a reconfigurable hardware processor, the hub 400 can provide enhanced imaging data, such as video, in real time, with little or no delay between capturing the imaging and displaying the enhanced imaging on a connected display during an imaging session. In some embodiments, the reconfigurable hardware processor 408 is a reconfigurable GPU. The main processing unit 406 and the auxiliary processing unit 410 can each be any suitable processor or combination of processors, such as a central processing unit, a graphics processing unit, a microcontroller, an ASIC, or an FPGA, or any combination thereof.

[0159] The hub 400 includes memory 412, which can be local memory located within the hub 400 or remote memory at a remote location that the hub 400 can access via a network connection. One or more portions of the memory 412 can be local, and one or more portions can be remotely located. The memory 412 can include one or more configuration files 414 that specify the configuration of the hub 400 for different imaging sessions, one or more software programs for execution by the primary processing unit 406 and / or the auxiliary processing unit 410, and one or more hardware logic files 418 for reconfiguring the reconfigurable hardware processor 408. The primary processing unit 406 can access the configuration files 414 to determine the processing requirements specified in the configuration files, can load the hardware logic files 418 onto the reconfigurable hardware processor 408 defined by the configuration files, and can load the software programs 416 onto the auxiliary processing units 410 specified in the configuration files 414. Thus, the data stored in the memory 412 can be used to configure the hub 400 for different imaging sessions.

[0160] The reconfigurable hardware processor 408 is communicatively coupled to the main processing unit 406. The main processing unit 406 can send a video data stream to the reconfigurable hardware processor 408 for processing and can receive processed video back from the reconfigurable hardware processor 408 for inclusion in a display feed. The main processing unit can load hardware logic files into the reconfigurable hardware processor 408 for reconfiguration of the reconfigurable hardware processor 408.

[0161] The auxiliary processing unit 410 is communicatively coupled to the main processing unit 406. The main processing unit 406 can send data to the auxiliary processing unit 410 for processing and can receive processing results for inclusion in a display feed. The main processing unit 406 can load software on the auxiliary processing unit 410 for processing imaging data.

[0162] The hub 400 is configured to combine information received from multiple connected devices into a display feed for display on a connected display. Thus, multiple information sources can be displayed simultaneously on a connected display. The hub 400 is configured to stitch together information received from connected devices according to a predefined layout. For example, the hub 400 can generate a display feed in which a first video stream is displayed in a first display segment, a second video stream is displayed in a second display segment, and additional information such as data, alarms, device status, metrics, etc. is displayed in a third display segment.

[0163] According to some embodiments, the main processing unit 406 is responsible for receiving data from connected devices and stitching the data together into a composite display feed. The main processing unit 406 can utilize the reconfigurable hardware processor 408 and / or the auxiliary processing unit 410 to process the received data for enhancing the display of the data.

[0164] The main processing unit 406 combines the information sources into a display feed according to one or more predefined display layouts, which specify the imaging type of information to be displayed and the relative size and position of the imaging information and other information for display. The predefined display layouts can be associated with different types of imaging sessions, such as different types of surgical sessions or different types of surgical or other medical procedures. Different types of procedures can involve different types of imaging equipment and / or different types of imaging processing algorithms, and the predefined display layouts can specify the type of information to be displayed for a given procedure. Predefined display layouts can be associated with different practitioners based on the practitioners' preferences. For example, the same information can be displayed differently for two different practitioners performing the same procedure. The predefined display layouts can be stored in the memory 412 as a configuration data file 414.

[0165] Figure 5A An example of a first predefined display layout 500 is illustrated, and Figure 5B An example of a second predefined display layout 520 is illustrated. The first layout 500 includes three sections for three different sources—502, 504, and 506. The term source refers to the different data outputs generated by the hub 400. A source can include data received from one or more connected devices, enhanced data generated by processing data received from one or more connected devices, or any combination thereof. Multiple sources can include or be based on the same data received from connected devices. For example, a first source can include a video stream received from a connected device, and a second source can include the same video stream enhanced with information extracted from the video stream or information received from another connected device.

[0166] In addition to defining the sources to be displayed, predefined display layouts also define the relative positions of different sources on the display and the relative sizes of different sources on the display. For example, in first layout 500, first source 502 is located above second source 504 in the left half of the display, with sources 502 and 504 being equal in size. Third source 506 is located in the right half of the display and is larger than the first and second sources. In contrast, layout 520 includes six different sources of equal size, arranged in two rows and three columns. Layout 520 includes first source 502, second source 504, and third source 506, in addition to three other sources. First source 502 and second source 504 are in different positions relative to layout 500, and third source 506 is a different size relative to layout 500. Layout 500 can be associated with a first practitioner who configures layout 500 according to their preferences, and layout 520 can be associated with a second practitioner. Layouts 500 and 520 can be associated with different types of imaging sessions, such as different types of surgical procedures, or they can be associated with the same type of surgical procedures. In some embodiments, both layouts are used in the same imaging session.For example, layout 500 may define the layout of a first display of an imaging system, and layout 520 may define the layout of a second display of an imaging system.

[0167] Hub 400 can configure a display feed based on one or more parameters associated with an imaging session. Hub 400 can be used for a variety of different types of medical procedures and / or by a variety of different practitioners. As used herein, a medical procedure can refer to a single (e.g., surgical) procedure with various tasks performed by a practitioner (e.g., a surgeon), or more than one procedure performed during a single session with a patient (e.g., a single surgical session on a patient). For example, an orthopedic surgical session involving the performance of an orthopedic procedure (e.g., drilling and / or implantation of a medical device) along with an imaging procedure (e.g., visualization of tissue space and / or blood flow / tissue perfusion) can be a single medical procedure or multiple medical procedures. Different types of medical procedures can utilize different types of imagers and other equipment. A display layout designed for one type of procedure may not be suitable for another. Furthermore, different practitioners may have different preferences regarding what type of information should be displayed and how it should be displayed. Therefore, hub 400 can process received data and generate display feeds differently based on the specific requirements or preferences of each medical imaging session.

[0168] The hub 400 can configure the processing of input data and the generation of display feeds based on one or more predefined configurations. The predefined configurations can be associated with one or more parameters of an imaging session. Examples of imaging session parameters can include the user (e.g., a practitioner), the procedure type, information associated with one or more connected input devices, and information associated with one or more connected output devices.

[0169] Hub 400 may receive user input specifying one or more parameter values ​​(such as via Figure 1 The hub 400 may further comprise a user interface 114 of the embodiment of the present invention and may select a predefined configuration based on one or more parameter values. The hub 400 may reconfigure the processing of one or more inputs and the generation of one or more display feeds based on the selected predefined configuration.

[0170] The predefined configuration may define a predefined display layout and may also define one or more data processing algorithms. The algorithms may be implemented, for example, in the reconfigurable hardware processor 408 and / or in the auxiliary processing unit 410. In some embodiments, the reconfigurable hardware processor 408 may be reconfigured according to the predefined configuration to perform the imaging data processing specified by the predefined configuration.

[0171] As explained above, different sources can be included in different layouts. Different sources can be data from different connected devices, but can also be different information extracted from the same connected device. To facilitate the generation of different data depending on the connected device and layout preferences from one imaging session to the next, the hub 400 can automatically reconfigure the processing of data received from the connected devices according to the requirements specified in the configuration data associated with the imaging session.

[0172] The hub 400 can receive an indication of an imaging session associated with a predefined configuration and can automatically configure the processing of the input data and the generation of display feeds accordingly. For example, in preparation for a surgical session, a nurse can input one or more parameters associated with the surgical session into the hub 400, such as via a keyboard, mouse, touch screen, or other input device, and the hub 400 can configure itself accordingly, which can include reconfiguring the reconfigurable hardware processor by loading one or more hardware logic files stored in the memory 412, and loading one or more software programs or modules on the auxiliary processing unit 410. The parameters can include the type of surgical procedure to be performed and the practitioner performing the surgical procedure. One or more predefined layouts can be associated with the surgical procedure type and / or practitioner, and the hub 400 can reconfigure itself to generate a display feed according to the predefined layout.

[0173] Figure 6An example of a medical imaging processing hub (such as hub 400) configured for a first imaging session is illustrated. The configured hub 600 includes a main processor 602, a reconfigurable hardware processor 604, and an auxiliary processor 606. Hub 600 includes multiple data inputs 608, three of which are connected to three different devices (630, 632, and 634), which can be cameras, camera control units, instrument control units, lighting control units, insufflators, cauterizers, or any other device or system used during an imaging session that generates data related to the imaging session. Hub 600 includes multiple video outputs 610. In the illustrated embodiment, two displays 640 and 642 are connected to two of the video outputs 610.

[0174] In a first configuration, one or more algorithms have been loaded into the reconfigurable hardware processor 604 for processing imaging data received from the device 632. The reconfigurable hardware processor 604 processes the data received from the device 632 and transmits the processed data to the main processor 602. The reconfigurable hardware processor 604 may receive the imaging data directly from the input 608, or may receive the data via the main processor 602. In some embodiments, the main processor 602 may process the imaging data according to the above description. Figure 2 The principles described in method 200 may be used to crop the image data and provide the cropped image data to the reconfigurable hardware processor 604 and / or the auxiliary processor 606. This may be advantageous in reducing the amount of imaging data that requires processing.

[0175] The auxiliary processor 606 executes a software-based program for processing data from the third connected device 634. The auxiliary processor 606 can output the processing results to the main processor 602 via, for example, a video output 612 (such as a video output on a motherboard of a CPU).

[0176] The main processor 602 is responsible for combining the different data sources into a display feed for transmission to the connected display 640. The main processor stitches together the processed data from the reconfigurable hardware processor 604, from the auxiliary processor 606, and directly from the first connected device 630. For example, the main processor can generate a display feed that positions these three sources in different sections of the display.

[0177] In some embodiments, the main processor 602 can be configured to provide multiple different display streams. In the illustrated embodiment, the main processor 602 includes two compositors 614 and 616 that can generate two different display feeds based on configuration data stored in memory, for example. The first compositor 614 is configured to combine data from the first connected device, the reconfigurable hardware processor 604, and the auxiliary processor 606 into a first display feed for transmission to the display 640. The second compositor 616 receives data input (e.g., video input) from the device 634 and generates a second display feed for transmission to the display 642.

[0178] The compositor may combine data sources and generate display feeds differently from one imaging session to the next. The compositor's handling of data may change based on configuration data stored in memory. For example, Figure 6 The data handling illustrated in A may be defined by a first configuration file. A second configuration file may specify a change in the data handling by, for example, specifying that the first compositor 614 generates a display feed based on data from only the first device 630 for display on the first display 640, and the second compositor 616 generates a display feed based on data from the device 632 and the device 634 for display on the second display 642. Different configurations may be associated with different types of procedures and / or different practitioners, for example, to support different imaging sessions.

[0179] Figure 7 Illustrated is a method 700 for configuring a medical imaging processing system, such as the hub 400, in accordance with some embodiments. As further described below, the method 700 includes reconfiguring a reconfigurable hardware processor, such as the reconfigurable hardware processor 408 of the hub 400, according to predefined configuration data associated with a medical imaging session. The reconfigurable hardware processor is configured to implement an imaging data processing algorithm defined by the configuration data. Utilizing a reconfigurable hardware processor to implement an imaging processing algorithm allows the hardware processor configuration to be customized to process imaging data according to a specified algorithm, and allows different algorithms to be implemented for different imaging sessions having different imaging input and / or display requirements. Thus, the reconfigurable hardware processor can provide advantages over general-purpose processors running software-based algorithms, which may not be able to provide the lower latency and higher bandwidth that the reconfigurable processor can provide, which are important for providing real-time processing of video for display during medical procedures.

[0180] At step 702, a reconfigurable hardware processor, such as the reconfigurable hardware processor 408 of the hub 400, is configured into a first configuration for a first medical imaging session. The configuration of the hardware processor may be based on first configuration data stored in a memory. The first configuration data may define one or more medical imaging processing algorithms to be implemented by the reconfigurable hardware processor. Once configured into the first configuration, the reconfigurable hardware processor implements the one or more medical imaging processing algorithms as defined in the configuration data. The reconfigurable hardware processor may be configured by loading one or more hardware logic files from a memory (which may be processed by a second processor, such as the main processing unit 406 of the hub 400) onto the reconfigurable hardware processor.

[0181] The first imaging session may include performing one or more medical procedures, such as surgical procedures, on the patient. The first imaging session may begin when a nurse or other user initializes the medical imaging system for a medical procedure or a series of medical procedures on the patient. The first medical imaging session may be completed when the medical procedure or all of the series of medical procedures on the patient are completed, or may be completed when the first of the series of procedures on the patient is completed. As an example of this latter scenario, a first medical procedure, such as a first surgical procedure, may be completed on the patient, which completes the first imaging session, and may be followed by a second medical procedure, such as a different surgical procedure performed by the same or a different surgeon. The second medical procedure may include a second imaging session.

[0182] The reconfigurable hardware processor, in its first configuration, is configured to receive medical imaging data and process at least a portion of the data using a first imaging processing algorithm. In some embodiments, the first configuration may also include the ability to process the received data using one or more additional processing algorithms. The reconfigurable hardware processor in the first configuration may have the ability to process multiple different imaging data sets (e.g., generated by different devices) using the first imaging processing algorithm and / or the additional imaging processing algorithms. For example, the reconfigurable hardware processor in the first configuration may receive a first data set generated by a first connected device and process the first data set using the first medical imaging processing algorithm, and may also receive a second data set generated by a second connected device and process the second data set using the second medical imaging processing algorithm.

[0183] In some embodiments, a reconfigurable hardware processor is configured in response to input indicating a first medical imaging session. For example, a user, such as an operating room nurse, may provide information to the medical imaging processing system specifying parameters associated with the imaging session, such as the type of medical procedure and / or the identity of the medical practitioner. The one or more parameters may be associated with first configuration data, and the system may access the first configuration data and configure the reconfigurable hardware processor according to the specifications of the first configuration data. The user input indicating the first medical imaging session may include user selection of a profile. The profile may be associated with one or more types of medical procedures and may define data processing and display layouts customized for the one or more types of medical procedures. The types of medical procedures may include endoscopic medical procedures (such as enteroscopy, colonoscopy, sigmoidoscopy, proctoscopy, rhinoscopy, otoscopy, cystoscopy, colposcopy, arthroscopy, thoracoscopy, etc.) and surgical procedures (such as biopsy, carotid endarterectomy, cholecystectomy, coronary artery bypass surgery, skin grafting, hysterectomy, and mastectomy).

[0184] The profile can be a physician profile that defines the type of data the physician wants to see in a layout preferred by the physician. The profile can be a default profile that includes a predefined layout and predefined data processing. The default profile can be based on one or more parameters of the medical imaging system detected by the medical imaging processing system, such as the number and type of inputs to the processing system and the number and type of display outputs from the processing system. In some embodiments, the default profile can be based on one or more parameters detected from received image data, such as a radius or diameter of the FOV, which can be associated with the type of imager (e.g., endoscope size).

[0185] At step 704, the system receives first medical imaging data generated during a first medical imaging session. The first medical imaging data is received from one or more devices connected to one or more inputs of the system. For example, the first medical imaging data may be a series of video frames received from an imager, such as an endoscopic imager. The first medical imaging data may include data from multiple devices connected to the system, such as multiple video feeds from multiple imagers.

[0186] At step 706, enhanced first medical imaging data is generated at least in part by processing the first medical imaging data using a first medical imaging processing algorithm implemented by the reconfigurable hardware processor in the first configuration. The reconfigurable hardware processor processes at least a portion of the first medical imaging data using the first medical imaging processing algorithm and any other algorithms that the reconfigurable hardware processor is configured to implement, as defined by the first configuration data. For example, in the first configuration, the reconfigurable hardware processor may implement a smoke detection algorithm that detects portions of a received image associated with smoke in the field of view and enhances the received image to reduce the appearance of smoke.

[0187] Some or all of the first medical imaging data can be routed to the reconfigurable hardware processor via, for example, a main processor, such as main processing unit 406 of hub 400. The main processor can receive the first medical imaging data and can route the data to the reconfigurable processor based on first configuration data. The first configuration data can specify that data received from the connected device should be processed using at least a first medical imaging processing algorithm. Based on this requirement, the main processor can direct the data received from the connected device to the reconfigurable hardware processor. In some embodiments, the reconfigurable processor receives the first medical imaging data directly from an input connection to the connected device - that is, without first routing the data through one or more additional processing units.

[0188] The processing of data by the reconfigurable hardware processor can be based on processing by other processing units of the system. For example, the processing of first medical imaging data by a first medical imaging processing algorithm can be based on information received from a second processing unit. The second processing unit can analyze some or all of the first medical imaging data, and the results of the analysis can be used by the reconfigurable hardware processor in implementing the first medical imaging processing algorithm. For example, in the embodiment discussed above where a smoke detection algorithm is implemented in a reconfigurable processor, an auxiliary processing unit, such as auxiliary processing unit 410 of hub 400, can receive some or all of the first imaging data to determine whether smoke is present in the imaged field of view. Upon detecting smoke, the auxiliary processing unit can notify the reconfigurable hardware processor (directly or via another processing unit, such as main processing unit 406), and in response, the reconfigurable hardware processor can begin processing the first imaging data to reduce the contribution of smoke in the data.

[0189] At step 708, the enhanced first medical imaging data generated by the reconfigurable hardware processor is displayed for viewing during the first medical imaging session. For example, the first medical imaging session may include an endoscopic procedure involving the use of a cautery tool, and the enhanced first medical imaging data may be an enhancement of the video feed generated by the endoscopic camera, wherein the appearance of smoke generated by the cautery tool has been reduced. This enhanced imaging may be displayed to the surgeon in real time, allowing the surgeon to better visualize the surgical field.

[0190] In some embodiments, the enhanced first medical imaging data is received from the reconfigurable hardware processor by another processing unit, such as the main processing unit 406 of the hub 400. The main processor can generate one or more display feeds including the enhanced first medical imaging data. The main processor can generate the one or more display feeds based at least in part on the first configuration data. For example, the main processor can combine the enhanced first medical imaging data with additional information (such as additional imaging received from another connected device) for display in a different portion of the display defined by the first configuration data.

[0191] In some embodiments, the display feed includes the enhanced first medical imaging data combined with other data. For example, the display feed may include the enhanced first medical imaging data for display in a first portion of a connected display and may include additional information for display in a second portion of the connected display. In some embodiments, the system generates multiple display feeds having different display configurations for displaying the enhanced imaging data and provides different display feeds to different displays.

[0192] Figure 8Ais a block diagram of a medical imaging processing system 800, according to one embodiment, illustrating steps 702-708 of method 700. A reconfigurable hardware processor 804 is configured to process imaging data received from a white light imager 810 via a first input port 808 using a first imaging processing algorithm, such as a smoke detection and removal algorithm. In response to user input associated with a first imaging session, the main processor 802 accesses first configuration data stored in a memory 812. Based on the specifications in the first configuration data, the main processor 802 reconfigures the reconfigurable hardware processor 804 by loading a hardware logic configuration file for the smoke reduction algorithm. Furthermore, the main processor 802 loads a smoke detection software program or module from the memory 812 to the auxiliary processor 806. The auxiliary processor 806, running the smoke detection software program or module, can detect the presence of smoke in the received imaging data and can instruct the reconfigurable hardware processor 804 to process the imaging data to reduce the effects of smoke in the imaging data. Before smoke is detected by the auxiliary processor 806, the reconfigurable hardware processor can simply pass the imaging data through for display without first processing the data for smoke removal. The main processor can provide one or more portions (such as one or more frames) of the received imaging data to the auxiliary processor 806, such as on a periodic basis, for detecting smoke and triggering smoke removal processing by the reconfigurable hardware processor 804. The system 800 outputs a display feed to the display 816 that includes the imaging data received from the white light imager 810, which has been enhanced by removing the contribution from smoke when smoke is detected in the imaging data.

[0193] Returning to method 700 , at step 710 , the reconfigurable hardware processor is reconfigured to a second configuration for a second medical imaging session based on second configuration data stored in the memory. The second configuration implements at least one medical imaging processing algorithm not implemented in the first configuration.

[0194] In some embodiments, the reconfigurable hardware processor is reconfigured in response to input indicating a second medical imaging session. For example, a user, such as an operating room nurse, may provide information to the medical imaging processing system specifying parameters associated with the second imaging session, such as the type of medical procedure and / or the identity of the practitioner. The one or more parameters may be associated with second configuration data, and the system may access the second configuration data and configure the reconfigurable hardware processor according to the specifications of the second configuration data. As discussed above, the user input indicating the second medical imaging session may include a user selection of a profile. Depending on the selected profile, the second imaging session may be associated with the same practitioner as the first imaging session—for example, if the same practitioner is transitioning from one type of medical procedure to another that may require a different display layout (such as due to different equipment connected to the medical imaging processing system or different enhancement algorithms). The second imaging session may be associated with the same type of medical procedure but with a different practitioner. For example, a first surgeon may perform one type of surgical procedure (e.g., a cholecystectomy) on a first patient in a first imaging session, and a second surgeon may perform the same type of surgical procedure (e.g., a cholecystectomy) on a second patient (e.g., later that same day or the following day).

[0195] A first imaging session may have been completed (e.g., the surgery or procedure associated with the first imaging session has been completed), and the imaging system may be configured for use in a subsequent second imaging session. The second imaging session may involve one or more different types of procedures and / or may include one or more different users, for which different imaging processing may be beneficial. Thus, the second configuration implements one or more image processing algorithms that were not implemented in the first imaging session. The reconfigurable hardware processor is reconfigured to implement the one or more image processing algorithms required for the second imaging session as defined by the second configuration data.

[0196] In some embodiments, the second imaging session may be a second surgical session for which the imaging system will be used. After the first imaging session is completed, the operating room may be set up for the second surgical session. The second surgical session may involve a different type of surgical procedure, a different practitioner, a different patient, etc. During the setup for the second surgical session, the imaging processing system may receive input indicative of the second surgical session. This input may be, for example, a selection of a surgical procedure type or a selection of a profile (e.g., a practitioner profile) made via a user interface to the imaging processing system. Based on this selection, the system may automatically reconfigure the reconfigurable processor based on configuration data associated with the second surgical session.

[0197] At step 712, the medical imaging processing system receives second medical imaging data generated during the second medical imaging session. The second medical imaging data may be received from the same one or more connected devices as the first medical imaging data, or from a different one or more connected devices.

[0198] At step 714, enhanced second medical imaging data is generated at least in part by processing the second medical imaging data using a second medical imaging processing algorithm implemented in the second configuration of the reconfigurable hardware processor. The reconfigurable hardware processor processes at least a portion of the first medical imaging data using the second medical imaging processing algorithm (which is not implemented in the first configuration) and any other algorithms that the reconfigurable hardware processor is configured to implement, which may or may not be implemented in the first configuration, as defined by the second configuration data. For example, in the second configuration, the reconfigurable hardware processor may implement an algorithm that processes a fluorescence image (e.g., a video frame) to determine one or more characteristics of blood flow through tissue (such as tissue perfusion, vessel location, blood flow rate or velocity, size of the imaged tissue, or any combination thereof), and generates enhanced imaging that modifies the fluorescence image (modifies image coloring, overlays data on the image, overlays contours on the image, etc.) based on the determined characteristics.

[0199] As in the first configuration described above, some or all of the second medical imaging data can be routed to the reconfigurable hardware processor via, for example, a main processor (such as main processing unit 406 of hub 400). The main processor can receive the second medical imaging data and can route the data to the reconfigurable processor based on second configuration data. The second configuration data can specify that data received from the connected device should be processed using at least a second medical imaging processing algorithm. Based on this requirement, the main processor can direct the data received from the connected device to the reconfigurable hardware processor. In some embodiments, the reconfigurable processor receives the first medical imaging data directly from the input, i.e., without first routing the data through one or more additional processing units.

[0200] At step 716, the enhanced second medical imaging data generated by the reconfigurable hardware processor is displayed for viewing during the second medical imaging session. Display of the enhanced second medical imaging data can assist a medical practitioner, such as a surgeon, during one or more procedures performed during the second medical imaging session. By leveraging the low latency and high bandwidth of the reconfigurable processor, the enhanced second medical imaging data can be displayed in real time.

[0201] In some embodiments, the enhanced second medical imaging data may be transferred from the reconfigurable hardware processor to another processing unit, such as Figure 4The main processing unit 406 of the reconfigurable hardware processor can generate a display feed including the enhanced second medical imaging data. The display feed can be transmitted by the main processor to one or more connected displays. In some embodiments, the reconfigurable hardware processor can transmit the enhanced second medical imaging data directly to an output connection having one or more connected displays.

[0202] Figure 8A is a block diagram of a medical imaging processing system 800, according to one embodiment, illustrating steps 710-716 of method 700. A reconfigurable hardware processor 804 is reconfigured to process imaging data received from a white light imager 810 and a fluorescence imager 818 (which can be part of the same imaging system and can be received on the same or different input ports) using a second imaging processing algorithm. The second imaging processing algorithm analyzes the fluorescence imaging to characterize portions of tissue based on, for example, the health of the tissue, the degree of blood flow in the tissue, or the degree of perfusion in the tissue, and overlays the characterization on the white light imaging. In response to user input associated with the second imaging session (e.g., an input indicating a surgical session for a new patient or an input indicating a new procedure for the same patient as the first imaging session), the main processor 802 accesses second configuration data stored in memory 812 and, based on the specifications in the first configuration data, reconfigures the reconfigurable hardware processor 804 by loading a hardware logic configuration file for the tissue characterization algorithm. Furthermore, the main processor 802 loads a reference marker software program or module from memory 812 to the auxiliary processor 806. Auxiliary processor 806, running a reference marker program or module, can determine, for example, the locations of maximum and / or minimum perfusion in the fluorescence imaging data. The reference markers generated by auxiliary processor 806 are added to the overlay generated by the reconfigurable hardware processor (this can be done by reconfigurable hardware processor 804, by main processor 802, or by a different processor of the system). The resulting enhanced imaging data is output to display 816 for visualization during the second imaging session.

[0203] According to some embodiments, a tissue characterization algorithm implemented in a reconfigurable hardware processor can provide an enhanced visual representation of a subject's tissue that may be more accurate in data representation and intuitive for clinicians to use in their clinical decision-making. The generated enhanced visual representation of tissue can be adapted to various types of tissue (e.g., various wounds, including chronic, acute, pressure ulcers, cancerous tissue) and can provide a framework for automatically classifying tissue (e.g., wound tissue, cancerous tissue) and / or predicting clinical outcomes (e.g., healing timeline of wound tissue, healing of cancerous tissue).

[0204] Tissue characterization algorithms can utilize machine learning or deep learning. Machine learning-based methods and systems facilitate solving problems for which there are no algorithmic solutions or for which the solutions are too complex to be found. Due to the complex nature of physiological processes occurring within the human body, medical diagnosis and tissue characterization based on tissue imaging are particularly well-suited tasks for machine learning algorithms. Machine learning can be used to discover medically relevant features and patterns in large datasets, helping clinicians make medical diagnoses more accurately, quickly, and consistently, regardless of their experience. In some embodiments, the tissue characterization algorithm includes identifying one or more attributes of data relevant to the clinical characterization of tissue, and classifying the data into a plurality of clusters based on the one or more attributes of the data, such that data within the same cluster is more similar to each other than data in different clusters, wherein the clusters characterize the tissue. In some variations, the algorithm may further include associating a corresponding cluster with each of a plurality of subregions in a time series of images, such as, for example, fluorescence images, and generating a subject space (cluster) map based on the associated clusters of the plurality of subregions in the subject time series of fluorescence images. The algorithm may further include receiving a plurality of subject space maps and receiving metadata associated with each subject space map, storing each subject space map and its associated clinical data in a record of a database, and using the records of the database as input to a supervised machine learning algorithm for generating a prediction model. The prediction model may be used to predict clinical data associated with the subject time series of fluorescence images of the subject.

[0205] Figure 9A and 9B Illustrated is a graphical user interface for configuring a medical imaging processing system such as hub 400 for a new imaging session. The user interface may be provided on, for example, a tablet computer connected to the system or on a touch screen of the system. Figure 9A The user interface 900 enables a user to configure an image processing system by selecting a specialty 902, a program 904, and / or a practitioner 906. Each selection can be associated with a different configuration, or a combination of selections can be associated with a configuration. For example, each practitioner selection can be associated with a different configuration that has been previously specified by the practitioner, while selecting a configuration may require selection of both a specialty and a program. The configuration can be stored locally in the system's memory or remotely, for example, in a hospital information system that is accessed, for example, via a network connection.

[0206] Figure 9BA user interface 910 for defining data sources and data source layouts is illustrated. Two display layouts (912 and 914) are associated with the illustrated configuration. Each display layout defines the data source, as well as the size and position of the data source. The first display layout 912 includes three different sources. As described above, the source can define the type of data displayed, which can be based on both the system generating the data and the type of processing performed on the data, for example, by a reconfigurable processor and / or other system modules. Similarly, different sources can be based on data from the same imaging system or other device. For example, source 1 can be a still image from an incoming video stream (e.g., as selected via voice commands from a practitioner), and source 3 can be a video stream. User interface 910 can enable the user to select, position, and size different sources. For example, the user can select available sources from a drop-down list that specifies, for example, all sources that the system can generate given input to the system or all sources that the system can generate. The user can drag source icons around the screen to reposition the sources and can resize the sources using, for example, gestures, mouse input, keyboard input, or any other suitable input.

[0207] Once the user completes the selection of a configuration profile, the medical imaging processing system may automatically configure itself according to the requirements defined in the selected configuration profile, according to the methods described above.

[0208] Systems for collecting, enhancing, and displaying medical imaging data (such as Figure 1 The system 100 may include one or more imaging systems for acquiring a time series of tissue images (eg, a time series of fluorescence images, a time series of white light images, etc.). In some embodiments, the imaging system is a fluorescence imaging system. Figure 10FIG2 is a schematic illustration of a fluorescence imaging system 1010 according to one embodiment. Fluorescence imaging system 1010 includes a light source 1012 for illuminating tissue of a subject to induce fluorescence emission from a fluorescent imaging agent 1014 in the subject's tissue (e.g., in blood, urine, lymph, cerebrospinal fluid, or other bodily fluid or tissue); an image acquisition component 1016 configured to generate a time series of fluorescence images and / or a subject time series based on the fluorescence emission; and a processor component 1018 configured to process the generated time series of fluorescence images / subject time series according to any variation of the methods described herein. Processor component 1018 may include a memory 1068 having instructions thereon, a processor module 1062 configured to execute the instructions on memory 1068 to process the time series of fluorescence images and / or the subject time series, and a data storage module 1064 for storing unprocessed and / or processed time series of fluorescence images and / or the subject time series. In some variations, the memory 1068 and the data storage module 1064 may be embodied in the same storage medium, while in other variations, the memory 1068 and the data storage module 1064 may be embodied in different storage media. The system 1010 may further include a communication module 1066 for transmitting images and other data (such as some or all of the time series of fluorescence images / subject time series or other input data, spatial maps, subject spatial maps, and / or tissue values ​​(quantizers)) to an imaging data processing hub, such as a computer program product, in accordance with the systems and methods discussed above. Figure 1 The imaging data processing hub 102.

[0209] In some variations, light source 1012 includes, for example, an illumination module 1020. Illumination module 1020 may include a fluorescence excitation source arranged to generate excitation light having a suitable intensity and a suitable wavelength for exciting fluorescent imaging agent 1014. Figure 11 As shown in FIG, the illumination module 1020 may include a laser diode 1022 (e.g., which may include, for example, one or more fiber-coupled diode lasers) arranged to provide excitation light to excite a fluorescent imaging agent (not shown) in the subject's tissue. Examples of other excitation light sources that may be used in various embodiments include one or more LEDs, arc lamps, or other light-emitting technologies of sufficient intensity and appropriate wavelength to excite the fluorescent imaging agent in the tissue. For example, one or more 793 nm conduction-cooled single-rod fiber-coupled laser diode modules from DILAS Diode Laser Co., Germany, may be used to excite a fluorescent imaging agent in blood, where the fluorescent imaging agent is a fluorescent dye with near-infrared excitation and emission characteristics.

[0210] In some variations, the light output from the light source 1012 may be projected through one or more optical elements to shape and direct the output being used to illuminate the tissue region of interest. The optical elements may include one or more lenses, light guides, and / or diffractive elements to ensure a flat field across substantially the entire field of view of the image acquisition assembly 1016. The fluorescence excitation source may be selected to emit at a wavelength close to the absorption maximum of the fluorescent imaging agent 1014 (e.g., indocyanine green (ICG), etc.). For example, Figure 11 As shown in FIG, the output 1024 from the laser diode 1022 may pass through one or more focusing lenses 1026 and then through a homogenizing light pipe 1028, such as, for example, those commonly available from Newport Corporation. Finally, the light may pass through an optical diffraction element 1032 (i.e., one or more optical diffusers), such as, for example, a ground glass diffraction element also available from Newport Corporation. Power to the laser diode 1022 may be provided, for example, by a high-current laser driver (such as those available from Lumina Power, Inc., USA). During the image acquisition process, the laser may optionally be operated in a pulsed mode. An optical sensor, such as a solid-state photodiode 1030, may be incorporated into the illumination module 1020 and may sample the illumination intensity generated by the illumination module 1020 via scattered or diffuse reflections from various optical elements. In some variations, an additional illumination source may be used to provide guidance when aligning and positioning the module over the region of interest.

[0211] Reference again Figure 10 In some variations, the image acquisition component 1016 may be a component of the fluorescence imaging system 1010 that is configured to acquire a time series of fluorescence images and / or a time series of the subject from fluorescence emissions from the fluorescent imaging agent 1014. The image acquisition component 1016 may include a camera module 1040. Figure 12 As shown in FIG, camera module 1040 can acquire an image of fluorescent emissions 1042 from a fluorescent imaging agent in tissue by collecting the fluorescent emissions using imaging optics (e.g., 1046a, 1046b, 1048, and 1050) and focusing the fluorescent emissions onto image sensor assembly 1044. Image sensor assembly 1044 can include at least one 2D solid-state image sensor. The solid-state image sensor can be a charge-coupled device (CCD), a CMOS sensor, a CID, or similar 2D sensor technology. The charge generated by the optical signal converted by image sensor assembly 1044 is converted into an electrical video signal, including both digital and analog video signals, by appropriate readout and amplification electronics in camera module 1040.

[0212] According to an exemplary variation of a fluorescence imaging system, a light source can provide an excitation wavelength of approximately 800 nm + / - 10 nm, and an image acquisition component can use an emission wavelength > 820 nm in conjunction with NIR-compatible optics for, for example, ICG fluorescence imaging. In an exemplary embodiment, the NIR-compatible optics can include a CCD monochrome image sensor with a GigE standard interface and a lens that is compatible with the sensor in terms of optical format and mounting format (e.g., C / CS mount).

[0213] In some variations, processor module 1062 comprises any computer or computing component, such as, for example, a tablet computer, a laptop computer, a desktop computer, a networked computer, or a dedicated stand-alone microprocessor. For example, processor module 1062 may include one or more central processing units (CPUs). In an exemplary embodiment, processor module 1062 is a quad-core 2.5 GHz processor having four CPUs, each of which is a microprocessor, such as a 64-bit microprocessor (e.g., as sold as an Intel Core i3, i5, or i7, or in the AMD Core FX series). However, in other embodiments, processor module 1062 may be any suitable processor having any suitable number of CPUs and / or other suitable clock speed.

[0214] The input to the processor module 1062 may be taken from, for example Figure 12 The image sensor 1044 of the camera module 1040 shown in FIG. Figure 11 The solid-state photodiode 1030 in the illumination module 1020 in the laser diode 1030, and / or any external control hardware (such as a foot switch or remote control). The output is provided to the laser diode driver and optical alignment aid. Figure 10 As shown in , in some variations, processor assembly 1018 may include a data storage module 1064 capable of saving a time series of images / subject time series, or data representing them, or other input data, to a tangible, non-transitory computer-readable medium, such as, for example, internal memory (e.g., a hard drive or flash memory), to enable recording and processing of acquired data. In some variations, processor module 1062 may include an internal clock to enable control of various components and ensure proper timing of illumination and sensor shuttering. In some variations, processor module 1062 may also provide a graphical display of user input and output. The fluorescence imaging system may optionally be configured with a communication unit 1066, such as a wired or wireless network connection or a video output connection, for transmitting the time series of fluorescence images as they are acquired or for playback after recording. Communication unit 1066 may additionally or alternatively transmit processed data, such as spatial maps, subject spatial maps, and / or tissue values.

[0215] exist Figure 10-12 In operation of the exemplary system described in

[10] , a subject is positioned relative to fluorescence imaging system 1010 such that a region of interest (e.g., a target tissue region) is located beneath light source 1012 and image acquisition assembly 1016, such that illumination module 1020 of light source 1012 produces a substantially uniform illumination field across substantially the entire region of interest. In some variations, images of the region of interest may be acquired for background subtraction purposes prior to administering fluorescent imaging agent 1014 to the subject. To acquire fluorescence images / subject fluorescence images, an operator of fluorescence imaging system 1010 may initiate acquisition of a time series of fluorescence images / subject time series by depressing a remote switch or foot control, or via a keyboard (not shown) connected to processor assembly 1018. As a result, light source 1012 is turned on, and processor assembly 1018 begins recording fluorescence image data / subject fluorescence image data provided by image acquisition assembly 1016. When operating in the pulsed mode of this embodiment, image sensor 1044 in camera module 1040 is synchronized to collect fluorescence emissions following laser pulses generated by diode laser 822 in illumination module 1020. In this way, the maximum fluorescence emission intensity is recorded and the signal-to-noise ratio is optimized. In this embodiment, the fluorescent imaging agent 1014 is administered to the subject and delivered to the region of interest via arterial flow. For example, shortly after the administration of the fluorescent imaging agent 1014, the acquisition of a time series of fluorescence images / subject time series is initiated, and a time series of fluorescence images is acquired from substantially the entire region of interest throughout the entry of the fluorescent imaging agent 1014. The fluorescence emission from the region of interest is collected by the collection optics of the camera module 1040. The remaining ambient and reflected excitation light is collected by subsequent optical elements in the camera module 1040 (e.g., Figure 12 The optical element 1050 in FIG. 1 , which may be a filter, attenuates the fluorescence emission so that the fluorescence emission can be collected by the image sensor assembly 1044 with minimal interference from light from other sources.

[0216] In some variations, following acquisition or generation of the time series of fluorescence images / subject time series, processor component 1018 (e.g., processor module 1062 or other processor) may then be activated to execute instructions stored on memory 1068 and process the imaging data prior to transmission to an imaging data processing system (e.g., hub 102 of system 100). System 1010 may transmit the spatial map / subject space map and / or any clinical correlations or diagnoses derived therefrom, or both, via connection 1066 for display to a user in a composite display feed as, for example, a grayscale or false color image, and / or for storage for subsequent use.

[0217] Figure 13 Shown Figure 1 The endoscopic surgical cart embodiment of the system 100 is shown. The cart 10 can be used, for example, in an operating room for endoscopic imaging and display during endoscopic procedures. The cart 10 includes an imaging system such as Figure 10 FIG10 is a fluoroscopic imaging system 1010 of the present invention. The imaging system includes a mirror assembly 11 that can be utilized in endoscopic procedures. The mirror assembly 11 incorporates an endoscope or scope 12, which is coupled to a camera head 16 via a coupler 13 located at the distal end of the camera head 16. A light source 14 provides light to the scope via a light guide 26, such as a fiber optic cable. The camera head 16 is coupled to a camera control unit (CCU) 18 via a cable 15. The CCU 18 is preferably connected to and in communication with the light source 14. Operation of the camera 16 is controlled in part by the CCU 18. The cable 15 transmits video image data from the camera head 16 to the CCU 18 and transmits various control signals bidirectionally between the camera head 16 and the CCU 18. In one embodiment, the image data output by the camera head 16 is digital.

[0218] A control or switch arrangement 17 is provided on the camera head 16 and allows the user to manually control various functions of the cart 10. Voice commands can be input into a microphone 25 mounted on a headset 27 worn by the surgeon and coupled to a voice control unit 23. The cart 10 may include a handheld control device 21, such as a tablet computer or PDA with a touch screen user interface, which may be coupled to the cart 10 as an additional control interface. The cart 10 also includes an imaging data processing hub 31, such as a Figure 1 The center 102 or Figure 4 The imaging data processing hub 400 is coupled to the imaging system via one or more cable connections for receiving images and / or video from the imaging system, processing the images and / or video, and generating a display feed for display on the display 20 according to the methods described herein. The imaging data processing hub can receive user input via a voice control unit and / or through a handheld control device.

[0219] The cart 10 may include one or more additional devices 33, such as an imaging recording device or a surgical tool control device, which may be coupled to the imaging data processing hub. The imaging data processing hub 31 may receive information from one or more additional devices 33, such as device alerts, device status, and device settings. In some embodiments, the additional device 33 is a video recorder, and the imaging data processing hub 31 may transmit one or more display feeds to the video recorder for recording.

[0220] A tangible, non-transitory computer-readable medium having computer-executable (readable) program code embedded thereon can provide instructions that, when executed, cause one or more processors to perform one or more of the methods described herein. The program code can be written in any suitable programming language and delivered to the processor in a variety of forms, including, for example, but not limited to, information permanently stored on a non-writable storage medium (e.g., a read-only memory device such as a ROM, CD-ROM disk, etc.), information revocably stored on a writable storage medium (e.g., a hard drive, etc.), or information transmitted to the processor via a communication medium, such as a local area network, a public network (such as the Internet), or any type of medium suitable for storing electronic instructions. When carrying computer-readable instructions that implement various embodiments of the methods described herein, such computer-readable media represent examples of the various embodiments. In various embodiments, tangible, non-transitory computer-readable media includes all computer-readable media, and the scope of the present invention is limited to computer-readable media where the media is both tangible and non-transitory.

[0221] A kit can include any portion of the systems described herein and a fluorescent imaging agent, such as, for example, a fluorescent dye (such as ICG) or any suitable fluorescent imaging agent. In another aspect, the kit can include a tangible, non-transitory computer-readable medium having embedded thereon computer-executable (readable) program code that, when executed, can provide instructions for causing one or more processors to perform one or more of the methods described herein for characterizing tissue and / or predicting clinical data. The kit can include instructions for using at least some of its components (e.g., for using the fluorescent imaging agent, for installing the computer-executable (readable) program code having embedded thereon the instructions, etc.). In yet another aspect, a fluorescent imaging agent, such as, for example, a fluorescent dye, is provided for use in the methods and systems described herein. In further variations, the kit can include any portion of the systems described herein or the entire system, and a fluorescent agent, such as, for example, a fluorescent dye (such as ICG) or any other suitable fluorescent agent or combination of fluorescent agents.

[0222] Example imaging agents for use in generating imaging data

[0223] According to some embodiments, in fluorescence medical imaging applications, the imaging agent is a fluorescent imaging agent, such as, for example, an ICG dye. When ICG is administered to a subject, it binds to blood proteins and circulates with the blood in the tissues. The fluorescent imaging agent (e.g., ICG) can be administered to the subject as a bolus injection (e.g., into a vein or artery) at a concentration suitable for imaging, such that the bolus circulates through the vasculature and passes through the microvasculature. In other implementations in which multiple fluorescent imaging agents are used, such agents can be administered simultaneously, for example, in a single bolus, or sequentially in separate boluses. In some embodiments, the fluorescent imaging agent can be administered via a catheter. In certain embodiments, the fluorescent imaging agent can be administered less than one hour before measuring the intensity of the signal generated by the fluorescent imaging agent. For example, the fluorescent imaging agent can be administered to the subject less than 30 minutes before the measurement. In yet other embodiments, the fluorescent imaging agent can be administered at least 30 seconds before the measurement is performed. In yet other embodiments, the fluorescent imaging agent can be administered simultaneously with the measurement.

[0224] According to some embodiments, the fluorescent imaging agent may be administered at various concentrations to achieve a desired circulating concentration in the blood. For example, in embodiments where the fluorescent imaging agent is ICG, it may be administered at a concentration of about 2.5 mg / mL to achieve a circulating concentration of about 5 mg / mL in the blood. to about 10 In various embodiments, the upper concentration limit for administering the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes clinically toxic in circulating blood, and the lower concentration limit is the instrument limit for collecting signal intensity data generated by the fluorescent imaging agent circulating in the blood to detect the fluorescent imaging agent. In various other embodiments, the upper concentration limit for administering the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes self-quenched. For example, the circulating concentration of ICG can be from about 2 to about 10 Thus, in one aspect, according to various embodiments, the method includes the steps of administering an imaging agent (e.g., a fluorescent imaging agent) to a subject and acquiring signal intensity data (e.g., a video) prior to processing the signal intensity data. In another aspect, the method excludes any step of administering an imaging agent to a subject.

[0225] According to some embodiments, a suitable fluorescent imaging agent for use in fluorescence imaging applications to generate fluorescence image data is an imaging agent that circulates with blood (e.g., a fluorescent dye that can circulate with, for example, blood components such as lipoproteins or serum plasma) and through the vasculature of tissues (i.e., macrovessels and microvessels), and generates a signal intensity from the imaging agent when exposed to appropriate light energy (e.g., excitation light energy or absorption light energy). In various embodiments, the fluorescent imaging agent comprises a fluorescent dye, an analog thereof, a derivative thereof, or a combination thereof. Fluorescent dyes include any non-toxic fluorescent dye. In certain embodiments, the fluorescent dye optimally emits fluorescence in the NIR spectrum. In certain embodiments, the fluorescent dye is or comprises a tricarbocyanine dye. In certain instances, the fluorescent dye is or comprises ICG, methylene blue, or a combination thereof. In other embodiments, the fluorescent dye is or comprises fluorescein isothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin, o-phthalaldehyde, fluorescamine, rose bengal, trypan blue, fluorogold, or a combination thereof, all of which are excitable using an excitation light wavelength appropriate for each dye. In some embodiments, analogs or derivatives of fluorescent dyes can be used. For example, fluorescent dye analogs or derivatives include fluorescent dyes that have been chemically modified but still retain their ability to fluoresce when exposed to light energy of an appropriate wavelength.

[0226] In various embodiments, the fluorescent imaging agent can be provided as a lyophilized powder, solid, or liquid. In certain embodiments, the fluorescent imaging agent can be provided in a vial (e.g., a sterile vial), which allows for reconstitution to the appropriate concentration by administering a sterile fluid with a sterile syringe. Reconstitution can be performed using any suitable carrier or diluent. For example, the fluorescent imaging agent can be reconstituted with an aqueous diluent immediately prior to administration. In various embodiments, any diluent or carrier that maintains the fluorescent imaging agent in solution can be used. As an example, ICG can be reconstituted with water. In some embodiments, once the fluorescent imaging agent is reconstituted, it can be mixed with additional diluents and carriers. In some embodiments, the fluorescent imaging agent can be conjugated to another molecule (such as a protein, peptide, amino acid, synthetic polymer, or sugar), for example, to enhance solubility, stability, imaging properties, or a combination thereof. Additional buffers can optionally be added, including Tris, HCl, NaOH, phosphate buffer, and / or HEPES.

[0227] It will be appreciated by those skilled in the art that, although fluorescent imaging agents are described in detail above, other imaging agents may also be used in conjunction with the systems, methods, and techniques described herein, depending on the optical imaging modality. Such fluorescent agents may be administered to body fluids (e.g., lymph, cerebrospinal fluid) or body tissues.

[0228] In some variations, fluorescent imaging agents used in conjunction with the methods, systems, and kits described herein can be used for blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof, which can be performed during invasive, minimally invasive, non-invasive surgical procedures, or a combination thereof. Examples of invasive surgical procedures that may involve blood flow and tissue perfusion include cardiac-related surgical procedures (e.g., CABG, with or without extracorporeal circulation) or reconstructive surgical procedures. Examples of non-invasive or minimally invasive surgical procedures include wound treatment and / or management (e.g., chronic wounds, such as pressure sores). In this regard, for example, changes in wound size (e.g., diameter, area) or changes in tissue perfusion in and / or around the wound can be tracked over time using the methods and systems. Examples of lymphatic imaging include identifying one or more lymph nodes, lymph node drainage, lymphatic mapping, or a combination thereof. In some variations, such lymphatic imaging may be associated with the female reproductive system (e.g., uterus, cervix, vulva).

[0229] In a variation related to cardiac applications, imaging agent(s) (e.g., ICG alone or in combination with another imaging agent) can be injected intravenously, for example, through a central venous line, bypass pump, and / or cardioplegia line, to circulate and / or perfuse the coronary vasculature, microvasculature, and / or graft. ICG can be administered as a diluted ICG / blood / saline solution down the graft vessel, such that the final concentration of ICG in the coronary arteries is approximately the same as or lower than that produced by injecting approximately 2.5 mg (i.e., 1 ml of 2.5 mg / ml) into a central line or bypass pump. ICG can be prepared by dissolving, for example, 25 mg of solid solution in 10 ml of a sterile aqueous solvent, which can be provided by the manufacturer with ICG. One milliliter of ICG solution can be mixed with 500 ml of sterile saline (e.g., by injecting 1 ml of ICG into a 500 ml saline bag). Thirty milliliters of the diluted ICG / saline solution can be added to 10 ml of the subject's blood, which can be obtained aseptically from the central arterial line or bypass pump. ICG in the blood binds to plasma proteins and helps prevent leakage from the blood vessels. The mixing of ICG with blood can be performed using standard aseptic techniques within the field of aseptic surgery. 10 ml of the ICG / saline / blood mixture can be applied to each graft. Rather than using a needle to inject ICG through the wall of the graft, ICG can be applied with the aid of a syringe attached to the (open) proximal end of the graft. When the graft has been harvested, the surgeon conventionally attaches an adapter to the proximal end of the graft so that they can attach a syringe filled with saline, seal the distal end of the graft, and inject saline down the graft, pressurizing the graft and thus assessing the integrity of the conduit (regarding leaks, side branches, etc.) before performing the first anastomosis. In other variations, the methods, dosages, or combinations thereof described herein in conjunction with cardiac imaging can be used in any vascular and / or tissue perfusion imaging application.

[0230] Lymphatic mapping is an essential component of effective surgical staging for cancers that spread through the lymphatic system, such as breast, gastric, and gynecological cancers. Removal of multiple lymph nodes from a specific lymph node basin can lead to serious complications, including acute or chronic lymphedema, paresthesia, and / or seroma formation. Indeed, if the sentinel lymph node is negative for metastases, the surrounding lymph nodes are most likely negative as well. For example, during breast cancer surgery, identification of tumor-draining lymph nodes (LNs) has become a crucial step in staging cancers that spread through the lymphatic system. LN mapping involves the use of dyes and / or radioactive tracers to identify LNs for biopsy or resection and subsequent pathological evaluation for metastases. The goal of lymph node dissection during surgical staging is to identify and remove LNs at high risk for local cancer spread. Sentinel lymph node (SLN) mapping has emerged as an effective surgical strategy in breast cancer treatment. It is generally based on the concept that metastases (spread of cancer to the axillary LNs), if present, should be located in the SLN, which is defined in the art as the first LN or group of lymph nodes to which cancer cells are most likely to spread from the primary tumor. If the SLN is negative for metastases, then the surrounding secondary and tertiary LNs should also be negative. The main benefit of SLN mapping is to reduce the number of subjects who undergo traditional partial or complete lymph node resection and, therefore, the number of subjects who suffer associated morbidities such as lymphedema and lymphocele.

[0231] The current standard of care for SLN mapping involves the injection of a tracer that identifies the lymphatic drainage pathway from the primary tumor. The tracer used may be a radioactive isotope (such as technetium-99 or Tc-99m) for intraoperative localization using a gamma probe. Radioactive tracer technology (known as scintigraphy) is limited to hospitals where access to the radioisotope requires the involvement of a nuclear physician and does not provide real-time visual guidance. A colored dye, isosulfan blue, has also been used, but it cannot be seen through the skin and fatty tissue. Furthermore, the blue dye results in a breast tattoo that persists for several months, subcutaneous injections may be associated with skin necrosis, and rare allergic reactions have been reported. Severe allergic reactions have occurred following isosulfan blue injections (approximately 2% of patients). Manifestations include respiratory distress, shock, angioedema, urticaria, and pruritus. Reactions are more likely to occur in subjects with a history of bronchial asthma or those who are allergic to triphenylmethane dye or have had a medication reaction. Isosulfan blue is known to interfere with oxygen saturation measurements by pulse oximetry and methemoglobin measurements by gas analyzers. The use of isosulfan blue may result in transient or long-term (tattoo-like) bluish discoloration.

[0232] In contrast, fluorescence imaging for use in SLN visualization, mapping according to various embodiments facilitates direct real-time visual identification of LNs and / or afferent lymphatics intraoperatively, facilitates real-time high-resolution optical guidance through skin and adipose tissue, blood flow visualization, tissue perfusion, or a combination thereof.

[0233] In some variations, visualization, classification, or both of lymph nodes during fluorescence imaging can be based on imaging with one or more imaging agents, which can be further based on visualization and / or classification using a gamma probe (e.g., technetium Tc-99m is a clear, colorless aqueous solution typically injected into the periareolar region according to standard of care), another commonly used color imaging agent (isosulfan blue), and / or other assessments (such as, for example, histology). A subject's breast can be injected twice with approximately 1% isosulfan blue (for comparative purposes) and twice with an ICG solution having a concentration of approximately 2.5 mg / ml. The isosulfan blue injection can precede the ICG injection, or vice versa. For example, using a TB syringe and a 30 G needle, an anesthetized subject can be injected with 0.4 ml (0.2 ml at each site) of isosulfan blue in the periareolar region of the breast. For the right breast, the subject can be injected at the 12 and 9 o'clock positions, and for the left breast, at the 12 and 3 o'clock positions. The total dose of isosulfan blue injected intradermally into each breast can be approximately 4.0 mg (0.4 ml of a 1% solution: 10 mg / ml). In another exemplary variation, the subject can receive an ICG injection first, followed by an isosulfan blue injection (for comparison). Immediately prior to ICG administration, a 25 mg vial of ICG can be reconstituted with 10 ml of sterile water for injection to produce a 2.5 mg / ml solution. For example, using a TB syringe and a 30G needle, the subject can be injected with approximately 0.1 ml (0.05 ml at each site) of ICG in the periareolar region of the breast (for the right breast, injections can be performed at the 12 and 9 o'clock positions, and for the left breast, injections can be performed at the 12 and 3 o'clock positions). The total dose of ICG injected intradermally into each breast can be approximately 0.25 mg (0.1 ml of a 2.5 mg / ml solution). For example, the ICG can be injected at a rate of 5 to 10 seconds per injection. When ICG is injected intradermally, the binding properties of the ICG protein cause it to be rapidly absorbed by the lymph and move to the LNs through conducting blood vessels. In some variations, ICG can be provided in the form of a sterile lyophilized powder containing 25 mg of ICG and no more than 5% sodium iodide. ICG can be packaged with an aqueous solvent consisting of sterile water for injection, which is used to reconstitute the ICG. In some variations, the ICG dose (mg) in sentinel lymph node mapping for breast cancer can range from about 0.5 mg to about 10 mg, depending on the route of administration. In some variations, the ICG dose can be about 0.6 mg to about 0.75 mg, about 0.75 mg to about 5 mg, or about 5 mg to about 10 mg.Administration routes can include, for example, subcutaneous, intradermal (e.g., into the periareolar area), subcutaneous, skin overlying the tumor, intradermal in the areola proximal to the tumor, subcutaneous into the areola, intradermal above the tumor, periareolar over the entire breast, or combinations thereof. NIR fluorescence-positive LNs (e.g., using ICG) can be represented as, for example, black-and-white NIR fluorescence image(s) and / or full or partial color (white light) images, full or partial desaturated white light images, enhanced color images, overlays (e.g., fluorescence with any other image), composite images (e.g., fluorescence incorporated into another image), and can have various colors, desaturation levels, or color ranges to highlight / visualize certain features of interest. Image processing can be further performed for further visualization and / or other analysis (e.g., quantification). According to the American Society of Breast Surgeons (ASBrS) practice guidelines for SLN biopsy in breast cancer patients, fluorescence imaging systems and methods based on various embodiments of ICG and SLNs can be used, alone or in combination with a gamma probe (Tc-99m), to visualize lymph nodes and lymphatic vessels (e.g., intraoperatively, in real time). Fluorescence imaging of LNs can begin at the injection site by tracing the lymphatic tract leading to the LN in the axilla. Once the LN is visually identified, LN mapping and identification can be completed through the incised skin. LN mapping can be performed until ICG-visualized lymph nodes are identified. For comparison, mapping can be performed using isosulfan blue until "blue" lymph nodes are identified. LNs identified using ICG alone or in combination with another imaging technique (e.g., isosulfan blue and / or Tc-99m) can be marked for resection. Subjects may have different stages of breast cancer (e.g., IA, IB, IIA).

[0234] In some variations, such as, for example, in gynecological cancers (e.g., uterine, endometrial, vulvar, and cervical malignancies), ICG can be administered interstitially for visualization of lymph nodes, lymphatic channels, or a combination thereof. When injected interstitially, ICG's protein-binding properties cause it to be rapidly absorbed by the lymphatic system and mobilized to the SLNs via conducting blood vessels. ICG can be provided for injection as a sterile lyophilized powder containing 25 mg of ICG (e.g., 25 mg per vial) and no more than 5% sodium iodide. The ICG can then be reconstituted with commercially available sterile water for injection prior to use. According to one embodiment, a vial containing 25 mg of ICG can be reconstituted in 20 ml of water for injection, resulting in a 1.25 mg / ml solution. A total of 4 ml of this 1.25 mg / ml solution will be injected into the subject (4 x 1 ml injections), for a total ICG dose of 5 mg per subject. Four (4) injections of 1 ml of a 10 mg / ml 1% isosulfan blue solution may also be administered into the cervix (for comparative purposes) for a total dose of 40 mg. The injections may be performed while the subject is under anesthesia in the operating room. In some variations, the ICG dose (mg) in sentinel lymph node detection and / or mapping for gynecological cancers may range from about 0.1 mg to about 5 mg, depending on the route of administration. In some variations, the ICG dose may be about 0.1 mg to about 0.75 mg, about 0.75 mg to about 1.5 mg, about 1.5 mg to about 2.5 mg, about 2.5 mg to about 5 mg. The route of administration may be, for example, cervical injection, peritumoral injection, hysteroscopic endometrial injection, or a combination thereof. To minimize spillage of isosulfan blue or ICG that may interfere with the mapping procedure when LNs are to be resected, mapping may be performed on the hemipelvis and with both isosulfan blue and ICG prior to resection of any LNs. LN mapping for clinical stage I endometrial cancer can be performed according to the NCCN Guidelines for Uterine Corpus Tumors, SLN Surgical Staging Algorithm for Endometrial Cancer; and SLN mapping for clinical stage I cervical cancer can be performed according to the NCCN Guidelines for Cervical Cancer, Surgical / SLN Mapping Algorithm for Early-Stage Cervical Cancer. Thus, LN identification can be based on ICG fluorescence imaging alone or in combination or co-administration with a colorimetric dye (isosulfan blue) and / or a radiotracer.

[0235] Lymph node visualization can be qualitative and / or quantitative. Such visualization can include, for example, lymph node detection, detection rate, and anatomical distribution of lymph nodes. Lymph node visualization according to various embodiments can be used alone or in combination with other variables (e.g., vital signs, height, weight, demographics, surgical prognostic factors, relevant medical history and underlying conditions, histological visualization and / or assessment, Tc-99m visualization and / or assessment, concomitant medications). Follow-up can occur on the day of discharge and at a subsequent date (e.g., one month).

[0236] Lymphatic fluid contains high levels of protein, so ICG can bind to endogenous proteins upon entering the lymphatic system. When used according to the methods and systems described herein, fluorescence imaging (e.g., ICG imaging) for lymphatic mapping offers the following exemplary advantages: a high signal-to-background ratio (or tumor-to-background ratio) because NIR does not generate significant autofluorescence; real-time visualization features for lymphatic mapping; tissue definition (i.e., structural visualization); rapid excretion and elimination after entering the vascular system; and avoidance of non-ionizing radiation. Furthermore, NIR imaging has superior tissue penetration (approximately 5 to 10 mm of tissue) compared to visible light (1 to 3 mm of tissue). For example, the use of ICG also facilitates visualization through the peritoneum covering para-aortic lymph nodes. While tissue fluorescence can be observed over extended periods with NIR light, it is not visible with visible light and therefore does not affect the pathological assessment or management of LNs. Furthermore, fluorescence is easier to detect during surgery than the blue staining of lymph nodes (isochromatic blue). In other variations, the methods, dosages, or combinations thereof as described herein in connection with lymphatic imaging may be used in any vascular and / or tissue perfusion imaging application.

[0237] Tissue perfusion involves the microcirculatory flow of blood per unit tissue volume, in which oxygen and nutrients are supplied to, and waste products are removed from, the capillary beds of perfused tissue. Tissue perfusion is a phenomenon related to, but distinct from, blood flow in blood vessels. Quantified blood flow through blood vessels can be expressed in terms of flow rate (i.e., volume / time) or velocity (i.e., distance / time). Tissue blood perfusion defines the movement of blood within a tissue volume through the microvasculature (such as arterioles, capillaries, or venules). Quantified tissue blood perfusion can be expressed in terms of blood flow through a tissue volume: blood volume / time / tissue volume (or tissue mass). Perfusion is associated with nutrient vessels (e.g., microvessels known as capillaries), which include vessels involved in the exchange of metabolites between blood and tissue, rather than larger, non-nutrient vessels. In some embodiments, quantification of target tissue can include calculating or determining a parameter or quantity (such as velocity, size, volume, time, distance / time, and / or volume / time) associated with the target tissue, and / or the amount of change as it relates to any one or more of the aforementioned parameters or quantities. However, compared to blood movement through larger diameter vessels, blood movement through individual capillaries can be highly erratic, primarily due to vasomotion, where spontaneous oscillations in vascular tone manifest as pulsations in red blood cell movement. In some embodiments, blood flow and tissue perfusion imaging as described herein in conjunction with the systems and methods can be used to image tumor tissue and distinguish such tissue from other tissues.

[0238] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described to best explain the principles of the technology and its practical application. This will enable others skilled in the art to best utilize the technology and various embodiments with various modifications as are suitable for the particular application contemplated.

[0239] Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications should be understood to be included within the scope of the present disclosure and examples as defined by the claims. Finally, the entire disclosures of the patents and publications mentioned in this application are incorporated herein by reference.

Claims

1. A method for configuring a medical imaging processing system, the method comprising: configuring a reconfigurable hardware processor of the medical imaging processing system to a first configuration for a first medical imaging session based on first configuration data stored in the memory, wherein the first configuration implements at least a first medical imaging processing algorithm; receiving first medical imaging data generated during a first medical imaging session; generating enhanced first medical imaging data at least in part by processing the first medical imaging data using a first medical imaging processing algorithm implemented in a first configuration, wherein the first configuration data includes at least one first hardware logic file to customize the reconfigurable hardware processor to a first hardware processor configuration for processing the first medical imaging data according to the first medical imaging processing algorithm; displaying the enhanced first medical imaging data for viewing during the first medical imaging session; reconfiguring the reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the second configuration implements at least a second medical imaging processing algorithm not implemented in the first configuration; receiving second medical imaging data generated during a second medical imaging session; generating enhanced second medical imaging data at least in part by processing the second medical imaging data using a second medical imaging processing algorithm implemented in a second configuration, wherein the second configuration data includes at least one second hardware logic file to customize the reconfigurable hardware processor to a second hardware processor configuration for processing the second medical imaging data according to the second medical imaging processing algorithm; and The enhanced second medical imaging data is displayed on the display for viewing during the second medical imaging session. 2 . The method of claim 1 , comprising receiving input indicative of a second medical imaging session, and in response to receiving the input, automatically reconfiguring the reconfigurable hardware processor to the second configuration.

3. The method of claim 2, wherein the input comprises a selection of a medical procedure type. The method of claim 2 , wherein the input comprises selection of a user profile.

5. The method of any one of claims 2-4, wherein the input comprises a selection of a default configuration profile. The method of claim 5 , wherein the default configuration profile is based on one or more connections from one or more external devices to the medical imaging processing system. The method of claim 6 , wherein the default configuration profile is based on a field of view of the connected external device.

8. The method of any one of claims 1-4, wherein the first configuration is associated with a first type of medical procedure and the second configuration is associated with a second type of medical procedure. 9 . The method of claim 8 , wherein the first medical imaging session comprises performing a first type of medical procedure on the patient and the second medical imaging session comprises performing a second type of medical procedure on the patient.

10. The method of any of claims 1-4, wherein the first configuration is associated with a first user profile and the second configuration is associated with a second user profile.

11. The method of claim 10, wherein the first medical imaging session comprises imaging a patient and the second medical imaging session comprises imaging a patient.

12. The method of claim 10, wherein the first configuration data and the second configuration data are both associated with the same type of medical procedure.

13. The method of any one of claims 1-4, wherein the first medical imaging session is a first surgical session and the second medical imaging session is a second surgical session.

14. The method of any one of claims 1-4, wherein the at least one medical imaging processing algorithm implemented in the second configuration comprises a smoke detection algorithm, and generating enhanced second medical imaging data comprises enhancing the sharpness of one or more portions of one or more images associated with smoke.

15. The method of any one of claims 1-4, wherein the first medical imaging processing algorithm is configured to detect features of the imaged tissue.

16. The method of claim 15, wherein the characteristic of the imaged tissue is tissue perfusion, blood vessel location, blood flow, size of the imaged tissue, or a combination thereof.

17. The method of any one of claims 1-4, wherein the enhanced second medical imaging data comprises an overlay on at least a portion of the second medical imaging data.

18. The method of any one of claims 1-4, wherein the reconfigurable hardware processor is reconfigured before imaging begins.

19. The method of any one of claims 1-4, wherein one or more medical imaging processing algorithms are implemented in both the first configuration and the second configuration.

20. The method of any one of claims 1-4, wherein the second medical imaging data comprises at least one of a video frame and an image.

21. The method of any one of claims 1-4, wherein the second medical imaging data is received from an endoscopic imaging system.

22. The method of claim 21, wherein the second medical imaging data is received from a camera control unit.

23. The method according to any one of claims 1-4, wherein the reconfigurable hardware processor is an FPGA or a GPU.

24. The method of any one of claims 1-4, comprising receiving second medical imaging data from a first device, receiving data from a second medical device, and outputting a display feed to a display, the display feed comprising the enhanced second medical imaging data and at least a portion of the data from the second medical device.

25. The method of claim 24 , comprising receiving, at a first processor, second medical imaging data and data from a second medical device, transmitting the second medical imaging data from the first processor to a reconfigurable hardware processor, receiving, at the first processor, enhanced second medical imaging data from the reconfigurable hardware processor, and generating, by the first processor, a display feed by combining the enhanced second medical imaging data with at least a portion of the data associated with the second medical device.

26. The method according to any of claims 1-4, wherein the first configuration data is stored in a remote memory and is received via a network connection.

27. A reconfigurable medical imaging processing system comprising: monitor; Memory; reconfigurable hardware processors; and A second processor is configured to: configuring a reconfigurable hardware processor to a first configuration for a first medical imaging session based on first configuration data stored in a memory, wherein the reconfigurable hardware processor in the first configuration is configured to implement at least a first medical imaging processing algorithm and generate enhanced first medical imaging data for display on a display at least in part by processing first medical imaging data using the first medical imaging processing algorithm, wherein the first configuration data includes at least one first hardware logic file to customize the reconfigurable hardware processor to the first hardware processor configuration for processing the first medical imaging data according to the first medical imaging processing algorithm, and Reconfiguring the reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the reconfigurable hardware processor in the second configuration is configured to implement at least a second medical imaging processing algorithm and generate enhanced second medical imaging data for display on a display at least in part by processing second medical imaging data using the second medical imaging processing algorithm, wherein the second configuration data includes at least one second hardware logic file to customize the reconfigurable hardware processor to the second hardware processor configuration for processing the second medical imaging data according to the second medical imaging processing algorithm.

28. The system of claim 27, wherein the second processor is configured to receive input indicative of a second medical imaging session and, in response to receiving the input, automatically reconfigure the reconfigurable hardware processor to the second configuration.

29. The system of claim 28, wherein the input comprises a selection of a medical procedure type.

30. The system of claim 28, wherein the input comprises a selection of a user profile.

31. The system of any one of claims 28-30, wherein the input comprises a selection of a default configuration profile.

32. The system of claim 31, wherein a default configuration profile is based on one or more connections from one or more external devices to the medical imaging processing system.

33. The system of claim 32, wherein the default configuration profile is based on a field of view of a connected external device.

34. The system of any one of claims 27-30, wherein the first configuration is associated with a first type of medical procedure and the second configuration is associated with a second type of medical procedure.

35. The system of claim 34, wherein the first medical imaging session comprises performing a first type of medical procedure on the patient and the second medical imaging session comprises performing a second type of medical procedure on the patient.

36. The system of any of claims 27-30, wherein the first configuration is associated with a first user profile and the second configuration is associated with a second user profile.

37. The system of claim 36, wherein the first medical imaging session comprises imaging the patient and the second medical imaging session comprises imaging the patient.

38. The system of claim 36, wherein the first configuration data and the second configuration data are both associated with the same type of medical procedure.

39. The system of any one of claims 27-30, wherein the first medical imaging session is a first surgical session and the second medical imaging session is a second surgical session.

40. The system of any one of claims 27-30, wherein the at least one medical imaging processing algorithm implemented in the second configuration comprises a smoke detection algorithm, and generating enhanced second medical imaging data comprises enhancing the sharpness of one or more portions of one or more images associated with smoke.

41. The system of any one of claims 27-30, wherein the first medical imaging processing algorithm is configured to detect features of the imaged tissue.

42. The system of claim 41, wherein the characteristic of the imaged tissue is tissue perfusion, blood vessel location, blood flow, size of the imaged tissue, or a combination thereof.

43. The system of any one of claims 27-30, wherein the enhanced second medical imaging data comprises an overlay on at least a portion of the second medical imaging data.

44. The system of any one of claims 27-30, wherein the system is configured to reconfigure the reconfigurable hardware processor before imaging begins.

45. The system of any one of claims 27-30, wherein one or more medical imaging processing algorithms are implemented in both the first configuration and the second configuration.

46. ​​The system of any one of claims 27-30, wherein the second medical imaging data comprises at least one of a video frame and an image.

47. The system of any one of claims 27-30, wherein the system is configured to receive the second medical imaging data from an endoscopic imaging system.

48. The system of claim 47, wherein the system is configured to receive the second medical imaging data from a camera control unit.

49. The system of any one of claims 27-30, wherein the reconfigurable hardware processor is an FPGA or a GPU.

50. The system of any one of claims 27-30, wherein the system is configured to receive second medical imaging data from a first device, receive data from a second medical device, and display at least a portion of the enhanced second medical imaging data and the data from the second medical device.

51. The system of claim 50, wherein the system is configured to receive the second medical imaging data and data from the second medical device at the second processor, transmit the second medical imaging data from the second processor to the reconfigurable hardware processor, receive the enhanced second medical imaging data at the second processor from the reconfigurable hardware processor, and generate a display feed for display by the second processor by combining the enhanced second medical imaging data with at least a portion of the data associated with the second medical device.

52. The system of any one of claims 27-30, wherein the first configuration data is stored in a remote memory and is received via a network connection.

53. A non-transitory tangible computer readable medium having computer executable program code embodied thereon to perform the method of any one of claims 1 to 26.

54. A kit for processing a time series of fluorescence images of tissue of a subject, the kit comprising the system of any one of claims 27 to 52 or the non-transitory tangible computer readable medium of claim 53 and a fluorescent imaging agent.

55. A fluorescent imaging agent for use in a method according to any one of claims 1 to 26, a system according to any one of claims 27 to 52, or a kit according to claim 54, for imaging a subject.

56. The fluorescent imaging agent of claim 55, wherein imaging the subject comprises imaging the subject during blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof.

57. The fluorescent imaging agent of claim 56, wherein blood flow imaging, tissue perfusion imaging and / or lymphatic imaging comprises blood flow imaging, tissue perfusion imaging and / or lymphatic imaging during an invasive surgical procedure, a minimally invasive surgical procedure, or during a non-invasive surgical procedure.

58. The fluorescent imaging agent of claim 57, wherein the invasive surgical procedure comprises a heart-related surgical procedure or a reconstructive surgical procedure.

59. The fluorescent imaging agent of claim 58, wherein the heart-related surgical procedure comprises a coronary artery bypass grafting procedure.

60. The fluorescent imaging agent of claim 59, wherein the coronary artery bypass grafting procedure is performed on or off-pump.

61. The fluorescent imaging agent of claim 57, wherein the non-invasive surgical procedure comprises a wound care procedure.

62. The fluorescent imaging agent of any one of claims 56-61, wherein lymphatic imaging comprises identification of lymph nodes, lymph node drainage, lymphatic mapping, or a combination thereof.

63. The fluorescent imaging agent of any one of claims 56-61, wherein lymphatic imaging relates to the female reproductive system.

64. Use of the method of any one of claims 1 to 26 in a system according to any one of claims 27 to 52 or a kit according to claim 54 for imaging a subject for lymphatic imaging.

65. Use of the method of any one of claims 1 to 26 in a system of any one of claims 27 to 52 or a kit of claim 54 for imaging a subject for blood flow imaging, tissue perfusion imaging, or a combination thereof.

Images