System and method for displaying medical imaging data

Abstract

To provide an imaging processing system that improves utilization efficiency of an FOV part in an imaging area.SOLUTION: A system of the present invention displays medical imaging data, and includes one or more data inputs, one or more processors, and one or more displays. The one or more data inputs are configured to receive first image data generated by a first medical imaging device. The first image data includes a field-of-vision (FOV) part and a non-FOV part (Step 202). The one or more processors identify the non-FOV part of the first image data (Step 204). By removing at least part of the non-FOV part of the first image data, cropped first image data is generated (Step 206). The cropped first image data to be displayed in a first part of the display and additional information to be displayed in a second part of the display are transmitted (Step 208).SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure generally relates to medical imaging, and more particularly to the processing of medical imaging for tissue visualization.

Background Art

[0002] (Cross-reference to Related Applications) This application claims the benefit of U.S. Provisional Application No. 62 / 775,622, filed on Dec. 5, 2018, the entire content of which is incorporated herein by reference.

[0003] With the emergence of high definition (HD) and Ultra HD / 4K resolution in surgical visualization, 16:9 aspect ratio surgical displays are becoming increasingly common. However, many minimally invasive surgical procedures still rely on an optical scope where the field of view in the imaging device is smaller than the sensing area of the imaging device. As a result, the image and video will have a circular field of view area within a substantial area of black pixels. In many cases, for example, only 44% of the available imaging area can be utilized, and as a result, only 44% of the available display area may be utilized. This is particularly true for smaller diameter scopes, such as a 4 mm diameter scope typically used in arthroscopic or otolaryngology / neuro (ENT / Neuro) procedures.

[0004] When a surgeon needs to view multiple information sources such as multiple low-usage optical scope images, the surgeon may switch the input on the primary surgical display between various imaging feeds, use a picture-in-picture mode or picture-by-picture mode on the surgical display, or may have to look at two different monitors that may be in two different locations in the operating room. None of these options can optimally utilize the visible area of the surgical display or may cause situations where the surgeon is forced to switch focus from one display to another.

Summary of the Invention

[0005] According to some embodiments, a medical imaging processing system is configured to process and combine medical imaging data to generate a display feed that provides an enhanced representation of medical imaging. According to some embodiments, the medical imaging processing system can combine multiple imaging data streams into one or more display streams to display data from multiple imaging sources and other imaging session-related information sources together in a single display layout. According to some embodiments, the use of the display layout can be optimized by removing unused portions of the imaging data, such as data generated by portions of the image sensor outside the imaging field of view. In some embodiments, the display feed can be generated according to imaging session-specific preference trends tailored to a particular type of imaging session and / or a user of a particular imaging system. In some embodiments, a reconfigurable hardware processor of the medical imaging processing system may be reconfigured from one imaging session to another, providing imaging data processing that is tailored to the next imaging session. Through one or more of these capabilities, the imaging processing system can provide an enhanced visualization of medical imaging that is tailored to the user's preference trends.

[0006] According to some embodiments, a method for configuring a medical imaging processing system is to configure a reconfigurable hardware processor of a medical imaging processing system into a first configuration for a first medical imaging session based on first configuration data stored in memory, wherein the first configuration generates enhanced first medical imaging data by at least partially implementing a medical imaging processing algorithm in the first configuration for a first medical imaging session, receiving first medical imaging data generated during the first medical imaging session, and processing the first medical imaging data using the first medical imaging processing algorithm implemented in the first configuration, thereby enabling enhanced first medical imaging data for observation during the first medical imaging session. The method includes displaying first medical imaging data, and reconfiguring a reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in memory, wherein the second configuration implements at least a second medical imaging processing algorithm not implemented in the first configuration, generates enhanced second medical imaging data by at least partially receiving second medical imaging data generated during the second medical imaging session, and processing the second medical imaging data using the second medical imaging processing algorithm implemented in the second configuration, and displays the enhanced second medical imaging data on a display for observation during the second medical imaging session.

[0007] In any of these embodiments, the method may include receiving an input indicating the second medical imaging session, and automatically reconfiguring the reconfigurable hardware processor to the second configuration in response to receiving the input.

[0008] In any of these embodiments, the input may include the selection of the type of medical procedure.

[0009] In any of these embodiments, the input may include the selection of a user profile.

[0010] In any of these embodiments, the input may include the selection of a default configuration profile.

[0011] In any of these embodiments, the default configuration profile may be based on one or more connections from one or more external devices to the medical imaging processing system.

[0012] In any of these embodiments, the default configuration profile may be based on the field of view of connected external devices.

[0013] In any of these embodiments, the first configuration may be associated with a first type of medical procedure, and the second configuration may be associated with a second type of medical procedure.

[0014] In any of these embodiments, the first medical imaging session may include the performance of a first type of medical procedure on the patient, and the second medical imaging session may include the performance of a second type of medical procedure on the patient.

[0015] In any of these embodiments, the first configuration may be associated with a first user profile, and the second configuration may be associated with a second user profile.

[0016] In any of these embodiments, the first medical imaging session may include imaging the patient, and the second medical imaging session may include imaging the patient.

[0017] In any of these embodiments, both the first and second configuration data can be associated with the same type of medical procedure.

[0018] In any of these embodiments, the first medical imaging session may be the first surgical session, and the second medical imaging session may be the second surgical session.

[0019] In any of these embodiments, at least one medical imaging processing algorithm implemented in the second configuration includes a smoke detection algorithm, and the generation of enhanced second medical imaging data may include enhancing the clarity of one or more portions of one or more images related to smoke.

[0020] In any of these embodiments, the first medical imaging processing algorithm may detect features of the imaged tissue.

[0021] In any of these embodiments, the features of the imaged tissue may be tissue perfusion, blood vessel location, blood flow rate, dimensions of the imaged tissue, or a combination thereof.

[0022] In any of these embodiments, the enhanced second medical imaging data may include an overlay on at least a portion of the second medical imaging data.

[0023] In any of these embodiments, the reconfigurable hardware processor may be reconfigured before the start of imaging.

[0024] In any of these embodiments, one or more medical imaging processing algorithms may be implemented in both the first and second configurations.

[0025] In any of these embodiments, the second medical imaging data may include at least one of video frames and images.

[0026] In any of these embodiments, the second medical imaging data may be received from an endoscopic imaging system.

[0027] In any of these embodiments, the second medical imaging data may be received from a camera control unit.

[0028] In any of these embodiments, the reconfigurable hardware processor may be an FPGA or a GPU.

[0029] In any of these embodiments, the method includes receiving second medical imaging data from a first device, receiving data from a second medical device, and outputting a display feed to a display, the display feed including enhanced second medical imaging data and at least a portion of the data from the second medical device.

[0030] In any of these embodiments, the method may include receiving, in a first processor, second medical imaging data and data from a second medical device, transmitting the second medical imaging data from the first processor to a reconfigurable hardware processor, receiving, in the first processor, enhanced second medical imaging data from the reconfigurable hardware processor, and generating, by the first processor, a display feed by combining the enhanced second medical imaging data and at least a portion of the data associated with the second medical device.

[0031] In any of these embodiments, the first configuration data may be stored in a remote memory and received via a network connection.

[0032] According to some embodiments, a method of displaying medical imaging data includes receiving first image data generated by a first medical imaging device, the first image data including a field of view (FOV) portion and a non-FOV portion, identifying the non-FOV portion of the first image data, generating cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and displaying the cropped first image data on a first portion of a display and additional information on a second portion of the display. [[ID=十七]]

[0033] In any of these embodiments, the non-FOV portion may be identified using edge detection.

[0034] In any of these embodiments, the first image data includes a series of video frames, and edge detection may be performed on multiple frames.

[0035] In any of these embodiments, the non-FOV portion can be identified using one or more measurements relating to the center position of the FOV portion and the dimensions of the FOV portion.

[0036] In any of these embodiments, the measurement values ​​relating to the center position of the FOV portion and the dimensions of the FOV portion may be determined during the initialization process of the imaging session.

[0037] In any of these embodiments, the initialization process of the imaging session may be a white balance process.

[0038] In any of these embodiments, the first image data includes a rectangular image or video frame, and the FOV portion may be a circular portion of the rectangular image or video frame.

[0039] In any of these embodiments, the first image data may include video frames.

[0040] In any of these embodiments, the first image data is received at a first input of the medical imaging processing system, and additional information is obtained based on data received at a second input of the medical imaging processing system.

[0041] In any of these embodiments, the method includes transmitting a display feed from a medical imaging processing system to a display, the display feed may include a combination of cropped first image data and additional information.

[0042] In any of these embodiments, the method may include receiving a second image data generated by a second medical imaging device, identifying a non-FOV portion of the second image data, generating a cropped second image data by removing at least a portion of the non-FOV portion of the second image data, and displaying the cropped second image data on a second portion of a display.

[0043] In any of these embodiments, the first image data may be received at a first input of the medical imaging processing system, and the second image data may be received at a second input of the medical imaging processing system.

[0044] In any of these embodiments, the method includes transmitting a display feed from a medical imaging processing system to a display, the display feed may include a combination of a cropped first image data and a cropped second image data.

[0045] In any of these embodiments, the cropped first image data and additional information may be arranged on the display based on configuration data stored in memory.

[0046] In any of these embodiments, the configuration data may include user-specified configuration data.

[0047] In any of these embodiments, configuration data may be received via a network connection.

[0048] In any of these embodiments, the first image data may be received from an endoscopic imaging system, an intraoperative C-arm imaging system, or an ultrasound system.

[0049] In any of these embodiments, the first image data may be received from a camera control unit.

[0050] In any of these embodiments, the additional information may include one or more of the following: patient data, metrics, graphs, images, device status, and video feeds.

[0051] According to some embodiments, a reconfigurable medical imaging processing system may include a display, memory, a reconfigurable hardware processor, and a second processor configured as follows: Based on first configuration data stored in memory, the reconfigurable hardware processor is configured to a first configuration for a first medical imaging session, wherein the reconfigurable hardware processor in the first configuration is configured to generate enhanced first medical imaging data for display on a display by implementing at least a first medical imaging processing algorithm and at least partially processing first medical imaging data using the first medical imaging processing algorithm; Based on second configuration data stored in memory, the reconfigurable hardware processor is configured to a second configuration for a second medical imaging session, wherein the reconfigurable hardware processor in the second configuration is configured to generate enhanced second medical imaging data for display on a display by implementing at least a second medical imaging processing algorithm and at least partially processing second medical imaging data using the second medical imaging processing algorithm.

[0052] In any of these embodiments, the second processor may be configured to receive an input indicating a second medical imaging session and, in response to receiving said input, to automatically reconfigure a reconfigurable hardware processor to a second configuration.

[0053] In any of these embodiments, the input may include the selection of the type of medical procedure.

[0054] In any of these embodiments, the input may include the selection of a user profile.

[0055] In any of these embodiments, the input may include the selection of a default configuration profile.

[0056] In any of these embodiments, the default configuration profile may be based on one or more connections from one or more external devices to the medical imaging processing system.

[0057] In any of these embodiments, the default configuration profile may be based on the field of view of connected external devices.

[0058] In any of these embodiments, the first configuration may be associated with a first type of medical procedure, and the second configuration may be associated with a second type of medical procedure.

[0059] In any of these embodiments, the first medical imaging session may include the performance of a first type of medical procedure on the patient, and the second medical imaging session may include the performance of a second type of medical procedure on the patient.

[0060] In any of these embodiments, the first configuration may be associated with a first user profile, and the second configuration may be associated with a second user profile.

[0061] In any of these embodiments, the first medical imaging session may include imaging the patient, and the second medical imaging session may include imaging the patient.

[0062] In any of these embodiments, both the first and second configuration data can be associated with the same type of medical procedure.

[0063] In any of these embodiments, the first medical imaging session may be a first surgical session, and the second medical imaging session may be a second surgical session.

[0064] In any of these embodiments, at least one medical imaging processing algorithm implemented in the second configuration includes a smoke detection algorithm, and the generation of enhanced second medical imaging data may include enhancing the clarity of one or more portions of one or more images related to smoke.

[0065] In any of these embodiments, the first medical imaging processing algorithm may be configured to detect features of the imaged tissue.

[0066] In any of these embodiments, the characteristics of the imaged tissue may be tissue perfusion, vascular location, blood flow rate, dimensions of the imaged tissue, or a combination thereof.

[0067] In any of these embodiments, the enhanced second medical imaging data may include an overlay on at least a portion of the second medical imaging data.

[0068] In any of these embodiments, the system may be configured to reconfigure a hardware processor that can be reconfigured before the start of imaging.

[0069] In any of these embodiments, one or more medical imaging processing algorithms may be implemented in both the first and second configurations.

[0070] In any of these embodiments, the second medical imaging data may include at least one of a video frame and an image.

[0071] In any of these embodiments, the system may be configured to receive second medical imaging data from the endoscopic imaging system.

[0072] In any of these embodiments, the system may be configured to receive second medical imaging data from a camera control unit.

[0073] In any of these embodiments, the reconfigurable hardware processor may be an FPGA or a GPU.

[0074] In any of these embodiments, the system may be configured to receive second medical imaging data from a first device, receive data from a second medical device, and display the enhanced second medical imaging data and at least a portion of the data from the second medical device.

[0075] In any of these embodiments, the system may be configured such that the second processor receives second medical image data and data from the second medical device, transmits the second medical imaging data from the second processor to a reconfigurable hardware processor, receives enhanced second medical imaging data from the reconfigurable hardware processor, and combines the enhanced second medical imaging data with at least a portion of the data related to the second medical device, thereby causing the second processor to generate a display feed for a display.

[0076] In any of these embodiments, the first configuration data may be stored in remote memory and received via a network connection.

[0077] According to some embodiments, a system for displaying medical imaging data includes one or more data inputs, one or more processors, and one or more displays, wherein one or more data inputs are configured to receive first image data generated by a first medical imaging device, the first image data including a field of view (FOV) portion and a non-FOV portion, and one or more processors are configured to identify the non-FOV portion of the first image data, generate cropped first image data by removing at least a portion of the non-FOV portion of the first image data, and transmit the cropped first image data for display on a first portion of a display and additional information for display on a second portion of one or more displays.

[0078] In any of these embodiments, one or more processors may be configured to identify non-FOV areas using edge detection.

[0079] In any of these embodiments, the first image data comprises a series of video frames, and one or more processors may be configured to identify non-FOV portions using edge detection performed on multiple frames.

[0080] In any of these embodiments, one or more processors may be configured to identify non-FOV portions using one or more measurements relating to the center position of the FOV portion and the dimensions of the FOV portion.

[0081] In any of these embodiments, one or more processors may be configured to determine measurements relating to the center position of the FOV portion and the dimensions of the FOV portion during the initialization process of the imaging session.

[0082] In any of these embodiments, the initialization process of the imaging session may be a white balance process.

[0083] In any of these embodiments, the first image data includes a rectangular image or video frame, and the FOV portion may be a circular portion of the rectangular image or video frame.

[0084] In any of these embodiments, the first image data may include video frames.

[0085] In any of these embodiments, one or more data inputs are configured to receive first image data at a first input of a medical imaging processing system, and additional medical imaging data is obtained based on data received at a second input of the medical imaging processing system.

[0086] In any of these embodiments, the medical imaging processing system is configured to transmit a display feed from the medical imaging processing system to a display, the display feed may include a combination of cropped first image data and additional medical imaging data.

[0087] In any of these embodiments, one or more data inputs are configured to receive second image data generated by a second medical imaging device, and one or more processors may be configured to identify non-FOV portions of the second image data, generate cropped second image data by removing at least a portion of the non-FOV portions of the second image data, and transmit the cropped second image data for display on a second portion of one or more displays.

[0088] In any of these embodiments, one or more data inputs may be configured to receive first image data at a first input of the medical imaging processing system and to receive second image data at a second input of the medical imaging processing system.

[0089] In any of these embodiments, the medical imaging processing system is configured to transmit a display feed from the medical imaging processing system to a display, the display feed may include a combination of cropped first image data and cropped second image data.

[0090] In any of these embodiments, the cropped first image data and additional medical imaging data may be arranged on the display based on configuration data stored in memory.

[0091] In any of these embodiments, the configuration data may include user-specified configuration data.

[0092] In any of these embodiments, the system is configured to receive configuration data via a network connection.

[0093] In any of these embodiments, one or more data inputs may be configured to receive first image data from an endoscopic imaging system, an intraoperative C-arm imaging system, or an ultrasound system.

[0094] In any of these embodiments, one or more data inputs may receive first image data from the camera control unit.

[0095] In any of these embodiments, the additional information may include one or more of the following: patient data, metrics, graphs, images, device status, and video feeds.

[0096] According to some embodiments, a non-temporary, tangible, computer-readable medium includes computer-executable program code embedded to perform any of the methods described above.

[0097] According to some embodiments, a kit for processing time-series fluorescence images of a subject's tissue comprises one of the systems described above and / or one of the non-transient, tangible, computer-readable media described above, as well as a fluorescence contrast agent.

[0098] According to some embodiments, a fluorescent contrast agent is provided for use in any of the above methods, any of the above systems, or in any of the above kits for imaging an object.

[0099] In any of these embodiments, imaging of the object may include imaging of the object in blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof.

[0100] In any of these embodiments, blood flow imaging, tissue perfusion imaging, and / or lymph imaging may include blood flow imaging, tissue perfusion imaging, and / or lymph imaging in invasive surgical procedures, minimally invasive surgical procedures, or non-invasive surgical procedures.

[0101] In any of these embodiments, the invasive surgical procedure may include cardiac-related surgical procedures or reconstructive surgical procedures.

[0102] In any of these embodiments, cardiac-related surgical procedures may include coronary artery bypass grafting (CABG) procedures.

[0103] In any of these embodiments, the CABG procedure may be performed on-pump or off-pump.

[0104] In any of these embodiments, non-invasive surgical procedures may include wound care procedures.

[0105] In any of these embodiments, lymph imaging may include identification of lymph nodes, lymph node drainage, lymph mapping, or a combination thereof.

[0106] In any of these embodiments, lymphoid imaging may be related to the female reproductive system.

[0107] Some embodiments include using any of the above methods, using any of the above systems, or using any of the above kits for imaging objects in lymphoid imaging.

[0108] Some embodiments include the use of any of the above methods, any of the above systems, or any of the above kits for imaging objects in blood flow imaging, tissue perfusion imaging, or a combination thereof.

[0109] It will be understood that any modifications disclosed herein in connection with the methods, systems, kits, and other embodiments of this disclosure may be combined. [Brief explanation of the drawing]

[0110] The present invention will be described below with reference to the attached drawings. [Figure 1] Figure 1 is a block diagram of a system for generating and displaying medical imaging data during a medical imaging session, according to several embodiments. [Figure 2] Figure 2 shows a method for displaying medical imaging data according to several embodiments. [Figure 3A] , [Figure 3B] Figure 3A shows an exemplary image generated by an endoscope image sensor, and Figure 3B shows two endoscope images displayed side by side on an exemplary display. [Figure 3C] Figure 3C shows an exemplary display for showing a cropped endoscopic image according to several embodiments. [Figure 3D] Figure 3D shows an exemplary display, according to several embodiments, that shows a cropped endoscopic image and additional imaging session-related data. [Figure 4] Figure 4 is a block diagram of a medical imaging data processing hub according to several embodiments. [Figure 5A] , [Figure 5B] Figure 5A shows an example of a first predefined display layout that can be generated by the hub in Figure 4, and Figure 5B shows an example of a second predefined display layout that can be generated by the hub in Figure 4. [Figure 6] Figure 6 shows an example of a medical imaging processing hub configured for a first imaging session according to several embodiments. [Figure 7] Figure 7 shows a method for configuring a medical imaging processing system according to several embodiments. [Figure 8A] , [Figure 8B] Figures 8A and 8B are block diagrams of a medical imaging processing system that performs the method shown in Figure 7, according to one embodiment. [Figure 9A] , [Figure 9B]Figures 9A and 9B show a graphical user interface for configuring a medical imaging processing system for a new imaging session, according to several embodiments. [Figure 10] Figure 10 is an exemplary depiction of an exemplary fluorescence imaging system according to several embodiments. [Figure 11] Figure 11 is an exemplary depiction of an exemplary illumination module of a fluorescence imaging system according to several embodiments. [Figure 12] Figure 12 shows an exemplary camera module of a fluorescence imaging system according to several embodiments. [Figure 13] Figure 13 shows an exemplary endoscopic imaging cart according to several embodiments. [Modes for carrying out the invention]

[0111] Here, various aspects and modifications of the systems and methods described herein will be described in detail. While several exemplary variations of the systems and methods are described herein, other variations of the systems and methods may include any suitable combination of all or some of the described aspects of the systems and methods. This specification describes systems and methods for generating enhanced medical imaging images for display in connection with a medical imaging session (e.g., during a medical imaging session). The systems and methods can process data from multiple imaging systems to generate enhanced imaging images, or combine multiple imaging datasets into a single display feed for displaying information from multiple sources on a single display. Image data can be processed to maximize display utilization to enable the presentation of more relevant information to the practitioner during the imaging session.

[0112] According to some embodiments, the system and method can process and combine imaging data in different ways based on the needs of each imaging session. The practitioner can define the information displayed during the imaging session to ensure that the data is presented in a way that is suitable for the practitioner, which can reduce the time required for the practitioner to adjust the data display.

[0113] In one embodiment, one or more reconfigurable hardware processors are reconfigured for each imaging session to provide imaging processing tailored to each imaging session. Reconfigurable hardware processors, such as field-programmable gate arrays (FPGAs), provide the low latency and high bandwidth required for real-time image processing, and also provide the ability to implement different algorithms or different combinations of algorithms for different data inputs or combinations of data inputs required for each imaging session, thereby providing imaging processing tailored to the different needs of different imaging sessions. This configurability and flexibility in the ability to process and combine different input data enables a single imaging processing system according to the embodiments described herein to support a wide variety of imaging sessions, including a wide variety of surgical procedures.

[0114] In the following description of various embodiments, the accompanying drawings are referenced as examples, illustrating specific implementable embodiments. It should be understood that other embodiments and examples are implementable and can be modified without departing from the scope of this disclosure.

[0115] Furthermore, it should be understood that the singular forms "a," "an," and "the" used in the following descriptions are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the phrase "and / or" as used herein means and encompasses one or more and all combinations of the related enumerated items. Furthermore, it should be understood that the phrases "includes," "including," "comprises," and / or "comprising," as used herein, identify the presence of the referred feature, integer, process, operation, element, component, and / or unit, but do not exclude the presence or addition of one or more other features, integers, processes, operations, elements, components, units, and / or groups thereof.

[0116] Certain aspects of this disclosure include the processing steps and instructions described herein in the form of algorithms. It should be noted that the processing steps and instructions of this disclosure may be implemented in software, firmware, or hardware, and if implemented in software, they may reside on different platforms used by various operating systems and may be downloaded to operate from such different platforms. As will be apparent from the following description, unless otherwise specified, throughout this specification, any use of terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” and “generating” is understood to refer to the operation and processing of a computer system or similar electronic processing unit that manipulates and transforms data represented as physical (electronic) quantities in computer system memory or registers, or in other such information recording devices, transmitting devices, or display devices.

[0117] In some embodiments, the disclosure also relates to apparatus for performing the operations described herein. Such apparatus may be specifically configured for a particular purpose, or may include a general-purpose computer that is selectively started or reconfigured by a computer program stored in the computer. Such computer programs may be stored on non-temporary computer-readable recording media, such as (but not limited to) floppy disks, USB flash drives, external hard drives, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROMs, EEPROMs, magnetic or optical cards, disks of any type including application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, each of which may be coupled to a computer system bus. Furthermore, the computers referred to herein may include a single processor, or may have an architecture employing multiple processors to enhance computing power.

[0118] The methods, apparatus, and systems described herein are not inherently related to any particular computer or other device. Furthermore, various general-purpose systems may be used for programs following the teachings herein, and it may prove convenient to construct more specialized devices to perform the necessary method steps. The structures required for these various systems will become apparent from the following description. Moreover, the present invention is not described with reference to any particular programming language. It will be understood that various programming languages ​​may be used to carry out the teachings of the present invention as described herein.

[0119] Figure 1 shows a system 100 for generating and displaying medical imaging data during a medical imaging session. System 100 includes a medical data processing hub 102 that processes data received from one or more imaging modalities 104 to generate one or more display feeds for displaying enhanced medical imaging images on one or more displays 106. One or more imaging modalities 104 may generate image data related to the treatment of a patient. The image data may be images or videos generated during the treatment of a patient that support one or more medical procedures, such as images captured by an endoscope camera during an endoscopic procedure on a patient. Examples of medical imaging modalities include, but are not limited to, endoscopic systems, open-field imaging systems, X-ray systems such as intraoperative C-arm systems, computed tomography systems, ultrasound systems, magnetic resonance imaging systems, and nuclear medicine systems.

[0120] In some embodiments, the hub 102 may receive data from one or more non-imaging devices 120 that are used in connection with a medical imaging session (e.g., during a medical imaging session) and may provide information that may be relevant to the display during the medical imaging session. Non-limiting examples of non-imaging devices include insufflation devices, lighting controllers, and voice control systems.

[0121] The hub 102 may receive image data from one or more imaging modalities 104 via one or more input ports 108. The hub 102 generates one or more display feeds using the received imaging data and transmits one or more display feeds to one or more displays 106 via one or more output ports 110. For example, the hub 102 may generate a display feed that includes enhanced imaging images of patient tissue based on imaging images generated by one or more imaging modalities 104, and the enhanced imaging images may be displayed on one or more displays 106 to assist the practitioner during patient treatment. The hub 102 may also transmit the display feeds to one or more recording devices 112 to record the enhanced imaging images for later retrieval. The input ports 108 and output ports 110 may be any suitable type of data transmission port, such as a DVI port, HDMI port, RS232 port, or IP port.

[0122] The hub 102 may be connected to one or more networks 116 via one or more network connections 118. One or more networks may be local networks such as hospital information systems, or broader networks such as wide area networks or the internet. The network connections 118 may be wired connections such as Ethernet connections, or wireless network connections such as Wi-Fi connections. In some embodiments, the hub 102 may access one or more networks 116 to retrieve configuration data stored at network locations in order to configure the hub for imaging sessions, and / or access one or more networks to receive updated software and / or updated hardware files in order to process imaging data.

[0123] One or more user interfaces 114 may be connected to the hub 102 to allow users to provide input to the hub 102. Users may input data related to the configuration of the hub 102 for an imaging session. User input may include, for example, the selection of a practitioner profile related to an upcoming imaging session, the selection of the type of imaging session or the type of procedure to be performed during the imaging session, or other relevant information. One or more user interfaces 114 may include a tablet, keyboard, mouse, voice control system, keypad, touchscreen, or any combination thereof.

[0124] As detailed below, the hub 102 processes the received medical imaging data and any other relevant data to generate an enhanced display feed for display on one or more displays 106 during an imaging session. According to some embodiments, the hub 102 can combine multiple imaging sources into a single display feed, process the received imaging data to generate richer imaging data, modify the imaging data to make better use of the display space, and / or reconfigure the processing of the imaging data according to the user's needs and preferences from one imaging session to the next.

[0125] Figure 2 shows a method 200 for displaying medical imaging data according to several embodiments. Method 200 may be performed by a medical imaging data processing hub, such as the medical imaging data processing hub 102 of system 100. Method 200 is performed to remove unused portions of the received imaging data in order to make better use of the display space, thereby providing the user with the ability to display more relevant information during the imaging session.

[0126] For example, in many conventional imaging systems, such as scope-based imaging systems including endoscopic imaging systems, a roughly circular area of ​​light from the scene is projected onto the photosensitive area of ​​one or more imaging sensors. This is because one or more sensors of the image sensor generally have a larger area than the area of ​​light provided by the scope's optical system. Therefore, the image captured by one or more sensors includes a field of view (FOV) area representing the light received from the field of view and a non-FOV area generated by the parts of one or more sensors (i.e., pixels) that do not receive light from the scene, often resulting in a rectangular image with a circular FOV area in the center representing the imaging scene, surrounded by a black non-FOV area (or nearly black due to sensor noise). When endoscopic images are displayed in a conventional manner, the majority of the display is occupied by the non-FOV area, displaying black pixels that do not provide useful information.

[0127] To illustrate this concept, an imaging system such as an exemplary endoscopic image 300 is shown in Figure 3A. Image 300 includes an FOV portion 302 generated by the portion of the sensor that receives light from the captured scene and a non-FOV portion 304 generated by the portion of the sensor that does not receive light from the scene. Figure 2B shows two images 300 displayed side by side on an exemplary display 350. As shown, a relatively large amount of display space is wasted due to the non-FOV portions of the two images. In some embodiments, the hub 102 can crop part or all of the non-FOV portion of the received image data.

[0128] Returning to Figure 2, in step 202, the first image data generated by the first medical imaging device is received by the medical imaging data processing hub. The first image data may be an image or a video frame and may include an FOV portion and a non-FOV portion. For example, the first image data may be a video frame generated by an endoscope camera, such as image 300 in Figure 3A. The frame may include an FOV portion generated by pixels of one or more sensors that receive light from an captured scene incident on one or more camera sensors, and a non-FOV portion generated by pixels of one or more camera sensors that do not receive light from the captured scene.

[0129] In step 204, the non-FOV portion of the first image data is identified. According to some embodiments, the non-FOV portion may be identified based on one or more predetermined parameters related to the FOV portion. Examples of predetermined parameters include the center of the FOV portion, the radius or diameter of the FOV portion, and the pixel position related to the FOV portion or non-FOV portion. Pixels outside the region defined by the predetermined parameters may be identified as non-FOV portions.

[0130] In some embodiments, parameters related to the FOV portion of data received from a connected device may be determined once and reused as new image data is received from the connected device to identify the non-FOV portion of the data received from the connected device. For example, the center and diameter (or radius) of the FOV portion may be determined based on image data received from the connected device, and this center and diameter (or radius) may be used to identify the non-FOV portion in future image data received from the device. In some embodiments, one or more edge detection algorithms may be used to detect the edges of the FOV portion, and the edge data may be used to identify the non-FOV portion of the image, or to determine the center and diameter (or radius) of the FOV portion, and thus to identify the non-FOV portion of the image.

[0131] In some embodiments, one or more parameters related to the FOV portion of image data received from a connected device are determined during the imaging session initialization phase of the connected device, in which an image is captured having a sharp boundary between the FOV portion and the non-FOV portion. This initialization phase may be a white balance phase, for example, in which the image sensor is directed towards a white surface, and the image sensor and / or associated light source are allowed to adjust one or more imaging parameters, such as gain and light intensity, based on the amount of light received from the white surface. During the white balance phase, the FOV portion of the image data generated by the image sensor directed towards the white background is relatively bright, and therefore has high contrast with the non-FOV portion, which is black, and exhibits sharp edges that can be easily detected using one or more edge detection algorithms.

[0132] In some embodiments, the medical imaging data processing hub receives an instruction from the connected device that the connected device is in an initialization phase, such as the white balance phase. In response to receiving this instruction, the medical imaging data processing hub performs edge detection processing to determine the position of the field of view (FOV) portion of the image data received from the connected device. The determined FOV portion position (e.g., center, diameter, pixel position, etc.) may be used to identify the non-FOV portion of the image data received thereafter.

[0133] In some embodiments, non-FOV areas may be identified by detecting the periphery of the FOV area for each received image or frame. In some embodiments, the periphery of the FOV area in the first image data may be detected, for example, using one or more edge detection algorithms.

[0134] In step 206, cropped first image data is generated by removing at least a portion of the non-FOV portion of the first image data. The one or more non-FOV portions to be removed may be selected based on any suitable cropping criterion, including a desired aspect ratio of the cropped image or a predefined size of the cropped image. For example, the cropping criterion may specify that the cropped image should be square, and based on this criterion, non-FOV portions outside the square containing the FOV portion are removed, resulting in a square cropped image. Alternatively, the cropping criterion may specify an aspect ratio, and based on this criterion, non-FOV portions of a rectangle containing the FOV portion are removed, resulting in a cropped image having the specified aspect ratio.

[0135] In some embodiments, the one or more cropping criteria used in step 206 may be based on one or more properties of the connected display. For example, the dimensions of the display may be used to determine the boundaries of the cropped image. The display dimensions may be divided into display sections, and the dimensions of the display sections may determine the boundaries of the cropped image. For example, in the exemplary display of Figure 2D, the first display section 220 may be sized such that the image to be displayed in section 220 is cropped to the width of the FOV portion of the image to be displayed in section 220, while the image to be displayed in the second section 222 may be cropped to the height of the FOV portion of the image to be displayed in section 222.

[0136] In some embodiments, the medical imaging data processing hub may receive information about the display area (i.e., pixel dimensions, spatial dimensions, etc.) from a connected display. In other embodiments, one or more display area parameters are user-defined.

[0137] In step 208, a display feed is generated based on the cropped first image data. The display feed is transmitted via one or more display connections to one or more connected displays, such as the display 106 of the system 100, and the cropped first image data is displayed on the displays. In some embodiments, the cropped first image may be displayed in a first portion of the display, and additional information may be displayed in a second portion of the display. Examples of additional information that may be displayed include one or more images, videos, patient data and / or patient metadata, the status of connected devices, metrics related to imaging, or any other connected device-related information, one or more graphs, etc. According to some embodiments, cropping the first image data can reduce the space occupied by the first portion of the display, thereby increasing the amount of display space available for displaying additional information. According to some embodiments, the cropped image data may be shown in a portion of the screen having the same height and / or width as the portion showing the uncropped image data, but cropping the image data can increase the field of view on the display.

[0138] In some embodiments, image data may be received from multiple connected devices, and image data from each connected device may be cropped according to method 200 described above. A display feed may be generated to display multiple cropped images on one or more connected displays. In some embodiments, additional information may be displayed along with one or more cropped images. The additional information may be based on data received from one or more connected devices. For example, a pneumoperitoneum system connected to a medical imaging data processing hub may transmit pneumoperitoneum pressure readings to the system, which may be combined with the cropped endoscopic images in the display feed for display on a display alongside the cropped endoscopic images.

[0139] Figure 3C shows an exemplary display 350 of Figure 3B having two cropped images 310 generated according to method 200. The cropped image 310 includes the FOV portion 302 of image 300 in Figure 3B, with the non-FOV portion removed. As illustrated, cropping an image allows it to be displayed larger. Cropping an image can also provide space for additional information to be displayed. For example, in Figure 3C, the cropped image 310 occupies the first portion 320 of the display 350, the second cropped image 326 occupies the second portion 322 of the display 350, and the exemplary graph 328 occupies the third portion 324 of the display screen. Thus, a medical imaging processing system such as hub 102 can make maximum use of the display screen for displaying medical imaging data and other information.

[0140] Figure 4 is a block diagram of a medical imaging data processing hub 400 according to one embodiment, which may be used in a medical imaging system such as system 100 in Figure 1, and is for processing multiple data streams from connected medical devices such as imaging devices and generating an optimized display layout for displaying useful information to a user such as a surgeon during a medical procedure. The hub 400 includes one or more input connections 402 for receiving data from connected devices. The hub 400 includes one or more outputs 404 for connecting to one or more display devices. The hub 400 includes a primary processing unit 406 that processes at least a portion of the data received from connected devices and generates a display feed for output to one or more connected displays.

[0141] The hub 400 includes a primary processing unit 406 for managing the processing of imaging data and generating a display feed using the processed data, a reconfigurable hardware processor 408 for processing the imaging data stream, and an auxiliary processing unit 410 for providing software-based processing of imaging data and other data.

[0142] The reconfigurable hardware processor 408 may be a field-programmable gate array (FPGA) that can be reconfigured by loading hardware logic files that define circuit connections within the FPGA. The reconfigurable hardware processor 408 can be repeatedly reconfigured to provide low-latency, high-bandwidth processing of imaging data and to provide different processing of imaging data for different imaging sessions. By leveraging the reconfigurable hardware processor, the hub 400 can provide enhanced imaging data, such as video, in real time during an imaging session with little to no delay between capturing the imaging data and displaying the enhanced imaging image on a connected display. In some embodiments, the reconfigurable hardware processor 408 is a reconfigurable GPU. The primary processing unit 406 and the auxiliary processing unit 410 may each be any suitable processor or combination of processors, such as a central processing unit, graphics processing unit, microcontroller, ASIC, FPGA, or any combination thereof.

[0143] The hub 400 includes memory 412, which may be local memory located within the hub 400, or remote memory accessible to the hub 400 via a network connection. One or more portions of memory 412 may be local, and one or more portions may be remote. Memory 412 may include one or more configuration files 414 specifying the configuration of the hub 400 for different imaging sessions, one or more software programs executed by the primary processing unit 406 and / or auxiliary processing unit 410, and one or more hardware logical files 418 for reconfiguring a reconfigurable hardware processor 408. The primary processing unit 406 may access the configuration file 414 to determine the processing requirements specified in the configuration file, load the hardware logical file 418 into the reconfigurable hardware processor 408 as defined by the configuration file, or load the software program 416 into the auxiliary processing unit 410 as specified in the configuration file 414. Thus, data stored in memory 412 can be used to configure the hub 400 for different imaging sessions.

[0144] The reconfigurable hardware processor 408 is communicatively connected to the primary processing unit 406. The primary processing unit 406 may send a video data stream to the reconfigurable hardware processor 408 for processing and may receive processed video from the reconfigurable hardware processor 408 for inclusion in the display feed. The primary processing unit may load a hardware logical file into the reconfigurable hardware processor 408 in order to reconfigure it.

[0145] The auxiliary processing unit 410 is communicatively coupled to the primary processing unit 406. The primary processing unit 406 may send data to the auxiliary processing unit 410 for processing and receive the processing results for inclusion in the display feed. The primary processing unit 406 may load software into the auxiliary processing unit 410 for processing the imaging data.

[0146] The hub 400 is configured to combine information received from multiple connected devices into a display feed for display on a connected display. Therefore, multiple information sources can be displayed simultaneously on the connected display. The hub 400 is configured to synthesize information received from connected devices according to a predefined layout. For example, the hub 400 may generate a display feed in which a first video stream is displayed in a first display section, a second video stream in a second display section, and additional information such as data, alerts, device status, and metrics is displayed in a third display section.

[0147] According to some embodiments, the primary processing unit 406 is responsible for receiving data from connected devices and piecing the data together into a composite display feed. The primary processing unit 406 may utilize a reconfigurable hardware processor 408 and / or an auxiliary processing unit 410 to process the received data to enhance the display of the data.

[0148] The primary processing unit 406 connects the information source to the display feed according to one or more predefined display layouts that specify the type of imaging information to be displayed and the relative size and position of the imaging information and other information for display. The predefined display layouts may be associated with different types of imaging sessions, such as different types of surgical sessions or different types of surgical or other medical procedures. Different types of procedures may include different types of imaging devices and / or different types of imaging processing algorithms, and the predefined display layouts may specify the type of information for display relating to a given procedure. The predefined display layouts may be associated with different practitioners depending on the practitioner's preference tendencies. For example, the same information may be displayed in different ways to two different practitioners performing the same procedure. The predefined display layouts may be stored in memory 412 as a configuration data file 414.

[0149] Figure 5A shows an example of a first predefined display layout 500, and Figure 5B shows an example of a second predefined display layout 520. The first layout 500 includes three sections for three different sources 502, 504, and 506. The term "source" refers to a clear data output generated by the hub 400. A source may include data received from one or more connected devices, enhanced data generated by processing data received from one or more connected devices, or any combination thereof. Multiple sources may include, or be based on, the same data received from connected devices. For example, a first source may include a video stream received from a connected device, and a second source may include the same video stream enhanced with information extracted from that video stream or information received from another connected device.

[0150] A predefined display layout defines the source to be displayed, as well as the relative positions and sizes of different sources on the display. For example, in the first layout 500, the first source 502 is positioned above the second source 504 in the left half of the display, and both sources 502 and 504 are of equal size. The third source 506 is positioned in the right half of the display and is larger than the first and second sources. In contrast, layout 520 includes six different sources of equal size arranged in two rows of three columns. Layout 520 includes the first, second, and third sources 502, 504, and 506 in addition to the three other sources. The first and second sources 502 and 504 are in different positions relative to layout 500, and the third source 506 is of a different size relative to layout 500. Layout 500 may be associated with a first practitioner who configured layout 500 according to their preference tendencies, and layout 520 may be associated with a second practitioner. Layouts 500 and 520 may be associated with different types of imaging sessions, such as different types of surgical procedures, or they may be associated with the same type of surgical procedure. In some embodiments, both layouts are used in the same imaging session. For example, layout 500 may define a layout for a first display of the imaging system, and layout 520 may define a layout for a second display of the imaging system.

[0151] Hub 400 may configure the display feed according to one or more parameters related to the imaging session. Hub 400 may be used for multiple different types of medical procedures and / or by multiple different practitioners. As used herein, a medical procedure may refer to a single procedure (e.g., surgery) involving various tasks performed by a practitioner (e.g., a surgeon), or multiple procedures performed in a single session with a patient (e.g., a single surgical session with a patient). For example, an orthopedic surgical session in which an orthopedic procedure (e.g., perforation and / or implantation of a medical device) is performed together with an imaging procedure (e.g., to visualize tissue space and / or blood flow / tissue perfusion) may be a single medical procedure or multiple medical procedures. Different types of medical procedures may utilize different types of image sensors and other equipment. A display layout designed for one type of procedure may not be as suitable for another type of procedure. Furthermore, different practitioners may have different preference tendencies regarding what types of information should be displayed and how the information should be displayed. Therefore, the hub 400 may process the received data and generate separate display feeds based on the specific requirements and priority trends of each medical imaging session.

[0152] The hub 400 may configure the processing of input data and the generation of display feeds based on one or more predefined configurations. The predefined configurations may be associated with one or more parameters of the imaging session. Examples of imaging session parameters may include the user (e.g., practitioner), the treatment type, information related to one or more connected input devices, and information related to one or more connected output devices.

[0153] Hub 400 may receive user input specifying one or more parameter values ​​(such as via the user interface 114 in Figure 1) and select a predefined configuration based on those one or more parameter values. Based on the selected predefined configuration, Hub 400 reconfigures the processing of one or more inputs and the generation of one or more display feeds.

[0154] A predefined configuration may define a predefined display layout, or it may define one or more data processing algorithms. The algorithms may be implemented, for example, in a reconfigurable hardware processor 408 and / or an auxiliary processing unit 410. In some embodiments, the reconfigurable hardware processor 408 may be reconfigured according to the predefined configuration to perform imaging data processing specified by the predefined configuration.

[0155] As described above, different sources can be included in different layouts. Different sources may be data from different connected devices, or they may be different information extracted from the same connected device. Depending on the connected devices and layout preferences, the hub 400 may automatically reconfigure the processing of data received from connected devices according to the requirements specified in the configuration data associated with the imaging session, in order to facilitate the generation of different data for each imaging session.

[0156] The hub 400 may receive instructions for an imaging session associated with a predefined configuration and automatically configure the processing of input data and the generation of a display feed accordingly. For example, in preparation for a surgical session, a nurse may input one or more parameters related to the surgical session to the hub 400, such as via a keyboard, mouse, touchscreen, or other input device, and the hub 400 may configure itself accordingly, which may include reconfiguring a reconfigurable hardware processor by loading one or more hardware logical files stored in memory 412 and loading one or more software programs or modules on the auxiliary processing unit 410. The parameters may include the type of surgical procedure to be performed and the surgeon performing the surgical procedure. One or more predefined layouts may be associated with the type of surgical procedure and / or the surgeon, and the hub 400 may configure itself to generate a display feed according to the predefined layout.

[0157] Figure 6 shows an example of a medical imaging processing hub, such as hub 400, configured for a first imaging session. The configured hub 600 includes a primary processor 602, a reconfigurable hardware processor 604, and an auxiliary processor 606. Hub 600 includes a number of data inputs 608, three of which are connected to three different devices (630, 632, and 634), which may be a camera, camera control unit, instrument control unit, lighting control unit, insufflation device, cauterizer, or any other device or system used during the imaging session to generate data related to the imaging session. Hub 600 includes a number of video outputs 610. In the illustrated embodiment, two displays 640 and 642 are connected to two of the video outputs 610.

[0158] In the first configuration, one or more algorithms are loaded into a reconfigurable hardware processor 604 to process the imaging data received from the apparatus 632. The reconfigurable hardware processor 604 processes the data received from the apparatus 632 and transmits the processed data to the primary processor 602. The reconfigurable hardware processor 604 may receive the imaging data directly from the input 608 or it may receive the data via the primary processor 602. In some embodiments, the primary processor 602 may crop the image data according to the principles described above with respect to method 200 in Figure 2 and provide the cropped image data to the reconfigurable hardware processor 604 and / or auxiliary processor 606. This may be advantageous in reducing the amount of imaging data that needs to be processed.

[0159] The auxiliary processor 606 executes a software-based program for processing data from the third connected device 634. The auxiliary processor 606 may output the processing results to the primary processor 602 via a video output 612, such as a video output on the motherboard for the CPU.

[0160] The primary processor 602 is responsible for combining different data sources into a display feed for transmission to the connected display 640. The primary processor combines processed data from the reconfigurable hardware processor 604, the auxiliary processor 606, and directly from the first connected device 630. For example, the primary processor may generate a display feed that places these three sources into different sections of the display.

[0161] In some embodiments, the primary processor 602 may be configured to provide multiple different display streams. In the illustrated embodiment, the primary processor 602 includes two compositors 614 and 616 that can generate two different display feeds according to configuration data stored, for example, in memory. The first compositor 614 is configured to combine data from a first connected device, a reconfigurable hardware processor 604, and an auxiliary processor 606 into a first display feed for transmission to the display 640. The second compositor 616 receives data input (e.g., video input) from device 634 and generates a second display feed for transmission to the display 642.

[0162] The compositor may vary the data source merging and display feed generation for each imaging session. The compositor's handling of data may be modified based on configuration data stored in memory. For example, the data processing shown in Figure 6A may be defined by a first configuration file. A second configuration file may specify this change in data processing, for example, by specifying that for display on the first display 640, the first compositor 614 generates a display feed based only on data from the first device 630, and for display on the second display 642, the second compositor 616 generates a display feed based on data from devices 632 and 634. Different configurations may be associated with different types of procedures and / or different practitioners, for example, to support different imaging sessions.

[0163] Figure 7 shows a method 700 for configuring a medical imaging processing system, such as a hub 400, according to several embodiments. As will be further described below, method 700 includes reconfiguring a reconfigurable hardware processor, such as the reconfigurable hardware processor of the hub 400, according to predefined configuration data associated with a medical imaging session. The reconfigurable hardware processor is configured to implement an imaging data processing algorithm defined by the configuration data. By utilizing a reconfigurable hardware processor to implement an imaging processing algorithm, it becomes possible to configure the hardware processor to be tailored to processing imaging data according to a specified algorithm, and to implement different algorithms for different imaging sessions with different imaging inputs and / or display requirements. Thus, a reconfigurable hardware processor can offer advantages over a general-purpose processor that runs software-based algorithms, which may not be able to provide the low latency and high bandwidth that a reconfigurable processor can provide, which is important for providing real-time processing of images for display during medical procedures.

[0164] In step 702, a reconfigurable hardware processor, such as the reconfigurable hardware processor 408 of the hub 400, is configured for a first configuration for a first medical imaging session. The configuration of the hardware processor may be based on first configuration data stored in memory. The first configuration data may define one or more medical imaging processing algorithms for the reconfigurable hardware processor to implement. Once configured for the first configuration, the reconfigurable hardware processor implements one or more medical imaging processing algorithms as defined in the configuration data. The reconfigurable hardware processor may also be configured by loading one or more hardware logical files from memory into the reconfigurable hardware processor (which may be processed by a second processor, such as the primary processing unit 406 of the hub 400).

[0165] The first imaging session may include the performance of one or more medical procedures on the patient, such as surgical procedures. The first imaging session may begin with a nurse or other user initializing the medical imaging system for a medical procedure or a series of medical procedures on the patient. The first medical imaging session may end when all of the medical procedures or a series of medical procedures on the patient are completed, or when the first of the series of procedures on the patient is completed. As an example of the latter scenario, the first medical procedure on the patient, such as a first surgical procedure, may be completed, thereby completing the first imaging session, followed by a second medical procedure, such as a different surgical procedure, performed by the same or a different surgeon. The second medical procedure may include a second imaging session.

[0166] The reconfigurable hardware processor, in its first configuration, is configured to receive medical imaging data and process at least a portion of the data using a first imaging processing algorithm. In some embodiments, the first configuration may include the ability to process the received data using one or more additional processing algorithms. The reconfigurable hardware processor in the first configuration may have the ability to process multiple different sets of imaging data (e.g., generated by different devices) using the first imaging processing algorithm and / or additional imaging processing algorithms. For example, the reconfigurable hardware processor in the first configuration may receive a first dataset generated by a first connected device and process the first dataset using a first medical imaging processing algorithm, or it may receive a second dataset generated by a second connected device and process the second dataset using a second medical imaging processing algorithm.

[0167] In some embodiments, a reconfigurable hardware processor is configured in response to an input indicating a first medical imaging session. For example, a user, such as a nurse in an operating room, may provide the medical imaging processing system with information specifying parameters related to the imaging session, such as the type of medical procedure and / or the status of the practitioner. One or more parameters may be associated with first configuration data, and the system may access the first configuration data and configure a reconfigurable hardware processor according to the specifications of the first configuration data. User input indicating a first medical imaging session may include user selection of a profile. A profile may be associated with one or more types of medical procedures and may define data processing and display layouts tailored to one or more types of medical procedures. Types of medical procedures may include endoscopic medical procedures such as endoscopy, colonoscopy, sigmoidoscopy, proctoscopy, rhinoscopy, otoscopy, cystoscopy, vaginoscopy, arthroscopy, and thoracoscopy, and surgical procedures such as biopsy, carotid endarterectomy, cholecystectomy, coronary artery bypass surgery, skin grafting, hysterectomy, and mastectomy.

[0168] The profile may be a practitioner profile that defines the type of data the practitioner wants to view in a preferred layout. The profile may also be a default profile that includes a predefined layout and predefined data processing. The default profile may be based on one or more parameters of the medical imaging system detected by the medical imaging processing system, such as the number and type of inputs to the processing system and the number and type of display outputs from the processing system. In some embodiments, the default profile may be based on one or more parameters detected from the received image data, such as the radius or diameter of the FOV, which may be associated with the type of image sensor (e.g., endoscope size).

[0169] In step 704, the system receives first medical imaging data generated during the first medical imaging session. The first medical imaging data is received from one or more devices connected to one or more inputs of the system. For example, the first medical imaging data may be a series of video frames received from an image sensor, such as an endoscope image sensor. The first medical imaging data may include data from multiple devices connected to the system, such as multiple video feeds from multiple image sensors.

[0170] In step 706, the enhanced first medical imaging data is generated by at least partially processing the first medical imaging data using a first medical imaging processing algorithm implemented by a reconfigurable hardware processor in a first configuration. The reconfigurable hardware processor processes at least a portion of the first medical imaging data using the first medical imaging processing algorithm and any other algorithms configured to be implemented by the reconfigurable hardware processor, as defined by the first configuration data. For example, in the first configuration, the reconfigurable hardware processor may implement a smoke detection algorithm that detects portions of the received image related to smoke in the field of view and enhances the received image to reduce the appearance of smoke.

[0171] Some or all of the first medical imaging data may be routed to a reconfigurable hardware processor by a primary processor, such as the primary processing unit 406 of the hub 400. The primary processor may receive the first medical imaging data and route the data to a reconfigurable processor according to first configuration data. The first configuration data may specify that the data received from the connected device should be processed using at least a first medical imaging processing algorithm. In accordance with this requirement, the primary processor may direct the data received from the connected device to the reconfigurable hardware processor. In some embodiments, the reconfigurable processor receives the first medical imaging data directly from the input connection to the connected device, i.e., without the data being routed through one or more additional processing units first.

[0172] The processing of data by the reconfigurable hardware processor may be based on processing by other processing units of the system. For example, the processing of first medical imaging data by the first medical imaging processing algorithm may be based on information received from a second processing unit. The second processing unit may analyze part or all of the first medical imaging data, and the results of this analysis may be used by the reconfigurable hardware processor in the implementation of the first medical imaging processing algorithm. For example, in an embodiment in which a smoke detection algorithm is implemented in the reconfigurable processor described above, an auxiliary processing unit such as the auxiliary processing unit 410 of the hub 400 may receive part or all of the first imaging data and determine whether or not smoke is present in the captured field of view. If smoke is detected, the auxiliary processing unit may notify the reconfigurable hardware processor (directly or via another processing unit such as the primary processing unit 406), and in response, the reconfigurable hardware processor may start processing the first imaging data to reduce the contribution of smoke to the data.

[0173] In step 708, enhanced first medical imaging data generated by a reconfigurable hardware processor is displayed for observation during the first medical imaging session. For example, the first medical imaging session may include an endoscopic procedure involving the use of a cauterizing tool, and the enhanced first medical imaging data may be an enhanced image feed generated by an endoscopic camera, with reduced appearance of smoke generated by the cauterizing tool. This enhanced image may be displayed to the surgeon in real time to allow the surgeon to better visualize the surgical field.

[0174] In some embodiments, the enhanced first medical imaging data is received from a hardware processor reconfigurable by another processing unit, such as the primary processing unit 406 of the hub 400. The primary processor may generate one or more display feeds containing the enhanced first medical imaging data. The primary processor may generate one or more display feeds based at least in part on the first configuration data. For example, the primary processor may combine the enhanced first medical imaging data with additional information, such as additional imaging images received from another connected device, for display on different parts of a display defined by the first configuration data.

[0175] In some embodiments, the display feed includes enhanced first medical imaging data combined with other data. For example, the display feed may include enhanced first medical imaging data for display on a first portion of a connected display and additional information for display on a second portion of the connected display. In some embodiments, the system generates multiple display feeds having different display configurations for displaying the enhanced imaging data and provides different display feeds to different displays.

[0176] Figure 8A is a block diagram of a medical imaging processing system 800 showing steps 702 to 708 of Method 700 according to one embodiment. A reconfigurable hardware processor 804 is configured to process imaging data received from a white light image sensor 810 via a first input port 808 using a first imaging processing algorithm, such as a smoke detection and removal algorithm. In response to user input related to a first imaging session, the primary processor 802 accesses first configuration data stored in memory 812, and based on the specifications of the first configuration data, the primary processor 802 reconfigures the reconfigurable hardware processor 804 by loading a hardware logic configuration file for a smoke reduction algorithm. Furthermore, the primary processor 802 loads a smoke detection software program or module from memory 812 to an auxiliary processor 806. The auxiliary processor 806, running the smoke detection software program or module, can detect the presence of smoke in the received imaging data and can instruct the reconfigurable hardware processor 804 to process the imaging data to reduce the effects of smoke in the imaging data. Prior to smoke detection by the auxiliary processor 806, the reconfigurable hardware processor may simply pass the image data for display without first processing the data for smoke removal. The primary processor may periodically provide the auxiliary processor 806 with one or more portions (one or more frames, etc.) of the received image data to detect smoke and trigger smoke removal processing by the reconfigurable hardware processor 804. When smoke is detected in the image data, the system 800 outputs a display feed to the display 816, which includes the image data received from the white light image sensor 810, enhanced by removing the contribution from the smoke.

[0177] Returning to method 700, in step 710, the reconfigurable hardware processor is reconfigured into a second configuration for a second medical imaging session based on second configuration data stored in memory. The second configuration implements at least one medical imaging processing algorithm that is not implemented in the first configuration.

[0178] In some embodiments, a reconfigurable hardware processor is reconfigured in response to an input indicating a second medical imaging session. For example, a user, such as a surgical nurse, may provide the medical imaging processing system with information specifying parameters related to the second imaging session, such as the type of medical procedure and / or the practitioner's identity. One or more parameters may be associated with second configuration data, and the system may access the second configuration data and configure the reconfigurable hardware processor according to the specifications of the second configuration data. User input indicating a second medical imaging session may include user selection of a profile, as described above. Depending on the selected profile, the second imaging session may be associated with the same practitioner as the first imaging session—for example, when the same practitioner transitions from one type of medical procedure to another type of medical procedure that may require a different display layout due to different equipment or different extension algorithms connected to the medical imaging processing system. The second imaging session may be associated with a different practitioner, even if it is the same type of medical procedure. For example, a first surgeon may perform a certain type of surgery (e.g., cholecystectomy) on a first patient during a first imaging session, and a second surgeon may perform the same type of surgery (e.g., cholecystectomy) on a second patient (for example, on the same day or the following day).

[0179] The first imaging session is completed (for example, the surgery or procedure associated with the first imaging session is completed), and the imaging system may be set up for use in a subsequent second imaging session. The second imaging session may include one or more different types of procedures and / or one or more different users for whom different imaging processing may be beneficial. Thus, the second configuration implements one or more image processing algorithms that were not implemented in the first imaging session. A reconfigurable hardware processor is reconfigured to implement one or more image processing algorithms required for the second imaging session, as defined by the second configuration data.

[0180] In some embodiments, the second imaging session may be a second surgical session in which the imaging system is to be used. After the completion of the first imaging session, the operating room may be set up for the second surgical session. The second surgical session may include a different type of surgery, a different practitioner, a different patient, etc. During the setup of the second surgical session, the imaging processing system may receive an input indicating the second surgical session. The input may be, for example, a selection of the type of surgical procedure, or a selection of a profile (e.g., a practitioner profile) made via a user interface to the imaging processing system. Based on this selection, the system may automatically reconfigure a reconfigurable processor based on configuration data associated with the second surgical session.

[0181] In step 712, the medical imaging processing system receives second medical imaging data generated during the second medical imaging session. This second medical imaging data may be received from the same connected device or group of devices as the first medical imaging data, or from a different connected device or group of devices.

[0182] In step 714, the enhanced second medical imaging data is generated by at least partially processing the second medical imaging data using a second medical imaging processing algorithm implemented in a second configuration of the reconfigurable hardware processor. The reconfigurable hardware processor processes at least a portion of the first medical imaging data using a second medical imaging processing algorithm (not implemented in the first configuration) and other algorithms configured to be implemented by the reconfigurable hardware processor, which may or may not be implemented in the first configuration, as defined by the second configuration data. For example, in the second configuration, the reconfigurable hardware processor may implement an algorithm that processes a fluorescence image (e.g., a video frame) to determine one or more features of blood flow through the tissue, such as tissue perfusion, vascular location, blood flow volume or velocity, dimensions of the imaged tissue, or any combination thereof, and then modifies the fluorescence image according to the determined features (e.g., correcting image coloration, overlaying data on the image, overlaying contours on the image), thereby generating an enhanced imaging image.

[0183] Similar to the first configuration described above, some or all of the second medical imaging data may be routed to a reconfigurable hardware processor by a primary processor, such as the primary processing unit 406 of the hub 400. The primary processor may receive the second medical imaging data and route the data to a reconfigurable processor according to the second configuration data. The second configuration data may specify that the data received from the connected device should be processed using at least the second medical imaging processing algorithm. In accordance with this requirement, the primary processor may direct the data received from the connected device to a reconfigurable hardware processor. In some embodiments, the reconfigurable processor receives the first medical imaging data directly from the input, i.e., without the data being routed through one or more additional processing units first.

[0184] In step 716, the enhanced second medical imaging data generated by the reconfigurable hardware processor is displayed for observation during the second medical imaging session. Displaying the enhanced second medical imaging data can assist the surgeon or other practitioner during one or more procedures performed during the second medical imaging session. By leveraging the low latency and high bandwidth of the reconfigurable processor, the enhanced second medical imaging data can be displayed in real time.

[0185] In some embodiments, the enhanced second medical image data may be transmitted from a reconfigurable hardware processor to another processing unit, such as the primary processing unit 406 in Figure 4, which can generate a display feed containing the enhanced second medical imaging data. The display feed may be transmitted by the primary processor to one or more connected displays. In some embodiments, the reconfigurable hardware processor may transmit the enhanced second medical imaging data directly to an output connection to one or more connected displays.

[0186] Figure 8A is a block diagram of a medical imaging system 800 illustrating steps 710-716 of Method 700 according to one embodiment. The reconfigurable hardware processor 804 is reconfigured to process imaging data received from the white light image sensor 810 and the fluorescence image sensor 818 (which may be part of the same imaging system or received at the same or different input ports) using a second imaging processing algorithm that analyzes fluorescence imaging images to characterize a portion of tissue according to the health of the tissue, the degree of blood flow in the tissue, or the degree of perfusion in the tissue, and superimposes the characterization onto the white light imaging image. In response to user input related to a second imaging session (e.g., input indicating a surgical session for a new patient, or input indicating a new procedure for the same patient as the first imaging session), the primary processor 802 reconfigures the reconfigurable hardware processor 804 by accessing second configuration data stored in memory 812 and loading a hardware logical configuration file for the tissue characterization algorithm based on the specifications of the first configuration data. Furthermore, the primary processor 802 loaded a reference marker software program or module from memory 812 into the auxiliary processor 806. The auxiliary processor 806, executing the reference marker program or module, can determine, for example, the locations of maximum and / or minimum perfusion in the fluorescence imaging data. The reference markers generated by the auxiliary processor 806 are added to an overlay generated by a reconfigurable hardware processor (which may be done by the reconfigurable hardware processor 804, the primary processor 802, or a different processor in the system). The resulting enhanced image data is output to the display 816 for visualization during a second imaging session.

[0187] According to some embodiments, a tissue characterization algorithm implemented on a reconfigurable hardware processor may provide an enhanced visual representation of a subject's tissue that is more accurate with respect to data representation and may be intuitive for clinicians to use in clinical decision-making. The generated, enhanced visual representation of tissue may be applicable to various types of tissue (e.g., various wounds including chronic, acute, pressure ulcers, and cancerous tissue) and may provide a framework for automatically classifying tissues (e.g., wound tissue, cancerous tissue) and / or predicting clinical outcomes (e.g., wound tissue healing timeline, cancerous tissue healing).

[0188] Tissue characterization algorithms may utilize machine learning or deep learning. Machine learning-based methods and systems facilitate the resolution of problems for which there are no algorithmic solutions or problems that are too complex to find solutions for. Medical diagnosis and tissue characterization based on tissue imaging are particularly well-suited tasks for machine learning algorithms due to the complex nature of the physiological processes that occur in the human body. Machine learning can help discover medically relevant features and patterns from large datasets, enabling more accurate, faster, and consistent medical diagnoses, regardless of the clinician's experience level. In some embodiments, a tissue characterization algorithm includes identifying one or more attributes of data relevant to the clinical characterization of a tissue, and classifying the data into multiple clusters based on one or more attributes of the data such that data within the same cluster is more similar to data in different clusters than data in different clusters, with the clusters representing the characteristics of the tissue. In some variations, the algorithm may further include associating each of the clusters with each of several subregions in a time-series image, such as a fluorescence image, and generating a subject space (cluster) map based on the clusters associated with the multiple subregions in the subject time-series fluorescence image. The algorithm may further include receiving multiple subject spatial maps, receiving metadata associated with each subject spatial map, storing each subject spatial map and its associated clinical data in a database record, and using the database records as input to a supervised machine learning algorithm for generating a predictive model. The predictive model may be used to predict clinical data associated with a subject's time-series fluorescence images.

[0189] Figures 9A and 9B show a graphical user interface for configuring a medical imaging processing system, such as a hub 400, for a new imaging session. The user interface may be provided, for example, on a tablet connected to the system or on the system's touchscreen. The user interface 900 in Figure 9A allows the user to configure the imaging processing system by selecting a specialty 902, a procedure 904, and / or a practitioner 906. Each selection may be associated with a different configuration, or a combination of selections may be associated with a configuration. For example, each practitioner selection may be associated with a different configuration previously specified by the practitioner, while both a specialty and a procedure selection may be required to select a configuration. The configurations may be stored locally in the system's memory or remotely in a hospital information system, accessed, for example, via a network connection.

[0190] Figure 9B shows a user interface 910 for defining data sources and data source layouts. Two display layouts (912 and 914) are associated with the illustrated configuration. Each display layout defines the data sources and their size and position. The first display layout 912 includes three different sources. As described above, a source can define the type of data to be displayed, which can be based on both the system that generates the data and the type of processing performed on the data by, for example, a reconfigurable processor and / or other system modules. Thus, different sources may be based on data from the same imaging system or other devices. For example, source 1 may be a still image from an input video stream (selected, for example, via a voice command from the practitioner), and source 3 may be a video stream. The user interface 910 may allow the user to select, arrange, and resize different sources. For example, the user may select available sources from a drop-down list that specifies all sources that the system can generate, or all sources that the system can generate by receiving input to the system. Users can rearrange sources by dragging source icons around the screen, or resize sources using gestures, mouse input, keyboard input, or other appropriate input methods.

[0191] Once the user has completed selecting a configuration profile, the medical imaging processing system can automatically configure itself according to the requirements defined in the selected configuration profile, in accordance with the method described above.

[0192] A system for collecting, enhancing, and displaying medical imaging data, such as system 100 in Figure 1, may include one or more imaging systems for acquiring time-series images of tissue (e.g., time-series fluorescence images, time-series white light images, etc.). In some embodiments, the imaging system is a fluorescence imaging system. Figure 10 is a schematic example of a fluorescence imaging system 1010 according to one embodiment. The fluorescence imaging system 1010 comprises a light source 1012 that irradiates the subject's tissue to induce fluorescence emission from a fluorescent contrast agent 1014 into the subject's tissue (e.g., in blood, urine, lymph, cerebrospinal fluid, or other body fluids or tissues), an image acquisition assembly 1016 configured to generate time-series and / or subject-time-series fluorescence images from the fluorescence emission, and a processor assembly 1018 configured to process the generated time-series / subject-time-series fluorescence images according to any variation of the method described herein. The processor assembly 1018 may include a memory 1068 having instructions thereon, a processor module 1062 configured to execute instructions on the memory 1068 to process time-series and / or subject time-series fluorescence images, and a data storage module 1064 for storing raw and / or processed time-series and / or subject time-series fluorescence images. In some variations, the memory 1068 and the data storage module 1064 may be embodied on the same recording medium, while in other variations, the memory 1068 and the data storage module 1064 may be embodied on different recording media. The system 1010 may further include a communication module 1066 for transmitting some or all of the time-series / subject time-series fluorescence images or other input data, spatial maps, subject spatial maps, and / or tissue quantifiers to an imaging data processing hub such as the imaging data processing hub 102 in Figure 1, in accordance with the systems and methods described above.

[0193] In some variations, the light source 1012 includes, for example, an illumination module 1020. The illumination module 1020 may include a fluorescence excitation source configured to generate excitation light having an appropriate intensity and wavelength for exciting a fluorescent contrast agent 1014. As shown in Figure 11, the illumination module 1020 may include a laser diode 1022 (which may include, for example, one or more fiber-coupled diode lasers) configured to provide excitation light for exciting a fluorescent contrast agent (not shown) in the tissue of a subject. Other examples of sources of excitation light that may be used in various embodiments include one or more LEDs, arc lamps, or other illumination techniques having sufficient intensity and an appropriate wavelength for exciting a fluorescent contrast agent in tissue. For example, the excitation of a fluorescent contrast agent in blood (where the fluorescent contrast agent is a fluorescent dye having near-infrared excitation and emission properties) can be performed using one or more 793 nm, conduction-cooled, single-bar, fiber-coupled laser diode modules from DILAS Diode Laser Co., Germany.

[0194] In some variations, the light output from the light source 1012 may be projected through one or more optical elements to form and guide an output used to illuminate a tissue region of interest. The optical elements may include one or more lenses, optical guides and / or diffracting elements to ensure a flat field over substantially the entire field of view of the image acquisition assembly 1016. The fluorescence excitation source may be selected to emit light at a wavelength close to the absorption maximum of the fluorescent contrast agent 1014 (e.g., indocyanine green (ICG)). For example, as shown in Figure 11, the output 1024 from the laser diode 1022 may pass through one or more focusing lenses 1026 and then through a homogenizing light pipe 1028, such as a light pipe commonly available from Newport Corporation, for example. Finally, the light may pass through an optical diffracting element 1032 (i.e., one or more light diffusers), such as a ground glass diffracting element, for example, available from Newport Corporation, for example. Power to the laser diode 1022 may be supplied by a high-current laser driver, such as one available from Lumina Power Inc., for example. The laser may optionally operate in pulse mode during the image acquisition process. An optical sensor, such as a solid-state photodiode 1030, may be incorporated into the illumination module 1020 to sample the illumination intensity generated by the illumination module 1020 via scattered or diffuse reflection from various optical elements. In some variations, an additional illumination source may be used to provide guidance when aligning and positioning the module over the region of interest.

[0195] Referring again to Figure 10, in some modifications, the image acquisition assembly 1016 may be a component of a fluorescence imaging system 1010 configured to acquire time-series and / or subject-time-series fluorescence images from fluorescence emission from a fluorescent contrast agent 1014. The image acquisition assembly 1016 may also include a camera module 1040. As shown in Figure 12, the camera module 1040 may acquire an image of fluorescence emission 1042 from a fluorescent contrast agent in tissue by using a system of imaging optics (e.g., 1046a, 1046b, 1048, and 1050) to collect the fluorescence emission and focus it onto an image sensor assembly 1044. The image sensor assembly 1044 may include at least one 2D solid-state image sensor. The solid-state image sensor may be a charge-coupled device (CCD), a CMOS sensor, a CID, or similar 2D sensor technology. The charge generated from the optical signal converted by the imaging sensor assembly 1044 is converted into an electrical video signal, which includes both digital and analog video signals, by appropriate readout and amplification electronics within the camera module 1040.

[0196] In an exemplary modification of the fluorescence imaging system, the light source may provide an excitation wavelength of approximately 800 nm + / - 10 nm, and the image acquisition assembly may use an emission wavelength greater than 820 nm with an NIR-compatible optical system, for example, for ICG fluorescence imaging. In an exemplary embodiment, the NIR-compatible optical system may include a CCD monochrome imaging sensor having a GigE standard interface and a lens compatible with the sensor in terms of optical format and mount format (e.g., C / CS mount).

[0197] In some variations, the processor module 1062 includes any computer or computing means, such as a tablet, laptop, desktop, network computer, or dedicated standalone microprocessor. For example, the processor module 1062 may include one or more central processing units (CPUs). In an exemplary embodiment, the processor module 1062 is a quad-core 2.5GHz processor having four CPUs, each CPU being a microprocessor such as a 64-bit microprocessor (e.g., commercially available as an INTEL Core i3, i5, or i7, or an AMD Core FX series). However, in other embodiments, the processor module 1062 may be any suitable number of CPUs and / or any suitable processor having other suitable clock speeds.

[0198] Inputs for the processor module 1062 may be obtained, for example, from the imaging sensor 1044 of the camera module 1040 shown in Figure 12, from the solid-state photodiode 1030 of the illumination module 1020 shown in Figure 11, and / or from any external control hardware such as a footswitch or remote control. Outputs are provided to the laser diode driver and optical alignment aid. In some modifications, as shown in Figure 10, the processor assembly 1018 may have a data storage module 1064 that has the function of storing time-series / subject time-series images, or data indicating them, or other input data, in a tangible, non-temporary computer-readable medium such as internal memory (e.g., hard disk or flash memory) to enable recording and processing of acquired data. In some modifications, the processor module 1062 may have an internal clock to enable control of various elements and to ensure the precise timing of the illumination and sensor shutters. In some modifications, the processor module 1062 may also provide user input and graphic display. The fluorescence imaging system may optionally be configured to include a communication unit 1066, such as a wired or wireless network connection or a video output connection, for transmitting time-series fluorescence images when acquiring them or when playing them back after recording. The communication unit 1066 may additionally or alternatively transmit processed data such as spatial maps, subject spatial maps, and / or tissue numerical values.

[0199] In the operation of the exemplary system shown in Figures 10-12, the subject is positioned relative to the fluorescence imaging system 1010 such that the illumination of the light source 1012 and the joules 1020 produce a substantially uniform field of illumination over substantially the entire region of interest, so that the region of interest (e.g., the target tissue region) is located below the light source 1012 and the acquisition assembly 1016. In some variations, an image of the region of interest may be acquired for background deduction purposes before administering the fluorescent contrast agent 1014 to the subject. To acquire fluorescence images / subject fluorescence images, the operator of the fluorescence imaging system 1010 may initiate the acquisition of time-series / subject time-series fluorescence images by pressing a remote switch or foot control, or via a keyboard (not shown) connected to the processor assembly 1018. As a result, the light source 1012 is turned on, and the processor assembly 1018 begins recording fluorescence image data / subject fluorescence image data provided by the image acquisition assembly 1016. When operating in pulse mode in this embodiment, the imaging sensor 1044 in the camera module 1040 is synchronized to collect fluorescence emission following laser pulses generated by the diode laser 822 in the illumination module 1020. In this way, the maximum fluorescence emission intensity is recorded and the signal-to-noise ratio is optimized. In this embodiment, the fluorescent contrast agent 1014 is administered to the subject and delivered to the region of interest via arterial flow. Acquisition of time-series / non-subject time-series fluorescence images is initiated, for example, immediately after administration of the fluorescent contrast agent 1014, and time-series fluorescence images are acquired from substantially the entire region of interest during the immersion of the fluorescent contrast agent. Fluorescence emission from the region of interest is collected by the acquisition optics of the camera module 1040. Residual ambient light and reflected excitation light are attenuated by a subsequent optical element in the camera module 1040 (e.g., optical element 1050 in Figure 12, which may be a filter), so that fluorescence emission can be acquired by the imaging sensor assembly 1044 with minimal interference from light from other light sources.

[0200] In some variations, following the acquisition or generation of time-series / subject time-series fluorescence images, the processor assembly 1018 (e.g., processor module 1062 or other processor) may then execute instructions stored in memory 1068 and begin processing the imaging data before sending it to the imaging data processing system (e.g., hub 102 of system 100). System 1010 may transmit via connection 1066 spatial maps / subject spatial maps and / or any clinical correlations or diagnoses derived therefrom, or both, for display to the user in a composite display feed, for example, as grayscale or colored images, and / or for storage for subsequent use.

[0201] Figure 13 shows an embodiment of the endoscopic surgical cart of system 100 of Figure 1. The cart 10 may be used, for example, in an operating room for endoscopic imaging and display during endoscopic surgery. The cart 10 includes an imaging system, such as the fluorescence imaging system 1010 in Figure 10. The imaging system includes a scope assembly 11 that can be used for endoscopic procedures. The scope assembly 11 contains an endoscope or scope 12 coupled to a camera head 16 by a coupler 13 located at the distal end of the camera head 16. Light is supplied to the scope by a light source 14 via an optical guide 26, such as an optical fiber cable. The camera head 16 is coupled to a camera control unit (CCU) 18 by an electrical cable 15. The CCU 18 is preferably connected to and communicates with the light source 14. The operation of the camera 16 is partially controlled by the CCU 18. The cable 15 transmits video data from the camera head 16 to the CCU 18 and transmits various control signals bidirectionally between the camera head 16 and the CCU 18. In one embodiment, the image data output by the camera head 16 is digital.

[0202] The camera head 16 is provided with a control or switch arrangement 17, allowing the user to manually control various functions of the cart 10. Voice commands may be input to a microphone 25 attached to a headset 27 worn by the surgeon and coupled to a voice control unit 23. The cart 10 may also include a handheld control device 21, such as a tablet or PDA having a touchscreen user interface, which may be coupled to the cart 10 as a further control interface. The cart 10 also includes an imaging data processing hub 31, such as the hub 102 in Figure 1 or the hub 400 in Figure 4, which is coupled to the imaging system via one or more cable connections to receive images and / or video from the imaging system, process the images and / or video, and generate a display feed for display on the display 20 according to the method described herein. The imaging data processing hub may receive user input via the voice control unit and / or through the handheld control device.

[0203] Cart 10 may include one or more additional devices 33, such as an imaging recording device or a surgical tool control device, which can be coupled to the imaging data processing hub. The imaging data processing hub 31 may receive information such as device warnings, device status, and device settings from one or more additional devices 33. In some embodiments, the additional device 33 is a video recorder, and the imaging data processing hub 31 may send one or more display feeds to the video recorder for recording.

[0204] A tangible, non-temporary computer-readable medium embedded with computer-executable (readable) program code can provide one or more processors with instructions to perform one or more of the methods described herein when executing instructions. The program code is written in any suitable programming language and can be delivered to the processor in many forms, including, but not limited to, information permanently stored in a non-writable storage medium (e.g., a read-only memory device such as a ROM or CD-ROM disk), information modifiablely stored in a writable storage medium (e.g., a hard drive), information transmitted to the processor via a communication medium such as a local area network or a public network such as the Internet, or any type of medium suitable for storing electronic instructions. When carrying computer-readable instructions that implement various embodiments of the methods described herein, such computer-readable medium represents an example of various embodiments. In various embodiments, tangible, non-temporary computer-readable medium includes all computer-readable media, and the scope of the present invention is limited to computer-readable media where the medium is both tangible and non-temporary.

[0205] The kit may include any part of the system described herein and a fluorescent contrast agent, such as a fluorescent dye like ICG or any suitable fluorescent contrast agent. In a further embodiment, the kit may include a tangible, non-temporary, computer-readable medium embedded with computer-executable (readable) program code, which, when executing instructions, can provide one or more processors with instructions to perform one or more methods for tissue characterization and / or prediction of clinical data as described herein. The kit may include instructions for using at least some of its components (e.g., instructions for using a fluorescent contrast agent, instructions for installing computer-executable (readable) program code with embedded instructions, etc.). In yet another embodiment, a fluorescent contrast agent, such as a fluorescent dye, for use in the method and system described herein is provided. In further modifications, the kit may include any part or all of the system described herein and a fluorescent contrast agent, such as a fluorescent dye like ICG or any other suitable fluorescent agent, or a combination of fluorescent agents.

[0206] Examples of contrast agents used to generate imaging data In some embodiments, in a fluorescence medical imaging application, the contrast agent is a fluorescent contrast agent such as ICG dye. When administered to a subject, ICG binds to blood proteins and circulates with the blood in the tissues. The fluorescent contrast agent (e.g., ICG) may be administered to the subject as a bolus injection (e.g., intravenous or arterial injection) at a concentration suitable for imaging so that the bolus circulates through the vascular system and passes through the microvessel system. In other embodiments where multiple fluorescent contrast agents are used, such agents may be administered simultaneously in a single bolus, for example, or sequentially in separate boluses. In some embodiments, the fluorescent contrast agent may be administered by catheter. In certain embodiments, the fluorescent contrast agent may be administered less than one hour before measuring the signal intensity produced by the fluorescent contrast agent. For example, the fluorescent contrast agent may be administered to the subject less than 30 minutes before the measurement. In yet another embodiment, the fluorescent contrast agent may be administered at least 30 seconds before the measurement is performed. In yet another embodiment, the fluorescent contrast agent may be administered simultaneously with the measurement.

[0207] According to several embodiments, the fluorescent contrast agent may be administered at various concentrations to achieve a desired circulating concentration in the blood. For example, in embodiments where the fluorescent contrast agent is ICG, it may be administered at a concentration of about 2.5 mg / mL to achieve a circulating concentration of about 5 μM to about 10 μM in the blood. In various embodiments, the upper limit for administration of the fluorescent contrast agent is the concentration at which the fluorescent contrast agent becomes clinically toxic in the circulating blood, and the lower limit is the instrument limit for acquiring signal intensity data generated from the fluorescent contrast agent circulating with the blood in order to detect the fluorescent contrast agent. In various other embodiments, the upper limit for administration of the fluorescent contrast agent is the concentration at which the fluorescent contrast agent self-quenches. For example, the circulating concentration of ICG may be in the range of about 2 μM to about 10 mM. Thus, in one embodiment, the method includes the steps of administering a contrast agent (e.g., a fluorescent contrast agent) to a subject and acquiring signal intensity data (e.g., images) before processing the signal intensity data according to various embodiments. In another embodiment, the method excludes the optional step of administering a contrast agent to the subject.

[0208] According to several embodiments, a fluorescent contrast agent suitable for use in a fluorescence imaging application for generating fluorescence image data is a contrast agent that circulates with the blood (e.g., a fluorescent dye that can circulate with blood components such as lipoproteins in the blood or serum plasma), can pass through the tissue's vascular system (i.e., large and small vessels), and produces a signal intensity when exposed to appropriate light energy (e.g., excitation or absorption light energy). In various embodiments, the fluorescent contrast agent includes a fluorescent dye, its analogues, its derivatives, or a combination thereof. The fluorescent dye includes any non-toxic fluorescent dye. In certain embodiments, the fluorescent dye optimally emits fluorescence in the near-infrared spectrum. In certain embodiments, the fluorescent dye is or includes a tricarbocyanine dye. In certain embodiments, the fluorescent dye is or includes ICG, methylene blue, or a combination thereof. In other embodiments, the fluorescent dyes are fluorescein isothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin, o-phthalaldehyde, fluorescein, rose bengal, trypan blue, fluorogold, or combinations thereof, each of which can be excited using an appropriate wavelength of excitation light for each dye. In some embodiments, analogs or derivatives of the fluorescent dyes may be used. For example, the fluorescent dye analogs or derivatives include fluorescent dyes that are chemically modified but retain the ability to fluoresce when exposed to light energy of an appropriate wavelength.

[0209] In various embodiments, the fluorescent contrast agent may be provided as a lyophilized powder, solid, or liquid. In certain embodiments, the fluorescent contrast agent may be provided in a vial (e.g., a sterile vial), which can be reconstituted to the desired concentration by administering a sterile fluid with a sterile syringe. Reconstitution can be carried out using any suitable carrier or diluent. For example, the fluorescent contrast agent may be reconstituted with an aqueous diluent immediately before administration. In various embodiments, any diluent or carrier that maintains the fluorescent contrast agent in solution can be used. As an example, ICG may be reconstituted with water. In some embodiments, once the fluorescent contrast agent is reconstituted, it may be mixed with additional diluents and carriers. In some embodiments, the fluorescent contrast agent may be bound to another molecule such as a protein, peptide, amino acid, synthetic polymer, or sugar to enhance, for example, solubility, stability, imaging properties, or a combination thereof. Additional buffers, including Tris, HCl, NaOH, phosphate buffer, and / or HEPES, may be optionally added.

[0210] Although details of fluorescent contrast agents have been described above, those skilled in the art will understand that other contrast agents may be used in connection with the systems, methods, and techniques described herein, depending on the optical imaging modality. Such fluorescent agents may be administered into body fluids (e.g., lymph, cerebrospinal fluid) or body tissues.

[0211] In some modifications, fluorescent contrast agents used in combination with the methods, systems, and kits described herein may be used for blood flow imaging, tissue perfusion imaging, lymphatic imaging, or combinations thereof, which may be performed during invasive surgical procedures, minimally invasive surgical procedures, non-invasive surgical procedures, or combinations thereof. Examples of invasive surgical procedures involving blood flow and tissue perfusion include cardiac-related surgical procedures (e.g., on-pump or off-pump CABG) and reconstructive surgical procedures. Examples of non-invasive or minimally invasive procedures include the treatment and / or management of wounds (e.g., chronic wounds such as pressure ulcers). In this regard, changes in the wound over time, such as changes in wound dimensions (e.g., diameter, area) or changes in wound and / or peri-wound tissue perfusion, may be tracked over time by the application of the methods and systems. Examples of lymphatic imaging include identification of one or more lymph nodes, lymph node drainage, lymphatic mapping, or combinations thereof. In some variations, such lymphatic imaging may be related to the female reproductive system (e.g., uterus, cervix, vulva).

[0212] In a variation relating to cardiac application, a contrast agent (e.g., ICG alone or in combination with other contrast agents) may be injected intravenously, for example, through a central venous circuit, bypass pump, and / or cardioprotective circuit, to fluidize and / or perfuse the coronary vascular system, microvascular system, and / or graft. ICG may be administered below the grafted vessel as diluted ICG / blood / saline so that the final concentration of ICG in the coronary artery is approximately the same as or less than that obtained by injecting approximately 2.5 mg (i.e., 1 ml of 2.5 mg / ml) into the central circuit or bypass pump. ICG may be prepared, for example, by dissolving 25 mg of solid in 10 ml of sterile aqueous solvent, which may be provided by the manufacturer together with ICG. 1 ml of ICG solution may be mixed with 500 ml of sterile saline (e.g., injecting 1 ml of ICG into a 500 ml saline bag). 30 ml of diluted ICG / saline may be added to 10 ml of subject blood obtained sterilely from a central venous circuit or bypass pump. ICG in the blood binds to plasma proteins, preventing leakage outside the blood vessels. Mixing of ICG and blood can be performed in a sterile surgical field using standard aseptic techniques. 10 ml of the ICG / saline / blood mixture can be administered to each graft. ICG can be administered using a syringe attached to the (open) proximal end of the graft, rather than by injecting it through the graft wall using a needle. When the graft is harvested, the surgeon routinely attaches an adapter to the proximal end of the graft so that the graft can be pressurized by attaching a syringe filled with saline, sealing the distal end of the graft, and injecting saline below the graft, allowing for assessment of the integrity of the conduit (in terms of leakage, side branches, etc.) before performing the first anastomosis. In other modifications, the methods, dosages, or combinations thereof described herein in relation to cardiac imaging may be used in any vascular and / or tissue perfusion imaging application.

[0213] Lymphatic mapping is a crucial part of effective surgical staging for cancers that spread via the lymphatic system (e.g., breast cancer, gastric cancer, gynecological cancers). While resection of multiple nodes from a specific lymphatic drainage area can lead to serious complications, including acute or chronic lymphedema, sensory disturbances, and / or seroma formation, in practice, if the sentinel lymph node is negative for metastasis, the surrounding lymph nodes are most likely to be negative as well. For example, in breast cancer surgery, identification of tumor dissipation lymph nodes (LNs) is a critical step in staging cancers that spread via the lymphatic system. LN mapping involves the use of dyes and / or radioactive tracers to identify LNs involved in either biopsy or resection, and subsequent pathological evaluation for metastasis. The goal of lymph node dissection during surgical staging is to identify and remove LNs at high risk of local cancer spread. Sentinel lymph node (SLN) mapping is emerging as an effective surgical strategy in the treatment of breast cancer. Generally, the concept is based on the idea that if metastasis (spread of cancer to the axillary lymph node) is present, it should be located in the subcutaneous lymph node (SLN). In this art, the SLN is defined as the group of nodes from which cancer cells are most likely to spread from the initial LN or primary tumor. If the SLN is negative for metastasis, the surrounding secondary and tertiary LNs should also be negative. The main advantage of SLN mapping is to reduce the number of subjects who undergo conventional partial or complete lymph node dissection, and consequently, to reduce the number of subjects who suffer from associated pathological conditions such as lymphedema and lymphocysts.

[0214] The current standard treatment for SLN mapping involves injecting a tracer to identify the lymphatic drainage pathway from the primary tumor. The tracer used may be a radioisotope (e.g., technetium-99 or Tc-99m) for intraoperative localization using a gamma probe. Radiotrace technology (known as scintigraphy) is limited to hospitals where radioisotopes are available, requires the involvement of a nuclear medicine physician, and does not provide real-time visual guidance. The colored dye isosulfan blue is also used, but this dye is not visible through the skin and adipose tissue. Furthermore, the blue staining can result in breast tattooing that lasts for several months, subcutaneous injection can cause skin necrosis, and allergic reactions with anaphylaxis have been reported in rare cases. Severe anaphylactic reactions occur after injection of isosulfan blue (about 2% of patients). Symptoms include dyspnea, shock, angioedema, urticaria, and pruritus. Reactions are more likely to occur in subjects with a history of bronchial asthma or those with allergies or drug reactions to triphenylmethane dye. Isosulfan blue is known to interfere with pulse oximetry for measuring oxygen saturation and gas analyzers for measuring methemoglobin. Use of isosulfan blue may result in transient or long-lasting (tattoo-like) blue discoloration.

[0215] In contrast, fluorescence imaging according to various embodiments for use in the visualization and mapping of SLNs facilitates direct, real-time visual identification of LNs and / or afferent lymphatic vessels during surgery, and facilitates real-time, high-resolution optical guidance through skin and adipose tissue, organizing of blood flow, tissue perfusion, or a combination thereof.

[0216] In some variations, visualization, classification, or both of lymph nodes during fluorescence imaging may be based on imaging with one or more contrast agents, which may further be based on visualization and / or classification using a gamma probe (e.g., technetium Tc-99m is a colorless aqueous solution typically injected into the periareolar region as standard care), another conventionally used colored contrast agent (isosulfan blue), and / or other evaluations such as histology. The subject's breast may be injected, for example, twice with approximately 1% isosulfan blue (for comparison) and twice with an ICG solution having a concentration of approximately 2.5 mg / ml. The injection of isosulfan blue may precede the injection of ICG, or vice versa. For example, using a TB syringe and a 30G needle, 0.4 ml (0.2 ml per site) of isosulfan blue can be injected into the periareolar region of the breast of an anesthetized subject. The subject may receive injections at the 12 o'clock and 9 o'clock positions in the right breast and at the 12 o'clock and 3 o'clock positions in the left breast. The total intradermal dose of isosulfan blue to each breast may be approximately 4.0 mg (0.4 ml of 1% solution: 10 mg / ml). In another exemplary variation, the subject may receive ICG injection first, followed by isosulfan blue (for comparison). One 25 mg vial of ICG may be reconstituted with 10 ml of sterile water for injection to obtain a 2.5 mg / ml solution immediately before ICG administration. Using a TB syringe and a 30 G needle, for example, the subject may receive approximately 0.1 ml of ICG (0.05 ml per site) in the periareolar region of the breast (injections may be performed at the 12 o'clock and 9 o'clock positions in the right breast and at the 12 o'clock and 3 o'clock positions in the left breast). The total intradermal dose of isosulfan blue to each breast may be approximately 0.25 mg (0.1 ml of 2.5 mg / ml solution). ICG can be injected, for example, at a rate of 5 to 10 seconds per injection. Upon intradermal injection, the protein-binding properties of ICG allow it to be rapidly taken up by the lymphatic system and transported to the lymphatic nuclei (LN) via ducts. In some modifications, ICG may be provided in the form of a sterile, lyophilized powder containing 5% or less sodium iodide and 25 mg of ICG.ICG may be packaged with an aqueous solvent consisting of sterile water for injection, which is used to reconstitute the ICG. In some variations, the ICG dose (mg) in breast cancer sentinel lymphoma mapping may range from about 0.5 mg to about 10 mg, depending on the route of administration. In some variations, the ICG dose may be about 0.6 mg to about 0.75 mg, about 0.75 mg to about 5 mg, or about 5 mg to about 10 mg. The route of administration may be, for example, subcutaneous, intradermal (e.g., periareola), subareola, skin overlapping the tumor, intradermal into the areola closest to the tumor, subcutaneous into the areola, intradermal over the tumor, periareola of the entire breast, or a combination thereof. NIR fluorescence-positive LNs (e.g., using ICG) may be represented, for example, as monochrome NIR fluorescence images and / or full or partial color (white light) images, full or partially unsaturated white light images, enhanced color images, overlays (e.g., fluorescence with any other image), composite images (e.g., fluorescence incorporated into other images) which may have various colors, various levels of desaturation, or various ranges of colors to highlight / visualize specific features of interest. Further image processing may be performed for further visualization and / or other analysis (e.g., quantification). Lymph nodes and lymphatic vessels may be visualized (e.g., in real time, during surgery) using fluorescence imaging systems and methods according to various embodiments, using ICG and SLN alone or in combination with a gamma probe (Tc-99m). Fluorescence imaging of LNs can be started from the injection site by tracing the lymphatic vessels leading to the axillary LN. Once a visual image of a lymph node (LN) is identified, the skin may be incised and LN mapping and identification performed, and LN mapping may be continued until nodes visualized by ICG are identified. For comparison, mapping with isosulfane blue may be performed until "blue" nodes are identified. LNs identified by ICG alone or in combination with another imaging technique (e.g., isosulfane blue, and / or Tc-99m) may be labeled to be excised.The subjects may have breast cancer at various stages (e.g., IA, IB, IIA).

[0217] In some variations, for example in gynecological cancers (e.g., malignant tumors of the uterus, endometrium, vulva, and cervix), ICG may be administered intercellularly for visualization of lymph nodes, lymphatic vessels, or a combination thereof. Upon intercellular injection, ICG is rapidly taken up by the lymphatic fluid due to its protein-binding properties and travels through the ducts to the SLN. ICG may be provided for injection in the form of a sterile lyophilized powder containing 25 mg of ICG (e.g., 25 mg / vial) and 5.0% or less of sodium iodide. The ICG may then be reconstituted with commercially available sterile water for injection before use. According to one embodiment, a vial containing 25 mg of ICG may be reconstituted with 20 ml of water for injection to obtain a 1.25 mg / ml solution. A total of 4 ml of this 1.25 mg / ml solution is injected into a subject (4 × 1 ml injection), resulting in a total ICG dose of 5 mg per subject. Alternatively, a 1 ml solution of 1% isosulfan blue 10 mg / ml (for comparative purposes) can be injected into the cervix four times for a total dose of 40 mg. This injection may be performed in the operating room while the subject is under anesthesia. In some variations, the dose (mg) of ICG in the detection and / or mapping of sentinel lymph nodes in gynecological cancers may range from approximately 0.1 mg to approximately 5 mg, depending on the route of administration. In some variations, the ICG dose may be approximately 0.1 mg to approximately 0.75 mg, approximately 0.75 mg to approximately 1.5 mg, approximately 1.5 mg to approximately 2.5 mg, or approximately 2.5 mg to approximately 5 mg. The route of administration may be, for example, cervical injection, perivolar injection, hysteroscopy-guided endometrial injection, or a combination thereof. To minimize the outflow of isosulfan blue or ICG that may interfere with the mapping procedure when resecting the LN, mapping may be performed on the semipelvic side, and mapping using both isosulfan blue and ICG may be performed before the resection of any LN.LN mapping for clinical stage I endometrial cancer may be performed according to the NCCN guidelines for uterine tumors and the SLN algorithm for surgical staging of endometrial cancer, and SLN mapping for clinical stage I cervical cancer may be performed according to the NCCN guidelines for cervical tumors and the surgical / SLN mapping algorithm for early cervical cancer. Accordingly, LN identification may be based on ICG fluorescence imaging alone, or in combination with or concurrently administered with a colorimetric dye (isosulfane blue) and / or a radiotracker.

[0218] Lymph node visualization may be qualitative and / or quantitative. Such visualizations may include, for example, lymph node detection, detection rate, and anatomical distribution of lymph nodes. Lymph node visualizations according to various embodiments may be used alone or in combination with other variables (e.g., vital signs, height, weight, demographics, surgical predictors, relevant medical history and underlying condition, histological visualization and / or evaluation, Tc-99m visualization and / or evaluation, concomitant medications). Follow-up visits may be made on the day of discharge and thereafter (e.g., one month).

[0219] Because lymphatic fluid contains high levels of protein, ICG can bind to endogenous proteins upon entering the lymphatic system. When used according to the methods and systems described herein, fluorescence imaging for lymph mapping (e.g., ICG imaging) offers the following exemplary advantages: high signal-to-background ratio (or tumor-to-background ratio) because NIR does not produce significant autofluorescence; real-time visualization features for lymph mapping; tissue definition (i.e., structural visualization); rapid excretion and removal after entering the vascular system; and avoidance of non-ionizing radiation. Furthermore, NIR imaging has superior tissue penetration (approximately 5-10 mm) compared to visible light (1-3 mm tissue). Also, the use of ICG, for example, facilitates visualization through the peritoneum covering the para-aortic nodes. Tissue fluorescence can be observed for extended periods with NIR light but not with visible light, and consequently does not affect pathological evaluation or LN treatment. In addition, fluorescence is more easily detected intraoperatively than blue staining (isosulfane blue) of lymph nodes. In other modifications, the methods, dosages, or combinations thereof described herein in relation to lymphatic vessel imaging may be used in any vascular and / or tissue perfusion imaging application.

[0220] Tissue perfusion relates to the microcirculation of blood per unit tissue volume, which is supplied with oxygen and nutrients and removes waste products from the capillary bed of the perfused tissue. Tissue perfusion relates to, but is distinct from, blood flow within blood vessels. Quantified blood flow through blood vessels can be expressed in terms of flow-defining language (i.e., volume / time) or velocity-defining language (i.e., distance / time). Tissue hemoperfusion defines the movement of blood through the microvascular system, such as arterioles, capillaries, or venules, within a tissue volume. Quantified tissue hemoperfusion can be expressed as blood flow through a tissue volume, i.e., blood volume / time / tissue volume (or tissue mass). Perfusion relates to nutrient vessels (e.g., microvessels known as capillaries), which include vessels involved in the exchange of metabolites between blood and tissue, rather than non-nutrient vessels of a larger diameter. In some embodiments, the quantification of the target tissue may involve calculating or determining parameters or quantities related to the target tissue, such as velocity, size, volume, time, distance / time, and / or volume / time, and / or changes in any one or more of the aforementioned parameters or quantities. However, compared to the movement of blood through larger diameter vessels, the movement of blood through individual capillaries is highly irregular, mainly due to vasomotor activity, and spontaneous oscillations of vascular tone manifest as pulsations in the movement of red blood cells. In some embodiments, blood flow and tissue perfusion imaging described herein in relation to systems and methods may be used to image tumor tissue and distinguish such tissue from other tissues.

[0221] The foregoing disclosures have been made for illustrative purposes with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the exact forms disclosed. In view of the above teachings, many modifications and variations are possible. The embodiments have been selected and disclosed to best illustrate the principles of the art and their practical applications. Thereafter, those skilled in the art can best utilize the art and its various embodiments by making various modifications to suit specific conceivable applications.

[0222] While the present disclosure and examples have been adequately described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art. Such changes and modifications should be understood to fall within the scope of the disclosure and examples as defined by the claims. Finally, the entire disclosures of the patents and publications referenced in this application are incorporated herein by reference.

Claims

1. A method for optimizing a medical imaging processing system, The aforementioned method, Based on first configuration data stored in memory, the reconfigurable hardware processor of the medical imaging processing system is configured into a first configuration for a first medical imaging session, wherein the first configuration implements at least a first medical imaging processing algorithm, and the first configuration is associated with a first user profile. Receiving the first medical imaging data generated during the first medical imaging session, By at least partially processing the first medical imaging data using the first medical imaging processing algorithm implemented in the first configuration, enhanced first medical imaging data is generated. For observation during the first medical imaging session, the enhanced first medical imaging data is displayed, Reconfiguring the reconfigurable hardware processor to a second configuration for a second medical imaging session based on second configuration data stored in the memory, wherein the second configuration implements at least a second medical imaging processing algorithm not implemented in the first configuration, and the second configuration is associated with a second user profile. Receiving the second medical imaging data generated during the second medical imaging session, By processing the second medical imaging data at least partially using the second medical imaging processing algorithm implemented in the second configuration, enhanced second medical imaging data is generated. The enhanced second medical imaging data is displayed on a display for observation during the second medical imaging session. A method characterized by the following:

2. Receiving an input indicating the second medical imaging session, This includes, in response to receiving the input, automatically reconfiguring the reconfigurable hardware processor to the second configuration. The method according to feature 1.

3. The method according to 2, characterized in that the input includes the selection of a type of medical procedure.

4. The method according to 2 or 3, characterized in that the input includes the selection of a user profile.

5. The method according to any one of claims 2 to 4, characterized in that the input includes the selection of a default configuration profile.

6. The method according to claim 5, characterized in that the default configuration profile is based on one or more connections from one or more external devices to the medical imaging processing system.

7. The method according to 6, characterized in that the default configuration profile is based on the field of view of a connected external device.

8. The first configuration described above is associated with a first type of medical procedure, The second configuration described above is associated with a second type of medical procedure. The method according to any one of claims 1 to 7, characterized by...

9. The method according to any one of claims 1 to 8, characterized in that both the first configuration data and the second configuration data are associated with the same type of medical procedure.

10. The first medical imaging session is the first surgical session, The second medical imaging session is the second surgical session. The method according to any one of claims 1 to 9, characterized by...

11. The at least one medical imaging processing algorithm implemented in the second configuration includes a smoke detection algorithm, The generation of the enhanced second medical imaging data includes enhancing the clarity of one or more portions of one or more images related to smoke. The method according to any one of claims 1 to 10, characterized by...

12. The method according to any one of claims 1 to 11, characterized in that the first medical imaging processing algorithm detects the characteristics of the imaged tissue.

13. The method according to 12, characterized in that the characteristics of the imaged tissue are tissue perfusion, location of blood vessels, blood flow rate, dimensions of the imaged tissue, or a combination thereof.

14. The method according to any one of claims 1 to 13, characterized in that the enhanced second medical imaging data includes an overlay on at least a portion of the second medical imaging data.

15. The method according to any one of claims 1 to 14, characterized in that the reconfigurable hardware processor is reconfigured before the start of imaging.

16. The method according to any one of claims 1 to 15, characterized in that one or more medical imaging processing algorithms are implemented in both the first and second configurations.

17. The method according to any one of claims 1 to 16, characterized in that the second medical imaging data includes at least one of a video frame and an image.

18. The method according to any one of claims 1 to 17, characterized in that the second medical imaging data is received from an endoscopic imaging system.

19. The method according to 18, characterized in that the second medical imaging data is received from a camera control unit.

20. The method according to any one of claims 1 to 19, characterized in that the reconfigurable hardware processor is an FPGA or a GPU.

21. Receiving the second medical imaging data from the first device, Receiving data from a second medical device, This includes outputting a display feed to the aforementioned display, The display feed includes the enhanced second medical imaging data and at least a portion of the data from the second medical device. The method according to any one of claims 1 to 20, characterized by...

22. The first processor receives the second medical imaging data and the data from the second medical device, Transmitting the second medical imaging data from the first processor to the reconfigurable hardware processor, The first processor receives the enhanced second medical imaging data from the reconfigurable hardware processor, The first processor generates the display feed by combining the enhanced second medical imaging data with at least a portion of the data related to the second medical device. The method according to feature 21.

23. The method according to any one of claims 1 to 22, characterized in that the first configuration data is stored in remote memory and received via a network connection.

24. A reconfigurable medical imaging processing system, The display and Memory and A reconfigurable hardware processor, A second processor configured as follows, including: Based on first configuration data stored in the memory, the reconfigurable hardware processor is configured to a first configuration for a first medical imaging session, wherein the first configuration is associated with a first user profile, and the reconfigurable hardware processor in the first configuration is configured to implement at least a first medical imaging processing algorithm and to generate enhanced first medical imaging data for display on the display by at least partially processing first medical imaging data using the first medical imaging processing algorithm. Based on the second configuration data stored in the memory, the reconfigurable hardware processor is reconfigured into a second configuration for a second medical imaging session, wherein the second configuration is associated with a second user profile, and the reconfigurable hardware processor in the second configuration is configured to implement at least a second medical imaging processing algorithm and to generate enhanced second medical imaging data for display on the display by at least partially processing second medical imaging data using the second medical imaging processing algorithm. A system characterized by the following features.

25. The system according to claim 24, wherein the second processor is configured to receive an input indicating the second medical imaging session and to automatically reconfigure the reconfigurable hardware processor to the second configuration in response to the receipt of said input.

26. The system according to claim 25, characterized in that the input includes the selection of a type of medical procedure.

27. The system according to claim 25 or 26, characterized in that the input includes the selection of a user profile.

28. The system according to any one of claims 25 to 27, characterized in that the input includes the selection of a default configuration profile.

29. The system according to claim 28, characterized in that the default configuration profile is based on one or more connections from one or more external devices to the medical imaging processing system.

30. The system according to claim 29, characterized in that the default configuration profile is based on the field of view of a connected external device.

31. The first configuration described above is associated with a first type of medical procedure, The second configuration described above is associated with a second type of medical procedure. The system according to any one of claims 24 to 30, characterized by the features described herein.

32. The first medical imaging session includes the performance of the first type of medical procedure on the patient, The second medical imaging session includes the performance of the second type of medical procedure on the patient. The system according to feature 31.

33. The first medical imaging session includes imaging the patient, The second medical imaging session includes imaging the patient. The system according to any one of claims 24 to 32, characterized by the features described herein.

34. The system according to any one of claims 24 to 33, characterized in that both the first configuration data and the second configuration data are associated with the same type of medical procedure.

35. The first medical imaging session is the first surgical session, The second medical imaging session is the second surgical session. The system according to any one of claims 24 to 34.

36. The at least one medical imaging processing algorithm implemented in the second configuration includes a smoke detection algorithm, The generation of the enhanced second medical imaging data includes enhancing the clarity of one or more portions of one or more images related to smoke. The system according to any one of claims 24 to 35, characterized by the features described herein.

37. The system according to any one of claims 24 to 36, characterized in that the first medical imaging processing algorithm is configured to detect features of the imaged tissue.

38. The system according to claim 37, characterized in that the characteristics of the imaged tissue are tissue perfusion, location of blood vessels, blood flow rate, dimensions of the imaged tissue, or a combination thereof.

39. The system according to any one of claims 24 to 38, characterized in that the enhanced second medical imaging data includes an overlay on at least a portion of the second medical imaging data.

40. The system according to any one of claims 24 to 39, characterized in that the system is configured to reconfigure the reconfigurable hardware processor before the start of imaging.

41. The system according to any one of claims 24 to 40, characterized in that one or more medical imaging processing algorithms are implemented in both the first and second configurations.

42. The system according to any one of claims 24 to 41, characterized in that the second medical imaging data includes at least one of a video frame and an image.

43. The system according to any one of claims 24 to 42, characterized in that the system is configured to receive the second medical imaging data from the endoscopic imaging system.

44. The system according to claim 43, characterized in that the system is configured to receive the second medical imaging data from the camera control unit.

45. The system according to any one of claims 24 to 44, characterized in that the reconfigurable hardware processor is an FPGA or a GPU.

46. The aforementioned system, The second medical imaging data is received from the first device. Receive data from the second medical device, Displaying the enhanced second medical imaging data and at least a portion of the data from the second medical device. The system according to any one of claims 24 to 45, characterized in that it is configured as follows.

47. The aforementioned system, In the second processor, the second medical imaging data and the data from the second medical device are received. The second medical imaging data is transmitted from the second processor to the reconfigurable hardware processor. The second processor receives the enhanced second medical imaging data from the reconfigurable hardware processor. The second processor generates a display feed for the display by combining the enhanced second medical imaging data with at least a portion of the data related to the second medical device. The system according to claim 46, characterized in that it is configured as follows.

48. The system according to any one of claims 24 to 47, characterized in that the first configuration data is stored in remote memory and received via a network connection.

Images