Methods and systems for automating surgical interventions

Abstract

A surgical system for providing improved video images of a surgical site, comprising: a system controller that receives and processes video images to determine a video signature corresponding to a condition that interferes with the quality of the video images, wherein the system controller interacts with a video enhancer to enhance the video images from a video capture device to automatically control the video enhancer to enhance the video images. The surgical system can also review the video images for a triggering event and automatically start or stop recording of the video images when the triggering event occurs.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application Serial No. 62 / 474,331, filed on March 21, 2017, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to the field of medical imaging, and more particularly to providing improved images of a surgical site for use during surgical procedures. Background Art

[0004] During typical endoscopic procedures, including laparoscopic surgical procedures, smoke is generated, which can interfere with the image of the surgical site being viewed. Better images during smoke generation are desired. A fast, simple, and reliable method of deploying medical or surgical equipment in a medical care area is also desired. Summary of the invention

[0005] According to various aspects, the present invention is directed to systems and methods for providing improved video images of a surgical site. The system includes: a tool for manipulating tissue at a surgical site; a light source for providing light to the surgical site; a video capture device for obtaining a video image at the surgical site; an image display for displaying the video image; and a system controller configured to maintain the quality of the video image obtained by the video capture device and provided to the image display. The system controller receives and processes the video image to determine a video signature corresponding to a condition that interferes with the quality of the video image. The system controller interacts with a video enhancer to enhance the video image from the video capture device to automatically control the video enhancer to enhance the video image transmitted from the video capture device to the image display when a condition that interferes with the quality of the video image is detected, so that a user is relieved from having to control the video enhancer to obtain an improved video image of the surgical site for viewing on the image display.

[0006] Another aspect of the present invention is to provide a system for providing an improved video image of a surgical site. The system includes: a tool for manipulating tissue at the surgical site; a suction system for providing suction at the surgical site; a light source for providing light to the surgical site; a video capture device for obtaining a video image at the surgical site; an image display for displaying the video image; and a system controller configured to maintain the quality of the video image obtained by the video capture device and provided to the image display. The system controller receives and processes the video image to determine a video signature corresponding to a condition that interferes with the quality of the video image. The system controller interacts with the video enhancer, the tool, and the adjustable suction system and controls at least one of the video capture device, the video enhancer, the tool, and the suction system to address the condition at the surgical site to restore the video image to an improved quality for viewing, so that the user is relieved from having to manually control any of the video capture device, the video enhancer, the tool, and the suction system to obtain an improved video image of the surgical site for viewing on the image display.

[0007] Another aspect of the present invention is to provide a method for controlling a surgical system to provide an improved image of a surgical site, the method comprising: manipulating tissue at the surgical site; irradiating the surgical site; obtaining a video image of the surgical site; analyzing the video image to determine the presence of a condition that interferes with the quality of the video image; configuring a system controller to interact with a video enhancer; in response to the presence of the condition and without control from an operator, controlling the video enhancer using the system controller to generate a video image with improved image quality; and displaying the video image with improved image quality.

[0008] Another aspect of the present invention is to provide an imaging system for observing video images of a surgical site. The system includes: a light source for providing light to the surgical site; a video capture device for obtaining video images at the surgical site; a video recorder that receives the video images; and a system controller that receives and processes the video images to determine a triggering event. The system controller interacts with the video recorder to at least one of automatically record the video images and automatically stop recording the video images when a triggering event is determined.

[0009] Another aspect of the present invention is to provide a method for controlling a surgical system for providing images of a surgical site, the method comprising: irradiating the surgical site; obtaining a video image at the surgical site; analyzing the video image to determine the presence of a trigger event; configuring a system controller to interact with a video recorder; and in response to sensing the trigger event, automatically controlling the video recorder using the system controller to record the video image.

[0010] Another aspect of the present invention is to provide a system for providing an improved video image of a surgical site. The system includes: a tool for manipulating tissue at the surgical site; a suction system for providing suction at the surgical site; a light source for providing light to the surgical site; a video capture device for obtaining a video image at the surgical site; an image display for displaying the video image; and a system controller configured to maintain or improve the quality of the video image obtained by the video capture device and provided to the image display. The system controller receives and processes the video image to determine a video signature corresponding to a condition that interferes with the quality of the video image. The system controller interacts with the video enhancer, the tool, and the suction system and controls at least one of the video capture device, the video enhancer, the tool, and the suction system to address the condition at the surgical site to achieve an improved quality of the video image for viewing, so that the user is relieved from having to manually control any of the video capture device, the video enhancer, the tool, and the adjustable suction system to obtain an improved video image of the surgical site for viewing on the image display. The video signature to be identified corresponds to a condition of smoke in the video image, and the system controller operates at least one of the video capture device, the video enhancer, the cautery tool, the insufflator, and the adjustable suction system in response to (1) the smoke characteristics determined by the system controller and (2) at least one other surgical input provided to the system controller. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] One or more embodiments of the present disclosure are illustrated as examples and should not be construed as limited to the particular embodiments depicted in the drawings, in which like reference numerals indicate like elements.

[0012] Figure 1 A schematic diagram of a surgical system according to an embodiment is illustrated.

[0013] Figure 2 A method of using surgical system 10 according to an embodiment is illustrated.

[0014] Figure 3 A method of processing an image according to an embodiment is illustrated.

[0015] Figure 4 is an image of an exemplary surgical site.

[0016] Figure 5 A workflow process according to an embodiment is illustrated.

[0017] Figure 6 is a first image of a feature tracker according to an embodiment.

[0018] Figure 7 is a second image of a feature tracker according to an embodiment.

[0019] Figure 8 is a list of possible exemplary features used according to embodiments.

[0020] Fig. 9 Illustrated are samples of 3D spaces determined using the processes described herein in various embodiments.

[0021] Fig.10 The steps for a classification training embodiment are illustrated.

[0022] Fig.11 The steps for a classification test embodiment are illustrated.

[0023] Fig.12 Illustrated is a domain logic flow diagram for enhancing a video image depending on factors other than cleansing the video image alone, according to an embodiment.

[0024] Fig.13 A method for automatically recording video images according to an embodiment is illustrated.

[0025] For convenience, certain terminology will be used in the following description for reference only and not for limitation. The terminology will include the words specifically mentioned, their derivatives and words of similar meaning. DETAILED DESCRIPTION

[0026] Reference will now be made in detail to embodiments and examples of various aspects and variations of the invention, examples of which are illustrated in the accompanying drawings. Although at least two variations of systems, methods, uses and kits are described, other variations of systems, methods, uses and kits may include aspects of the systems, methods, uses and kits described herein with combinations of all or some of the described aspects combined in any suitable manner.

[0027] Figure 1An embodiment of a surgical system 10 for performing a surgical procedure is illustrated. The surgical system 10 may include a tool 12 for manipulating tissue at a surgical site 16 under the control of a tool controller 14, a fluid input system 18 for providing fluid to the surgical site 16, a suction system 20 (e.g., adjustable) for providing suction at the surgical site 16 for controlling removal of fluid from the surgical site 16, a light source 22 for providing light to the surgical site 16 under the control of a light source controller 24, a video capture device 26 for obtaining a video signal of a video image at the surgical site 16, a camera control unit 28 for controlling the video capture device 26, and an image display 32 for displaying the video image. In the illustrated example, tool controller 14 may be individually operated to control tool 12, fluid input system 18 may be individually operated to provide fluid to surgical site 16, suction system 20 may be individually operated to aspirate fluid from surgical site 16, light source controller 24 may be individually operated to adjust light source 22, and / or camera control unit 28 may be individually operated to control video capture device 26. Furthermore, as shown in FIG. Figure 1 As shown in FIG. 1 , the system controller 30 communicates with the tool controller 14 to control the tool 12, with the fluid input system 18 to provide fluid to the surgical site 16, with the suction system 20 to aspirate fluid from the surgical site 16, with the light source controller 24 to adjust the light source 22, and with the camera control unit 28 to control the video capture device 26. The fluid input system 18 can provide fluid to the surgical site 16 through its own device or through another device 31 (e.g., an endoscope) that also has the video capture device 26 connected to it and receives light from the light source 22.

[0028] In the illustrated example, the surgical system 10 can be employed in a number of different surgical procedures. For example, the surgical system 10 can be used during an endoscopic procedure (including a laparoscopic procedure), where the tool 12 can be a cauterization tool, the fluid input system 18 can be an insufflator for providing gas to the surgical site 16, and the suction system 20 can aspirate gas and potentially smoke from the surgical site 16. Alternatively, the surgical system 10 can be used, for example, during an arthroscopic procedure, where the tool 12 can be a cutting tool, the fluid input system 18 can be a liquid pump for providing fluid (e.g., saline solution) to the surgical site 16, and the suction system 20 can aspirate fluid (e.g., saline solution and, potentially, blood) from the surgical site 16. It is contemplated that the procedure (e.g., an endoscopic procedure) can be employed using one or more robotic devices for robotic surgery.

[0029] The illustrated surgical system 10 may also provide the video images captured by the video capture device 26 to an image display 32 to be viewed by a person in the operating room. The image display 32 may be a single display or multiple displays. In addition, the image display 32 may be incorporated into the same housing as the housing of the system controller 30, or the housing of the system controller 30 may include another display for the video images in addition to the image display 32. The surgical system 10 may also include a video recorder 36 as a stand-alone device that is incorporated into the housing of the system controller 30 or communicates with the surgical system 10 from a remote location. An example of an integrated system controller 30 and video recorder 36 is the SDC3 HD Information Management System (with Device Control) sold by Stryker Corporation of Kalamazoo, Michigan. The video images may also be processed in a video enhancer 38 to clean up the video images before they are transmitted to any display of the image display 32, the system controller 30, and the video recorder 36. The video enhancer 38 and the system controller 30 can be configured to maintain the quality of the video image obtained by the video capture device 26 and provided to any display of the image display 32, the system controller 30, and the video recorder 36. An example of a video enhancer 38 is a clarity video enhancer such as sold by Stryker Corporation of Kalamazoo, Michigan. The video enhancer 38 can be used to adjust the video sent to it by changing the brightness, color, contrast or other characteristics of the video to enable better viewing of relevant portions of the video image. For example, the video enhancer 38 can be used to change the brightness, contrast or color to enable identification of certain areas of the surgical site 16 relative to other areas of the surgical site 16 (e.g., to make the contrast between smoke and adjacent areas more obvious).

[0030] Figure 2 A general method 40 of using a surgical system 10 as disclosed herein is illustrated according to an embodiment. As an initial step, a video image is captured using the video capture device 26 at step 42. Thereafter, at step 44, the video image is analyzed to determine whether a trigger condition has occurred. If it is determined at step 44 that a trigger condition has occurred, the surgical system 10 takes further action at step 46.

[0031] Figure 3 An embodiment of a general method 40 is illustrated, wherein Figure 3 The method includes a method 50 for controlling the surgical system 10 to provide an improved image of the surgical site 16. Figure 3In the method of , a first step includes capturing a video image using the video capture device 26 at step 52. At step 54, the video image is processed to determine whether there is a video signature corresponding to a condition that interferes with the quality of the video image. If there is a video signature corresponding to a condition that interferes with the quality of the video image, the system controller 30 may interact with the video enhancer 38 at step 56 to automatically control the video enhancer 38 to enhance the video image delivered from the video capture device 26. Because the system controller 30 may automatically control the video enhancer 38 at step 56 to enhance the video image delivered from the video capture device 26, a user of the surgical system 10 using the method 50 may be relieved from having to manually control the video enhancer 38 to obtain an improved video image of the surgical site 16. The improved video image may be displayed on the image display 32, any display of the system controller 30, and / or may be saved in the video recorder 36.

[0032] In the illustrated example, in addition to or in lieu of controlling the video intensifier 38, the system controller 30 may control the tools 12, the fluid input system 18, and / or the suction system 20 to clear the image at step 56. For example, if the surgical system 10 is used during a laparoscopic procedure, the cauterization tool 12 may be adjusted to a lower power to produce less smoke, the insufflator 18 may be increased to add more gas to the surgical site 16, and / or the suction rate from the suction system 20 may be increased to suction gas and, potentially, smoke from the surgical site 16. Alternatively, if the surgical system 10 is used during an arthroscopic procedure, the cutting tool 12 may be adjusted to a lower power to produce less blood and / or debris, the fluid pump 18 may be increased to add more surgical fluid to the surgical site 16, and / or the suction rate from the suction system 20 may be increased to suction surgical fluid and blood from the surgical site 16.

[0033] One aspect of the embodiment is to provide Figure 3 Method 50 is provided for determining when the system controller 30 should control elements of the surgical system 10 and / or process video images in response to smoke. Automatic detection of smoke and subsequent automation of surgical equipment (e.g., tools 12, fluid input system 18, etc.) can play a significant role in reducing or eliminating manual control of elements of the surgical system 10 and / or manual activation of processes to clean images, thereby reducing surgical procedure time and making it simpler to perform surgical procedures (and potentially reducing surgical risks and errors).

[0034] In the illustrated example, step 54 of method 50 includes processing the image to determine the presence and quality of smoke at the surgical site 16. As examples, gaseous byproducts such as smoke discussed above can be easily seen and smelled during laparoscopic cholecystectomy, electrocautery, laser tissue resection, and ultrasonic (harmonic) surgical scalpel tissue dissection. The average aerodynamic size of the generated smoke particles varies greatly depending on the energy method used to generate the smoke. Figure 4 An example of a tool 12 being used at a surgical site 16 (at a point in time before any smoke is present) to generate smoke is illustrated.

[0035] Step 54 of identifying smoke can be performed by analyzing the video image. In a first example, a digital device can be used to analyze the video image during the actual surgical procedure (e.g., the analysis can occur a predetermined number of times per second) to record the possibility of a smoke state or a smoke-free state. If step 54 of the first example identifies the possibility of the presence of smoke, the method may include a further step of determining the density and diffusion of the smoke based on the recorded state (smoke or no smoke) probability. In a second example, the video image can be analyzed to determine whether there is smoke together with the characteristics of the smoke (e.g., the appearance or disappearance of smoke, changes in smoke intensity, changes in smoke diffusion, etc.) and other elements of the video (e.g., the appearance or disappearance of blood and other fluids, etc.). It is contemplated that the characteristics of the smoke can be determined based on inputs such as procedure type, camera settings, and surgeon preferences. The foregoing list is illustrative and not exhaustive.

[0036] Figure 5 An embodiment of a training method 60 is illustrated for developing a process for determining the presence and quality of smoke at a surgical site 16 at step 54. The method 60 may include identifying smoke by a digital device. The method 60 for identifying smoke may include a first step 62 of dividing a video image at the surgical site 16 into small blocks, such as blocks of frames of a particular time amount (e.g., one second) of consecutive frames of video or a particular number of frames (e.g., 60 frames) of consecutive frames of video. The method 60 may include an additional step 64 of identifying a set of pixels whose convex hull in a first frame of the block under review covers as much of the frame space as possible. For example, Figure 6 A set of possible pixels marked by crosses are shown positioned throughout the first frame of the block under review.

[0037] In the illustrated example, the subsequent step 66 is to track the set of pixels as it evolves from the first frame to the end of the block of the frame under review (see Figure 7, where the adjusted contrast better illustrates the direction and magnitude of the movement of the tracked pixel 67). An algorithm may be used to track a set of pixels. For example, the Kanade LucasTomasi (KLT) algorithm may be used to track a set of pixels. In some embodiments, tracking may produce a motion vector for each pixel in the set of tracked pixels. Given the optical flow vectors of the tracked pixels across the entire block, the subsequent step 68 (see Figure 8) is to generate features of pixels, including temporal and / or frequency domain statistics of the temporal and / or spatial evolution of these pixels. Features of pixels may include temporal values, frequency domain values, and / or statistical values. Examples of temporal values ​​include (1) an average spatial displacement of one pixel across the entire block averaged over all pixels, (2) the variance of the spatial displacement of one pixel across the entire block averaged over all pixels, (3) the entropy of the pixel displacement of one pixel across the entire block averaged over all pixels, (4) the average angular displacement of one pixel across the entire block averaged over all pixels, (5) the variance of the angular displacement of one pixel across the entire block averaged over all pixels, (6) the entropy of the angular displacement of one pixel across the entire block averaged over all pixels, (7) the average spatial displacement of all tracked pixels in a given frame averaged across all frames in the block, (8) the variance of the spatial displacement of all tracked pixels in a given frame averaged across all frames in the block, (9) the entropy of the angular displacement of one pixel across the entire block averaged across all frames in the block. entropy of the spatial displacement of all tracked pixels in a given frame, (10) the average angular displacement of all tracked pixels in a given frame averaged across all frames in the block, (11) the variance of the angular displacement of all tracked pixels in a given frame averaged across all frames in the block, (12) the entropy of the angular displacement of all tracked pixels in a given frame averaged across all frames in the block, (13) the correlation between consecutive spatial displacements of a pixel across the entire block averaged over all pixels, (14) the correlation between consecutive angular displacements of a pixel across the entire block averaged over all pixels, (15) the correlation between spatial displacements of neighboring pixels in a given frame averaged across all frames in the block and / or (16) the correlation between angular displacements of neighboring pixels in a given frame averaged across all frames in the block.Examples of frequency domain values ​​include (1) given a fast Fourier transform (FFT) of a sequence of spatial displacements of one pixel across the entire block, the ratio of energy in a 20%-40%, 40%-60%, 60%-80%, or 80%-100% band of the Nyquist frequency relative to energy in a 0%-20% band averaged across all pixels (e.g., including up to 4 features, with one feature derived from each ratio), (2) given a FFT of a sequence of angular displacements of one pixel across the entire block, the ratio of energy in a 20%-40%, 40%-60%, 60%-80%, or 80%-100% band of the Nyquist frequency relative to energy in a 0%-20% band averaged across all pixels (e.g., including (e.g., including up to 4 features), (3) given the FFT of the set of spatial displacements of all tracked pixels in a frame, the ratio of the energy in the 20%-40%, 40%-60%, 60%-80%, 80%-100% bands of the Nyquist frequency relative to the energy in the 0%-20% band averaged across all frames in the block (e.g., including up to 4 features), and (4) given the FFT of the set of angular displacements of all tracked pixels in a frame, the ratio of the energy in the 20%-40%, 40%-60%, 60%-80%, 80%-100% bands of the Nyquist frequency relative to the energy in the 0%-20% band averaged across all frames in the block (e.g., including up to 4 features). Examples of statistical values ​​include (1) the percentage of tracked pixels that exhibit significant motion, (2) the percentage of tracked pixels that cannot be tracked for an entire block of the video, (3) the percentage of frames covered by tracked particles across a block of the video, and / or (4) the rate of change of coverage percentage across a block of the video. Fig. 9 A sample 3D feature space is illustrated. The features may be initially manually selected during system construction based on sample training data.

[0038] In the illustrated example, for each block of the video, a subsequent step 70 includes generating the output of the feature generation block as a vector of N numbers representing the values ​​of the features generated in step 68. A subsequent step 72 is to mark (e.g., manually during training of the system) which of the blocks correspond to the presence of surgical smoke (smoke) and which do not correspond to the presence of surgical smoke (smoke), where such data is aggregated in an aggregator. The set of feature vectors from the M blocks of the video (the MxN matrix of data) can then be presented to the trainer for training at step 74 along with the labeled Mxl vectors. For example, the trainer can be a kernel-based support vector machine (SVM), a Gaussian mixture model, or a neural network. Fig.10 The steps of classification training are illustrated. Once training is complete, the classifier model can then be applied to a collection of test videos for classification testing to determine the accuracy of the prediction engine in terms of true positives, false positives, true negatives, and false negatives at step 76. Fig.11 Steps for classifying a test according to an embodiment are illustrated.

[0039] return Figure 3 After determining the presence of smoke in step 54 of method 50 for controlling surgical system 10 to provide improved images of surgical site 16, if smoke is detected, system controller 30 may automatically control video enhancer 38 to enhance the video images transmitted from video capture device 26 and / or may automatically control other surgical devices (e.g., tool 12 via tool controller 14, suction system 20, etc.) to improve conditions at surgical site 16 at step 56. It is contemplated that step 56 may automatically include controlling video enhancer 38 and / or other surgical devices independently of one another or in combination with one another. For example, suction system 20 may be controlled in tandem with controlling video enhancer 38, wherein the amount of suction from suction system 20 is limited by the amount of enhancement performed by video enhancer 38. Moreover, it is contemplated that step 56 may automatically include controlling video enhancer 38 and / or other surgical devices in light of the characteristics of the smoke (e.g., density, diffusion rate, etc.).

[0040] In the illustrated example, domain logic may be employed to select the type of response to decontaminate the image at step 56. For example, the system controller 30 may determine the manner in which to proceed at step 56 after determining characteristics of the smoke and video (e.g., haze, grayscale average, intensity, degree of diffusion of the smoke, etc.), determining a list of surgical devices connected thereto (automatically determined upon connection and / or manually entered), determining the type of procedure (e.g., automatically pulled from a schedule database in communication with the system controller 30 or manually entered), and / or determining the size of the orifice or natural opening (e.g., determined by analyzing images from the camera unit 26 if the camera unit 26 is turned on while the insertion portion of the device 31 is inserted into the orifice or natural opening). The above list is illustrative and not exhaustive.

[0041] It is contemplated that the method 50 for controlling the surgical system 10 to provide an improved image of the surgical site 16 may include a learning algorithm that may improve the step 56 of automatically controlling the video enhancer 38 and / or other surgical devices. For example, the system controller 30 may monitor the video enhancer 38 and / or other surgical devices to determine whether the video enhancer 38 and / or other surgical devices have been manually adjusted after the system controller 30 has automatically controlled the video enhancer 38 and / or other surgical devices to cleanse the image at step 56. If the video enhancer 38 and / or other surgical devices are manually adjusted, the system controller 30 may employ domain logic steps that are employed to select the type of response to cleanse the image at step 56. For example, the system controller 30 may learn that the tool 12 is not automatically reduced in power sufficiently because the power is manually reduced after step 56; that the suction system 20 does not automatically provide sufficient suction if the suction system 20 is manually adjusted to increase suction after step 56; or that the video enhancer 36 does not automatically adjust to a desired level if the video enhancer 38 is manually controlled after step 56. The above list is illustrative and not exhaustive. After the manual adjustment, the domain logic steps can be adjusted to correspond to the manual invention so that the next time the method 50 is performed, the video enhancer 38 and / or other surgical equipment automatically adjusts to the point after the manual adjustment as outlined above, so that the video enhancer 38 and / or other surgical equipment does not have to be manually adjusted again. It is contemplated that the domain logic steps can be saved for a particular surgeon, a particular procedure, and / or for all uses of the video enhancer 38 and / or other surgical equipment. It is further contemplated that simulation can be used to teach and adjust the domain logic steps as outlined above.

[0042] As outlined above, the smoke detection and image enhancement process can allow for automatic detection of surgical smoke during tissue removal to automatically trigger digital enhancement of surgical footage and / or to trigger mechanical smoke ventilation / reduction to enable the surgeon to better see images of the surgical site 16. The goal of the aspects discussed herein is to detect the presence of such smoke in surgical videos, the intensity of the smoke, and the extent of the diffusion of the smoke, so that the image processing algorithms for defogging (e.g., in the video enhancer 38) can be automatically turned on / off, as well as turning on / off or reducing the power (e.g., ventilation) of surgical equipment associated with the smoke. Over time, the system can learn (e.g., each individual surgeon's) preferred smoke ventilation and defogging levels, and automatically tend to the best settings each time.

[0043] As outlined above, aspects of some embodiments relate to centralized and automated control mechanisms during minimally invasive surgical procedures. While the smoke monitoring and prevention system is outlined above, another aspect of some embodiments is to use the same process with other aspects of the surgical procedure that can use centralized and automated control mechanisms during minimally invasive surgical procedures to improve the surgical procedure. Depending on the type of procedure, different kinds and numbers of instruments (e.g., the surgical equipment outlined above) can be connected to the system controller 30. Each procedure can have one or more triggering events (e.g., the presence of smoke) that can warrant activating at least one of the instruments at different settings in order to present the surgeon with the best possible quality of surgical video that the surgeon is most comfortable viewing.

[0044] Therefore, if Figure 3 In the method outlined in , a centralized and automated control mechanism is used during minimally invasive surgery to improve surgical procedures. A trigger detection step can operate on a sequence of digital images obtained from a camera (e.g., an endoscopy camera). The image sequence can be processed in hardware (e.g., on a field programmable gate array) or in software (e.g., on a processor) using image / video processing and computer vision algorithms to identify trigger events. Based on the determination of the identification of the trigger event, the domain logic configures the relevant instrument to the best known settings applicable to the scenario. Moreover, the learning step fine-tunes the value of the best known settings based on manual surgeon (or other in training) intervention. Again, using smoke detection as an example, the first step is to identify smoke by digital means using an image / video processing algorithm to pre-process the digital image / video sequence emerging from the camera (e.g., an endoscopy camera) to identify relevant features and landmarks. The features and landmarks are then fed to the computer vision algorithm to identify smoke along with various aspects of smoke (or the absence of smoke). The algorithm can be trained using a large number of training images and videos prior to use to be able to distinguish between various scenarios (e.g., the presence or absence of smoke) with a desired level of accuracy. Based on the input of the feature vector, the algorithm can not only determine whether smoke is present, but also provide a confidence metric indicating the level of certainty. The algorithm can be further enhanced to quantify the intensity and / or spatial diffusion of surgical smoke within the visual space. Once the presence of smoke is determined, the system controller 30 can control the equipment to ventilate the smoke, adjust the instrument causing the smoke, and / or process the video image to present the surgeon with the best possible surgical continuous footage while mitigating the effects of the smoke.

[0045] Although the smoke example outlined above, it is possible to use Figure 3Many other features and events that occur during surgery can be detected using the processes outlined in , and further actions can be taken to enhance the surgery. For example, during sinus surgery, the surgeon may attempt to scrape off a nasal polyp and may encounter excessive blood that fouls the tip of the endoscope, thereby obstructing the camera. Typically, the surgeon manually rinses the nose with saline to clear the scope tip and view the surgical footage again. An automatic blood detection algorithm can trigger an automatic rinse or other tip cleaning device to eliminate the need for manual intervention by the surgeon. The amount of saline to be injected and / or the amount of tip cleaning can be learned by the algorithm over time to suit different surgeon preferences.

[0046] In one aspect of an embodiment, in addition to making the surgeon's life easier by eliminating repetitive and manual interventions, the system can also improve the quality of patient care and reduce the duration of surgical procedures, thereby saving costs and time for everyone involved in the surgical procedure. Furthermore, in the case of surgical smoke, the system can also reduce the risk of surgical personnel being exposed to hazardous compounds, such as those present in the gaseous byproducts of tissue dissection.

[0047] In the method 50 as outlined above, one of the primary goals is to cleanse the image in step 56. However, it is contemplated that step 56 may be adjusted such that activation and control of the video enhancer 38 and / or other surgical equipment is dependent upon factors other than cleaning the image alone. For example, during step 56 of controlling the video enhancer 38 and / or other surgical equipment, consideration may be given to minimizing procedure completion time, health issues for the patient and operating room staff, and optimal use of surgical equipment. As a specific example, the system controller 30 may know from input or from a surgical schedule database that the surgical procedure will take 5 hours and that the filter for the suction system 20 has only a 4 hour life when used at maximum suction. In a specific example, the suction system 20 may be reduced to not operate at maximum suction to extend the life of the filter such that step 56 is modified dependent upon factors other than cleaning the image alone. Alternatively, if the system controller 30 knows that the process will take 5 hours and the filter of the suction system 20 has only a 4 hour life when used at maximum suction, the system controller 30 can adjust the power level of the tool 12 to produce less smoke so that the suction system 20 will not have to overuse its filter, but still produce a decontaminated image in step 56 because the amount of smoke is reduced. Fig.12 A domain logic flow diagram is illustrated for performing step 56 when the video enhancer 38 and / or other surgical equipment is activated and controlled depending on factors other than decontaminating the image alone.

[0048] Fig.13 Another embodiment of a general method 40 is illustrated, wherein Fig.13 The method includes a method 80 for automatically recording video images from the video capture device 26. Fig.13 In the method, the first step includes capturing a video image using the video capture device 26 at step 82. In step 84, the video image is processed to determine whether a triggering event has occurred. If the triggering event has occurred, then at step 86 the system controller 30 automatically controls the video recorder 36 to record a video image from the video capture device 26 or to stop recording a video image from the video capture device 26. It is contemplated that the triggering event may be any event at the beginning of an occasion where it is desired to record a surgical procedure, or any event where it is desired to pause or stop recording a surgical procedure. For example, the video capture device 26 may be connected to an endoscope 31. During a laparoscopic cholecystectomy surgical procedure, the endoscope 31 may be introduced into the body via a trocar (not shown) to the surgical site 16, wherein the video capture device 26 undergoes certain known activities before and during insertion into the surgical site 16, such as camera white balancing, threading the endoscope 31 through the trocar, first exposure to the surgical site 16, and viewing blood. During step 84, each of the above examples can be determined and set as a triggering event, so that when the event occurs, the video recorder 36 automatically records the video images from the video capture device 26 at step 86. The video images will continue to be captured at step 82, thereby looking for the next triggering event. If the next triggering event occurs (e.g., an extended pause activity), the video recorder 36 automatically stops recording the video images from the video capture device 26 at step 86. It is contemplated that manual intervention can be used to start recording, and that the recording can be automatically stopped using method 80, and that manual intervention can be used to stop recording after automatically recording using method 80. Method 80 can be used to automatically start or stop recording multiple times during a surgical procedure. It is contemplated that method 80 can be used in combination with methods 40 and 50 (e.g., step 54 and step 84 can be the same event).

[0049] Example Imaging Agents for Imaging Tissue in a Surgical Site

[0050] In various embodiments, the systems and methods described herein can be used in medical imaging including various optical modalities, such as, for example, white light imaging, fluorescence imaging (e.g., using endogenous and exogenous fluorophores), or a combination thereof. In embodiments including fluorescent medical imaging applications, the imaging agent used in combination with the methods, systems, uses, and kits described herein is a fluorescent imaging agent, such as, for example, indocyanine green (ICG) dye. When ICG is supplied to a subject, it binds to blood proteins and circulates in tissues with the blood. The fluorescent imaging agent (e.g., ICG) can be supplied to a subject (e.g., into a vein or artery) as a bolus injection at a concentration suitable for imaging, so that the bolus circulates in the vasculature and passes through the microvasculature. In other embodiments in which multiple fluorescent imaging agents are used, such agents can be supplied simultaneously, for example, in a single bolus or sequentially in separate boluses. In some embodiments, the fluorescent imaging agent can be supplied by a catheter. In certain embodiments, the fluorescent imaging agent can be supplied less than one hour in advance of performing a measurement of the signal intensity caused by the fluorescent imaging agent. For example, the fluorescent imaging agent may be administered to the subject less than 30 minutes prior to taking a measurement. In other embodiments, the fluorescent imaging agent may be administered at least 30 seconds prior to taking a measurement. In other embodiments, the fluorescent imaging agent may be administered simultaneously with taking a measurement.

[0051] In some embodiments, the fluorescent imaging agent can be supplied at various concentrations to achieve a desired circulating concentration in the blood or other body tissues or in the fluid to which the fluorescent agent is supplied. For example, in an embodiment where the fluorescent imaging agent is ICG, it can be supplied at a concentration of about 2.5 mg / mL to achieve a circulating concentration of about 5 μM to about 10 μM in the blood. In various embodiments, the upper concentration limit for supplying the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes clinically toxic in the circulating blood or other body tissues or fluids, and the lower concentration limit is the instrument limit for acquiring signal intensity data to detect the fluorescent imaging agent, which is caused by the fluorescent imaging agent circulating with the blood or in other body tissues or fluids. In various other embodiments, the upper concentration limit for supplying the fluorescent imaging agent is the concentration at which the fluorescent imaging agent becomes self-quenching. For example, the circulating concentration of ICG can be in the range of from about 2 μM to about 10 mM. Therefore, in one aspect, the method described herein may include the following steps: before processing the signal intensity data, the imaging agent (e.g., fluorescent imaging agent) is supplied to the object and the signal intensity data (e.g., video) is acquired. In another aspect, the method may exclude any step of administering an imaging agent to the subject.

[0052] In embodiments, suitable fluorescent imaging agents used in fluorescent imaging applications alone or in combination with other imaging to generate fluorescent image data are imaging agents that can circulate with blood (e.g., fluorescent dyes that can circulate with, for example, components of blood (such as lipoproteins in blood or serum plasma)) and through the vasculature of tissues (i.e., macrovessels and microvasculature), and when the imaging agent is exposed to appropriate light energy (e.g., excitation light energy or absorption light energy), a signal intensity is caused by it. In some variations, the fluorescent imaging agent includes a fluorescent dye, an analog thereof, a derivative thereof, or a combination of these. The fluorescent dye includes any non-toxic fluorescent dye. In certain embodiments, the fluorescent dye emits fluorescence in the near-infrared spectrum. In certain embodiments, the fluorescent dye is or includes a tricarbocyanine dye. In certain embodiments, the fluorescent dye is or includes indocyanine green (ICG), methylene blue, or a combination thereof. In other embodiments, the fluorescent dye is or includes fluorescein isothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin, o-phthalaldehyde, fluorescamine, red bengal, trypan blue, fluorogold, or a combination thereof, which can be excited using an excitation light wavelength suitable for each dye. In some embodiments, analogs or derivatives of fluorescent dyes can be used. For example, fluorescent dye analogs or derivatives include fluorescent dyes that have been chemically modified but still retain their ability to fluoresce when exposed to light energy of the appropriate wavelength.

[0053] In an embodiment, the fluorescent imaging agent may be provided as a lyophilized powder, solid or liquid. In certain embodiments, the fluorescent imaging agent may be provided in a vial (e.g., a sterile vial), which may allow for reconstitution to a suitable concentration by supplying a sterile fluid using a sterile syringe for use as a supporting component for use with the systems and methods described herein. Reconstruction may be performed using any appropriate carrier or diluent. For example, the fluorescent imaging agent may be reconstructed using an aqueous diluent immediately before supply. In various embodiments, any diluent or carrier that maintains the fluorescent imaging agent in a solution may be used. As an example, ICG may be reconstructed using water. In some embodiments, once the fluorescent imaging agent is reconstructed, it may be mixed with additional diluents and carriers. In some embodiments, the fluorescent imaging agent may be conjugated to another molecule (such as a protein, peptide, amino acid, synthetic polymer or sugar), for example to enhance solubility, stability, imaging properties or a combination thereof. Additional buffers may be optionally added, and the additional buffers include Tris, HCl, NaOH, phosphate buffered saline and / or HEPES.

[0054] Those skilled in the art will appreciate that although fluorescent imaging agents are described in detail above, other imaging agents may be used in conjunction with the systems, methods, and techniques described herein, depending on the optical imaging modality.

[0055] In some variations, fluorescent imaging agents used in combination with the methods, systems, uses, and kits described herein may be used for blood flow imaging, tissue perfusion imaging, lymphatic imaging, or a combination thereof, or for imaging tissue or body structures (anatomy) (e.g., urinary system imaging including ureteral imaging), which may be performed during an invasive surgical procedure, a minimally invasive surgical procedure, or a non-invasive surgical procedure in combination with invasive and minimally invasive procedures. Examples of lymphatic imaging include identification of one or more lymph nodes, lymph node drainage, lymphatic mapping, or a combination thereof. In some variations, such lymphatic imaging may be associated with the female reproductive system (e.g., uterus, cervix, vulva).

[0056] In some variations related to any vascular application, (one or more) imaging agents (e.g., ICG alone or in combination with another imaging agent) can be injected intravenously. For example, imaging agents can be injected intravenously through a central venous line, a bypass pump and / or a cardioplegia line and / or other vascular system to flow and / or perfuse the coronary vasculature, microvasculature and / or graft or other blood vessels. ICG can be supplied downward along a graft vessel or other vascular system as a diluted ICG / blood / saline solution, so that the final concentration of ICG in the coronary artery or other vascular system depending on the application is approximately the same as or lower than the concentration caused by injecting about 2.5 mg (i.e., 1 ml of 2.5 mg / ml) into a central line or bypass pump. ICG can be prepared by dissolving, for example, 25 mg of a solid in 10 ml of a sterile aqueous solvent (which can be provided by the manufacturer together with ICG). One milliliter of ICG solution can be mixed with 500 ml of sterile saline (e.g., by injecting 1 ml of ICG into a 500 ml saline bag). Thirty milliliters of diluted ICG / saline solution can be added to 10 ml of the object's blood, which can be obtained from a central arterial line or bypass pump in a sterile manner. The ICG in the blood binds to plasma proteins and helps prevent leakage of blood vessels. The mixing of ICG with blood can be performed using standard aseptic techniques in the field of aseptic surgery. For each graft, ten ml of ICG / saline / blood mixture can be supplied. ICG can be supplied by a syringe attached to the (open) proximal end of the graft, rather than by injecting ICG through the wall of the graft using a needle. When harvesting the graft, the surgeon routinely attaches an adapter to the proximal end of the graft so that they can attach a syringe filled with saline, seal the distal end of the graft and inject saline down along the graft, thereby pressurizing the graft and therefore assessing the integrity of the catheter (relative to leakage, side branches, etc.) before performing the first anastomosis. In other variations, the method, dosage, or combination thereof combined with cardiac imaging as described herein can be used in any vascular and / or tissue perfusion imaging applications.

[0057] Lymphatic mapping is an important part of effective surgical staging for cancers that spread through the lymphatic system (e.g., breast cancer, gastric cancer, gynecological cancer). Removing multiple nodes from a specific node basin can lead to serious complications, including acute or chronic lymphedema, paresthesia and / or serous formation, and in fact, if the sentinel lymph node is negative for metastasis, the surrounding nodes are also likely to be negative. For example, in breast cancer surgery, the identification of tumor-draining lymph nodes (LN) has become an important step in staging cancers that spread through the lymphatic system. LN mapping involves the use of dyes and / or radioactive tracers to identify LNs for biopsy or resection, as well as subsequent metastatic pathological assessments. The goal of lymphadenectomy during surgical staging is to identify and remove LNs at high risk of local spread of cancer. Sentinel lymph node (SLN) mapping has been shown to be an effective surgical strategy in the treatment of breast cancer. It is generally based on the concept that if metastasis (spread of cancer to axillary LNs) is present, it should be located in the SLNs, which are defined in the art as the first LN or grouping of nodes to which cancer cells are most likely to spread from the primary tumor. If the SLNs are negative for metastasis, the surrounding secondary and tertiary LNs should also be negative. The main benefit of SLN mapping is to reduce the number of subjects who undergo traditional partial or complete lymph node dissection, and therefore reduce the number of subjects who suffer from associated diseases such as lymphedema and lymphocele.

[0058] Fluorescence imaging according to various embodiments may include applications for SLN visualization, mapping, facilitating direct real-time visual identification of LNs and / or afferent lymphatic channels during surgery, facilitating real-time high-resolution optical guidance through skin and adipose tissue, visualization of blood flow, tissue perfusion, or a combination thereof.

[0059] In some variations, visualization, classification, or both of lymph nodes during fluorescence imaging can be based on imaging of one or more imaging agents, which can be further based on visualization and / or classification using a gamma probe (e.g., Technetium Tc-99m is a clear, colorless aqueous solution and is typically injected into the areolar region as per standard of care), another conventionally used colored imaging agent (isosulfan blue), and / or other assessments, such as, for example, histology. ICG can be packaged with an aqueous solvent consisting of sterile water for injection, which is used to reconstitute the ICG. In some variations, the ICG dose (mg) in breast cancer sentinel lymph node mapping can range from about 0.5 mg to about 10 mg depending on the route of delivery. In some variations, the ICG dose can be from about 0.6 mg to about 0.75 mg, from about 0.75 mg to about 5 mg, from about 5 mg to about 10 mg. The delivery route can be, for example, subcutaneous, intradermal (e.g., into the areola region), subareolar, skin overlying the tumor, intradermal in the areola closest to the tumor, subcutaneous into the areola, intradermal above the tumor, areola over the entire breast, or a combination thereof. NIR fluorescence-positive LNs (e.g., using ICG) can be represented, for example, as (one or more) black and white NIR fluorescence images and / or as all or part color (white light) images, all or part desaturated white light images, enhanced color images, overlays (e.g., fluorescence with any other image), composite images (e.g., fluorescence incorporated into another image), which can have various colors, various levels of desaturation, or various ranges of colors to highlight / visualize certain features of interest. Processing of the images can be further performed for further visualization and / or other analysis (e.g., quantification). According to various embodiments of the combination of gamma probes (Tc-99m) used for ICG and SLN alone or in accordance with the American Society of Breast Surgeons (ASBrS) practice guidelines for SLN biopsy in breast cancer patients, lymph nodes and lymphatic vessels can be visualized (e.g., intraoperatively, in real time) using fluorescence imaging systems and methods. Fluorescence imaging of LNs can be started from the injection site by tracing the lymphatic channels leading to the LNs in the axilla. Once the visual image of the LN is identified, LN mapping and identification of the LN can be completed through the incised skin, and LN mapping can be performed until the ICG visualization node is identified. For comparison, mapping using isosulfan blue can be performed until the "blue" node is identified. LNs identified using ICG alone or in combination with another imaging technique (e.g., isosulfan blue and / or Tc-99m) can be marked for resection. The subject may have various stages of breast cancer (e.g., IA, IB, IIA).

[0060] In some variations, such as, for example, in gynecological cancers (e.g., uterine cancer, endometrial cancer, vulvar cancer, and cervical malignancies), ICG may be provided interstitially for visualization of lymph nodes, lymphatic channels, or a combination thereof. When injected interstitially, the protein binding properties of ICG cause it to be rapidly absorbed by the lymph and move to the SLN through the conducting blood vessels. ICG may be provided for injection in the form of a sterile lyophilized powder containing 25 mg of ICG (e.g., 25 mg / vial) with no more than 5.0% sodium iodide. The ICG may then be reconstituted with commercially available water for injection (sterile) prior to use. According to an embodiment, a vial containing 25 mg of ICG may be reconstituted in 20 ml of water for injection, resulting in a 1.25 mg / ml solution. A total of 4 ml of this 1.25 mg / ml solution is to be injected into the subject at a total dose of 5 mg of ICG per subject (4x1 ml injections). The cervix may also be injected with 1 ml of 1% isosulfan blue 10 mg / ml solution (for comparison purposes) four (4) times at a total dose of 40 mg. The injections may be performed while the subject is anesthetized in the operating room. In certain variations, the ICG dose (mg) in gynecological cancer sentinel lymph node detection and / or mapping may range from about 0.1 mg to about 5 mg depending on the route of delivery. In some variations, the ICG dose may be about 0.1 mg to about 0.75 mg, about 0.75 mg to about 1.5 mg, about 1.5 mg to about 2.5 mg, about 2.5 mg to about 5 mg. The route of delivery may be, for example, cervical injection, vulvar peritumoral injection, hysteroscopic endometrial injection, or a combination thereof. In order to minimize spillage of isosulfan blue or ICG interfering with the mapping process when LNs are to be resected, mapping may be performed on the hemipelvis, and mapping may be performed using both isosulfan blue and ICG prior to resection of any LNs. LN mapping can be performed for clinical stage I endometrial cancer according to the NCCN Uterine Cancer Guidelines, SLN Algorithm for Surgical Staging of Endometrial Cancer; and SLN mapping can be performed for clinical stage I cervical cancer according to the NCCN Cervical Cancer Guidelines, Surgery / SLN Mapping Algorithm for Early Cervical Cancer. Thus, identification of LNs can be based on ICG fluorescence imaging alone, or in combination or co-administration with a colorimetric dye (isosulfan blue) and / or a radioactive tracer.

[0061] The visualization of lymph nodes can be qualitative and / or quantitative. Such visualization can include, for example, lymph node detection, detection rate, and anatomical distribution of lymph nodes. The visualization of lymph nodes according to various embodiments can be used alone or in combination with other variables (e.g., vital signs, height, weight, demographics, surgical prediction factors, relevant medical history and basic conditions, histological visualization and / or evaluation, Tc-99m visualization and / or evaluation, concomitant medications). Regular review visits can occur on the date of discharge and on subsequent dates (e.g., one month).

[0062] Lymphatic fluids include high levels of protein, so ICG can bind to endogenous proteins when entering the lymphatic system. When used according to the methods and systems described herein, fluorescence imaging (e.g., ICG imaging) for lymphatic mapping provides the following example advantages: high signal to background ratio (or tumor to background ratio) because NIR does not generate significant autofluorescence; real-time visualization features for lymphatic mapping; tissue definition (i.e., structural visualization); rapid excretion and elimination after entering the vascular system and avoiding non-ionizing radiation. In addition, NIR imaging has superior tissue penetration (approximately 5 to 10 mm tissue) than visible light (1 to 3 mm tissue). The use of ICG, for example, also helps in visualization through the peritoneum covering the node beside the aorta. Although tissue fluorescence can be observed for an extended period of time using NIR light, it cannot be seen using visible light, and therefore does not affect the pathological assessment or treatment of LN. Moreover, compared with blue staining of lymph nodes (isosulfan blue), it is easier to detect fluorescence intraoperatively. In other variations, methods, dosages, or combinations thereof as described herein in connection with lymphatic imaging may be used in any vascular and / or tissue perfusion imaging application.

[0063] In various embodiments, methods, systems, uses, fluorescent agents, and kits can be used for tissue perfusion imaging. Tissue perfusion refers to microcirculatory blood flow per unit tissue volume, in which oxygen and nutrients are provided to the capillary beds of the perfused tissue and waste products are removed from them. Tissue perfusion is a phenomenon related to but different from blood flow in blood vessels. Quantified blood flow through blood vessels can be expressed in terms that define flow (i.e., volume / time) or define velocity (i.e., distance / time). Tissue blood perfusion defines the movement of blood through microvasculature (such as arterioles, capillaries, or venules within a tissue volume). Quantified tissue blood perfusion can be expressed in terms of blood flow through a tissue volume, i.e., blood volume / time / tissue volume (or tissue mass). Perfusion is associated with nutrient vessels (e.g., microvessels called capillaries), which include vessels associated with the exchange of metabolites between blood and tissue, rather than large diameter non-nutrient vessels.

[0064] Embodiments include a kit for imaging tissue in a surgical site, wherein the kit includes a method as used in any of the methods described herein. Figure 1 Another embodiment includes the use of the kit of the foregoing statements for lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomy imaging, or a combination thereof. Another embodiment includes the use of the kit of the foregoing statements for lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomy imaging, or a combination thereof. Figure 1A fluorescent imaging agent 100 for use with a surgical system for imaging tissue in a surgical site 16 in conjunction with any of the methods described herein. Another embodiment includes the fluorescent imaging agent 100 of the preceding statement, wherein imaging tissue in a surgical site includes imaging blood flow, tissue perfusion, lymphatic tissue, tissue anatomy, or a combination thereof. Another embodiment includes a fluorescent imaging agent for use with a surgical system for imaging tissue in a surgical site 16 in conjunction with any of the methods described herein. Figure 2 , 3 or any of the methods of 13 for use in conjunction with a fluorescent imaging agent 100 for imaging tissue in a surgical site. Another embodiment includes the fluorescent imaging agent 100 of the preceding statement, wherein imaging tissue in a surgical site includes imaging blood flow, tissue perfusion, lymphatic tissue, tissue anatomy, or a combination thereof. Another embodiment includes using Figure 1 A system in conjunction with any of the methods described herein for lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomy imaging, or a combination thereof. Another embodiment includes using Figure 2 , 3 Or the method of 13 is used for lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomy imaging or a combination thereof.

[0065] Although the present disclosure has been illustrated and described in conjunction with various embodiments shown and described in detail, it is not intended to be limited to the details shown, because various modifications and structural changes can be made without departing from the scope of the present disclosure in any way. Without departing from the scope of the present disclosure in any way, various modifications of the arrangement of forms, parts, steps, details and the operating order of the illustrated embodiments and other embodiments of the present disclosure can be made, and for those skilled in the art, the various modifications will be obvious when referring to the description. It is therefore envisaged that the attached claims will cover such modifications and embodiments because they fall within the true scope of the present disclosure. For the purpose of clarity and simplicity, the features are described herein as parts of the same or separated embodiments, however, it will be appreciated that the scope of the present disclosure includes embodiments with all or some combinations of the described features. For the terms "for example" and "such as" and their grammatical equivalents, unless otherwise expressly stated, the phrase "and without limitation" is understood to follow. As used herein, the singular forms of "one", "one" and "said" include plural indicators unless the context clearly indicates otherwise.

Claims

1. A system for providing improved video images of a surgical site, comprising: tools for manipulating tissue at the surgical site; a light source for providing light to the surgical site; a video capture device for obtaining video images of the surgical site; An image display, used for displaying the video image; as well as a system controller configured to maintain the quality of the video image obtained by the video capture device and provided to the image display, wherein the system controller receives and processes the video image to detect a degradation in the quality of at least one video frame of the video image by determining a video signature corresponding to a condition that interferes with the quality of the video image; The system controller interacts with a video enhancer to enhance the at least one video frame of the video image from the video capture device to automatically control the video enhancer to enhance the at least one video frame of the video image transmitted from the video capture device to the image display when a reduction in the quality of the at least one video frame of the video image is detected, so that a user is relieved from having to control the video enhancer to obtain the improved video image of the surgical site for viewing on the image display.

2. The system according to claim 1, comprising: a suction system for providing suction at the surgical site; The system controller interacts with the video enhancer, the tool, and the suction system and controls at least one of the video capture device, the video enhancer, the tool, and the suction system to address the condition at the surgical site to restore the video image to an improved quality for observation, thereby eliminating the need for a user to manually control any of the video capture device, the video enhancer, the tool, and the suction system to obtain the improved video image of the surgical site for observation on the image display.

3. The system of claim 2, wherein the video signature to be identified corresponds to a condition of smoke in the video image, and the system controller operates at least one of the video capture device, the video enhancer, the tool, and the suction system in response to (1) smoke characteristics determined by the system controller and (2) at least one other surgical input provided to the system controller.

4. The system according to any one of claims 1 to 3, further comprising: an insufflator for providing gas to the surgical site; as well as A suction system for providing suction at the surgical site to control the removal of gas from the surgical site; wherein the tool is a cautery tool.

5. The system of claim 4, wherein: the system controller interacts with the tool, the insufflator, and the aspiration system; and The system controller also automatically controls at least one of the tool, the insufflator, and the aspiration system to address the condition at the surgical site to restore the video image to an improved quality.

6. The system of claim 4, wherein: wherein the video signature to be identified corresponds to the condition of smoke in the video image, and the system controller operates at least one of the tool, the insufflator, and the suction system in response to the sensed amount of smoke.

7. The system of claim 4, wherein: The video signature to be identified corresponds to the condition of smoke in the video image.

8. The system of claim 4, wherein: The video signature to be identified corresponds to the condition of blood in the video image.

9. The system according to claim 4, further comprising: a fluid pump system for providing fluid to the surgical site; as well as a suction system that provides suction for controlled removal of fluid from the surgical site; The tool is a cutting tool.

10. The system according to any one of claims 1 to 3, further comprising: an insufflator for providing gas to the surgical site; and Wherein the tool is a cautery tool.

11. The system of claim 10, wherein: The system controller also automatically controls the insufflator to address the conditions at the surgical site to restore the video image to improved quality.

12. The system according to claim 10, further comprising: a fluid pump system for providing fluid to the surgical site; The tool is a cutting tool.

13. The system of claim 3, wherein: The at least one other surgical input includes a characteristic of at least one of the tool, the suction system, the light source, and the video capture device.

14. The system of claim 3, wherein: The at least one other surgical input includes a surgical procedure type.

15. The system of claim 3, wherein: The at least one other surgical input comprises a surgeon preference.

16. The system of any one of claims 1-15, which is used for lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomical structure imaging, or a combination thereof.

17. The system of any one of claims 1-15 for use in robotic surgery.

18. A method for providing an improved image of a surgical site, comprising: obtaining a video image of the surgical site; detecting a reduction in quality of at least one video frame of the video image by determining the presence of a condition that interferes with the quality of the video image; configuring the system controller to interact with the video enhancer; in response to a degradation in the quality of the at least one video frame of the video image and without control from an operator, controlling, with the system controller, the video enhancer to generate the at least one video frame of the video image having improved image quality; as well as The video image is displayed with the improved image quality.

19. A kit for imaging tissue in a surgical site, the kit comprising a fluorescent imaging agent and the system of any one of claims 1-15.

20. The kit of claim 19, for use in lymphatic imaging, blood flow imaging, tissue perfusion imaging, tissue anatomical structure imaging, or a combination thereof.

21. The kit of claim 19 for use in robotic surgery.

22. A fluorescent imaging agent for use with the system of any one of claims 1-15 for imaging tissue in a surgical site.

23. The fluorescent imaging agent of claim 22, wherein imaging tissue in the surgical site comprises imaging blood flow, tissue perfusion, lymphatic tissue, tissue anatomy, or a combination thereof.

24. A fluorescent imaging agent for use with the method of claim 18 for imaging tissue in a surgical site.

25. The fluorescent imaging agent of claim 24, wherein imaging tissue in the surgical site comprises imaging blood flow, tissue perfusion, lymphatic tissue, tissue anatomy, or a combination thereof.

26. The fluorescent imaging agent according to any one of claims 22-25, further for use in robotic surgery.

Images