Surgical Instrument Tracking Using NIR Fluorescent Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In surgical robotic systems, the low visibility of surgical instruments in monochromatic NIR imaging modes makes it difficult for surgeons to accurately position and move instruments relative to fluorescent tissue structures, as the instruments do not fluoresce and are not clearly visible within the endoscopic field of view.
Innovation Solution
The surgical robotic system incorporates an imaging system that can operate in multiple modes, including an overlay mode that combines white and NIR images, and a monochromatic mode that displays the NIR/ICG signal for enhanced differentiation. Additionally, the system uses fluorescent fiducial markers on surgical instruments to enhance their visibility, allowing for better instrument tracking and visualization during procedures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If monochromatic NIR imaging mode is used to enhance visualization of fluorescent tissue structures, then the differentiation of tissue structures is improved, but the visibility of surgical instruments deteriorates
Solution Approach 1:
The patent applies color changes by mapping NIR signal intensity to different visible colors in an overlay image. Instruments are rendered in colors that contrast with the tissue fluorescence colors, allowing simultaneous visualization of both instruments and tissue structures in monochromatic NIR mode. This resolves the contradiction by transforming the invisible instrument signals into visible color representations without compromising tissue structure differentiation.
Solution Approach 2:
The patent merges white light imaging and NIR fluorescence imaging into a single overlay image that displays both instrument positions and tissue fluorescence simultaneously. By combining these two imaging modalities into one composite visualization, the system allows surgeons to see both instruments and fluorescent tissue structures in monochromatic NIR mode, eliminating the need to switch between modes and resolving the visibility contradiction.
2Difficulty of detecting and measuring
If imaging mode is switched to white light mode to visualize instruments, then the visibility of instruments is improved, but the differentiation of fluorescent tissue structures deteriorates
Solution Approach 1:
The system combines white light and NIR fluorescence imaging data into a single overlay image that preserves both instrument visibility and tissue fluorescence differentiation. Surgeons can view instruments and fluorescent tissue structures simultaneously in monochromatic NIR mode without switching to white light mode, maintaining both visualization quality and tissue differentiation.
Solution Approach 2:
Instruments are displayed in specific colors within the overlay image that distinguish them from fluorescent tissue structures. This color differentiation allows both instruments and tissue to be clearly visualized simultaneously, eliminating the need to switch imaging modes and preserving tissue structure differentiation while improving instrument visibility.
3Adaptability or versatility
If multiple imaging modes are provided to address different visualization needs, then the versatility of the imaging system is improved, but the complexity of mode switching and operation deteriorates
Solution Approach 1:
The patent merges the functionality of multiple imaging modes (white light and monochromatic NIR) into a single overlay image mode. This eliminates the need for mode switching while providing the benefits of both modes simultaneously, improving ease of operation while maintaining versatility through the integrated visualization capability.
Solution Approach 2:
The overlay image mode serves multiple functions simultaneously: it visualizes instruments, displays tissue fluorescence, and provides depth information. This multi-functional single mode replaces the need for separate imaging modes, improving ease of operation while maintaining the versatility needed for different surgical visualization requirements.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables clear visualization of surgical instruments within the endoscopic field of view, even in monochromatic NIR imaging modes, thereby improving the surgeon's ability to accurately position and move instruments during surgical procedures, reducing the need to switch between imaging modes.
Implementation Method 1
instrument tracking and visualization in near infra-red (NIR) imaging mode using NIR reflection
Implementation Method 2
fluorescence excitation light excites fluorophores in the tissue, which emit fluorescence light at an emission wavelength, which is typically greater than the excitation wavelength
Data Source
AI summary
A surgical robotic system is paired with an imaging system that may operate in a plurality of imaging modes, including white light and near infrared (NIR) light. The robotic system includes a plurality of robotic arms, one of which controls a laparoscopic camera providing video data based on the selected imaging mode to an image processing device. One or more other arms control one or more corresponding robotic instruments, which include fiducial markers formed from a fluorescent material detectable in the NIR imaging mode. The markers allow for visualization of the instruments in the NIR imaging mode, which is monochromatic. The image processing device also generates masked or virtual overlays of the instrument and registers the overlays with the instruments in the monochromatic video feed, providing additional visualization. Registration of the overlays is performed using fiducial markers, which act like landmarks for aligning the overlay to the actual image of the instrument in the video feed.