Multi-Sensor Surgical Imaging for Real-Time NIR Fluorescence Overlay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing minimally invasive surgical systems face challenges in providing simultaneous visualization of near-infrared (NIR) fluorescence and visible-range images without degrading spatial resolution or requiring time-consuming mode switching, which hinders real-time observation of fluorescence during surgical procedures.
Innovation Solution
A method and system that utilizes a multi-sensor camera with correlated sensors to predict NIR-induced fluorescence information from visible-range data, allowing concurrent display of an approximate full-color image with overlaid NIR fluorescence, using non-linear color mapping to enhance visualization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a multi-sensor camera with NIR capability is used to capture both visible and NIR fluorescence simultaneously, then real-time fluorescence visualization is achieved, but device complexity increases
Solution Approach 1:
The patent combines multiple sensors (visible spectrum sensors and NIR sensor) into a single integrated camera system, allowing simultaneous capture of both visible and NIR fluorescence images without requiring separate imaging devices or time-multiplexed operation
Solution Approach 2:
The imaging system is designed to perform multiple functions using a single multi-sensor camera: capturing visible light images for anatomical reference and NIR fluorescence images for functional visualization, eliminating the need for separate specialized imaging devices
2Measurement precision
If separate imaging modes are used for visible and NIR fluorescence, then measurement precision is maintained, but time consumption increases due to mode switching
Solution Approach 1:
The multi-sensor camera enables continuous simultaneous capture of both visible and NIR fluorescence images without interrupting the imaging process or requiring switching between different operational modes, maintaining uninterrupted real-time visualization
Solution Approach 2:
The system performs preliminary calibration and establishes correlation between visible and NIR sensor responses before actual imaging, enabling real-time prediction and combination of fluorescence information without delay during surgical procedures
3Loss of information
If NIR fluorescence is overlaid on visible images, then information completeness is improved, but color accuracy deteriorates due to non-linear mapping
Solution Approach 1:
The system transforms the color representation parameters by applying non-linear color mapping functions that adapt the color space to simultaneously preserve anatomical color information from visible images and functional fluorescence information from NIR images, optimizing the display for surgical decision-making
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables real-time visualization of NIR fluorescence on visible-range images without degrading spatial resolution, allowing surgeons to operate continuously while observing fluorescence, thus improving surgical efficiency and accuracy.
Implementation Method 1
a third sensor configured to capture the surgical scene as illuminated by a near-infrared (NIR) light source
Data Source
AI summary
An illustrative method includes receiving data representing information captured using a first sensor and a second sensor of a multi-sensor camera associated with a surgical device. The sensors are configured to capture a surgical scene comprising biological tissues as illuminated by a light source configured to emit wavelengths in the visible spectrum corresponding to sensing capabilities of the sensors, respectively. The method further comprises receiving data representing information captured using a third sensor of the multi-sensor camera, the third sensor configured to capture the surgical scene as illuminated by a NIR light source, generating a first visual representation of the surgical scene based on the data representing the information captured using the sensors, combining the first visual representation with the information captured using the third sensor to generate a second visual representation of the surgical scene, and presenting the second visual representation of the surgical scene on a display device.