Endoscope Image Signal Processing for Depth-Based Illumination Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Minimally invasive surgical procedures face challenges in capturing clear video due to non-uniform scene brightness, where anatomy at greater depths is poorly illuminated, leading to over- or under-exposure issues in endoscope images, which can be exacerbated by auto-exposure mechanisms.
Innovation Solution
The system generates high dynamic range (HDR) video frames by performing depth estimation for each pixel, capturing multiple images at a higher frame rate, and adjusting image capture parameters such as light source output and sensor settings to compensate for brightness fall-off with depth, merging these images to create a composite frame that maintains consistent illumination across the surgical site.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If auto-exposure mechanisms are used to adjust image capture parameters, then overall image brightness is improved, but non-uniform illumination across different depths is worsened
Solution Approach 1:
The scene is segmented into multiple depth ranges, with separate image capture parameters determined for each depth range. This allows independent optimization of exposure settings for near, mid, and far regions, resolving the contradiction between overall brightness and illumination uniformity across different depths
Solution Approach 2:
Different image capture parameters are applied to different spatial regions (depth ranges) within the scene. Each region receives locally optimized exposure settings based on its specific illumination conditions, rather than applying a single global parameter set that cannot simultaneously satisfy uniformity and overall brightness requirements
2Manufacturing precision
If multiple images are captured at higher frame rate to generate HDR video, then image quality is improved, but processing time and device complexity are worsened
Solution Approach 1:
Depth information is determined in advance for each pixel before the actual image capture and compositing processes. This preliminary depth estimation enables efficient organization and processing of multiple captured images, reducing computational complexity during the HDR generation phase while maintaining high image quality
Solution Approach 2:
The system dynamically adjusts image capture parameters based on real-time depth information and scene conditions. By adaptively modifying exposure settings for different depth ranges and combining multiple dynamically captured images, the system achieves high image quality without requiring static, overly complex processing pipelines
Data Source
AI summary
One example method includes obtaining, from an endoscope and between first and second video frames of a real-time video having a frame rate, a preliminary image of a scene during a surgical procedure, the first image comprising a plurality of pixels, wherein the first and second video frames are consecutive video frames in the real-time video; determining, for each pixel of the plurality of pixels, a depth within the scene; determining first image capture parameters for the scene based on a scene illumination setting and a first set of pixels within a first range of depths in the scene; capturing, between the first and second consecutive output video frames, a first image using the first image capture parameters; determining an illumination correction for a second set of pixels at a second range of depths within the scene; capturing, between the first and second consecutive output video frames, a second image using the illumination correction; generating a composite image based on the first and second images; and outputting the composite image as the second output video frame.