Endoscopic Video Fusion for Higher Resolution and Wider Field of View
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Endoscopic imaging systems are limited by resolution and field of view, which can prolong interventions and hinder the provision of complete anatomical information to physicians.
Innovation Solution
An endoscopic system that applies spatial transforms to image frames, converting them into cylindrical coordinates, generates a map image with higher resolution, aligns and fuses current frames with the map to enhance spatial resolution and field of view, and incorporates EM tracking for precise anatomical guidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the endoscopic imager uses standard resolution imaging, then the device complexity is low, but the spatial resolution of the image is insufficient
Solution Approach 1:
The patent transforms the 2D image data into a 3D volumetric representation by stacking multiple 2D image frames along the temporal dimension and applying temporal convolutional networks. This dimensional transformation enables super-resolution reconstruction without requiring additional physical sensors or complex optical systems, thereby improving spatial resolution while avoiding increased device complexity
Solution Approach 2:
The system creates multiple virtual copies of the low-resolution image data through temporal stacking and generative adversarial network synthesis. By generating multiple high-resolution versions from a single low-resolution input through iterative refinement, the system achieves enhanced spatial resolution without adding physical imaging components
2Area of stationary object
If the endoscopic imager uses a narrow field of view, then the device complexity is low, but the field of view coverage is insufficient
Solution Approach 1:
The patent extends the 2D field of view into the temporal dimension by stacking multiple sequential 2D frames to create a 3D volumetric representation. This allows the system to synthesize a wider effective field of view by combining information from multiple time points without requiring a physically wider lens or multiple cameras
Solution Approach 2:
The system performs preliminary registration and alignment of multiple image frames before fusion, preparing the data in advance for comprehensive field of view synthesis. By pre-aligning frames based on temporal relationships and anatomical landmarks, the system can seamlessly combine views to expand the effective field of view without real-time complex processing
3Manufacturing precision
If multiple image frames are processed to enhance resolution, then the spatial resolution improves, but the processing time increases
Solution Approach 1:
The system uses periodic temporal sampling of image frames, processing only selected frames at regular intervals rather than continuously processing every frame. This periodic approach maintains high spatial resolution through sufficient temporal sampling while reducing overall processing time by skipping redundant frames between sampled points
Solution Approach 2:
The system performs preliminary downsampling and feature extraction from multiple frames before the final super-resolution reconstruction. By pre-processing frames to extract essential features and reduce data dimensionality beforehand, the system reduces the computational burden of the main reconstruction algorithm, thereby decreasing processing time while preserving spatial resolution quality
Data Source
AI summary
An endoscopic system includes an endoscopic imager configured to capture image frames of a target site within a living body and a processor configured to apply a spatial transform to a preliminary set of image frames, the spatial transform converting the image frames into cylindrical coordinates; calculate a map image from the spatially transformed image frames, each pixel position in the map image being defined with a vector of fixed dimension; align a current image frame with the map image and apply the spatial transform to the current image frame; fuse the spatially transformed current image frame to the map image to generate a fused image; and apply an inverse spatial transform to the fused image to generate an enhanced current image frame having a greater spatial resolution than the current image frame. The system also includes a display displaying the enhanced current image frame.