Endoscope Viewpoint Difference Estimation Using Depth Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing endoscopic navigation methods face challenges in accurately determining the viewpoint difference between real and virtual endoscopic images, particularly in tubular structures like the bronchus, due to noise and differences in image representation, which can hinder precise navigation to target positions.
Innovation Solution
An information processing apparatus and method that utilize a processor to estimate viewpoint differences by combining real and virtual images, along with depth images generated from these images, using trained models to transform and align the images accurately, thereby improving navigation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real endoscopic images are used for navigation, then the visual representation of the tubular structure interior is clear, but the viewpoint position cannot be accurately determined due to noise and lack of depth information
Solution Approach 1:
The patent combines real endoscopic images with virtual endoscopic images and depth images to create a composite navigation system. The real image provides visual clarity of the tubular structure interior, while the virtual image and depth image compensate for the lack of viewpoint position information, achieving accurate navigation through information fusion
Solution Approach 2:
The patent introduces virtual endoscopic images and depth images as intermediary elements to bridge the gap between real endoscopic images and viewpoint position determination. These intermediaries provide the missing depth and position information that cannot be obtained from real images alone
2Measurement precision
If virtual endoscopic images are generated from three-dimensional images, then the viewpoint position can be determined, but the visual clarity and texture detail are reduced compared to real endoscopic images
Solution Approach 1:
The patent merges real endoscopic images with high visual clarity and texture detail, and virtual endoscopic images with accurate viewpoint position information. This combination allows the system to maintain image quality while achieving precise navigation through the integration of multiple image sources
3Measurement precision
If image transformation is applied to align real and virtual images, then the viewpoint difference can be estimated, but the processing complexity and time increase
Solution Approach 1:
The patent replaces complex mechanical image alignment procedures with a learning model-based approach. The learning model automatically estimates viewpoint differences by learning from training data, substituting iterative mechanical transformation processes with a more efficient computational model that reduces processing complexity
Data Source
AI summary
An information processing apparatus including at least one processor, wherein the processor is configured to: use at least one of a real image, which represents an interior wall of a tubular structure, or a first depth image, which represents a distance for each pixel from a viewpoint of the real image to the interior wall, and at least one of a virtual image, which represents, in a pseudo manner, the interior wall as viewed from a virtual viewpoint predetermined in a three-dimensional image, or a second depth image, which represents a distance for each pixel from the virtual viewpoint to the interior wall, to perform viewpoint difference estimation processing of estimating a viewpoint difference between the viewpoint of the real image and the virtual viewpoint; and use at least one of the first depth image or the second depth image in the viewpoint difference estimation processing.