Sensorless Endoscope Navigation via Virtual Viewpoint Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing endoscope navigation methods require sensors to accurately position the endoscope, which can be cumbersome and limit precise navigation without depth information in fluoroscopic images.
Innovation Solution
An image processing device that acquires three-dimensional images and radiation images to derive virtual viewpoints from real endoscopic images, allowing for sensorless navigation by matching virtual and real endoscopic images, adjusting viewpoints based on reliability criteria, and sequentially displaying virtual and real images for precise positioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a sensor is provided in the endoscope to detect the position of the endoscope, then the position detection accuracy is improved, but the device complexity and ease of operation are worsened
Solution Approach 1:
The patent creates a virtual copy of the endoscopic scene by synthesizing a virtual endoscopic image from the three-dimensional image data. This virtual image serves as a surrogate for direct sensor measurement, allowing position estimation without physical sensors in the endoscope. The virtual viewpoint position is derived by matching features between the virtual endoscopic image and the actual endoscopic image, effectively copying the measurement function through image processing rather than direct sensing.
Solution Approach 2:
The patent introduces a three-dimensional image (CT or MRI data) as an intermediary medium between the endoscope and the position detection system. Instead of directly measuring endoscope position with sensors, the system uses the pre-acquired three-dimensional image as a reference model to indirectly infer the endoscope's position and orientation by comparing it with the actual endoscopic views. This intermediary approach eliminates the need for sensors while maintaining navigation capability.
2Productivity
If fluoroscopic imaging is performed to display real-time images during treatment, then the navigation capability is improved, but the depth information is lost making accurate positioning difficult
Solution Approach 1:
The patent transitions from two-dimensional fluoroscopic imaging to three-dimensional virtual endoscopic imaging by utilizing three-dimensional image data (CT or MRI). This dimensional enhancement restores depth information that is inherently lost in flat fluoroscopic images. The system generates a virtual endoscopic image with full three-dimensional spatial context, allowing operators to perceive depth and spatial relationships accurately while maintaining real-time navigation capability.
Solution Approach 2:
The patent creates a virtual copy of the endoscopic scene with complete three-dimensional information by rendering a virtual endoscopic image from the three-dimensional image data. This virtual image preserves depth cues and spatial relationships that are absent in conventional fluoroscopic images, effectively copying the rich three-dimensional information into a format that can be displayed and used for navigation in real-time.
3Measurement precision
If virtual endoscopic images are generated from three-dimensional images to assist navigation, then the depth perception is improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent performs preliminary processing of the three-dimensional image data before the actual navigation procedure. The three-dimensional image (CT or MRI) is pre-acquired and pre-processed to create a ready-to-use virtual model of the patient's anatomy. This preliminary action allows the virtual endoscopic images to be generated rapidly during the procedure by simply rendering different viewpoints of the pre-processed three-dimensional data, rather than processing raw three-dimensional image data in real-time.
Data Source
AI summary
A processor acquires a three-dimensional image of a subject, acquires a radiation image of the subject having a lumen structure into which an endoscope is inserted, acquires a first real endoscopic image in the lumen structure of the subject captured at a first time point by the endoscope, derives a provisional virtual viewpoint in the three-dimensional image of the endoscope using the radiation image and the three-dimensional image, derives a virtual viewpoint at the first time point in the three-dimensional image of the endoscope using the provisional virtual viewpoint, the first real endoscopic image, and the three-dimensional image, and derives a virtual viewpoint at a second time point after the first time point in the three-dimensional image of the endoscope using the first real endoscopic image and a second real endoscopic image captured by the endoscope at the second time point.