Bronchoscope Guidance via 3D-2D Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current bronchoscopy guidance methods are limited by slow registration processes, instability in tracking due to coplanar points, and scale ambiguity in essential matrix calculations, which hinder precise navigation during lung cancer assessments.
Innovation Solution
A system and method utilizing fast coarse tracking followed by gradient-based fine registration, employing 3D-2D pose estimation and correlation-based cost functions to continuously guide bronchoscope motion, addressing drift errors through iterative refinement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Powell's method is used for image registration, then registration accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The patent segments the registration process into two distinct stages: a fast coarse registration phase that provides initial alignment, and a subsequent fine registration phase that refines accuracy. This segmentation allows the system to achieve both speed (in the coarse phase) and precision (in the fine phase), resolving the contradiction between processing speed and registration accuracy.
2Extent of automation
If essential matrix calculation is used for tracking, then motion estimation is achieved, but scale ambiguity and instability from coplanar points worsen measurement precision
Solution Approach 1:
The patent introduces an intermediary depth map derived from CT image data as a mediator between the 2D video frames and 3D pose estimation. This depth information serves as additional constraints that resolve the scale ambiguity inherent in essential matrix calculations and provide stability when tracking coplanar points, thereby improving measurement precision while maintaining automated motion estimation.
3Reliability
If optical-flow approach is used for tracking, then continuous motion tracking is achieved, but computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary action by pre-computing a depth map from CT image data before the bronchoscopy procedure begins. This pre-computed depth information is then used during the procedure to constrain and guide the optical-flow tracking, reducing the computational search space and processing time while maintaining continuous and reliable motion tracking throughout the procedure.
Data Source
AI summary
A method provides guidance to the physician during a live bronchoscopy or other endoscopic procedures. The 3D motion of the bronchoscope is estimated using a fast coarse tracking step followed by a fine registration step. The tracking is based on finding a set of corresponding feature points across a plurality of consecutive bronchoscopic video frames, then estimating for the new pose of the bronchoscope. In the preferred embodiment the pose estimation is based on linearization of the rotation matrix. By giving a set of corresponding points across the current bronchoscopic video image, and the CT-based virtual image as an input, the same method can also be used for manual registration. The fine registration step is preferably a gradient-based Gauss-Newton method that maximizes the correlation between the bronchoscopic video image and the CT-based virtual image. The continuous guidance is provided by estimating the 3D motion of the bronchoscope in a loop. Since depth-map information is available, tracking can be done by solving a 3D-2D pose estimation problem. A 3D-2D pose estimation problem is more constrained than a 2D-2D pose estimation problem and does not suffer from the limitations associated with computing an essential matrix. The use of correlation-based cost, instead of mutual information as a registration cost, makes it simpler to use gradient-based methods for registration.