Augmented 3D Instrument Pose Estimation in Fluoroscopy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Medical procedures involving instruments that lack vision capabilities or are outside the camera's viewing range face challenges in navigating within the human anatomy due to the lack of three-dimensional context in two-dimensional fluoroscopic images, which can lead to unsafe contact with anatomy or other instruments.
Innovation Solution
A medical system that generates an augmented representation of two-dimensional images by incorporating three-dimensional pose information of instruments using sensor data and robotic data, providing operators with a three-dimensional context for navigation and control through an augmented visualization interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two-dimensional fluoroscopic images are used for instrument navigation, then the imaging process is simple and fast, but the three-dimensional context is lost leading to unsafe contact with anatomy
Solution Approach 1:
The patent overlays three-dimensional instrument pose representations (position and orientation) onto two-dimensional fluoroscopic images. This adds dimensional information about instrument location and orientation without requiring full three-dimensional imaging, thus recovering lost spatial context while maintaining the simplicity of two-dimensional imaging.
Solution Approach 2:
The system uses sensor data (electromagnetic trackers, optical sensors) as an intermediary to capture instrument pose information that is then integrated with the fluoroscopic images. This intermediary provides the missing three-dimensional context without requiring complex three-dimensional imaging hardware.
2Measurement precision
If instruments lack vision capabilities or are outside camera viewing range, then the instrument design is simpler, but navigation precision deteriorates
Solution Approach 1:
External sensors (electromagnetic trackers, optical motion capture systems) serve as intermediaries to measure instrument position and orientation without requiring vision capabilities built into the instruments themselves. This maintains instrument simplicity while achieving precise navigation through external measurement systems.
Solution Approach 2:
The system transitions from relying on instrument-mounted cameras (two-dimensional visual feedback) to using external sensor systems that provide three-dimensional pose data. This dimensional enhancement allows precise tracking of instruments even when they lack onboard vision capabilities.
3Reliability
If three-dimensional imaging systems are used to provide context, then navigation safety improves, but the system complexity and cost increase
Solution Approach 1:
Instead of using full three-dimensional imaging systems (CT, MRI, or 3D fluoroscopy), the patent adds dimensional information about instrument pose to two-dimensional images through overlay graphics. This provides necessary spatial context without the complexity and cost of three-dimensional imaging hardware.
Solution Approach 2:
The system creates a virtual copy or representation of the instrument's three-dimensional pose using sensor data and overlays it on the two-dimensional image. This virtual model provides the necessary spatial context without requiring physical three-dimensional imaging capabilities.
Data Source
AI summary
The present disclosure relates to systems, devices, and methods for augmenting a two-dimensional image with three-dimensional pose information of instruments shown in the two-dimensional image.