Markerless Medical Device Pose Tracking With Single-Camera Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing medical device tracking methods, such as stereo-based optical tracking and electromagnetic tracking, are costly, prone to occlusion errors, and lack precision, especially when dealing with complex medical environments and substantial device rotations.
Innovation Solution
A markerless pose estimation system using a single camera, such as a color depth camera, identifies medical devices like ultrasound transducers by extracting key features from images and comparing them to ground truth templates, while accounting for occlusions by modeling the occluding objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereo-based optical tracking with markers is used, then tracking precision is improved, but hardware cost and device complexity increase
Solution Approach 1:
The patent removes the marker component from the tracking system, extracting only the essential optical detection function. By using a single camera to capture natural surface features and patterns of the medical device without requiring additional markers, the system achieves markerless tracking that reduces hardware complexity and cost while maintaining precision through advanced image processing algorithms
Solution Approach 2:
The patent creates a virtual copy of the medical device's visual appearance by capturing and storing reference images of the device from multiple angles and positions. This reference database allows the single camera to recognize and track the device by comparing real-time images against the stored references, eliminating the need for physical markers while maintaining accurate tracking identification
2Measurement precision
If markers are added to the device body, then tracking identification accuracy is improved, but device complexity and cost increase
Solution Approach 1:
The patent enables the medical device to be tracked using its own inherent visual characteristics and surface patterns without requiring external markers. The device's natural appearance, geometry, and surface features serve as the tracking identifiers, allowing the system to recognize and track the device autonomously based on its self-contained visual properties rather than requiring additional components
3Device complexity
If a single camera is used for tracking, then hardware cost is reduced, but tracking precision and reliability deteriorate
Solution Approach 1:
The patent transitions from two-dimensional image processing to three-dimensional spatial reasoning by capturing images from multiple angles and positions. The single camera system reconstructs three-dimensional information about the device's pose, orientation, and position by analyzing changes in the two-dimensional images across different time points and spatial configurations, thereby achieving precise tracking despite using only one camera
Solution Approach 2:
The patent performs preliminary capture and storage of reference images of the medical device from various angles and positions before actual tracking begins. This pre-acquired reference database enables the single camera to accurately identify and track the device by comparing real-time images against the comprehensive reference set, compensating for the limited view angle of a single camera through thorough pre-characterization of the device's visual appearance
4Reliability
If marker-based tracking is used, then tracking reliability is improved, but susceptibility to occlusion increases
Solution Approach 1:
The patent segments the tracking task into multiple independent components: capturing images from the single camera, detecting the device in the image, determining pose information, and handling occlusion cases. This segmentation allows the system to process visual information in manageable steps and apply specialized algorithms for each function, improving overall reliability while reducing susceptibility to occlusion through robust image processing and multiple verification approaches
Data Source
Figure 1~2
Figure 3~4
AI summary
The pose of medical device in use, where the device may be occluded, is estimated using a single camera. Multiple cameras and/or a marker are not needed. The medical device is detected in the camera image during use to determine the pose. Various further approaches may be used to deal with occlusion. Comparison with a template for that type of medical device may be used to deal with occlusion. Modeling the occlusion, such as the hand, may be used to deal with occlusion.