Surgical Tool Fiducial Marker Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for surgical tool tracking in minimally-invasive robotic surgery face challenges such as inaccurate kinematics-based pose estimation, occlusion, adverse lighting conditions, and high costs associated with optical and electromagnetic trackers, which hinder the reliability and accuracy of image-derived tool pose estimation.
Innovation Solution
The use of image-derived data from reference features on surgical tools, including multiple markers with identification and positional information, processed by a processor to determine the tool's state, providing improved accuracy and reliability in tool tracking by reducing sensitivity to occlusions and adverse conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical trackers are used for surgical tool tracking, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses image-derived data from 2D images to create a virtual copy or representation of the surgical tool's 3D pose, eliminating the need for physical optical trackers. The system processes images from existing surgical cameras and generates tool pose estimates through image processing algorithms, thereby copying the tracking function without requiring dedicated tracking hardware.
Solution Approach 2:
The patent replaces mechanical/optical tracking systems with an image processing-based system. Instead of using optical trackers that require dedicated cameras and complex calibration, the system substitutes these with standard surgical imaging equipment and computational algorithms that derive tool pose from 2D images through feature detection and geometric transformation.
2Measurement precision
If electromagnetic trackers are used for surgical tool tracking, then measurement precision is improved, but cost increases
Solution Approach 1:
The system creates a computational representation of tool pose from 2D image data, copying the essential tracking information without requiring expensive electromagnetic tracking hardware. The image processing pipeline generates 3D pose estimates that replicate the functionality of electromagnetic trackers using only standard surgical imaging equipment.
Solution Approach 2:
The patent substitutes electromagnetic tracking systems with an image processing approach. Instead of relying on electromagnetic fields and specialized sensors, the system uses 2D image capture and computational geometry to derive tool pose, replacing the entire electromagnetic tracking mechanism with a software-based solution.
3Speed
If kinematics-based pose estimation is used, then update rate is improved, but measurement precision deteriorates
Solution Approach 1:
The patent merges kinematics-based pose estimation with image-derived pose estimation into a unified system. The kinematics component provides high-speed updates while the image processing component provides accurate reference measurements. By combining these two approaches, the system achieves both high update rates and high precision through data fusion and mutual correction.
Solution Approach 2:
The system uses image-derived pose estimates as feedback to correct and refine kinematics-based pose estimates. The accurate but slower image processing results are fed back into the kinematics model to correct accumulated errors, while the fast kinematics updates provide continuous motion tracking. This feedback loop ensures both speed and accuracy are maintained.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool-located reference features. A method includes: capturing a first image of a tool that includes multiple features that define a first marker, where at least one of the features of the first marker includes an identification feature; determining a position for the first marker by processing the first image; determining an identification for the first marker by using the at least one identification feature by processing the first image; and determining a tool state for the tool by using the position and the identification of the first marker.