3D Tool Tracking Fusion for Minimally Invasive Robotic Surgery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing minimally invasive surgical techniques face challenges in accurately tracking surgical tools, especially when they are outside the endoscope's field of view or occluded, leading to difficulties in tool identification and orientation determination, which can result in inaccurate positioning and potential tissue damage.
Innovation Solution
A hybrid approach combining non-endoscopically derived tool state information, such as system kinematics and sensor data, with endoscopically derived information from camera images to predict and correct the tool's position and orientation, using Bayesian filters and Kalman filters for real-time tracking and calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If only endoscopic camera images are used for tool tracking, then the tool can be visualized within the field of view, but the tool cannot be tracked when it is outside the field of view or occluded
Solution Approach 1:
The patent combines multiple independent tracking systems: endoscopic vision-based tracking and non-endoscopic sensor-based tracking (electromagnetic, acoustic, or mechanical sensors). By merging these complementary systems, the patent ensures continuous tool state information is available whether the tool is visible to the endoscope or not, resolving the contradiction between field-of-view dependency and tracking reliability.
Solution Approach 2:
The patent introduces sensor-based tracking systems as intermediary mechanisms that provide tool position and orientation data independently of endoscopic visualization. These sensors act as mediators that continue to track the tool even when occluded or outside the camera field of view, bridging the information gap when visual tracking fails.
2Measurement precision
If visual tracking methods are used to identify the tool, then the tool can be located within the camera view, but the tool orientation cannot be determined when occluded or outside view
Solution Approach 1:
The patent merges visual tracking data with sensor-based tracking data to simultaneously determine both tool position and orientation. The sensor systems provide complementary information about tool state that is independent of visual occlusion, ensuring both position and orientation can be reliably determined even when the tool is not visible to the endoscope.
3Ease of operation
If the tool is moved outside the endoscope's field of view to access difficult surgical sites, then surgical dexterity is improved, but tool identification and positioning accuracy deteriorate
Solution Approach 1:
The patent introduces non-endoscopic sensor systems as intermediary tracking mechanisms that continue to provide accurate tool position and orientation data even when the tool is moved outside the endoscope's field of view. This allows surgeons to access difficult sites with improved dexterity while maintaining positioning accuracy through the sensor-based tracking intermediary.
4Adaptability or versatility
If multiple tools are used simultaneously to perform complex surgical procedures, then surgical capability is enhanced, but tool differentiation and identification become difficult
Solution Approach 1:
The patent applies local quality by assigning unique identification characteristics to different tools. Each tool has distinct sensor signatures, markers, or identification codes that allow the system to differentiate between multiple tools even when they are in close proximity or performing similar functions, resolving the identification difficulty while maintaining enhanced surgical capability.
Data Source
AI summary
Methods and system perform tool tracking during minimally invasive robotic surgery. Tool states are determined using triangulation techniques or a Bayesian filter from either or both non-endoscopically derived and endoscopically derived tool state information, or from either or both non-visually derived and visually derived tool state information. The non-endoscopically derived tool state information is derived from sensor data provided either by sensors associated with a mechanism for manipulating the tool, or sensors capable of detecting identifiable signals emanating or reflecting from the tool and indicative of its position, or external cameras viewing an end of the tool extending out of the body. The endoscopically derived tool state information is derived from image data provided by an endoscope inserted in the body so as to view the tool.