Video-Based Robotic Surgical Tool Tracking for Out-of-View Safety
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In robotic surgeries, accurately determining the position and force of surgical tools relative to the camera is challenging due to errors in kinematic predictions, leading to potential patient injury from out-of-view tools or excessive force application.
Innovation Solution
Implementing video processing to identify surgical tools in images, compare their positions and types with predicted data, and adjust tool operation based on discrepancies to ensure safety, including disabling out-of-view tools and reducing excessive force.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of moving object
If kinematic chain models are used to predict surgical tool positions, then the surgeon can control tools with extended range of motion, but small deflections or errors in joint measurements become relatively large errors in predicted tool positions
Solution Approach 1:
The system captures images from a camera, identifies the actual positions of surgical tools in the images, compares these identified positions with the predicted positions from the kinematic chain model, and uses this feedback to detect and correct errors in tool position prediction
Solution Approach 2:
The patent replaces reliance on purely mechanical kinematic chain calculations with vision-based optical measurement. By using image processing to identify tool positions directly from camera images, the system substitutes mechanical prediction with optical observation, eliminating accumulation of mechanical measurement errors
2Loss of information
If the surgeon visually monitors surgical tools through the camera, then the surgeon can see tool positions, but there is no haptic feedback so the surgeon cannot sense the force applied to patient anatomy
Solution Approach 1:
The system calculates the difference between identified tool positions from images and predicted positions from the kinematic model, then uses this position discrepancy to estimate the force being applied by the surgical tool, providing the surgeon with force feedback information
Solution Approach 2:
The patent introduces an intermediary force estimation mechanism that translates visual position information into force information. The system acts as a mediator between the visual camera system and the surgeon, converting image-based position data into estimated force values that the surgeon can use to control tool force application
3Manufacturing precision
If the robotic surgical device operates with high precision tools, then the surgery can be performed minimally invasively, but it is difficult to detect when tools are outside the camera view or applying excessive force
Solution Approach 1:
The system continuously captures images, identifies tool positions, compares them with predicted positions, and uses this feedback loop to detect when tools are outside the camera view or applying excessive force, providing real-time monitoring capability
Solution Approach 2:
The patent replaces difficult mechanical sensing of tool positions and forces with vision-based identification. By using image processing to detect tool positions and comparing with kinematic predictions, the system substitutes complex mechanical sensing with optical detection, making it easier to monitor tool status
Data Source
AI summary
One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.