Synthetic Surgical Robot View for Lost Tool Recovery and Collision Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In minimally invasive robotic surgical systems, surgeons face challenges in visualizing and managing the spatial relationships between multiple surgical tools, particularly when they are outside the endoscope's field of view, leading to potential tool interference and confusion regarding tool reorientation and movement.
Innovation Solution
A robotic surgical system that includes a patient-side cart with multiple robotic arm assemblies, an image capture device, and a display that generates a synthetic representation of the robotic arms, tools, and image capture device, allowing for a three-dimensional projection of the endoscope's field of view and providing collision detection and lost tool recovery features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the surgeon relies on the endoscope's field of view to monitor tool positions, then the surgeon can see the surgical site in detail, but the surgeon cannot see portions of tools or entire tools outside the field of view, leading to potential tool interference and confusion
Solution Approach 1:
The system creates a synthetic image that is a virtual copy of the physical robotic surgical system, displaying tool positions, robotic arm positions, and image capture device positions. This synthetic representation provides complete visibility of all tool locations outside the endoscope's field of view, allowing the surgeon to monitor the entire surgical environment without losing spatial information.
Solution Approach 2:
The system adds a virtual dimension by overlaying the synthetic image (showing complete spatial relationships) with the real endoscopic view. This multi-dimensional display approach allows the surgeon to simultaneously access both the detailed surgical field view and the comprehensive tool position information that would otherwise be invisible.
2Adaptability or versatility
If the surgeon uses multiple working tools during surgery, then the surgical capabilities are enhanced, but the complexity of managing and tracking multiple tool positions and orientations increases
Solution Approach 1:
The system continuously provides visual feedback by updating the synthetic image to reflect current positions of multiple tools, robotic arms, and the image capture device. This real-time feedback mechanism helps the surgeon understand the spatial relationships between multiple tools and make informed decisions about tool manipulation and reorientation.
Solution Approach 2:
The synthetic image serves multiple functions simultaneously: it displays tool positions, shows robotic arm configurations, indicates image capture device orientation, and provides collision detection warnings. This multi-functional display reduces the need for separate monitoring systems for each tool.
3Adaptability or versatility
If the endoscope is manipulated to various positions and orientations to access different surgical sites, then the surgical access is improved, but the surgeon becomes confused about the general location of tools relative to the surgical site
Solution Approach 1:
The system creates a virtual representation that maintains consistent spatial relationships between all components regardless of endoscope manipulation. The synthetic image displays tool positions and orientations in a coordinated reference frame that preserves spatial context, allowing the surgeon to understand tool locations relative to the surgical site even when the endoscope is repositioned.
Solution Approach 2:
The synthetic image acts as a virtual copy that mirrors the physical system's spatial relationships. When the endoscope moves to new positions and orientations, the synthetic image updates to reflect these changes while maintaining accurate relative positioning information, preventing surgeon confusion about tool locations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.