3D Surgical Robot Representation for Tool Position and Collision Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In minimally invasive robotic surgical systems, surgeons face challenges in visualizing and managing the spatial arrangement of multiple surgical tools, particularly when they are outside the endoscope's field of view, leading to potential tool interference and difficulty in reorienting tools relative to the surgical site.
Innovation Solution
A method that generates a synthetic image of the surgical site, including the patient side cart and tools, which provides a broader view beyond the endoscope's field of view, using tool tracking and kinematic data to update the image in real-time, allowing for collision detection and lost tool recovery, and is displayed alongside the actual endoscopic view for enhanced situational awareness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If the surgeon relies on the endoscope's field of view to monitor tool positions, then the visual detail of the surgical site is improved, but the surgeon cannot see portions of tools or entire tools outside the field of view, leading to potential tool interference
Solution Approach 1:
The system creates a virtual copy of the surgical environment using a 3D graphical user interface that replicates the spatial relationships of all tools and the surgical site. This virtual model allows the surgeon to view tool positions from multiple perspectives simultaneously, including portions outside the physical endoscope's field of view, while maintaining accurate spatial correspondence with the real surgical environment.
Solution Approach 2:
The system transitions from a two-dimensional endoscope view to a three-dimensional virtual representation of the surgical field. This dimensional enhancement allows the surgeon to perceive tool positions, orientations, and potential interferences that cannot be observed in the limited 2D endoscopic view, providing comprehensive spatial awareness.
2Adaptability or versatility
If the endoscope is manipulated to various positions and orientations to view the surgical site, then the visibility of different surgical areas is improved, but the surgeon becomes confused about the general location of tools and may not understand how to avoid inter-tool interference
Solution Approach 1:
The virtual 3D graphical interface provides continuous feedback about tool positions, orientations, and spatial relationships relative to the surgical site. As the endoscope moves or tools are repositioned, the virtual model updates in real-time to reflect current configurations, helping the surgeon maintain spatial awareness and understand tool locations regardless of endoscope orientation.
Solution Approach 2:
The virtual graphical interface serves multiple functions simultaneously: it displays tool positions, shows spatial relationships, predicts potential interferences, and provides orientation context. This multi-functional display consolidates various types of spatial information in a single unified view, eliminating the need to mentally integrate information from multiple sources.
3Adaptability or versatility
If multiple working tools are used in the surgical system, then the versatility of surgical procedures is improved, but the risk of tool interference and complexity of managing tool positions increases
Solution Approach 1:
The virtual 3D graphical interface acts as an intermediary between the physical tools and the surgeon's perception. It translates complex spatial relationships and tool configurations into an intuitive visual representation, allowing the surgeon to manage multiple tools more effectively by providing a comprehensive overview of their positions and potential interactions.
Solution Approach 2:
The system performs preliminary analysis of potential tool interferences by continuously monitoring virtual tool positions and orientations. Before actual interference occurs, the system can alert the surgeon to potential conflicts, allowing preventive action to be taken. This predictive capability reduces the complexity of managing multiple tools by providing advance warning of spatial conflicts.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A synthetic representation of a robot tool for display on a user interface of a robotic system. The synthetic representation may be used to show the position of a view volume of an image capture device with respect to the robot. The synthetic representation may also be used to find a tool that is outside of the field of view, to display range of motion limits for a tool, to remotely communicate information about the robot, and to detect collisions.