Surgical Robot 3D Tool Visualization Beyond Endoscope View
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In minimally invasive robotic surgery, surgeons face challenges in visualizing the entire surgical tool configuration, including potential inter-tool collisions and lost tools outside the endoscope's field of view, leading to confusion and safety risks.
Innovation Solution
A robotic surgical system that includes a robot with a linkage, a kinematic component, and a display that provides a synthetic representation of the tool configuration based on joint state information, enabling collision detection and lost tool recovery through a three-dimensional model superimposed on the endoscope's view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the surgeon relies solely on the endoscope's field of view to monitor tool positions, then the surgical visualization is simple and direct, but the surgeon cannot see portions of tools or entire tools outside the field of view, leading to inability to detect inter-tool interference
Solution Approach 1:
The patent creates a synthetic three-dimensional representation (virtual model) of the robot and tools that replicates the physical tool configuration. This virtual copy is generated from joint state information and linkage structure data, allowing the surgeon to view the complete tool arrangement including portions outside the endoscope's field of view without adding physical complexity to the surgical environment.
Solution Approach 2:
The patent transitions from two-dimensional endoscopic views to a three-dimensional synthetic representation of the robot and tools. This dimensional enhancement provides comprehensive spatial information about tool positions and configurations, enabling the surgeon to perceive inter-tool interference and lost tools that cannot be seen in traditional 2D endoscopic images.
2Adaptability or versatility
If the endoscope is manipulated to various positions and orientations to view the surgical site, then the surgical flexibility is improved, but the surgeon becomes confused about the general location of tools and cannot understand how to avoid inter-tool interference
Solution Approach 1:
The system continuously provides visual feedback by updating the synthetic three-dimensional representation based on real-time joint state information from the robot. This feedback loop maintains an accurate virtual model that reflects the actual physical configuration, helping the surgeon understand tool locations and adjust master manipulators to avoid interference regardless of endoscope position or orientation.
Solution Approach 2:
The synthetic three-dimensional representation acts as an intermediary between the physical robot/tools and the surgeon's perception. This virtual model mediates the surgeon's understanding of tool configurations, providing intuitive spatial information that bridges the gap between complex robot mechanics and surgeon control, especially when the endoscope is positioned at various orientations.
3Adaptability or versatility
If multiple working tools are used in the surgical system, then the surgical capability is enhanced, but the risk of inter-tool interference outside the field of view increases
Solution Approach 1:
The system performs preliminary collision detection by continuously monitoring the synthetic three-dimensional representation of tool positions and configurations. Before actual inter-tool interference occurs, the virtual model allows the surgeon to anticipate potential collisions and adjust tool positions proactively, preventing harmful interactions between multiple working tools.
Data Source
AI summary
A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera configured to capture an image of a camera field of view. The system further comprises an input device, a display, and a processor. The processor is configured to display a first synthetic image including a first synthetic image of the tool. The first synthetic image of the tool includes a portion of the tool outside of the camera field of view. The processor is also configured to receive a user input at the input device and responsive to the user input, change the display of the first synthetic image to a display of a second synthetic image including a second synthetic image of the tool that is different from the first synthetic image of the tool.