Synthetic Surgical Robot View for Tool Tracking Beyond Endoscope FOV
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In minimally invasive robotic surgical systems, surgeons face challenges in visualizing the entire surgical site, including tool interference and orientation, due to limited field of view and lack of feedback on tool positions outside the endoscope's view, leading to potential collisions and lost tool issues.
Innovation Solution
A robotic surgical system that includes a robot with a linkage, a kinematic component for joint state information, and a display component that provides a synthetic graphical representation of the robot and tools, along with collision detection and tool tracking, enabling better visualization and control of tool movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If the endoscope is manipulated to various positions and orientations to view the surgical site, then the surgeon can observe different areas of the surgical field, but the surgeon loses awareness of tool positions and orientations outside the field of view, leading to potential tool interference
Solution Approach 1:
The system creates a virtual copy of the surgical environment using a 3D graphical model that mirrors the actual physical positions of tools and endoscope. This virtual model is continuously updated based on encoder feedback from the robotic system, allowing the surgeon to see tool positions that are not visible in the real endoscopic view.
Solution Approach 2:
The system transitions from a 2D endoscopic view to a 3D graphical representation that adds spatial dimensionality. This 3D model displays tool positions, orientations, and potential interference zones in three-dimensional space, providing comprehensive spatial awareness that cannot be achieved with traditional 2D endoscopic imaging alone.
2Adaptability or versatility
If multiple working tools are used in the surgical system, then the surgical capabilities are enhanced, but the complexity of tracking and monitoring tool positions increases, making it difficult to detect tool interference
Solution Approach 1:
The 3D graphical model serves multiple functions simultaneously: it displays tool positions, shows endoscope orientation, indicates potential tool interference zones, and provides spatial context for surgical planning. This single unified visualization system handles all monitoring tasks for multiple tools without requiring separate display systems.
Solution Approach 2:
The system continuously receives feedback from encoders on the robotic manipulators and updates the 3D graphical model in real-time. This closed-loop feedback mechanism automatically tracks tool positions and orientations, calculating and displaying potential interference zones without requiring manual intervention or complex manual tracking procedures.
3Measurement precision
If the field of view is limited to the distal ends of working tools within the endoscope's view, then the image quality is maintained, but the surgeon cannot see portions of tools or the entire surgical site, leading to confusion about tool locations
Solution Approach 1:
The system merges the high-quality 2D endoscopic view with the comprehensive 3D graphical model into a unified display. The 2D view provides detailed visual information of the surgical site with high image quality, while the 3D model provides contextual spatial information about tool positions and orientations, combining the advantages of both visualization modes.
Data Source
AI summary
A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera. The system also comprises an input device, a display, and a processor. The processor is configured to, in a first mode, command the first robotic arm to move the camera in response to a first input received from the input device to capture an image of the tool and present the image as a displayed image on the display. The processor is configured to, in a second mode, display a synthetic image of the first robotic arm in a boundary area around the captured image on the display, and in response to a second input, change a size of the boundary area relative a size of the displayed image.