Eye Tracking Augmented Reality Surgical Robot
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current surgical robotic systems lack effective integration of real-time image analysis and augmented reality to enhance the surgeon's visual feedback and decision-making during procedures, relying primarily on eye-tracking for command inputs and limited tactile feedback.
Innovation Solution
Incorporating an eye-tracker and computer vision/image processing algorithms to analyze real-time surgical images, providing augmented reality overlays and active assistance, such as alerts, information, and haptic feedback, to help the surgeon navigate and perform tasks more accurately and safely.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If eye tracking is used for command input control, then ease of operation is improved, but loss of information occurs as the surgeon cannot simultaneously monitor surgical images and control commands
Solution Approach 1:
The patent introduces an intermediary system that captures the surgeon's gaze direction and translates it into command inputs without requiring the surgeon to move their eyes from the surgical field. The eye tracking system acts as a mediator between the surgeon's visual attention and the control system, allowing command input while maintaining continuous visual monitoring of the surgical site.
Solution Approach 2:
The patent replaces manual mechanical control input (hand movements to control devices) with an optical-based eye tracking system. This substitution allows the surgeon to issue commands through natural eye movements rather than physical manipulation of control devices, eliminating the need to shift attention from the surgical images to control interfaces.
2Productivity
If traditional control interfaces are used, then device complexity is reduced, but productivity decreases due to time spent switching between visual monitoring and command input
Solution Approach 1:
The patent implements a multi-functional control system that can operate through multiple input modalities: traditional manual control devices for complex or deliberate commands, and eye tracking for rapid, intuitive control. This universal control architecture allows the system to adapt to different surgical scenarios and surgeon preferences, improving productivity without oversimplifying the control capabilities.
Solution Approach 2:
The system performs preliminary analysis of the surgeon's gaze patterns and anticipates intended commands before explicit input is given. By pre-processing visual attention data and predicting upcoming control actions, the system reduces latency and prepares command execution in advance, thereby improving surgical workflow efficiency without adding significant complexity.
Data Source
AI summary
A system augments a surgical procedure by providing information to the user about regions of a surgical site the user is viewing on a displayed endoscopic image. The system uses an eye tracker to determine the region of the displayed image being viewed by the user, and applies computer vision to the image data to identify structures, conditions or disease states within the region. The system provides information to the user, generates alerts, or modifies operational settings for surgical devices based on the identified structures, conditions or disease states.