Eye-Tracking Instrument Assignment for Robotic Surgery Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current surgical robotic systems with eye tracking technology face limitations in efficiently assigning and controlling surgical instruments using gaze direction, particularly in differentiating objects or regions within a displayed surgical site for precise robotic manipulations and computer vision recognition.
Innovation Solution
The system employs eye tracking to allow surgeons to assign instruments to robotic manipulators and aid computer vision in recognizing instruments on an endoscopic display by using gaze input, with methods involving visual overlays, computer vision algorithms, and gaze-based input sequences to facilitate instrument pairing and image segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If eye tracking is used to control instrument assignment, then ease of operation is improved, but reliability of instrument identification deteriorates due to difficulty in differentiating objects within the displayed surgical site
Solution Approach 1:
The system performs preliminary actions by detecting the surgeon's gaze direction and using it to pre-select or pre-assign instruments before final confirmation is needed. The eye tracking system continuously monitors gaze position and prepares instrument assignment options based on where the surgeon is looking, reducing the operational steps required while maintaining accurate identification through confirmation mechanisms.
2Ease of operation
If eye tracking gaze direction is used for command selection, then ease of operation is improved, but measurement precision deteriorates due to difficulty in accurately identifying specific instruments or regions
Solution Approach 1:
The displayed surgical site is segmented into multiple distinct regions or zones, each corresponding to specific instruments or anatomical structures. The eye tracking system divides the complex visual field into manageable segments, allowing the surgeon's gaze to be mapped to specific segmented regions rather than requiring precise identification of individual instruments within a cluttered display. This segmentation approach maintains measurement precision while improving ease of operation.
3Productivity
If eye tracking is integrated with computer vision for instrument recognition, then productivity is improved, but device complexity increases due to integration of multiple systems
Solution Approach 1:
The eye tracking system and computer vision system are merged into a unified instrument recognition and assignment platform. The gaze detection data from the eye tracker is combined with the visual data from the endoscopic camera and computer vision algorithms, creating an integrated system that processes both ocular movement information and visual scene understanding simultaneously. This merging reduces overall system complexity compared to having separate independent systems while maintaining enhanced productivity.
Data Source
AI summary
A robotic surgical system includes an eye gaze sensing system in conjunction with a visual display of a camera image from a surgical work site. Detected gaze of a surgeon towards the display is used as input to the system. This input may be used by the system to assign an instrument to a control input device (when the user is prompted to look at the instrument), or it may be used as input to a computer vision algorithm to aid in object differentiation and seeding information, facilitating identification/differentiation of instruments, anatomical features or regions.