Gaze-Directed Optical Imaging for Noncontact Surgical Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Effective communication between users of an optical imaging system, such as surgeons during a surgical operation, is hindered by the need to point at specific points within the surgical cavity, which is time-consuming and risky, disrupting the workflow and often avoided.
Innovation Solution
An apparatus that generates an adjusted image of a sample based on gaze direction data, emphasizing a region of interest to facilitate communication by highlighting the user's focus area, using processors and storage devices to process sensor and gaze direction data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the surgeon uses the tip of the tool to point at a specific point within the surgical cavity, then communication between users is enabled, but the workflow is interrupted and time is lost
Solution Approach 1:
The system introduces an intermediary mechanism (eye tracking technology and image processing system) that translates the surgeon's gaze into visual indicators on the display. This mediator enables communication without requiring the surgeon to physically point with the tool, thus maintaining workflow continuity while effectively conveying spatial information to the team
Solution Approach 2:
The system creates a visual copy or representation of the surgeon's line of sight by overlaying indicators on the displayed surgical field. This copy allows the team to understand where the surgeon is looking without interrupting the surgical procedure, effectively communicating spatial information through a digital representation rather than physical gesturing
2Loss of information
If the surgeon uses the tip of the tool to point at a specific point within the surgical cavity, then communication between users is enabled, but the surgical procedure becomes risky
Solution Approach 1:
The eye tracking system serves as a safe intermediary that conveys the surgeon's attention and spatial reference without requiring physical contact with tissues. This eliminates the risk associated with using the tool tip for pointing while maintaining effective communication of spatial information to the surgical team
Solution Approach 2:
The system replaces the mechanical action of physically pointing with the tool tip with an optical/electronic mechanism (eye tracking and visual overlay). This substitution eliminates the physical risks of touching tissues with instruments while achieving the same communication goal through non-contact visual indicators
3Loss of time
If the surgeon avoids communicating to maintain workflow continuity, then time is saved, but communication between users deteriorates
Solution Approach 1:
The system provides automatic, continuous communication by tracking the surgeon's eye movements and automatically generating visual indicators of the surgeon's focus area. This self-service mechanism eliminates the need for the surgeon to consciously initiate communication actions, maintaining workflow continuity while continuously providing spatial information to the team
Data Source
Figure 1a~1b
Figure 2~4
Figure 3
AI summary
Examples relate to an apparatus for an optical imaging system comprising one or more processors and one or more storage devices. The apparatus is configured to obtain sensor data of an optical imaging sensor of the optical imaging system. The sensor data is indicative of an image of a sample. Further, the apparatus is configured to obtain gaze direction data of a sensor. The gaze direction data is indicative of a gaze direction of a user of the optical imaging system. The apparatus is further configured to generate region data indicative of a region of interest that the user gazes at on the image of the sample. The region data is generated based on the gaze direction data. Further, the apparatus is configured to generate output data indicative of an adjusted image of the sample to be displayed on a display device. The output data is generated based on the sensor data and the region data. The adjusted image of the sample comprises an emphasized area indicative of the region of interest, such that the emphasized area is emphasized relative to the region of interest in the image of the sample. The apparatus is further configured to transmit the output data for displaying on the display device.