Gaze-Based Tracking for Pixel-Mapped Annotation Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current slide scoring protocols in clinical and research labs lack explicit annotation requirements and non-obtrusive methods for collecting detailed spatial information about a pathologist's focus during slide examination, which is crucial for training AI systems.
Innovation Solution
Implement gaze-based tracking to automatically create a training dataset by mapping a user's monitored gaze and manipulations to pixels of an image, using a system that integrates with microscopes and displays to capture and register gaze data without interrupting the user's workflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation methods are used to collect gaze data, then detailed spatial information can be obtained, but the process requires significant manual labor and interrupts user workflow
Solution Approach 1:
The system automatically collects gaze data through integrated eye-tracking technology without requiring user intervention. The pathologist's natural viewing behavior is captured and mapped to digital slide coordinates autonomously, eliminating the need for manual annotation while preserving detailed spatial information about regions of interest
Solution Approach 2:
Manual annotation processes are replaced with automated optical tracking systems. Eye-tracking hardware and software automatically capture gaze coordinates and map them to image pixels, substituting the mechanical manual pointing and clicking process with an automated optical-mechanical system that operates transparently in the background
2Productivity
If gaze tracking systems are integrated with microscopes and displays, then automatic annotation data collection is enabled, but device complexity increases
Solution Approach 1:
The system integrates multiple functions into a unified platform: eye-tracking data acquisition, gaze coordinate mapping to image space, automatic annotation generation, and training dataset creation all occur within a single integrated system. This multi-functionality reduces the need for separate devices and processes, managing complexity through consolidation rather than proliferation of components
Solution Approach 2:
A coordinate mapping system serves as an intermediary layer between the physical gaze tracking hardware and the digital image annotation output. This intermediary component translates eye-tracking coordinates into image pixel coordinates, handling the complexity of coordinate system transformations and device calibration internally while presenting a simple annotation generation interface to users
3Reliability
If detailed gaze information is collected and mapped to image pixels, then training dataset quality improves, but data processing and storage requirements increase
Solution Approach 1:
The system extracts only the essential annotation elements from raw gaze data: unique region coordinates, dwell time thresholds, and confidence metrics. Rather than storing every raw gaze point, the system processes the continuous gaze stream and extracts discrete annotation events that capture the essential spatial and temporal patterns of pathologist attention, reducing data volume while preserving training quality
Data Source
AI summary
There is provided a computer implemented method of automatically creating a training dataset comprising a plurality of records, wherein a record includes: an image of a sample of an object, an indication of monitored manipulations by a user of a presentation of the sample, and a ground truth indication of a monitored gaze of the user viewing the sample on a display or via an optical device mapped to pixels of the image of the sample, wherein the monitored gaze comprises at least one location of the sample the user is viewing and an amount of time spent viewing the at least one location.