Human-in-the-loop eye tracking with dynamic library updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile eye-tracking technology faces challenges in analyzing eye movement data in dynamic environments due to the identification of three-dimensional objects of interest, which can be viewed from multiple perspectives and are often occluded, making it difficult to automate the analysis of real-world data.
Innovation Solution
An eye tracking system that includes a video capture component, a library of feature sets for areas of interest, an image annotation component to extract and match features, and a verification component that sends uncertain data to human experts for labeling, with the library dynamically updated based on expert input, allowing for robust analysis of mobile eye-tracking data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated feature matching is used to identify objects in dynamic environments, then analysis speed is improved, but identification accuracy deteriorates due to occlusion and multiple perspectives
Solution Approach 1:
The system implements a feedback mechanism where human experts review and correct automated object identification results. The verified labels are fed back into the library to improve future automated matching, creating a continuous improvement loop that addresses accuracy deterioration while maintaining high analysis speed through automated processing.
Solution Approach 2:
A human expert verification component acts as an intermediary between automated feature matching and final object identification. This intermediary reviews uncertain matches and provides corrective feedback, resolving the contradiction by allowing automated high-speed processing while human expertise ensures accuracy for ambiguous cases.
2Measurement precision
If human experts manually annotate all video data, then identification accuracy is improved, but analysis time and cost increase significantly
Solution Approach 1:
The system enables self-service through automated feature extraction and matching that performs the majority of object identification work independently. Human experts only intervene for uncertain cases, allowing the system to serve itself for routine tasks while human expertise provides accuracy improvements only where needed, significantly reducing total time and cost.
Solution Approach 2:
Instead of requiring complete manual annotation of all data, the system applies partial human action only to uncertain or ambiguous object identifications. This partial verification approach maintains high accuracy for critical cases while avoiding the time cost of manual annotation for all data points.
3Adaptability or versatility
If the library is updated with new object labels from human experts, then adaptability to new objects is improved, but system complexity increases
Solution Approach 1:
The system uses feedback from human expert annotations to dynamically update the object library. New object labels and features provided by experts feed back into the system, enabling automatic adaptation to new objects and environments without manual reconfiguration, thus improving versatility while managing complexity through automated library maintenance.
Solution Approach 2:
The object library is designed as a dynamic structure that automatically updates and evolves based on new data from human experts. Rather than a static comprehensive database, the library dynamically adapts to new objects and contexts, improving versatility while keeping complexity manageable through automated updates and incremental growth.
Data Source
AI summary
Systems and methods for eye tracking include a video capture component that provides a video of an environment and a set of gaze tracking coordinates for the video. A library stores a plurality of feature sets representing respective areas of interest with respective object labels. An image annotation component extracts a plurality of features from a region of interest and matches the extracted features to an area of interest in the library to produce a selected area of interest and a confidence value. A verification component accepts the selected area of interest if the confidence value meets a threshold value and sends the region of interest to a human expert to assign an object label if the confidence value does not. The library is dynamically updated in response to input from the human expert to add new areas of interest and new object labels to the library.