Gaze-Tracked Emotion Prediction for Seamless Sensory Reading Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to automatically generate sensory augmentation routines that match a reader's emotional response or storyline context, disrupting reading flow and comprehension when music or ambient settings are manually changed.
Innovation Solution
A system using gaze tracking and emotional prediction techniques to generate sensory augmentation routines, including music, lighting, or sounds, that adapt to the reader's emotional response and text context, seamlessly adjusting to enhance reading experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual music selection is used to match emotional response, then emotional alignment improves, but reading flow is disrupted and comprehension decreases
Solution Approach 1:
The system performs preliminary action by predicting the reader's future emotional response to upcoming text passages before they are read. The gaze tracking system identifies where the reader is looking, and the emotion processing unit predicts the emotional state associated with that text segment, allowing the music to be pre-selected and seamlessly transitioned without interrupting the reading flow.
Solution Approach 2:
The system uses feedback from gaze tracking to continuously monitor the reader's attention and predict their emotional response in real-time. This feedback loop allows the music selection to be dynamically adjusted based on the reader's actual engagement with the text, maintaining emotional alignment while preserving reading flow.
2Adaptability or versatility
If music is changed to match storyline context, then emotional response improves, but reading speed decreases due to interruptions
Solution Approach 1:
The system predicts the reader's emotional response to upcoming text passages before they are read. The gaze tracking system identifies where the reader is looking, and the emotion processing unit predicts the emotional state associated with that text segment, allowing the music to be pre-selected and seamlessly transitioned without interrupting the reading flow.
Solution Approach 2:
The system maintains continuity of useful action by ensuring that music transitions occur seamlessly without interrupting the reading process. The music is updated in advance based on predicted emotional responses, and transitions are timed to occur during natural pauses in reading, maintaining both context matching and reading speed.
3Adaptability or versatility
If automated sensory augmentation is implemented, then reading immersion improves, but system complexity increases
Solution Approach 1:
The system achieves universality by integrating multiple functions into a unified platform: gaze tracking for attention monitoring, emotion processing for emotional state prediction, and music generation for sensory augmentation. These components work together as a cohesive system that enhances reading immersion through automated, context-aware sensory augmentation.
Data Source
AI summary
A method for reading sensory augmentation comprising displaying a text on a reading device, determining a reader's current location in the text with a gaze tracking system, identifying a subsequent location in the text, processing the text at the subsequent location with an emotion processing unit to predict an emotional response to the text at the subsequent location, generating a sensory augmentation routine based on the predicted emotional response, and initiating playback of the sensory augmentation routine at the subsequent location in the text. The sensory augmentation routine can include a musical composition, a sound, and/or an ambient lighting effect. The gaze tracking system can be configured to receive data from the reading device related to the text currently displayed thereon and to receive data from an image capture device related to eye movement tracking of the reader.