Gaze-Based Virtual Effect Indicators for Directional Sound Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interactive media titles, including virtual reality games, face accessibility issues for players with sensory disabilities such as blindness, low vision, hearing loss, and deafness, as they often miss directional sounds crucial to the gameplay experience.
Innovation Solution
The system detects a player's gaze to generate virtual effect indicators correlated with directional sounds, using gaze data to identify when a player is not focusing on the source location of directional sounds, and then presents virtual indicators on the client device to indicate the sound source.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If directional sound is used to convey incoming characters or action in interactive media titles, then the gameplay experience is enhanced, but players with hearing loss or deafness cannot access this information
Solution Approach 1:
The patent introduces gaze detection as an intermediary mechanism to bridge the gap between directional sound and player awareness. The system detects where the player is looking and uses this gaze information to determine when to provide visual effect indicators, thereby mediating the communication of directional sound information to players with hearing disabilities without requiring the player to actually hear the directional sound.
Solution Approach 2:
The patent substitutes the acoustic channel with a visual channel supported by gaze detection. Instead of relying on the player's auditory system to detect directional sound, the system uses eye tracking (gaze data) to understand player attention and provides corresponding visual indicators. This replacement of the acoustic mechanism with a vision-based mechanism enables accessibility for players with hearing loss.
2Adaptability or versatility
If visual effect indicators are provided for directional sounds, then accessibility is improved, but the system complexity increases due to gaze detection requirements
Solution Approach 1:
The patent makes the gaze detection system multi-functional by using the same gaze data for multiple purposes: determining when visual effect indicators should be provided, tracking player attention throughout the game, and potentially adjusting other accessibility features. This universal use of gaze data justifies the added system complexity by providing benefits beyond just directional sound indication.
Solution Approach 2:
The system continuously monitors gaze data and uses this feedback to dynamically control the provision of visual effect indicators. The gaze detection creates a closed-loop system where player attention is continuously measured and used to adjust the accessibility assistance, allowing the complex gaze detection system to be justified through its intelligent, adaptive control of visual cues.
Data Source
AI summary
A method and system for providing gaze-based generation of virtual effects indicators correlated with directional sounds is disclosed. Gaze data is tracked via a camera associated with a client device to identify a point of focus within a three-dimensional virtual environment towards which one or both eyes of the player are focused. When the point of focus indicated by the gaze data when the point of focus does not move towards the source location within the three-dimensional virtual environment when the directional sound is received indicates that a virtual effect indicator associated with the directional sound type of the indicated directional sound is should be generated.