Spatial Audio Rendering with Augmented Reality Cues for Illusion Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio systems struggle to accurately render spatial audio due to inaccuracies in head-related transfer functions (HRTFs), environment-related transfer functions (ERTFs), and head tracking, leading to a collapse of the spatial audio illusion when visual cues are removed.
Innovation Solution
An audio system that integrates HRTFs, ERTFs, and head tracking with visual references in the form of sound source icons displayed in augmented reality, allowing listeners to perceive spatial audio as originating from specific locations even when visual cues are no longer visible.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If visual references (sound source icons) are displayed in augmented reality to enhance spatial audio perception, then the durability of spatial audio illusion is improved, but the device complexity increases
Solution Approach 1:
The patent combines audio rendering components (HRTF, ERTF, head tracking) with visual reference display (augmented reality sound source icons) into an integrated system. This merging allows the visual and audio components to work together synergistically, enhancing the durability of spatial audio illusion while managing system complexity through unified architecture.
Solution Approach 2:
The visual references (sound source icons) act as intermediaries between the audio system and the listener's perception. These visual cues mediate the spatial audio experience by providing reference points that help the brain maintain the illusion of externalized sound sources, thereby extending the durability of the spatial audio effect.
2Reliability
If HRTF, ERTF, and head tracking algorithms are used to render spatial audio, then spatial audio perception is achieved, but inaccuracies cause the illusion to collapse
Solution Approach 1:
The patent applies visual references (sound source icons) as a compensatory measure before the spatial audio illusion can collapse due to inaccuracies in HRTF, ERTF, or head tracking. These visual cues provide a reference framework that cushions against the degrading effects of imperfect audio rendering algorithms, maintaining the illusion for longer periods.
Solution Approach 2:
The system uses visual references as a feedback mechanism to reinforce spatial audio perception. By displaying sound source icons at designated positions in the augmented reality view, the system provides continuous visual feedback that helps the listener's brain maintain the correct spatial mapping, compensating for inaccuracies in the audio rendering algorithms.
Data Source
AI summary
An audio system and method of using the audio system to augment spatial audio rendition is described. The audio system can include a device to receive user inputs designating positions on an augmented reality view of a listening environment. Sound source icons can be presented in the augmented reality view at the designated positions. The sound source icons can visually represent sound sources at locations in the listening environment that correspond to, but are different than, the positions. One or more processors of the audio system can apply head-related transfer functions, which correspond to the locations in the listening environment, to audio input signals to generate binaural audio signals. The audio system can include a headset that uses the binaural audio signals to render spatialized audio localizing sounds to the locations in the listening environment. Other aspects are also described and claimed.