AR Glasses Audio Enhancement Using Eye-Gaze Sound Locking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In noisy environments, AR glasses struggle to effectively capture sound from specific sources of interest due to directional microphones not aligning with the user's focus, leading to interference from other sound sources.
Innovation Solution
An AR glasses system using a microphone array to detect sound source distribution, mark positions on lenses, lock onto a target sound source based on eye gaze direction, enhance the audio component associated with the target sound source, and output through an in-ear headphone.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If directional microphones are used to capture sound from specific directions, then noise from other directions is suppressed, but the sound source capture does not align with user's focus direction
Solution Approach 1:
The system dynamically adjusts the sound capture direction by tracking user eye gaze movements in real-time. The microphone array's effective capture direction is made dynamic rather than fixed, allowing it to adapt to where the user is looking. This resolves the contradiction by making the directional capture aligned with user focus through continuous adjustment based on eye tracker data.
Solution Approach 2:
The system uses eye tracking feedback to determine user focus direction and adjusts the sound capture accordingly. The eye tracker provides continuous feedback about where the user is looking, and this feedback loop enables the audio system to automatically align its capture direction with user attention, solving the misalignment problem between fixed directional microphones and dynamic user focus.
2Quantity of substance
If multiple sound sources are present in noisy environment, then ambient sounds are captured, but interference from unwanted sound sources increases
Solution Approach 1:
The system applies different processing qualities to different sound sources based on their spatial location and significance. Sound sources in the user's focus direction receive enhanced processing and amplification, while sound sources from other directions are suppressed or filtered. This local differentiation allows ambient sound capture while minimizing noise interference from unwanted directions.
Solution Approach 2:
The audio system segments the soundscape into multiple independent sound source channels, each processed differently. By separating and independently processing each detected sound source, the system can selectively amplify the target sound while suppressing others, achieving both ambient awareness and noise reduction simultaneously.
3Quantity of substance
If conventional audio processing is used, then all ambient sounds are captured, but the target sound source cannot be effectively isolated
Solution Approach 1:
The system performs preliminary sound source detection and classification before final audio processing. By identifying and categorizing different sound sources in advance using the microphone array and eye gaze data, the system can then apply targeted processing to isolate the desired sound while maintaining awareness of the ambient soundscape. This preliminary segmentation enables effective target isolation without losing ambient context.
Data Source
AI summary
The present disclosure provides AR glasses, an audio enhancing method and device therefor, as well as a readable storage medium. The audio enhancing method for AR glasses worn by a user in a surrounding environment includes: detecting a distribution of sound sources in the surrounding environment using a microphone array; marking a position of each sound source in the distribution of sound sources on lenses of the AR glasses; locking onto one of the distribution of sound sources as target sound source based on an eye gazing direction of the user; extracting and enhancing an audio component associated with a voiceprint characteristic of the target sound source from an audio signal received by the microphone array, configures to obtain an enhanced audio signal; and outputting the enhanced audio signal to the user through an in-ear headphone.