Gaze-Directed Display Eyewear for Speech Isolation in Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hearing aids collect sounds from multiple directions, making it difficult for users to isolate and perceive sounds of interest, especially in noisy environments.
Innovation Solution
Display eyewear equipped with gaze-tracking technology and directional microphones to focus on the user's gaze direction, isolating and presenting speech or sound from that direction on a display panel or through auditory transducers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional hearing aids collect sounds from multiple directions, then the device can capture a broader range of audio input, but the user cannot isolate and perceive specific sounds of interest in noisy environments
Solution Approach 1:
The audio field is segmented by direction using multiple microphone elements arranged in an array. Each microphone captures sound from its specific orientation, allowing the system to separate and process sounds from different spatial locations independently. This segmentation enables selective enhancement of speech from the gazed-at direction while filtering out sounds from other directions.
Solution Approach 2:
The system applies different processing qualities to sounds from different directions. Sounds originating from the direction the user is gazing at receive enhanced processing and amplification, while sounds from other directions are attenuated or filtered. This local quality differentiation allows the hearing aid to optimize speech intelligibility for the relevant direction without uniformly amplifying all ambient noise.
2Stress or pressure
If hearing aids amplify all collected sounds uniformly, then overall sound level increases, but speech recognition in noisy environments deteriorates
Solution Approach 1:
The hearing aid employs dynamic processing that continuously adjusts amplification levels based on the user's gaze direction. As the user moves their gaze, the system dynamically shifts which directional channel receives enhanced amplification. This dynamic adaptation ensures that speech recognition accuracy is maintained by always amplifying the sound from the currently relevant direction rather than using a static amplification setting.
Solution Approach 2:
The system introduces gaze direction as an intermediary parameter that mediates between the multiple microphone inputs and the final audio output. The gaze tracker provides directional information that acts as a selector, determining which spatial channel should receive enhanced amplification. This intermediary mechanism allows the system to intelligently prioritize speech from the gazed-at direction while suppressing other sounds.
3Measurement precision
If directional microphones and gaze tracking are added to hearing aids, then speech isolation capability improves, but device complexity increases
Solution Approach 1:
The hearing aid system integrates multiple functions into a unified architecture: the microphone array serves both general sound collection and directional speech isolation; the gaze tracker provides both user interface control and spatial filtering information; and the processing system handles both ambient sound amplification and selective speech enhancement. This multi-functionality reduces the need for separate dedicated components for each function.
Solution Approach 2:
The system merges the gaze tracking functionality with the audio processing pipeline, using the same directional information for both display orientation and audio beamforming. The microphone array is combined with speech recognition algorithms to create an integrated speech enhancement system that leverages both spatial and spectral information. This merging reduces overall system complexity by sharing common components and processing pathways.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Some embodiments provide display eyewear with auditory enhancement. In general, one aspect disclosed features a head-wearable apparatus comprising: a microphone; a display panel visible to the wearer; a gaze tracker configured to determine a direction of a gaze of a wearer of the head-wearable apparatus; and a controller configured to: extract speech from sound collected by the microphone from the determined direction, and present the extracted speech on the display panel.