Wearable Audio Directional Filtering via Camera-Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current wearable devices for lifelogging, such as smartphones, are not ideal due to their size and design, limiting their ability to provide enhanced interaction with the environment through feedback and advanced functionality based on image and audio analysis.
Innovation Solution
A wearable system comprising a microphone and camera that determines the user's looking direction to selectively amplify audio from that direction, while attenuating audio from other directions, and transmits the processed audio to a hearing interface device for improved auditory feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If audio signals are selectively amplified based on looking direction, then auditory feedback quality is improved, but device complexity increases due to integration of camera, microphone array, and signal processing components
Solution Approach 1:
The patent combines multiple functional components (camera, microphone array, processor, and hearing interface device) into an integrated wearable system. The camera captures images to determine looking direction, while microphones capture audio signals, and the processor integrates both data streams to selectively amplify audio from the user's field of view. This merging of functions resolves the contradiction by creating a unified system where the complexity is organized and coordinated to deliver high-quality directional auditory feedback.
Solution Approach 2:
The processor acts as an intermediary that receives and processes both visual data from the camera and audio data from the microphones. It determines the user's looking direction based on image analysis and uses this information to selectively amplify audio signals from corresponding directions. This intermediary processing step enables the system to coordinate multiple components effectively, improving auditory feedback quality while managing system complexity through centralized intelligence.
2Loss of information
If multiple audio signals are transmitted simultaneously to the speaker, then auditory information completeness is improved, but signal interference increases
Solution Approach 1:
The system transmits audio signals in a coordinated manner where the first audio signal (environmental sound) and the second audio signal (processed directional sound) are transmitted at different times or with different priorities. The processor manages the timing and sequencing of signal transmission to the speaker, ensuring that important directional audio information is delivered without being overwhelmed by continuous environmental noise. This periodic or prioritized transmission reduces interference while maintaining information completeness.
Solution Approach 2:
The system applies different processing qualities to different audio signals. The second audio signal, which contains the directionally filtered and amplified sound from the user's field of view, is processed with higher priority and enhanced quality. The first audio signal (raw environmental sound) is transmitted with lower priority or reduced processing. This local quality differentiation ensures that the most important auditory information (directional sound) is delivered with high fidelity while minimizing interference from less critical background noise.
Data Source
AI summary
System and methods for selectively amplifying audio signals are disclosed. In one implementation, a method includes receiving at least one image of a plurality of images captured by a wearable camera; receiving a first audio signal representative of the sounds captured by a microphone; determining a looking direction of a user; and processing the first audio signal by amplifying audio coming from the looking direction of the user and attenuating audio coming from at least one other direction; receiving a second audio signal representative of the sounds captured by a hearing interface device; transmitting the second audio signal to a speaker associated with the hearing interface device; transmitting an additional audio signal to the speaker, wherein the transmission of the additional audio signal at least partially overlaps the transmission of the second audio signal; and transmitting the processed first audio signal to the speaker.