Hearing System Camera-Based Sound Source Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hearing systems fail to effectively enhance speech intelligibility in adverse listening conditions by not being able to accurately identify and track specific speakers in multi-person environments, as they lack the capability to optimize beam forming based on the location of the intended speaker.
Innovation Solution
A method and system that utilize a camera and an auxiliary device to capture images, process them to determine the directional angle of the sound source relative to the user, and transmit consolidated data to the hearing device for improved sound source tracking and beam forming, while maintaining energy efficiency by offloading complex image analysis to the auxiliary device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image processing is performed in the hearing device to determine sound source location, then speech intelligibility is improved, but energy consumption increases
Solution Approach 1:
An auxiliary device serves as an intermediary between the camera and the hearing device. The auxiliary device performs computationally intensive image processing to determine sound source location, while the hearing device only receives and uses the processed location data. This mediator approach allows the hearing device to benefit from accurate sound source tracking without bearing the energy cost of complex image processing.
Solution Approach 2:
The system divides the overall processing task into two segments: (1) image capture and complex image processing performed by the auxiliary device, and (2) sound source location application performed by the hearing device. This segmentation allows each component to specialize in its optimal function, with the auxiliary device handling computationally intensive tasks and the hearing device handling audio processing and beamforming.
2Measurement precision
If beam forming is optimized to track speaker location, then speech intelligibility is improved, but device complexity increases
Solution Approach 1:
The auxiliary device acts as a mediator that handles the complexity of image processing and sound source localization algorithms. The hearing device receives pre-processed location information and only needs to implement beamforming based on provided directional data, significantly reducing the computational complexity burden on the hearing device itself.
Solution Approach 2:
The auxiliary device is designed to perform multiple functions: capturing images via camera, processing images to identify speakers, determining sound source locations, and transmitting this information to the hearing device. This multi-functional approach consolidates complexity into a single external device rather than requiring the hearing device to perform all these functions.
Data Source
Figure 1~2
Figure 3a~3b
Figure 4~5a
AI summary
A method for operating a hearing system comprising a hearing device (1), a camera (11, 12) and an auxiliary device (21), the method comprising the steps of providing an input signal to said hearing device (1), capturing an image or a sequence of images of at least sections of a surrounding of a user (10) wearing said hearing device (1), processing said image or said sequence of images in said auxiliary device (21) for obtaining consolidated data of a sound source (15) being important for said user (10), transmitting said consolidated data to the hearing device (1), generating an output signal in said hearing device (1) by processing the audio signal and by taking into account said consolidated data, and feeding said output signal to an output transducer of said hearing device (1). Thereby, a hearing system having improved capabilities is obtained.