Hearing System 3D Sound Localization via IMU Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hearing aids with only left and right microphones are limited in determining the 3D direction of sound sources due to ambiguity in elevation and azimuth, and cannot distinguish between user and sound source movement.
Innovation Solution
Incorporating 3D gyroscopes, 3D accelerometers, and magnetometers (IMUs) to estimate the orientation and translation of the hearing device, allowing for the synthesis of a 3D DOA sensor from a 2D DOA sensor array, and using processors to estimate the 3D location of sound sources based on time differences and sensor array configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only left and right hearing devices with single microphones are used, then the device complexity is low, but the measurement precision of 3D direction of arrival is insufficient due to ambiguity in elevation and azimuth
Solution Approach 1:
The patent transitions from 2D DOA estimation (azimuth only) to 3D DOA estimation (azimuth and elevation) by incorporating vertical microphone spacing in the sensor array. The microphone array includes microphones positioned at different heights (e.g., mic1 at height h1, mic2 at height h2), enabling the system to resolve elevation angles and eliminate the front-back ambiguity inherent in 2D arrays. This dimensional expansion allows accurate 3D localization of sound sources.
Solution Approach 2:
The patent divides the hearing system into multiple independent hearing devices (left and right hearing aids), each containing its own microphone array. This segmentation allows each device to independently perform DOA estimation while maintaining spatial separation, which provides additional geometric constraints for resolving 3D source location. The multiple devices work together to overcome the limitations of a single array configuration.
2Adaptability or versatility
If hearing devices are made stationary, then the measurement precision of sound source location is high, but the adaptability to user movement is poor
Solution Approach 1:
The patent incorporates motion sensors (accelerometers, gyroscopes) that continuously monitor the movement of the hearing devices and provide feedback to the signal processing system. This feedback allows the system to dynamically adjust the DOA estimation calculations by compensating for device motion, thereby maintaining accurate sound source localization even when the user is moving. The system distinguishes between movement of the hearing devices and movement of the sound source.
Solution Approach 2:
The patent transitions from assuming stationary hearing devices to a dynamic model where device motion is explicitly accounted for. The system uses real-time motion data from inertial sensors to update the geometric relationships between the microphone arrays and sound sources, enabling accurate 3D localization during user activities such as walking, turning, or changing head position. This dynamic adaptation maintains measurement precision across varying motion conditions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate 3D direction and location estimation of sound sources, overcoming the limitations of 2D ambiguity and movement confusion, thereby improving sound source separation and noise attenuation in complex environments.
Implementation Method 1
HAs equipped with 3D gyroscopes, 3D accelerometers and 3D magnetometers, so-called Inertial Measurements Units, IMUs for short
Implementation Method 2
HAs equipped with 3D gyroscopes, 3D accelerometers and 3D magnetometers, so-called Inertial Measurements Units, IMUs for short
Implementation Method 3
HAs equipped with 3D gyroscopes, 3D accelerometers and 3D magnetometers, so-called Inertial Measurements Units, IMUs for short
Implementation Method 4
A sensor array of M input transducers, e.g. microphones, each for providing an electric input signal representing said sound in said environment
Implementation Method 5
extracting sensor array configuration specific data τij of said sensor array indicative of differences between a time of arrival of sound from said localized sound source S at said respective input transducers
Data Source
AI summary
A hearing system is adapted to be worn by a user and configured to capture sound in an environment of the user and comprises a) a sensor array comprising M transducers for providing M electric input signals representing said sound and having a known geometrical configuration relative to each other; b) a detector unit for detecting movements over time of the hearing system, and providing location data of said sensor array at different points in time t, t=1, . . . , N; c) a first processor for receiving said electric input signals and—in case said sound comprises sound from a localized sound source S—for extracting sensor array configuration specific data τij of said sensor array indicative of differences between a time of arrival of sound from said localized sound source S at said respective input transducers, at said different points in time t, t=1, . . . , N; and d) a second processor configured to estimate data indicative of a location of said localized sound source S relative to the user based on corresponding values of said location data and said sensor array configuration data at said different points in time t, t=1, . . . , N.