Auditory Attention Decoding via Deep Neural Network Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hearing aids struggle to effectively separate the desired acoustic source from multiple acoustic sources in noisy environments due to similarities in spectro-temporal characteristics, leading to poor performance and associated social isolation and cognitive decline.
Innovation Solution
The use of end-to-end deep neural networks (DNNs) for auditory attention decoding, which process multi-channel EEG data and acoustic signals to compute similarity scores, allowing for the identification of the attended acoustic source in a single step, leveraging non-linear processing to optimize similarity functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional linear methods (e.g., least-squares) are used for stimulus reconstruction and correlation in AAD, then the system is simpler and more interpretable, but the ability to capture non-linear neural processing and optimize similarity functions is limited
Solution Approach 1:
The patent replaces traditional linear mechanical/mathematical systems (least-squares regression, linear correlation) with a neural network-based system that can model non-linear relationships. The neural network learns complex non-linear mappings between EEG signals and acoustic sources, substituting the simple linear algebraic approach with a more sophisticated computational model that better captures the complexity of neural processing.
Solution Approach 2:
The patent transforms the problem from a linear parameter optimization space to a non-linear parameter space by using neural network weights and activations. Instead of optimizing linear correlation coefficients, the system optimizes non-linear transformations through multiple layers of neurons, changing the fundamental parameters being adjusted to achieve better similarity scoring.
2Reliability
If hearing aids use traditional noise reduction algorithms, then the device complexity remains manageable, but the performance in separating multiple acoustic sources with similar spectro-temporal characteristics deteriorates
Solution Approach 1:
The patent replaces traditional signal processing algorithms (spectral subtraction, beamforming) with a neural network-based approach. Instead of using hand-crafted features and linear filtering, the system uses deep neural networks to learn robust representations of acoustic sources directly from the mixture, achieving better separation accuracy despite increased computational complexity.
Solution Approach 2:
The patent combines multiple data modalities (EEG signals, acoustic recordings, spectral features) into a composite input for the neural network. By fusing these different types of information in a unified model, the system achieves superior source separation performance that leverages the complementary strengths of each data type.
3Measurement precision
If more EEG channels are used to improve AAD performance, then the measurement precision increases, but the ease of operation and portability of the system decreases
Solution Approach 1:
The patent performs preliminary processing of EEG signals through the neural network to extract the most informative features and dimensions. By pre-processing the high-dimensional EEG data through learned transformations, the system identifies and emphasizes the critical channels and time-points that contribute most to attention decoding, effectively reducing the impact of having many channels while maintaining accuracy.
Data Source
AI summary
In one aspect of the present disclosure, method includes: receiving neural data responsive to a listener's auditory attention; receiving an acoustic signal responsive to a plurality of acoustic sources; for each of the plurality of acoustic sources: generating, from the received acoustic signal, audio data comprising one or more features of the acoustic source, forming combined data representative of the neural data and the audio data, and providing the combined data to a classification network configured to calculate a similarity score between the neural data and the acoustic source using one or more similarity metrics; and using the similarity scores calculated for each of the acoustic sources to identify, from the plurality of acoustic sources, an acoustic source associated with the listener's auditory attention.