Neural Signal Audio Correlation for Real-Time Attended Sound Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing auditory attention detection methods fail to detect attended sounds in real time due to inefficiencies in comparing pre-recorded EEG signals with audio signals, particularly for sounds with dynamically changing acoustic properties, such as natural language speech.
Innovation Solution
A system and method that analyze correlations between audio signals and neural signals in real time to determine the sound to which a user is attending, using features extracted from both signal types, and an audio output device that adjusts volume based on user attention levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pre-recorded EEG signals are compared with audio signals one by one to detect attended sounds, then detection accuracy can be achieved, but real-time detection cannot be accomplished due to inefficiency
Solution Approach 1:
The patent segments the audio signal into multiple short-time frames and processes each frame independently through the trained neural network model. This allows parallel processing of multiple audio segments simultaneously, enabling real-time detection while maintaining accuracy. The EEG signals are also segmented into corresponding time frames to match the audio segments for correlation analysis.
Solution Approach 2:
The patent performs preliminary training of the neural network model using paired EEG and audio data before actual detection. The model learns the relationship between neural responses and audio characteristics in advance. During real-time operation, the pre-trained model is directly applied to process incoming audio and EEG signals without requiring real-time training, thus achieving both speed and accuracy.
2Measurement precision
If average neural responses to repetitive sounds are calculated to obtain ERP responses, then detection results can be obtained, but efficiency is reduced due to the need for repetitive stimulation
Solution Approach 1:
The patent replaces the traditional mechanical approach of repetitive auditory stimulation and ERP averaging with a data-driven neural network model. Instead of requiring repetitive presentation of the same sound to elicit measurable ERP responses, the system uses a trained neural network to directly analyze the relationship between EEG signals and audio content, eliminating the need for repetitive stimulation while maintaining detection accuracy.
Solution Approach 2:
The patent changes the fundamental parameter of detection from time-locked ERP averaging to continuous neural response analysis. By using a neural network that processes EEG and audio signals in their natural temporal progression, the system captures dynamic neural responses to varying audio content without requiring the audio to be repeated or synchronized to specific time points.
3Reliability
If traditional AAD methods are used, then detection can be performed for static audio signals, but proper detection results cannot be obtained for sounds with dynamically changing acoustic properties like natural language speech
Solution Approach 1:
The patent implements a dynamic detection system that processes audio and EEG signals in continuous time frames, allowing the detection algorithm to adapt to changing acoustic properties. The neural network model is trained on diverse audio including natural speech with varying characteristics, enabling it to handle dynamic sounds. Each time frame is processed independently, capturing the instantaneous relationship between neural activity and audio content regardless of how the audio characteristics change over time.
Data Source
AI summary
A system for detecting a sound to which a user is attending based on neural signals includes an audio signal collection unit to collect audio signals including two or more sounds from a surrounding environment around the user; a neural signal collection unit to collect the neural signals of the user; an attended sound detection unit to analyze correlations between the two or more sounds included in the audio signals and the neural signals of the user in real time and determine the sound to which the user is attending based on the correlations; a database unit to store a result of the detection; and an output unit to select and output the stored individual audio signal or output the result of detecting the attended sound in real time according to the presence or absence of the audio signal in the surrounding environment around the user.