Target Sound Event Localization With Class-Specific Neural Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sound event localization and detection (SELD) systems face challenges in accurately localizing sound sources that move, are obscured by reverberation, or mixed with interfering sounds, especially when dealing with a large number of classes, and require extensive training data for each class.
Innovation Solution
A class-specific SELD system that processes spatial and spectral features of acoustic mixtures using a neural network trained to identify and localize a single target sound event, incorporating FiLM blocks and convolution layers to estimate direction of arrival (DOA) and distance, even with limited training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a DNN outputs ACCDOA representations for all classes at every time instant, then comprehensive sound event detection is achieved, but computational complexity and resource consumption increase significantly
Solution Approach 1:
The system segments the sound event detection task by creating separate detection heads for different classes (e.g., speech, noise, music). Each detection head processes only its specific class using a dedicated ACCDOA representation, rather than computing all classes simultaneously. This segmentation reduces the computational burden at each time instant while maintaining comprehensive detection coverage across all sound event classes.
2Measurement precision
If class-specific models are trained for each sound event class, then detection accuracy for specific classes improves, but training data requirements become extremely large
Solution Approach 1:
The system merges the common feature extraction backbone with class-specific detection heads. The shared backbone processes the acoustic input and extracts general features, while each class-specific head receives the same features and applies class-conditioned ACCDOA representations. This merging allows the system to achieve class-specific detection accuracy without requiring separate training data sets for each class, as the shared backbone learns from all classes collectively.
3Reliability
If localization is performed for all classes at all times, then complete sound event monitoring is achieved, but processing time and computational resources are wasted on irrelevant classes
Solution Approach 1:
The system dynamically activates only the necessary class-specific detection heads based on the current acoustic context. Instead of computing ACCDOA representations for all classes at every time instant, the system determines which classes are relevant to the current sound mixture and processes only those. This dynamic approach maintains complete monitoring capability while significantly reducing processing time by avoiding computation for irrelevant classes.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Embodiments of the present disclosure disclose a system and method for localization of a target sound event. The system collects a first digital representation of an acoustic mixture of sounds of a plurality of sound events, by using an acoustic sensor. The system receives a second digital representation of a sound corresponding to the target sound event. Further, the first digital representation and the second digital representation are processed by a neural network to produce a localization information indicative of a location of an origin of the target sound event with respect to a location of the acoustic sensor.