Sound Event Detection Transfer Learning for New Audio Classes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sound event detection systems are difficult to update to recognize new sounds without requiring significant computing resources and retraining, and maintaining large datasets becomes unwieldy as more sounds are added.
Innovation Solution
Utilize transfer learning techniques to generate an updated sound event detection model by retaining a previously trained model and adding an output layer with adapter networks, using a reduced dataset for training to detect additional sound classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional audio classification methods are used, then implementation complexity is reduced, but detection accuracy deteriorates due to inability to handle diverse sound events and background noise
Solution Approach 1:
The audio classification task is segmented into multiple specialized classification heads, each trained to detect specific sound event types (e.g., dog barking, car horn, glass breaking). This segmentation allows the system to achieve high detection accuracy for each specific sound type while maintaining a unified network architecture that doesn't excessively increase overall complexity.
Solution Approach 2:
The patent introduces temporal dimension by processing audio signals across multiple time frames and using recurrent neural network layers. This dimensional expansion enables the model to capture temporal patterns and contextual information, significantly improving detection accuracy for transient sound events without requiring proportionally higher computational complexity.
2Adaptability or versatility
If multiple sound event types are detected simultaneously, then detection coverage is improved, but false detection rate increases due to background noise confusion
Solution Approach 1:
Different parts of the network (classification heads) are specialized for detecting specific sound event types with locally optimized features. Each classification head focuses on particular acoustic characteristics of its target sound type, enabling the system to maintain high detection coverage across multiple sound types while reducing false detections through specialized local processing.
Solution Approach 2:
The loss function incorporates feedback mechanisms that penalize false positive predictions across all classification heads. During training, the system receives feedback about incorrect classifications and adjusts weights to reduce false detection rates, allowing simultaneous detection of multiple sound types with improved reliability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device includes a processor configured to receive audio data samples and provide the audio data samples to a first neural network to generate a first output corresponding to a first set of sound classes. The processor is further configured to provide the audio data samples to a second neural network to generate a second output corresponding to a second set of sound classes. A second count of classes of the second set of sound classes is greater than a first count of classes of the first set of sound classes. The processor is also configured to provide the first output to a neural adapter to generate a third output corresponding to the second set of sound classes. The processor is further configured to provide the second output and the third output to a merger adapter to generate sound event identification data based on the audio data samples.