Audio Event Detection Using Multi-Scale Frame Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio event detection systems face challenges such as false positives, missed positives, data variance, and the inability to accurately determine the start and end times of audio events, primarily due to frame-by-frame prediction variability in audio data processing.
Innovation Solution
The proposed system employs a multi-model subsampling and upsampling process using convolutional recurrent neural networks (CRNNs) and region proposal networks (RPNs) to extract audio features, adjust time windows, and classify events, providing more accurate detection of audio event onset and offset times by processing audio data into subsampled and upsampled frames, which are then scored and combined to determine the likelihood of an event occurrence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frame-by-frame prediction is used for audio event detection, then the system can process audio data in real-time, but the detection accuracy deteriorates due to variability in predictions
Solution Approach 1:
The patent segments the audio data processing into multiple independent models (e.g., different time window sizes, different feature extraction approaches). Each model processes the same audio data independently and produces a prediction. These segmented predictions are then combined through voting or averaging to produce the final detection result, reducing variability while maintaining real-time processing capability.
Solution Approach 2:
The patent merges predictions from multiple independent models to produce a consolidated detection result. By combining the outputs of several models that process audio data through different approaches (different window sizes, different feature extractions), the system reduces the variability inherent in single-model frame-by-frame predictions while preserving real-time processing throughput.
2Measurement precision
If multiple models are used for audio event detection, then detection accuracy improves, but system complexity increases
Solution Approach 1:
The system segments the detection task into multiple specialized models, each handling a specific aspect of audio event detection (e.g., different time windows, different feature types). This segmentation allows each model to be simpler and more focused, while the collective ensemble achieves high accuracy. The modular structure also makes the system more manageable despite the multiple components.
Solution Approach 2:
The patent uses multiple copies of detection models with different configurations (different window sizes, different feature extractions) to process the same audio input. These copied models provide diverse perspectives on the same data, improving accuracy through ensemble methods while maintaining relatively simple individual model structures that are easier to train and deploy.
3Measurement precision
If audio data is processed with multiple time window sizes, then event timing precision improves, but processing time increases
Solution Approach 1:
The patent applies periodic action by using multiple models with different time window sizes that process audio data at different scales. Each model operates periodically on the same input data with its specific window configuration, and their results are combined to achieve precise event timing detection. This approach extracts timing information at multiple temporal resolutions without requiring sequential processing of all windows.
Solution Approach 2:
The system employs partial action by selectively applying different time window sizes based on the specific detection needs. Rather than processing all possible window sizes equally, the system uses a subset of window configurations that provide sufficient timing precision for the application, balancing accuracy requirements with processing time constraints.
Data Source
AI summary
An audio event detection system that subsamples input audio data using a series of recurrent neural networks to create data of a coarser time scale than the audio data. Data frames corresponding to the coarser time scale may then be upsampled to data frames that match the finer time scale of the original audio data frames. The resulting data frames are then scored with a classifier to determine a likelihood that the individual frames correspond to an audio event. Each frame is then weighted by its score and a composite weighted frame is created by summing the weighted frames and dividing by the cumulative score. The composite weighted frame is then scored by the classifier. The resulting score is taken as an overall score indicating a likelihood that the input audio data includes an audio event.