Hierarchical LSTM Audio Classification with Transient Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio classification methods struggle to achieve high accuracy and real-time decision-making, particularly in distinguishing between speech, vocal music, non-vocal music, and background noise, due to limitations in decision time resolution and model complexity.
Innovation Solution
The proposed method employs a hierarchical audio classification system using three stages of LSTM networks for binary classification, along with a parallel transition detection LSTM network. This approach utilizes short-term audio normalization and high-quality audio features extracted from raw PCM samples, enabling real-time classification with high accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single-stage audio classification model is used, then the device complexity is low, but the classification accuracy is insufficient to distinguish between speech, vocal music, non-vocal music, and background noise
Solution Approach 1:
The patent divides the audio classification task into three separate binary classification stages: (1) speech/music detection, (2) vocal/non-vocal music detection, and (3) background noise detection. Each stage uses its own LSTM network specialized for that specific classification task, allowing each model to be simpler and more accurate for its narrow function while collectively achieving high overall accuracy.
Solution Approach 2:
The patent transitions from a single multi-class classification problem to a hierarchical structure of multiple binary classification problems. By adding the dimension of hierarchical decision-making, the system can process audio signals through multiple classification layers, each making decisions based on the previous stage's output, thereby improving accuracy without requiring a single overly complex model.
2Loss of time
If the audio classification system processes audio signals in real-time, then the decision time resolution is high, but the accuracy may be compromised due to limited processing time
Solution Approach 1:
The patent extracts and prepares audio features in advance before classification. By pre-processing the audio signal to extract relevant features (spectral characteristics, temporal patterns, etc.) and normalizing them, the system reduces the processing time required during actual classification, enabling real-time decisions while maintaining high accuracy through thorough feature extraction.
Solution Approach 2:
The system dynamically adjusts the classification process based on the audio signal characteristics. The hierarchical structure allows the system to adapt its processing depth and speed based on the complexity of the audio signal, maintaining real-time performance while achieving high accuracy through flexible processing strategies.
3Measurement precision
If the system uses deep learning models for audio classification, then the classification accuracy is high, but the computational resources and model complexity increase significantly
Solution Approach 1:
Instead of using a single complex deep learning model for multi-class classification, the patent segments the classification task into three simpler binary classification problems, each handled by its own LSTM network. This segmentation reduces the complexity of individual models while maintaining high overall accuracy through hierarchical decision-making.
Solution Approach 2:
The patent applies deep learning (LSTM networks) selectively only where necessary - specifically for the complex classification tasks that require it, while using simpler processing for other stages. This partial application of complex models only where needed achieves high accuracy without unnecessarily increasing overall system complexity.
4Measurement precision
If the system extracts high-quality audio features from raw PCM samples, then the classification accuracy is improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent performs audio feature extraction and normalization as preliminary actions before the classification stage. By pre-processing the raw PCM samples to extract relevant features and normalize them, the system reduces the computational burden during actual classification, enabling high accuracy with acceptable processing times for real-time applications.
Data Source
AI summary
This invention provides a method and system for hierarchical audio classification, aimed at achieving high accuracy in prediction with enhanced decision time resolution. The disclosed method uses a perfectly tagged database which utilizes innovative techniques of labelling. Further data augmentation is also done using signal processing techniques like audio mixing and blending of different types of data. The disclosed method applies short term audio normalization on database for normalized training and prediction using AI based Long Short-Term Memory (LSTM) networks. The method and system employ the LSTM networks in a hierarchical structure to classify audio into desired 3 or more audio classes which include at least a background noise audio class. Decision time accuracy is improved by running the LSTM predictors over time overlapped slices and by using a separate transition detection neural network.