Audio Scene Classification Using Temporal Correlation Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a need for an audio scene classification method with low latency and high accuracy that can efficiently process multi-channel audio signals, particularly for immersive sound experiences in home environments, which requires effective down-mixing and up-mixing based on scene classification results.
Innovation Solution
The method involves using neural networks to extract feature vectors from audio signals and calculate temporal correlation vectors, allowing for accurate scene classification and subsequent down-mixing or up-mixing of multi-channel audio signals, enabling flexible channel layout transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional scene classification methods are used, then device complexity is reduced, but classification accuracy and latency performance deteriorate
Solution Approach 1:
The audio scene classification system is segmented into multiple specialized neural networks: a first neural network for extracting audio features, a second neural network for temporal correlation analysis, and a third neural network for scene classification. This segmentation allows each component to be optimized for its specific function, improving overall classification accuracy while managing system complexity through modular design.
Solution Approach 2:
The patent introduces temporal correlation as an additional dimension for scene classification. By analyzing not only the current audio frame but also temporal relationships with previous frames through the second neural network, the system achieves more accurate classification by considering time-based patterns in audio scenes.
2Measurement precision
If more scene types are classified, then classification accuracy improves, but processing latency increases
Solution Approach 1:
The first neural network continuously extracts audio features from incoming audio frames in advance, preparing data for subsequent temporal correlation analysis and classification. This preliminary feature extraction enables the system to quickly classify new scenes without extensive processing delays.
Solution Approach 2:
The system maintains continuous processing by constantly extracting audio features and computing temporal correlations with previous frames. This continuous action allows the third neural network to immediately classify new audio scenes by comparing current features with the ongoing temporal context, reducing latency for multiple scene types.
3Measurement precision
If temporal correlation analysis is performed, then classification accuracy improves, but computational complexity increases
Solution Approach 1:
The patent extracts temporal correlation as a separate, dedicated function performed by the second neural network. This network specifically computes the correlation between current audio features and previous frame features, isolating this computationally intensive task from the main classification process and enabling more accurate scene classification through specialized temporal analysis.
Data Source
AI summary
An audio processing method includes obtaining a first audio signal corresponding to a first frame; extracting a first feature vector by inputting the first audio signal to a first neural network; obtaining a temporal correlation vector representing a similarity between the first feature vector and at least one second feature vector extracted from at least one second audio signal corresponding to at least one second frame that is temporally before the first frame; and classifying a scene of the first audio signal by inputting the first feature vector, the at least one second feature vector, and the temporal correlation vector to a second neural network.