Audio Scene Recognition Using Multi-Channel Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for accurately recognizing audio scenes in digital broadcasting, such as distinguishing between commentator speech and crowd sound in sports content, are inadequate, leading to inefficient execution of audio-related functions and user experience.
Innovation Solution
An electronic device employs a plurality of neural networks, including a CNN for time-frequency analysis, a DNN for spectral envelope analysis, and an RNN for combining channel results, to preprocess and recognize audio scenes by separating input audio signals into mid and side channels, calculating probabilities, and determining scene classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single neural network is used for audio scene recognition, then the system complexity is reduced, but the recognition accuracy deteriorates
Solution Approach 1:
The patent divides the audio scene recognition task into multiple specialized neural networks, each handling specific aspects: a first neural network processes mid-channel audio features, a second neural network processes side-channel audio features, and a third neural network integrates results. This segmentation allows each network to specialize in specific acoustic characteristics, improving overall recognition accuracy while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent introduces a new dimension to the recognition system by processing audio signals through multiple channels (mid-channel and side-channel) simultaneously. The first neural network analyzes one dimension (mid-channel) while the second neural network analyzes another dimension (side-channel), and the third network synthesizes both dimensions. This multi-dimensional approach captures more comprehensive audio scene information, significantly improving recognition accuracy.
2Measurement precision
If multiple neural networks are used for audio scene recognition, then the recognition accuracy is improved, but the processing time increases
Solution Approach 1:
By segmenting the recognition task into parallel neural networks that process different audio channels independently, the system can perform computations simultaneously rather than sequentially. The first neural network processes mid-channel signals while the second processes side-channel signals in parallel, and only the final integration step requires sequential processing. This segmentation dramatically reduces total processing time compared to a single sequential network.
Solution Approach 2:
The patent applies partial action by having each neural network focus on specific portions of the audio signal (mid-channel or side-channel) rather than processing the entire signal. This allows each network to operate faster on smaller data subsets, and the results are combined by the third network. The excessive action of processing multiple channels separately provides redundant information that improves robustness and accuracy while maintaining acceptable processing times.
3Productivity
If audio signals are processed in original format, then the information loss is minimized, but the processing efficiency deteriorates
Solution Approach 1:
The patent transforms the audio signal from the time domain to the frequency domain by converting it into a spectrogram. This dimensional change reveals additional information about the audio signal's frequency content over time, which is crucial for accurate scene recognition. The spectrogram representation maintains information integrity while enabling more efficient pattern recognition through frequency-based analysis.
Solution Approach 2:
The patent extracts key features from the audio signal by separating it into mid-channel and side-channel components, and further extracts spectral envelope information. By taking out and analyzing these specific features through specialized neural networks, the system achieves efficient processing without losing critical information. The extraction of spectral envelope and other acoustic features allows for compact representation that maintains recognition accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device and method of recognizing an audio scene are provided. The method of recognizing an audio scene includes: separating, according to a predetermined criterion, an input audio signal into channels; recognizing, according to each of the separated channels, at least one audio scene from the input audio signal by using a plurality of neural networks trained to recognize an audio scene; and determining, based on a result of the recognizing of the at least one audio scene, at least one audio scene included in audio content by using a neural network trained to combine audio scene recognition results for respective channels, wherein the plurality of neural networks includes: a first neural network trained to recognize the audio scene based on a time-frequency shape of an audio signal, a second neural network trained to recognize the audio scene based on a shape of a spectral envelope of the audio signal, and a third neural network trained to recognize the audio scene based on a feature vector extracted from the audio signal.