Audio Source Separation with Onset Detection and Latency Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current audio processing techniques are inadequate for effectively separating and mixing audio content, particularly in situations where original audio sources need to be isolated and enhanced from mixed audio signals, often resulting in imperfect separation and residual noise.
Innovation Solution
An electronic device and method that performs source separation using blind source separation techniques, followed by onset detection and latency compensation to generate an enhanced separated source by mixing the audio signal with the separated source based on the onset detection signal, thereby improving the quality of the separated audio signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If source separation is performed on mixed audio content, then original audio sources can be isolated, but residual noise and imperfect separation occur
Solution Approach 1:
An onset detection signal is introduced as an intermediary element to control the mixing process. This signal detects onset phases in the separated source and uses this information to dynamically adjust the mixing between the original audio content and the separated source, thereby reducing residual noise while maintaining separation quality
Solution Approach 2:
Onset detection is performed in advance on the separated source to identify onset phases before the final mixing operation. This preliminary detection allows the system to prepare appropriate mixing gains that compensate for latency and improve separation accuracy at critical transient moments
2Manufacturing precision
If mixing is performed to enhance separated sources, then sonic quality improves, but latency and timing errors occur
Solution Approach 1:
The system uses onset detection as a feedback mechanism to monitor the separated source and dynamically adjust the mixing gains. By detecting onset phases and using this information to control the mixing process, the system compensates for latency and timing errors, maintaining synchronized alignment between the original audio content and separated sources
Solution Approach 2:
The mixing gains are dynamically changed based on the onset detection signal. When onsets are detected, the system adjusts the gain parameters to favor the separated source; during other periods, it balances the mix to reduce latency artifacts. This dynamic parameter adjustment optimizes both sonic quality and timing accuracy
3Manufacturing precision
If blind source separation is used to separate audio sources, then original sources can be extracted, but crosstalk between sources remains
Solution Approach 1:
Instead of attempting complete separation of all sources, the system applies partial mixing based on onset detection. By selectively mixing the original audio content with the separated source only when onsets are detected, the system achieves sufficient source isolation for critical moments while accepting some degree of crosstalk during non-onset periods, thereby improving overall source isolation quality
Data Source
AI summary
An electronic device comprising circuitry configured to perform (402; 702; 1204) source separation (201) based on a received audio input to obtain a separated source, to perform onset detection (202) on the separated source to obtain an onset detection signal and to mix (405; 706; 1207) the audio signal with the separated source based on the onset detection signal to obtain an enhanced separated source.