Audio Processing With Latency-Compensated Frame Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The synchronization of music accompaniment and recorded audio is affected by significant latency in existing audio recording systems, leading to a poor karaoke experience due to desynchronization during audio mixing.
Innovation Solution
A method and apparatus that calculate a reference timestamp based on play and capture latencies to synchronize audio playback and recording, optimizing the audio mixing process by aligning audio frames using a reference timestamp.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the terminal plays music accompaniment and records audio simultaneously, then karaoke service is provided, but latency causes desynchronization between music and recording
Solution Approach 1:
The system performs preliminary actions by calculating play latency and capture latency before audio mixing, determines a reference timestamp based on these latencies, and uses this timestamp to pre-align the audio frames. This preliminary synchronization action ensures that when music and recording are played simultaneously for karaoke, they remain synchronized despite the inherent latencies in playback and recording systems.
2Device complexity
If audio mixing is performed without latency compensation, then processing is simple, but music accompaniment and recorded audio are not synchronized
Solution Approach 1:
The system performs preliminary actions by calculating play latency and capture latency before audio mixing, determines a reference timestamp based on these latencies, and uses this timestamp to pre-align the audio frames. This preliminary synchronization action ensures that when music and recording are played simultaneously for karaoke, they remain synchronized despite the inherent latencies in playback and recording systems.
Data Source
AI summary
The disclosure relates to a method, an apparatus, a device, a storage medium and a program product for processing audio. The method includes: capturing an external sound to obtain a second audio during playing a first audio to the outside; determining a play duration of the first audio based on a current system time and a play latency; calculating a reference timestamp based on the play duration of the first audio and a capture latency; processing the second audio based on the reference timestamp to obtain a third audio; and performing audio mixing processing on the first audio and the third audio to obtain a target audio.