VoD Audio-Video Frame Synchronization via Server-Side Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video on demand (VoD) services, there is a synchronization issue between video and audio frames due to the delay in transmitting audio frames corresponding to a specific playback time, causing a mismatch and delay in playing content from a requested time.
Innovation Solution
A content delivery server processes search requests by deleting audio frames before the requested playback time and reconstructs content data to include only frames from the corresponding video frame to the audio frame at that time, ensuring synchronization and transmitting this data at a high speed to reduce delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If audio frames are transmitted with a preset time delay after video frames with the same PTS, then audio-video synchronization is maintained in normal playback, but when seeking to a specific playback time, the audio frame corresponding to that time has not been transmitted yet, causing playback delay
Solution Approach 1:
The system performs preliminary actions by transmitting audio frames ahead of their scheduled playback time in the normal sequence. When a seek request is received, the necessary audio frames have already been transmitted or are available in the buffer, eliminating the waiting delay while maintaining proper synchronization at the target playback time.
Solution Approach 2:
The system dynamically adjusts the transmission timing of audio frames based on the seek request. Instead of a fixed preset delay, the transmission schedule is modified to ensure audio frames corresponding to the requested playback time are available, creating a dynamic adaptation to user actions while preserving synchronization.
2Reliability
If the system waits for the audio frame corresponding to the requested playback time to be transmitted before playing, then audio-video synchronization is ensured, but the user experiences a delay before playback starts
Solution Approach 1:
Audio frames are preliminarily transmitted or buffered before the actual playback time arrives. This preliminary action ensures that when the user seeks to a specific time, the audio data is already available, allowing immediate playback without waiting for transmission while maintaining synchronization.
Solution Approach 2:
The system changes the temporal dimension of audio frame transmission by decoupling transmission timing from playback timing. Audio frames can be transmitted in advance and stored in a buffer, allowing the playback system to access them immediately when needed without being constrained by the original transmission schedule.
3Loss of time
If audio frames before the requested playback time are deleted and reconstructed content data is transmitted, then playback delay is reduced and startup speed is improved, but the complexity of data processing and reconstruction increases
Solution Approach 1:
The system extracts and removes audio frames that correspond to times before the requested playback point from the transmission stream. By taking out only the necessary audio frames that need to be transmitted or buffered, the system reduces unnecessary data transmission while maintaining synchronization, balancing complexity reduction with functionality.
Solution Approach 2:
The audio stream is segmented into individual frames with specific PTS values. This segmentation allows the system to selectively identify, extract, and transmit only the relevant audio frames corresponding to or after the requested playback time, making the reconstruction process more manageable and efficient rather than handling the entire audio stream.
Data Source
AI summary
The present disclosure provides a method and a device for synchronization between a video frame and an audio frame on the basis of high-speed transmission of a VoD content. A content delivery server includes a transceiver and a processors. The transceiver receives, from a content requesting terminal, a search request for a specific playback time within a VOD content. In response to the search request, the processor deletes one or more audio frames existing between a video frame that corresponds to the requested playback time and an audio frame that corresponds to the requested playback time; and generates content data reconstructed to include all or part of frames existing in a period of time from the video frame that corresponds to the requested playback time to the audio frame that corresponds to the requested playback time. The transceiver transmits the reconstructed content data to the content-requesting terminal.