Audio Segmentation for Trick-Play Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional adaptive streaming systems do not provide playback of audio content during trick-play modes such as rewind, fast forward, and seek, limiting user experience in accessing specific parts of media content, especially during live events like sporting events.
Innovation Solution
The system encodes audio and video content into segments and generates synchronization information associating audio segments with video frames, allowing playback devices to provide audio content during trick-play modes by switching between alternative streams based on network conditions and maintaining a queue of audio segments for sequential playback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional adaptive streaming systems are used for trick-play modes, then network bandwidth adaptation is maintained, but audio content playback is not provided during trick-play modes
Solution Approach 1:
The audio content is divided into multiple segments that correspond to different time ranges of the media content. Each audio segment is associated with specific video frames, allowing the playback device to efficiently retrieve and play back only the relevant audio segment during trick-play operations without processing the entire audio stream, thus enabling audio playback during fast-forward/rewind while maintaining manageable system complexity
Solution Approach 2:
Audio segments are pre-encoded and stored separately before playback, with each segment associated with specific video frame ranges through synchronization information. This preliminary preparation allows the playback device to quickly switch between different audio segments during trick-play modes without real-time processing complexity, enabling seamless audio playback during fast-forward or rewind operations
2Ease of operation
If audio content is provided during trick-play modes, then user experience is enhanced, but synchronization between audio and video becomes more complex
Solution Approach 1:
Both audio content and video frames are divided into corresponding segments, with each audio segment linked to specific video frame ranges. This segmentation approach simplifies synchronization during trick-play by allowing the playback device to directly map selected video frames to their corresponding pre-segmented audio, reducing the complexity of real-time audio-video synchronization while enhancing user experience
Solution Approach 2:
Synchronization information acts as an intermediary data structure that associates audio segments with specific video frame ranges. This intermediary layer decouples the audio and video streams, allowing independent processing and efficient re-synchronization during trick-play operations without requiring complex real-time coordination between audio and video playback, thus simplifying the overall synchronization mechanism
3Reliability
If multiple audio segments are buffered for trick-play, then playback continuity is maintained, but memory requirements increase
Solution Approach 1:
Audio content is segmented into smaller units corresponding to specific video frame ranges, allowing the playback device to buffer only the necessary audio segments for the current trick-play operation rather than maintaining a large continuous buffer. This segmentation enables playback continuity during fast-forward or rewind by loading only the required audio segments into memory, reducing overall memory requirements while maintaining reliable playback
Solution Approach 2:
The system buffers audio segments selectively based on the current playback position and direction, loading only the portions needed for the immediate trick-play operation rather than pre-buffering entire audio tracks. This partial buffering approach maintains playback continuity during trick-play modes while minimizing memory buffer size requirements by avoiding unnecessary audio data storage
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for providing playback of audio content during a trick-play mode in accordance with embodiments of the invention are disclosed. Media content can include audio content and video content. The audio content is divided into segments and the video content includes frames. Synchronization information associates a presentation time of each of the segments of the audio content with a presentation time of one or more frames associated with the images of the video content. When a trick-play mode is requested, the next frame from the video content to present is determined. The synchronization information is used to determine the segments of the audio content associated with the next frame. The determined segments audio content are presented as the frames are displayed in trick-play mode.