Media Device Pipeline Architecture for Streaming Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Media devices experience video freezes and audio defects during streaming, leading to an undesirable user experience due to issues like delays in processing, bandwidth changes, and synchronization errors between audio and video.
Innovation Solution
Implementing a secure zone and non-secure zone architecture in media devices, using buffer queues to manage video frames, and applying adaptive bitrate streaming to mitigate these issues, along with dropping video frames to synchronize audio and video effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adaptive bitrate streaming is implemented to handle bandwidth changes, then the reliability of streaming content is improved, but the complexity of the media device increases
Solution Approach 1:
The patent segments the media processing into separate secure and non-secure zones with dedicated buffer queues for video frames and audio samples. This segmentation allows independent management of different media types and processing stages, improving reliability through isolated failure containment while managing complexity through modular architecture.
Solution Approach 2:
The patent introduces buffer queues as intermediary structures between the network layer and rendering layer. These buffers act as mediators that decouple the streaming process from the display process, allowing the system to absorb bandwidth variations and synchronization errors without propagating failures through the entire system, thus improving reliability while maintaining manageable complexity.
2Reliability
If video frames are dropped to synchronize audio and video, then the synchronization between audio and video is improved, but the loss of video information increases
Solution Approach 1:
The patent implements feedback mechanisms where the system monitors synchronization status between audio and video streams. When desynchronization is detected, the system selectively drops video frames based on feedback from synchronization status, allowing dynamic adjustment to maintain audio-video sync while minimizing unnecessary frame loss through intelligent feedback-driven decisions.
Solution Approach 2:
The patent changes the parameter of video frame retention dynamically based on synchronization status. When audio-video sync is compromised, the system modifies the video processing parameters to drop specific frames, thereby restoring synchronization. This parameter change approach allows the system to balance information retention with synchronization requirements.
3Stability of the object's composition
If buffer queues are used to manage video frames and audio samples, then the stability of media playback is improved, but the device complexity increases
Solution Approach 1:
The patent segments the buffer queue management into separate secure and non-secure zones, with dedicated queues for video frames and audio samples. This segmentation stabilizes playback by isolating different media types in separate memory regions, preventing corruption spread, while managing complexity through clear functional separation and dedicated management paths for each queue type.
Data Source
AI summary
Media devices may stream a wide variety of content for presentation. Described herein are systems, devices, and methods for improving the presentation of the content. The media device may execute a pipeline architecture comprising a sequential execution of a plurality of threads. The threads may include a decryption thread, a decoding thread, and a rendering thread. The media device may include buffer queues between the threads, which enable continuous content presentation when one or more of the threads fail to communicate with another thread. The media device may pause the presentation of content by suspending the execution of the rendering thread. In one implementation, the media device replaces a stale frame that is displayed after performing a seek function with a black frame. In another example, to maintain content presentation, the media device drops video frames if an audio video lag event occurs.