Timing Manager Adjusts Media Stream Frame Timestamps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming protocols face challenges in handling irregular timing information, leading to undesirable artifacts such as stuttering video and audio pops due to mobile devices' limited compute capacity and shared resource access, which causes deviations in frame processing rates.
Innovation Solution
A timing manager adjusts the processing of streamed media by modifying the expected timestamps and durations of frames to accommodate irregular timing data, ensuring a more regular cadence and synchronization of stream components, allowing for greater drift in sync conditions compared to regular timing data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If streaming protocols deliver media frames at advertised frame rates, then playback quality is maintained, but irregular timing information from mobile devices causes frame duplication and audio artifacts
Solution Approach 1:
The system dynamically adjusts the expected frame rate based on actual received timing information. Instead of rigidly adhering to the advertised frame rate, the timing manager continuously adapts the expected timestamps and durations to match the actual irregular timing patterns from mobile devices, preventing frame duplication and audio artifacts while maintaining playback quality
Solution Approach 2:
The system changes the timing parameters (expected timestamps and durations) dynamically in response to detected irregularities. When irregular timing information is detected, the timing manager modifies these parameters to accommodate the actual timing patterns, allowing the system to handle both regular and irregular timing information effectively
2Productivity
If mobile devices share compute capacity across multiple tasks, then device resource utilization is improved, but timing information irregularity increases
Solution Approach 1:
The timing manager implements a feedback mechanism that monitors actual frame timing information and uses this feedback to adjust expected timestamps and durations. This closed-loop approach allows the system to compensate for timing irregularities caused by shared resource access, maintaining playback quality even when mobile devices utilize compute capacity across multiple tasks
Solution Approach 2:
The system self-adjusts its timing expectations based on the actual timing information received from the mobile device. The timing manager autonomously detects irregularities and modifies processing parameters without requiring external intervention, allowing mobile devices to efficiently share resources while the receiving system automatically adapts to the resulting timing variations
Data Source
AI summary
Techniques for accommodating irregular timing information in streamed media data are described. According to some embodiments, at least a portion of a media stream that includes a video component is received, the video component including a plurality of video frames and video timing data, and wherein the media stream includes an indication of a video frame rate. A first duration of a first video frame of the plurality of video frames is determined to be different than an expected video frame duration, wherein the expected video frame duration is based at least in part on the indication of the video frame rate. A timestamp of at least one video frame of the plurality of video frames is adjusted to change the first duration.