Buffer Reduction via Selective Frame Dropping in Live Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Adaptive bitrate (ABR) streaming techniques result in significant end-to-end latency during live streaming events due to excessive video buffering, which persists even when network conditions stabilize, leading to delayed video playback and a poor user experience.
Innovation Solution
Implementing a frame dropping mechanism that reduces the quantity of buffered video frames by selectively dropping non-reference frames based on predetermined thresholds and frequency criteria, while ensuring audio and video synchronization, thereby reducing latency without altering the frame rate or fast-forwarding the video output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video buffering is increased to minimize playback stalling during network deterioration, then playback reliability is improved, but end-to-end latency increases
Solution Approach 1:
The patent implements dynamic buffer management by adjusting the buffer flush threshold based on network conditions. When network conditions stabilize or improve, the threshold is reduced to flush accumulated frames from the buffer, thereby reducing latency. When network conditions deteriorate, the threshold is increased to accumulate more frames for playback reliability. This dynamic adjustment resolves the contradiction between maintaining high playback reliability and minimizing end-to-end latency.
2Stability of the object's composition
If multiple seconds of video are buffered in the player video frame buffer, then playback stability during unstable network conditions is improved, but end-to-end latency increases significantly
Solution Approach 1:
The patent implements periodic buffer management by continuously monitoring network conditions and periodically adjusting the buffer flush threshold. The system periodically evaluates whether to flush buffered frames based on current network stability, allowing it to maintain playback stability when needed while periodically reducing latency when network conditions improve. This periodic action resolves the contradiction between playback stability and end-to-end latency.
3Loss of time
If the buffer flush threshold is reduced to decrease end-to-end latency, then live streaming timeliness is improved, but playback stalling may occur during network deterioration
Solution Approach 1:
The patent implements feedback-based buffer management by continuously monitoring network conditions and using this feedback to dynamically adjust the buffer flush threshold. When network conditions are stable or improving, the feedback triggers a reduction in the threshold to decrease latency. When network conditions deteriorate, the feedback triggers an increase in the threshold to maintain playback continuity. This feedback mechanism resolves the contradiction between reducing end-to-end latency and preventing playback stalling.
Data Source
AI summary
The techniques described herein may, for example, allow a quantity of buffered video frames to be reduced, for example for use in adaptive bitrate (ABR) streaming and other streaming transmissions. The quantity of buffered video frames may be reduced by dropping certain selected buffered frames from a resulting video output that is displayed to the viewer. This may reduce end-to-end latency without requiring a change in frame rate or fast forwarding of the video output, which may sometimes not be available or permitted by a resulting video player. These buffer reduction techniques may sometimes be employed upon a determination that transmission conditions have stabilized and that holding large quantities of buffered video may be unnecessary. In some examples, the disclosed techniques may be particularly advantageous for live streaming scenarios, in which end-to-end latency may be particularly problematic for the viewer.