Zero-Copy Adaptive Bitrate Video Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional adaptive bitrate video streaming methods introduce latency and resource overhead by copying video frames into a transmission buffer, which does not address delays occurring earlier in the video streaming pipeline.
Innovation Solution
Implementing zero-copy adaptive bitrate video streaming, where video frames are captured, encoded, and transmitted without being copied into a buffer, allowing for real-time adjustment of video quality based on frequency of discarded frames and network bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video frames are copied into a transmission buffer for adaptive bitrate streaming, then bandwidth adaptation is achieved, but latency and resource cost increase
Solution Approach 1:
The patent extracts the video frame data directly from the capture pipeline and feeds it to the encoder without copying into a transmission buffer. This eliminates the buffering step while maintaining adaptive bitrate capability through direct encoder control and selective frame discarding based on bandwidth conditions.
Solution Approach 2:
The patent performs preliminary actions by proactively discarding frames at the encoder input based on predicted bandwidth conditions, rather than buffering frames and then deciding what to transmit. This preliminary frame discarding prevents latency accumulation while still achieving bandwidth adaptation.
2Adaptability or versatility
If video frames are copied into a transmission buffer, then adaptive bitrate streaming is enabled, but resource cost increases
Solution Approach 1:
The patent extracts frame data directly through memory sharing between capture and encoding components, eliminating the need for buffer memory allocation. This reduces the quantity of memory resources required while maintaining adaptive bitrate functionality.
Solution Approach 2:
The patent eliminates unnecessary copying operations by using zero-copy techniques where frame data is referenced directly between capture and encoding modules, reducing both memory usage and processing overhead associated with data duplication.
3Loss of time
If frames are discarded when encoder backlog occurs, then latency is reduced, but video quality may deteriorate
Solution Approach 1:
The patent applies partial discarding of frames only when necessary due to encoder backlog or bandwidth constraints, rather than discarding all frames or buffering all frames. This selective partial action maintains quality for frames that can be processed while preventing latency accumulation.
Solution Approach 2:
The patent uses feedback from encoder backlog status and bandwidth conditions to dynamically control frame discarding decisions. This feedback mechanism ensures frames are discarded only when necessary, maintaining video quality while managing latency and bandwidth adaptation.
Data Source
AI summary
A method includes capturing video frames of a video item to be transmitted as a livestream and delivering the video frames to a queue for an encoder of a user device. The method further includes determining that a subset of the video frames have been discarded. The discarding indicates that the subset of the video frames did not enter the encoder. The method further includes determining a frequency of discarded video frames. Responsive to the frequency exceeding a threshold frequency, the method includes reducing quality of the video item transmitted as the livestream based on the frequency by reducing one or more of frame rate of the user device, resolution of the user device, or encoder bitrate of the encoder.