Live Video Frame Duplication and Extension for Streaming Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Live encoding systems face challenges in efficiently handling gaps in data transmission and server load, leading to potential failures in delivering frames to clients in real-time, which can result in streaming errors and increased hardware resource utilization.
Innovation Solution
Implementing techniques such as frame replication and extension to adjust bitrates, resolutions, and timestamps, allowing live encoding systems to compensate for network gaps and server overload without re-encoding, thereby maintaining continuous streaming and reducing hardware load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If frame duplication and extension techniques are implemented, then streaming continuity and reliability are improved, but device complexity increases
Solution Approach 1:
The system performs frame duplication and extension in advance before actual playback or transmission gaps occur. By pre-duplicating frames and extending their temporal coverage, the system ensures that continuous frames are available even when input streams experience gaps or delays, thereby maintaining streaming continuity without reactive complexity.
Solution Approach 2:
The patent creates duplicate copies of video frames and extends their temporal validity periods. These copied and extended frames serve as substitutes when original frames are unavailable, ensuring uninterrupted playback. This copying approach simplifies the handling of gaps compared to complex real-time interpolation or re-encoding methods.
2Loss of time
If frame duplication and extension are used to compensate for network gaps, then loss of time is reduced, but manufacturing precision of timestamp alignment deteriorates
Solution Approach 1:
The system dynamically adjusts the temporal parameters (timestamps and duration fields) of duplicated and extended frames. By modifying these parameters to reflect the extended validity period, the system maintains accurate synchronization information despite the frames covering longer time intervals, thus preserving timestamp precision while reducing streaming delays.
Solution Approach 2:
The patent implements dynamic timestamp adjustment where the end timestamps of extended frames are flexibly updated based on actual frame availability and playback progress. This dynamic parameter adjustment allows the system to adapt to varying network conditions while maintaining precise temporal alignment between extended frames and the overall stream timeline.
3Adaptability or versatility
If multiple adaptive bitrate streams are generated through live encoding, then adaptability is improved, but use of energy and computational resources worsens
Solution Approach 1:
Instead of performing full re-encoding for each adaptive bitrate stream, the system duplicates reference frames from the input stream and extends their temporal coverage. These duplicated frames are then distributed to multiple output streams at different bitrates, significantly reducing the computational energy required compared to traditional multi-rate encoding approaches.
Solution Approach 2:
The extended frames serve multiple functions simultaneously: they compensate for network gaps, provide content for multiple adaptive bitrate streams, and maintain synchronization across different output formats. This multi-functionality eliminates the need for separate encoding processes for each bitrate, reducing overall computational energy consumption while maintaining adaptability.
Data Source
AI summary
Embodiments of the invention provide for live encoding systems that can replicate a current encoded frame instead of re-encoding said current frame, and then adjust the replicated frame to different bitrates, resolutions, and/or contexts as necessary for the several different adaptive bitrate streams. In addition, various embodiments of the invention can extend a duration of a current frame being repackaged and/or re-encoded. Utilizing these and other techniques, live encoding systems in accordance with embodiments of the invention can more efficiently handle gaps in received data, slower feeding of data, and/or heavy loads on server hardware.