Dual Long-Term Reference Frame Error Recovery in Video Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video stream processing technologies face challenges in recovering from packet loss due to the large size and quality degradation associated with intra frames, and previous solutions like Gradual Decoder Refresh and Long Term Reference (LTR) frames are inefficient in minimizing network congestion and ensuring quick error recovery.
Innovation Solution
The use of two LTR frames for error recovery, where the decoder provides a fast update request listing present LTR frames, allowing the encoder to send P frames referencing the most recent LTR frame, thereby minimizing the use of intra frames and reducing network congestion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If I frames are used for error recovery, then video quality is recovered, but frame size increases significantly causing network congestion
Solution Approach 1:
The patent segments the reference frame storage by maintaining separate long-term reference frame buffers at both encoder and decoder sides. This allows selective storage and retrieval of reference frames without requiring complete I-frame transmissions, thereby reducing the quantity of data needed for error recovery while maintaining reliability.
Solution Approach 2:
The patent performs preliminary action by pre-storing long-term reference frames in buffers before packet loss occurs. When error recovery is needed, these pre-stored frames are immediately available for use, eliminating the need to transmit large I-frames and thus avoiding network congestion while ensuring rapid recovery.
2Reliability
If I frames are used for error recovery, then video quality is recovered, but intra frame beating behavior degrades quality
Solution Approach 1:
The patent uses copying by retrieving and reusing previously decoded long-term reference frames from buffer memory rather than decoding new I-frames. This copying approach maintains consistent video quality by avoiding the blurry/sharp transitions inherent in I-frame decoding, while still providing complete error recovery.
3Loss of energy
If GDR is used to refresh decoder gradually, then network bandwidth peak is avoided, but error recovery period is extended
Solution Approach 1:
The patent implements feedback mechanisms where the encoder receives information about the decoder's long-term reference frame buffer status. This feedback allows the encoder to make informed decisions about which frames to transmit and how to optimize error recovery, achieving rapid recovery without creating network bandwidth peaks.
4Quantity of substance
If LTR frames are used for error recovery without back channel, then I frame usage is reduced, but uncertainty about decoder LTR frame presence complicates recovery
Solution Approach 1:
The patent resolves the complexity issue by introducing feedback channels where the decoder reports its long-term reference frame buffer status to the encoder. This feedback enables the encoder to know exactly which LTR frames are available at the decoder, simplifying the error recovery decision process while maintaining reduced frame sizes.
Solution Approach 2:
The patent uses an intermediary communication mechanism (feedback channel) between encoder and decoder to exchange information about LTR frame availability. This intermediary solves the uncertainty problem without requiring large I-frames, thereby maintaining the benefits of small frame sizes while reducing recovery complexity.
Data Source
AI summary
Utilizing two LTR frames for improved error recovery. By using two LTR frames, much better performance is achieved in terms of error recovery as the likelihood of the decoder having one of the two LTR frames is very high. When the decoder determines a frame is lost, the decoder provides a fast update request (FUR). The FUR includes a listing of the LTR frames present at the decoder. With this indication of the LTR frames present at the decoder, the encoder utilizes one of the LTR frames, preferably the most recent, to use as a reference to send the next frame as a P frame. The P frame is sent with an indication of the LTR frame used as reference. The use of two LTR frames and the feedback of LTR frames present at the decoder allows the minimization of the use of I frames for error recovery.