Low-Latency Video Streaming Error Resilience via Distant Reference Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video streaming techniques face issues with error resilience in low-latency live video streaming, where bitstream errors lead to visual degradation and bandwidth consumption due to the propagation of errors through temporal prediction, which are not effectively addressed by current error handling methods.
Innovation Solution
The solution involves a decoder sending an error message to the encoder to use a distant, error-free reference frame for encoding a correction frame, rather than relying on preceding corrupted frames, thereby stopping the propagation of visual impairments and improving visual perception quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the decoder uses temporal prediction with preceding frames for error recovery, then error concealment is achieved, but visual degradation propagates through subsequent frames
Solution Approach 1:
The patent extracts the harmful corrupted frames from the temporal prediction chain by introducing an I-frame that references only the clean reference frame, isolating the corruption to a single frame rather than allowing it to propagate through multiple subsequent frames
Solution Approach 2:
The patent performs preliminary error detection and notification before the corruption wave can propagate extensively. The decoder detects errors and sends notifications to the encoder, which then proactively inserts correction frames before the corruption spreads through the temporal prediction chain
2Reliability
If I-frames or intra-refresh sequences are used to stop error propagation, then corruption is corrected, but bandwidth consumption increases
Solution Approach 1:
Instead of using full intra-refresh sequences that refresh entire frames, the patent applies partial correction by inserting I-frames only at specific locations where error propagation needs to be stopped, using the minimum necessary correction action to contain the corruption wave
Solution Approach 2:
The patent changes the reference frame parameter dynamically - switching from predictive coding (using preceding frames) to intra coding (using only reference frames) when errors are detected, thereby controlling error propagation while managing bandwidth usage
3Reliability
If error concealment is performed at the decoder, then visual degradation is reduced, but the decoder complexity increases
Solution Approach 1:
The error correction system serves itself by having the encoder proactively generate correction frames based on decoder error notifications, reducing the burden on the decoder to perform complex error concealment operations and shifting some functionality to the encoder
Data Source
AI summary
Various aspects of the subject technology relate to systems, methods, and machine-readable media for recovering lost information in a data stream. The method includes receiving the data stream having consecutive frames from a server, each frame encoded based on information from a preceding frame in an encoding order as reference. The method also includes detecting an error in the data stream. The method also includes sending an error message to the server regarding the error. The method also includes receiving a corrected frame encoded based on a distant reference frame that was received prior to detection of the error. The method also includes decoding the corrected frame.