Video Stream Error Propagation Tracking and Refresh
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video stream encoding methods face challenges in error resilience, particularly in low latency systems, due to high end-to-end latency, compression efficiency loss, and rapid error propagation, especially when using predictive video coders and periodic intra-frame coding, which can result in jerky video and loss of lip sync.
Innovation Solution
A method that initializes, updates, and resets an error propagation metric differently for static and non-static regions to minimize refresh of static areas, using a pixel dependency count and probabilistic distortion estimate to select refresh macroblocks, and codes them as inter macroblocks with respect to a reliable reference frame to improve coding efficiency and reduce error propagation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic intra-frame coding is used to provide error resilience, then error propagation is reduced, but compression efficiency is reduced and more bits are required
Solution Approach 1:
The patent applies local quality by differentiating between static and non-static regions in the video frame. Static regions are refreshed less frequently while non-static regions are refreshed more frequently based on their motion characteristics. This selective refresh strategy reduces the overall number of intra-coded blocks needed, thereby improving compression efficiency while maintaining error resilience in critical areas.
Solution Approach 2:
The patent implements dynamics by making the refresh pattern adaptive rather than fixed. The encoder dynamically determines which regions need refreshing based on motion estimation and error propagation metrics. This dynamic approach allows the system to adjust the refresh rate and spatial distribution of intra-blocks according to actual video content and channel conditions, optimizing the balance between reliability and compression efficiency.
2Reliability
If periodic intra-frame coding is used to break the prediction dependency chain, then error propagation is stopped, but end-to-end latency increases
Solution Approach 1:
The patent applies periodic action through committed refresh periods, where intra-blocks are inserted at regular intervals to control error propagation. By establishing a predictable periodic refresh pattern, the system can manage latency more effectively than ad-hoc refresh methods, as the timing of error correction is known in advance and can be synchronized with video playback requirements.
Solution Approach 2:
The patent reduces latency impact by applying different refresh strategies to different regions. Static regions use longer refresh periods reducing their latency contribution, while non-static regions use shorter periods. This localized approach ensures that the overall system latency remains acceptable while still providing effective error propagation control where needed.
3Reliability
If motion adaptive refresh is used to select refresh macroblocks, then error propagation is reduced, but implementation complexity and processing requirements increase
Solution Approach 1:
The patent simplifies motion adaptive refresh by using local quality metrics (motion estimation results) to directly guide refresh decisions. Instead of complex global optimization, the system independently evaluates each macroblock's motion characteristics and applies refresh based on simple thresholds and rules, reducing computational complexity while maintaining effectiveness.
Solution Approach 2:
The patent applies segmentation by dividing the video frame into macroblocks and processing them independently based on their individual error propagation metrics. This segmented approach allows complex error propagation control to be implemented through simple local operations rather than complex global calculations, reducing overall processing complexity.
4Productivity
If non-uniform distribution of refresh macroblocks is used, then coding efficiency is improved, but rapid error propagation occurs due to lack of spatial relationship
Solution Approach 1:
The patent addresses this contradiction by using local quality metrics to guide spatial distribution of refresh blocks. By analyzing motion characteristics and error propagation metrics at the macroblock level, the system can cluster refresh blocks in regions where they are most needed while maintaining coding efficiency. This localized spatial awareness prevents rapid error propagation without requiring uniform distribution.
Data Source
AI summary
A method and device for tracking error propagation and refreshing a video stream is provided. The proposed subject matter comprises of an error propagation tracking method that works in the sub-sampled domain to reduce computational cycles and memory bandwidth. Further, the tracking based update of the error propagation metric is done differently for static and non-static regions to avoid unnecessary refresh of static areas. Through suitable thresholding of the metric at a macroblock (MB) level, a set of refresh MBs are selected for each frame. These refresh MBs are coded either as an intra MB or as an inter MB that is predicted from one or more reliable reference frames (—frames that are known to be available at the decoder with negligible errors—). Such inter coding of refresh MBs improves the compression efficiency when compared to pure intra coding of refresh MBs. Further, variants to the threshold selection are presented that result in temporally uniform distribution of the number of refresh MBs and a strict refresh scheme wherein all MBs are guaranteed to be with negligible errors following a packet loss within a committed refresh period. In addition, to using the error propagation metric, spatial connectivity to already chosen refresh MBs is used in the selection of additional refresh MBs within a frame and across frames; this reduces the rate of error propagation due to part of a macroblock predicting from older, erroneous neighboring MBs and in turn requiring more refresh MBs on the average per frame.