Extended LTR Frame Retention for Recurring Scene Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression standards like H.264 and H.265 incur efficiency losses when updating long-term reference frames, as the entire frame is updated and the prior frame is discarded, leading to inefficient encoding when static backgrounds reappear.
Innovation Solution
Implement an encoder that selects and retains extended long-term reference frames (eLTR) for variable or predetermined durations, signaled to the decoder for efficient frame prediction, using online or offline methods to optimize memory usage and compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire LTR frame is updated and the prior frame is discarded, then the prediction accuracy is improved for current scenes, but the compression efficiency deteriorates when static backgrounds reappear
Solution Approach 1:
The patent changes the parameter of LTR frame retention from binary (keep or discard) to variable retention time. The encoder signals a retention time parameter that determines how long an LTR frame should be kept in the reference picture list, allowing flexible adaptation to different scene conditions and optimizing the trade-off between prediction accuracy and compression efficiency.
Solution Approach 2:
The patent introduces dynamic retention time management for LTR frames. Instead of a static keep-or-discard approach, the system dynamically adjusts how long reference frames are retained based on scene characteristics, enabling the encoder to adapt to changing video content and maintain both prediction accuracy and compression efficiency.
2Productivity
If the encoder signals a newly decoded frame to be saved as an LTR frame, then the compression performance is improved for static backgrounds, but the memory usage and complexity increase
Solution Approach 1:
The patent introduces a retention time parameter that controls how long LTR frames are kept in memory. By adjusting this parameter, the system can optimize the balance between compression performance (which benefits from longer retention) and memory usage/complexity (which increase with longer retention), allowing flexible adaptation to different application requirements.
Solution Approach 2:
The patent applies partial retention by keeping only the necessary number of LTR frames for the specified retention time rather than maintaining all possible reference frames indefinitely. This partial action approach reduces memory usage and decoder complexity while still achieving compression performance improvements for static background scenes.
3Productivity
If LTR frames are retained for extended periods, then the compression efficiency is improved for recurring scenes, but the error-resilience deteriorates due to propagation of errors over time
Solution Approach 1:
The patent implements dynamic retention time management that allows the system to adaptively adjust how long LTR frames are kept. This dynamic approach enables optimization of the trade-off between compression efficiency (improved by longer retention for recurring scenes) and error-resilience (deteriorated by long-term error propagation), allowing flexible adaptation to different video content and error conditions.
Solution Approach 2:
The patent employs feedback mechanisms where the encoder monitors scene changes and video content characteristics to dynamically adjust LTR frame retention times. This feedback-driven approach allows the system to extend retention for stable recurring scenes to improve compression while limiting retention during scenes with potential errors or significant changes, thereby maintaining error-resilience.
Data Source
AI summary
A video encoder encodes a plurality of pictures with one of the pictures being a long term reference picture. The encoder provides signaling in a bitstream for a decoder to reconstruct pictures by selectively activating the long term reference picture for use as a reference picture, maintaining the long term reference picture in the buffer while being inactive as a reference picture, changing the long term reference picture from inactive to active as a reference picture, and marking the long term reference picture as being unused for reference so that it is no longer eligible to be used as a reference picture for a subsequent picture and is removed as a reference picture.