Video Encoder Long Term Reference Picture Retention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression standards, such as H.264 and H.265, face inefficiencies when updating long-term reference frames, as the entire frame is updated and the prior frame is discarded, leading to reduced compression efficiency when static backgrounds reappear in scenes.
Innovation Solution
The implementation of explicit signaling for extended long-term reference picture retention, where selected frames can be retained in the reference list for specified durations and marked as inactive or active, allowing for selective frame retention and reuse, improving compression efficiency by maintaining relevant frames without discarding them.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire LTR frame is updated and the prior frame is discarded, then the compression performance is improved by allowing better prediction, but the compression efficiency is reduced when the static background reappears in the video
Solution Approach 1:
The patent applies the discarding and recovering principle by allowing LTR frames to be discarded when updated and then recovered when needed again. The encoder signals LTR frame updates to decoders, and when a previously discarded LTR frame is needed again (when the static background reappears), it can be recovered by re-encoding and re-transmitting that frame in the bitstream. This resolves the contradiction by temporarily discarding frames to improve prediction accuracy during scene changes, then recovering them when they become useful again for compression efficiency.
Solution Approach 2:
The patent implements dynamics by making the LTR frame retention and update process adaptive rather than static. The encoder dynamically decides when to update LTR frames based on scene analysis, and the system adapts to scene changes by switching between using current frames and recovered LTR frames as reference. This dynamic approach allows the system to optimize prediction accuracy when backgrounds change while maintaining compression efficiency when static backgrounds reappear.
2Productivity
If LTR frames are retained for extended periods, then compression efficiency is improved by avoiding re-encoding of static backgrounds, but memory requirements and decoding complexity increase
Solution Approach 1:
The patent limits long-term frame retention by implementing a mechanism where LTR frames are discarded after a certain period or when updated, and only recovered when explicitly needed. This prevents the indefinite accumulation of reference frames in memory, thereby controlling memory requirements and decoding complexity while still achieving compression efficiency benefits when static backgrounds are present for extended periods.
3Measurement precision
If LTR frames are updated frequently, then prediction accuracy is maintained for changing backgrounds, but the overhead for signaling updates increases
Solution Approach 1:
The patent applies partial action by updating LTR frames selectively rather than frequently or universally. The encoder performs partial updates only for the specific LTR frames that need updating based on scene analysis, rather than updating all LTR frames frequently. This reduces the signaling overhead in the bitstream while maintaining prediction accuracy for the frames that actually need updates.
Data Source
AI summary
An encoder for video encoding and decoding with long term reference picture retention. The encoder generates an encoded bitstream for a compliant decoder. At the decoder, coded pictures are decoded using a list of references pictures, of which at least one is a long term reference picture. A first picture is decoded using the long term reference picture. A second picture is decoded without using the long term reference picture which is inactive but is retained in a buffer. A subsequent picture is then decoded again using the long term reference picture.