Video Coding Patch Cache for Low Correlation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems face inefficiencies due to poor prediction correlation when reference frames are inadequate, particularly in scenarios with scene changes, camera movement, or high noise levels, as they typically store a limited number of reference frames that fail to provide effective prediction.
Innovation Solution
A video coding system that includes an encoder with a coding engine, a decoder, a reference picture cache, and a patch cache, where prediction patches are generated from various sources such as predefined image content, evicted reference pictures, and image data from prior sessions, allowing for improved prediction references even when temporal correlation is low.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a limited number of reference frames are stored in the reference picture cache, then memory usage is controlled, but prediction accuracy deteriorates when temporal correlation is low
Solution Approach 1:
The patent introduces a new dimension by creating a patch cache that stores prediction patches independently from the reference picture cache. This allows the system to maintain a limited number of reference frames while supplementing them with additional prediction patches from multiple sources (evicted reference pictures, predefined image content, prior session data), thereby improving prediction accuracy without proportionally increasing memory usage.
Solution Approach 2:
The system dynamically changes the parameter of prediction source diversity by collecting patches from multiple sources and adjusting the composition of the patch cache based on current coding needs. This allows adaptation to different temporal correlation scenarios while maintaining controlled memory usage through selective patch retention.
2Productivity
If reference frames are evicted from the reference picture cache to control memory, then memory management is efficient, but prediction opportunities are lost
Solution Approach 1:
The patent extracts prediction patches from evicted reference pictures before they are completely removed from memory. When reference pictures are evicted from the reference picture cache, their pixel data is captured and stored as prediction patches in the patch cache, thereby recovering potential prediction value that would otherwise be lost.
Solution Approach 2:
The system implements a discard and recover mechanism where reference pictures are discarded from the reference picture cache to maintain memory efficiency, but their essential prediction content is recovered and preserved in the patch cache for future use, thus maintaining prediction availability without compromising memory management.
3Reliability
If prediction patches are stored in both encoder and decoder caches, then prediction consistency is improved, but communication overhead increases
Solution Approach 1:
The system implements self-service by having both encoder and decoder independently maintain and populate their own patch caches using the same collection rules and sources. This allows both sides to generate consistent prediction references without requiring extensive communication about patch contents, reducing communication overhead while maintaining prediction consistency.
Data Source
AI summary
In a video coding system, an encoder may include a coding engine to predictively code input video, a decoder to reconstruct reference pictures generated by the coding engine, a reference picture cache to store the reconstructed reference pictures, a patch cache to store prediction patches generated from other sources, and a prediction search unit to search among the reference picture cache and the patch cache to generate prediction references for use by the coding engine while coding input video. The prediction patches may be assembled from a variety of sources including: predefined image content, reference pictures being evicted from the reference picture cache, image content of prior coding sessions and image data stored by applications on a common terminal where the encoder resides. A decoder may store prediction patches in its own patch cache for synchronous decoding.