Video Compression Using Feature-Based Reference Frame Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video compression, the time-consuming search for the best reference frame in a large memory of reference frames hinders efficient encoding and decoding, especially in applications like video conferencing where periodic IDR-frames are not necessary, leading to increased computational load and bitrate.
Innovation Solution
Classifying reference frames based on features allows for a subset of frames to be searched during motion prediction, reducing the search time and computational load by using feature-based classification and motion search within a smaller subset of frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a large number of reference frames are stored in long-term reference memory, then video coding efficiency and quality are improved, but search time and computational load increase
Solution Approach 1:
The patent segments the large reference frame memory into multiple subsets organized by scene characteristics (e.g., indoor, outdoor, close-up, wide-angle). Instead of searching the entire memory, the encoder identifies the current scene type and searches only the corresponding subset, dramatically reducing search time while maintaining access to relevant reference frames for high-quality coding.
Solution Approach 2:
The patent performs preliminary organization of reference frames into scene-based subsets during the encoding process. By pre-classifying and storing reference frames according to scene characteristics, the system prepares the search structure in advance, allowing for rapid retrieval without exhaustive searching when encoding subsequent frames.
2Loss of energy
If periodic IDR-frames are eliminated in video conferencing, then bandwidth requirements are reduced, but decoder memory management becomes more complex
Solution Approach 1:
The patent implements a universal long-term reference memory structure that handles both periodic IDR-frame scenarios and continuous P-frame scenarios. The same memory architecture and scene-based organization mechanism work for different video conferencing conditions, eliminating the need for separate memory management strategies while reducing bandwidth through efficient P-frame coding.
Solution Approach 2:
The patent introduces scene characteristics as an intermediary layer between the video content and the reference frame memory. This intermediary enables intelligent routing and organization of reference frames based on scene type, simplifying memory management by providing a systematic way to handle reference frames without requiring complex periodic flushing operations.
Data Source
AI summary
Particular embodiments generally relate to video compression. In one embodiment, a store of reference frames is provided in memory. The reference frames may be classified based on a plurality of classifiers. The classifiers may correspond to features that are found in the reference frame. A frame to encode is then received. The frame is analyzed to determine features found in the frame. As macroblocks in the frame are encoded, a macroblock is analyzed to determine which feature may be included in the macroblock. The feature is used to determine a classifier, which is used to determine a subset of the reference frames. The subset is then searched to determine a reference frame for the macroblock.