Video Decoder Just-in-Time Post-Processing for Memory Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As video codecs advance to support higher resolutions and frame rates, the memory requirements for storing reconstructed and processed video frames become significant, especially in devices with limited space, leading to a need for improved memory management in video coding hardware architectures.
Innovation Solution
The proposed solution involves separating decoding operations from post-processing operations, storing only reconstructed frames in memory until an output indication is received, and applying post-processing just-in-time to generate processed frames, thereby reducing memory usage and bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If both reconstructed frames and processed frames are stored in memory simultaneously, then video quality and processing requirements are met, but memory usage becomes excessive
Solution Approach 1:
The patent extracts the post-processing operation from the decoding pipeline by introducing a separate post-processing unit that operates on decoded frames only when needed for output. This separates the memory-intensive storage of reconstructed frames from the temporary processing of processed frames, allowing the system to store only one reconstructed frame at a time while maintaining video quality through just-in-time post-processing.
2Ease of operation
If decoded frames are stored in memory for post-processing, then processing can be performed, but memory bandwidth requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-decoding video frames and storing them in memory before they are needed for output. The post-processing unit then retrieves these pre-decoded frames and applies processing operations only when an output indication is received. This separates the decoding phase from the processing phase, reducing memory bandwidth requirements during processing while maintaining processing capability.
3Measurement precision
If high-resolution and high-frame-rate video is processed, then video quality is improved, but memory requirements increase significantly
Solution Approach 1:
The patent introduces dynamic frame management where the system maintains a single reconstructed frame in memory at any given time, dynamically retrieving and processing frames only when output is indicated. This dynamic approach allows the system to handle high-resolution and high-frame-rate video by processing frames on-demand rather than maintaining all frames in memory simultaneously, thus reducing memory requirements while preserving video quality.
Data Source
AI summary
Systems and techniques are provided for decoding video data and/or processing video data. In some examples, a codec system decodes an encoded video frame to generate a decoded video frame, and stores the decoded video frame in a memory. In response to an indication that a processed video frame is to be output, the codec system retrieves the decoded video frame from the memory, processes the decoded video frame to generate the processed video frame, and outputs the processed video frame.