Interleaved Sub-block Processing for Video Coding Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In digital video coding, the processing of video data is hindered by data dependencies between pixels, leading to increased latency, decreased throughput, and larger chip area requirements due to idle components waiting for processing results, particularly in intra coding where neighboring pixels must be processed before a video block.
Innovation Solution
The method involves interleaving the processing order of sub-blocks of luminance and color difference blocks, allowing for more efficient component utilization by processing sub-blocks in a pipeline-like manner, reducing idle time and dependencies between luminance sub-blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video blocks are processed in sequential order (first video block completely before second video block), then processing simplicity is maintained, but latency increases and throughput decreases due to idle components waiting for processing results
Solution Approach 1:
The patent divides video blocks into sub-blocks (e.g., 4x4 or 8x8 sub-blocks from a 16x16 video block) and processes them independently in an interleaved manner. This segmentation allows different processing stages to work on different sub-blocks simultaneously, increasing throughput while maintaining manageable processing complexity through modular design
Solution Approach 2:
The patent implements dynamic processing where the processing order is changed from static sequential to dynamic interleaved processing. Processing units can dynamically switch between working on different video blocks based on data availability and processing stage completion, optimizing throughput without excessive complexity
2Loss of time
If components wait for processing results before proceeding to next stage, then processing accuracy is maintained, but latency increases due to idle waiting time
Solution Approach 1:
The patent performs preliminary actions by preparing and buffering data for multiple video blocks before actual processing. Reference blocks and prediction data are pre-computed and stored in buffers, allowing processing units to immediately begin work without waiting for previous blocks to complete, thereby reducing latency while maintaining processing reliability through pre-prepared data
Solution Approach 2:
The patent introduces intermediary buffers and memory structures that decouple the dependency between processing stages. These intermediaries hold intermediate results and allow subsequent processing stages to proceed independently without directly waiting for previous stages to complete, reducing latency while preserving data integrity and processing reliability
3Area of stationary object
If all sub-blocks of one video block are processed completely before moving to the next video block, then processing simplicity is maintained, but chip area increases due to larger buffers needed to hold intermediate results
Solution Approach 1:
By segmenting video blocks into smaller sub-blocks and processing them in an interleaved pattern across multiple video blocks, the patent reduces the amount of intermediate data that needs to be held in buffers at any one time. This segmentation allows reuse of buffer memory across different sub-blocks and video blocks, reducing total chip area requirements
Solution Approach 2:
The patent maintains continuous useful action by keeping processing units actively working on different sub-blocks of different video blocks simultaneously. This continuous processing eliminates idle buffer waiting time and allows smaller buffers to be reused more frequently, reducing the total buffer memory capacity needed on the chip
Data Source
AI summary
Systems and methods for interleaving video sub-blocks in video coding are described herein. In one aspect, an apparatus includes a memory and a video coder. The memory stores a first video block and a second video block. The first video block and the second video block include sub-blocks. The video coder processes a first sub-block of the first video block according to a first process and a second process, and processes a second sub-block of the first video block according to the first process and the second process after processing the first sub-block of the first video block according to the first process and the second process. Further, the video coder processes a first sub-block of the second video block according to the first process before processing the second sub-block of the first video block according to the first process.