Video Decoder Tile Partitioning for Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video encoding and decoding methods, particularly in AVC/H.264 standards, face limitations in parallel processing efficiency due to rigid block partitioning schemes like slices and slice groups, which impose unnecessary restrictions on decoder implementation and reduce flexibility in exploiting dependencies between blocks, leading to suboptimal compression efficiency and parallel processing capabilities.
Innovation Solution
The introduction of tiles as a group of NxM adjacent blocks with dependency breaks at tile boundaries allows for flexible partitioning independent of transmission order, enabling efficient parallel processing across multiple cores and minimizing the need for additional headers, thus decoupling encoding from transmission order and allowing for better exploitation of horizontal and vertical dependencies in images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If rigid block partitioning schemes like slices and slice groups are used, then decoder implementation is constrained, but parallel processing efficiency is reduced
Solution Approach 1:
The video frame is divided into multiple tiles, where each tile is further segmented into prediction blocks and transform blocks. This hierarchical segmentation allows different levels of parallel processing: tiles can be processed in parallel across multiple threads, while within each tile, prediction blocks can be independently decoded. This resolves the contradiction by providing both decoder implementation flexibility through configurable tile boundaries and improved parallel processing efficiency through fine-grained parallelism.
Solution Approach 2:
The patent introduces a two-dimensional tile grid structure overlaying the traditional one-dimensional slice organization. This dimensional change allows parallel processing to occur both horizontally and vertically across tile boundaries, while maintaining flexible decoder implementation through configurable tile dimensions. The 2D tile structure enables more efficient exploitation of spatial locality and memory access patterns compared to conventional slice-based approaches.
2Adaptability or versatility
If conventional slice-based partitioning is used, then transmission order is fixed, but compression efficiency is suboptimal
Solution Approach 1:
The patent enables dynamic configuration of tile boundaries and dimensions based on content characteristics and processing requirements. Tile widths and heights can be adjusted to optimize compression efficiency for different video content types, while maintaining flexible transmission ordering. This dynamic adaptability allows the system to achieve better compression efficiency by adapting partitioning strategies to specific content patterns, resolving the contradiction between partitioning flexibility and compression efficiency.
3Device complexity
If rigid block partitioning is used, then dependency management is simplified, but parallel processing capability is limited
Solution Approach 1:
By segmenting the video frame into independent tiles with clearly defined boundaries, the patent simplifies dependency management at the tile level while enabling fine-grained parallel processing within each tile. Prediction blocks within a tile can be processed independently once reference data from previous frames is available, allowing multiple threads to work simultaneously on different tiles and prediction blocks. This hierarchical segmentation resolves the contradiction by maintaining simple inter-tile dependencies while enabling complex intra-tile parallelism.
Data Source
Figure 1
Figure 2
Figure 3a
AI summary
A video decoder, method and computer program product allow for processing of a video frame encoded in rectangular tiles. An interface receives a bit stream in tile order within a video frame that was encoded into rectangular tiles. A processor decodes the video frame while respecting dependency breaks at tile boundaries; the rectangular tiles include an integer number of two-dimensional blocks of pixels. A tile shape is defined by N x M two- dimensional blocks of pixels, respective values of N and M need not be identical for each of the rectangular tiles, and information regarding tile shape for each tile being conveyed from an encoder to the decoder. The decoder determines N and M for each tile from the information, and tiles have dependency breaks therebetween.