Video Subpicture Partitioning for CTU-Aligned Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as HEVC and VVC, face challenges in efficiently partitioning and encoding/decoding video pictures into subpictures, particularly in managing the number of subpictures and aligning their boundaries with Coding Tree Units (CTUs) to optimize processing and reduce computational complexity.
Innovation Solution
The proposed method involves determining the number of subpictures for a current picture based on allowed maximum slices, partitioning the picture into subpictures aligned with CTU grids, and encoding/decoding each subpicture, with syntax elements like sps_num_subpics_minus1 and pps_rect_slice_flag controlling the subpicture layout and slice structure, allowing for efficient video bitstream generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of subpictures is increased to improve processing parallelism and reduce latency, then video processing efficiency is improved, but device complexity and computational overhead increase
Solution Approach 1:
The picture is divided into multiple subpictures, each containing one or more slices. This segmentation allows parallel processing of different subpictures, improving video processing efficiency and reducing latency while maintaining manageable complexity through structured organization
Solution Approach 2:
The number of subpictures is made dynamically configurable through the sps_num_subpics_minus1 syntax element, allowing the system to adapt the degree of parallelism based on processing capabilities and performance requirements, thus balancing productivity improvement against device complexity
2Ease of operation
If subpicture boundaries are aligned with CTU grids to simplify processing, then ease of operation is improved, but manufacturing precision of boundary alignment is constrained
Solution Approach 1:
The picture is segmented into CTUs (Coding Tree Units) which serve as the basic grid for further partitioning. Subpicture boundaries are explicitly aligned with CTU boundaries, simplifying processing by ensuring that all coding operations occur within well-defined rectangular regions without cross-boundary dependencies
Solution Approach 2:
The CTU size is configurable through sps_log2_ctu_size_minus5 syntax element, allowing the system to adjust the grid resolution. This parameter change enables flexibility in balancing processing simplicity against alignment precision requirements for different application scenarios
3Device complexity
If the maximum number of slices per picture is limited to reduce complexity, then device complexity is reduced, but productivity of video encoding/decoding is constrained
Solution Approach 1:
By introducing subpictures as an intermediate structure between slices and the whole picture, the system can have multiple slices per subpicture and multiple subpictures per picture. This hierarchical segmentation allows the maximum number of slices to be distributed across multiple subpictures, effectively increasing total slice capacity while maintaining manageable complexity within each subpicture
Solution Approach 2:
The patent adds a new dimension to the hierarchy by inserting subpictures between the picture level and slice level. This creates a two-level partitioning structure (subpictures × slices) that increases productivity by allowing more concurrent processing units while keeping individual slice complexity manageable
Data Source
AI summary
Video processing methods and apparatuses include receiving input video data associated with a current picture composed of multiple Coding Tree Units (CTUs) for encoding or decoding, determining a number of subpictures, partitioning the current picture into one or more subpictures, and encoding or decoding each subpicture in the current picture. Each subpicture contains multiple complete CTUs and boundaries of each subpicture are aligned with grids of the current picture in units of CTUs. The number of subpictures in the current picture is limited by an allowed maximum number of slices.