Region Packing in Coded Video With ROI Reconstruction Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards lack an interoperable solution to pack regions of interest (ROIs) within a smaller picture coded in a video bitstream, which enables reconstruction of target pictures using only the information contained within the bitstream.
Innovation Solution
An SEI message is introduced to signal the locations and sizes of ROIs within a coded picture, allowing for flexible resampling and reconstruction of target pictures, with optional signaling of corresponding region locations and background inclusion, using region IDs to determine precedence and resampling ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire picture is coded at full resolution, then the reconstruction quality is maintained, but the bitrate and processing complexity increase
Solution Approach 1:
The picture is divided into multiple regions of interest (ROIs) and non-ROI areas. Each ROI is coded separately with full resolution while non-ROI areas are downsampled. This segmentation allows selective high-quality coding only where needed, reducing overall bitrate while maintaining reconstruction quality in critical regions.
Solution Approach 2:
Different regions of the picture are assigned different coding qualities. ROIs receive full resolution coding for high quality reconstruction, while non-ROI regions are coded at lower resolution. This local quality differentiation optimizes the bitrate-quality tradeoff by concentrating bits where they provide maximum benefit.
2Measurement precision
If the entire picture is coded at full resolution, then the reconstruction quality is maintained, but the processing complexity increases
Solution Approach 1:
The processing workload is segmented by identifying and separating ROI regions from non-ROI regions. Only the ROI regions undergo full resolution processing, while non-ROI regions are processed at lower resolution. This segmentation reduces overall processing complexity by limiting high-complexity operations to only necessary regions.
Solution Approach 2:
Processing complexity is distributed non-uniformly across the picture. High complexity processing is applied locally to ROI regions where full resolution is needed, while lower complexity processing is applied to non-ROI regions. This local differentiation reduces total processing complexity while maintaining quality where required.
3Quantity of substance
If regions are packed into a smaller picture, then the bitrate is reduced, but the ability to reconstruct the original picture is lost
Solution Approach 1:
Metadata describing the ROI regions, their locations, and their mapping to the original picture is prepared and transmitted alongside the coded packed picture. This preliminary action preserves the necessary information to reconstruct the original picture by guiding the decoder on how to place and scale the ROI regions back to their original positions and dimensions.
Solution Approach 2:
Metadata acts as an intermediary between the packed coded picture and the original picture reconstruction. This metadata contains the mapping information, region definitions, and scaling factors that enable the decoder to transform the packed picture back into the original picture structure, preventing information loss during the packing process.
4Productivity
If region packing is implemented, then coding efficiency is improved, but interoperability between different systems is reduced
Solution Approach 1:
The region packing mechanism is designed with universal metadata structures and standardized region definitions that can be understood by different coding systems. The metadata format serves multiple functions: it describes ROI locations, scaling relationships, and reconstruction parameters, enabling interoperability between encoders and decoders from different vendors or systems while maintaining coding efficiency benefits.
Data Source
AI summary
An example apparatus includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine rectangular regions of a picture; pack the rectangular regions of the picture into a packed picture; code the packed picture; signal the coded packed picture; and signal metadata describing the rectangular regions packed into the coded packed picture.