Region Packing Video Encoding with SEI-Based ROI Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies are inefficient for applications where video content is primarily consumed by machines, as they do not effectively compress and decode regions of interest, leading to unnecessary data transmission and processing.
Innovation Solution
A video encoding and decoding system that detects regions of interest, tightly packs them into contiguous areas, and transmits region parameters as Supplemental Enhancement Information (SEI), allowing for efficient compression and reconstruction at the decoder.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional video coding is used to compress all video frames, then compression is achieved, but machine task performance deteriorates due to loss of region-specific information
Solution Approach 1:
The video frame is segmented into regions of interest (ROIs) and non-ROI areas. The encoder identifies ROIs based on object detection algorithms and processes only these regions with high precision, while applying standard compression to the remaining areas. This segmentation allows the system to reduce overall data transmission volume while maintaining high measurement precision for machine tasks that require detailed analysis of specific regions.
Solution Approach 2:
Different quality levels are applied to different regions of the video frame. Regions of interest receive high-quality encoding with preserved spatial and spectral information, while non-ROI regions are compressed more aggressively. This local quality differentiation ensures that machine tasks receiving the compressed video data can achieve high performance on critical regions without the computational burden of processing entire high-quality frames.
2Loss of information
If all video data is transmitted to ensure complete information, then information completeness is maintained, but data transmission volume increases unnecessarily
Solution Approach 1:
The encoder extracts only the essential information from video frames by identifying and prioritizing regions of interest. Object detection algorithms extract salient objects and their spatial-spectral characteristics, which are then encoded with high fidelity. Background and non-critical regions are either omitted or compressed heavily, significantly reducing data transmission volume while maintaining information completeness for machine task requirements.
Solution Approach 2:
Instead of processing the entire video frame uniformly, the system applies partial processing only to necessary regions. The encoder performs detailed analysis and high-quality encoding on a subset of pixels corresponding to ROIs, while applying minimal or no processing to the remainder. This partial action approach reduces data transmission volume without sacrificing the information completeness needed for machine vision tasks.
3Productivity
If region packing is implemented to compress only regions of interest, then data compression efficiency improves, but system complexity increases
Solution Approach 1:
The encoder integrates multiple functions into a unified processing pipeline: object detection, region identification, spatial-spectral feature extraction, and selective compression are all performed within a single encoding framework. This multi-functionality approach improves compression efficiency by processing only relevant regions while managing system complexity through integrated architecture rather than separate independent modules.
Solution Approach 2:
The system performs preliminary object detection and region identification before the actual compression process. By pre-segmenting the video frame into ROIs and non-ROIs based on detected objects, the encoder can subsequently apply compression operations only to the identified regions. This preliminary action streamlines the compression process, improving efficiency while keeping complexity manageable through automated detection algorithms.
Data Source
AI summary
Systems and methods for video coding and decoding using region packing are provided. At an encoder, an encoded bitstream is generated having packed frames with a plurality of regions of interest therein. At least a portion of the region parameters to reconstruct the packed frame are encoded as Supplemental Enhancement Information. At the decoder, the encoded bitstream is decoded and parameters sufficient to place the regions within a reconstructed frame are extracted, including parameters extracted from Supplemental Enhancement Information. A reconstructed frame is generated which substantially maintains the spatial relationship and size of regions of interest in the original video frame.