Decoder-Side ROI Video Processing with Reliability Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current digital video decoding technologies face challenges in efficiently identifying and processing region-of-interest (ROI) portions of video frames, particularly in determining ROI reliability and applying appropriate decoding and error concealment techniques without pre-existing assistance information.
Innovation Solution
A method and device for decoder-side ROI video processing that determines the availability of ROI assistance information, using bitstream domain or pixel domain analysis to identify and refine ROIs, and applies preferential decoding, post-processing, and error concealment based on reliability, with different techniques for intra-coded and inter-coded data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If decoder-side ROI processing is applied without encoder assistance, then ROI identification can be performed autonomously at the decoder, but the complexity of the decoding process increases
Solution Approach 1:
The decoding process is segmented into multiple stages: initial ROI extraction from bitstream, reliability analysis stage, and refined ROI identification stage. This segmentation allows the system to perform autonomous ROI identification while managing complexity by breaking down the task into manageable steps with different complexity levels for different frame types.
Solution Approach 2:
The system dynamically adjusts the ROI processing approach based on the frame type (intra-coded vs. inter-coded) and reliability analysis results. For high-reliability frames, simpler extraction methods are used; for low-reliability frames, more complex analysis is applied. This dynamic adaptation resolves the contradiction by making complexity conditional rather than universal.
2Productivity
If bitstream domain ROI extraction is used, then processing efficiency is improved, but ROI identification reliability may be insufficient without encoder assistance
Solution Approach 1:
A preliminary reliability analysis is performed on the extracted ROI data before final ROI identification is completed. This preliminary action assesses the quality and reliability of the bitstream domain extraction, allowing the system to determine whether additional processing is needed. This resolves the contradiction by adding a quality check that maintains efficiency while improving reliability assurance.
Solution Approach 2:
The system implements feedback through reliability analysis that evaluates the quality of ROI extraction from the bitstream. Based on this feedback, the system determines whether the extracted ROI is reliable enough for direct use or whether additional processing and refinement are required. This feedback mechanism ensures reliability without completely sacrificing processing efficiency.
3Manufacturing precision
If preferential decoding and error concealment are applied to ROI areas, then visual quality of ROIs is enhanced, but computational complexity and power consumption increase
Solution Approach 1:
Preferential decoding and error concealment are applied selectively only to identified ROI areas rather than uniformly across the entire frame. This local quality approach enhances visual quality where it matters most (in the ROI) while avoiding unnecessary processing in non-ROI areas, thereby reducing overall computational complexity and power consumption while maintaining enhanced ROI quality.
4Measurement precision
If multiple ROI extraction processes are implemented for different frame types, then ROI identification accuracy is improved, but device complexity increases
Solution Approach 1:
The system dynamically selects the appropriate ROI extraction process based on the frame type (intra-coded or inter-coded). Rather than implementing all processes simultaneously in a static structure, the system activates only the relevant process for each frame type, improving accuracy when needed while managing complexity through conditional execution rather than permanent structural complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure is directed to decoder-side region-of-interest (ROI) video processing. A video decoder determines whether ROI assistance information is available. If not, the decoder defaults to decoder-side ROI processing. The decoder-side ROI processing may estimate the reliability of ROI extraction in the bitstream domain. If ROI reliability is favorable, the decoder applies bitstream domain ROI extraction. If ROI reliability is unfavorable, the decoder applies pixel domain ROI extraction. The decoder may apply different ROI extraction processes for intra-coded (I) and inter-coded (P or B) data. The decoder may use color-based ROI generation for intra-coded data, and coded block pattern (CBP)-based ROI generation for inter-coded data. ROI refinement may involve shape-based refinement for intra-coded data, and motion- and color-based refinement for inter-coded data.