Real-time HDR to SDR Reshaping Using Sliding Window Statistics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional real-time single-layer backward compatible codecs face challenges in efficiently generating reversible standard dynamic range (SDR) from high dynamic range (HDR) images, leading to computational intensity and delays, especially in handling highly dynamic scenes and user adjustments.
Innovation Solution
A real-time reshaping architecture and algorithm that utilize statistical and central tendency sliding windows to determine forward and backward reshaping functions, enabling efficient conversion of HDR images to SDR while maintaining temporal stability and reversibility, using techniques like multiple regression and dynamic tone mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional real-time single-layer backward compatible codecs generate reversible SDR from HDR images using approximation techniques, then backward compatibility and reversibility are maintained, but computational intensity increases and processing delays occur
Solution Approach 1:
The patent applies dynamics by making the reshaping function adaptive rather than static. The forward and backward reshaping functions are dynamically determined based on statistical and central tendency sliding windows that analyze temporal characteristics of video sequences. This allows the system to optimize the reshaping process in real-time based on actual content, reducing processing delays while maintaining reversibility.
Solution Approach 2:
The patent employs preliminary action by pre-computing and storing statistical characteristics and central tendency values in sliding windows before the actual reshaping operation. These pre-processed statistical data are then used to quickly determine the reshaping functions without performing complex calculations during real-time encoding, thereby reducing processing delay while maintaining accuracy.
2Adaptability or versatility
If conventional codecs use approximation to generate SDR from HDR, then backward compatibility is achieved, but computational intensity increases
Solution Approach 1:
The patent applies parameter changes by transforming the reshaping process from a fixed approximation method to a dynamic parameter-based approach. Statistical parameters and central tendency values are extracted from video sequences and used to determine optimal reshaping functions. This allows the system to adapt the reshaping parameters to specific content characteristics, achieving backward compatibility with reduced computational intensity by avoiding unnecessary approximations.
Solution Approach 2:
The patent implements self-service by enabling the system to automatically determine optimal reshaping functions based on the statistical characteristics of the input video sequence itself. The sliding window mechanisms allow the codec to self-optimize its reshaping parameters based on actual content, eliminating the need for heavy external approximation algorithms while maintaining backward compatibility.
3Productivity
If real-time reshaping is implemented with statistical and central tendency sliding windows, then processing speed improves, but system complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the video sequence into sliding windows that contain a manageable number of frames. Statistical and central tendency calculations are performed on these segmented windows rather than on entire sequences, which reduces the computational burden per frame and enables real-time processing. This segmentation approach increases productivity while keeping the complexity of individual processing units manageable.
Solution Approach 2:
The patent employs partial action by calculating and storing only the necessary statistical parameters and central tendency values within sliding windows, rather than processing every possible parameter combination. This selective approach focuses computational resources on the most critical characteristics needed for reshaping, improving processing speed while avoiding the excessive complexity of complete analysis.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Real-time forward reshaping, comprising selecting a statistical sliding window that indexes with the current frame, having also, a look-back frame and a look-ahead frame, determining whether they are part of the current scene, determining a noise parameter, a luma transfer function and a luma forward reshaping function based on the luma transfer function and the noise parameter within the current scene, selecting a central tendency sliding window of the current frame and the look-back frame within the current scene, and determining a central tendency luma forward reshaping function. The chroma reshaping comprises analyzing statistics for the extended dynamic range (EDR) weights and EDR upper bounds, mapping these to standard dynamic range (SDR) weights and SDR upper bounds based on the central tendency luma forward reshaping function, determining a chroma content-dependent polynomial and a central tendency chroma forward reshaping polynomial and generating chroma MMR coefficients.