Adaptive HDR Video Composition with Dynamic Color Space Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video composition systems face challenges in effectively mixing High Dynamic Range (HDR) and Wide Color Gamut (WCG) video components with Standard Dynamic Range (SDR) and Standard Color Gamut (SCG) graphical/user interface elements, as they struggle to adapt color information and luminance to varying display conditions, leading to suboptimal visual output.
Innovation Solution
A method and system that adaptively mix HDR video components with SDR/SCG and/or HDR/WCG graphics/UI components by converting video data to a high dynamic range or wide gamut color space, performing statistical analysis, and assembling frames based on display characteristics, ambient conditions, and user commands to optimize the color space for the display device, thereby improving visibility and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video data and graphics/UI elements with disparate dynamic ranges and color spaces are mixed using existing video compositing systems, then the mixing process becomes complex and computationally intensive, but the visual output quality deteriorates due to inability to adapt color information and luminance to varying display conditions
Solution Approach 1:
The patent introduces an intermediate color space conversion step where HDR video data is converted to the display's color space (sRGB for SDR displays or wider gamut for HDR displays) before compositing with UI elements. This intermediary conversion acts as a mediator that harmonizes the disparate color spaces and dynamic ranges, enabling accurate color mixing without excessive computational complexity. The conversion uses transfer functions and gamma correction to map luminance and color values appropriately.
Solution Approach 2:
The system dynamically changes color space parameters (gamma, transfer function, white point, primary chromaticities) based on the display device's capabilities and ambient conditions. By adjusting these parameters adaptively, the system maintains color accuracy across different display types without requiring a completely different compositing pipeline for each scenario.
2Device complexity
If UI elements are rendered in the same color space as video (sRGB), then the compositing process simplifies, but HDR video content loses its dynamic range benefits and visual quality
Solution Approach 1:
The system dynamically determines the appropriate color space for UI elements based on the video content's dynamic range and the display's capabilities. Rather than statically rendering all UI in sRGB, the system adapts UI rendering to match the video's color space characteristics when appropriate, maintaining both simplicity and HDR quality through conditional logic that assesses content and display properties.
3Manufacturing precision
If video data is processed with full HDR and WCG support throughout the pipeline, then color accuracy improves, but power consumption and processing overhead increase
Solution Approach 1:
The patent segments the video processing pipeline into distinct stages: HDR-to-display-color-space conversion, compositing, and final output. By segmenting the processing, the system applies full HDR accuracy only where necessary (during the conversion and initial compositing stages) then transitions to the display's native color space for subsequent processing and output. This reduces overall computational overhead while maintaining color accuracy in critical operations.
4Adaptability or versatility
If the compositing system adapts to varying ambient conditions and display characteristics, then visual output optimization improves, but system complexity and computational requirements increase
Solution Approach 1:
The system incorporates feedback mechanisms that detect ambient light conditions and display characteristics, then use this information to adjust color space conversion parameters and luminance mapping. The feedback loop continuously monitors display output and environmental conditions, automatically tuning the compositing parameters to optimize visual quality without requiring manual intervention or complex configuration.
Data Source
AI summary
A method and system for adaptively mixing video components with graphics/UI components, where the video components and graphics/UI components may be of different types, e.g., different dynamic ranges (such as HDR, SDR) and/or color gamut (such as WCG). The mixing may result in a frame optimized for a display device's color space, ambient conditions, viewing distance and angle, etc., while accounting for characteristics of the received data. The methods include receiving video and graphics/UI elements, converting the video to HDR and/or WCG, performing statistical analysis of received data and any additional applicable rendering information, and assembling a video frame with the received components based on the statistical analysis. The assembled video frame may be matched to a color space and displayed. The video data and graphics/UI data may have or be adjusted to have the same white point and/or primaries.