Real-Time HDR Video Processing via Pipelined Image Alignment and Tone Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital cameras capture images as low dynamic range (LDR), limiting the dynamic range of colors and resulting in lost details in dark or bright areas due to clipping, which hinders the preservation of image details and contrast ratios.
Innovation Solution
A frame-pipelined approach that combines multiple input image frames captured with different exposure settings to create a single high dynamic range (HDR) output frame in real time, using a pipelined architecture that includes image alignment, mixing, and tone mapping stages to enhance details and recover lost information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple image frames with different exposure settings are captured and combined to create HDR output, then the dynamic range and image details are improved, but the processing complexity and computational requirements increase
Solution Approach 1:
The patent divides the HDR processing into distinct pipeline stages: image alignment, image mixing, and tone mapping. Each stage processes images independently and passes results to the next stage, allowing parallel processing and reducing overall complexity. The image alignment stage handles registration, the mixing stage combines multiple exposures, and the tone mapping stage adjusts the final output, with each stage operating autonomously.
Solution Approach 2:
The patent performs image alignment and registration as preliminary actions before the actual HDR mixing process. By pre-aligning the captured frames using feature detection and geometric transformation, the system prepares the images in advance, reducing the computational burden during the time-critical mixing and tone mapping stages.
2Loss of information
If real-time HDR processing is implemented to preserve image details, then the image quality is improved, but the processing time and computational load increase
Solution Approach 1:
The patent implements a continuous pipelined processing architecture where image capture, alignment, mixing, and tone mapping occur continuously without interruption. While one set of images is being processed in the mixing stage, the next set is being aligned, and the previous set is being tone-mapped, ensuring continuous HDR video output without batch processing delays.
Solution Approach 2:
By segmenting the processing into parallel stages that can operate simultaneously on different image sets, the system achieves real-time processing. The pipeline architecture allows multiple images to be processed at different stages concurrently, maintaining real-time throughput while performing computationally intensive HDR operations.
3Device complexity
If conventional LDR capture is used, then the device complexity is reduced, but the dynamic range and contrast ratio are limited
Solution Approach 1:
The patent changes the exposure parameter by capturing multiple images at different exposure settings (under-exposed, properly exposed, and over-exposed frames). This parameter variation allows the system to capture a wider dynamic range of illumination intensities, preserving details in both dark and bright regions that would be lost in conventional single-exposure LDR capture.
Solution Approach 2:
The patent merges multiple LDR images captured at different exposure settings into a single HDR image. By combining the information from under-exposed, properly exposed, and over-exposed frames through weighted averaging and tone mapping, the system achieves extended dynamic range while using conventional LDR camera hardware.
Data Source
AI summary
Embodiments are directed towards enabling digital cameras to digitally process a captured a Low Dynamic Range image sequence at a real time video rate, and to convert the image sequence into an High Dynamic Range (HDR) image sequence using a pipelined architecture. Two or more image frames are captured using different exposure settings and then combined to form a single HDR output frame in a video sequence. The pipelined architecture operate on adjacent image frames by performing an image alignment, an image mixing, and a tone mapping on the adjacent image frames to generate the HDR image sequence.