Multi-Sensor HDR Video Merging for Real-Time Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image and video capture devices struggle to reproduce high dynamic range (HDR) due to limitations in dynamic range, especially when capturing moving subjects, requiring post-processing that is challenging for video production.
Innovation Solution
A method and apparatus that processes pixel streams from multiple sensors in a frame-independent manner, using a pipeline with synchronization and kernel operations to produce real-time HDR video by merging pixel values from sensors with different exposures, correcting for spectral differences, and replacing saturated pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional sensors are used to capture images, then the device complexity is low, but the dynamic range is limited and cannot reproduce high dynamic range scenes
Solution Approach 1:
The patent divides the imaging function into multiple specialized sensors, each optimized for specific exposure ranges. Instead of using a single sensor to capture the entire dynamic range, the system segments the capture task across multiple sensors with different exposure settings, allowing each sensor to operate within its optimal range while collectively achieving high dynamic range capture.
Solution Approach 2:
The patent combines data from multiple sensors with different exposures through a merging process. The system integrates information from sensors capturing different exposure levels, selectively combining their outputs to reconstruct the full high dynamic range scene. This merging of multiple data streams enables the system to achieve dynamic range capabilities beyond what any single sensor can provide.
2Measurement precision
If post-processing is used to create HDR images from multiple sensors, then the dynamic range is extended, but the processing time increases and real-time video production becomes challenging
Solution Approach 1:
The patent performs preliminary actions by capturing multiple exposure levels simultaneously during the imaging process itself, rather than acquiring a single image and then processing it later. The multiple sensors capture different exposure ranges at the same moment, pre-preparing the data needed for HDR reconstruction before any post-processing occurs. This eliminates the time delay associated with sequential capture methods.
Solution Approach 2:
The patent establishes continuity of useful action by implementing a real-time processing pipeline that continuously merges data from multiple sensors as they stream in. Instead of batch processing complete frames, the system continuously integrates pixel data from multiple exposure streams, maintaining an uninterrupted HDR video output. This continuous processing eliminates idle time between capture and processing operations.
3Measurement precision
If multiple sensors are used to capture HDR video, then the dynamic range is extended, but the device complexity and data processing requirements increase
Solution Approach 1:
The patent implements universality by designing a processing pipeline that handles multiple sensor data streams through a unified merging architecture. The same merging operations and processing logic are applied regardless of which sensors are used or how many sensors are present. This multi-functional approach allows the system to accommodate different sensor configurations while maintaining a consistent processing framework, reducing overall system complexity.
4Ease of operation
If frame-based processing is used for multiple sensor images, then the processing is straightforward, but the processing speed decreases and real-time production is compromised
Solution Approach 1:
The patent segments the processing task from the frame structure by treating pixel data as independent streams rather than bound units. Instead of processing complete frames as atomic units, the system divides the processing into individual pixel operations that can be handled independently. This segmentation allows processing to proceed at the pixel level without waiting for complete frame data, dramatically increasing processing speed while maintaining simplicity through consistent per-pixel operations.
Data Source
AI summary
Systems and methods of the invention merge information from multiple image sensors to provide a high dynamic range (HDR) video. The present invention provides for real-time HDR video production using multiple sensors and pipeline processing techniques. According to the invention, multiple sensors with different exposures each produces an ordered stream of frame-independent pixel values. The pixel values are streamed through a pipeline on a processing device. The pipeline includes a kernel operation that identifies saturated ones of the pixel values. The streams of pixel values are merged to produce an HDR video.


