Multi-Sensor HDR Video Merging for Real-Time Capture

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional image and video capture devices struggle to reproduce high dynamic range (HDR) due to limitations in dynamic range, especially when capturing moving subjects, requiring post-processing that is challenging for video production.

Innovation Solution

A method and apparatus that processes pixel streams from multiple sensors in a frame-independent manner, using a pipeline with synchronization and kernel operations to produce real-time HDR video by merging pixel values from sensors with different exposures, correcting for spectral differences, and replacing saturated pixels.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional sensors are used to capture images, then the device complexity is low, but the dynamic range is limited and cannot reproduce high dynamic range scenes

Engineering Contradiction:
Improvedynamic rangeVSAvoiddevice complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent divides the imaging function into multiple specialized sensors, each optimized for specific exposure ranges. Instead of using a single sensor to capture the entire dynamic range, the system segments the capture task across multiple sensors with different exposure settings, allowing each sensor to operate within its optimal range while collectively achieving high dynamic range capture.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent combines data from multiple sensors with different exposures through a merging process. The system integrates information from sensors capturing different exposure levels, selectively combining their outputs to reconstruct the full high dynamic range scene. This merging of multiple data streams enables the system to achieve dynamic range capabilities beyond what any single sensor can provide.

Inventive Principle:
Principle #5Merging (Combining)

2Measurement precision

If post-processing is used to create HDR images from multiple sensors, then the dynamic range is extended, but the processing time increases and real-time video production becomes challenging

Engineering Contradiction:
Improvedynamic rangeVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by capturing multiple exposure levels simultaneously during the imaging process itself, rather than acquiring a single image and then processing it later. The multiple sensors capture different exposure ranges at the same moment, pre-preparing the data needed for HDR reconstruction before any post-processing occurs. This eliminates the time delay associated with sequential capture methods.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent establishes continuity of useful action by implementing a real-time processing pipeline that continuously merges data from multiple sensors as they stream in. Instead of batch processing complete frames, the system continuously integrates pixel data from multiple exposure streams, maintaining an uninterrupted HDR video output. This continuous processing eliminates idle time between capture and processing operations.

Inventive Principle:
Principle #20Continuity of useful action

3Measurement precision

If multiple sensors are used to capture HDR video, then the dynamic range is extended, but the device complexity and data processing requirements increase

Engineering Contradiction:
Improvedynamic rangeVSAvoiddevice complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent implements universality by designing a processing pipeline that handles multiple sensor data streams through a unified merging architecture. The same merging operations and processing logic are applied regardless of which sensors are used or how many sensors are present. This multi-functional approach allows the system to accommodate different sensor configurations while maintaining a consistent processing framework, reducing overall system complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

4Ease of operation

If frame-based processing is used for multiple sensor images, then the processing is straightforward, but the processing speed decreases and real-time production is compromised

Engineering Contradiction:
Improveprocessing simplicityVSAvoidprocessing speed
Core Design Contradiction:
Ease of operationVSProductivity

Solution Approach 1:

The patent segments the processing task from the frame structure by treating pixel data as independent streams rather than bound units. Instead of processing complete frames as atomic units, the system divides the processing into individual pixel operations that can be handled independently. This segmentation allows processing to proceed at the pixel level without waiting for complete frame data, dramatically increasing processing speed while maintaining simplicity through consistent per-pixel operations.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS20250247491A1Devices and methods for high dynamic range video
Publication Date: 2025.07.31 CONTRAST INC
  • US20250247491A1 patent drawing
  • US20250247491A1 patent drawing
  • US20250247491A1 patent drawing

AI summary

Systems and methods of the invention merge information from multiple image sensors to provide a high dynamic range (HDR) video. The present invention provides for real-time HDR video production using multiple sensors and pipeline processing techniques. According to the invention, multiple sensors with different exposures each produces an ordered stream of frame-independent pixel values. The pixel values are streamed through a pipeline on a processing device. The pipeline includes a kernel operation that identifies saturated ones of the pixel values. The streams of pixel values are merged to produce an HDR video.