Light-Field Image Processing Sensor Saturation Compensation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Light-field image processing technologies face challenges in reducing color artifacts, projection artifacts, and enhancing dynamic range, particularly due to sensor saturation and microlens modulation, which affect the accuracy and quality of captured images.

Innovation Solution

The system processes light-field image data by capturing flat-field images, converting them into modulation images, and applying these on a per-pixel basis to correct for sensor saturation and microlens modulation, using techniques such as demodulation, demosaicing, and automatic white balance to improve image quality and reduce artifacts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If light-field image processing is performed without compensation, then processing speed is maintained, but color artifacts and projection artifacts increase while dynamic range is reduced

Engineering Contradiction:

Improveimage qualityVSAvoidprocessing complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by capturing flat-field images before processing the main light-field image data. These flat-field images are used to pre-compute modulation correction factors that are then applied during the main processing stage, reducing artifacts without requiring complex real-time calculations on the original image data

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary approach by using flat-field images as a mediator between the captured light-field data and the final processed image. The flat-field images serve as reference data that enables compensation for sensor saturation and microlens modulation effects through demodulation processes

Inventive Principle:

Principle #24Intermediary (Mediator)

2Quantity of substance

If sensor saturation occurs during capture, then dynamic range is extended, but color accuracy deteriorates due to saturation artifacts

Engineering Contradiction:

Improvedynamic rangeVSAvoidcolor accuracy

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent implements feedback by using the flat-field images to generate modulation correction factors that are fed back into the processing of the main light-field image. This feedback loop allows the system to identify and correct saturation artifacts by comparing the expected flat-field response with the actual captured data

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent converts the harmful effect of sensor saturation into a beneficial process by using the saturation information from flat-field images to compute correction factors. The saturation artifacts in the flat-field images are processed to create lookup tables that compensate for similar saturation effects in the main image, turning the problem into a correction mechanism

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

3Productivity

If microlens modulation is present, then light-field data is captured, but projection artifacts and color artifacts increase

Engineering Contradiction:

Improvedata capture capabilityVSAvoidprojection artifacts

Core Design Contradiction:

ProductivityVSObject-generated harmful factors

Solution Approach 1:

The patent replaces direct mechanical correction of the microlens array with an optical processing approach. Instead of physically adjusting the microlenses, the system uses digital signal processing techniques including demodulation and lookup table-based compensation to correct the modulation effects introduced by the microlens array in software

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Reliability

If flat-field images are captured and processed, then color artifacts are reduced, but processing time increases

Engineering Contradiction:

Improvecolor accuracyVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs the time-consuming flat-field image capture and modulation factor computation during the calibration phase before actual image processing. This preliminary action separates the heavy computational workload from the real-time processing, allowing fast demodulation and correction during the main imaging operation using pre-computed lookup tables

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS8948545B2Compensating for sensor saturation and microlens modulation during light-field image processing

Publication Date: 2015.02.03 GOOGLE LLC

AI summary

According to various embodiments, the system and method of the present invention process light-field image data so as to reduce color artifacts, reduce projection artifacts, and/or increase dynamic range. These techniques operate, for example, on image data affected by sensor saturation and/or microlens modulation. Flat-field images are captured and converted to modulation images, and then applied on a per-pixel basis, according to techniques described herein.