Per-pixel white balance kernel for mixed illumination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Correcting color casts in photographs captured under mixed lighting conditions is challenging due to varying illumination sources, requiring manual skill and time, especially when each pixel has a different lighting combination.
Innovation Solution
Automatically determining a per-pixel white balance kernel by capturing two images in quick succession, one with and one without flash, to generate a white-balanced image without user input, using a white balance engine that includes components for handling shadow, specularity, and motion effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual white balancing is performed to correct color casts under mixed illumination, then color accuracy is improved, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system captures a reference image without flash before capturing the main image under mixed illumination. This preliminary reference image contains information about the ambient lighting conditions, which is then used to automatically compute and apply white balance corrections to the main image, eliminating the need for manual color correction and significantly reducing post-processing time
Solution Approach 2:
The patent introduces a computational intermediary process that analyzes the reference image to extract lighting characteristics and generates a white balance transformation matrix. This intermediary computational step bridges the gap between the raw captured image and the final color-corrected image, automating what would otherwise require manual color grading expertise
2Measurement precision
If manual white balancing is performed to correct color casts under mixed illumination, then color accuracy is improved, but operational complexity increases
Solution Approach 1:
The system performs self-service by automatically analyzing the reference image to determine ambient lighting conditions and computing the appropriate white balance corrections without requiring user intervention. The algorithm autonomously identifies color casts and applies corrections based on the captured reference data, making the process accessible to users without color correction expertise
Solution Approach 2:
The patent replaces the mechanical/manual process of color correction with an automated computational system. Instead of requiring manual adjustment of color balances by skilled operators, the system uses image processing algorithms to automatically analyze lighting conditions and apply corrections, substituting human expertise with computational analysis
3Extent of automation
If flash photography is used to capture reference image, then automatic white balance computation is enabled, but motion artifacts may occur between the two images
Solution Approach 1:
The system uses periodic action by capturing the reference image and main image in rapid succession with a fixed temporal pattern. The flash is triggered at a specific interval after the reference image capture, ensuring consistent timing that minimizes motion artifacts while providing sufficient time for the camera and subject to remain stable
Solution Approach 2:
The reference image is captured as a preliminary action before the main image, establishing a temporal baseline. By capturing the reference frame first and immediately using it for white balance computation, the system minimizes the time window during which motion artifacts could develop, while still allowing adequate preparation time for the flash discharge
Data Source
AI summary
Embodiments of the present invention facilitate automatically white-balancing images captured under mixed lighting conditions. More particularly, some embodiments are directed to leveraging flash photography to capture two images in quick succession, one with the flash activated and one without. By combining these two images, a per-pixel white balance kernel can be automatically determined and used to generate a white-balanced image without requiring any user input or user assumptions about the lighting in the scene. In embodiments, the white balance kernels for pixels that lie in the shadow or appear as specular spots are masked and can be interpolated from the white balance kernels at unmasked kernels. Motion effects between the flash and non-flash images may also be reduced.