Portrait Shadow Removal Through Masked Lighting Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
It is challenging to automatically alter photographs to eliminate shadows while maintaining proper exposure in natural lighting environments, often resulting in high contrast shadow boundaries that detract from aesthetic appeal.
Innovation Solution
A machine learning model is trained using well-lit and shadowed image pairs to generate synthetic images, which are scored and adjusted to create aesthetically pleasing portraits by softening or removing shadows, utilizing a mask to combine and manipulate lighting conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If conventional image post-processing techniques are used to eliminate shadows, then shadow removal is achieved, but proper exposure to the subject's face cannot be maintained and high contrast shadow boundaries are created
Solution Approach 1:
The patent segments the image processing task into multiple components: shadow detection, mask generation, and selective manipulation. By dividing the face image into shadowed and non-shadowed regions using a mask, the system can apply different processing operations to each region, allowing shadow elimination while preserving proper exposure in non-shadowed areas.
Solution Approach 2:
The patent applies local quality by using a mask to apply different processing operations to different regions of the image. The shadowed regions are selectively manipulated to eliminate shadows, while non-shadowed regions maintain their original exposure characteristics. This localized approach ensures that shadow removal does not compromise the overall exposure accuracy of the subject's face.
2Object-affected harmful factors
If conventional image post-processing techniques are used to eliminate shadows, then shadow removal is achieved, but high contrast shadow boundaries are created that reduce aesthetic appeal
Solution Approach 1:
The patent employs dynamic processing by applying blur operations with varying intensities to different regions of the mask. The blur amount is adjusted based on the distance from shadow boundaries, creating a dynamic transition zone that smooths high contrast boundaries. This dynamic approach transforms the static, abrupt shadow boundaries into smooth, aesthetically pleasing gradients.
Solution Approach 2:
The patent changes the blur parameter selectively across different regions of the image. By applying different blur amounts to different zones (higher blur near shadow boundaries, lower blur in shadowed regions), the system transforms the sharp shadow boundaries into smooth transitions, eliminating the high contrast effect while preserving the shadow elimination benefit.
3Shape
If machine learning models are trained to generate synthetic images with adjusted lighting, then aesthetic appeal is improved, but processing complexity and computational resources increase
Solution Approach 1:
The patent applies preliminary action by pre-training machine learning models to generate synthetic images with various lighting conditions and shadow patterns. This pre-training phase creates a library of learned transformations that can be quickly applied during actual image processing. The complex learning work is performed in advance, allowing faster and less computationally intensive application during runtime.
Solution Approach 2:
The patent uses copying by generating synthetic images that replicate the desired lighting conditions and shadow patterns. Instead of performing complex real-time calculations, the system creates synthetic copies of the target image with adjusted lighting based on pre-trained models. These synthetic images serve as templates that can be blended with the original image to achieve the desired aesthetic quality with reduced computational overhead.
Data Source
AI summary
Systems and methods described herein may relate to potential methods of training a machine learning model to be implemented on a mobile computing device configured to capture, adjust, and/or store image frames. An example method includes supplying a first image frame of a subject in a setting lit within a first lighting environment and supplying a second image frame of the subject lit within a second lighting environment. The method further includes determining a mask. Additionally, the method includes combining the first image frame and the second image frame according to the mask to generate a synthetic image and assigning a score to the synthetic image. The method also includes training a machine learning model based on the assigned score to adjust a captured image based on the synthetic image.