Portrait Image Relighting Using Surface Orientation and Light Energy Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile device users lack access to professional-grade portrait lighting capabilities and tools, limiting their ability to enhance images with desirable lighting effects.
Innovation Solution
Implementing a convolutional neural network on mobile devices to adjust lighting conditions in captured images, using machine learning to apply lighting models that enhance image quality and user control over lighting effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If professional portrait lighting equipment and controlled studio settings are used, then image quality and lighting control are improved, but device complexity and accessibility are worsened
Solution Approach 1:
The patent creates a computational copy of professional portrait lighting behavior through machine learning models. Instead of requiring physical lighting equipment, the system learns from training data containing images with professional lighting and synthesizes similar lighting effects computationally, making professional-grade lighting accessible on mobile devices
Solution Approach 2:
The patent replaces mechanical lighting systems (physical lights, reflectors, studio equipment) with a computational system based on machine learning. The neural network model substitutes for physical lighting equipment by analyzing input images and applying learned lighting transformations, eliminating the need for complex mechanical lighting gear
2Illumination intensity
If professional portrait lighting equipment and controlled studio settings are used, then image quality and lighting control are improved, but ease of operation is worsened
Solution Approach 1:
The patent implements self-service by enabling the mobile device to automatically perform portrait lighting enhancement without requiring user expertise. The machine learning model autonomously analyzes the input image, determines appropriate lighting adjustments, and applies enhancements, allowing any user to achieve professional results without learning complex lighting techniques
Solution Approach 2:
The system copies the expertise and judgment of professional portrait photographers into the machine learning model. By training on data from professional photography, the model encapsulates professional lighting knowledge, allowing ordinary users to access professional-level results without needing to learn or understand professional techniques
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Apparatus and methods related to applying lighting models to images of objects are provided. An example method includes applying a geometry model to an input image to determine a surface orientation map indicative of a distribution of lighting on an object based on a surface geometry. The method further includes applying an environmental light estimation model to the input image to determine a direction of synthetic lighting to be applied to the input image. The method also includes applying, based on the surface orientation map and the direction of synthetic lighting, a light energy model to determine a quotient image indicative of an amount of light energy to be applied to each pixel of the input image. The method additionally includes enhancing, based on the quotient image, a portion of the input image. One or more neural networks can be trained to perform one or more of the aforementioned aspects.