Convolutional Neural Network Portrait Lighting via Bilateral Grid

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In photography, especially with mobile terminal devices, achieving ideal lighting for portraits is challenging due to difficulties in obtaining professional lighting distribution, leading to issues like dark faces, lack of three-dimensional effect, and uneven lighting, which complicates image processing and results in low efficiency.

Innovation Solution

An image lighting method using a convolutional neural network to extract local and global feature information, generate a bilateral grid matrix based on pooling results, and perform affine transformation to achieve a more natural and adaptive lighting effect.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If traditional image processing methods are used for portrait lighting, then the processing complexity increases, but the lighting effect quality deteriorates due to inability to achieve professional lighting distribution

Engineering Contradiction:

Improvelighting effect qualityVSAvoidprocessing complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent replaces traditional mechanical/image processing methods with a neural network-based system. The neural network automatically learns and applies professional lighting distributions, substituting complex manual processing with an intelligent system that achieves high-quality lighting effects without increasing processing complexity for the user.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The neural network performs self-learning and self-adjustment to optimize lighting effects. The system automatically adapts to different portraits and lighting conditions without requiring manual intervention, thereby maintaining high lighting quality while keeping the processing system simple for the end user.

Inventive Principle:

Principle #25Self-service

2Manufacturing precision

If professional lighting distribution is applied to achieve ideal portrait effects, then the lighting quality improves, but the ease of operation deteriorates due to difficulty in obtaining proper illumination

Engineering Contradiction:

Improvelighting qualityVSAvoidease of obtaining proper illumination

Core Design Contradiction:

Manufacturing precisionVSEase of operation

Solution Approach 1:

The patent replaces the need for physical professional lighting equipment and manual lighting adjustment with a computational neural network system. This substitution allows users to achieve professional lighting effects through software processing alone, dramatically improving ease of operation while maintaining high lighting quality.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The neural network system provides universal lighting solutions that work across different lighting conditions, portrait types, and devices. A single system handles multiple lighting scenarios that would otherwise require different professional lighting setups, making the process easy to operate while maintaining consistent high quality.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Ease of operation

If traditional lighting methods are used in less-than-ideal photography environments, then the simplicity is maintained, but the lighting effect deteriorates with dark faces and uneven illumination

Engineering Contradiction:

Improvesimplicity of methodVSAvoidlighting effect

Core Design Contradiction:

Ease of operationVSManufacturing precision

Solution Approach 1:

The patent replaces simple traditional lighting methods with an intelligent neural network system that automatically adapts to various environmental conditions. The neural network analyzes the input image and applies appropriate lighting corrections, maintaining simplicity of operation while dramatically improving lighting effects in challenging environments.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The neural network dynamically adjusts lighting parameters based on the specific characteristics of each portrait and lighting condition. Rather than using fixed simple rules, the system adaptively modifies lighting distributions in real-time, achieving high-quality results across varying environments while keeping the user interface simple.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11455516B2Image lighting methods and apparatuses, electronic devices, and storage media

Publication Date: 2022.09.27 SHENZHEN SENSETIME TECH CO LTD

AI summary

An image lighting method includes: determining a convolutional neural network corresponding to a lighting operation type of an initial image; obtaining local feature information and global feature information of initial image according to convolutional neural network; obtaining fusion feature information of initial image according to convolutional neural network based on local feature information and global feature information; obtaining a maximum pooling result map and a minimum pooling result map according to convolutional neural network based on a luminance component map of initial image; obtaining a bilateral grid matrix of initial image based on fusion feature information, maximum pooling result map, and minimum pooling result map; and performing affine transformation on initial image according to bilateral grid matrix to obtain a target image, the target image being an image obtained after lighting initial image according to lighting operation type.