Neural Light Redistribution for Mobile Specular Highlight Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices lack computational resources to perform high-quality image enhancement features such as reducing specular highlights and redistributing light in images, which are typically available only on more powerful computing devices.
Innovation Solution
A neural network-based system on mobile devices adjusts specular and diffuse components of images by redistributing light energy without predicting albedo, using a U-net architecture and training on a light stage computational illumination system to enhance images in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a neural network predicts albedo to perform light redistribution and specular highlight removal, then image enhancement quality improves, but computational resource requirements increase
Solution Approach 1:
The patent extracts and removes the albedo prediction step from the traditional light redistribution pipeline. By directly predicting the relit image without intermediate albedo estimation, the system eliminates unnecessary computational operations while maintaining enhancement quality, directly resolving the contradiction between image quality and computational resource usage on mobile devices
Solution Approach 2:
The patent uses a lightweight neural network architecture that learns to directly map input images to relit output images, effectively creating a simplified computational model that copies the essential light redistribution function without requiring the complex albedo prediction pathway, thereby reducing computational overhead while preserving image enhancement effectiveness
2Manufacturing precision
If traditional light redistribution methods are used to soften shadows and remove specular highlights, then image quality improves, but device complexity increases
Solution Approach 1:
The patent replaces complex traditional image processing algorithms with a trained neural network model that performs light redistribution through learned patterns. This substitution simplifies the device architecture by using a compact neural network instead of multiple complex processing stages, reducing device complexity while maintaining high image quality through the network's ability to directly predict relit images
Data Source
AI summary
Apparatus and methods related to light redistribution in images are provided. An example method includes receiving, by a computing device, an input image comprising a subject. The method further includes adjusting, by a neural network, one or more of a specular component or a diffuse component associated with the input image. The adjusting involves redistributing a per-pixel light energy of the input image. The method additionally includes predicting, by the neural network, an output image comprising the subject with the adjusted one or more of the specular component or the diffuse component.