Image-to-Material Translation via Shadow Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to accurately perform image-to-material translation, especially with heterogeneous materials and complex shading, leading to inaccuracies, destructive functionality, and high computing resource consumption, which negatively affects user experience.
Innovation Solution
The use of cascaded U-Net machine learning models to derive albedo and normal maps from input images, removing shadow and highlight data to capture color and geometric properties, thereby generating accurate visual renderings without requiring flash photography and reducing computing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If existing machine learning systems are used for image-to-material translation, then the process can be automated, but the accuracy deteriorates especially with heterogeneous materials and complex shading
Solution Approach 1:
The patent segments the complex image-to-material translation task into distinct components: shadow map generation, highlight map generation, and albedo map reconstruction. By dividing the automated process into specialized sub-tasks, each handled by specific machine learning models, the system achieves both high automation and high accuracy even with heterogeneous materials and complex shading
Solution Approach 2:
The patent introduces intermediate maps (shadow maps and highlight maps) as mediators between the input image and the final albedo map. These intermediate representations allow the system to automatically separate different lighting effects, improving both the automation capability and the accuracy of material reconstruction
2Illumination intensity
If flash photography is used to capture input images, then lighting conditions can be controlled, but shadow and highlight data that obscures geometric features is introduced
Solution Approach 1:
The patent extracts and removes shadow and highlight data from the input image through automated machine learning processes. By taking out these problematic lighting effects computationally, the system recovers geometric feature information that would otherwise be obscured, eliminating the need for flash photography while preserving all necessary visual information
3Loss of information
If multiple images are used as input, then more complete data can be captured, but computing resource consumption increases
Solution Approach 1:
The patent uses a single input image and creates computational copies through generated intermediate maps (shadow maps, highlight maps) rather than requiring multiple physical images. This approach captures complete material information while minimizing computing resource consumption by processing one image through multiple computational passes rather than processing multiple images
Data Source
AI summary
Various disclosed embodiments are directed to image-to-material translation based on delighting an input image, thereby allowing proper capturing of the color and geometry properties of the input image for generating a visual rendering. This, among other functionality described herein, improves the inaccuracies, user experience, and computing resource consumption of existing technologies.