Neural Network Material Map Generation Using Photometric Stereo
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for creating physically-based digital materials are either bulky and expensive, requiring extensive computer processing power or provide low-quality results due to the need for numerous image captures, making it challenging to achieve accurate and high-quality digital material renderings.
Innovation Solution
The system captures real-world materials using images with different lighting patterns, including area lights, and employs a neural network to generate material maps, with optimization through a differentiable renderer, reducing the number of captures needed and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If goniophotometers with rotating lighting apparatus are used to capture images at broad range of angles, then measurement precision and manufacturing precision are improved, but device complexity, cost, and processing time increase significantly
Solution Approach 1:
The patent uses a camera to capture images of the material sample instead of complex goniophotometer instrumentation. Multiple images are taken from different lighting angles and processed computationally to generate material maps, replacing mechanical rotation and complex optical systems with a simpler imaging approach.
Solution Approach 2:
The patent replaces the mechanical rotating lighting apparatus of goniophotometers with computational methods. Instead of physically rotating lights and cameras to capture data from multiple angles, the system uses multiple static images with different lighting patterns and processes them through algorithms to achieve the same material characterization.
2Measurement precision
If goniophotometers capture images at broad range of angles, then measurement precision is improved, but loss of time and productivity deteriorate due to slow operation
Solution Approach 1:
The patent captures multiple images of the material sample simultaneously or in rapid succession from different lighting angles before processing. By preparing all necessary images in advance through a simple capture process, the system avoids time-consuming mechanical rotations during measurement, enabling quick material characterization.
3Manufacturing precision
If a large number of image captures are acquired to improve digital material quality, then manufacturing precision is improved, but use of energy and computer processing power increase
Solution Approach 1:
The patent uses a limited set of strategically chosen lighting patterns and camera angles to capture the essential material properties. Instead of capturing images from all possible angles, the system selects specific lighting configurations that provide sufficient information for accurate material map generation, reducing computational requirements while maintaining quality.
4Ease of operation
If handheld camera with flashlight is used to reduce device complexity, then ease of operation is improved, but measurement precision and manufacturing precision deteriorate
Solution Approach 1:
The patent uses a standard camera device that can serve multiple purposes: capturing images for material characterization, storing images for later processing, and potentially sharing images through communication interfaces. This universal device replaces specialized instrumentation while achieving comparable results through computational processing.
Solution Approach 2:
The patent improves measurement precision by varying lighting parameters (intensity, angle, pattern) and processing parameters (algorithm selection, processing intensity) rather than relying on complex hardware. By controlling these parameters computationally, the system achieves high-quality material maps using simple, portable equipment.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate and efficient creation of high-quality digital material maps with reduced computational requirements, allowing for precise rendering of real-world materials in applications like video game development and augmented reality.
Implementation Method 1
a diffuse component image and a specular component image may be obtained for each lighting pattern by rotating a polarization filter when capturing images of the material
Data Source
AI summary
Methods, system, and computer storage media are provided for generating physical-based materials for rendering digital objects with an appearance of a real-world material. Images depicted the real-world material, including diffuse component images and specular component images, are captured using different lighting patterns, which may include area lights. From the captured images, approximations of one or more material maps are determined using a photometric stereo technique. Based on the approximations and the captured images, a neural network system generates a set of material maps, such as a diffuse albedo material map, a normal material map, a specular albedo material map, and a roughness material map. The material maps from the neural network may be optimized based on a comparison of the input images of the real-world material and images rendered from the material maps.