PBR Material Map Identification Using Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Physically Based Rendering (PBR) assets with unknown or missing material maps face challenges in rendering photorealistic images due to lack of material definitions or inaccurate linking during import/export processes, leading to poor quality results.
Innovation Solution
The technology employs image analysis using machine classifiers like convolutional neural networks for material map type classification and Generative Adversarial Networks (GANs) to generate synthetic material maps, ensuring correct type assignment and correlation of material maps, even when identity information is lost or absent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If material maps are imported from another application, then the PBR asset can be used in the new application, but the material-map type information may be lost during the import process
Solution Approach 1:
The system performs preliminary classification of material maps using a trained machine learning model before rendering. The model analyzes visual characteristics of material maps and predicts their types (e.g., diffuse, normal, roughness, metallic) in advance, so that even without metadata, the rendering engine can correctly process each map type with appropriate shading models and parameters.
Solution Approach 2:
A machine learning classification model serves as an intermediary between the imported material maps and the rendering engine. The model takes raw material map images as input and outputs predicted type classifications, bridging the gap between format-agnostic bitmap data and the type-specific processing requirements of the PBR pipeline.
2Adaptability or versatility
If material maps are exported to another application, then the asset can be shared, but the identity information and correlations between material maps may be lost
Solution Approach 1:
The system creates visual copies or embeddings of material map characteristics that can be analyzed by the machine learning model. Instead of relying on fragile metadata that may not transfer between applications, the system uses the actual visual content of the material maps themselves as the basis for type identification, ensuring that the information needed for correct rendering is preserved in a format-independent manner.
3Measurement precision
If a machine classifier is used to classify material map types, then unknown material maps can be identified, but the classification accuracy may be insufficient without proper training data
Solution Approach 1:
The system performs preliminary training of the machine learning model using a dataset of example material maps with known types. This training phase creates a pre-trained classifier that can then be applied to unknown material maps without requiring additional training at runtime. The pre-training captures the visual characteristics of different material map types, enabling accurate classification when the asset is imported or exported.
Data Source
AI summary
Embodiments of the technology described herein, make unknown material-maps in a Physically Based Rendering (PBR) asset usable through an identification process that relies, at least in part, on image analysis. In addition, when a desired material-map type is completely missing from a PBR asset the technology described herein may generate a suitable synthetic material map for use in rendering. In one aspect, the correct map type is assigned using a machine classifier, such as a convolutional neural network, which analyzes image content of the unknown material map and produce a classification. The technology described herein also correlates material maps into material definitions using a combination of the material-map type and similarity analysis. The technology described herein may generate synthetic maps to be used in place of the missing material maps. The synthetic maps may be generated using a Generative Adversarial Network (GAN).