Visual Neural Network Material Replacement in 3D Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face challenges in accurately and efficiently extending the size and variability of synthesized datasets for three-dimensional models, lacking flexibility, accuracy, and efficiency in generating and modifying digital images, especially for machine-learning applications.
Innovation Solution
The system converts physically-based-rendering materials to procedural materials, using a visual neural network to generate deep visual features and replace materials in digital scenes with visually similar ones from a source dataset, enhancing the flexibility, accuracy, and efficiency of digital image generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional systems are used to generate synthesized digital images, then the process is straightforward, but the flexibility, accuracy, and efficiency in extending dataset size and variability are insufficient
Solution Approach 1:
The patent replaces conventional mechanical material replacement methods with a neural network-based visual feature comparison system. The neural network automatically extracts and compares visual features of materials, enabling flexible and efficient dataset generation without manual intervention, thereby resolving the contradiction between adaptability and productivity.
Solution Approach 2:
The system changes the parameters of procedural materials by adjusting visual features extracted through the neural network. This allows dynamic modification of material properties to generate diverse synthesized images, improving both flexibility and efficiency in dataset extension.
2Manufacturing precision
If PBR materials are used in digital scenes, then rendering accuracy is maintained, but the ability to efficiently generate material variations is limited
Solution Approach 1:
The patent creates visual feature representations (copies) of PBR materials through the neural network. These feature copies can be quickly compared and modified to generate material variations without reprocessing the original complex PBR materials, thus maintaining accuracy while reducing processing time.
Solution Approach 2:
The system performs preliminary extraction of visual features from PBR materials before actual material replacement is needed. This pre-processing step stores essential visual characteristics that can be rapidly accessed and modified later, reducing the time required for generating material variations while preserving rendering accuracy.
3Measurement precision
If manual material replacement is performed in digital scenes, then control over material selection is precise, but the process is time-consuming and inefficient
Solution Approach 1:
The patent replaces manual material selection and replacement processes with an automated neural network system that compares visual features. This automated system maintains precise material matching through learned visual特征 recognition while dramatically increasing the speed of material replacement, resolving the contradiction between measurement precision and productivity.
Data Source
AI summary
Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a visual neural network to replace materials in a three-dimensional scene with visually similar materials from a source dataset. Specifically, the disclosed system utilizes the visual neural network to generate source deep visual features representing source texture maps from materials in a plurality of source materials. Additionally, the disclosed system utilizes the visual neural network to generate deep visual features representing texture maps from materials in a digital scene. The disclosed system then determines source texture maps that are visually similar to the texture maps of the digital scene based on visual similarity metrics that compare the source deep visual features and the deep visual features. Additionally, the disclosed system modifies the digital scene by replacing one or more of the texture maps in the digital scene with the visually similar source texture maps.