3D Object Modeling with Layered Depth Images for Fast Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating three-dimensional models are time-consuming and resource-intensive, particularly when creating virtual try-on augmented reality applications for large electronic catalogs, due to the high computational demands of existing techniques like Neural Radiance Fields, Periodic Implicit Generative Adversarial Networks, and DeepSDF, which require extensive sampling and computing resources.
Innovation Solution
The use of layered depth images to represent three-dimensional models, trained using machine learning models, allows for efficient generation of three-dimensional models with reduced computing resources, enabling practical differentiable rendering and improved scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If professional photogrammetry rig with approximately one hundred cameras is used to capture initial scans, then the three-dimensional model quality is improved, but the device complexity and resource requirements increase significantly
Solution Approach 1:
The patent segments the complex photogrammetry process into two distinct phases: (1) an initial scanning phase using a professional rig with approximately one hundred cameras to capture high-quality reference data, and (2) a generation phase using a trained machine learning model that processes the segmented reference data to produce three-dimensional models. This segmentation allows the complex hardware to be used only once for data collection, while subsequent models are generated through software processing.
Solution Approach 2:
The patent applies preliminary action by using the professional photogrammetry rig to capture initial scans and create a training dataset before the machine learning model is deployed. The model is trained on this pre-captured reference data, enabling it to generate three-dimensional models without requiring the complex hardware during actual model generation. This preliminary data collection and model training phase eliminates the need for repeated use of the complex photogrammetry rig.
2Manufacturing precision
If three-dimensional artist manually creates the final three-dimensional model from scans, then the model quality is improved, but the time consumption increases to at least eight hours
Solution Approach 1:
The patent replaces the manual mechanical process of three-dimensional artists manually creating models from scans with an automated machine learning system. The trained model automatically processes the initial scans and generates three-dimensional models through computational algorithms, eliminating the need for manual manipulation and significantly reducing creation time from at least eight hours to a fraction of that time while maintaining quality.
Solution Approach 2:
The machine learning model is designed to be self-sufficient in generating three-dimensional models from scan data without requiring manual intervention. Once trained on reference datasets, the model independently processes new scan inputs and produces high-quality three-dimensional outputs autonomously, freeing artists from repetitive manual modeling tasks.
3Manufacturing precision
If existing techniques like Neural Radiance Fields, Periodic Implicit Generative Adversarial Networks, and DeepSDF are used, then the three-dimensional model detail is improved, but the computing resource requirements increase significantly
Solution Approach 1:
The patent creates a trained machine learning model that serves as a computational copy or surrogate of the complex rendering processes used in techniques like Neural Radiance Fields. Instead of repeatedly executing computationally intensive rendering algorithms for each three-dimensional model generation, the system uses the pre-trained model to approximate and generate models efficiently, capturing the essential detailed features without the full computational overhead of the original techniques.
Data Source
AI summary
The system generates a three-dimensional model with layered depth images based on an input two-dimensional image. For training, layered depth images are derived from existing three-dimensional models. The system trains a machine learning model to predict multiple layered depth images from an input image of an object. The system compares the generated, multiple layered depth images to the derived layered depth images for the object to update the machine learning model during training. At inference time, the system receives an input image for an object. The system applies the machine learning model to the input image to output predicted layered depth images. The system generates a three-dimensional model from the predicted layered depth images.