Neural Network Training Database Generation for 2D to 3D Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for converting 2D images into 3D models using neural networks face challenges due to the lack of a comprehensive database of pairs of input 2D images and output 3D models, which affects the fidelity of the conversion process.
Innovation Solution
A computer-implemented method and system for generating a database for training neural networks, involving the steps of obtaining 3D models through various methods, rendering them into 2D formats, and collecting pairs of rendered 2D image frames and corresponding 3D models for training the neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If neural networks are used to convert 2D images into 3D models, then the conversion process can be automated, but the fidelity of the conversion is poor due to lack of training data
Solution Approach 1:
The patent applies preliminary action by pre-collecting and preparing volumetric data from multiple sources (volumetric videos, depth maps, multi-view images) before training the neural network. This pre-prepared database of 2D-3D image pairs serves as training data, enabling the network to learn accurate conversion patterns in advance, thereby improving conversion fidelity while maintaining automation.
2Manufacturing precision
If a comprehensive database of 2D-3D image pairs is created, then the training quality improves, but the data collection and processing complexity increases
Solution Approach 1:
The patent applies universality by creating a multi-functional database generation system that can process various types of input data (volumetric videos, depth maps, multi-view images) through a unified pipeline. The system performs multiple functions including data collection, preprocessing, 3D model generation, and pair creation within a single integrated framework, thereby managing complexity while producing comprehensive training data.
Solution Approach 2:
The patent applies copying by generating multiple 2D renderings from the same 3D model at different viewpoints and conditions. Instead of requiring entirely separate 3D models for each 2D image, the system creates multiple views by rendering copies of the underlying 3D structure, efficiently generating numerous 2D-3D pairs from a single source.
3Manufacturing precision
If volumetric data is captured from multiple sources, then the diversity and quality of training data increases, but the time and resources required for data collection increase
Solution Approach 1:
The patent applies merging by combining data from multiple sources (volumetric videos, depth maps, multi-view images) into a single unified training database. Instead of treating each source separately, the system integrates these diverse data streams, merging them into comprehensive 2D-3D image pairs that leverage the strengths of each source while streamlining the overall process.
Data Source
AI summary
A computer-implemented method of generating of a database for training a neural network configured for converting 2d images into 3d models comprising steps of: (a) obtaining 3d models; (b) rendering said 3d models in a 2d format from at least one view point; and (c) collecting pairs further comprising said rendered 2d image frame and said corresponding sampled 3d models each.