Single-Image 3D Object Texture Mapping with Sampler-Refiner Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating virtual 3D human models from images often result in low similarity with the user due to limitations in scanning technology or the number of provided images, and lack detailed texture representation.
Innovation Solution
An apparatus and method for generating a 3D object texture map by blending a sampling image obtained from a sampler network and a refined image obtained from a refiner network, using a curriculum-trained sampler network and a loss-function-trained refiner network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a single image is used to generate a 3D human model, then the ease of manufacture is improved, but the texture quality and detail representation deteriorate
Solution Approach 1:
The texture generation process is divided into two distinct stages: a sampler network that generates initial texture from a single image, and a refiner network that enhances texture details. This segmentation allows each network to specialize in specific tasks, with the sampler focusing on overall texture synthesis and the refiner focusing on detail enhancement, thereby resolving the contradiction between ease of manufacture and texture quality.
Solution Approach 2:
The sampler network performs preliminary texture generation before the refiner network processes it. By pre-generating a complete but low-detail texture map first, the system establishes a foundation that the refiner can then enhance. This preliminary action enables the final high-quality texture to be achieved through sequential processing rather than requiring complex simultaneous operations.
2Device complexity
If conventional scanning methods are used, then the manufacturing process is simplified, but the similarity with the user deteriorates due to limitations in scanning technology
Solution Approach 1:
Instead of using physical scanning devices that have inherent limitations, the system creates a digital copy of the user's appearance through neural networks. The sampler and refiner networks learn from training data to synthesize realistic texture that captures user-specific details, producing a digital replica that overcomes the physical limitations of scanning technology while maintaining simplicity in the manufacturing process.
3Loss of time
If only posture and form reconstruction is performed, then the processing time is reduced, but the texture detail representation deteriorates
Solution Approach 1:
The system maintains continuous useful action by implementing a two-stage processing pipeline where the sampler network continuously generates base texture and the refiner network continuously enhances it. Both networks operate in sequence without interruption, ensuring that neither processing time is excessively increased nor texture details are lost, as each stage builds upon the previous one efficiently.
Data Source
AI summary
There is provided an apparatus for generating a 3D object texture map. The apparatus may comprise: a memory; and a processor, wherein the processor is configured to: generate a partial texture image by mapping object information in an input image into a texture space; obtain a sampling image by inputting the partial texture image to a sampler network trained according to a curriculum for selecting at least one of a training partial texture image, an aligned partial image in which the training partial texture image is aligned in the texture space, or an augmented partial image augmented from the aligned partial image; obtain a blending mask and a refined image by inputting the sampling image to a refiner network; and generate a 3D object texture map by blending the sampling image and the refined image based on the blending mask.