Aligned UV Texture Mapping for Distortion-Free 3D Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for texture synthesis in 3D models fail to account for semantic information, leading to distorted textures, especially for complex shapes, and require manual supervision or ground-truth data for alignment.
Innovation Solution
A neural network-based approach that learns to embed 3D surfaces into an aligned UV space using a texture alignment module, cutting shapes into pieces for UV mapping, and using a masking network to reduce distortion, allowing for unsupervised texture alignment and synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If spherical texture maps are used for texture synthesis, then the texture can be generated for arbitrary surface topology, but severe distortions are introduced for thin parts such as animal limbs
Solution Approach 1:
The patent segments the 3D shape surface into multiple patches or regions, each of which is independently mapped to a 2D texture domain. This segmentation allows each patch to be textured without the severe distortions that occur when the entire shape is mapped using a spherical texture map, particularly for thin parts like animal limbs.
Solution Approach 2:
The patent transitions from a spherical texture map approach (2D surface embedded in 3D space) to a UV mapping approach that directly maps 3D surface points to a 2D plane in a way that preserves topology. This dimensional transformation allows arbitrary surface topology to be represented without the distortion artifacts inherent in spherical mappings.
2Ease of manufacture
If implicit texture fields are used for texture synthesis, then texture can be generated without explicit texture mapping, but the output textures are overly smoothed
Solution Approach 1:
The patent replaces the implicit texture field approach (which uses continuous neural networks to predict textures) with an explicit UV mapping approach combined with a neural network that predicts aligned UV coordinates. This substitution allows the system to maintain the ease of automatic texture generation while avoiding the excessive smoothing problem by preserving high-frequency texture details through the explicit mapping and conditioning mechanism.
3Ease of manufacture
If warping a spherical mesh template to the target shape is used, then texture map can be obtained for texture synthesis, but semantic information in texture synthesis is not accounted for
Solution Approach 1:
The patent introduces a feedback mechanism where the neural network conditions the UV coordinate prediction on both the 3D shape representation and the target texture image. This feedback loop allows the system to iteratively refine the UV mapping to better align with semantic information in the texture, enabling the system to account for semantic content while maintaining automatic texture synthesis.
Data Source
AI summary
Approaches presented herein can utilize a network that learns to embed three-dimensional (3D) coordinates on a surface of one or more 3D shapes into an aligned two-dimensional (2D) texture space, where corresponding parts of different 3D shapes can be mapped to the same location in a texture image. Alignment can be performed using a texture alignment module that generates a set of basis images for synthesizing textures. A trained network can generate a basis shared by all shape textures, and can predict input-specific coefficients to construct the output texture for each shape as a linear combination of the basis images, then deform the texture to match the pose of the input. Such an approach can ensure alignment of textures, even in situations with at least somewhat limited network capacity. To unwrap shapes of complex structure or topology, a masking network can be utilized that cuts the shape into multiple pieces to reduce the distortion in the 2D mapping.