3D Human Model Parsing With Texture-Space Mask Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D model parsing techniques lack robustness and accuracy in segmenting human models, particularly in identifying and labeling different parts of the human body, such as garments, due to occlusion and variation in clothing styles.
Innovation Solution
A system that combines multi-view image space segmentation and texture space segmentation, using machine learning models to generate and label 3D masks, including a promptable segmentation model for texture space masks and a semantic branch for accurate labeling, followed by back-projection and vertex voting to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing 3D model parsing techniques are used, then the process is simpler, but the segmentation accuracy and robustness deteriorate due to occlusion and clothing variation
Solution Approach 1:
The patent divides the 3D model parsing into multiple independent components: a semantic branch that generates initial segmentation masks, a texture space mask generation module, and a label assignment module. This segmentation of the processing pipeline enables each component to specialize in specific tasks, improving overall segmentation accuracy while managing complexity through modular design
Solution Approach 2:
The patent introduces texture space masks as an intermediary representation between the input 3D model and the final segmented output. These masks serve as a mediating structure that captures clothing and texture information in a standardized coordinate system, enabling more accurate label assignment despite occlusions and clothing variations
2Measurement precision
If multi-view image space segmentation and texture space segmentation are combined, then labeling accuracy improves, but computational complexity increases
Solution Approach 1:
The patent performs preliminary segmentation in image space to generate candidate masks before transforming to texture space. This preliminary action filters out obvious non-target regions early in the pipeline, reducing the computational burden of subsequent texture space processing while maintaining high labeling accuracy for relevant regions
Solution Approach 2:
The patent transforms the segmentation problem from 3D model space to 2D texture space, enabling the use of efficient 2D image processing techniques. This dimensional transformation allows leveraging well-optimized 2D convolutional neural networks while achieving accurate 3D model labeling through the texture atlas mapping
3Reliability
If promptable segmentation model is used for texture space masks, then segmentation robustness improves, but processing time increases
Solution Approach 1:
The patent applies promptable segmentation selectively to identify and segment clothing regions, rather than processing the entire 3D model uniformly. By focusing computational resources on locally challenging regions with occlusions or complex textures, the system achieves improved robustness while minimizing overall processing time
Data Source
AI summary
Techniques and systems are provided for three-dimensional model segmentation. For instances, a process can include: rendering a three-dimensional (3D) model of an object based on a 3D mesh model and a texture of the object; generating a view of the 3D model based on the rendered 3D model; generating a texture space mask based on the texture of the object using a first machine learning (ML) model; generating a labeled mask based on a second ML model; assigning labels to the texture space mask based on the labeled mask to obtain a labeled texture space mask; and generating a 3D mask based on the labeled texture space mask.