Dynamic PIFu Enrollment Using Deformation-Aware 3D Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing avatar generation systems inaccurately represent users, require high-performance general and graphics processors, and do not work well on power-constrained mobile devices like smartphones or computing tablets.
Innovation Solution
A method for determining a 3D occupation field using multiple images from different viewpoints, incorporating a feature network, classification network, and deformation data to improve 3D reconstruction on mobile devices by sampling feature points and fusing classification values across images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing avatar generation systems are used, then avatar generation can be performed, but the systems inaccurately represent the user and require high-performance processors
Solution Approach 1:
The system segments the avatar generation process into distinct modules: a feature network for extracting user characteristics from images, a classification network for categorizing features, and a mesh generation component for constructing the 3D avatar. This segmentation allows each module to be optimized independently, improving accuracy while reducing overall computational complexity on mobile devices.
Solution Approach 2:
The patent introduces deformation data as an intermediary element that bridges 2D images and 3D avatar reconstruction. The deformation data captures geometric transformations and is integrated with classification results to accurately represent user features in 3D space, improving measurement precision without requiring high-performance processors.
2Power
If high-performance processors are used for avatar generation, then processing power is sufficient, but the systems do not work well on power-constrained mobile devices
Solution Approach 1:
The patent employs lightweight neural network architectures and optimized algorithms that can be executed on mobile devices with limited computational resources. The system uses efficient feature extraction and classification methods that consume less power compared to high-performance processor requirements, enabling avatar generation on power-constrained devices.
Solution Approach 2:
The system changes computational parameters by using deformation data representation and efficient fusion algorithms that reduce computational complexity. The classification network processes features with optimized parameters that lower power consumption while maintaining processing capability on mobile devices.
3Manufacturing precision
If multiple images from different viewpoints are processed, then 3D reconstruction accuracy is improved, but computational requirements increase
Solution Approach 1:
The system performs preliminary processing by extracting deformation data from multiple images before the main 3D reconstruction process. The feature network pre-processes images to identify key characteristics, and the classification network pre-categorizes features, reducing the computational load during final avatar generation while maintaining high reconstruction accuracy.
Solution Approach 2:
Deformation data serves as an intermediary that consolidates information from multiple images. Instead of processing all images directly for 3D reconstruction, the system first computes deformation data that captures essential geometric transformations, then uses this compact representation to achieve accurate reconstruction with reduced computational requirements.
Data Source
AI summary
Generating a 3D representation of a subject includes obtaining a set of images of a subject. For each sample point, a classifier value is obtained based on each image. The classifier value indicates a relationship of the sample point to an interior or exterior of a volume of the subject. In addition, deformation data is determined for the subject across the image. The classifier values are fused based on the deformation data, and a 3D occupation field is determined for the subject based on the fused classifier values.