Single-Image 3D Avatar Generation via Neural Network Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing avatar generation systems inaccurately represent users, require high-performance processors, and do not function well on power-constrained mobile devices like smartphones or tablets.
Innovation Solution
The system determines a 3D shape and joint locations of a subject from a single image using a feature network, classification network, and joint network, which predicts surface features, depth values, and joint positions, enabling accurate avatar representation without high-performance graphics processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing avatar generation systems are used, then avatar generation can be performed, but the systems require high-performance processors and do not function well on power-constrained mobile devices
Solution Approach 1:
The system segments the avatar generation process into distinct neural network components: a feature network for extracting surface features, a classification network for determining 3D occupancy, and a joint network for predicting joint locations. This segmentation allows each component to be optimized independently for mobile deployment, reducing overall power consumption while maintaining accuracy
Solution Approach 2:
The patent replaces traditional graphics processing (GPU-based rendering and mesh processing) with neural network-based processing that can be executed on mobile CPUs or NPUs. This substitution eliminates the need for high-performance graphics processors while achieving comparable or superior avatar reconstruction accuracy through learned feature representations
2Measurement precision
If existing avatar generation systems are used, then avatar generation can be performed, but the systems inaccurately represent the user
Solution Approach 1:
The system transitions from 2D image processing to 3D volume classification by introducing depth as a third dimension. The classification network evaluates 3D points within a volume bounded by the subject, enabling accurate reconstruction of complex 3D geometries and joint locations from 2D input images, thereby improving representation accuracy
Solution Approach 2:
The feature network serves as an intermediary that extracts meaningful surface features from input images and transforms them into feature vectors suitable for 3D classification. This intermediary processing step enables the classification network to accurately determine 3D occupancy without directly processing raw pixel data, improving accuracy while managing system complexity
3Measurement precision
If existing avatar generation systems are used, then avatar generation can be performed, but they require high-performance general and graphics processors
Solution Approach 1:
The neural network-based system is designed to be universally applicable across different device types. The same feature network, classification network, and joint network can be deployed on mobile devices, tablets, or servers without requiring device-specific optimizations, enabling accurate avatar generation on power-constrained mobile devices while maintaining compatibility across the full range of computing platforms
Data Source
AI summary
Generating a 3D representation of a subject includes obtaining an image of a physical subject. Front depth data is obtained for a front portion of the physical subject. Back depth data is obtained for the physical subject based on the image and the front depth data. A set of joint locations is determined for the physical subject from the image, the front depth data, and the back depth data.