3D Human Model Generation Using Reference Pose Feature Grids
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training high-resolution human mesh models, such as NeRF or SMPL models, is resource-intensive and computationally demanding, requiring vast amounts of data and extensive processing power, limiting accessibility to organizations with significant computational resources.
Innovation Solution
A machine learning-based approach that estimates features from images, inverse warps them into a reference pose, generates filtered features using a feature grid, projects these features into 2D planes, and combines them to create a 3D model, reducing memory requirements and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution human mesh models (NeRF or SMPL) are trained using traditional methods, then model quality and resolution are improved, but computational resource requirements and memory usage increase significantly
Solution Approach 1:
The patent segments the 3D modeling process into distinct components: feature extraction from images, inverse warping to reference pose, feature grid generation, and model synthesis. This segmentation allows each component to be optimized independently, reducing overall computational requirements while maintaining model quality.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing feature grids for reference poses before actual modeling occurs. These pre-computed feature grids are then reused across multiple modeling operations, significantly reducing the computational burden during actual model generation while maintaining high resolution and quality.
2Reliability
If traditional training methods are used for high-resolution mesh models, then accurate 3D representations are achieved, but accessibility to devices with limited resources is reduced
Solution Approach 1:
The patent creates simplified copies of 3D features through 2D feature grids that capture essential geometric information. These feature grid copies serve as efficient proxies for full 3D representations, enabling accurate modeling on devices with limited computational resources while maintaining reliability of the final 3D output.
3Manufacturing precision
If extensive processing power is allocated to model training, then high-resolution output is achieved, but processing time and computational cost increase
Solution Approach 1:
The patent performs feature grid computation as a preliminary action that can be done once and reused. This pre-computation approach maintains high resolution output quality while significantly reducing processing time for subsequent modeling operations, as the computationally intensive feature extraction is performed only once rather than repeatedly.
Data Source
AI summary
Systems and techniques are disclosed for generating a three-dimensional (3D) model. For example, a process can include estimating a plurality of features associated with at least a portion of images; inverse warping the plurality of features into reference pose features having a reference pose; generating filtered reference pose features by selecting features from the reference pose features based on a distance of the selected features from corresponding features from the reference pose; generating modified reference pose features by modifying the filtered reference pose features based on a feature grid associated with a reference model associated with the reference pose; projecting the filtered reference pose features into one or more two dimensional (2D) planes; identifying first features associated with the person from the one or more 2D planes; and generating a 3D model of the person having a pose using the first features, the modified reference pose features, and pose information.