3D Human Model Generation via Part-Specific Neural Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating photorealistic 3D human models, such as computer graphics and 3D scanning, require significant time, cost, and expertise, and neural rendering methods struggle with high-resolution image generation and detailed rendering of small body parts, leading to blurred images and increased learning and rendering times.
Innovation Solution
A method that predicts appearance control parameters for each body part based on perspective projection and trains a neural rendering model to generate a photorealistic 3D model with controllable external deformation, using part-specific normalized images and control parameters to update a canonical 3D model and synthesize rendered images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If neural rendering method is used to generate 3D model, then operators' intervention is reduced and realistic 3D image is generated, but image quality becomes blurred when external deformation occurs and learning/rendering time increases for high resolution
Solution Approach 1:
The patent divides the 3D model generation process into multiple neural rendering models, each specialized for specific body parts (head, hands, torso, legs). This segmentation allows each model to focus on rendering details of its designated region, preventing the blurring that occurs when a single model attempts to render the entire body at high resolution. The controlled canonical 3D model is also segmented into multiple body parts for targeted rendering.
2Manufacturing precision
If high resolution image rendering is attempted, then image quality improves, but learning time and rendering time increase significantly
Solution Approach 1:
By segmenting the rendering task across multiple specialized neural rendering models (head model, hands model, torso model, legs model), each model processes a smaller portion of the image at high resolution. This reduces the computational burden on each individual model, thereby decreasing both learning time and rendering time while maintaining high overall image quality.
Solution Approach 2:
The patent applies local quality by having different neural rendering models with different levels of detail specialization for different body parts. Each model is trained to render its specific body part with high fidelity, allowing the overall system to achieve high resolution rendering without requiring all models to process at maximum computational intensity simultaneously.
3Adaptability or versatility
If whole-body image is used for training, then comprehensive 3D model is generated, but details of small areas (hands, face) become blurred
Solution Approach 1:
The training process is segmented into multiple specialized neural rendering models, each trained on images and data specific to particular body parts. The head model is trained on head images, the hands model on hand images, and so forth. This segmentation ensures that each small body part receives dedicated training attention, preventing the blurring that occurs when a single model tries to learn all body parts from whole-body images.
Solution Approach 2:
The patent implements local quality by creating specialized neural rendering models for different body parts with varying levels of detail requirements. Each model is optimized to capture the specific characteristics and details of its designated body part, ensuring that small areas like hands and faces are rendered with appropriate detail while maintaining comprehensive whole-body coverage.
Data Source
AI summary
An electronic device for generating a 3D model and a method of operating the electronic device. The method includes: receiving an image including a person to be modeled; generating, from the image, part-specific normalized images including perspective projection characteristics for respective parts of a body of the person; outputting part-specific control parameters including part-specific appearance control parameters representing appearance of the person from the part-specific normalized images; updating a canonical 3D model in fixed pose and size by accumulating appearance information of the person based on the part-specific control parameters; receiving control information for controlling a 3D model of the person from a user and controlling the canonical 3D model based on the control information; generating part-specific rendered images of the 3D model based on the canonical 3D model; and generating a 3D model of the person by synthesizing the part-specific rendered images.