3D Human Image Generation With Tri-Plane Pose Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating images of virtual humans using neural networks are limited to rigid object modeling, which restricts quality and controllability, or are not generative, failing to synthesize novel identities and appearances.
Innovation Solution
A generator neural network that processes camera and pose parameters to generate a 3D representation of a human in a desired pose, using a tri-plane representation and deformation networks to produce high-quality, customizable images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If classical graphics modeling and rendering techniques are used, then high-quality images are achieved, but the process requires pre-captured templates, multi-camera systems, controlled studios, and long-term work of artists
Solution Approach 1:
The patent replaces classical graphics modeling and rendering techniques with a neural network-based system. The neural network learns to generate photorealistic images of humans in arbitrary poses by training on image datasets, eliminating the need for complex multi-camera systems, controlled studios, and manual artist work while maintaining high image quality
Solution Approach 2:
The patent transforms the input representation from fixed templates to parametric human models with controllable pose parameters and appearance parameters. This allows the system to generate images with desired poses and appearances by adjusting parameters rather than requiring pre-captured templates and manual manipulation
2Ease of manufacture
If neural networks are used for image synthesis, then low cost is achieved, but known methods are limited to rigid object modeling or single subject representations
Solution Approach 1:
The patent implements a dynamic pose representation system where the neural network processes pose parameters to generate images of humans in arbitrary poses. The system uses a parametric human model that can be deformed to different poses through parameter adjustments, enabling flexible and generative image synthesis rather than being limited to rigid objects or single fixed poses
Solution Approach 2:
The patent creates a universal neural network system that can handle both rigid object modeling and articulated human representations. The system processes camera parameters, pose parameters, and appearance parameters to generate diverse images of different human identities and appearances in various poses, making it applicable to multiple tasks including virtual try-on, avatar creation, and image editing
3Ease of operation
If articulated human representations are learned for single subjects, then pose control is achieved, but the method is not generative and cannot synthesize novel identities and appearances
Solution Approach 1:
The patent separates the image generation process into distinct parameter components: camera parameters for view control, pose parameters for body configuration, and appearance parameters for identity and appearance characteristics. This segmentation allows independent control of pose while maintaining generative capability for novel identities through the appearance parameter space
Solution Approach 2:
The patent introduces an intermediate parametric human model representation that serves as a bridge between input parameters and final image generation. The neural network processes camera parameters, pose parameters, and appearance parameters through this intermediate representation to generate images with desired poses and novel appearances, enabling both pose control and generative capability
Data Source
AI summary
Camera parameters describing a view angle, and pose parameters describing a shape and a pose of a parametric human body model, are processed to generate geometry information (which characterizes a 3D geometry of the human), and the appearance information (which characterizes a RGB appearance of the human). These in turn are processed to generate the image of the human. In the image, the human is depicted viewed from the view angle and with the body of the human having the shape and the pose described by the pose parameters.