Head Avatar Generation From Shot Video With Neural Implicit Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional avatar creation technology requires expertise or complex systems, leading to reduced graphic precision and difficulty in modifying avatars.
Innovation Solution
A method and device for creating a head avatar using a smartphone camera, involving neural implicit functions (NIF) to generate precise head meshes and textures from shot videos, optimizing differences between input images and rendering images, and allowing user-modifiable avatars.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional avatar creation technology is used, then expertise or complex systems are required, but this leads to reduced graphic precision and difficulty in modifying created avatars
Solution Approach 1:
The patent replaces conventional mechanical/complex system-based avatar creation with a neural network-based automated system. The neural network learns from input images to generate 3D head models, substituting the need for expert operators and complex manual systems with an intelligent automated system that achieves higher precision while reducing operational complexity
Solution Approach 2:
The patent uses 2D input images as copies or representations of the target subject's face, and through neural network processing, generates accurate 3D head model copies. This copying approach allows the system to capture precise facial features from simple images without requiring complex scanning equipment or expert intervention
2Adaptability or versatility
If conventional avatar creation technology is used, then complex systems are required, but this leads to difficulty in modifying created avatars
Solution Approach 1:
The patent segments the avatar creation process into distinct components: input image processing, neural network learning, 3D mesh generation, and texture mapping. This segmentation allows each component to be independently optimized and modified, enhancing adaptability while maintaining system manageability and reducing overall complexity
Solution Approach 2:
The patent creates a dynamic avatar system where the neural network can be retrained with new input images, and the generated 3D models can be modified by adjusting network parameters or input data. This dynamic nature allows easy adaptation and modification without requiring complex reconfiguration of the entire system
Data Source
AI summary
According to a method and device for generating a head avatar using a shot video, a head mesh of neutral look and a head texture map of neutral look are generated based on a base mesh of a base model and at least one shooting image representing a face of neutral look among a plurality of shooting images generated from a camera shot video, and a head mesh reflecting a certain facial expression and a head texture reflecting the certain facial expression are generated based on at least one shooting image representing a face of a certain facial expression among the plurality of shooting images, the head mesh of neutral look, and the head texture map of neutral look.