Morphable 3D Meshes Disentangle Identity and Expression Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing content generation techniques face challenges in creating diverse 3D assets due to the tight coupling of mesh details with specific users or configurations, and the fixed topology of meshes limits the generation of different types of images or objects, requiring additional tools for small movements or deformations.
Innovation Solution
A machine learning system that disentangles identity and expression by training models directly from 3D points, using multilayer perceptrons to generate and manipulate embeddings for geometry, expression, and color, allowing independent changes and morphing of 3D models with a topology-agnostic representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If meshes are tightly coupled to specific users or configurations, then mesh details can be precisely controlled, but it becomes difficult to obtain meshes for a variety of users and objects
Solution Approach 1:
The patent segments a mesh into multiple layers, where the first layer contains user-specific details and the second layer contains configuration-specific details. This segmentation allows the mesh to be adapted to different users and configurations by selectively applying or modifying specific layers, thereby maintaining manufacturing precision while improving adaptability.
2Stability of the object's composition
If meshes have a fixed topology, then structural stability is maintained, but it is not possible to generate different types of images or images of different objects
Solution Approach 1:
The patent introduces a morphing capability that allows the mesh topology to dynamically transition between different states. The system can morph between a first mesh and a second mesh by interpolating vertex positions, enabling the generation of different image types while maintaining structural stability during the transition process.
3Manufacturing precision
If additional tools or procedures are used for blending changes between vertex positions, then deformation accuracy can be improved, but system complexity increases
Solution Approach 1:
The patent merges the deformation and blending operations into a unified morphing process. By combining these functions into a single systematic approach that operates on the layered mesh structure, the system achieves accurate deformation without requiring separate complex tools or procedures for each operation.
Data Source
AI summary
Approaches presented herein provide systems and methods for disentangling identity from expression input models. One or more machine learning systems may be trained directly from three-dimensional (3D) points to develop unique latent codes for expressions associated with different identities. These codes may then be mapped to different identities to independently model an object, such as a face, to generate a new mesh including an expression for an independent identity. A pipeline may include a set of machine learning systems to determine model parameters and also adjust input expression codes using gradient backpropagation in order train models for incorporation into a content development pipeline.