Neural Pose Transfer via SPAdaIN Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for neural pose transfer between human meshes with different identities require extensive manual adjustments, auxiliary data, and retraining, limiting their generalization and efficiency, especially when dealing with new identities or noisy conditions.
Innovation Solution
A deep learning architecture using shared weights convolution layers and SPAdaIN (Spatially-Adaptive Instance Normalization) for neural pose transfer, which extracts pose features and applies them to target meshes without additional input, enabling generalization across different vertex orders and identities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional deformation transfer methods are used, then pose transfer between meshes with different identities is achieved, but extensive manual labeling and auxiliary data are required
Solution Approach 1:
The patent extracts and removes the requirement for manual landmark annotation and auxiliary mesh data from the pose transfer pipeline. By using only the source mesh and target mesh as inputs, the method eliminates the need for complex data preparation while maintaining pose transfer capability across different identities
Solution Approach 2:
The system performs pose transfer automatically without requiring external auxiliary data or manual intervention. The network learns to handle shape gaps and identity differences autonomously through its architecture, making the process self-sufficient and eliminating the need for user-annotated correspondences
2Extent of automation
If deep models are used for pose transfer, then automated pose transfer is achieved, but generalization ability is limited when source and target meshes have different identities
Solution Approach 1:
The patent designs a universal pose transfer network that can handle multiple identity types and shape variations without requiring retraining. The architecture is designed to be identity-agnostic, allowing the same model to transfer poses between different human identities, animals, or even fictional characters seamlessly
Solution Approach 2:
The method changes the approach by using adaptive instance normalization with style parameters that are learned during training. This allows the model to capture and generalize pose information while adapting to different identities and shapes, enabling automatic pose transfer across diverse targets without manual intervention
3Measurement precision
If auxiliary data and retraining are used for new identities, then pose transfer accuracy is maintained, but time consumption increases
Solution Approach 1:
The patent performs preliminary learning during training where the network is exposed to diverse identity data and learns to generalize pose transfer capabilities. This preliminary training enables the model to handle new identities accurately without requiring subsequent retraining, as the architecture is designed to be inherently adaptable to new targets
Data Source
AI summary
A network for neural pose transfer includes a pose feature extractor, and a style transfer decoder, wherein the pose feature extractor comprises a plurality of sequential extracting stacks, each extracting stack consists of a first convolution layer and an Instance Norm layer sequential to the first convolution layer. The style transfer decoder comprises a plurality of sequential decoding stacks, a second convolution layer sequential to the plurality of decoding stacks and a tan h layer sequential to the second convolution layer. Each decoding stack consists of a third convolution layer and a SPAdaIn residual block. A source pose mesh is input to the pose feature extractor, and an identity mesh is concatenated with the output of the pose feature extractor and meanwhile fed to each SPAdaIn residual block of the style transfer decoder. A system thereof is also provided.