On-Device Body Animation from One Image with Texture Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to create realistic animations of a person's body from a single photograph, lacking efficiency and accuracy in generating dynamic movements without requiring server-side computational resources.
Innovation Solution
A method and system for single image-based real-time body animation using a generic model trained on diverse datasets, involving segmentation, pose estimation, 3D model reconstruction, and texture mapping, allowing for dynamic pose generation and video creation on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing technologies are used to create body animation from a single photograph, then animation can be generated, but the process lacks efficiency and accuracy in generating dynamic movements
Solution Approach 1:
The patent segments the body into multiple joints and segments, dividing the complex animation task into manageable parts. Each joint can be independently controlled and animated, allowing for precise pose estimation and efficient dynamic movement generation while maintaining overall body coherence
Solution Approach 2:
The patent implements a dynamic animation system that generates realistic movements by applying forces and motions to the segmented body parts. The system creates dynamic poses and movements that evolve over time, enabling efficient and accurate animation of complex actions like dancing, fighting, and acrobatics
2Reliability
If server-side computational resources are used for body animation, then high-quality animation can be achieved, but internet connectivity and external resources are required
Solution Approach 1:
The patent implements a self-service animation system where the mobile device performs all necessary computational tasks locally. The system includes integrated neural networks and processing units that enable the device to independently execute pose estimation, 3D model reconstruction, and animation generation without requiring external server resources or internet connectivity
Solution Approach 2:
The patent replaces traditional server-based computational systems with an on-device neural network and processing architecture. This substitution eliminates the need for external computational resources while maintaining high animation quality through locally-executed deep learning models and optimized algorithms
3Measurement precision
If a generic model trained on diverse datasets is used, then accurate pose and texture rendering can be achieved, but the model complexity increases
Solution Approach 1:
The patent employs a universal generic model that can handle diverse body types, poses, and movements through a single integrated architecture. The model is trained on diverse datasets to recognize and render various human forms, enabling accurate pose and texture rendering across different scenarios without requiring multiple specialized models
Solution Approach 2:
The patent performs preliminary training of the generic model on diverse datasets during the development phase. This preliminary action prepares the model with pre-learned patterns and representations that enable accurate pose estimation and texture rendering in real-time applications without requiring complex on-device training procedures
Data Source
AI summary
Systems and methods for text and audio-based real-time face reenactment are provided. An example method includes receiving an input image including a body of a person, fitting a model to the body in the input image, where the model is configured to generate an output image including the body adopting a pose based on a set of pose parameters, generating, based on the input image and the model, a three-dimensional (3D) mesh of the body, generating a texture map for the 3D mesh, modifying the texture map to modify an appearance of at least a portion of the body, and generating, based on the modified texture map and the set of pose parameters, the output image of the body adopting the pose with the modified appearance.