Single-Image 3D Body Animation for Real-Time Mobile Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for body animation from a single image lack the ability to create realistic and efficient animations in real-time without requiring server-side computational resources, particularly on mobile devices.
Innovation Solution
A method and system for single image-based real-time body animation using a computing device that segments an input image, fits a generic 3D model to the image, applies pose parameters to generate animated frames, and generates a video featuring the animated body, utilizing neural networks for segmentation and pose estimation, and a motions database for pose parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images are captured and processed to generate body animation, then the animation quality and accuracy are improved, but the processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary actions by capturing multiple images in advance and pre-processing them to extract body keypoint information before real-time animation generation. This allows the computationally intensive image processing to be completed beforehand, so that only lightweight processing is needed during real-time animation generation, resolving the contradiction between animation accuracy and processing time.
Solution Approach 2:
The system segments the animation generation process into distinct stages: image capture phase, keypoint extraction phase, and animation generation phase. By dividing the workflow, the system can perform detailed analysis on captured images without affecting real-time performance during the actual animation rendering, thus maintaining both high accuracy and efficient processing.
2Measurement precision
If multiple images are captured and processed to generate body animation, then the animation quality and accuracy are improved, but the device complexity increases
Solution Approach 1:
The system introduces an intermediary component that extracts body keypoint information from captured images. This intermediary layer simplifies the overall processing complexity by converting complex image data into structured keypoint representations, which are then easier to process for animation generation, thus maintaining animation accuracy while reducing device complexity.
Solution Approach 2:
The system uses pre-processed keypoint information as a simplified copy or representation of the original complex image data. Instead of processing full-resolution images during animation generation, the system works with extracted keypoint coordinates, which preserve the essential information needed for accurate animation while significantly reducing computational complexity.
3Measurement precision
If body keypoint information is extracted from captured images, then the animation generation accuracy is improved, but the processing time increases
Solution Approach 1:
The system performs keypoint extraction as a preliminary action during the image capture phase rather than during real-time animation generation. By extracting body keypoint information in advance when computational resources are more readily available, the system maintains high animation accuracy while improving processing efficiency during the actual animation rendering phase.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are systems and methods for single image-based body animation. An example method includes receiving a input image, the input image including a body of a person, segmenting the input image into a body portion and a background portion, wherein the body portion includes pixels corresponding to the body of the person, fitting a model to the body portion, wherein the model is configured to receive pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose, receiving a series of further pose parameters, each of the series of further pose parameters representing one of further poses of the body, providing each of the series of further pose parameters to the model to generate a series of output images of the body adopting the further poses, and generating, based on the series of output images, an output video.