3D Morphable Model for Human Body Reshaping in Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video editing tools lack the capability to easily and efficiently manipulate the body shape and proportions of human actors in video footage, requiring hours of manual work even for skilled professionals, due to the complexity of performing holistic and spatiotemporally coherent edits.
Innovation Solution
A system using a 3D morphable model of human shape and pose learned from full body scans, allowing for interactive modification of shape attributes like height, waist girth, and muscularity, with marker-less motion estimation and image-based warping for spatiotemporal coherence, enabling quick and easy reshaping of human actors in video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual retouching operations are used to change body proportions, then visual perfection can be achieved, but the process requires hours of manual work
Solution Approach 1:
The patent uses a 3D morphable model as a digital copy of the human body that can be manipulated independently from the original video frames. This model captures the essential geometric structure and allows for parameter-based editing of body shape, pose, and proportions without requiring frame-by-frame manual retouching
Solution Approach 2:
The patent represents body shape and pose variations through a small number of semantic parameters (e.g., height, weight, muscularity, pose angles) that control the 3D morphable model. By adjusting these parameters, complex body transformations are achieved automatically, replacing hours of manual work with simple parameter adjustments
2Ease of operation
If local operations are used for body reshaping, then specific areas can be modified, but holistic body appearance correlation makes reshaping very hard
Solution Approach 1:
The patent segments the body into a parametric 3D model with distinct anatomical regions and joints, allowing local modifications to be applied to specific body parts while the overall model maintains holistic consistency through its structured representation of body geometry and pose
Solution Approach 2:
The 3D morphable model serves as an intermediary between local editing operations and the final holistic body appearance. Local modifications are applied to the model parameters, which then propagate through the entire model to ensure consistent global appearance, resolving the conflict between local ease of operation and holistic precision
3Ease of manufacture
If body shape manipulation is performed in a single video frame, then editing can be done, but spatiotemporal coherence in video requires additional complexity
Solution Approach 1:
The patent performs preliminary actions by tracking and fitting the 3D morphable model to each video frame in advance, establishing temporal consistency of body pose and shape throughout the sequence. This preliminary tracking creates a coherent temporal foundation that simplifies subsequent editing operations while maintaining spatiotemporal coherence
Solution Approach 2:
The patent uses a dynamic 3D morphable model that adapts to changing body poses and shapes across video frames. The model includes temporal smoothing and consistency constraints that automatically maintain spatiotemporal coherence during editing, allowing simple frame-by-frame operations to produce coherent video-wide results
Data Source
AI summary
The invention concerns a computer-implemented method for tracking and reshaping a human-shaped figure in a digital video comprising the steps: acquiring a body model of the figure from the digital video, adapting a shape of the body model, modifying frames of the digital video, based on the adapted body model and outputting the digital video.