3D Volumetric Mesh Editing with SMPL Retargeting and Keyframes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Editing and modifying 3D volumetric data generated from multi-view cameras is difficult due to its continuous nature and varying mesh topologies, requiring high cost and time to achieve consistent modifications.
Innovation Solution
A method involving the separation of head parts from volumetric data, generation of an SMPL model, and subsequent editing processes such as fitting, rigging, and retargeting to facilitate easier manipulation of the 3D mesh data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 3D volumetric data is continuously synthesized from multi-view images, then the appearance and motion of the subject are recorded vividly, but editing and modification become very difficult
Solution Approach 1:
The patent segments the continuous 3D volumetric data into discrete keyframe models at intervals. By dividing the temporal sequence into discrete editable units, editors can modify individual keyframes without having to process the entire continuous sequence, thus reducing editing difficulty while preserving the overall fidelity of the recorded subject.
Solution Approach 2:
The patent extracts specific frame data (keyframes) from the continuous volumetric sequence at predetermined intervals. This extraction creates discrete, manageable units that can be edited independently, solving the problem of difficult modification in continuous data while maintaining the essential appearance and motion characteristics of the subject.
2Manufacturing precision
If multiple 3D mesh models are modified to achieve consistent editing, then the desired modification is achieved, but the cost and time required increase significantly
Solution Approach 1:
By segmenting the volumetric sequence into discrete keyframe models, the patent enables editors to achieve consistent modifications by working on representative keyframes only. The modified keyframes then serve as templates for the entire sequence, dramatically reducing the time and cost compared to modifying every frame individually while maintaining consistency.
Solution Approach 2:
The patent creates representative keyframe models that can be copied and applied across multiple frames. By editing these copied keyframe templates, the modification propagates consistently throughout the sequence, reducing the manual effort and cost required compared to individual frame editing.
3Stability of the object's composition
If 3D mesh data with varying mesh topologies is synthesized continuously, then the data represents the subject accurately over time, but consistent modification requires high cost and long time
Solution Approach 1:
The patent segments the continuously synthesized volumetric data into discrete keyframe models with stabilized mesh topologies. By creating discrete units with consistent mesh structures at key intervals, the system reduces the complexity of handling varying topologies while maintaining temporal representation accuracy through the sequence of keyframes.
Data Source
AI summary
Disclosed is an editing method for edit 3D volumetric data to edit 3D mesh data in which head parts are removed from the volumetric data, an SMPL model is generated from the volumetric data, a 3D model is generated by transferring the head parts to the SMPL model, and then editing is performed, and includes: (a) receiving a 3D mesh sequence composed of a series of continuous frames; (b) estimating a 3D pose from a sequence of the 3D mesh; (c) separating a head part from a 3D mesh of a keyframe; (d) estimating a 3D mesh and pose data of an SMPL model; (e) generating a 3D base model of the keyframe; (f) editing the estimated 3D pose of the keyframes, and generating a sequence of 3D poses; and (g) editing the 3D base model to generate a 3D editing model and animating the 3D editing model.