Realistic Digital Human Movement via AI Realization Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital human characters in the metaverse exhibit awkward movements due to limitations in muscle structure representation, high costs, and errors in motion capture and data refinement processes.
Innovation Solution
An apparatus and method that utilize a pretrained realization model to generate realistic movements for digital human characters by converting predetermined regions of a live-action video into corresponding regions of a realistically visualized 3D digital human video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion capture equipment and data refinement processes are used to improve digital human movements, then movement realism is improved, but cost and time consumption increase significantly
Solution Approach 1:
The patent creates virtual reference videos by rendering 3D digital human characters with realistic movements, then uses these virtual copies as training data to teach AI models. This copying approach eliminates the need for expensive motion capture equipment while achieving realistic movements through automated AI learning from synthesized reference data.
Solution Approach 2:
The patent replaces the mechanical motion capture system (physical sensors, markers, and equipment) with an AI-based system that learns movement patterns from video data. This substitution eliminates complex physical measurement devices while achieving similar or better results through machine learning.
2Reliability
If high-budget digital human models with detailed muscle structures are used, then movement quality is improved, but processing performance and terminal compatibility deteriorate
Solution Approach 1:
The patent transfers movement patterns from high-quality reference videos to simplified digital human models through AI learning. This allows lightweight models to replicate realistic movements without requiring the computational resources needed for high-detail models, achieving quality-performance balance.
Solution Approach 2:
The patent changes the approach from modifying model parameters (muscle structures, joint configurations) to learning movement patterns from video data. This parameter transformation allows simplified models to achieve realistic movements through AI-driven motion transfer rather than complex anatomical modeling.
3Measurement precision
If motion capture markers are attached to capture natural movements, then movement accuracy is improved, but errors accumulate through multiple processing stages
Solution Approach 1:
The patent creates virtual reference videos that serve as error-free training data, copying realistic movement patterns directly into the AI model without physical measurement errors. This eliminates marker attachment errors, tracking inaccuracies, and manual refinement mistakes that plague traditional motion capture pipelines.
Solution Approach 2:
The patent introduces AI models as an intermediary between reference videos and target digital humans. This intermediary learns and transfers movement patterns automatically, eliminating manual data refinement steps where errors typically accumulate during marker correction and joint position adjustment.
4Reliability
If professional designers manually refine motion capture data, then movement naturalness is improved, but productivity and cost decrease
Solution Approach 1:
The patent enables the AI model to automatically learn and refine movement patterns from video data without human intervention. The system serves itself by generating training data, learning patterns, and applying movements autonomously, replacing expensive manual refinement work with automated machine learning processes.
Solution Approach 2:
The patent copies natural movement patterns directly from video data into the AI model through automated learning, eliminating the need for professional designers to manually interpret and refine motion capture data. This copying process achieves naturalness automatically through AI pattern recognition.
Data Source
AI summary
Disclosed herein is an apparatus and method for realistic movements of a digital human character. The apparatus includes memory in which at least one program is recorded and a processor for executing the program. The program performs generating a first video by realistically visualizing a video in which a 3D digital human character is rendered and generating a second video by making movements in the first video realistic based on a pretrained realization model, and the realization model may be pretrained based on a fourth video that is generated by converting a predetermined region of a live-action video of a person into a corresponding predetermined region of a third video obtained by realistically visualizing a video in which a 3D digital human character is rendered.


