Automated 3D Facial Animation via Landmark Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The animation of facial expressions for 3D models is technically challenging and often requires extensive manual processes, making it inefficient and time-consuming.

Innovation Solution

A system and method that identifies landmarks from captured images, calculates morph target weights based on facial expressions, and animates 3D models using a combination of deep learning approaches and morph target blending techniques, allowing for automated and efficient animation of facial expressions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual animation processes are used for 3D model facial expressions, then animation quality can be controlled, but the process becomes time-consuming and inefficient

Engineering Contradiction:

Improveanimation efficiencyVSAvoidanimation time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical animation processes with an automated computer vision system that uses landmark detection, expression classification, and morph target blending to generate facial animations automatically from images, eliminating time-consuming manual keyframing while maintaining animation quality

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system captures facial expressions from real-world images by detecting landmarks and copying the expression data to drive the 3D model's morph targets, creating accurate facial animations without manual intervention by directly transferring expression characteristics from source images to the animated model

Inventive Principle:

Principle #26Copying

2Productivity

If automated landmark detection and expression classification are implemented, then animation productivity increases, but system complexity increases

Engineering Contradiction:

Improveanimation automationVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent breaks down the complex animation task into distinct modular components: landmark detection module, neutral frame identification module, expression classification module, and morph target blending module. Each module handles a specific aspect of the process, making the overall system more manageable and easier to implement despite the increased automation capability

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system employs universal algorithms for landmark detection and expression classification that can process various types of facial images and expressions through a single integrated framework, reducing the need for multiple specialized systems and thereby managing complexity while maintaining high productivity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10198845B1Methods and systems for animating facial expressions

Publication Date: 2019.02.05 LOOMAI INC

AI summary

Systems and methods for animating expressions of 3D models from captured images of a user's face in accordance with various embodiments of the invention are disclosed. In many embodiments, expressions are identified based on landmarks from images of a user's face. In certain embodiments, weights for morph targets of a 3D model are calculated based on identified landmarks and/or weights for predefined facial expressions to animate expressions for the 3D model.