Automated Mouth Shape Animation via Viseme Intensity Parsing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for generating mouth shape animations in animation scenarios require manual production by artists and animators, resulting in low efficiency and significant production time.

Innovation Solution

A method and apparatus for generating mouth shape animations by performing feature analysis on target audio to generate viseme feature flow data, parsing each set of viseme feature data to obtain viseme information and intensity information, and controlling a virtual face to produce corresponding mouth shape animations, thereby automating the animation generation process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual production of mouth shapes and animations by artists and animators is used, then the quality and control of animation can be maintained, but the production time increases and efficiency decreases

Engineering Contradiction:

Improveanimation generation efficiencyVSAvoidproduction time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent replaces the manual mechanical process of artists creating mouth shapes with an automated system that performs feature analysis on audio to generate viseme feature flow data, which then automatically controls the virtual face to produce mouth shape animations

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service animation generation by automatically analyzing audio input and generating corresponding mouth shape animations without requiring manual intervention from artists or animators

Inventive Principle:

Principle #25Self-service

2Productivity

If automated audio-based animation generation is implemented, then production efficiency improves and time is reduced, but the complexity of the generation system increases

Engineering Contradiction:

Improveanimation generation efficiencyVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the animation generation process into distinct modules: feature analysis module that processes audio, parsing module that extracts viseme information and intensity information, and control module that animates the virtual face, making the complex system more manageable and maintainable

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces viseme feature flow data as an intermediary between audio input and animation output, serving as a structured representation that bridges the gap between raw audio features and animation control parameters

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20240203015A1Mouth shape animation generation method and apparatus, device, and medium

Publication Date: 2024.06.20 TENCENT TECHNOLOGY (SHENZHEN) CO LTD

AI summary

A mouth shape animation generation method, system, apparatus, and computer-readable medium is described. The process may include: performing feature analysis based on a target audio, to generate viseme feature flow data; the viseme feature flow data including a plurality of sets of ordered viseme feature data; each set of viseme feature data being corresponding to one audio frame in the target audio (202); separately parsing each set of viseme feature data, to obtain viseme information and intensity information corresponding to the viseme feature data; the intensity information being used for characterizing a change intensity of a viseme corresponding to the viseme information (204); and controlling, according to the viseme information and the intensity information corresponding to the sets of viseme feature data, a virtual face to change, so as to generate a mouth shape animation corresponding to the target audio (206).