Virtual Character Facial Expression Switching Synchronized With Speech

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional systems for controlling virtual characters require human operators to switch facial expressions, leading to increased costs and delayed timing that causes user incompatibility.

Innovation Solution

A system that automates facial expression switching in virtual characters by generating behavior instruction data based on player inputs, reducing the need for human operators and synchronizing expressions with speech in real-time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If manual switching of facial expressions is used, then facial expression control is achieved, but human costs increase and switching timing is delayed

Engineering Contradiction:

Improvefacial expression switchingVSAvoidswitching timing delay

Core Design Contradiction:

Extent of automationVSLoss of time

Solution Approach 1:

The system enables self-service automation by having the virtual character automatically select and switch facial expressions based on speech recognition results, eliminating the need for human operators to manually control expression switching while achieving synchronized timing with speech

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system uses speech recognition to detect what the performer is saying, then feeds this information back to automatically select appropriate facial expressions, creating a closed-loop control system that synchronizes expressions with speech content in real-time

Inventive Principle:

Principle #23Feedback

2Extent of automation

If manual switching of facial expressions is used, then facial expression control is achieved, but human costs increase

Engineering Contradiction:

Improvefacial expression switchingVSAvoidoperation system

Core Design Contradiction:

Extent of automationVSDevice complexity

Solution Approach 1:

The system replaces the mechanical manual operation of switching facial expressions with an automated information processing system that uses speech recognition and pre-defined expression mapping, substituting human labor with computational processes

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentEP3988189B1Program, method, and terminal device

Publication Date: 2025.12.10 COLOPL

Figure 1
Figure 2
Figure 3

AI summary

The present invention is to, on a basis of data including information capable of specifying a motion input by a performer who plays a character arranged in a virtual space and a speech of the performer, produce a behavior of the character, to enable the character viewed from a predetermined point of view in the virtual space to display on a predetermined display unit, and to enable a sound depending on the speech of the performer to output. A behavior of a mouth of the character in the virtual space is controlled based on the data, a next facial expression to be taken as the facial expression of the character is specified in accordance with a predetermined rule, and switching control is performed on the facial expression of the character in the virtual space to the facial expression, which is specified by the step of specifying, with lapse of time.