Face Model Fitting for Marker Placement Accuracy

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for accurately placing facial markers for digital human motion tracking are prone to positional deviations, leading to degraded quality in computer graphic facial expression reproduction due to reliance on manual alignment and limited accuracy in image analysis.

Innovation Solution

An information processing apparatus and method that displays a distribution image of marker points on a performer's face image, fitting a face model to the image using position information from placed markers, and adjusting the image position based on fitting results to enhance marker placement accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If manual marker placement with reference image is used, then marker placement can be performed, but it takes time and positional deviation occurs

Engineering Contradiction:
Improvemarker placement accuracyVSAvoidmarker placement time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system displays the superimposed face model and marker distribution image in real-time during marker placement, providing visual feedback to the operator. This allows immediate verification of marker positions and enables corrective adjustments, thereby improving placement accuracy while maintaining efficient workflow

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

A computer-generated face model serves as an intermediary between the reference image and the actual marker placement on the performer's face. The model is superimposed on the live face image to provide precise anatomical guidance, eliminating the need for manual interpretation of static reference images and reducing positional deviation

Inventive Principle:
Principle #24Intermediary (Mediator)

2Manufacturing precision

If image analysis for feature point extraction is used, then marker placement time is reduced, but positional accuracy is limited due to image analysis accuracy

Engineering Contradiction:
Improvemarker placement accuracyVSAvoidimage analysis system complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system merges multiple approaches by combining automated feature point extraction from image analysis with manual marker placement guidance. The extracted feature points are superimposed on the live face image to assist the operator, leveraging both automated speed and human precision to achieve high accuracy while managing system complexity

Inventive Principle:
Principle #5Merging (Combining)

3Manufacturing precision

If three-dimensional model superimposition is used, then marker placement guidance is improved, but mechanism for exact superimposition is lacking

Engineering Contradiction:
Improvemarker placement accuracyVSAvoidsuperimposition alignment ease
Core Design Contradiction:
Manufacturing precisionVSEase of operation

Solution Approach 1:

The system provides real-time visual feedback by displaying the superimposed three-dimensional face model and marker distribution image overlaid on the performer's actual face. This allows the operator to verify alignment accuracy and make adjustments, making the complex superimposition process easy to control and validate

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS20240378833A1Information processing apparatus, information processing method, and program
Publication Date: 2024.11.14 SONY GROUP CORP
  • US20240378833A1 patent drawing
  • US20240378833A1 patent drawing
  • US20240378833A1 patent drawing

AI summary

An information processing apparatus (10) includes a display control unit (14) and a fitting unit (13). The display control unit (14) displays a distribution image indicating distribution of a plurality of marker points extracted from a face model of a performer in a superimposed manner on a face image of the performer. The fitting unit (13) fits the face model to the face image by using position information of a placed marker each time the marker is placed on a face of the performer after the distribution image is superimposed, and adjusts a superimposed position of the distribution image based on a fitting result.