Face Recognition via 3D Expression Normalization

Solution Overview

Problem

Existing face recognition methods perform poorly when facial expressions change, particularly in unconstrained environments, as they fail to effectively utilize information from all face regions and require large learning databases, limiting recognition accuracy.

Innovation Solution

A method that fits a two-dimensional face image into a three-dimensional face model, normalizes it to a neutral expression using learned parameters, and converts it back into a two-dimensional image for recognition, allowing for accurate face recognition without information loss and without modifying existing algorithms.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If existing face recognition methods are used, then recognition accuracy is excellent in neutral expression state, but performance deteriorates when facial expression changes

Engineering Contradiction:

Improveface recognition accuracyVSAvoidrobustness to expression changes

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent transforms two-dimensional face images into three-dimensional face models to capture depth information and facial geometry that are lost in 2D images. This dimensional transition enables the system to represent facial expressions more accurately and perform normalization in 3D space, thereby maintaining recognition accuracy across different expressions.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent applies expression normalization by adjusting parameters of the three-dimensional face model to transform various facial expressions into a neutral expression state. This parameter-based transformation allows the system to compensate for expression variations and maintain consistent recognition performance regardless of the original facial expression.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If region-based methods are used to avoid expression-changed areas, then recognition can be performed, but information from essential regions is limitedly used

Engineering Contradiction:

Improverecognition performanceVSAvoidinformation loss from face regions

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

By transitioning to three-dimensional modeling, the patent enables utilization of the entire face surface including regions that appear distorted in 2D images due to expressions. The 3D representation preserves geometric relationships and allows comprehensive use of facial information for recognition.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent performs expression normalization on the three-dimensional face model before recognition, transforming expression-affected regions into their neutral states in advance. This preliminary normalization ensures that all face regions, including those seriously changed by expressions, can be effectively used for recognition without information loss.

Inventive Principle:

Principle #10Preliminary action

3Quantity of substance

If three-dimensional modeling technique is used, then more face information can be captured, but expression changes are not yet dealt with

Engineering Contradiction:

Improveface information quantityVSAvoidrecognition accuracy under expression changes

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent applies expression normalization by adjusting parameters of the three-dimensional face model to transform various facial expressions into a neutral expression state. This parameter-based transformation allows the system to compensate for expression variations and maintain consistent recognition performance regardless of the original facial expression.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent performs expression normalization on the three-dimensional face model before recognition, transforming expression-affected regions into their neutral states in advance. This preliminary normalization ensures that all face regions, including those seriously changed by expressions, can be effectively used for recognition without information loss.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9811716B2Method for face recognition through facial expression normalization, recording medium and device for performing the method

Publication Date: 2017.11.07 KOREA INST OF SCI & TECH

AI summary

A method for face recognition through facial expression normalization includes: fitting an input two-dimensional face image into a three-dimensional face model by using a three-dimensional face database; normalizing the three-dimensional face model into a neutral-expression three-dimensional face model by using a neutral-expression parameter learned from the three-dimensional face database; converting the neutral-expression three-dimensional face model into a neutral-expression two-dimensional face image; and recognizing the neutral-expression two-dimensional face image from a two-dimensional face database. Accordingly, face recognition may be performed with high reliability without a loss of information.