Hierarchical Face Alignment Model Selection for Landmark Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current face alignment technologies face challenges in accurately locating facial key points across varying facial rotations and expressions, leading to reduced accuracy and robustness in applications like face recognition and 3D reconstruction.

Innovation Solution

A multi-task convolutional neural network-based system that hierarchically categorizes facial characteristics, selects a corresponding face alignment model, and uses cascaded regression to obtain precise facial landmarks, addressing the limitations by integrating face detection, pose estimation, and expression analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single face alignment model is used for all facial images, then the device complexity is reduced, but the measurement precision of facial landmarks deteriorates across varying facial rotations and expressions

Engineering Contradiction:

Improvenumber of face alignment modelsVSAvoidaccuracy of facial landmark detection

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent segments the face alignment task by dividing facial characteristics into hierarchical categories (first facial characteristic categories with multiple first levels, and second facial characteristic categories). Different face alignment models are trained for different segments (different combinations of facial characteristic categories), allowing specialized processing for each segment while maintaining overall system accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically selects which face alignment model to use based on the detected facial characteristics category set. The selection process adapts to the specific input image by identifying relevant facial characteristics and choosing the corresponding specialized model, making the system flexible and context-aware rather than static.

Inventive Principle:

Principle #15Dynamics

2Measurement precision

If multiple face alignment models are used for different facial characteristics, then the measurement precision improves, but the device complexity increases

Engineering Contradiction:

Improveaccuracy of facial landmark detectionVSAvoidnumber of face alignment models

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

Each face alignment model is designed to handle multiple facial characteristic categories simultaneously. The models are trained on multi-task convolutional neural networks that can process various facial characteristics (pose, expression, etc.) within a single model, reducing the total number of models needed while maintaining comprehensive coverage.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent applies local quality by training different face alignment models with different levels of detail and complexity appropriate for specific facial characteristic categories. Models dealing with more complex characteristics use more sophisticated processing, while simpler characteristics use more efficient models, optimizing the overall system performance.

Inventive Principle:

Principle #3Local quality

3Speed

If face alignment is performed without hierarchical categorization, then the processing speed is faster, but the adaptability to different facial conditions deteriorates

Engineering Contradiction:

Improveprocessing speed of face alignmentVSAvoidrobustness across facial rotations and expressions

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary action by detecting facial characteristics and categorizing them into hierarchical categories before selecting the appropriate face alignment model. This pre-processing step enables the system to quickly identify the relevant model and apply it, avoiding the need to evaluate all possible models and improving both speed and adaptability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a hierarchical categorization dimension for facial characteristics, organizing them into multiple levels (first levels and second levels) and categories. This additional dimensional structure allows the system to efficiently navigate and select models based on the specific combination of characteristics present, improving adaptability without sacrificing processing speed.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12033364B2Method, system, and computer-readable medium for using face alignment model based on multi-task convolutional neural network-obtained data

Publication Date: 2024.07.09 GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD

AI summary

A method includes receiving a facial image; receiving a facial image; and obtaining, using a multi-task convolutional neural network, a detected face location and a facial characteristic category set of a plurality of first facial characteristic categories; selecting a first face alignment model from a plurality of face alignment models based on the facial characteristic category set; and obtaining, using the first face alignment model, a plurality of facial landmarks. The first facial characteristic categories are arranged hierarchically. A hierarchy of the first facial characteristic categories includes a plurality of first levels corresponding to a plurality of corresponding facial characteristics. The facial characteristic category set includes the first facial characteristic categories of a path of the hierarchy of the first facial characteristic categories.