Facial Expression Dataset Augmentation via Dimensional Emotion Space Morphing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited availability of large, balanced, and high-quality datasets for training and testing facial expression analysis systems, particularly for dimensional affect models, due to the difficulty in collecting and annotating images across various expressions and intensities, leads to imbalanced datasets and high costs.
Innovation Solution
A computer-implemented method that augments datasets by generating expression and intensity variation images through morphing existing facial images in a continuous dimensional emotion space, allowing for the creation of balanced and annotated datasets efficiently and cost-effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large sets of images are collected across many subjects and expressions, then dataset quantity increases, but annotation cost and time increase prohibitively
Solution Approach 1:
The patent generates synthetic facial expression images by morphing between existing apex expression images, creating copies of intermediate expression states without requiring actual photos or manual annotation for each expression level
Solution Approach 2:
The method pre-processes apex expression images to establish dimensional embeddings and then automatically generates intermediate expression images with predicted labels, eliminating the need for time-consuming manual annotation of each expression intensity level
2Measurement precision
If experienced annotators are hired to label large corpus of images, then annotation quality improves, but cost and complexity increase prohibitively
Solution Approach 1:
The patent replaces the mechanical process of manual annotation by experienced annotators with an automated machine learning system that computes dimensional embeddings and generates expression images with predicted labels based on learned patterns from training data
3Reliability
If multiple annotations per image are required, then reliability of emotion labels improves, but cost and complexity increase further
Solution Approach 1:
The system generates multiple synthetic variations of facial expressions through morphing operations, creating multiple annotated samples that represent different expression intensities and variations without requiring multiple manual annotations of the same image
4Adaptability or versatility
If continuous dimensional emotion labels are used, then expression intensity coverage improves, but difficulty of assessment and assignment increases
Solution Approach 1:
The patent replaces the difficult manual assessment of continuous dimensional emotion labels with an automated system that computes embeddings in a pre-defined dimensional space and automatically assigns labels based on the position of morphed images within that space
Solution Approach 2:
The system transforms the complex task of assessing continuous dimensional emotion labels into a simpler parameter-based approach where labels are derived from the position of images in a pre-established dimensional embedding space, making label assignment automatic and consistent
Data Source
AI summary
In a computer-implemented method of augmenting a dataset used in facial expression analysis, a first facial image and a second facial image are added to a training/testing dataset and mapped to two respective points in a continuous dimensional emotion space. The position of a third point in the continuous dimensional emotion space between the first two points is determined. Augmentation is achieved when a labelled facial image is derived from the third point based on its position relative to the first and second facial expression.