Curriculum-Guided CNN Training for Emotion Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional emotion recognition models trained on limited datasets suffer from design biases and require cumbersome manual tagging, leading to inaccurate emotion predictions in images.
Innovation Solution
A curriculum-guided, webly supervised training method using a convolutional neural network that leverages an emotion hierarchy and weakly labeled stock images to iteratively adjust weightings and improve emotion classification accuracy, reducing biases and eliminating the need for manual tagging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional fully supervised training with manually tagged images is used, then training data can be obtained, but the process is inefficient, cumbersome, and leads to model overfitting on dataset biases
Solution Approach 1:
The system uses self-supervised learning where the model automatically generates its own training labels by predicting emotion categories from image data without requiring manual annotation. The neural network learns to associate visual patterns with emotion categories through unsupervised pre-training and semi-supervised fine-tuning, eliminating the need for human annotators while maintaining training quality
Solution Approach 2:
The patent employs data augmentation techniques that create synthetic training data by transforming and copying existing images through various operations (rotation, flipping, color jittering, etc.). This generates diverse virtual training examples that replicate the complexity of real-world data without requiring additional manual labeling, effectively copying the learning patterns from limited annotated data
2Reliability
If conventional training datasets are used, then models can be trained, but strong design biases (positive and negative biases) are introduced leading to inaccurate emotion recognition
Solution Approach 1:
The training process is segmented into distinct phases: unsupervised pre-training on large unlabeled datasets to learn general visual representations, followed by semi-supervised fine-tuning on smaller labeled datasets. This segmentation allows the model to first learn from diverse unlabeled data, reducing bias exposure during the critical learning phase, before gradually adapting to labeled data with known biases
Solution Approach 2:
The patent converts the harmful effect of biased labeled data into a benefit by using bias as a regularization signal. The model is trained to recognize that certain labels (especially negative emotions) are underrepresented in conventional datasets, and this representation imbalance is leveraged to improve generalization performance on real-world data where such biases naturally occur
3Adaptability or versatility
If limited training datasets are used, then data collection is simplified, but the model lacks diversity of visual concepts and generalization ability
Solution Approach 1:
The patent transitions from two-dimensional labeled data (image + manual label) to multi-dimensional training representations by incorporating unsupervised pre-training on high-dimensional unlabeled data. This dimensional expansion allows the model to learn rich visual features from the entire image space without requiring labels, effectively increasing the training data volume in feature space while maintaining simplicity in data collection
Data Source
AI summary
Disclosed herein are embodiments of systems, methods, and products for a webly supervised training of a convolutional neural network (CNN) to predict emotion in images. A computer may query one or more image repositories using search keywords generated based on the tertiary emotion classes of Parrott's emotion wheel. The computer may filter images received in response to the query to generate a weakly labeled training dataset labels associated with the images that are noisy or wrong may be cleaned prior to training of the CNN. The computer may iteratively train the CNN leveraging the hierarchy of emotion classes by increasing the complexity of the labels (tags) for each iteration. Such curriculum guided training may generate a trained CNN that is more accurate than the conventionally trained neural networks.