Emotion Recognition via Crowdsourced Body Movement Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in automatically recognizing human emotional expressions from bodily movements in unconstrained environments due to the complexity and subjectivity of bodily expressions, lack of standard annotation labels, and technical limitations in pose estimation and data collection.
Innovation Solution
A scalable and reliable crowdsourcing approach for collecting in-the-wild emotion data, utilizing the Body Language Dataset (BoLD) and Laban Movement Analysis (LMA) features, combined with deep learning models to predict emotional expressions from body movements, enabling the development of an Automated Recognition of Bodily Expression of Emotion (ARBEE) system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional emotion recognition systems are used, then they can work in controlled lab settings, but they fail in real-world unconstrained environments due to lack of standard annotation labels and subjective interpretations
Solution Approach 1:
The patent introduces a crowdsourced annotation platform as an intermediary between the system and real-world video data. This platform enables diverse annotators to provide labels for bodily expressions in unconstrained environments, while the system aggregates and reconciles these subjective interpretations to create a reliable, standardized annotation framework that bridges controlled methodology with real-world applicability
Solution Approach 2:
The patent changes the annotation parameter framework from fixed, controlled lab parameters to flexible, context-aware parameters that can handle diverse real-world scenarios. By allowing annotators to label expressions based on contextual understanding rather than rigid pose constraints, the system achieves adaptability to unconstrained environments while maintaining annotation reliability through crowdsourced consensus
2Measurement precision
If body movement models are combined to improve prediction accuracy, then the system can better recognize emotional expressions, but the computational complexity and processing time increase
Solution Approach 1:
The patent segments the emotion recognition task into multiple specialized body movement models, each trained on specific subsets of bodily expressions or anatomical regions. These segmented models process different aspects of body language independently and then their outputs are combined through a coordination layer, achieving high accuracy while managing computational complexity through modular architecture
Solution Approach 2:
The patent performs preliminary actions by pre-processing and aligning multiple body movement models before combining them. The system pre-aligns feature spaces and normalizes outputs from different models, enabling more efficient combination while reducing the computational burden during the actual recognition task
3Adaptability or versatility
If multiple body movement models are trained on diverse datasets, then the system can handle various expression types, but the data collection and processing requirements increase
Solution Approach 1:
The patent creates a universal annotation framework that serves multiple functions: it labels diverse expression types, provides ground truth for training models, and establishes a standardized vocabulary for body movement analysis. This multi-functional approach enables the system to handle various expression types without proportionally increasing data collection requirements, as the same crowdsourced platform serves all training needs
Data Source
AI summary
An emotion analysis and recognition system including an automated recognition of bodily expression of emotion (ARBEE) system is described. The system may include program instructions executable by a processor to: receive a plurality of body movement models, each body movement model generated based on a crowdsourced body language dataset, calculate at least one evaluation metric for each body movement model, select a highest ranked body movement model based on the at least one metric calculated for each body movement model, combine the highest ranked body movement model with at least one other body movement model of the plurality of body movement models, calculate at least one evaluation metric for each combination of body movement models, and determine a highest ranked combination of body movement models to predict a bodily expression of emotion.