Soft Label Generation for Emotion Recognition Data Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional emotion recognition techniques face challenges in improving accuracy with limited learning data, as they require a large amount of data and can only utilize utterances with majority-voted correct labels for model learning, leading to decreased data availability and recognition performance.
Innovation Solution
The use of correct emotion soft labels, which assign probabilities to multiple emotion classes based on listener agreements, allowing for the inclusion of utterances without majority-voted labels in model learning and enhancing data utilization, thereby improving recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a time-series model based on deep learning (such as LSTM-RNN) is used to achieve high emotion recognition accuracy, then estimation accuracy is improved, but a large amount of learning data (tens of thousands of utterances) is required, which is very difficult to collect
Solution Approach 1:
The patent changes the parameter representation from hard labels (single emotion class) to soft labels (probability distribution over multiple emotion classes). This allows the model to learn from uncertain or ambiguous cases where listeners disagree, effectively increasing the usable training data without requiring tens of thousands of utterances. The soft label approach transforms the data requirement constraint by enabling gradient-based learning from partial or uncertain annotations.
2Reliability
If correct emotion labels are decided by majority vote among multiple listeners, then reliability of labels is improved, but only utterances with clear majority votes can be used for learning, causing loss of potentially useful data
Solution Approach 1:
The patent introduces soft labels as an intermediary representation between hard majority-vote labels and model training. Instead of directly using binary correct/incorrect label assignments, the soft labels provide a probabilistic bridge that preserves information from listeners who disagreed. This intermediary representation allows the model to learn from the distribution of listener opinions rather than discarding uncertain cases, thereby maintaining reliability while increasing data utilization.
3Device complexity
If hard emotion labels (single correct class) are used for model learning, then simplicity of learning process is maintained, but information about emotion similarities and listener disagreements is lost
Solution Approach 1:
The patent transforms the label parameter from discrete (single emotion class) to continuous (probability distribution). This parameter change enables the learning process to capture emotion similarities through the soft label distributions, where related emotion classes will have non-zero probabilities. Although this increases computational complexity slightly, it preserves valuable information about emotion relationships and listener uncertainty that hard labels would discard.
Data Source
AI summary
With correct emotion classes selected as correct values of an emotion of an utterer of a first utterance from among a plurality of emotion classes C1, . . . , CK by listeners who have listened to the first utterance, as an input, the numbers of times ni that emotion classes Ci have been selected as the correct emotion classes are obtained, and rates of the numbers of times nk to a sum total of the numbers of times n1, . . . , nK or smoothed values of the rates are obtained as correct emotion soft labels tk(s) corresponding to the first utterance.