Semi-Supervised Emotion Inference Using RNN and VAE
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current information processing devices face challenges in accurately inferring emotions from observational data, particularly when data from multiple sources, such as motion and voice, are incomplete or missing, as existing methods require labeled training data and may include unnecessary information in the inference process.
Innovation Solution
An information processing device employing semi-supervised learning using a combination of recurrent neural networks (RNN) and variational autoencoders (VAE) to infer emotions as latent factors from both motion and voice data, allowing for interpolation of missing data and correlation between different modalities, enabling emotion inference even when one type of data is missing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If semi-supervised learning with RNN and VAE is used to infer emotions from multiple data sources, then emotion inference accuracy is improved, but device complexity increases
Solution Approach 1:
The patent combines RNN and VAE into a unified semi-supervised learning framework that processes multiple data sources (motion, voice, etc.) simultaneously. The RNN component captures temporal dependencies while the VAE component learns latent representations, and their integration enables joint processing of heterogeneous data modalities to infer emotions with higher accuracy despite the increased system complexity
Solution Approach 2:
The patent introduces a latent variable z as an intermediary that connects multiple observational data sources. This latent variable serves as a mediator that captures the underlying emotional state, allowing the system to integrate information from diverse data modalities (motion data x1, voice data x2, etc.) through a common representation space, thereby improving inference accuracy while managing complexity through structured intermediate representation
2Reliability
If multiple data sources are correlated and missing data is interpolated, then robustness is improved, but loss of information increases
Solution Approach 1:
The patent uses the VAE's generative capability to create copies or imputations of missing data based on the learned latent representation. When certain observational data sources are missing, the model generates plausible synthetic data copies from the latent variable z, allowing the system to maintain robustness by compensating for missing information through statistically informed data reconstruction
Solution Approach 2:
The patent transforms the problem of missing data by changing the parameter space from direct observational data to latent variables. By operating in the latent space z rather than the original data space, the model can effectively handle missing observations through probabilistic inference and parameter estimation, thereby improving robustness while managing information loss through intelligent parameter transformation and inference
Data Source
AI summary
Information processing including, a first data acquirer that acquires first observational data; a second data acquirer that acquires second observational data; and a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model.