Multimodal Emotion Sensing for Accurate Robot User State Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing psychological state information acquisition methods, such as those based solely on radio wave reflections from wireless devices, lack accuracy in determining user emotions.
Innovation Solution
A psychological state information acquisition apparatus that combines a camera for capturing images and a biological information acquirer using electromagnetic waves to enhance accuracy by utilizing multimodal information, including emotional and biological data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If psychological state information is acquired solely based on radio wave reflections from a wireless device, then the device complexity is reduced, but the measurement precision deteriorates
Solution Approach 1:
The patent combines multiple information acquisition methods (camera-based image capture and electromagnetic wave-based biological information acquisition) into a unified system. The camera captures facial images while the electromagnetic wave device measures biological signals, and both data streams are integrated to comprehensively determine psychological state, thereby improving measurement precision without excessive complexity increase
Solution Approach 2:
The robot system performs multiple functions using integrated sensors: it captures images for facial expression analysis, acquires biological information through electromagnetic waves, and synthesizes both data types to determine psychological states. This multi-functional approach allows a single system to handle diverse measurement tasks, improving accuracy while managing complexity
2Measurement precision
If multiple types of information (image and biological information) are combined to acquire psychological state information, then the measurement precision is improved, but the device complexity increases
Solution Approach 1:
The system divides the psychological state acquisition task into distinct functional modules: a camera module for capturing facial images, an electromagnetic wave module for acquiring biological information, and a synthesis module for integrating both data types. This segmentation allows each component to specialize in specific measurement tasks, improving overall precision while managing complexity through modular design
Solution Approach 2:
The robot serves as an intermediary that receives and processes both image data from the camera and biological information from the electromagnetic wave device. It synthesizes these diverse data types through pattern recognition algorithms, acting as a mediator that transforms multiple input streams into unified psychological state information, thereby improving measurement precision while managing system complexity
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The apparatus achieves improved accuracy in acquiring psychological states, enabling robots to provide enhanced therapeutic effects through tailored interactions based on user emotions.
Implementation Method 1
a camera (Im1) configured to capture an image (Im) of a user
Implementation Method 2
a biological information acquirer (vital sensor 14) configured to acquire biological information (B) of the user by using electromagnetic waves
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A psychological state information acquisition apparatus is provided for acquiring psychological state information of a user with excellent accuracy. The psychological state information acquisition apparatus includes a camera configured to capture an image of a user, a biological information acquirer configured to acquire biological information of the user by using electromagnetic waves, and an information outputter configured to output psychological state information, which is information related to a psychological state of the user obtained based on the image of the user captured by the camera and the biological information acquired by the biological information acquirer.