A robot-oriented multi-modal fusion emotion computing method and system

Abstract

The present invention provides a robot-oriented multi-modal fusion emotion computing method, including: acquiring multi-modal information, capturing the language information and non-linguistic information of people interacting with the robot in real time; constructing processing channels for different information to perform feature classification and recognition, including feature classification and recognition of language information and non-linguistic information; process multi-modal information, and map information to PAD three-dimensional space through PAD model and OCC model; carry out the fusion of each modal information at the decision-making level Timing alignment, to calculate the emotional dimension space based on timing. The present invention also provides a robot-oriented multi-modal fusion emotional computing system. By acquiring the user's multi-modal information, including non-linguistic and linguistic information, using the PAD model, OCC model, and linear regression model to respectively After the language information is fused, the linear regression model is used for final fusion to achieve better and more accurate emotional computing for robots.

Description

technical field [0001] The invention relates to the field of information technology, in particular to a robot-oriented multimodal fusion emotion computing method and system. Background technique [0002] From the current point of view, there are few related studies on multi-modal fusion. The current methods have not achieved the fusion of multi-modal information, and most of them are language partial information. Most of the research has the following defects: 1. It is limited to the information collection and acquisition of a certain mode; 2. It only recognizes the language part, and cannot recognize the user's emotion well; 3. The non-language information part is only for the interaction object. Facial expressions are used for emotional computing, but various signals such as physiological information, facial expressions, body language, and visual information are not accurately integrated; 4. There is no fusion method of language and non-linguistic multimodal information, a...

AI Technical Summary

Problems solved by technology

[0002] From the current point of view, there are few related studies on multi-modal fusion. The current method does not achieve the fusion of multi-modal information, and most of them are language partial information.

Most of the research has the following defects: 1. It is limited to the information collection and acquisition of a certain mode; 2. It only recognizes the language part, and cannot recognize the user's emotion well; 3. The non-language information part is only for the interaction object. Facial expressions are used for emotional computing, but various signals such as physiological information, facial expressions, body language, and visual information are not accurately integrated; 4. There is no fusion method of language and non-linguistic multimodal information, and the corresponding emotional computing method ; 5. Robots basically do not use multi-modal affective computing

Images