Multi-modal emotion recognition method for medical care robot

Abstract

A multi-modal emotion recognition method for a medical care robot comprises the steps that multi-modal emotion information is collected, and video information and audio information of a patient are collected; performing expression self-attention emotion feature extraction and action self-attention emotion feature extraction according to the video information, and performing voice self-attention emotion feature extraction and text self-attention emotion feature extraction according to the audio information; performing mutual attention mechanism-based emotion feature fusion on the four self-attention emotion features to obtain a complete multi-modal emotion feature; performing context emotion feature extraction based on a graph convolutional neural network on the multi-modal emotion features to obtain the multi-modal emotion features containing context information; sentiment classification recognition is carried out on the multi-mode sentiment features containing context information to obtain a sentiment label result; and performing voice interaction and display according to the emotion label result. According to the method, the emotion recognition accuracy of people can be improved.

Description

technical field [0001] The invention relates to the technical field of emotion recognition, in particular to a multimodal emotion recognition method for a medical care robot. Background technique [0002] With the continuous development of human-computer interaction technology, emotional interaction has become a key research content in human-computer interaction. Emotion recognition is the key to emotional interaction. Its purpose is to enable machines to perceive human emotional states and make robots more Humanize. Multimodal emotion recognition technology has broad application prospects in the field of medical care robots. Using various sensors mounted on the medical care robot, multi-modal signals containing emotional features such as facial expressions, behaviors, voice signals, etc. of patients are obtained, and deep learning and other methods are used to achieve feature extraction and fusion, and analyze and predict patients' emotions. It makes the medical care robo...

AI Technical Summary

Problems solved by technology

[0006] In order to overcome the problem that existing medical care robots lack multi-modal emotion recognition ability and multi-modal emotional information lacks context information, the present invention provides a multi-modal emotional feature that utilizes facial expressions, behavior actions, and voice signals, and introduces multi-modal A multi-modal emotion recognition method for medical care robots based on dynamic contextual emotional information

Images