Emotion recognition method, system and device based on multi-modal feature fusion and medium

A feature fusion and emotion recognition technology, applied in the field of emotion recognition, can solve problems such as unconsidered, low recognition efficiency, high model complexity, etc., to achieve the effect of improving efficiency, improving accuracy, and reducing model complexity
CN113807249APending Publication Date: 2021-12-17GUANGZHOU UNIVERSITY
0 Cites 5 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
GUANGZHOU UNIVERSITY
Publication Date
2021-12-17

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an emotion recognition method, system and device based on multi-modal feature fusion and a medium, and the method comprises the steps: obtaining preset first voice information and corresponding first visual information, and carrying out the feature extraction of the first voice information and the first visual information, and obtaining a voice feature image and an expression feature image; performing feature fusion on the voice feature image and the expression feature image to obtain a first multi-modal feature, and constructing a training data set according to the first multi-modal feature; inputting the training data set into a pre-constructed convolutional neural network for training to obtain a trained multi-modal feature recognition model; and identifying the emotion of the person to be tested according to the multi-modal feature identification model. On one hand, the model complexity is reduced, the model training and emotion recognition efficiency is improved, on the other hand, the influence of the voice features and the expression features on the emotion recognition result of the model is considered, the emotion recognition accuracy is improved, and the emotion recognition method can be widely applied to the technical field of emotion recognition.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The present invention relates to the technical field of emotion recognition, in particular to an emotion recognition method, system, device and medium based on multimodal feature fusion. Background technique

[0002] Emotion recognition is an important part of realizing full human-computer interaction. Emotion recognition can be applied in many different fields. For example, emotion recognition can be used to monitor and predict fatigue status. The task of emotion recognition is challenging because human emotions lack temporal boundaries and different people express emotions in different ways. Despite the current rich experience in emotion recognition for inferring the subject's emotion from speech or other forms, such as visual information (facial gestures), the accuracy of single-modal emotion recognition is not high, and the generalization ability is poor .

[0003] With the advent of deep neural networks in the past decade, there have been many b...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More