Feature fusion and decision fusion mixed multi-modal emotion recognition method

A technology of emotion recognition and decision fusion, applied in character and pattern recognition, acquisition/recognition of facial features, biological neural network model, etc., can solve problems such as inability to process difference signals, and achieve good recognition effect, good variability and scalability Effect
CN112800875AInactive Publication Date: 2021-05-14BEIJING INSTITUTE OF TECHNOLOGYGY
6 Cites 12 Cited by

Patent Information

Authority / Receiving Office
CN Ā· China
Patent Type
Applications(China)
Current Assignee / Owner
BEIJING INSTITUTE OF TECHNOLOGYGY
Publication Date
2021-05-14
Estimated Expiration
Not applicable Ā· inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a feature fusion and decision fusion mixed multi-mode emotion recognition method, and belongs to the field of mode recognition and emotion recognition. The implementation method comprises the following steps: 1, constructing an image emotion recognition network by using a convolutional neural network framework, and obtaining image features and image emotion states; 2, constructing a text emotion recognition network by using a recurrent neural network framework, and obtaining text features and a text emotion state; and 3, constructing a multi-modal information fusion emotion recognition network, constructing a main classifier for fusing the image emotion state and the text emotion state and obtaining main emotion classification, constructing an auxiliary classifier for fusing the image features and the text features and obtaining auxiliary emotion classification, and fusing the main emotion classification and the auxiliary emotion classification to obtain final emotion classification. According to the method, information complementation among multi-modal information is utilized, the problem of low emotion recognition accuracy caused by factors such as information fuzziness or missing of single-modal information is avoided, and a new thought is provided for multi-modal data fusion and emotion recognition.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The present invention relates to the fields of data fusion, neural network, emotion recognition, etc., and more specifically, relates to a multi-modal information fusion emotion recognition method based on hybrid fusion. Background technique

[0002] Human beings express emotional information through multiple modalities such as facial expressions, gestures, voices, and language. Emotional behavior is an important indicator reflecting human satisfaction. With the development of artificial intelligence technology, emotion recognition is an important means to achieve good human-computer interaction. Emotion recognition is to obtain the mapping relationship between the external appearance characteristics of emotion and the internal emotional state by extracting the characteristics of emotional signals, so as to identify the internal emotional type of the recognized object. Emotion recognition has very broad application prospects in machine services, healt...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More