Bimodal emotion recognition method and system based on 3D convolutional neural networks

A technology of convolutional neural network and emotion recognition, which is applied in the field of machine learning and pattern recognition, can solve problems such as the difficulty in establishing a human trunk motion feature library and facial expression feature library, and the inability to extract human body trunk motion features, and achieve strong representation capabilities and generalization ability, improving accuracy, and overcoming limitations
CN108596039AActive Publication Date: 2018-09-28NANJING UNIV OF POSTS & TELECOMM
7 Cites 43 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
NANJING UNIV OF POSTS & TELECOMM
Publication Date
2018-09-28

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a bimodal emotion recognition method and system based on 3D convolutional neural networks. According to the method, first, two 3D convolutional neural networks for face emotionrecognition and body gesture emotion recognition are constructed respectively, and network model parameters are optimized based on a training set and a verification set of a bimodal face and body gesture database; second, the two neural networks obtained after optimization are tested based on a test set of the bimodal face and body gesture database to obtain a face emotion recognition confusion matrix and a body gesture emotion recognition confusion matrix; and last, priori knowledge of the face emotion recognition confusion matrix and the body gesture emotion recognition confusion matrix isutilized to fuse bimodal recognition results of a newly input face video sequence and body gesture video sequence, and a bimodal emotion classification result is obtained. Through the method, the 3D convolutional neural networks and a bimodal fusion algorithm are adopted, the subjectivity of manual feature design is avoided, the limitation of single-modal emotion recognition is overcome, and the accuracy and robustness of emotion recognition can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of machine learning and pattern recognition, and relates to a video emotion recognition method and system, in particular to a dual-mode emotion recognition method and system based on a 3D convolutional neural network. Background technique

[0002] With the rapid development of science and technology, human beings rely more and more on computers, and the ability of human-computer interaction has been valued by researchers. One of the important goals of the development of computer science is how to realize the anthropomorphic computer, which has become a hot issue in this field. A key problem that needs to be solved in human-computer interaction is to realize the computer's emotion recognition ability.

[0003] Emotion recognition ability is an important aspect of computer intelligence, which reflects the computer's ability to judge the emotional state of the operator or interlocutor through the acquired information. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More