Multi-mode emotion recognition and classification method

A technology of emotion recognition and classification method, applied in character and pattern recognition, acquisition/recognition of facial features, instruments, etc., can solve the problems of model irrelevance, low fusion efficiency, inability to extract spatiotemporal features from video data, etc., to improve fusion Efficiency and improved accuracy

Active Publication Date: 2018-03-16
BEIJING NORMAL UNIVERSITY
View PDF3 Cites 37 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem that the effective spatio-temporal features cannot be extracted from the video data for emotion recognition in the prior art, and whether the fusion in the early stage or the late stage is used in emotion recognition, similar fusion methods have the characteristics of model independence and are not sufficient. Using the effective information in each modality, there is a common problem of low fusion efficiency, and a multi-modal emotion recognition and classification method is provided.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-mode emotion recognition and classification method
  • Multi-mode emotion recognition and classification method
  • Multi-mode emotion recognition and classification method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0043] refer to figure 1 , figure 1 It is a flow chart of a multi-modal emotion recognition and classification method provided by an embodiment of the present invention, the method includes:

[0044] S1. Receive the data to be tested, the data to be tested includes a video containing a human face and a corresponding video containing body movements, preprocess the video containing a human face and the corresponding video containing body movements, and obtain a human face Image time series and body image time series.

[0045] Specifically, by receiving videos containing human facial expressions and videos containing body movements at the same time, after preprocessing the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-mode emotion recognition and classification method. The method comprises the following steps of: processing a to-be-detected video comprising a face and a correspondingvideo comprising a body action in a same time, so as to convert the videos into an image time sequence which consists of image frames; extracting time features and space features in the image time sequence; carrying out multi-feature level fusion on the features on the basis of the obtained multilayer deep time and space features; and carrying out decision level fusion on a classification result so as to recognize an emotion type of a task in the to-be-detected video from multi-mode. According to the method, effective information in each mode is sufficiently utilized and the emotion recognition rate is enhanced.

Description

technical field [0001] The present invention relates to the technical field of computer processing, and more specifically, to a multimodal emotion recognition and classification method. Background technique [0002] Emotion recognition is an emerging research field interdisciplinary in computer science, cognitive science, psychology, brain science, neuroscience, etc. Its research purpose is to let computers learn to understand human emotional expressions, and ultimately enable them to recognize, The ability to understand emotions. Therefore, as a very challenging interdisciplinary subject, emotion recognition has become a research hotspot in the fields of pattern recognition, computer vision, big data mining and artificial intelligence at home and abroad, and has important research value and application prospects. [0003] In the existing emotion recognition technology, the research trend of emotion recognition presents two obvious characteristics. On the one hand, the data...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00
CPCG06V40/174G06V40/172G06V40/168
Inventor 孙波何珺余乐军曹斯铭
Owner BEIJING NORMAL UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap