Multi-mode based emotion recognition method

An emotion recognition, multi-modal technology, applied in the direction of character and pattern recognition, multiple biometrics, biometrics recognition, etc., can solve the problems of people's facial expressions, spoken words, body, voice, intonation and physiological characteristics. , to achieve the effect of improving sentiment analysis, reducing costs, and improving ease of use

Active Publication Date: 2018-11-13
南京云思创智信息科技有限公司
View PDF7 Cites 129 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, it does not integrate people's facial expressions, spoken words, body posture, voice intonation and physiological characteristics, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-mode based emotion recognition method
  • Multi-mode based emotion recognition method
  • Multi-mode based emotion recognition method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] The invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0061] Because the production of any emotion is accompanied by certain changes in the body, such as facial expressions, muscle tension, and visceral activities. Using these signal changes to directly perform emotion recognition is the so-called basic recognition method, also known as a single-modal emotion recognition method. The current main methods include facial images, language, text, gestures and physiological signals. The present invention proposes a more complete and accurate emotion recognition method by integrating, corresponding and inferring the understanding of emotions by computers in each of the following single modes.

[0062] The multimodal emotion recognition method based on the present embodiment proposes, its system is made up of following several parts ( figure 1 It is a schematic diagram of a multimodal emotion recogniti...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-mode based emotion recognition method, comprising a data collection device, an output device and an emotion analysis software system, wherein the emotion analysis software system performs comprehensive analysis and reasoning on data obtained by the data collection device and finally outputs a result to the output device. The specific steps are as follows: an emotionrecognition step based on facial image expressions, an emotion recognition step based on voice signals, an emotion analysis step based on text semantic, an emotion recognition step based on human gestures, an emotion recognition step based on physiological signals, a semantic comprehension step based on multi-round dialogues, and a multi-mode emotion semantic fusion association judgment step basedon timing sequence. The multi-mode based emotion recognition method provided by the invention has the advantages of breaking through the five kinds of single-mode emotion recognition, innovatively performing comprehensive judgment on the information of multiple single modes by using a deep neural network through neural network coding, deep correlation and understanding, greatly improving the accuracy and being suitable for most general inquiry interaction application scenes.

Description

technical field [0001] The present invention relates to the technical field of emotion recognition, in particular, to multimodal emotion recognition methods based on machine learning, deep learning, computer vision, natural language processing, speech recognition, human action recognition, and non-contact physiological detection. Background technique [0002] Emotion recognition is a technology for judging people's emotional changes. It mainly collects people's external expressions and behavior changes to infer people's psychological state. In modern society, emotion recognition technology is widely used in the development of smart devices, sales guide robots, health management, advertising and marketing, etc. Emotion is a state that integrates people's feelings, thoughts, and behaviors. It includes people's psychological responses to external or self-stimuli, as well as the physiological responses that accompany such psychological responses. In various human-computer inter...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06F17/27G06K9/62G10L25/63G10L25/30
CPCG10L25/30G10L25/63G06F40/30G06V40/70G06F18/25
Inventor 俞旸凌志辉
Owner 南京云思创智信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap