Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-modal emotion recognition method and system fusing voice and micro-expressions

An emotion recognition and micro-expression technology, applied in the field of emotion recognition, can solve the problems of low accuracy and large error of emotion recognition results, and achieve the effects of ensuring semantic integrity, reducing errors, and expanding comprehensiveness.

Pending Publication Date: 2021-02-02
JIANGXI UNIV OF SCI & TECH
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems of low accuracy and large errors in the emotion recognition results of the existing emotion recognition technology, the purpose of the present invention is to provide a multi-modal emotion recognition method and system that integrates speech and micro-expressions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal emotion recognition method and system fusing voice and micro-expressions
  • Multi-modal emotion recognition method and system fusing voice and micro-expressions
  • Multi-modal emotion recognition method and system fusing voice and micro-expressions

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] A multi-modal emotion recognition method that integrates speech and micro-expressions, such as figure 1 shown, including the following steps:

[0060] S101: Classify and process the historical micro-expression data, and establish a micro-expression database after setting the emotional fluctuation value for each classification sub-library in sequence according to the emotional development trend; among them, the emotional development trend is various, which can be determined according to the specific application scenario Settings, for example: excited-happy-happy-smile-calm-disappointed-worry-irritated-angry; the classification sub-library can contain one micro-expression category or multiple micro-expression categories with high similarity;

[0061] S102: Establish a speech emotion database according to the historical speech emotion representation vocabulary, and establish an emotion correlation function between speech and micro-expression according to the micro-expressi...

Embodiment 2

[0084] A multi-modal emotion recognition system that integrates speech and micro-expressions, such as figure 2 shown, including:

[0085] The expression database construction module is used to classify and process the historical micro-expression data, and establish the micro-expression database after setting the emotional fluctuation value for each classification sub-library in sequence according to the emotional development trend; the speech database construction module is used to Emotional representation vocabulary establishes a voice emotion database, and establishes the emotion correlation function between voice and micro-expression according to the micro-expression database and voice emotion database; the data processing module is used to simultaneously obtain the voice information and facial image information of the same target object, and After the information is preprocessed, the emotion representation vocabulary is extracted, and the facial image information is image...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multi-modal emotion recognition method and system fusing voice and micro-expressions, and relates to the technical field of situation recognition. The method comprises the steps of: establishing a voice emotion database and an emotion association function; acquiring voice information and face image information of the same target object at the same time, and extracting emotion representation vocabularies and micro-expression data; obtaining an emotion correlation function and an emotion fluctuation value corresponding to the micro-expression according to a matching result; establishing an emotion recognition network, and decomposing step by step to obtain a plurality of emotion recognition lines; obtaining a corresponding emotion fluctuation value, and establishingan emotion recognition curve; and selecting a qualified emotion recognition line according to a preset fluctuation degree after the emotion fluctuation degree is calculated. According to the invention, the authenticity of the real-time emotion of the target object represented by the voice information and the face image information is enhanced, the probability that the same situation reflects different situations is reduced, the accuracy of the emotion recognition result is improved, and the error of the emotion recognition result is reduced.

Description

technical field [0001] The invention relates to the technical field of emotion recognition, more specifically, it relates to a multi-modal emotion recognition method and system that integrates speech and micro-expressions. Background technique [0002] Emotion recognition is an emerging research field interdisciplinary in computer science, cognitive science, psychology, brain science, neuroscience, etc. Its research purpose is to let computers learn to understand human emotional expressions, and ultimately enable them to recognize, The ability to understand emotions. Therefore, as a very challenging interdisciplinary subject, emotion recognition has become a research hotspot in the fields of pattern recognition, computer vision, big data mining and artificial intelligence at home and abroad, and has important research value and application prospects. [0003] At present, there are two ways for emotion recognition, one is to detect physiological signals such as breathing, he...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06F40/289G06F40/30G10L25/63
CPCG06F40/289G06F40/30G10L25/63G06V40/174G06V40/172
Inventor 邓志娟许春冬钟少君唐明田
Owner JIANGXI UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com