Serial-parallel combined multi-mode emotion information fusion and identification method

Abstract

The present invention discloses a serial-parallel combined multi-mode emotion information fusion and identification method belonging to the emotion identification technology field. The method mainly comprises obtaining an emotion signal; pre-processing the emotion signal; extracting an emotion characteristic parameter; and fusing and identifying the characteristic parameter. According to the present invention, firstly, the extracted voice signal and facial expression signal characteristic parameters are fused to obtain a serial characteristic vector set, then M parallel training sample sets are obtained by the sampling with putback, and sub-classifiers are obtained by the Adabost algorithm training, and then difference of every two classifiers is measured by a dual error difference selection strategy, and finally, vote is carried out by utilizing the majority vote principle, thereby obtaining a final identification result, and identifying the five human basic emotions of pleasure, anger, surprise, sadness and fear. The method completely gives play to the advantage of the decision-making level fusion and the characteristic level fusion, and enables the fusion process of the whole emotion information to be closer to the human emotion identification, thereby improving the emotion identification accuracy.

Description

technical field [0001] The invention belongs to the technical field of emotion recognition, and in particular relates to a serial-parallel combination multi-mode emotion information fusion and recognition method. Background technique [0002] In recent years, research on emotion recognition has become a hot topic in the field of human-computer interaction. There are two main categories of research on emotion recognition at home and abroad, one is single-mode emotion recognition, and the other is multi-mode emotion recognition. The so-called single-mode emotion recognition is to obtain the emotional state of the current object only from a single information channel, such as voice signals, facial expression signals or physiological signals (blood pressure, body temperature, pulse, ECG, EEG, skin resistance, etc.). Although the research on emotion recognition relying solely on speech signals, facial expression signals or physiological parameters has achieved certain results, t...

AI Technical Summary

Problems solved by technology

These two methods have their own advantages and disadvantages. The fusion technology of the decision-making layer takes into account the different importance of different modes for emotion recognition, so they give different weights to different modes through subjective perception experiments, but this result obtained through subjective perception experiments It is doubtful whether the weights can be applied to other situations

The fusion technology of the feature layer is closer to the process of human recognition of emotion, and can make better use of statistical machine learning technology, but this method does not take into account the different importance of different patterns when recognizing different emotions, so this method cannot maximize Take full advantage of the advantages of multi-modal integration

It can be seen that the current research on emotional information fusion technology is still in its infancy, and the corresponding basic theory and method framework are still lacking.

Images