Multi-modal emotion analysis method and system based on deep learning for acupuncture

Abstract

The invention provides a multi-modal emotion analysis method and system based on deep learning for acupuncture, and belongs to the technical field of emotion recognition. According to the method, feature extraction is carried out on facial expression features by adopting Gabor wavelet transform, feature extraction and dimension reduction processing are carried out on electroencephalogram signals by adopting UPLBP, then sparse linear fusion is carried out on the facial expression features and the electroencephalogram signal features to fuse the facial expression features and the electroencephalogram signal features into a unified and normalized feature vector, and the feature vector is transformed into a tensor form; training is carried out in a CNN-LSTM network, redundant information is removed and predicted emotion classification information is obtained, and a loss function and a correct rate of the network ar calculated by comparing the predicted emotion classification information with actual emotion classification information. According to the invention, the expression features and the electroencephalogram signal features are fused, redundant information is removed, training isconducted through the CNN-LSTM network, the predicted emotion classification information is obtained, and the emotion recognition accuracy is improved.

Description

technical field [0001] The present invention relates to the technical field of emotion recognition, in particular to a deep learning-based multimodal emotion analysis method and system for acupuncture and moxibustion. Background technique [0002] Emotion recognition is an important branch in the field of image recognition, which is the process of identifying people's emotions based on their external expressions or internal physiological signals. Although emotion is an internal subjective experience, it is always accompanied by some external manifestation, that is, certain behavioral characteristics that can be observed, which will be expressed in people's face, posture and tone of voice. At the same time, in addition to external manifestations, emotions also generate physiological signals related to emotions in the human body. Heart rate and EEG signals are physiological signals caused by emotions. Emotional analysis and recognition involves multiple disciplines, including...

AI Technical Summary

Problems solved by technology

[0008] 1. Existing emotion recognition methods, although non-physiological expressions such as facial expressions and voice are combined with physiological factors such as EEG and heart rate for emotion recognition, but among them, the collection of voice signals will cause interference problems such as EEG and heart rate , which affects the accuracy of emotion recognition

[0009] 2. Existing emotion recognition methods use methods such as PCA and LBP to process eigenvalues. There are random and empirical dimensions for dimension reduction, and at the same time, it will bring problems such as incomplete image edge feature extraction information.

[0010] 3. The existing emotion recognition method adopts linear fusion method for the fusion of feature values, and there are problems such as the immobilization of different feature weights, and the influence of each feature weight caused by personality factors is not considered.

[0011] 4. Existing clinical acupuncture studies on the emotional changes of patients or normal people before and after acupuncture mostly use subjective evaluation methods such as scale evaluation, lacking digital and quantitative indicators

Images