Deep learning-based multimodal emotion analysis method and system for acupuncture and moxibustion

Abstract

The invention provides a deep learning-based multimodal emotion analysis method and system for acupuncture and moxibustion, belonging to the technical field of emotion recognition. The present invention uses Gabor wavelet transform for feature extraction on facial expression features, uses UPLBP to perform feature extraction on EEG signals, performs dimensionality reduction processing, and then uses sparse linear fusion of facial expression features and EEG signal features to form a unified , normalized eigenvectors, and transform eigenvectors into tensor form, input CNN‑LSTM network for training, remove redundant information and obtain predicted emotion classification information, and calculate network loss by comparing predicted emotion classification information with actual emotion classification information function and accuracy. The present invention fuses expression features and EEG signal features, removes redundant information, and obtains predicted emotion classification information through CNN-LSTM network training, thereby improving the accuracy of emotion recognition.

Description

technical field [0001] The present invention relates to the technical field of emotion recognition, in particular to a deep learning-based multimodal emotion analysis method and system for acupuncture and moxibustion. Background technique [0002] Emotion recognition is an important branch in the field of image recognition, which is the process of identifying people's emotions based on their external expressions or internal physiological signals. Although emotion is an internal subjective experience, it is always accompanied by some external manifestation, that is, certain behavioral characteristics that can be observed, which will be expressed in people's face, posture and tone of voice. At the same time, in addition to external manifestations, emotions also generate physiological signals related to emotions in the human body. Heart rate and EEG signals are physiological signals caused by emotions. Emotional analysis and recognition involves multiple disciplines, including...

AI Technical Summary

Problems solved by technology

[0008] 1. Existing emotion recognition methods, although non-physiological expressions such as facial expressions and voice are combined with physiological factors such as EEG and heart rate for emotion recognition, but among them, the collection of voice signals will cause interference problems such as EEG and heart rate , which affects the accuracy of emotion recognition

[0009] 2. Existing emotion recognition methods use methods such as PCA and LBP to process eigenvalues. There are random and empirical dimensions for dimension reduction, and at the same time, it will bring problems such as incomplete image edge feature extraction information.

[0010] 3. The existing emotion recognition method adopts linear fusion method for the fusion of feature values, and there are problems such as the immobilization of different feature weights, and the influence of each feature weight caused by personality factors is not considered.

[0011] 4. Existing clinical acupuncture studies on the emotional changes of patients or normal people before and after acupuncture mostly use subjective evaluation methods such as scale evaluation, lacking digital and quantitative indicators

Images