Suicide emotion perception method based on multi-modal fusion of voice and micro-expressions

Abstract

The invention discloses a suicide emotion perception method based on multi-mode fusion of voice and micro-expression. The method comprises the following steps: collecting videos and audios by using Kinect with an infrared camera; analyzing image frames and audios in the video by using different methods and converting the image frames and audios into corresponding feature texts; fusing the featuretexts, namely performing dimension reduction processing to obtain fused features; and classifying the fusion features by using a SoftMax activation function, and judging whether the emotion belongs tosuicide emotions or not. According to the invention, the multi-modal data is aligned with the text layer. The text intermediate representation and the proposed fusion method form a frame fusing speech and facial expressions. According to the invention, the dimensionality of voice and facial expressions is reduced, and two pieces of information are unified into one component. The Kinect is used for data acquisition, and the invention has the advantages of being non-invasive, high in performance and convenient to operate.

Description

technical field [0001] The invention belongs to the field of emotion perception, in particular to a suicide emotion perception method based on multimodal fusion of voice and micro-expression. Background technique [0002] In daily life, we often encounter some people who can't think about suicide, or even commit suicide because of a small unsatisfactory situation. This will cause huge psychological harm to those who love them, and also bring spiritual harm to the society. and material loss. In fact, these people will have corresponding physiological abnormalities such as speech, body, and expression before committing suicide. If we can carefully observe and understand them through some legitimate techniques, it may save a fresh life. [0003] In addition to mastering the corresponding psychological knowledge and improving the popularization mechanism of psychological courses, it is also an effective method to use scientific and technological means to observe these abnormal ...

AI Technical Summary

Problems solved by technology

It can be seen that most emotion recognition methods focus on the study of a single factor, which is obviously one-sided, because some people can control their inner emotions well without expressing them, which will make the research results less reliable

Images