Suicide emotion perception method based on multi-modal fusion of voice and micro-expressions

A technology of emotion perception and micro-expression, applied in the field of emotion perception, can solve the problems of insufficient reliability of research results and achieve the effect of convenient operation and high performance
CN112101096APending Publication Date: 2020-12-18SOUTH CHINA UNIV OF TECH
0 Cites 4 Cited by

Patent Information

Authority / Receiving Office
CN ยท China
Current Assignee / Owner
SOUTH CHINA UNIV OF TECH
Publication Date
2020-12-18

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a suicide emotion perception method based on multi-mode fusion of voice and micro-expression. The method comprises the following steps: collecting videos and audios by using Kinect with an infrared camera; analyzing image frames and audios in the video by using different methods and converting the image frames and audios into corresponding feature texts; fusing the featuretexts, namely performing dimension reduction processing to obtain fused features; and classifying the fusion features by using a SoftMax activation function, and judging whether the emotion belongs tosuicide emotions or not. According to the invention, the multi-modal data is aligned with the text layer. The text intermediate representation and the proposed fusion method form a frame fusing speech and facial expressions. According to the invention, the dimensionality of voice and facial expressions is reduced, and two pieces of information are unified into one component. The Kinect is used for data acquisition, and the invention has the advantages of being non-invasive, high in performance and convenient to operate.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of emotion perception, in particular to a suicide emotion perception method based on multimodal fusion of voice and micro-expression. Background technique

[0002] In daily life, we often encounter some people who can't think about suicide, or even commit suicide because of a small unsatisfactory situation. This will cause huge psychological harm to those who love them, and also bring spiritual harm to the society. and material loss. In fact, these people will have corresponding physiological abnormalities such as speech, body, and expression before committing suicide. If we can carefully observe and understand them through some legitimate techniques, it may save a fresh life.

[0003] In addition to mastering the corresponding psychological knowledge and improving the popularization mechanism of psychological courses, it is also an effective method to use scientific and technological means to observe these abnormal ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More