Psychological stress analysis method based on multi-modal fusion

A technology of psychological pressure and analysis method, applied in the field of emotion recognition, to achieve the effect of providing accuracy

Active Publication Date: 2021-08-13
INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF10 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] Existing technologies mainly model and analyze psychological pressure on facial expressions, voice, and physiological signals, and then make fusion decisions to obtain psychologi

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Psychological stress analysis method based on multi-modal fusion
  • Psychological stress analysis method based on multi-modal fusion
  • Psychological stress analysis method based on multi-modal fusion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0069] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as recited in the appended claims.

[0070] Such as figure 1 The multimodal fusion psychological stress analysis method provided by the embodiment of the present application includes:

[0071] S11: Segment the long audio and video into short audio and video with faces and voices, and process the short audio and video into frames to obtain t image sequences and t frames of voice signals;

[0072] S11-1: Input audio and video files...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-modal fusion psychological stress analysis method, which comprises the following steps of: segmenting a long audio/video into a short audio/video with a face and voice, and framing the short audio/video to obtain an image sequence and a voice signal; performing facial feature extraction on the image sequence to obtain a facial frame sequence; applying an optical flow method to extract optical flows from adjacent frames of the face frame sequence to obtain an optical flow sequence; fusing a face frame sequence and the optical flow sequence, and performing linear mapping to obtain a face embedding vector; extracting a region of interest from the image sequence to obtain a sequence of interest, and performing linear mapping to obtain a physiological signal embedding vector; extracting basic acoustic characteristics of the voice signal by taking a frame as a unit, and performing linear mapping to obtain an acoustic embedded vector; extracting emotional features from the voice signal and the image sequence; fusing the features according to the time sequence of the frame sequence to obtain a spatio-temporal feature vector; and inputting the spatio-temporal feature vectors into a model, and performing softmax classification to obtain a psychological stress level.

Description

technical field [0001] This application relates to the field of emotion recognition, and in particular to a method for analyzing psychological stress with multimodal fusion. Background technique [0002] Facial features include facial expression features and facial movement characteristics. Facial expressions and facial movements revealed by people inadvertently can express people's true psychological state. There is a correlation between changes in micro-expressions such as eyebrows, mouth, eyes, and forehead and their psychological stress. At the same time, facial movements when people speak are also correlated with their psychological stress. [0003] Voice features mainly include voice prosody features, voice spectrum features, and voice quality features; voice prosody features are acoustically expressed as fundamental frequency, duration, and energy parameters; voice spectrum features include spectrum, spectrum envelope, cepstral coefficient, formant, etc.; Voice quali...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/32G06K9/62G16H20/70G06N3/04G10L25/63G10L25/30
CPCG16H20/70G10L25/63G10L25/30G06V40/174G06V40/168G06V10/25G06N3/045G06F18/253
Inventor 陶建华何宇刘斌连政
Owner INST OF AUTOMATION CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap