Multi-modal fusion method and device for psychological stress detection

A modality and attention technology, applied in the computer field, can solve problems such as inability to fully express the psychological state, lack of psychological stress detection methods and devices, etc.

Active Publication Date: 2019-10-08
TSINGHUA UNIV
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are also some inherent problems in the data related to physiological signals. For example, the physiological data related to the state of extreme excitement and extreme stress are very similar. Therefore, the data related t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal fusion method and device for psychological stress detection
  • Multi-modal fusion method and device for psychological stress detection
  • Multi-modal fusion method and device for psychological stress detection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0045] Before introducing the solutions provided by the embodiments of the present invention, the origin of the present invention will be briefly described. When adolescents suffer from psychological stress, their daily activities (such as counting steps) and sleep conditions (difficulty falling asleep, waking up early, etc.) often have some abnor...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiments of the invention provide a multi-modal fusion method and device for psychological stress detection. The multi-modal fusion method is based on an attention-enhanced feature matrix of physiological data->text, physiological data->pictures, text->physiological data, text->pictures, pictures->physiological data and pictures->text, based on a feedforward fully-connected neural network,a fusion feature matrix of the text, the pictures and the physiological data is obtained; then based on importance weights of the text, the pictures, and the physiological data and the fusion featurematrix of the text, the pictures and the physiological data, a fusion representation matrix of three modes is obtained; and finally, based on the fusion representation matrix of the three modes and the feedforward fully-connected network, a pressure classification vector reflecting the psychological stress problem is obtained. By merging text picture data and physiological related data, not only the deficiencies caused by the subjectivity of text and picture data are compensated, but also certain inherent problems of physiological related data are solved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a multimodal fusion method and device for psychological stress detection. Background technique [0002] With the increasing pressure of social competition, the psychological pressure of teenagers has gradually become a more and more serious problem. Excessive psychological stress can lead to many physical and psychological problems, which makes psychological stress detection more and more important. [0003] Existing psychological stress detection work focused on social media only focuses on text and picture content. However, text and picture content are subjective and sometimes cannot express the true psychological state. [0004] Some existing works related to physiological signals have proved the effectiveness of physiological signals in detecting psychological stress, such as heart rate variability, electrocardiogram, current skin response, electroencephalogram, blood pres...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): A61B5/16
CPCA61B5/165A61B5/7264
Inventor 冯铃张慧君曹檑
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap