Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-modal emotional pressure recognition method and device, computer equipment and storage medium

An identification method and multimodal technology, applied in the field of computer equipment and storage media, devices, and multimodal emotional stress identification methods, can solve problems such as low accuracy and difficult application, and achieve the effect of improving accuracy.

Active Publication Date: 2021-07-02
SOUTH CHINA UNIV OF TECH +1
View PDF11 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The current research on human emotion recognition mainly focuses on several specific emotions provided by internationally renowned databases such as happiness, anger, sadness, etc., and few researchers pay attention to human pressure; moreover, in the use of human physiological data for research At this time, EEG, ECG and other data need to be collected by professional instruments, which are difficult to be widely used in real life.
In addition, in the existing unimodal emotional stress recognition, the recognition algorithms are mostly traditional machine learning algorithms, and the accuracy rate is not high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal emotional pressure recognition method and device, computer equipment and storage medium
  • Multi-modal emotional pressure recognition method and device, computer equipment and storage medium
  • Multi-modal emotional pressure recognition method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0065] This embodiment provides a multimodal emotional stress recognition method, which performs data preprocessing on face video images and photoplethysmograms collected under the psychological experiment paradigm, and constructs training samples. Build deep learning models using attentional convolutional neural networks, gated recurrent units, and fully connected layers. During the training process, the feature vector of the face image is extracted through the attentional convolutional neural network, and after being fused with the feature vector of the photoplethysmography wave, they are jointly input to the gated recurrent unit to obtain the feature vector containing time information, thereby improving The spatial information and time information contained in the training samples are well extracted, and finally the feature vector containing time information is input to the fully connected layer to obtain the recognition result. The modal emotional stress recognition method...

Embodiment 2

[0100] Such as Figure 5 As shown, the present embodiment provides a multimodal emotional stress recognition device, including obtaining multimodal data module 501, building a deep learning model module 502, training deep learning model module 503 and emotional stress recognition module 504, each module The specific functions are as follows:

[0101] The multimodal data acquisition module 501 is configured to acquire multimodal data and perform preprocessing to obtain a training sample set; wherein, the multimodal data includes face video image data and photoplethysmography data. The specific steps are as follows: acquire face video images and photoplethysmography physiological signals of people in the stress-induced state of the psychological experiment paradigm, perform data preprocessing on the data of the two modalities, and construct a training sample set.

[0102]The deep learning model building module 502 is used to build a deep learning model by using the attention co...

Embodiment 3

[0106] Such as Figure 6 As shown, this embodiment provides a computer device, which may be a computer, a server, etc., and includes a processor 602 connected through a system bus 601 , a memory, an input device 603 , a display 604 and a network interface 605 . Wherein, the processor 602 is used to provide computing and control capabilities, and the memory includes a non-volatile storage medium 606 and an internal memory 607. The non-volatile storage medium 606 stores an operating system, a computer program and a database, and the internal memory 607 is The operating system and the computer program in the non-volatile storage medium 606 provide an environment for running. When the computer program is executed by the processor 602, the multimodal emotional stress recognition method of the above-mentioned embodiment 1 is realized, as follows:

[0107] Obtaining multimodal data and performing preprocessing to obtain a training sample set; wherein the multimodal data includes face...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a multi-modal emotional pressure recognition method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining multi-modal data, and carrying out the preprocessing of the multi-modal data, and obtaining a training sample set, wherein the multi-modal data comprises face video image data and photoelectric volume pulse wave data; constructing a deep learning model by using an attention convolutional neural network, a gating circulation unit and a full connection layer; training the deep learning model by using the training sample set until the deep learning model converges; and inputting a to-be-identified sample into the trained deep learning model to obtain an emotion pressure identification result. According to the method, the selected multi-modal data has internal association when representing the emotional pressure, and spatial information and time information in the multi-modal data are fully mined and fused through the deep learning model, so that the deep learning model pays more attention to the part, which can best represent the emotional pressure, in the data, and thus the accuracy of emotional pressure recognition is improved.

Description

technical field [0001] The invention belongs to the field of artificial intelligence, and in particular relates to a multimodal emotional pressure recognition method, device, computer equipment and storage medium. Background technique [0002] Emotional stress refers to the psychological tension reaction or state formed by an individual under the influence of emotions such as anxiety or fear. In modern society, with the faster pace of work and life, people are facing various pressures. Such as social environment pressure, work pressure, personal achievement pressure and so on. Studies have shown that long-term exposure to emotional stress will produce a series of adverse consequences, causing direct damage to people's physical and mental health. Therefore, it is of great significance to identify and evaluate people's emotional stress. At present, researchers have used human emotional behaviors such as facial expressions, voice, posture, etc., and physiological patterns su...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): A61B5/16A61B5/00A61B5/02
CPCA61B5/165A61B5/0077A61B5/0064A61B5/0033A61B5/004A61B5/02A61B5/7264A61B5/7267A61B5/7235
Inventor 王毓邢晓芬徐向民殷瑞祥
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com