Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A facial expression and posture bimodal fusion expression recognition method based on deep learning

A technology of facial expression and deep learning, applied in the field of image recognition, can solve problems such as slow progress and difficult research on facial expression recognition, and achieve the effect of improving adaptability and promoting practical application

Inactive Publication Date: 2019-06-14
HARBIN ENG UNIV
View PDF2 Cites 17 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Early research on facial expression recognition was difficult and slow

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A facial expression and posture bimodal fusion expression recognition method based on deep learning
  • A facial expression and posture bimodal fusion expression recognition method based on deep learning
  • A facial expression and posture bimodal fusion expression recognition method based on deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The present invention will be further described below in conjunction with the accompanying drawings.

[0039] as attached Figure 7 As shown, it is a flow chart of the facial expression and posture dual-modal fusion expression recognition method based on deep learning, which specifically includes the following steps:

[0040] 1. Establish a face-posture dual-modal expression database;

[0041] RAF-DB dataset: This dataset downloads more than 30,000 images from the Filckr image social network using keywords related to expressions, and invites 315 volunteers who have received expression-related knowledge training to annotate these images. The experiment uses 12271 pictures given in the data set as the training set and 3068 pictures as the test set.

[0042] FER2013 data set: The main source of this data set is downloaded through the keyword search of Google Images, so it is closer to the facial expressions under natural conditions. At the same time, the samples are ver...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of image recognition, and particularly relates to a facial expression and posture bimodal fusion expression recognition method based on deep learning. Themethod comprises the following steps of establishing an image database in a natural environment, and carrying out preprocessing and enhancement processing on an obtained image; proposing an SE-GoogleNet network to carry out feature extraction on the processed data image; selecting a convolutional part of a Caffemodel of the GoogleNet model based on action classification to directly carry out knowledge migration training; and inputting the two pre-trained convolutional neural networks into an SPP layer for feature fusion, and finally sending the fused convolutional neural networks into an LSTMto realize final classification of bimodal expression recognition. According to the present invention, an LSTM model based on face and limb dual-channel feature fusion is designed, the pyramid poolingis used, and then the size matching problem during feature fusion is solved; and in combination with transfer learning, the model can realize recognition of bimodal expressions under the condition offewer databases, so that the adaptability of the model to various natural environments is improved, and the application prospect is wide.

Description

technical field [0001] The invention belongs to the technical field of image recognition, and in particular relates to a facial expression and posture dual-mode fusion expression recognition method based on deep learning. Background technique [0002] There are three basic forms of expression of emotion: expression, voice and language. Due to the unique non-contact, universality and authenticity of expressions, it can best reflect the true emotions of human beings in real life. Therefore, through the recognition of facial expressions, intelligent human-computer interaction and the prediction of one's own emotional fluctuations can be effectively realized. Expressions are divided into two categories: facial expressions and gesture expressions. Early research on facial expression recognition was difficult and progress was slow. With the outstanding achievements of deep learning in the field of computer vision in recent years. Deep learning structures and theories were quic...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/06
Inventor 王科俊陈静张欣怡孙丽莹
Owner HARBIN ENG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com