Real-time expression recognition method based on multichannel parallel convolutional neural network (MPCNN)

A convolutional neural network and facial expression recognition technology, applied in the field of facial expression recognition that integrates multi-feature extraction, can solve problems such as not many research results, achieve the effects of expanding data volume, increasing network learning ability, and improving performance

Active Publication Date: 2017-12-19
CHONGQING UNIV OF POSTS & TELECOMM
View PDF3 Cites 67 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] To sum up, although researchers have done a lot of research on facial expression recognition, there are still not many research results on deep learning in facia

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time expression recognition method based on multichannel parallel convolutional neural network (MPCNN)
  • Real-time expression recognition method based on multichannel parallel convolutional neural network (MPCNN)
  • Real-time expression recognition method based on multichannel parallel convolutional neural network (MPCNN)

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0049] The technical solutions in the embodiments of the present invention will be described clearly and in detail below with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only some of the embodiments of the invention.

[0050] The technical scheme that the present invention solves the above-mentioned technical problems is:

[0051] figure 1 This system block diagram mainly includes:

[0052] A real-time expression recognition method based on a multi-channel parallel convolutional neural network, comprising two steps of building a multi-channel parallel convolutional neural network (MPCNN) model and real-time expression recognition:

[0053] The steps of constructing the MPCN model include:

[0054] Step 1: Extract facial expression images containing RGB images and Depth images from the facial expression dataset containing color and depth images;

[0055] Step 2: Preprocess the images of the facial express...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a real-time expression recognition method based on a multichannel parallel convolutional neural network (MPCNN). The method comprises the steps of: extracting expression data, which contains RGB and Depth images, from a facial-expression data set; carrying out local binarization preprocessing (LBP) and facial-key-point extraction preprocessing on the color images, carrying out gradient preprocessing on the depth images, dividing the preprocessed images into two parts of a training set and a test set, and building the multichannel parallel convolutional neural network; sending the preprocessed images in the training set to the network for training to obtain depth channel, lbp channel and key-point channel recognition models which learn facial-expression outlines, stereoscopic distribution and key-point features; and adopting maximum confidence to fuse classification results of the three recognition models to obtain a final expression recognition model, and building a real-time expression recognition system. The method enhances the robustness of the recognition network, and effectively improves the performance of the real-time expression recognition system.

Description

technical field [0001] The invention belongs to the fields of image recognition, human-computer interaction and artificial intelligence, in particular to an expression recognition method based on deep learning and fusion of multi-feature extraction. Background technique [0002] Facial expression is an important carrier of human communication and an important way of non-verbal communication. It can not only express human emotional state, cognitive activities and personality characteristics, but also rich in human behavior information and human emotional state, Other factors such as mental state and health state are closely related. Psychologist Mehrabian proposed that in the process of human communication, only 7% of the information is expressed through language, and 38% is conveyed through auxiliary language, such as rhythm, voice, intonation, etc., and facial expressions are the largest part. ——Able to reach 55% of the total amount of information. Therefore, a lot of val...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/34G06K9/46G06K9/62
CPCG06V40/175G06V40/168G06V10/267G06V10/40G06V10/467G06F18/254
Inventor 蔡林沁周锴徐宏博陈富丽虞继敏
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products