Real-time expression recognition method based on multichannel parallel convolutional neural network (MPCNN)

Abstract

The invention discloses a real-time expression recognition method based on a multichannel parallel convolutional neural network (MPCNN). The method comprises the steps of: extracting expression data, which contains RGB and Depth images, from a facial-expression data set; carrying out local binarization preprocessing (LBP) and facial-key-point extraction preprocessing on the color images, carrying out gradient preprocessing on the depth images, dividing the preprocessed images into two parts of a training set and a test set, and building the multichannel parallel convolutional neural network; sending the preprocessed images in the training set to the network for training to obtain depth channel, lbp channel and key-point channel recognition models which learn facial-expression outlines, stereoscopic distribution and key-point features; and adopting maximum confidence to fuse classification results of the three recognition models to obtain a final expression recognition model, and building a real-time expression recognition system. The method enhances the robustness of the recognition network, and effectively improves the performance of the real-time expression recognition system.

Description

technical field [0001] The invention belongs to the fields of image recognition, human-computer interaction and artificial intelligence, in particular to an expression recognition method based on deep learning and fusion of multi-feature extraction. Background technique [0002] Facial expression is an important carrier of human communication and an important way of non-verbal communication. It can not only express human emotional state, cognitive activities and personality characteristics, but also rich in human behavior information and human emotional state, Other factors such as mental state and health state are closely related. Psychologist Mehrabian proposed that in the process of human communication, only 7% of the information is expressed through language, and 38% is conveyed through auxiliary language, such as rhythm, voice, intonation, etc., and facial expressions are the largest part. ——Able to reach 55% of the total amount of information. Therefore, a lot of val...

AI Technical Summary

Problems solved by technology

[0009] To sum up, although researchers have done a lot of research on facial expression recognition, there are still not many research results on deep learning in facial expression recognition. How to overcome different lighting, head postures, and complex backgrounds The impact of other practical factors is still a very thorny issue

Images