Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A multi-view facial expression recognition method based on mobile terminal

A facial expression recognition and multi-view technology, applied in the field of facial expression recognition, can solve the problem of large deep learning models and achieve the effect of improving recognition accuracy

Inactive Publication Date: 2019-03-01
CHINA UNIV OF GEOSCIENCES (WUHAN)
View PDF11 Cites 45 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

With the deepening of the network and the surge of data driven by big data, the accuracy of the training results has increased and at the same time it has brought about a problem: the deep learning model is getting bigger and bigger, often hundreds of megabytes, which can only be allocated to tens of megabytes It is unacceptable for the mobile app of the space, and model compression and optimization must be carried out

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A multi-view facial expression recognition method based on mobile terminal
  • A multi-view facial expression recognition method based on mobile terminal
  • A multi-view facial expression recognition method based on mobile terminal

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0031] The embodiment of the present invention provides an expression attention region learning based on multi-view facial expression recognition on a mobile terminal, including the following steps:

[0032] S1. Cut out part of the face image area from each picture, and perform data enhancement to obtain a data set for training the AA-MDNet model;

[0033] Data enhancement includes random cropping, panning, flipping, color dithering, brightness changes, saturation changes, contrast changes, and sharpness changes.

[0034] S2. Use the GAN model to extend the data set obtained in step S1;

[0035] The GAN model includes four parts: generative model G, image discrimination model D ep , Identity Discrimination Model D id And expression classifier C, generative model G includes encoder G e And decoder G d ; Encoder G e And decoder G d The input data is encoded, analyzed, decoded and reconstructed to generate images, both of which are composed of convolutional layers and fully connected lay...

Embodiment 2

[0042] The embodiment of the present invention provides a posture and expression classification example of a multi-view facial expression recognition method based on a mobile terminal, including:

[0043] 1. Data preprocessing

[0044] Data enhancement: The data sets used to train the AA-MDNet model are KDEF, BU-3DFE and SFEW. In order to better perform expression classification, before starting to train AA-MDNet, it is necessary to perform data enhancement on face images to increase the diversity of samples and minimize interference factors. First, for a picture, crop out part of the face image to reduce other interference factors (background, etc.). During training, perform data enhancement (random cropping, translation, flipping, color jitter, brightness change, saturation change, contrast change, sharpness change) to improve the generalization ability of the model, prevent overfitting, and improve accuracy.

[0045] Generative Adversarial Network (GAN) Extended Data Set: The SF...

Embodiment 3

[0089] The training process of a mobile-based multi-view facial expression recognition method is implemented as follows:

[0090] GAN model training: GAN is used to enrich the data set. Before training AA-MDNet, train the GAN model and save the model file.

[0091] (1) GAN model loss value calculation

[0092] The loss value of the generative model G: Since the generative model is directly related to the two decision models, its own loss value is combined with the encoder G e And decoder G d The loss value of can better train the model, the calculation formula is as follows

[0093] loss G =loss EG +aloss G_ep +bloss E_id

[0094] Among them, the values ​​of a and b are very small, and the default is 0.0001; loss EG , Loss G_ep , Loss E_id Represents the loss value of the generative model, the loss value of the encoder and the loss value of the decoder, respectively.

[0095] Discriminant model D ep The loss value:

[0096] loss D_ep =loss D_ep_input +loss D_ep_G

[0097] Where lo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multi-view facial expression recognition method based on a mobile terminal, which comprises the steps of cutting out a face region from each image, and carrying out data enhancement to obtain a facial expression recognition method used for training data set of AA-MDNet model; the multi-attitude data set is obtained by GAN model extension, and the multi-attitude data set is obtained by GAN model extension. Using ADN multi-scale clipping method to clip; enter the cropped image into AA-MDNet model, The input image extracts features from DenseNet, a densely connected subnetwork, Then, based on the extracted features, an attention adaptive network (ADN) is used to obtain the position parameters of the attention area of the expression and posture, and the image of the area is scaled from the input image according to the position parameters, which is used as the input of the next scale. Learning the multi-scale high-level feature fusion, we can get the high-level features with global and local fusion features. Finally, we can classify the facial posture and expression categories. The invention has very important significance in the fields of human-computer interaction, face recognition, computer vision and the like.

Description

Technical field [0001] The invention relates to the field of facial expression recognition, in particular to a multi-view facial expression recognition method based on a mobile terminal. Background technique [0002] Humans mainly rely on body language and natural language to transmit emotions and information. Natural language mainly refers to written records in the time dimension, but words alone are not enough to describe the recorded information in detail. Facial expressions are part of the human body's (physical) language, a physical and psychological response, and are usually used to convey emotions. If the machine can recognize facial expressions, it will have broad application prospects in remote education, medical and other industries, and promote the development of human-computer interaction, emotional computing, machine vision and other fields, so the research on facial expression recognition algorithms is of great significance. For example, the driver’s facial expres...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06K9/62
CPCG06V40/174G06V40/172G06V40/168G06F18/214
Inventor 刘袁缘王勋广蒋捷方芳谢忠罗忠文覃杰
Owner CHINA UNIV OF GEOSCIENCES (WUHAN)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com