Face motion unit detection method based on physical characteristics and distribution characteristics

A technology of face movement and distribution characteristics, applied in the computer field, can solve the problems of poor detection effect, poor AU detection and detection effect, and shallow network layers, and achieve the effect of accelerating the convergence speed.

Pending Publication Date: 2019-11-15
SHANGHAI JIAO TONG UNIV
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0018] The DRML algorithm has the following disadvantages: 1) The basis of the algorithm is AlexNet, and the number of network layers is relatively shallow, which leads to poor learning effect and poor AU detection effect; 2) it does not consider the timing information and the uneven distribution of different AUs, which will inevitably lead to The detection effect of AU with large sample size is slightly better, but the detection effect of AU with small sample size is very poor
[0026] The R-T1 algorithm has the following disadvantages: 1) The input data requires not only pictures, but also feature point information, so the input data of the R-T1 network is more than that of only pictures; 2) It does not consider the timing information and the distribution of different AUs The unbalanced characteristics will inevitably lead to the detection effect of the algorithm on those AUs with a large sample size is slightly better, but the detection effect on AUs with a small sample size is very poor; 3) The problem of unbalanced positive and negative sample data of the same AU is not considered. The number of concentrated negative samples (AU does not appear) far exceeds the number of positive samples (AU appears). The imbalance of positive and negative samples will affect the training of the algorithm, resulting in low AU detection effect

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Face motion unit detection method based on physical characteristics and distribution characteristics
  • Face motion unit detection method based on physical characteristics and distribution characteristics
  • Face motion unit detection method based on physical characteristics and distribution characteristics

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0052] This embodiment provides a human face motion unit detection method based on physical characteristics and distribution characteristics. The method is based on a pre-trained human face motion unit (AU) detection model to perform multi-label learning and classification on a group of picture sequences to obtain human face motion. Unit detection results, the human face motion unit detection model includes a sequentially connected cross-splicing network and a long-short-term memory network.

[0053] This embodiment is based on the deep learning framework of caffe, and implements the face motion unit detection model on the Ubuntu system. The face motion unit detection model does not need to add a special layer, and only needs to modify the existing layer structure of caffe to complete , the implementation difficulty is very low, and the test performance on the public AU detection algorithm datasets BP4D, DISFA, and GFT is very good, surpassing other algorithms that currently pe...

Embodiment 2

[0080] In order to test the performance of the present invention in this embodiment, the detection method of the present invention is used for testing on three public AU detection data sets. In the experiment, the detection method of the present invention is referred to as CCT (cross-concat and temporal network) for better The function of the cross-concat block is clearly shown, and the network that removes the LSTM in the CCT is referred to as CC.

[0081] There are two general measurement standards for the effect of AU detection algorithms, F1score and AUC. Among them, F1score is the harmonic mean of precision and recall, and AUC is the area under the ROC curve. The higher the F1 score and AUC, the better the detection effect of the algorithm.

[0082] Table 2, Table 3 and Table 4 show the comparative experimental results on these three data sets respectively.

[0083] Table 2 Algorithm comparison results on the BP4D dataset (the bold table with square brackets is the best...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a face motion unit detection method based on physical characteristics and distribution characteristics. The method is characterized in that a group of picture sequences are processed based on a pre-trained human face motion unit detection model, a human face motion unit detection result is obtained, and the human face motion unit detection model comprises a cross splicingnetwork and a long-short-term memory network which are connected in sequence. Compared with the prior art, the problem of unbalanced data distribution among different face motion units is considered and solved for the first time, and the detection effect of the face motion units is further improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method for detecting human face motion units based on physical characteristics and distribution characteristics. Background technique [0002] Human facial expression analysis is an important field of artificial intelligence, and the detection of human facial motion unit (AU, action unit,) is crucial for human facial expression analysis. Human facial expressions are caused by facial muscle movements, and the facial movement coding system refers to the movement of one or more muscles as a personal facial movement unit. Almost all facial expressions can be represented by one facial motion unit or a combination of multiple facial motion units. For example: a smile can be represented by a combination of raised cheeks (AU6) and raised mouth corners (AU12), such as image 3 shown. [0003] Face motion unit detection is to detect which types of face motion units appear on a person...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V40/176G06V20/41G06N3/045G06F18/214
Inventor 胡巧平申瑞民姜飞
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap