Facial action unit recognition method and device based on joint learning and optical flow estimation

A facial action and recognition method technology, applied in the field of computer vision, can solve the problems of poor versatility and low recognition rate, and achieve the effects of improving recognition accuracy, improving expression ability, and strong robustness

Active Publication Date: 2021-06-18
CHINA UNIV OF MINING & TECH
View PDF6 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a facial action unit recognition method and device based on joint learning and optical flow estimation, which automatically extracts ...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Facial action unit recognition method and device based on joint learning and optical flow estimation
  • Facial action unit recognition method and device based on joint learning and optical flow estimation
  • Facial action unit recognition method and device based on joint learning and optical flow estimation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0073] The present invention will be specifically introduced below in conjunction with the accompanying drawings and specific embodiments.

[0074] In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", " The orientations or positional relationships indicated by "vertical", "horizontal", "top", "bottom", "inner" and "outer" are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and Simplified descriptions, rather than indicating or implying that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and thus should not be construed as limiting the invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and should not be understood as indicating or implying relative...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a facial action unit recognition method and device based on joint learning and optical flow estimation. The method comprises the following steps: firstly extracting an original image pair needed by model training from video data to form a training data set, then preprocessing the original image pair to obtain an amplified image pair, constructing a convolutional neural network module I to extract multi-scale regional features of the amplified image pair, constructing a convolutional neural network module II to extract static global features of the amplified image pair, constructing a convolutional neural network module III to extract optical flow features of the amplified image pair, and finally, constructing a convolutional neural network module IV, fusing the static global features and the optical flow features, and carrying out facial action unit recognition. An end-to-end deep learning framework is adopted to jointly learn action unit recognition and optical flow estimation, action unit recognition is promoted by utilizing the relevance between tasks, the motion condition of facial muscles in a two-dimensional image can be effectively recognized, and unified facial action unit recognition system construction is achieved.

Description

technical field [0001] The invention relates to a facial action unit recognition method and device based on joint learning and optical flow estimation, belonging to computer vision technology. Background technique [0002] Optical flow estimation is a fundamental research task in computer vision, which is the bridge and link connecting images and videos. The core idea is to estimate the pixel-by-pixel correspondence given two frames of images before and after. This can also be approximately understood as a projected motion field of a 3D object on a 2D image plane. Optical flow plays an important role in behavior understanding, video processing, motion prediction, multi-view 3D reconstruction, autonomous driving, and real-time localization and mapping (SLAM). [0003] In order to study human facial expressions in more detail, Ekman, a famous American emotional psychologist, proposed the Facial Action Coding System (FACS) for the first time in 1978, and made important improv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/46G06K9/62G06F17/16G06N3/04G06N3/08
CPCG06F17/16G06N3/08G06V40/23G06V40/161G06V40/168G06V10/44G06N3/045G06F18/214
Inventor 邵志文孙莹周勇
Owner CHINA UNIV OF MINING & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products