Real-time video face key point detection method based on deep learning

Abstract

The invention relates to a real-time video face key point detection method based on deep learning, and the method employs a convolutional neural network to carry out the key point detection of a single frame, employs a depth separable convolution to improve the model detection rate, employs a boundary heat map as an additional subtask of an original network to improve the constraint of a global face structure of the original network. The method improves the detection accuracy of an original network, is used for solving a data imbalance loss function of a heat map, improves the generalization capability of a model for a large attitude sample under a limited sample, and improves the inter-frame smoothness through an optical flow loss function. In the detection process, for a frame of which the confidence is lower than a key point confidence threshold due to an extremely large angle, fitting is carried out by utilizing 3DMM to obtain dense key point coordinates, 68-point sampling is carried out on the obtained dense key points according to a projection error between minimum frames, and the consistency with the previous frame is kept. The method has the advantages of real-time performance, capability of utilizing global inter-frame information, high detection accuracy of a face large posture condition and the like.

Description

technical field [0001] The technical solution of the present invention relates to face key point detection, in particular to a real-time video face key point detection method based on deep learning. Background technique [0002] Face key point detection is to detect the coordinates of key points such as eyebrows, eyes, nose, etc. for the input face area. Face key point detection plays a key role in the practical application of face-related research in face recognition, 3D face reconstruction, facial expression cloning, and micro-expression analysis. [0003] Face key point detection in images is easily affected by background, face pose, dramatic expression changes, occlusion, lighting, etc., while video key point detection also has high requirements for inter-frame smoothness and real-time performance in addition to the above. Existing key point detection methods for video often only use optical flow to track local key points to improve speed, and cannot use global inter-fr...

AI Technical Summary

Problems solved by technology

[0005] The present invention aims at the shortcomings of the face key point detection algorithm currently applied to video, such as poor real-time performance, inability to use global inter-frame information, and poor detection accuracy for large gestures of the face, and proposes a real-time video face key point based on deep learning Detection method

Images