Human image key point detection method and system based on feature fusion

Abstract

The invention relates to a portrait key point detection method based on feature fusion, and the method comprises the steps: S1, sending a portrait image into a face detection network for face detection and cutting, and converting coordinate information in a training data set into thermodynamic diagram information; s2, the portrait image is sent to a regression network based on Transform and Convotion feature fusion to be trained, the regression network is of a parallel structure, low-level semantic features of the portrait image are captured through Convotion, high-level semantic features in the portrait image are captured through Transform, obtained feature maps are subjected to jump connection, and a thermodynamic diagram containing coordinate information is jointly coded; s3, combining the N thermodynamic diagrams of the N key points in the same channel on the basis of a Convoltion and Transform feature fusion regression network, generating a thermodynamic diagram with boundary information, and outputting the thermodynamic diagrams of N + 1 channels; and S4, decoding the first N thermodynamic diagrams of the output thermodynamic diagrams to obtain accurate coordinate information of N key points. The method and the system are favorable for improving the detection precision and the operation speed.

Description

technical field

[0001] The present invention relates to the technical field of computer vision, in particular to a method and system for detecting key points of a portrait based on feature fusion. Background technique

[0002] In recent years, with the vigorous development of Convolution network and deep learning in the field of computer vision, computer vision tasks on face images have also been practically applied. Among them, the facial key point detection technology based on deep learning is one of the current research hotspots in the field of image detection. The development trend of facial key point detection in virtual reality, facial expression recognition, face reconstruction, face tracking, and portrait beauty is obvious. .

[0003] The Transformer network was not born out of the boom in computer vision. The Transformer model was proposed by the Google team in 2017 for the purpose of applying to the field of NLP (natural language processing). The self-attention ...

Images