Human image key point detection method and system based on feature fusion

Abstract

The invention relates to a portrait key point detection method based on feature fusion, and the method comprises the steps: S1, sending a portrait image into a face detection network for face detection and cutting, and converting coordinate information in a training data set into thermodynamic diagram information; s2, the portrait image is sent to a regression network based on Transform and Convotion feature fusion to be trained, the regression network is of a parallel structure, low-level semantic features of the portrait image are captured through Convotion, high-level semantic features in the portrait image are captured through Transform, obtained feature maps are subjected to jump connection, and a thermodynamic diagram containing coordinate information is jointly coded; s3, combining the N thermodynamic diagrams of the N key points in the same channel on the basis of a Convoltion and Transform feature fusion regression network, generating a thermodynamic diagram with boundary information, and outputting the thermodynamic diagrams of N + 1 channels; and S4, decoding the first N thermodynamic diagrams of the output thermodynamic diagrams to obtain accurate coordinate information of N key points. The method and the system are favorable for improving the detection precision and the operation speed.

Description

technical field [0001] The present invention relates to the technical field of computer vision, in particular to a method and system for detecting key points of a portrait based on feature fusion. Background technique [0002] In recent years, with the vigorous development of Convolution network and deep learning in the field of computer vision, computer vision tasks on face images have also been practically applied. Among them, the facial key point detection technology based on deep learning is one of the current research hotspots in the field of image detection. The development trend of facial key point detection in virtual reality, facial expression recognition, face reconstruction, face tracking, and portrait beauty is obvious. . [0003] The Transformer network was not born out of the boom in computer vision. The Transformer model was proposed by the Google team in 2017 for the purpose of applying to the field of NLP (natural language processing). The self-attention ...

AI Technical Summary

Problems solved by technology

The author believes that there is often a potential connection between the two tasks of face detection and face key point detection. However, most methods do not effectively combine the two tasks. In this paper, in order to make full use of the potential connection between the two tasks, A multi-task cascaded face detection framework is proposed, which simultaneously performs face detection and face key point detection

[0005] However, the current mainstream portrait key point detection algorithms all use the form of heat map regression. This method needs to render a Gaussian heat map, because the maximum value point in the heat map directly corresponds to the key point, which leads to two problems. , first, this type of algorithm needs to maintain a relatively high-resolution heat map, which leads to the characteristics of this type of regression network that are difficult to regress

The second is this type of algorithm. The maximum value point in the heat map directly represents the corresponding key point position. It is often insufficient to render an accurate heat map only by relying on the traditional method of deepening the number of convolutional layers.

Images