Three-dimensional shape recovery and attitude estimation method and device based on depth image

Abstract

The invention provides a three-dimensional shape recovery and attitude estimation method and device based on a depth image, and the method employs a cascading deep learning network based on point cloud to recover the three-dimensional shape and three-dimensional joint point coordinates in a real-time and high-precision manner. According to the method, a parameterized model layer is provided, and based on a predefined parameterized model, model parameters are recovered through a neural network to recover a three-dimensional shape and corresponding joint point coordinates. According to the method, a joint self-adaptive adjustment sub-network is provided, the internal relation between the three-dimensional shape and the joint points is unbound, and the relation between the three-dimensional shape and the joint point coordinates marked by different frameworks is learned in a self-adaptive mode. Meanwhile, an existing data set is lack of three-dimensional shape annotation, so that the invention provides weak supervision of the three-dimensional shape provided by the Chamfer loss. Through actual use verification, the method has the advantages of high automation degree, high precision andreal-time performance, and can meet professional or popular application requirements.

Description

technical field [0001] The invention belongs to the field of computer vision and computer image processing, and in particular relates to a method and device for restoring and estimating the three-dimensional shape of a human hand based on a depth image, which is applicable to the whole or part of the human body (such as human hands and faces), large animals and other objects. Background technique [0002] With the development of computer vision and artificial intelligence, human-computer interaction technology has transitioned from keyboard and mouse to natural human-computer interaction. After satisfying the basic interaction needs of human beings, the natural human-computer interaction method puts more emphasis on the interactive experience, trying to get rid of the limitations of the interactive interface and equipment, so that the human-computer interaction is as convenient and natural as human communication. The emergence and rapid development of artificial intelligence...

AI Technical Summary

Problems solved by technology

[0007] The present invention proposes a cascaded neural network to solve the problem of skeleton inconsistency between training data sets, adaptively learn the relationship between the three-dimensional shape and the joint point coordinates under the current skeleton structure, and use the Chamfer loss function to provide weak supervision information of the three-dimensional shape, real-time high Progressive restoration of the 3D shape and joint point coordinates of objects such as human hands

Images