Human hand three-dimensional posture estimation method and device based on three-dimensional point cloud

Abstract

The invention relates to a human hand three-dimensional posture estimation method and device based on three-dimensional point cloud which mainly solves the problem of how to recover the three-dimensional posture of a human hand from the human hand point cloud obtained by a single depth map, and has the main technical difficulties of disordered point cloud arrangement, higher noise, rich gesture changes of the human hand, self-shielding of the human hand caused by a shooting angle and the like. The invention provides a human hand posture estimation algorithm based on a deep neural network by which the features can be adaptively extracted from the rich training data. Meanwhile, the local and global features of the point cloud can be predicted while the three-dimensional positions of the joint points of the human hand are regression in real time, the generalization ability of the network is improved through the internal connection of joint labeling, and the problem that the generalizationability of the features extracted by a single-task network is poor, is solved. The actual use verifies that the method and the device have the advantages of high automation degree, high precision andreal-time performance, and can meet the professional or popular application requirements.

Description

technical field [0001] The invention belongs to the fields of computer vision and computer image processing, and in particular relates to a method and device for estimating a three-dimensional pose of a human hand based on a three-dimensional point cloud. Background technique [0002] In recent years, with the development of computer vision technology and the substantial improvement of computer computing power, vision-based human-computer interaction has made great progress. Through image or video processing, the computer has the ability to "see", and the statistical method that relies on large-scale data can enable the computer to have the ability to understand and analyze. This is the most natural and convenient way in human-computer interaction. Among them, the interaction based on body movements is one of the core problems of visual human-computer interaction, including pose estimation, gesture recognition, face recognition, facial expression recognition and 3D reconstru...

AI Technical Summary

Problems solved by technology

[0005] The 3D hand pose estimation method is divided into traditional non-visual algorithm and visual algorithm. The non-visual algorithm is mainly based on the data glove method. The idea of ​​this method is to use some commonly used sensors to detect and track the key points of the human hand. Such methods have obvious disadvantages: high cost, unnatural interaction mode, easy separation of key points, etc.

Vision algorithms are divided into two types: model-driven and data-driven. Among them, model-based methods need to use efficient optimization algorithms in high-dimensional parameter spaces to quickly converge to the global optimum, and the general optimization objective functions are non- Convex function, which requires a better initialization weight, otherwise it is easy to fall into a local optimum; on the other hand, the optimization process requires a large number of iterative operations, resulting in no guarantee of real-time performance. Currently, it can only be used in some offline environments

There are traditional methods and deep learning methods based on data-driven methods. The idea of ​​image feature method in traditional methods is: first extract features from the image, such as edge feature extraction, etc., and use the nearest neighbor search method in the marked human hand pose database Find the closest pose as the final estimation result. The limitations of this method are also obvious: it depends too much on the integrity of the database, and cannot get poses that do not exist in the database.

Images