An inference network for 3D coordinate estimation of human joint and a method thereof

Abstract

A 3D coordinate estimation inference network for human joint and a method thereof, taking 3D coordinate estimation as the key point location problem of discretized 3D spatial attitude inference, Instead of directly regressing the 3D coordinates of joints (x, y, z), CNN is trained to predict the probability of each voxel of each joint in the volume, so as to form a 3D thermodynamic map, so as to improve the accuracy of 3D coordinate data for human posture estimation, reduce the non-linearity of the task of directly regressing joints, and improve the learning effect. The inference network is a model structure with n (n >= 2) order hourglass network (Hourglass) as the center and m order (m >= 2) cascade.

Description

technical field [0001] The invention relates to an inference network and method for estimating 3D coordinates of human body joints, and belongs to the technical field of virtual reality. Background technique [0002] The 3D pose estimation of the human body is to accurately estimate the 3D positions of several joints of the human body (such as the head, shoulder, elbow, etc.). Due to the loss of depth information, estimating the position of the 3D joint points of the human body from the RGB video stream is a great challenge in the field of computer vision. [0003] With the development of convolutional neural network (Convolutional Neural Networks, hereinafter referred to as CNN) technology, especially in the field of pattern classification, since the network avoids the complicated preprocessing of images, it can directly input the original image, thus obtaining more Wide range of applications. [0004] At present, more and more tasks in computer vision are solved using CN...

AI Technical Summary

Problems solved by technology

Divide a task into two stages, and the errors generated in each stage will accumulate, which will cause greater errors in the final result performance

The second is that in this solution, the performance of 3D pose estimation depends entirely on the 2D results, which will cause a part of the information to be lost.

However, the methods used in the prior art do not make full use of intermediate features, resulting in large errors in the final 3D inference results.

Images