Method for acquiring 3d (three-dimensional) coordinates of human skeleton joint points on basis of deep learning

Abstract

The invention provides a method for acquiring 3d (three-dimensional) coordinates of human skeleton joint points on the basis of deep learning. The method comprises the following steps of: data preparation, i.e., loading a standard FBX model to unity software, setting a joint rotation quaternion of the FBX model and acquiring 3d coordinates and 2d coordinates of each joint point; data preprocessing, i.e., carrying out normalization processing on 3d coordinate data and 2d coordinate data and inputting into a convolutional neural network; network training, i.e., calculating a training set loss and a verification set loss; and real-time acquisition of the 3d coordinates of the human joint points, i.e., after detecting out the 2d coordinates of the human joint points in an image and carrying out normalization processing, inputting to the convolutional neural network, and combining and utilizing anti-normalization to acquire the 3d coordinates of the human joint points under a camera coordinate system. Compared to the related art, the method for acquiring the 3d coordinates of the human skeleton joint points on the basis of deep learning, which is provided by the invention, is low in hardware cost, high in accuracy and wide in application range.

Description

technical field [0001] The invention belongs to the field of artificial intelligence, and in particular relates to a method for acquiring three-dimensional coordinates of joint points of human bones based on deep learning. Background technique [0002] With the development and progress of society, the degree of intelligentization of home appliances such as voice control is getting higher and higher, which meets people's needs for intelligent home furnishing. However, this type of intelligent products also has great limitations. People cannot experience it. Therefore, there is a need for the emergence of some smart products that are more in line with human operating habits. For example, in the field of smart home, by detecting gestures and actions, it is judged what operations people have performed to drive smart products; Judging whether the behavior of the human body matches the standard template based on the received human body posture. [0003] In related technologies, ...

AI Technical Summary

Problems solved by technology

[0003] In related technologies, there have been some studies on human body pose estimation, mainly in the following three categories, one is an optical capture instrument like opti-track, which pastes multiple Mark points on the human body and detects the position of the Mark points , to finally determine the position of the joint points of the human body. This method requires a lot of hardware assistance, and its use is limited. It does not use popularization and promotion.

The second category is similar to Kinect, which obtains the coordinates of the three-dimensional joint points of the human body through multiple cameras and binocular positioning. Similarly, this method has large limitations, many hardware assistance, and high cost. It does not use popularization and promotion.

The third category is that the coordinates of the two-dimensional joint points of the human body are detected through the deep learning algorithm, but the conversion to the three-dimensional coordinates has not been realized, and further information about the joints of the human body cannot be obtained.

Some conversion algorithms rely on limited training data, and after conversion, they only have good effects on part of the image processing, and cannot be widely used

Images