A method for estimating that posture of a three-dimensional human body based on a video stream

Abstract

A method for estimating that posture of a three-dimensional human body based on a video stream. The method based on depth learning is used to estimate 3D pose of human body in video stream, which avoids many defects caused by two-dimensional vision analysis errors, and makes full use of the temporal relationship between video frames to improve the accuracy and real-time of 3D pose inference results of video stream. The method includes, for n(n >= 2) th frame of video, (1) inputting two-dimensional image of current frame and using shallow neural network module to generate image shallow map; 2)inputting a two-dimensional joint thermodynamic map of the human body generated by the (n-1)th frame and an image shallow map generated by the current frame to an LSTM module to generate a deep-levelcharacteristic map; 3) outputting the deep image characteristic map generated by the current frame to the residual module to generate a two-dimensional human body joint thermodynamic map of the current frame; 4) outputting the human body two-dimensional joint thermodynamic map of the current frame to a three-dimensional joint inference module for carrying out two-dimensional to three-dimensionalspatial mapping; and superimposing the three-dimensional human body joint thermodynamic map generated in each frame to generate a video stream of three-dimensional human body posture estimation.

Description

technical field [0001] The invention relates to a method for estimating three-dimensional human body poses for two-dimensional image and video streams, and belongs to the technical field of virtual reality. Background technique [0002] The 3D pose estimation of the human body is to accurately estimate the 3D positions of several joints of the human body (such as the head, shoulder, elbow, etc.). Due to the loss of depth information, estimating the position of 3D joint points of the human body from a 2D RGB video stream is a great challenge in the field of computer vision. [0003] With the development of Deep Convolutional Networks (Deep Convolutional Networks), more and more technological innovations focus on 3D human skeleton detection based on end-to-end deep neural networks. The more common 3D human pose estimation methods currently have the following two technical routes: [0004] Two-stage 3D joint point inference, as attached figure 1 As shown, the method is divid...

AI Technical Summary

Problems solved by technology

[0006] As mentioned above, the existing 3D human pose estimation has the following technical defects: A. Generally, the 3D coordinates of human joint points are directly output, which is very difficult for the network to learn, because the learning task from the feature space to the 3D pose space is a highly Non-linear learning tasks have high nonlinear shortcomings; B. When performing 3D inference of joint points, the intermediate features of the neural network have not been fully utilized, and it is difficult to combine feature information of different scales and dimensions, resulting in poor inference results. Poor; C. In the process of 3D attitude inference based on video stream, the amount of calculation increases greatly, so that the final inference effect cannot meet the real-time requirements, and the actual application effect is poor; D. In the process of 3D attitude inference based on video stream , does not use the space-time relationship between each frame, so it cannot solve the problem of joint points being occluded and disappearing

Images