End-to-end multi-view three-dimensional human body posture estimation method and system and storage medium

Abstract

The invention relates to an end-to-end multi-view three-dimensional human body posture estimation method and system and a storage medium, and the method comprises the steps: loading a pre-trained two-dimensional human body posture estimation network, and enabling a picture of each current view to serve as the input of the network; generating a thermodynamic diagram through a two-dimensional humanbody posture estimation network, and taking the thermodynamic diagram as the input of an LSTM thermodynamic diagram time sequence information extraction network; inputting the thermodynamic diagram into an LSTM initialization thermodynamic diagram time sequence information extraction network and an LSTM thermodynamic diagram time sequence information extraction network according to the value of the time sequence step length T to obtain a cell state and a hidden state; feeding the obtained hidden state into a decoder network to obtain a decoded thermodynamic diagram; fusing the thermodynamic diagram and the decoded thermodynamic diagram to obtain a thermodynamic diagram Ht (p) fused with time and space information; sending the thermodynamic diagram Ht (p) into a soft-argmax linear algebraictriangulation network to obtain a 2D point position; and solving an overdetermined equation on the homogeneous three-dimensional coordinate vector, and adopting a differentiable DLT-SII algorithm toobtain a final three-dimensional human body posture estimation point.

Description

technical field [0001] The present invention relates to the field of computer vision, in particular to an end-to-end multi-view 3D human pose estimation method, system and storage medium based on a deep learning network combined with temporal features and spatial features. Background technique [0002] Human pose estimation is one of the important tasks of computer vision, and it has a wide range of applications in the fields of human-computer interaction, animation production, and behavior recognition. Among them, the existing research directions of human pose estimation mainly include two-dimensional human pose estimation and three-dimensional human pose estimation, although two-dimensional human pose estimation has self-occlusion, motion blur, semantic ambiguity caused by clothing, different lighting conditions, human body However, the existing research has made good research progress in the field of two-dimensional human pose estimation, and can estimate the two-dimensio...

AI Technical Summary

Problems solved by technology

[0006] In view of the above problems, the object of the present invention is to provide an end-to-end multi-view 3D human pose estimation method, system and storage medium, which can process 2D images input from multiple views and output corresponding 3D human poses, while avoiding the introduction of Poor performance caused by timing information

Images