End-to-end multi-view three-dimensional human body posture estimation method and system and storage medium

Active Publication Date: 2021-03-26
UNIVERSITY OF CHINESE ACADEMY OF SCIENCES
View PDF3 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of the above problems, the object of the present invention is to provide an end-to-end multi-view 3D human pose estimation method, system and storage medium, w

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • End-to-end multi-view three-dimensional human body posture estimation method and system and storage medium
  • End-to-end multi-view three-dimensional human body posture estimation method and system and storage medium
  • End-to-end multi-view three-dimensional human body posture estimation method and system and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0122] Example:

[0123] In this embodiment, the Human3.6M dataset (Human3.6M: Large Scale Datasets and Predictive Methods for 3D HumanSensing in Natural Environments), the largest multi-view 3D human pose estimation dataset at present, is used, which consists of four datasets in time. Synchronized 50Hz camera shooting, using the marker-based MoCap system to collect 3D human pose data, the data set contains a total of 3.6 million images, consisting of 11 sets of data including 5 sets of female data and 6 sets of male data. The 1st, 5th, 6th, 7th, and 8th data sets of 1.5 million images are used as training sets, and the 9th and 11th sets of data are used as test sets. One-fifth of the complete training set and two-dimensional human pose data sets COCO and MPII are extracted at intervals of 4 frames as the training set of the two-dimensional human pose estimation network Resnet-152, so that the training samples have samples similar to the complete training data. distribution, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an end-to-end multi-view three-dimensional human body posture estimation method and system and a storage medium, and the method comprises the steps: loading a pre-trained two-dimensional human body posture estimation network, and enabling a picture of each current view to serve as the input of the network; generating a thermodynamic diagram through a two-dimensional humanbody posture estimation network, and taking the thermodynamic diagram as the input of an LSTM thermodynamic diagram time sequence information extraction network; inputting the thermodynamic diagram into an LSTM initialization thermodynamic diagram time sequence information extraction network and an LSTM thermodynamic diagram time sequence information extraction network according to the value of the time sequence step length T to obtain a cell state and a hidden state; feeding the obtained hidden state into a decoder network to obtain a decoded thermodynamic diagram; fusing the thermodynamic diagram and the decoded thermodynamic diagram to obtain a thermodynamic diagram Ht (p) fused with time and space information; sending the thermodynamic diagram Ht (p) into a soft-argmax linear algebraictriangulation network to obtain a 2D point position; and solving an overdetermined equation on the homogeneous three-dimensional coordinate vector, and adopting a differentiable DLT-SII algorithm toobtain a final three-dimensional human body posture estimation point.

Description

technical field [0001] The present invention relates to the field of computer vision, in particular to an end-to-end multi-view 3D human pose estimation method, system and storage medium based on a deep learning network combined with temporal features and spatial features. Background technique [0002] Human pose estimation is one of the important tasks of computer vision, and it has a wide range of applications in the fields of human-computer interaction, animation production, and behavior recognition. Among them, the existing research directions of human pose estimation mainly include two-dimensional human pose estimation and three-dimensional human pose estimation, although two-dimensional human pose estimation has self-occlusion, motion blur, semantic ambiguity caused by clothing, different lighting conditions, human body However, the existing research has made good research progress in the field of two-dimensional human pose estimation, and can estimate the two-dimensio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/049G06N3/08G06V40/20G06N3/047G06N3/048G06N3/045G06F18/25
Inventor 薛健牛泽海吕科
Owner UNIVERSITY OF CHINESE ACADEMY OF SCIENCES
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products