Three-dimensional human body posture estimation method based on feature fusion and sample enhancement

Abstract

The invention discloses a three-dimensional human body posture estimation method based on feature fusion and sample enhancement, and relates to a three-dimensional human body posture estimation and performance optimization method. The method comprises the following steps: firstly, carrying out body part classification and pixel point three-dimensional coordinate regression on a human body in a picture by adopting a full convolutional network based on a candidate region; secondly, auxiliary network sample enhancement is adopted, and signal supplementation is conducted on sample positions without initial annotations; and finally, performing feature fusion on the model and an existing 2D attitude recognition model with a good effect, and performing advantage complementarity with local regression coordinates from the perspective of global attitude. A human body posture estimation framework based on multi-task parallelism is constructed through a feature fusion technology, and an effectivetheory and method are provided for advantage complementation of two-dimensional posture recognition and three-dimensional posture recognition; an auxiliary network based on data enhancement is established in a mode of simulating semi-supervised learning, and a new thought is provided for improving the generalization ability of a posture recognition model.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular relates to a three-dimensional human body posture estimation and performance optimization method. Background technique [0002] With the rise of artificial intelligence, more and more deep learning systems such as recognition and classification of human behavior characteristics are applied to real life scenes; Human beings understand the behavioral patterns of the world, which has a wide range of application paths and application values ​​in real-world scenarios such as human-computer interaction, AR, and VR. [0003] However, in common computer vision human pose recognition models, the computer has little knowledge of the 3D world. In contrast, for humans, even when viewing 2D pictures with perspective, occlusion, depth, and interrelated human bodies in the scene, the human eye can still understand and interpret in 3D space. In deep learning models, understanding human pose fro...

AI Technical Summary

Problems solved by technology

[0004] At present, most of the research based on 3D pose recognition is based on two types of methods: (1) 3D pose recognition based on auxiliary equipment: by adding auxiliary hardware devices (such as depth sensors, six-view cameras, wearable devices, etc.), the method of collecting sample data Three-dimensional characteristics, the collected three-dimensional data is directly used as network input for training; this method has problems such as complex equipment, high cost, and huge amount of calculation, and it is difficult to be extended to daily life and applications; (2) 3D model based on 3D skeleton fitting Pose recognition: By fitting the 3D human skeleton model to the 2D picture, the 3D pose recognition of the human body in the input image is realized; this method currently has problems such as lack of sample size, fuzzy prediction results, ambiguity, and wrong flips, etc. problems, resulting in unsatisfactory accuracy

[0005] Therefore, the existence of these problems leads to 3D pose recognition analysis is still in the incomplete stage

In view of the lack of sample size, ambiguity in three-dimensional space, and local ambiguity in existing pose recognition methods, a more specific and accurate recognition method for 3D human poses in complex real-world scenes is needed

Images