Multi-view real-time human motion, gesture, expression, texture reconstruction system

Abstract

The present invention provides a multi-viewpoint real-time human body movement, gesture, expression, and texture reconstruction system. The multi-viewpoint real-time human body movement, gesture, expression, and texture reconstruction system includes: enclosing a plurality of camera frames out of the capture area, and using a camera calibration method Calibrate the camera internal parameters and camera external parameters of the multiple cameras; collect human body images through the multiple cameras after calibration, and process the human body images so that the human body images are transcoded into RGB images, and then complete the single Objective human body posture estimation; obtain the heat map and joint affinity of each joint of the human body through the single-purpose human body posture estimation, and perform non-maximum suppression on the joint heat map to obtain the coordinates of each joint; thereby obtain the three-dimensional joint coordinates of the human body, thereby obtaining 3D reconstruction model of human body. This system uses deep learning to complete the estimation of human body posture. In the experimental environment, it can fit and render human body models of multiple people in real time.

Description

Technical field[0001]The present invention relates to the field of computer visual technology, in particular to a multi-view real-time human motion, gesture, expression, and texture reconstruction system.Background technique[0002]With the improvement of computer computing power and the continuous iteration of graphics cards, deep learning techniques have developed rapidly, which greatly promotes the computer visual field. The current reconstruction technique is mainly divided into two kinds. One is to use ordinary RGB cameras, with a multi-purpose feature point matching and triangularization to obtain depth information, the other is to directly utilize depth cameras to obtain a depth map, such as Apple The new iPhoneX published a depth camera to complete the face rebuild, and push this technology to the consumer.[0003]However, with respect to the RGB camera, the disadvantage of the depth camera is large, the depth detection distance is limited, and the price is expensive, so the RGB...

AI Technical Summary

Problems solved by technology

[0005] It solves the problem that most of the traditional methods for human body reconstruction use wearable sensors or green screen segmentation, which have high environmental requirements

Images