Human body 3D reconstruction method and device for complex dynamic scene under multi-view camera

Abstract

The invention discloses a human body 3D reconstruction method and device for a complex dynamic scene under a multi-view camera. The method comprises the following steps: photographing a target human body object at multiple views to obtain 2D images at multiple views under the same moment; predicting all the parts of the human body in the video sequence through a model after deep network learning to obtain the human body skeleton information of the human body; matching the movement and rough contour of the character in the images through a virtual 3D human body model according to the skeleton information to obtain the human body contour information of the human body; and carrying out human body 3D modeling through a Virtual Hull method according to the calibrated internal and external parameter information of the camera at multiple views as well as the human body skeleton information and the human body contour information. Through the method, 3D reconstruction of a human body object ina video sequence can be realized through camera calibration and other processes, and therefore, accurate character object segmentation is realized, and the accuracy and reliability of reconstruction are improved.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method and device for three-dimensional reconstruction of a human body in complex dynamic scenes under multi-view cameras. Background technique [0002] In computer vision, 3D reconstruction is the process of reconstructing 3D information from single-view or multi-view images. Since the information of a single video is incomplete, 3D reconstruction needs to use prior knowledge, while multi-view 3D reconstruction can use more The information of the two-dimensional image of the viewpoint is used to reconstruct the three-dimensional model. However, most of the current 3D reconstruction algorithms do not use 2D information accurately and comprehensively enough, and the calculation process relies too much on information provided by external devices, such as depth information provided by depth cameras, or on the segmentation results of objects and backgrounds, etc. , resulti...

AI Technical Summary

Problems solved by technology

However, most of the current 3D reconstruction algorithms do not use 2D information accurately and comprehensively, and the calculation process relies too much on information provided by external devices, such as depth information provided by depth cameras, or on the segmentation results of objects and backgrounds, etc. , resulting in the reconstructed result is still relatively rough

Images