Method and device for 3D reconstruction of gigapixel scene crowd based on implicit field

Abstract

The present invention provides a method and device for three-dimensional reconstruction of a billion-pixel scene crowd based on an implicit field, wherein the method includes: acquiring a billion-pixel scene image; detecting people and people in the billion-pixel scene and segmenting them; The segmented scene images are extracted by deep neural network, and the shape and posture of the person are estimated; the images of different perspectives of the human body are rendered using the high-precision scanned human body model as training data; the human body reconstruction under occlusion is trained; Visibility-based importance sampling is used to enhance the density of sampling points and increase the weight of the visible part of the 3D human body to enhance the details of the model. Convolutional neural network is used to extract features of training images, and attention mechanism is used to associate 3D human prior and implicit field model to complete high-precision reconstruction of human body under occlusion. The 3D human body obtained in the gigapixel scene is combined with the implicit field model to generate a high-resolution 3D reconstruction model of the crowd.

Description

technical field [0001] The present invention relates to the technical field of three-dimensional reconstruction in computer vision, and in particular, to a method and device for three-dimensional reconstruction of a billion-pixel scene crowd based on an implicit field. Background technique [0002] In recent years, with the development of computer vision technology and the improvement of computing hardware performance, the field of 3D reconstruction in computer vision has developed rapidly, among which 3D reconstruction of human body is a research hotspot. High-precision and accurate 3D reconstruction of the human body has great application value in intelligent communication, security monitoring, entertainment services and other fields. [0003] With the proposal of implicit field and the development of deep learning, great progress has been made in 3D reconstruction of human body from a single RGB image. However, the accuracy of the implicit field method is limited by the ...

AI Technical Summary

Problems solved by technology

However, the accuracy of the implicit field method is limited by the resolution, and it is difficult to apply to scenes with many people and dense crowds. It is also difficult to obtain stable reconstruction results under different lighting environments, and there are many limitations.

Images