Panorama-based self-supervised learning scene point cloud completion data set generation method

Abstract

The invention belongs to the technical field of three-dimensional reconstruction in the field of computer vision, and provides a panorama-based self-supervised learning scene point cloud completion data set generation method. A panoramic RGB image, a panoramic depth map and a panoramic normal map under the same viewpoint are used as input, and paired incomplete point clouds and target point clouds with RGB information and normal information can be generated to construct a self-supervised learning data set of a training ground scenic spot cloud complementation network. The key point of the invention is the processing of the stripe problem and the point-to-point shielding problem in the shielding prediction based on viewpoint conversion and the equirectangular projection and conversion process. According to the method, the acquisition mode of real scene point cloud data is simplified; a shielding prediction idea of viewpoint conversion; and a viewpoint selection strategy is designed.

Description

technical field [0001] The invention belongs to the field of three-dimensional reconstruction (3D Reconstruction) in the field of computer vision. The specific realization result is to use the panorama as input and generate a data set suitable for training scene point cloud completion network through self-supervised learning. Background technique [0002] In the process of capturing 3D scenes, there are inevitably some occluded areas, how to recover missing information from these occluded areas has become a very active research field in recent years. [0003] However, there are currently two key problems in the scene-level point cloud reconstruction method. First, in more complex scenes, it is difficult for the robot to move flexibly. The acquisition of multiple perspectives is time-consuming and laborious, and the global scene reconstruction effect is even more difficult to guarantee. Second, in an open environment, there are various types of indoor scenes, and it is diffic...

AI Technical Summary

Problems solved by technology

[0003] However, there are currently two key problems in the scene-level point cloud reconstruction method. First, in more complex scenes, it is difficult for the robot to move flexibly, the acquisition of multiple perspectives is time-consuming and laborious, and the global scene reconstruction effect is even more difficult to guarantee.

Second, in an open environment, there are various types of indoor scenes, and it is difficult to obtain sufficient data for supervised training, resulting in poor adaptability of traditional scene reconstruction methods, and it is difficult to reconstruct real point cloud scenes with better quality

However, this work is mainly for object completion under the scene layout, and does not include edge areas such as walls, ceilings, floors, etc., and does not consider the relationship between objects, lacking integrity

Images