A 3D reconstruction method of indoor scenes based on rgb-d images

Abstract

The invention discloses a three-dimensional reconstruction method of indoor scenes based on RGB-D images, which uses semantic segmentation results to repair depth image holes, provides object outline and category information for three-dimensional reconstruction, and obtains the shape and appearance of objects according to prior knowledge, thereby Provide more accurate data for 3D reconstruction. 3D reconstruction provides 3D spatial information for semantic segmentation, and solves mis-segmentation caused by overlapping objects and being affected by illumination in 2D image segmentation. Using multi-level camera pose estimation, sparse feature matching provides a rough estimated pose, and then through dense geometric and photometric optimization methods, an accurate camera pose is obtained to provide a more accurate camera pose for the reconstruction model. In the reconstruction process, local optimization is performed on each frame, and a key frame mechanism is added at the same time to establish global optimization and closed-loop detection, and establish constraints on the spatial points corresponding to the key frame pixels, effectively suppressing error accumulation, further optimizing the camera pose, and improving reconstruction results. accuracy.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and more specifically relates to a method for three-dimensional reconstruction of indoor scenes based on RGB-D images. Background technique [0002] The principle of the depth camera Kinect is that the infrared emitter emits infrared rays and irradiates the surface of the object to form random reflection speckles, which are then received by the depth sensor, and then the system chip operates to generate a depth image. For transparent materials and planes with missing textures, infrared rays cannot be reflected to form speckle or the effect is poor, resulting in a depth image with holes. At present, most research works use bilateral filtering method to preprocess the depth image simply. [0003] In the prior art, the 3D reconstruction based on RGB-D images mainly includes: Newcombe et al. directly calculate the 3D coordinates of the spatial points through the preprocessed depth image, and...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the defects of the prior art, the purpose of the present invention is to solve the technical problems of the prior art that the reconstruction accuracy is not high and cannot solve the mis-segmentation caused by the hole in the depth image

Images