Multi-angle consistent plane detection and analysis method for monocular video scene three dimensional structure

Abstract

The invention discloses a multi-angle consistent plane detection and analysis method for a monocular video scene three dimensional structure. The method includes inputting a monocular video to extract a key frame and generating a semi-dense point cloud containing noise; extracting a two-dimensional line segment in the key frame image, back-projecting the two-dimensional line segment to the three-dimensional space to obtain the corresponding point cloud; projecting the line segment obtained by single frame extraction into other key frames, filtering out the noise points in the point cloud to obtain the point cloud meeting the constraint according the constraint of multi-angle consistence and fitting the point cloud to obtain three-dimensional line segments; extracting the intersected line segments from the three-dimensional line segments and constructing a plane according to the constraint that line intersection is absolutely in the same plane, and performing detection and analysis in the three-dimensional point cloud including noise according to the constraint of multi-angle consistence to obtain the plane in a monocular video scene; and applying the reconstructed plane to augmented reality according to users' needs. The method has great performance in plane reconstruction and virtual-real fusion, and can be widely applied to the field of augmented reality.

Description

technical field [0001] The invention relates to a multi-view consistent plane detection and analysis method for a three-dimensional structure of a monocular video scene, which belongs to the fields of computer vision technology, three-dimensional reconstruction and augmented reality. Background technique [0002] In augmented reality applications, the two most important problems to be solved are restoring scene structure and fusion of virtual and real scenes. The first problem is more basic and basic. Only when the accurate scene structure is obtained can the virtual object be placed in the expected position and the accurate fusion of virtual and real scenes can be realized. [0003] In most of the current real-time single-camera reconstruction and recovery scene structure methods, the simultaneous localization and map reconstruction (SLAM, Simultaneous Localization And Mapping) method can often only obtain sparse or semi-dense point clouds rich in noise. Some methods that ...

AI Technical Summary

Problems solved by technology

[0008] To sum up, the monocular camera scene reconstruction based on the method of feature point alignment can only get sparse point cloud, and cannot get the 3D structure of the scene, while DTAM and other methods based on direct image alignment can restore the 3D scene and obtain dense points. Cloud, which is used to restore the three-dimensional structure of the scene, but often requires the assistance of GPU, which requires high computing power

However, the work of using LSD-SLAM method to obtain semi-dense point cloud and reconstruct the scene only considers the point cloud information for 3D structure reconstruction, and does not consider whether the 3D structure exists in the real 3D scene.

Images