Fast multilevel imagination and reality occlusion method at actuality enhancement environment

Abstract

The invention relates to a fast multilevel imagination and reality occlusion method at an actuality enhancement environment, comprising the following steps of: acquiring videos of a real scene by using a two-path video camera; extracting a pair of keyframes from a two-path video stream at set intervals to carry out the resolving of a dense depth map, building a three-dimensional model of a real object participating in occlusion, and extracting sparse feature points; tracking the sparse feature points of all middle frames of the two-path video stream, and estimating the posture of a current camera by combining with the positions of the sparse feature points in an image; acquiring the three-dimensional information of the real object according to a model built for the last time; moving and rotating the model of the real object according to the posture of the current camera; carrying out depth relation comparison by utilizing the three-dimensional information of the recently regulated real object with a registered three-dimensional virtual object; and respectively processing the three-dimensional tracking of the middle frames and the three-dimensional reconstruction of the keyframes in parallel at different threads. The invention is suitable for unknown and variable environments without modeling in advance and can meet the requirements for real time.

Description

technical field [0001] The invention relates to a fast multi-level virtual-real occlusion method in an augmented reality environment, which combines 3D reconstruction based on binocular stereo matching and 3D pose estimation based on sparse feature point tracking, and is applied to augmented reality with dual cameras . The invention does not need modeling in advance, is applicable to unknown and changing environments, and can meet real-time requirements. It belongs to the field of virtual reality, image processing and display technology. Background technique [0002] The application of augmented reality technology in fields such as teleoperation robots requires a certain degree of interaction between virtual objects and real objects. The visual occlusion relationship is the most basic kind of interaction. In a simple augmented reality application that does not consider the occlusion relationship, the virtual object image is always completely covered on the video image. Th...

AI Technical Summary

Problems solved by technology

However, the use of hierarchical dynamic programming algorithm for frame-by-frame dense stereo matching has the problem of long calculation time, and relatively fast algorithms often cannot obtain high-quality depth maps.

This makes real-time processing of live video streams difficult

Images