A virtual and real occlusion processing method based on depth image data stream

Abstract

The invention relates to a virtual-reality occlusion handling method based on depth image data flow. The virtual-reality occlusion handling method comprises three parts of construction of a scene point cloud model, three-dimensional space registration and virtual-reality occlusion handling and rendering. Firstly filtering and other processing operation are performed on depth data acquired by a depth camera and the normal vector of each point is calculated; then the camera attitude is calculated by using an iterative closest point algorithm according to the point cloud carrying the normal vector and the point cloud obtained by projection from a three-dimensional scene model through the last frame of camera attitude; then the point cloud of the current frame is fused into the three-dimensional scene point cloud model; when the scene is reconstructed, the color image feature points acquired by the depth camera are calculated in real time and three-dimensional space registration is performed by matching with the template image feature points; and then the space position relation and the occlusion relation of the virtual object and the three-dimensional scene are processed by the obtained camera attitude and rendered in real time. The method can be operated in real time on the present mainstream equipment, and the great virtual-reality occlusion effect can also be obtained when the resolution of the input data is low.

Description

Technical field [0001] The invention belongs to the field of computer vision and computer graphics image processing, and specifically is a virtual and real occlusion processing method based on a depth image data stream. The method can be used when the input data resolution is low and the depth data has holes and noise. Estimate the camera pose in real time and reconstruct the point cloud model of the 3D scene, and process the occlusion relationship between the virtual object and the 3D scene in real time according to the camera pose, and fuse the occlusion processing result with the color image to achieve a virtual and real fusion effect , Is of great significance to the research of 3D reconstruction system and real-time augmented reality (AR) technology. Background technique [0002] Augmented reality is a technology that superimposes virtual objects into the real environment to achieve the fusion effect of virtual and real. In recent years, it has become a research hotspot in t...

AI Technical Summary

Problems solved by technology

[0004] Some of the above methods are time-consuming, such as the extraction of feature points, the optimization of energy equations and other steps, the lack of three-dimensional information of the real scene, it is difficult to deal with the occlusion relationship between virtual objects and real scenes, and the fusion of virtual and real lacks realism

Images