Three-dimensional scene reconstruction method and system based on mixed reality glasses and application

Abstract

The invention provides a three-dimensional scene reconstruction method and system based on mixed reality glasses and application. The three-dimensional scene reconstruction method based on the mixed reality glasses comprises the following steps of collecting a sample image and a depth map corresponding to the sample image; training a depth estimation model by using the sample image and the corresponding depth map; and performing three-dimensional reconstruction of a real-time scene by using the obtained RGB target image and the finally trained depth estimation model. According to the three-dimensional scene reconstruction method based on the mixed reality glasses, the real-time depth estimation model is adopted, so that three-dimensional reconstruction can be carried out under the condition that the portable equipment is not supported by a depth sensor; by adopting the three-dimensional reconstruction technology, the time for reconstructing the three-dimensional scene of the portable equipment can be shortened, and the effect of reconstructing the scene in a three-dimensional manner is enhanced.

Description

technical field [0001] The present application belongs to the technical field of image processing, and in particular relates to a three-dimensional scene reconstruction method, reconstruction system and application based on mixed reality glasses. Background technique [0002] Mixed reality technology (Mix Reality, MR) is a form between virtual scenes and real scenes, including augmented reality and augmented virtuality, and refers to a new visual environment generated by merging reality and virtuality. In the new visualization environment physical objects and digital objects coexist and interact in real time. 3D scene reconstruction is a core content in the field of computer vision, and has a wide range of application values ​​in computer animation, electronic entertainment, virtual display and other fields. [0003] At present, 3D reconstruction on the side of mixed reality glasses needs the support of hardware such as depth sensors, and the 3D reconstruction algorithm nee...

AI Technical Summary

Problems solved by technology

[0003] At present, 3D reconstruction on the side of mixed reality glasses needs the support of hardware such as depth sensors, and the 3D reconstruction algorithm needs to rely on the depth map obtained by the depth sensor; however, some portable devices are not equipped with depth sensors, so the existing 3D reconstruction technology cannot be extended to portable devices

Images