3D scene reconstruction method, reconstruction system and application based on mixed reality glasses

Abstract

The present application provides a three-dimensional scene reconstruction method based on mixed reality glasses, a reconstruction system and an application. The three-dimensional scene reconstruction method based on mixed reality glasses includes the following steps: collecting sample images and their corresponding depth maps; using the sample images and their corresponding depth maps The depth map trains the depth estimation model; uses the acquired RGB target image and the final trained depth estimation model to perform 3D reconstruction of the real-time scene. The three-dimensional scene reconstruction method based on mixed reality glasses provided by the present application can perform three-dimensional reconstruction without the support of a depth sensor in a portable device by using a real-time depth estimation model; by using a three-dimensional reconstruction technology, the time for reconstructing a three-dimensional scene of a portable device can be reduced , to enhance the effect of the 3D reconstructed scene.

Description

technical field [0001] The present application belongs to the technical field of image processing, and in particular relates to a three-dimensional scene reconstruction method, reconstruction system and application based on mixed reality glasses. Background technique [0002] Mixed reality technology (Mix Reality, MR) is a form between virtual scenes and real scenes, including augmented reality and augmented virtuality, which refers to a new visual environment generated by merging reality and virtuality. In the new visualization environment, physical and digital objects coexist and interact in real time. 3D scene reconstruction is a core content in the field of computer vision, and has a wide range of application values ​​in computer animation, electronic entertainment, virtual display and other fields. [0003] At present, 3D reconstruction on the mixed reality glasses requires the support of hardware such as depth sensors, and the 3D reconstruction algorithm needs to rely...

AI Technical Summary

Problems solved by technology

[0003] At present, 3D reconstruction on the side of mixed reality glasses needs the support of hardware such as depth sensors, and the 3D reconstruction algorithm needs to rely on the depth map obtained by the depth sensor; however, some portable devices are not equipped with depth sensors, so the existing 3D reconstruction technology cannot be extended to portable devices

Images