An illumination estimation method based on monocular camera

Abstract

The invention relates to computer graphics, which is an illumination estimation method based on a monocular camera. The method comprises the following steps: a monocular camera acquires an RGB image as an input of a depth estimation; Convolution neural network for depth estimation of monocular camera is constructed and trained. The RGB image is input to the trained convolution neural network for depth estimation, and the depth prediction value is obtained, and the depth prediction map is output. The depth prediction value is upsampled to match the size of the depth prediction map with the RGBimage, and the upsampled depth prediction value is used as the input of the illumination estimation. The RGB image is transformed into CIELab color space, and the luminance channel information is usedas the input of illumination estimation. The spherical harmonic coefficients of the light source information in all directions of the real scene can be obtained by using the luminance channel information and the depth prediction value to estimate the illumination. This method uses monocular camera to obtain the light source information in each direction of the real scene, and effectively improvesthe fidelity of rendering virtual object in AR technology.

Description

technical field [0001] The invention relates to the fields of computer graphics and artificial intelligence, in particular to an illumination estimation method based on a monocular camera. Background technique [0002] There are two main categories of existing techniques for illumination estimation. One is to obtain additional scene information with the help of external hardware devices such as fisheye cameras and depth cameras, so as to estimate the lighting conditions of the real scene. This type of lighting estimation technology requires external hardware devices, which will undoubtedly increase the cost of lighting estimation, and mobile devices such as mobile phones need to consider portability, and it is almost impossible to configure these hardware devices. The other is to place some regular objects with known reflection characteristics in the scene and shoot them, and estimate the lighting information according to the lighting effect of the regular objects in the im...

AI Technical Summary

Problems solved by technology

The current mainstream augmented reality technology only achieves a rough estimate of the ambient light intensity in a given camera image (such as Google's ARCore platform), and cannot estimate the light source information in all directions in the real environment.

There are also some studies that can estimate the light source information in all directions in the real environment, but it is necessary to capture the light information with the help of external hardware devices or light probes that invade the lens

However, existing lighting estimation techniques are limited to additional hardware devices or invasive light probes, and are not suitable for mobile devices with only monocular cameras.

Images