Multi-modal information-based light field depth estimation method

Abstract

The invention discloses a multi-modal information-based light field depth estimation method, which comprises the following steps of: acquiring light field image data by using a light field camera based on a micro-lens array to obtain a four-dimensional light field image array, extracting a centermost visual angle image as a central view, exporting a group of focus slices as a focus stack, and carrying out data expansion; constructing a convolutional neural network, taking the focus stack and the corresponding center view as the input of a network model, and obtaining the input tensor of a focus stack flow and the input tensor of a center view flow; training the constructed convolutional neural network; and testing on the light field test set by using the trained neural network, and carrying out verifying on an actual focusing slice acquired by the mobile phone. According to the light field depth estimation method provided by the invention, multi-mode information of the light field can be fully utilized, and more accurate depth estimation is realized on a light field data set; the obtained depth information is more complete, and the edge is clearer; and practical application of a common consumer-level mobile phone terminal can be achieved.

Description

technical field [0001] The present invention relates to the technical field of light field depth estimation, in particular to a method for estimating light field depth based on multimodal information. Background technique [0002] Depth estimation is a key issue in the process of 3D reconstruction, and its purpose is to obtain the distance information between the target object and the photographer. The depth information of the scene can help people better understand the geometric structure of the scene, and at the same time provide data support for other visual tasks, and has important applications in the fields of scene restoration, action recognition and saliency detection. Therefore, depth estimation has become a hot research problem in computer vision. [0003] Common depth estimation methods usually extract scene depth information from single or multiple 2D images captured by conventional cameras. However, since the imaging process of traditional cameras only consider...

AI Technical Summary

Problems solved by technology

However, simple late fusion does not capture the complementarity between cross-modalities well

These problems make the predicted depth map less accurate, and there is a lot of room for improvement in some challenging scenarios, which is a problem that needs to be focused on for depth estimation based on focus stacks

Images