Light field image depth estimation method based on deep convolutional neural network

Abstract

The invention discloses a light field image depth estimation method based on a deep convolutional neural network. The method comprises the following steps of extracting a central sub-aperture image from 4D light field data of a scene; calculating and generating a horizontal EPI composite image and a vertical EPI composite image according to the 4D light field data; designing a deep convolutional neural network taking the central sub-aperture image, the horizontal EPI synthetic image and the vertical EPI synthetic image as input and the disparity map as output; training an involved deep convolutional neural network by taking the average absolute error as a loss function; and receiving a central sub-aperture image, a horizontal EPI synthetic image and a vertical EPI synthetic image generatedby 4D light field data of a given scene by using the successfully trained deep convolutional neural network, and calculating to obtain a disparity map of the scene. The deep convolutional neural network designed by the invention adopts a multi-stream input and skip layer connection system structure, is beneficial to the fusion of multi-source input information and shallow layer deep layer featureinformation, and improves robustness of depth estimation.

Description

technical field [0001] The invention belongs to the technical field of computer vision and artificial intelligence, and specifically relates to a method for estimating the depth of a light field image based on a deep convolutional neural network. Background technique [0002] A. Gershun et al. proposed the concept of light field in the first half of the 20th century to describe the radiation characteristics of light in three-dimensional space. However, light field imaging technology lags behind the development of theoretical concepts. Light field imaging devices such as camera arrays, camera shifters, coded apertures, and microlens arrays have emerged one after another. Among them, microlens light field cameras have entered the field of consumer electronics and have great industrial application and academic research value. [0003] Depth estimation is to determine the distance between the object point in the scene and the imaging system. It is one of the basic problems in co...

AI Technical Summary

Problems solved by technology

[0005] At present, the light field depth estimation method based on deep learning mostly uses convolutional neural network as a tool for feature extraction, and innovative methods are proposed in the face of network architecture and data enhancement, but there are still insufficient utilization of EPI texture structure and feature extraction capabilities. Limited, insufficient ability to deal with occlusion and noise

Images