Multi-view stereo depth estimation method based on adversarial training

Abstract

The invention discloses a multi-view stereo depth estimation method based on adversarial training. In an adversarial training network, mapping between a network learning image and corresponding depthmap is generated, and a a discrimination network learns and distinguishes whether the depth map is from a generation module or a reference depth. During training, a generation loss function and a cross entropy adversarial loss function are combined to train the whole network. According to the method, the deep learning capability of multi-view stereo depth estimation is improved through adversarialtraining, and the contextual information of the GAN in space and time is collected in the image depth direction, so that the network is allowed to be combined with more global information. Accordingto the adversarial training network, the adversarial training of the generation module and the discrimination module is utilized, gradient penalty is adopted as an adversarial loss function of soft constraint, the training process of the original generative adversarial network is improved, the memory occupation and running time during network training and testing are remarkably reduced. And the multi-view three-dimensional depth prediction precision is improved.

Description

technical field [0001] The invention relates to the fields of object detection and three-dimensional reconstruction of computer vision, and in particular to a multi-view stereo depth estimation method based on adversarial training. Background technique [0002] Depth estimation from stereo images is the core problem of many stereo vision tasks and has applications in many fields, such as 3D reconstruction, autonomous driving, object detection, robot navigation and virtual reality, augmented reality, etc. Depth estimation is a computer vision task that aims to estimate depth from 2D images. This task inputs RGB images taken from multiple perspectives of the same scene, and outputs a depth map of the scene after processing. The depth map contains information about the distance of the objects in the image from the point of view. [0003] A typical depth estimation algorithm includes 4 steps: feature extraction, feature matching, depth calculation and depth refinement. Each s...

AI Technical Summary

Problems solved by technology

However, these methods still have some limitations

In particular, existing network operations have huge memory consumption and require powerful computing and processing capabilities, which cannot be handled for high-resolution scenes

Images