Real-time depth and confidence prediction method based on binocular camera

Abstract

A real-time depth and confidence prediction method based on a binocular camera comprises the steps that a neural network model of a specific framework is constructed, the neural network model adopts aneural network framework of encoding and decoding, three data sets are prepared, an encoding part of a neural network is trained on a classification data set, and parameters of the encoding part of the neural network are frozen; the method also includes training parameters of a decoding part of the neural network on the artificially synthesized data set, unfreezing all parameters of the neural network after loss convergence, continuing training on the artificially synthesized data set, finely adjusting the parameters of the neural network on the data set of the real scene, and testing the neural network on the test set. According to the invention, only 2D convolution is adopted in the neural network, branches for predicting the confidence coefficient are increased, multiple kinds of information are aggregated through series operation in the sub-networks, the obtained neural network model has higher reasoning speed in a low-end GPU and embedded equipment with lower energy consumption,and the corresponding confidence coefficient can be given.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method for predicting real-time depth and confidence based on a binocular camera. Background technique [0002] Visual depth information is an important part of allowing computers to understand our physical world. Accurate and fast prediction of depth will have a positive and important impact on various fields such as unmanned driving, 3D reconstruction, mapping and positioning. Existing visual depth prediction methods mainly focus on devices such as monocular cameras, binocular cameras, and lidars. Monocular cameras use a single picture to predict the depth of an object. In principle, it cannot distinguish the size of the image. Whether a small object is a small object close to the camera or a large object far away from the camera, its scale is uncertain, and the depth prediction scheme of lidar is currently expensive and can only perform sparse depth prediction. The ...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art, and provide a real-time depth and confidence prediction based on a binocular camera that solves technical problems such as slow reasoning, lack of corresponding confidence, and blurred object outlines in the prior art. method

Images