Real-time dense depth estimation method based on sparse measurement and monocular RGB image

Abstract

The invention discloses a real-time dense depth estimation method based on sparse measurement and a monocular RGB image, and the method comprises the steps of extracting more useful information from sparse depth measurement through employing a self-attention mechanism and a long and short dense jump connection technology. Meanwhile, a lightweight network design method for real-time depth estimation is provided in combination with a depth supervision technology. An experiment result verifies the effectiveness of a self-attention mechanism, a long and short dense jump connection technology and adeep supervision technology. Experimental results show that by using the method provided by the invention, the network prediction precision and the reasoning speed can be balanced to the greatest extent so as to obtain the maximum efficiency. According to the depth error estimated in real time through the method, under the condition that the sparse sampling rate is smaller than 1 / 10000, the precision of an indoor data set NYU-Depth-v2 is within 30 cm, and the precision of an outdoor data set KITTI is within 4 m.

Description

technical field [0001] The invention belongs to the technical field of robot vision positioning and navigation, and in particular relates to a real-time dense depth estimation method based on sparse measurement and monocular RGB images. Background technique [0002] Dense depth estimation plays an important role in fields such as unmanned aerial vehicles, intelligent navigation, and augmented reality. The current mainstream depth acquisition scheme consists of a high-resolution camera and a low-resolution depth sensor. These sensors are usually expensive and cannot obtain dense depth, so they are not practical for most applications. Furthermore, the accuracy and reliability of RGB-based depth estimation is far from being practical, despite more than a decade of research efforts dedicated to improving it through deep learning methods. Therefore, high-accuracy dense real-time depth estimation from single images and sparse depth measurements acquired by monocular cameras and l...

AI Technical Summary

Problems solved by technology

[0003] Compared to the problem of depth estimation from only one RGB or grayscale image, a major advantage of sparse sample-based methods is that the sparse depth measurements can be considered as part of the output ground truth, however, most current Depth estimation methods based on sparse samples all follow a similar network design to methods based on single-frame RGB images, which leads to insufficient utilization of sparse information

In response to this problem, the present invention attempts to use the self-attention mechanism and long-short dense jump connections to further improve the depth estimation accuracy based on sparse samples. In addition, the research on monocular depth estimation in the past almost all focused on improving the accuracy, resulting in intensive calculations. Type algorithms cannot be easily adopted in robotic systems, since most systems have limited computing and storage resources, especially for tiny devices, a key challenge is to balance the cost of computing time with the accuracy of the algorithm

Images