Based on CNN semantic segmentation depth prediction method and device

Abstract

The present application discloses a depth prediction method and device based on CNN semantic segmentation, and relates to the field of semantic segmentation. The method includes: acquiring a current frame from a data set, calculating an ORB feature and a map point cloud of the current frame; according to the ORB feature and map point cloud of the current frame, determining that the current frame is selected as a key when a specified condition is satisfied frame; the CNN convolutional neural network is used to associate the key frame with the depth map to make depth prediction, and the CNN is used to label the depth map with semantic labels to achieve semantic association. The device includes: a calculation module, a selection module and a prediction module. This application no longer inputs each image into the model, which solves the memory problem, and adjusts the depth regression through CNN, which optimizes the problem of inaccurate absolute scale during 3D reconstruction.

Description

technical field [0001] The present application relates to the field of semantic segmentation, and in particular, to a method and device for depth prediction based on CNN semantic segmentation. Background technique [0002] Semantic SLAM (Simultaneous Localization And Mapping) is applied in image semantic segmentation, semantic map construction and other directions, expanding the research content of traditional SLAM problems. At present, there have been some researches that integrate semantic information into SLAM. For example, using the geometric consistency between the images obtained in the SLAM system to promote image semantic segmentation, or using the results of semantic segmentation / mapping to promote SLAM localization / loop closure, etc. [0003] Existing semantic SLAM models such as MaskFusion are essentially RGBD-SLAM+ semantic segmentation mask-rcnn, that is, the fusion of geometric SLAM model and deep learning SLAM model, which mainly solves the problem of understa...

AI Technical Summary

Problems solved by technology

[0004] However, the semantic SLAM model based on deep learning has no concept of key frames, and is calculated one by one. The monocular SLAM based on geometry cannot solve the depth problem, and is not robust enough in geometric consistency.

Images