Robot indoor environment three-dimensional semantic map construction method based on deep learning

Abstract

The invention provides a robot indoor environment three-dimensional semantic map construction method based on deep learning. The method comprises the following steps: firstly, an RGB image sequence and a depth image sequence of an indoor environment are collected through a depth camera; ORB feature extraction and matching are carried out on each collected frame of RGB image, and a key frame is determined; the extracted and matched feature point pairs are used to calculate a pose transformation matrix T between two adjacent frames of images through an ICP algorithm; semantic segmentation is performed on the determined key frame by using a trained deep learning network to obtain images of the key frames classified according to pixels; and point cloud splicing is performed on the segmented key frame images in combination with the calculated transformation matrix T and the depth images corresponding to the key frames to obtain a semantic map which can be understood by the robot. Semantic map construction can be carried out by directly utilizing the segmented key frame images, and semantic fusion does not need to be carried out after the environmental geometric map is established.

Description

technical field [0001] The invention belongs to the technical field of mobile robot visual environment perception, in particular to a method for constructing a three-dimensional semantic map of a robot's indoor environment based on deep learning. Background technique [0002] Home service robots generally have the following three core technologies: environmental perception, human-computer interaction, and motion control. The perception and understanding of the environment is undoubtedly a core technology for indoor mobile robots to perform tasks. The traditional method of obtaining information about the surrounding environment of the robot is to establish an indoor environment map through 2D laser SLAM (Simulntenous Localization And Mapping, simultaneous positioning and mapping) technology, which has great limitations. First, the map created by lidar is two-dimensional and lacks three-dimensional spatial information. When navigating and avoiding obstacles, it can only avoid...

AI Technical Summary

Problems solved by technology

This method needs to construct the geometric map of the environment first, then segment the color image to obtain semantic information, and then fuse the semantic information and geometric information to obtain the semantic map of the environment, which requires a large amount of calculation and requires high computing power of the computer. Not conducive to deployment on mobile robots

Chinese patent CN104732587B discloses a depth sensor-based indoor 3D semantic map construction method. In the method proposed, each frame of color images collected needs to be semantically segmented. However, it contains a large number of redundant images, which reduces the construction efficiency

Images