Construction method and device of three-dimensional semantic map, electronic equipment and storage medium

Abstract

The invention relates to a construction method and device of a three-dimensional semantic map, electronic equipment and a storage medium, and the method comprises the steps: obtaining an environment image set, carrying out semantic segmentation of the environment image set according to a trained semantic segmentation model, and obtaining a semantic image sequence; and projecting each frame of semantic image of the semantic image sequence to a pre-established three-dimensional coordinate system to obtain a first point cloud set, the first point cloud in the first point cloud set corresponding to each frame of semantic image; filtering the first point cloud set to obtain a filtered first point cloud set; clustering the first point cloud in the filtered first point cloud set to obtain a second point cloud set; and filtering the second point cloud set to obtain a three-dimensional semantic map. According to the invention, a color image sequence and a depth image sequence are combined as the input of the semantic segmentation model, so that the semantic prediction capability can be improved, filtering is carried out hierarchically based on the point cloud with semantics, the cache can be saved, and the real-time performance can be improved.

Description

technical field [0001] The present application relates to the technical field of machine vision, and in particular to a method, device, electronic equipment and storage medium for constructing a three-dimensional semantic map. Background technique [0002] Since the 21st century, artificial intelligence has received more and more attention in the computer field, its theory and technology have become increasingly mature, and its application fields have continued to expand. Robots are the most direct and common application of artificial intelligence technology. In the future, robots will be used as new production tools to improve human productivity and continuously expand the field of human activities. Intelligent robots can perform correct dynamic decision-making and planning, behavior control, task execution and other activities in real scenes like humans. The basis and key lies in the robot's correct perception of the world. For many tasks, such as rescuing trapped people ...

AI Technical Summary

Problems solved by technology

However, the current semantic mapping system is difficult to achieve real-time, in addition to the 2D semantic segmentation network takes a lot of time, and a large part of the reason is that the semantic fusion algorithm based on probability is difficult to achieve real-time in principle.

In addition, the existing semantic fusion algorithms all use the semantic information of n frames before and after the fixed fusion, without considering that if the perspective difference between the two frames is too large, the captured textures are also very different, and the predicted semantic categories are likely to be different. Integrating such semantic conflicts due to large viewing angle changes has little meaning for the semantic segmentation of the entire scene, and may even have a bad effect on correct predictions.

Images