Terrain semantic perception method based on vision and vibration tactile fusion

Abstract

The invention provides a terrain semantic perception method based on vision and vibration touch fusion. The terrain semantic perception method comprises the steps: firstly, giving an implementation method of vision three-dimensional semantic mapping based on ORB_SLAM2 and semantic segmentation; secondly, in combination with a terrain semantic classification and recognition method based on CNN-LSTM, giving a realization thought and a fusion strategy of vision / touch fusion; and finally, based on the blue whale XQ unmanned vehicle platform, the Kinect V1.0 visual sensing unit and the vibration sensing unit, carrying out algorithm testing in a real object environment. Therefore, the semantic marking precision of the method obtained by comparing a test result with a real environment can meet the application requirements; and meanwhile, whether the terrain semantic cognition is good or not can be obviously compared according to the fusion result of whether the vibration touch exists or not,so that more reliable sensing capacity can be provided for the patroller through fusion of the vibration touch and the terrain semantic cognition, and the vibration touch can still provide terrain cognition precision within a limited range even under the condition of visual failure.

Description

technical field [0001] The invention belongs to the technical field of terrain semantic perception, and in particular relates to a terrain semantic perception method based on fusion of vision, vibration and touch. Background technique [0002] Research on terrain semantic perception technology has not yet been reported in depth, but there have been some studies in the fields of environmental semantic mapping and semantic segmentation. The following is an analysis of the current technology development status and trends from these aspects. [0003] Kostavelis gave an overview of the research on semantic mapping of mobile robots, and analyzed it from multiple aspects such as category composition, development trend, and practical application. In 2016, the Davison team proposed a dense three-dimensional semantic mapping method based on convolutional neural network, that is, SemanticFusion. Through the combination of CNN and dense SLAM system, the improvement from traditional geom...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the problem of insufficient awareness of terrain semantics in the perception mode in the prior art, and propose a terrain semantics perception method based on the fusion of vision and vibration and touch

Images