Visual semantic SLAM system and method based on neural network technology

Abstract

The invention discloses a visual semantic SLAM system and method based on a neural network technology, and the method comprises the steps: carrying out the screening of key frames from an inputted image, and generating a key frame queue; obtaining matching information of a target by using target tracking; continuously carrying out target detection pose estimation and state estimation on the generated key frames, sequentially obtaining the key frames to carry out semantic analysis, carrying out foreground and background separation, and carrying out local mapping; sequentially acquiring each frame to perform image expression, generating a scene description index library, performing back-end optimization on the basis, establishing a new global map by using the scene description index libraryand a local map, performing loop detection, and performing continuous optimization and information updating on the map. According to the system and the method designed by the invention, the SLAM optimization performance can be improved, the understanding and description of environment semantics are enhanced, a map which is more beneficial to understanding and practicability is established, and therobustness and the expansion capability are better.

Description

technical field [0001] The invention belongs to the fields of simultaneous positioning and mapping and image semantics in computer vision, and specifically relates to a visual semantic SLAM (Simultaneous Localization and Mapping) system and method based on neural network technology. Background technique [0002] Synchronous positioning and map construction technology is a relatively popular research field in recent years. This technology can effectively solve the two main problems of robots locating themselves in unknown environments and sensing the surrounding environment at the same time. At present, after decades of development, visual SLAM has formed a relatively traditional mature framework. For example, Mur-Artal et al. proposed ORB-SLAM (A Versatile and Accurate Monocular SLAM System.IEEE Transactions on Robotics, vol.31 , no.5, pp.1147-1163), the feature point method used in the ORB-SLAM system has high requirements on the texture scene, and the established spatial s...

AI Technical Summary

Problems solved by technology

Its innovation lies in the use of hole convolution for semantic pixel-level segmentation. The system has rich semantic information, but the system only uses its pixel information to directly construct semantic maps in the point cloud without further use. The semantic information is not fully utilized, and the technology uses single

[0005] The existing SLAM technology semantic information is relatively lacking and single-use. Pure visual SLAM is limited by the environment in real-world applications, requiring additional sensors for technology fusion, such as Bluetooth, gyroscope, infrared and other devices, forming a complex technology fusion SLAM system

As the computing power and algorithm development of GPU devices keep pace with the times, it provides conditions for the implementation of complex neural network systems. As neural network technology is more and more widely used in the field of images, the understanding of semantic information of scenes is getting better and better. Abstraction, but at present, there is still no complete, general and robust SLAM system

Images