Robust vision SLAM method based on semantic prior and deep learning features

Abstract

The invention relates to a robust vision SLAM method based on semantic prior and deep learning features. The method comprises the following steps: (1), building a vision SLAM framework based on deep learning feature extraction, enabling a tracking thread of the framework to input an image obtained by a camera sensor into a deep neural network, and extracting depth feature points; (2) based on a lightweight semantic segmentation network model, performing semantic segmentation on the input video sequence to obtain a semantic segmentation result, and obtaining semantic prior information of a dynamic object in the scene; (3) removing the depth feature points extracted in the step (1) according to the semantic prior information in the step (2), removing the feature points on the dynamic object,and improving the positioning precision in the dynamic scene; and (4) obtaining a static point cloud corresponding to the key frame selected by the tracking thread according to the semantic segmentation result in the step (2), performing static point cloud splicing according to the key frame pose obtained in the step (3), and constructing a dense global point cloud map in real time.

Description

technical field [0001] The present invention relates to a robust visual SLAM method based on semantic prior information and deep learning features, which is a visual SLAM algorithm that combines semantic prior information and more robust deep learning features, and has better performance in weak texture and dynamic scenes. good adaptability. Background technique [0002] Visual SLAM uses cameras and other sensors as sensors. The cost is low and the information obtained is closer to the cognitive level of human beings. It has been widely used in mobile robots and other fields. Compared with the outdoor scene, the indoor scene has no drastic illumination changes, and the robot moves at a lower speed, so it is the main workplace of the mobile robot. Although indoor navigation is safer than outdoor environments, the indoor environment is more complex and obstacles are denser. To apply SLAM technology to indoor robot navigation, at least two challenges need to be solved. [000...

AI Technical Summary

Problems solved by technology

[0006] Regardless of whether it is an indoor or outdoor environment, the existence of dynamic objects is undoubtedly one of the major challenges to the positioning accuracy and robustness of visual SLAM. The mainstream visual SLAM framework does not consider the impact of dynamic objects, assuming that the surrounding environment is static, and The changes in the acquired images are all caused by the movement of the camera, which brings challenges to the SLAM algorithm for data association between different frames

For example, the movement of furniture in the home and the walking of people in the office. If these dynamic objects are constructed as part of the environment, on the one hand, the estimation accuracy of the visual SLAM algorithm's own pose will be reduced.

On the other hand, simply using geometric information to construct maps for dynamic scenes, the static maps constructed are not enough for indoor navigation

Images