Visual SLAM method based on semantic segmentation of deep learning

A technology of semantic segmentation and deep learning, applied in the field of computer vision sensing, it can solve the problems of different visual appearance, inability to realize loopback detection, etc., and achieve the effect of enhancing robustness, improving accuracy and excellent accuracy.

Pending Publication Date: 2020-12-25
ARMY ENG UNIV OF PLA
View PDF0 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When the robot returns to the previously visited place again, the dynamic object object disappears, and the same scene matching is performed at this time, however, this has a different visual appearance for the robot, and the loop detection process cannot be achieved

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Visual SLAM method based on semantic segmentation of deep learning
  • Visual SLAM method based on semantic segmentation of deep learning
  • Visual SLAM method based on semantic segmentation of deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0068] A visual SLAM method based on semantic segmentation of deep learning:

[0069] Step 1. After obtaining the image data collected by the RGB-D depth camera, extract the ORB feature points of the original RGB image, and use the DUNet semantic segmentation network to perform pixel-level semantic segmentation on the RGB image to obtain the semantic segmentation mask. The DUNet network model loaded in our visual SLAM system is trained based on the PASCAL VOC dataset, which can detect a total of 20 categories when used for segmentation tasks. If in the real application process, more than Pascal VOC data is encountered We can also use the larger MS COCO dataset to train our deep learning network. The input of the DUNet network is the original RGB image of size h×w×3, and the output of the network is a matrix of size h×w×n, where h represents the pixel height of the image, w represents the pixel width of the image, and n represents the Number of dynamic objects. For each outpu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a visual SLAM method based on semantic segmentation of deep learning, and relates to the technical field of computer vision sensing. The method comprises the following steps: acquiring an image through an RGBD depth camera, and performing feature extraction and semantic segmentation to obtain extracted ORB feature points and a pixel-level semantic segmentation result; detecting a moving object based on a multi-view geometric dynamic and static point detection algorithm, and deleting ORB feature points; and executing initialization mapping: sequentially executing tracking, local mapping and loopback detection threads, constructing an octree three-dimensional point cloud map of a static scene according to the key frame pose and a synthetic image obtained by a static background restoration technology, and finally realizing a dynamic scene-oriented visual SLAM method based on deep learning semantic segmentation. The precision of camera pose estimation and trajectoryevaluation of the visual SLAM in the dynamic scene is improved, and the robustness, stability and accuracy of the performance of a traditional visual SLAM system in the dynamic scene are enhanced.

Description

technical field [0001] The invention relates to the technical field of computer vision sensing, in particular to a visual SLAM method combining semantic segmentation and a moving and static point detection algorithm based on multi-view geometry. Background technique [0002] SLAM, the full name is Simultaneous Localization and Mapping, that is, simultaneous positioning and mapping. It is the basic technology for some robot applications, such as industrial automation robots, self-driving cars and UAV obstacle avoidance navigation. In the mid-1980s, Smith et al. first proposed SLAM application technology. This computer vision technology refers to that when an autonomous robot moves in an unknown scene, it uses special external sensors to obtain environmental information data and combines previous position information to estimate its current motion posture, while gradually building incremental A map of the external environment. After decades of development and progress, SLAM ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/73G06T5/00G06K9/46G06K9/34G06K9/00
CPCG06T7/73G06T2207/10016G06V20/10G06V10/267G06V10/462G06T5/77
Inventor 艾勇保芮挺赵晓萌方虎生符磊何家林陆明刘帅赵璇
Owner ARMY ENG UNIV OF PLA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap