Semantic mapping and positioning method based on priori laser point cloud and depth map fusion

Abstract

The invention discloses a semantic mapping and positioning method based on priori laser point cloud and depth map fusion. The method comprises the steps of S1, collecting priori laser point cloud data; S2, acquiring a depth image and an RGB image, generating RGB-D point cloud based on the depth image, and initializing and registering priori laser point cloud and RGB-D point cloud; S3, camera poseconstraints are provided by the registered prior laser point cloud to perform camera pose correction; S4, a three-dimensional geometric point cloud map is created by adopting a front and back window optimization method; S5, geometric increment segmentation is carried out on the three-dimensional geometric point cloud map, object recognition and semantic segmentation are carried out on the RGB image, geometric increment segmentation and semantic segmentation results are fused, and a 3D geometric segmentation map of semantic enhanced geometric segmentation is obtained; and S6, semantic association and segmentation probability allocation updating is carried out on the object to complete construction of a semantic map. Accumulated errors of large-scale indoor mapping and positioning can be effectively eliminated, and precision and real-time performance are high.

Description

technical field [0001] The invention relates to the technical field of positioning and map construction, in particular to a semantic mapping and positioning method based on fusion of prior laser point clouds and depth maps. Background technique [0002] At present, the main technical difficulties hindering the integration of virtual reality and virtual reality of augmented reality in large indoor and outdoor scenes, unmanned driving and robot navigation and positioning are the dynamic tracking and recording of camera positions and poses in real scenes and the construction of 3D semantic maps. Semantic map refers to assigning corresponding attribute values ​​to traditional 3D point cloud maps (for example: ground, walls or buildings). The requirements for accuracy and robustness cannot be met, and the error of the inertial attitude sensor gradually increases with time drift. The real-time 3D mapping and camera position and attitude calculation using computer vision and 3D lid...

AI Technical Summary

Problems solved by technology

Currently, application scenarios are usually limited to small workspaces, but robust and real-time localization of unknown cameras in large indoor and outdoor scenes remains challenging and may prevent widespread use of localization and mapping techniques; moreover, Due to the complex and dynamic changes in the lighting conditions and texture features of large indoor and outdoor scenes, feature points are often lost or the matching between point cloud frames based on a single RGB video image or lidar point cloud is often lost. Therefore, in an unknown environment, a single sensor cannot meet the requirements. Robustness and Real-time Requirements for 3D Semantic Reconstruction and Localization of Large Scenes

Images