Slam method and system based on laser radar point cloud and camera image data fusion

Abstract

The invention provides a laser radar-based point cloud and camera image data fusion slam method and system, and the method comprises the steps: extracting a key frame, carrying out the object instancesegmentation of a key frame image, and obtaining an object instance in the image; performing object segmentation on the point cloud of the key frame to obtain an object in a point cloud space; fusingand unifying the object instance in the image and the object object in the point cloud space to obtain an object set; matching the objects of the front frame and the rear frame according to the object set; performing solving to obtain the pose of the camera according to the point cloud matching error of the front and rear frames, the re-projection error of the image and the object category errorof the feature points in the front and rear frames; and registering the image carrying the object instance information into a point cloud map according to the pose of the camera to obtain the point cloud map with image semantic information. According to the method, the robustness of object instance segmentation is improved, and semantic constraints are added to an optimization equation, so that the precision of the solved pose is higher.

Description

technical field [0001] The present invention relates to the technical field of mobile robot positioning and navigation, in particular to a slam method based on laser radar point cloud and camera image data fusion. In particular, it relates to a slam method for mobile robots based on multi-sensor fusion. Background technique [0002] In the field of positioning and navigation of mobile robots, Simultaneous Localization and Mapping (SLAM) refers to the robot determining its own pose in the working environment and building a map of the environment at the same time. The essence of the SLAM problem is that the robot models the environment and estimates its own pose when the environment is unknown. In order to establish a model of the surrounding environment, it is necessary to determine its own pose. The environment is modeled. [0003] SLAM problems can often be divided into front-end data association and back-end loopback optimization. The existing SLAM methods based on lidar...

AI Technical Summary

Problems solved by technology

The traditional slam method relies on low-level feature information such as points and lines to estimate the pose, and its robustness is low in complex outdoor environments, especially in scenes with sparse features

Images