A Depth Camera Localization Method Based on Convolutional Neural Network

Abstract

The present invention provides a depth camera positioning method based on a convolutional neural network, comprising the following steps: establishing a convolutional neural network, the convolutional neural network includes a graph fusion generation module and a graph neural network module, and the graph fusion generation module It is used to jointly encode the color image obtained by the depth camera and the depth image to generate an interconnected fusion image, and the graph neural network module is used to process the fusion image; the convolutional neural network is trained based on an offline sample set, and the The above offline sample set includes batch images taken by the depth camera in the area to be located and the camera position when each frame of image is taken; when positioning is required, use the depth camera to take a set of color images and depth images in the area to be located as input , outputting the position of the camera for capturing the set of image data through the convolutional neural network. Through the method of the present invention, the color image acquired by the depth camera and the depth image can be fused and input to calculate the camera pose.

Description

technical field [0001] The invention relates to the technical field of robot vision positioning, in particular to a depth camera positioning method based on a convolutional neural network. Background technique [0002] Positioning refers to determining the position and heading of the mobile robot relative to the global coordinates in the operating environment, and is the most basic link in the navigation of the mobile robot. At present, according to the different types of sensors and information used, the mainstream positioning technologies include magnetic stripe positioning, laser positioning, two-dimensional code positioning, visual positioning, etc. Among them, visual positioning technology is similar to human beings who determine their own position by observing the surrounding environment with their eyes, and capture image information through a visual camera, process and calculate it, so as to obtain the position of the robot in space. Due to the low cost of visual cam...

AI Technical Summary

Problems solved by technology

There is no complete set of methods for directly merging pose calculations using color maps and depth maps, and the information of the two is not deeply fused, resulting in a large computational complexity

At the same time, since there are certain problems in the positioning stability of a single color image or a single depth image, the reliability of the above simple combination of the two to calculate the pose is further reduced, and it is prone to positioning failures.

[0010] In terms of using convolutional neural networks to perform pose calculations on depth maps, since ordinary convolutions cannot fully extract the features of unstructured point clouds, adjacent pixels can be far apart in space.

And for the task of judging the pose, the normalization of the depth map will lose most of the information. Therefore, the current general-purpose convolutional neural network cannot successfully process the depth map information.

Images