A real-time color and depth panoramic image stitching method

Abstract

The invention relates to a real-time color and depth panoramic image splicing method, which aligns the depth image and the color image generated by the depth sensor, uses the registration information of the color image to perform image transformation and image fusion on the depth image, and then generates the color and depth panorama. Image; the image splicing process is optimized. In the same scene, the registration parameters of the same group type are estimated only once. Subsequent images directly use this registration parameter for transformation and fusion, which greatly reduces the image splicing time and achieves Panoramic image stitching in real time. The spliced ​​depth panorama of the present invention has smaller distortion, the splicing method is more flexible, and the generation of a color panorama and a depth panorama can be completed at the same time. The present invention proposes an evaluation standard based on average structural similarity to evaluate the geometric quality and photometric quality of the generated spliced ​​image, so that the system can always perform splicing with optimal splicing parameters, and the quality of image splicing is higher.

Description

technical field [0001] The invention relates to a real-time color and depth panorama image mosaic method, which can efficiently and quickly mosaic the image sequences generated by the RGBD (Red-Green-Blue-Depth) sensor array with a smaller field of view represented by Kinect, and simultaneously generate Color panoramas and depth panoramas. Background technique [0002] With the development of computer vision, the traditional computer vision technology based on two-dimensional color image processing has been unable to meet people's requirements for applying computer vision to the three-dimensional physical world. Depth image, as an image that can directly reflect object distance information, has received more and more attention. A depth image is an image that has the distance from the image sensor to each point in the scene as a pixel value. In contrast to color images, it is not affected by changes in lighting in the environment. In recent years, with the introduction of ...

AI Technical Summary

Problems solved by technology

[0008] Aiming at the defects that the depth map lacks feature points and cannot apply the stitching method of color images, the present invention first aligns the depth image of each Kinect with the color image, and then uses the registration information of the color image to register the depth image, and then completes The splicing of depth images; the image registration step in the image splicing process is very time-consuming. In order to realize the real-time splicing of panoramas, the present invention optimizes the image splicing process: in the same scene, the registration between Kinects with the same relative position relationship The parameters are only estimated and optimized once, and subsequent images are directly transformed and fused using this registration parameter

Images