A real-time color and depth panoramic image stitching method

Abstract

The invention relates to a real-time color and depth panorama image splicing method. A depth image generated by a depth sensor is aligned with a color image, image transformation and image fusion of the depth image are carried out through utilizing the registration information of the color image, and the color and depth panorama image is generated; an image splicing flow is optimized, in one same occasion, only one-time estimation of registration parameters in a same class form is carried out, the registration parameter is directly utilized to carry out transformation and fusion for later images, the image splicing time is substantially shortened, and splicing timeliness of the panorama image is realized. The method is advantaged in that distortion of the spliced depth panorama image is smaller, the splicing method has larger flexibility, and generation of the color panorama and the depth panorama can be simultaneously accomplished. Evaluation standards based on average structure similarities are proposed, geometric quality and luminosity quality of the generated spliced images are evaluated, a system is made to realize splicing with the optimal splicing parameter, and image splicing quality is higher.

Description

technical field [0001] The invention relates to a real-time color and depth panorama image mosaic method, which can efficiently and quickly mosaic the image sequences generated by the RGBD (Red-Green-Blue-Depth) sensor array with a smaller field of view represented by Kinect, and simultaneously generate Color panoramas and depth panoramas. Background technique [0002] With the development of computer vision, the traditional computer vision technology based on two-dimensional color image processing has been unable to meet people's requirements for applying computer vision to the three-dimensional physical world. Depth image, as an image that can directly reflect object distance information, has received more and more attention. A depth image is an image that has the distance from the image sensor to each point in the scene as a pixel value. In contrast to color images, it is not affected by changes in lighting in the environment. In recent years, with the introduction of ...

AI Technical Summary

Problems solved by technology

[0008] Aiming at the defects that the depth map lacks feature points and cannot apply the stitching method of color images, the present invention first aligns the depth image of each Kinect with the color image, and then uses the registration information of the color image to register the depth image, and then completes The splicing of depth images; the image registration step in the image splicing process is very time-consuming. In order to realize the real-time splicing of panoramas, the present invention optimizes the image splicing process: in the same scene, the registration between Kinects with the same relative position relationship The parameters are only estimated and optimized once, and subsequent images are directly transformed and fused using this registration parameter

Images