Panoramic feature splicing system with virtual viewpoint and method thereof

Abstract

The invention discloses a panoramic feature splicing system with a virtual viewpoint and a method thereof. The panoramic feature splicing system comprises a panoramic camera and an image splicing module. The panoramic camera is provided with a depth sensor. The output end of the panoramic camera is connected with the image splicing module. The image splicing module comprises a panoramic camera inner-and-outer parameter calculating unit, a mapping matrix calculating unit, an image amplifying and fusing unit and an image hole filling unit. The method of the invention comprises the steps of S1, a camera inner-and-outer parameter calculating step, S2, a mapping matrix calculating step, S3, an image amplifying and fusing step and S4, an image hole filling step. The panoramic feature splicing system with the virtual viewpoint and the method thereof effectively process problems such as visual viewpoint position, image information, homography, image fusion, etc. in panoramic splicing. Furthermore high image quality after splicing is ensured.

Description

technical field [0001] The invention relates to a panoramic close-up mosaic system and a method thereof for a virtual viewpoint. Background technique [0002] As digital cameras become more common and more and more users are able to experience digital products, automatic image digital processing technology has also become an upsurge of technical researchers; two popular image processing methods, red-eye removal and color correction, have been included Commercial software package, used primarily by photographers; another common application is image stitching, which produces a high-quality panoramic image from multiple overlapping images of the same scene. [0003] From the invention of the computer to the present, after decades, information technology has grown from scratch, digital information has penetrated into various fields closely related to human life, and human beings are moving towards a highly informationized society; for a long time, people have been The pursuit o...

AI Technical Summary

Problems solved by technology

[0008] On the basis of the algorithm proposed by Mori, S.Zinger et al. further proposed an algorithm that filters the depth map, compares the changing areas before and after filtering to find the unreliable area, and maps it in reverse. The algorithm of Kwan-JungOh has improved PSNR and image contour edge, but both of these two schemes need to map the whole image multiple times, which is computationally intensive and time-consuming; Based on the background, use the background points around the hole to fill the hole

This method is limited to only filling small holes; P.Merkle et al. proposed an algorithm for image block processing, which has better PSNR performance, but the effect of the contour edge is not as good as the previous two algorithms; Ya-MeiFeng et al. People proposed an algorithm for asymmetric reference viewpoints, which can greatly reduce the amount of data and obtain high-quality images, but the reliability is not enough in complex occlusion environments; in the algorithms proposed by Karsten Müller and Aljoscha Smolic et al., The use of layered depth video (leveled depth video, LDV) can reduce the amount of transmitted data, and can effectively remove artifacts through edge extraction, but layered video requires preprocessing, which reduces real-time performance, and its edge detection method will part of the reliable area Divided into an unreliable area, adding additional overhead

Images