4K real-time video panorama stitching method based on GPU acceleration

Abstract

The 4K real-time video panorama stitching method based on GPU acceleration includes the following steps: the hardware device takes the stream directly to the GPU memory; calculates the camera parameters and saves the parameters; calculates the image brightness and chromaticity correction parameters; calculates the mask map, and uses it for subsequent frames. Image fusion; use the calculated camera parameters to map the image to the projection plane, and overlap the overlapping areas of the two images together; use the brightness and chromaticity correction parameters calculated in step 3 to compensate the image; use image pyramid and best stitching The stitching technology is used to fuse the images to obtain a panoramic mosaic image for display and output preservation. The invention adopts the Gaussian multi-layer pyramid method to extract high-frequency detail information of images for fusion, and all operations are accelerated by GPU, so as to solve the problem of ghost and double image in the splicing process and meet the real-time requirement.

Description

technical field [0001] The invention belongs to the technical field of image splicing, in particular to a 4K real-time video panorama splicing method based on GPU acceleration. Background technique [0002] Panoramic stitching technology is the technology of stitching several overlapping images (which may be obtained at different times, different angles of view or different sensors) into a large seamless high-resolution image. Image stitching goes through image collection and image processing. , image feature processing, image matching, model building, image fusion and other steps, and finally complete the stitching of panoramic images. [0003] At present, there are many researches on video stitching technology at home and abroad, and the research focuses on the two parts of single image registration and image fusion. There are few researches on the complex problems encountered in the process of continuous video stream stitching. The mainstream method of image registration...

AI Technical Summary

Problems solved by technology

This method extracts less image detail information. When the brightness and chromaticity of the mosaic image are slightly different, the transition zone of the mosaic panorama is obvious and the visual effect is poor.

At the same time, the key to video panorama stitching technology is real-time performance. At present, most practitioners use CPU+GPU architecture to realize it. According to the performance of different graphics cards, they can only splice 2 to 4 channels of 2K video at most. , and there is no real-time compensation, resulting in uneven brightness and darkness of the mosaic image, and large color difference

In addition, considering that the image captured by the hardware image acquisition device may have vignetting (bright in the middle of the image and dark around it), there is no processing, which leads to the same problem in stitching panoramas

Images