Panoramic video mosaicing system and method based on camera automatic calibration and color correction

Abstract

The invention discloses a panoramic video mosaicing system and method based on camera automatic calibration and color correction. The system comprises a camera automatic calibration module and a color on-line correction module. The camera automatic calibration module comprises an initial homography matrix calculation submodule, an overlapping region calculation submodule, an overlapping region regulation submodule, a feature matching submodule and a homography matrix correction submodule. The color on-line correction module comprises an overlapping region color correction submodule and an integral image color correction submodule. The method comprises a camera automatic calibration step S1, a color on-line correction step S2 and a panoramic image mosaicing step S3. A homography matrix between images to be mosaicked is corrected through a scaling matrix and a skewing matrix during automatic calibration to make the homography matrix more accurate; and when overlapping region color correction is carried out, a conversion relationship matrix is corrected by utilizing continuous three-frame image information to enable color transformation between frames in a panoramic image of a video to be more smooth and fluent.

Description

technical field [0001] The present invention relates to a panoramic video stitching system and method based on camera automatic calibration and color correction, Background technique [0002] At present, a large number of cameras are installed in government buildings, military bases, banks, schools, shopping malls, and stadiums. With the rise of video conferencing, distance education, robot navigation and other fields, a single camera cannot record all objects in a large scene due to the limitation of viewing angle. , the emergence of panoramic stitching technology meets the needs of large fields of view in various fields; today's society is fiercely competitive, and people are usually busy with work, so most of the choice of travel destinations during holidays depends on the promotion of tourist attractions, but traditional pictures cannot be presented to the public. The all-round display of scenic spots by tourists can no longer meet the needs of tourists, so it cannot att...

AI Technical Summary

Problems solved by technology

[0004] However, the traditional panoramic video stitching technology usually has the unavoidable problem of color difference in the same frame and between frames caused by graphic distortion and lighting; therefore, we need to perform geometric correction and color correction on the images to be stitched during the stitching process; Correction aims at overlapping areas between different projection images, and geometric alignment of the overlapping areas is required. There are a series of key issues such as "screen segmentation", "overlapping area extraction", "global geometric alignment" and "multi-image edge fusion". Greater engineering difficulty; and better geometric correction results depend on the accuracy of identifying overlapping regions, which in turn depends on the reliability of camera calibration and the accuracy of camera parameters. Traditional camera calibration relies on accurate camera internal parameters and checkerboard a series of tedious processes

In the case of large errors in camera parameters, traditional camera calibration cannot obtain better geometric correction results; the automatic scene balance algorithm used in traditional color correction tends to adjust the exposure so that the average color of the image is neutral gray, This is acceptable to consumers for many security purposes, but unacceptable for users with specific scenes or high color fidelity requirements such as live football games, house inspections and tourism; for example, simple balanced photography ID images Making the background white often distorts the subject's face, hair, and eye shading, brightens the face, and washes out features; alternatively, balancing toward a neutral gray in such a scene may be affected by the subject's clothing. The effect of shading, and produces a colored background; similarly, removing the blue tint from a sky, snow, or ice scene results in color changes beyond acceptable levels, also for images with dark shadows; lightens shadows or changes shadows A color may affect the entire image by making shadows too bright or by unacceptably shifting other area colors of the image; similar unacceptable results can occur when color balancing is performed to correct for skewing of other colors; and, For video, the traditional color correction is only to correct the color of the image at the same moment, but there will be a large color difference between the images due to the angle change and uneven lighting when shooting

Images