Image stitching method and device, electronic device, and storage medium

Abstract

The present disclosure provides image stitching method and device, electronic equipment and storage medium. The method comprises: acquiring original images and corresponding mask images of a same scene captured by a first camera and a second camera; determining an overlapping field of view region of the first camera and the second camera based on the original images and the mask images; acquiring block information of the overlapping field of view region on the first original image and the second original image; calculating a color difference compensation value between the first camera and the second camera based on the block information; performing global compensation correction on one of the first original image and the second original image based on the color difference compensation value to obtain a corrected image; and stitching the corrected image with the other uncorrected original image of the first original image and the second original image to obtain a target stitched image. The present disclosure adopts a global color difference compensation strategy, which can eliminate color difference and brightness difference of images captured by different cameras to generate a high-quality stitched image.

Claims

1. An image stitching method, characterized in that, include: Obtain the original images and their corresponding mask images captured by the first camera and the second camera in the same scene, wherein the original image captured by the first camera is the first original image, the original image captured by the second camera is the second original image, the mask image corresponding to the first original image is the first mask image, and the mask image corresponding to the second original image is the second mask image. Based on the first original image, the second original image, the first mask image, and the second mask image, the overlapping field of view of the first camera and the second camera is determined; Obtain the block information of the overlapping field of view region on the first original image and the second original image; Calculate the color difference compensation value between the first camera and the second camera based on the block information; Based on the color difference compensation value, a global compensation correction is performed on one of the first original image and the second original image to obtain the corrected image; as well as The corrected image is then stitched together with another uncorrected original image from the first original image and the second original image to obtain the target stitched image.

2. The method according to claim 1, characterized in that, Based on the first original image, the second original image, the first mask image, and the second mask image, determining the overlapping field of view region of the first camera and the second camera includes: The first original image and the second original image are respectively subjected to projection transformation to obtain the first original projected image and the second original projected image. The first mask image and the second mask image are respectively subjected to projection transformation to obtain the first mask projection image and the second mask projection image. Obtain the overlapping region between the first original projected image and the second original projected image; and On the first and second mask projection images, a mask region at the same position as the overlapping region is determined, and the mask region is used as the overlapping field of view region of the adjacent first and second cameras.

3. The method according to claim 1, characterized in that, Obtaining the block information of the overlapping field of view region in the first original image and the second original image includes: The overlapping field of view is regularized to obtain an equivalent regular graphic; The equivalent regular graphic is divided into grid blocks on the first original image, and pixel statistics of each block are obtained to obtain the first block information; and The equivalent regular graphic is divided into grid blocks on the second original image, and the pixel statistics of each block are obtained to obtain the second block information.

4. The method according to claim 3, characterized in that, Calculating the chromatic difference compensation value between the first camera and the second camera based on the block information includes: The first block information and the second block information are merged to construct a block information set; According to the preset classification rules, the block information set is divided into two different categories: a first category and a second category. Count the number of blocks in each category corresponding to the first camera and the second camera; Select the category that contains the maximum number of said blocks; In the selected category, calculate the pixel means of the blocks corresponding to the first camera and the second camera, denoted as meanP1 and meanP2 respectively; and Calculate the color difference compensation value between the first camera and the second camera based on meanP1 and meanP2.

5. The method according to claim 4, characterized in that, The category that selects the one containing the largest number of the blocks includes: Compare the number of blocks in the first category and the number of blocks in the second category, where the number of blocks in the first category is denoted as BM1 and the number of blocks in the second category is denoted as BM2; If BM1 > BM2, then take the first category as the selected category; If BM1 < BM2, then take the second category as the selected category; and If BM1 = BM2, then do nothing.

6. The method according to claim 5, characterized in that, Calculating the color difference compensation value between the first camera and the second camera based on meanP1 and meanP2 includes: Judge whether BM1 and BM2 satisfy BM1 > BM2 + T or BM1 < BM2 + T, where T is a preset threshold; If so, calculate the difference between meanP1 and meanP2, and take the difference as the color difference compensation value between the first camera and the second camera; and Otherwise, take the color difference compensation value used when stitching the previous frame image as the color difference compensation value between the first camera and the second camera.

7. The method according to claim 6, characterized in that, Performing global compensation correction on one of the first original image and the second original image based on the color difference compensation value to obtain the corrected image includes: Perform time-domain filtering on the color difference compensation value to obtain the filtered color difference compensation value; and Use the formula newI2 = I2 + newD12 to calculate the corrected image, where I2 represents one of the first original image and the second original image, newD12 represents the filtered color difference compensation value, and newI2 represents the corrected image.

8. The method according to claim 5, characterized in that, Calculating the color difference compensation value between the first camera and the second camera based on meanP1 and meanP2 includes: Judge whether BM1 and BM2 satisfy BM1 > BM2 + T or BM1 < BM2 + T, where T is a preset threshold; If so, calculate the ratio between meanP1 and meanP2, and take the ratio as the color difference compensation value between the first camera and the second camera; and Otherwise, take the color difference compensation value used when stitching the previous frame image as the color difference compensation value between the first camera and the second camera.

9. The method according to claim 8, characterized in that, Performing global compensation correction on one of the first original image and the second original image based on the color difference compensation value to obtain the corrected image includes: Perform time-domain filtering on the color difference compensation value to obtain the filtered color difference compensation value; and The corrected image is calculated using the formula newI2 = I2 × newD12, where I2 represents one of the first original image and the second original image, newD12 represents the filtered color difference compensation value, and newI2 represents the corrected image.

10. The method according to claim 9, characterized in that, The time-domain filtering of the color difference compensation value includes: applying an infinite impulse response digital filter to the color difference compensation value in the time domain, wherein the infinite impulse response digital filter is applied using the following formula: newD12=alpha*D12+beta*lastD12 Where alpha and beta represent preset filtering coefficients, D12 represents the color difference compensation value to be filtered, lastD12 represents the color difference compensation value used when stitching the previous frame image, the initial value of lastD12 is 0, and newD12 represents the filtered color difference compensation value.

11. The method according to claim 1, characterized in that, The corrected image is stitched together with another uncorrected original image from the first original image and the second original image to obtain the target stitched image, which includes: Reprojection transformations are performed on the corrected image and another uncorrected original image from the first and second original images, respectively, to obtain a first re-mapped image and a second re-mapped image; and The first and second remapping images are fused to obtain the target stitched image.

12. The method according to claim 1, characterized in that, Also includes: Before acquiring the original images and their corresponding mask images captured by the first and second cameras in the same scene, the first and second cameras are calibrated offline.

13. An image stitching device, characterized in that, include: The memory is configured to store information associated with the image; as well as At least one processor is electrically coupled to the memory and is configured to: Obtain the original images and their corresponding mask images captured by the first camera and the second camera in the same scene, wherein the original image captured by the first camera is the first original image, the original image captured by the second camera is the second original image, the mask image corresponding to the first original image is the first mask image, and the mask image corresponding to the second original image is the second mask image. Based on the first original image, the second original image, the first mask image, and the second mask image, the overlapping field of view of the first camera and the second camera is determined; Obtain the block information of the overlapping field of view region on the first original image and the second original image; Calculate the color difference compensation value between the first camera and the second camera based on the block information; Based on the color difference compensation value, a global compensation correction is performed on one of the first original image and the second original image to obtain the corrected image; as well as The corrected image is then stitched together with another uncorrected original image from the first original image and the second original image to obtain the target stitched image.

14. An electronic device, characterized in that, include: The memory is configured to store executable programs; as well as A processor is configured to execute the program causing the electronic device to perform the image stitching method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the image stitching method according to any one of claims 1 to 12.

Images