An improved image stitching method
By combining field-of-view correction, feature point registration, and brightness adjustment with the shortest path algorithm and Laplacian fusion, the problems of computational time and artifacts in high-resolution image stitching are solved, achieving efficient and seamless image stitching.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI FULLHAN MICROELECTRONICS
- Filing Date
- 2023-11-23
- Publication Date
- 2026-06-23
AI Technical Summary
Existing image stitching techniques are computationally time-consuming in high-resolution image processing, making real-time computation difficult, and are prone to artifacts during image fusion.
After field-of-view correction and feature point registration, grayscale conversion and brightness adjustment are performed. A feature similarity map is calculated, and the shortest path algorithm is used to determine the optimal seam line on the downsampled image. Finally, the image is fused using the Laplacian fusion algorithm.
It reduces computation time, improves image consistency, removes artifacts in image fusion, and achieves efficient image stitching.
Smart Images
Figure CN117522762B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and specifically relates to an improved image stitching method. Background Technology
[0002] Image stitching is a technique that combines two or more images with partially overlapping areas into a single image with a larger field of view. Because it contains more information, it is widely used in surveillance and vehicle recording.
[0003] Image stitching mainly involves three steps: image preprocessing, image registration, and image fusion. Among these, image fusion is a crucial step in image stitching technology. Image fusion methods include alpha fusion, guided filtering-based fusion, Laplacian pyramid-based fusion, wavelet transform-based fusion, and so on. Regardless of the method, determining the fusion boundary is a critical step. Even after preprocessing and registration, images are rarely perfectly identical in overlapping areas, and conventional methods can lead to artifacts in the merged image. To improve the quality of the stitched image, methods for calculating the optimal seam line have been extensively studied. The goal is to find a continuous top-to-bottom line that minimizes the content differences between the two images on either side of this line. Using this line as the boundary, the merged image can achieve maximum smoothness and seamlessness in the overlapping areas.
[0004] Common methods for calculating the optimal stitching line include graph cut, shortest path, and dynamic programming. These methods all transform image pixels into vertices of a graph and artificially define the relationships between pixels as edges, thus converting the problem of calculating the optimal stitching line into a graph theory problem. However, as image resolution gradually increases, the computation time of these methods also increases, making real-time computation particularly difficult in video processing. Summary of the Invention
[0005] To address the aforementioned problems in the existing technology, this invention provides an improved image stitching method, implemented through the following steps:
[0006] S1: Obtain a first stitched image and a second stitched image; perform field-of-view correction on the first stitched image and the second stitched image to obtain a first calibration image and a second calibration image; perform feature point registration on the first calibration image and the second calibration image to obtain the overlapping area of the images;
[0007] S2: Perform grayscale conversion on the overlapping areas of the images to obtain a first image grayscale histogram and a second image grayscale histogram. Adjust the brightness of the first calibration image and the second calibration image according to the first image grayscale histogram and the second image grayscale histogram to obtain a first equalized image and a second equalized image. Calculate the average pixel difference between the first equalized image and the second equalized image to obtain a feature similarity map.
[0008] S3: Perform a downsampling operation on the feature similarity map to obtain a downsampled feature map; perform data processing on the feature similarity map and the downsampled feature map to obtain feature similarity map data and downsampled feature map data; and use the shortest path algorithm to calculate the shortest path on the downsampled feature map data.
[0009] S4: Perform an upsampling operation on the vertices in the shortest path to obtain new graph data;
[0010] S5: Repeat steps S3 and S4 to obtain the shortest path of the new image data, obtain the optimal stitching line based on the shortest path, and use the Laplacian fusion algorithm to fuse the first stitched image and the second stitched image based on the optimal stitching line to obtain the stitched result image.
[0011] Specifically, the brightness adjustment method in S2 is as follows:
[0012] The second brightness-replaced image is obtained by performing pixel brightness replacement on the second stitched image based on the gray-level mapping of the first image gray-level histogram and the second image gray-level histogram;
[0013] Calculate the ratio Y of the brightness change between the second brightness replacement image and the second calibration image. new / Y2, based on the ratio, the second calibration image is subjected to luminance gain conversion to obtain a second luminance converted image, and the conversion formula is:
[0014]
[0015] Among them, R new G new B new R2, G2, and B2 are the RGB channel values of the second calibration image after brightness gain conversion, and R2, G2, and B2 are the RGB channel values of the second calibration image before brightness gain conversion.
[0016] The correction parameters are calculated for the second brightness conversion image and the first calibration image of the overlapping region, using the following formula:
[0017]
[0018] Wherein, R1, G1, and B1 are the RGB channel values of the first calibration image. The second brightness conversion image and the first calibration image are corrected according to the correction parameters to obtain the first equalization image and the second equalization image.
[0019] Specifically, the method for calculating the average pixel difference in S2 is as follows:
[0020] The overlapping region is expanded, and the expanded region is converted into HSV format to obtain a first HSV image and a second HSV image. The first HSV image and the second HSV image are then split according to the H, S, and V channels to obtain grayscale image groups.
[0021] A Laplacian pyramid is constructed for the grayscale image group, and the absolute values of the differences between the H, S, and V components of the first HSV image and the second HSV image in the grayscale image group are calculated based on the number of layers of the Laplacian pyramid.
[0022] The absolute values of the differences are magnified to obtain full-scale difference maps of H, S, and V. The full-scale difference maps are then weighted and averaged to obtain the feature similarity map.
[0023] Specifically, the data processing in S3 treats pixels as vertices of graph data and the sum of a pixel and the pixels in the two adjacent rows as edges of the graph data.
[0024] Specifically, the shortest path algorithm in S3 is implemented as follows:
[0025] The pixels in the middle row of the downsampled feature map data are taken as the starting point of the shortest path, and the distance from the starting point of the shortest path to the pixels in the left, middle and right positions of the adjacent upper and lower rows is calculated.
[0026] Traverse the pixels in the middle row, and take the pixel with the minimum distance as the shortest path point. Obtain the shortest path point set based on the minimum distance from the shortest path point to the left, middle and right pixels in the adjacent upper and lower rows. Connect the starting point of the shortest path with the shortest path points in the shortest path point set to obtain the shortest path.
[0027] The beneficial effects of this invention are as follows:
[0028] (1) Before image stitching, perform brightness and color preprocessing on the images to reduce image differences and improve the consistency of the stitched images;
[0029] (2) First, the optimal seam line is calculated on the downsampled image, and then the complex calculation that was originally on the high-resolution image is simplified to the low-resolution image by upsampling, which greatly reduces the calculation time.
[0030] (3) The Laplacian pyramid is used to calculate the feature differences of the image, and a weighted summation method is used to increase the weight of high-frequency information, so as to remove artifacts that may be generated by image fusion to the greatest extent. Attached Figure Description
[0031] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.
[0032] Figure 1 This is an improved image stitching method in the present invention. Detailed Implementation
[0033] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.
[0034] Please see Figure 1 An improved image stitching method includes the following steps:
[0035] S1: Obtain a first stitched image and a second stitched image; perform field-of-view correction on the first stitched image and the second stitched image to obtain a first calibration image and a second calibration image; perform feature point registration on the first calibration image and the second calibration image to obtain the overlapping area of the images;
[0036] S2: Perform grayscale conversion on the overlapping areas of the images to obtain a first image grayscale histogram and a second image grayscale histogram. Adjust the brightness of the first calibration image and the second calibration image according to the first image grayscale histogram and the second image grayscale histogram to obtain a first equalized image and a second equalized image. Calculate the average pixel difference between the first equalized image and the second equalized image to obtain a feature similarity map.
[0037] S3: Perform a downsampling operation on the feature similarity map to obtain a downsampled feature map; perform data processing on the feature similarity map and the downsampled feature map to obtain feature similarity map data and downsampled feature map data; and use the shortest path algorithm to calculate the shortest path on the downsampled feature map data.
[0038] S4: Perform an upsampling operation on the vertices in the shortest path to obtain new graph data;
[0039] S5: Repeat steps S3 and S4 to obtain the shortest path of the new image data, obtain the optimal stitching line based on the shortest path, and use the Laplacian fusion algorithm to fuse the first stitched image and the second stitched image based on the optimal stitching line to obtain the stitched result image.
[0040] In this embodiment, the two input images are preprocessed, and their fields of view are corrected to the same plane through intrinsic and extrinsic parameter calibration and cylindrical projection. Image registration is performed using feature point matching to obtain the specific range of the overlapping area between the two images, and the two images are corrected vertically to achieve row alignment and cropping out excess areas.
[0041] The second brightness-replaced image is obtained by performing pixel brightness replacement on the second stitched image based on the gray-level mapping of the first image gray-level histogram and the second image gray-level histogram;
[0042] Calculate the ratio Y of the brightness change between the second brightness replacement image and the second calibration image. new / Y2, based on the ratio, the second calibration image is subjected to luminance gain conversion to obtain a second luminance converted image, and the conversion formula is:
[0043]
[0044] Among them, R new G new B new R2, G2, and B2 are the RGB channel values of the second calibration image after brightness gain conversion, and R2, G2, and B2 are the RGB channel values of the second calibration image before brightness gain conversion.
[0045] The correction parameters are calculated for the second brightness conversion image and the first calibration image of the overlapping region, using the following formula:
[0046]
[0047] Wherein, R1, G1, and B1 are the RGB channel values of the first calibration image. The second brightness conversion image and the first calibration image are corrected according to the correction parameters to obtain the first equalization image and the second equalization image.
[0048] Specifically, the method for calculating the average pixel difference in S2 is as follows:
[0049] The overlapping region is expanded, and the expanded region is converted into HSV format to obtain a first HSV image and a second HSV image. The first HSV image and the second HSV image are then split according to the H, S, and V channels to obtain grayscale image groups.
[0050] A Laplacian pyramid is constructed for the grayscale image group, and the absolute values of the differences between the H, S, and V components of the first HSV image and the second HSV image in the grayscale image group are calculated based on the number of layers of the Laplacian pyramid.
[0051] The absolute values of the differences are magnified to obtain full-scale difference maps of H, S, and V. The full-scale difference maps are then weighted and averaged to obtain the feature similarity map.
[0052] Specifically, the data processing in S3 treats pixels as vertices of graph data and the sum of a pixel and the pixels in the two adjacent rows as edges of the graph data.
[0053] Specifically, the shortest path algorithm in S3 is implemented as follows:
[0054] The pixels in the middle row of the downsampled feature map data are taken as the starting point of the shortest path, and the distance from the starting point of the shortest path to the pixels in the left, middle and right positions of the adjacent upper and lower rows is calculated.
[0055] Traverse the pixels in the middle row, and take the pixel with the minimum distance as the shortest path point. Obtain the shortest path point set based on the minimum distance from the shortest path point to the left, middle and right pixels in the adjacent upper and lower rows. Connect the starting point of the shortest path with the shortest path points in the shortest path point set to obtain the shortest path.
[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. An improved image stitching method, characterized by, Includes the following steps: S1: Obtain a first stitched image and a second stitched image; perform field-of-view correction on the first stitched image and the second stitched image to obtain a first calibration image and a second calibration image; perform feature point registration on the first calibration image and the second calibration image to obtain the overlapping area of the images; S2: Perform grayscale conversion on the overlapping areas of the images to obtain a first image grayscale histogram and a second image grayscale histogram. Adjust the brightness of the first calibration image and the second calibration image according to the first image grayscale histogram and the second image grayscale histogram to obtain a first equalized image and a second equalized image. Calculate the average pixel difference between the first equalized image and the second equalized image to obtain a feature similarity map. S3: Perform a downsampling operation on the feature similarity map to obtain a downsampled feature map; perform data processing on the feature similarity map and the downsampled feature map to obtain feature similarity map data and downsampled feature map data; and use the shortest path algorithm to calculate the shortest path on the downsampled feature map data. S4: Perform an upsampling operation on the vertices in the shortest path to obtain new graph data; S5: Repeat steps S3 and S4 to obtain the shortest path of the new image data, obtain the optimal stitching line based on the shortest path, and use the Laplacian fusion algorithm to fuse the first stitched image and the second stitched image based on the optimal stitching line to obtain the stitched result image.
2. The method of claim 1, wherein, The specific method for brightness adjustment in S2 is as follows: The second brightness-replaced image is obtained by performing pixel brightness replacement on the second stitched image based on the gray-level mapping of the first image gray-level histogram and the second image gray-level histogram; calculating a ratio Y of the luminance variation of the second luminance replacement image and the second calibration image new Y2, the second calibration image is converted to a second luminance conversion image according to the ratio, and the conversion formula is: wherein R new , G new , B new are the RGB channel values of the second calibration image after luminance gain conversion, and R2, G2, B2 are the RGB channel values of the second calibration image before luminance gain conversion. The correction parameters are calculated for the second brightness conversion image and the first calibration image of the overlapping region, using the following formula: Wherein, R1, G1, and B1 are the RGB channel values of the first calibration image. The second brightness conversion image and the first calibration image are corrected according to the correction parameters to obtain the first equalization image and the second equalization image.
3. The method of claim 1, wherein, The specific method for calculating the average pixel difference in S2 is as follows: The overlapping region is expanded, and the expanded region is converted into HSV format to obtain a first HSV image and a second HSV image. The first HSV image and the second HSV image are then split according to the H, S, and V channels to obtain grayscale image groups. A Laplacian pyramid is constructed for the grayscale image group, and the absolute values of the differences between the H, S, and V components of the first HSV image and the second HSV image in the grayscale image group are calculated based on the number of layers of the Laplacian pyramid. The absolute values of the differences are magnified to obtain full-scale difference maps of H, S, and V. The full-scale difference maps are then weighted and averaged to obtain the feature similarity map.
4. The method of claim 1, wherein, The data processing in S3 uses pixels as vertices of graph data and the sum of pixels in the two adjacent rows as edges of the graph data.
5. The method according to claim 1, characterized in that, The specific implementation of the shortest path algorithm in S3 is as follows: The pixels in the middle row of the downsampled feature map data are taken as the starting point of the shortest path, and the distance from the starting point of the shortest path to the pixels in the left, middle and right positions of the adjacent upper and lower rows is calculated. Traverse the pixels in the middle row, and take the pixel with the minimum distance as the shortest path point. Obtain the shortest path point set based on the minimum distance from the shortest path point to the left, middle and right pixels in the adjacent upper and lower rows. Connect the starting point of the shortest path with the shortest path points in the shortest path point set to obtain the shortest path.