A multi-band fusion processing method, system, computer readable storage medium and computer program product for seamless stitching of overlapping images

Abstract

The application relates to the technical field of computer vision and digital image processing, and discloses a multi-band fusion processing method and system for seamless splicing of overlapping images, a computer readable storage medium and a computer program product, the method comprising the following steps: performing multi-band decomposition on overlapping area images to obtain sampling images of K different frequency bands; based on a set fusion weight, performing weighted fusion on the sampling images of two overlapping area images, and reconstructing the fusion result to generate a fusion image of the two overlapping area images; and using the fusion image to replace the overlapping area images to splice two original images to be spliced, thereby obtaining a spliced image. The overlapping area images are subjected to multi-scale decomposition, and a differentiated fusion strategy is adopted in different frequency bands, so that low-frequency brightness smooth transition and high-frequency detail retention are realized, thereby obtaining a spliced image which has no obvious joint, consistent brightness and clear details.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to a multi-band fusion processing method, system, computer-readable storage medium, and computer program product for seamless stitching of overlapping images. Background Technology

[0002] In applications such as panoramic imaging, machine vision inspection, industrial vision stitching, and large field-of-view imaging, it is often necessary to stitch together multiple images with overlapping areas to form a composite image with a larger field of view and more complete information.

[0003] Existing image stitching technologies generally include steps such as image registration, overlapping area determination, seam finding, and pixel fusion. Among these, how to smoothly fuse the overlapping areas after geometric registration is a key factor affecting the final stitching quality.

[0004] In existing technologies, common splicing and blending methods include direct linear weighted blending, fixed weight replacement, and simple seam trimming. However, these methods have the following shortcomings in practical applications: (1) Significant differences in brightness: Due to different shooting conditions, exposure parameters or changes in lighting, the images in the overlapping areas often have inconsistent brightness. Simple linear fusion is prone to produce obvious brightness jumps at the seams. (2) High-frequency detail blurring: When there are slight registration errors between images or there are moving objects in the scene, direct fusion can easily lead to high-frequency detail blurring or ghosting; (3) Obvious seam artifacts: Seam treatment with fixed width or fixed position is difficult to adapt to complex scenes and is prone to producing visible splicing marks at the edges or texture areas.

[0005] Therefore, there is an urgent need in this field for an image stitching and fusion method that can effectively suppress brightness differences in the seam area and reduce stitching artifacts while preserving image details. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-band fusion processing method, system, computer-readable storage medium, and computer program product for seamless stitching of overlapping images, so as to solve or at least partially solve the technical problems mentioned in the background art.

[0007] To achieve this objective, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-band fusion processing method for seamless stitching of overlapping images, comprising: Find the optimal seam path on the overlapping area image of the original images to be stitched, and obtain the overlapping area seam image; Multi-band decomposition is performed on the overlapping region image to obtain sampled images of K different frequency bands; Based on the differences in the seam images of the overlapping region, the fusion weights of the sampled images are set; Based on the set fusion weights, the sampled images of the two overlapping regions are weighted and fused, and the fusion result is reconstructed to generate a fused image of the two overlapping regions. Replace the overlapping area image with the fused image, and then stitch the two original images to be stitched together to obtain the stitched image.

[0008] Optionally, finding the optimal seam path on the overlapping region image of the original images to be stitched, and obtaining the overlapping region seam image, includes: The overlapping regions of the two original images to be stitched are segmented to obtain two overlapping region images; let the two original images be the first original image and the second original image, and let the overlapping region image of the first original image be the first overlapping region image, and the overlapping region image of the second original image be the second overlapping region image. Calculate the overlap region error between two overlapping region images to obtain an overlapping region difference map; the formula for calculating the overlap region error is: ; In the formula, e 直 The overlapping region error is represented by p and q, which represent two horizontally adjacent pixels in the overlapping region image, respectively. I1 represents the gray value of the first overlapping region image, I2 represents the gray value of the second overlapping region image, and ||| represents the L2 norm calculation. The minimum cumulative error is calculated for all paths on the difference map of the overlapping region using the minimum error objective function, resulting in the minimum cumulative error map; the minimum error objective function is as follows: ; In the formula, (i,j) represents the pixel coordinates on the difference map of the overlapping region, and e i,j E represents the error of the current pixel. i-1,j-1 E i-1,j and E i-1,j+1 E represents the minimum cumulative error value on the top left, top, and top right sides of the current pixel, respectively. i,j This represents the minimum cumulative error across all paths of the current pixel. Finally, the optimal splicing seam path is obtained by backtracking from the minimum cumulative error map, and the seam image of the overlapping area is obtained.

[0009] Optionally, the method for obtaining the optimal splicing seam path is as follows: In the last row of the overlapping region image, y=H, the pixel with the smallest error is taken as the seam endpoint; the seam endpoint (y endx end This can be represented as: , This indicates the position where the function value is minimized; Starting from the end of the seam, backtrack upwards, subtracting 1 from the row coordinate and taking the column coordinate of the pixel with the smallest error in the previous row, until backtracking to the first row, to obtain the optimal seam path: The coordinates of the pixels on the optimal seam path can be expressed as: .

[0010] Optionally, the coordinates of the pixels on the optimal seam path are specifically: .

[0011] Optionally, the step of performing multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands specifically involves constructing a Laplacian pyramid structure on the overlapping region image to obtain K Laplacian pyramid components.

[0012] Optionally, constructing a Laplacian pyramid structure on the overlapping region image to obtain K Laplacian pyramid components specifically includes: Gaussian filtering and downsampling are performed layer by layer on the overlapping region image to construct a K-layer Gaussian pyramid; The Laplacian pyramid components of the corresponding level are obtained by performing a difference operation on adjacent Gaussian pyramids. The method is as follows: Let G be the k-th level of the Gaussian pyramid. k , for G k+1 Perform upsampling to make its size consistent with G. k Consistent, resulting in Expand(G) k+1 ), G k With Expand(G) k+1 Subtracting pixel by pixel, we obtain the k-th layer Laplacian pyramid component L. k ,Right now: .

[0013] Optionally, the method for setting the fusion weight of the sampled images based on the differences in the seam images of the overlapping regions is as follows: Let the overlapping region seam image obtained based on the first overlapping region image be the first overlapping region seam image, and the overlapping region seam image obtained based on the second overlapping region image be the second overlapping region seam image; Differential Gaussian smoothing is applied to the seam images of the first and second overlapping regions according to the pyramid level to obtain the fusion weight w of the k-th Laplacian pyramid component. 1k and w2k Among them, w 1k +w 2k =1; The method for weighted fusion of sampled images from two overlapping regions based on a set fusion weight is as follows: We perform a weighted fusion of the k-th level Laplace pyramid components to obtain the k-th level merged pyramid component R. k The calculation method is as follows: R k =L 1k* w 1k +L 2k* w 2k L 1k and L 2k These represent the Laplacian components of the first and second overlapping region images at the k-th layer, respectively. The method for reconstructing the fusion result to generate a fused image is as follows: The merged pyramid components are upsampled and stacked layer by layer from top to bottom to complete the reconstruction of the fused image; the reconstruction relationship can be expressed as: S k =R k +expand(S k+1 In the formula, S k This represents the fusion of pyramids.

[0014] Optionally, after obtaining the stitched image, the process also includes: Local brightness homogenization is performed on the stitching boundaries of the stitched images.

[0015] Optionally, the method for performing local brightness homogenization processing on the splicing boundaries of the spliced ​​images is as follows: Using the stitching boundary of the stitched image as the center reference, expand the area of ​​interest (ROI) to both sides by a preset pixel width to obtain the ROI region: Extract a first ROI image corresponding to the ROI region from the fused image, and extract a second ROI image corresponding to the ROI region from the original image; Perform secondary multi-band fusion processing on the first ROI image and the second ROI image to generate a smoothed fused ROI image; The image of the fused ROI is used to replace the corresponding region in the stitched image to obtain the final stitched image.

[0016] Optionally, before finding the optimal seam path on the overlapping area image of the original images to be stitched and obtaining the overlapping area seam image, the method further includes: Acquire at least two original images that have overlapping regions; The method for segmenting the overlapping regions of the two original images to be stitched together to obtain two overlapping region images is as follows: Using the geometric registration parameters between the two original images to be stitched, the overlapping regions in the two original images are segmented to obtain the first overlapping region image and the second overlapping region image.

[0017] Secondly, the present invention provides a multi-band fusion processing system for seamless stitching of overlapping images, comprising: The overlapping region analysis module is used to find the optimal seam path on the overlapping region image of the original images to be stitched, and to obtain the overlapping region seam image. The multi-band decomposition module is electrically connected to the overlapping region analysis module and is used to perform multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands. The fusion module is electrically connected to the overlapping region analysis module and is used to set the fusion weight of the sampled images based on the differences in the seam images of the overlapping regions; and is also electrically connected to the multi-band decomposition module, which is used by the probe to perform weighted fusion of the sampled images of the two overlapping regions based on the set fusion weight, and to reconstruct the fusion result to generate a fused image of the two overlapping regions. The stitching module, electrically connected to the fusion module, is used to replace the overlapping area image with the fused image, and to stitch the two original images to be stitched together to obtain the stitched image.

[0018] Optionally, the overlapping region analysis module is specifically used to: segment out the overlapping regions of the two original images to be stitched together, and obtain two overlapping region images; Let the two original images be the first original image and the second original image, and let the overlapping region of the first original image be the first overlapping region image and the overlapping region of the second original image be the second overlapping region image. The overlapping region analysis module is also used to calculate the overlapping region error between two overlapping region images, obtaining an overlapping region difference map; wherein, the formula for calculating the overlapping region error is: ; In the formula, e 直 The overlapping region error is represented by p and q, which represent two horizontally adjacent pixels in the overlapping region image, respectively. I1 represents the gray value of the first overlapping region image, I2 represents the gray value of the second overlapping region image, and ||| represents the L2 norm calculation. The overlapping region analysis module is also used to calculate the minimum cumulative error for all paths on the overlapping region difference map using a minimum error objective function, thereby obtaining a minimum cumulative error map; wherein, the minimum error objective function is: ; (i,j) represents the pixel coordinates on the difference map of the overlapping region, e i,j E represents the error of the current pixel. i-1,j-1 E i-1,j and E i-1,j+1 E represents the minimum cumulative error value on the top left, top, and top right sides of the current pixel, respectively. i,j This represents the minimum cumulative error across all paths of the current pixel. The overlapping region analysis module is also used to backtrack from the minimum cumulative error map to obtain the optimal splicing seam path and obtain the overlapping region seam image.

[0019] Optionally, the method for obtaining the optimal splicing seam path is as follows: In the last row of the overlapping region image, y=H, the pixel with the smallest error is taken as the seam endpoint; the seam endpoint (y end x end This can be represented as: , This indicates the position where the function value is minimized; Starting from the end of the seam, backtrack upwards, subtracting 1 from the row coordinate and taking the column coordinate of the pixel with the smallest error in the previous row, until backtracking to the first row, to obtain the optimal seam path: The coordinates of the pixels on the optimal seam path can be expressed as: .

[0020] Optionally, the coordinates of the pixels on the optimal seam path are specifically: .

[0021] Optionally, the multi-band decomposition module is specifically used to: construct a Laplacian pyramid structure for the overlapping region image to obtain K Laplacian pyramid components.

[0022] Optionally, constructing a Laplacian pyramid structure on the overlapping region image to obtain K Laplacian pyramid components specifically includes: Gaussian filtering and downsampling are performed layer by layer on the overlapping region image to construct a K-layer Gaussian pyramid; The Laplacian pyramid components of the corresponding level are obtained by performing a difference operation on adjacent Gaussian pyramids. The method is as follows: Let G be the k-th level of the Gaussian pyramid. k , for G k+1 Perform upsampling to make its size consistent with G. k Consistent, resulting in Expand(G) k+1 ), G k With Expand(G)k+1 Subtracting pixel by pixel, we obtain the k-th layer Laplacian pyramid component L. k ,Right now: .

[0023] Optionally, the method for setting the fusion weight of the sampled images based on the differences in the seam images of the overlapping regions is as follows: Let the overlapping region seam image obtained based on the first overlapping region image be the first overlapping region seam image, and the overlapping region seam image obtained based on the second overlapping region image be the second overlapping region seam image; Differential Gaussian smoothing is applied to the seam images of the first and second overlapping regions according to the pyramid level to obtain the fusion weight w of the k-th Laplacian pyramid component. 1k and w 2k Among them, w 1k +w 2k =1; The method for weighted fusion of sampled images from two overlapping regions based on a set fusion weight is as follows: We perform a weighted fusion of the k-th level Laplace pyramid components to obtain the k-th level merged pyramid component R. k The calculation method is as follows: R k =L 1k* w 1k +L 2k* w 2k L 1k and L 2k These represent the Laplacian components of the first and second overlapping region images at the k-th layer, respectively. The method for reconstructing the fusion result to generate a fused image is as follows: The merged pyramid components are upsampled and stacked layer by layer from top to bottom to complete the reconstruction of the fused image; the reconstruction relationship can be expressed as: S k =R k +expand(S k+1 In the formula, S k This represents the fusion of pyramids.

[0024] Optionally, the multi-band fusion processing system further includes a brightness uniformization processing module, which is electrically connected to the stitching module and is used to perform local brightness uniformization processing on the stitching boundary of the stitched image.

[0025] Optionally, the method for performing local brightness homogenization processing on the splicing boundaries of the spliced ​​images is as follows: Using the stitching boundary of the stitched image as the center reference, expand the area of ​​interest (ROI) to both sides by a preset pixel width to obtain the ROI region: Extract a first ROI image corresponding to the ROI region from the fused image, and extract a second ROI image corresponding to the ROI region from the original image; Perform secondary multi-band fusion processing on the first ROI image and the second ROI image to generate a smoothed fused ROI image; The image of the fused ROI is used to replace the corresponding region in the stitched image to obtain the final stitched image.

[0026] Optionally, the multi-band fusion processing system further includes an image input module, which is electrically connected to the overlapping region analysis module and is used to acquire at least two original images with overlapping regions. The method for segmenting the overlapping regions of the two original images to be stitched together to obtain two overlapping region images is as follows: Using the geometric registration parameters between the two original images to be stitched, the overlapping regions in the two original images are segmented to obtain the first overlapping region image and the second overlapping region image.

[0027] Thirdly, the present invention provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement a multi-band fusion processing method for seamless stitching of overlapping images as described above.

[0028] Fourthly, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements a multi-band fusion processing method for seamless stitching of overlapping images as described above.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows: The multi-band fusion processing method for seamless stitching of overlapping images proposed in this application decomposes the overlapping area image at multiple scales and adopts differentiated fusion strategies in different frequency bands to achieve smooth transition of low-frequency brightness and preservation of high-frequency details, thereby obtaining stitched images with no obvious seams, consistent brightness and clear details. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of a multi-band fusion processing method for seamless stitching of overlapping images, provided in an embodiment of the present invention.

[0032] Figure 2a The first original image to be stitched is provided in an embodiment of the present invention.

[0033] Figure 2b The second original image to be stitched is provided in an embodiment of the present invention.

[0034] Figure 3a The first overlapping region image to be stitched is provided in an embodiment of the present invention.

[0035] Figure 3b The image shows the second overlapping region to be stitched, as provided in an embodiment of the present invention.

[0036] Figure 4 The overlapping region difference diagram provided for the embodiments of the present invention.

[0037] Figure 5 The minimum cumulative error diagram provided for embodiments of the present invention.

[0038] Figure 6a The first overlapping area seam image provided for an embodiment of the present invention.

[0039] Figure 6a The second overlapping area seam image provided in the embodiment of the present invention.

[0040] Figure 7 This is a stitched image of the first original image and the second original image.

[0041] Figure 8 This is the first ROI image extracted from the stitched image.

[0042] Figure 9 This is the second ROI image extracted from the first original image.

[0043] Figure 10 This is a fused ROI image generated after fusing the first ROI image and the second ROI image.

[0044] Figure 11 This is the final stitched image of the first original image and the second original image provided in the embodiments of the present invention.

[0045] Figure 12 This is a system architecture diagram of a multi-band fusion processing system for seamless stitching of overlapping images, provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0047] Example 1: Please refer to Figure 1 , Figure 1 A flowchart of a multi-band fusion processing method for seamless stitching of overlapping images is provided in an embodiment of the present invention. The method includes: Step 110: Obtain at least two original images with overlapping regions.

[0048] This embodiment uses the two original images shown in Figure 2 as an example, wherein, Figure 2a The first original image to be stitched together. Figure 2b The second original image is to be stitched together, and there is an overlapping area between the first and second original images.

[0049] Step 120: Find the optimal seam path on the overlapping area image of the original images to be stitched, and obtain the overlapping area seam image.

[0050] Step 120 specifically includes: Step 121: Separate the overlapping regions of the two original images to be stitched together to obtain two overlapping region images; Let the two original images be the first original image and the second original image, and let the overlapping region of the first original image be the first overlapping region image, as shown below. Figure 3a As shown, the overlapping region image of the second original image is the second overlapping region image, as follows. Figure 3b As shown.

[0051] Step 122: Calculate the overlap error between the two overlapping region images to obtain the overlap region difference map; The formula for calculating the error in the overlapping region is: ; In the formula, e 直 The overlapping region error is represented by p and q, which represent two horizontally adjacent pixels in the overlapping region images, respectively. I1 represents the gray value of the first overlapping region image, I2 represents the gray value of the second overlapping region image, and ||| represents the L2 norm calculation. The resulting overlapping region difference map is shown below. Figure 4 As shown.

[0052] Step 123: Calculate the minimum cumulative error for all paths on the overlapping region difference map using the minimum error objective function to obtain the minimum cumulative error map; The objective function for minimizing the error is: ; (i,j) represents the pixel coordinates on the difference map of the overlapping region, e i,j E represents the error of the current pixel. i-1,j-1 E i-1,j and E i-1,j+1 E represents the minimum cumulative error value on the top left, top, and top right sides of the current pixel, respectively. i,j This represents the minimum cumulative error across all paths of the current pixel. The minimum cumulative error diagram is shown below. Figure 5 As shown.

[0053] Step 124: Backtrack from the minimum cumulative error map to obtain the optimal splicing seam path and obtain the seam image of the overlapping area.

[0054] The specific method for obtaining the optimal splicing seam path is as follows: In the last row of the overlapping region image, y=H, the pixel with the smallest error is taken as the seam endpoint; the seam endpoint (y end x end This can be represented as: , This indicates the position where the function value is minimized; Starting from the end of the seam, backtrack upwards, subtracting 1 from the row coordinate and taking the column coordinate of the pixel with the smallest error in the previous row, until backtracking to the first row, to obtain the optimal seam path: The coordinates of the pixels on the optimal seam path can be expressed as: .

[0055] As an optional implementation, in this embodiment, the coordinates of the pixel points on the optimal seam path are specifically as follows: .

[0056] The obtained first overlapping area seam image is as follows Figure 6a As shown, the obtained seam image of the second overlapping region is as follows. Figure 6b As shown.

[0057] In this embodiment, the method for segmenting the overlapping region is as follows: Using the geometric registration parameters between the two original images to be stitched, the overlapping regions in the two original images are segmented to obtain the first overlapping region image and the second overlapping region image.

[0058] Step 130: Perform multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands.

[0059] In this step, multi-band decomposition is implemented using a pyramid structure, including but not limited to the Gaussian pyramid and the Laplace pyramid: Specifically, step 130 is implemented as follows: constructing a Laplacian pyramid structure for the overlapping region image to obtain K Laplacian pyramid components, specifically including: Step 131: Perform Gaussian filtering and downsampling on the overlapping region image layer by layer to construct a K-layer Gaussian pyramid; Step 132: Perform a difference operation on adjacent Gaussian pyramid levels to obtain the Laplace pyramid components for the corresponding levels. The method is as follows: Let G be the k-th level of the Gaussian pyramid. k , for G k+1 Perform upsampling to make its size consistent with G. k Consistent, resulting in Expand(G) k+1 ), G k With Expand(G) k+1 Subtracting pixel by pixel, we obtain the k-th layer Laplacian pyramid component L. k ,Right now: .

[0060] In this step, low-frequency components are used to characterize the overall brightness and slowly changing areas of the image, achieving brightness consistency through weight transitions over a wider spatial range; high-frequency components are used to characterize details such as edges and textures, avoiding detail blurring through fusion over a narrower range.

[0061] Step 140: Based on the differences in the seam images of the overlapping regions, set the fusion weights of the sampled images.

[0062] Let the overlapping region seam image obtained based on the first overlapping region image be the first overlapping region seam image, and the overlapping region seam image obtained based on the second overlapping region image be the second overlapping region seam image; Differential Gaussian smoothing is applied to the seam images of the first and second overlapping regions according to the pyramid levels to obtain the fusion weight w of the k-th layer Laplacian pyramid component. 1k and w 2k Among them, w 1k +w 2k =1.

[0063] Step 150: Based on the set fusion weights, perform weighted fusion on the sampled images of the two overlapping regions, and reconstruct the fusion result to generate a fused image of the two overlapping regions.

[0064] The method for weighted fusion of sampled images is as follows: We perform a weighted fusion of the k-th level Laplace pyramid components to obtain the k-th level merged pyramid component R. k The calculation method is as follows: R k =L 1k* w 1k +L 2k* w 2k L 1k and L 2k These represent the Laplacian components of the first and second overlapping region images at the k-th layer, respectively. The method for reconstructing the fusion results is as follows: The merged pyramid components are upsampled and stacked layer by layer from top to bottom to complete the reconstruction of the fused image; the reconstruction relationship can be expressed as: S k =R k +expand(S k+1 In the formula, S k This represents the fusion of pyramids.

[0065] Step 160: Replace the overlapping area image with the fused image, and stitch the two original images to be stitched together to obtain the stitched image.

[0066] like Figure 7 As shown, it is a stitched image obtained by fusing the first original image and the second original image.

[0067] Step 170: Perform local brightness homogenization processing on the splicing boundary of the spliced ​​image.

[0068] To eliminate brightness differences between adjacent areas after image stitching, this embodiment further performs localized fine-tuning of brightness uniformity around the stitching boundary. The method is as follows: Using the stitching boundary of the stitched image as the center reference, expand the area of ​​interest (ROI) to both sides by a preset pixel width to obtain the ROI region: Extract the first ROI image corresponding to the ROI region from the fused image, such as... Figure 8 As shown, a second ROI image is extracted from the original image corresponding to the ROI region, as follows. Figure 9 As shown; Perform secondary multi-band fusion processing on the first and second ROI images to generate a smoothed fused ROI image, such as... Figure 10 As shown; The image of the corresponding region in the stitched image obtained in step 160 is replaced with the image of the fused ROI to obtain the final stitched image with continuous and uniform brightness and no obvious brightness difference, such as... Figure 11 As shown.

[0069] The above methods can effectively eliminate brightness differences at the seams and reduce the visually perceptible splicing boundaries.

[0070] In summary, the multi-band fusion processing method for seamless stitching of overlapping images provided in this embodiment is based on the following core idea: decomposing the image information of the overlapping area into multiple spatial frequency bands, performing large-scale smooth fusion in the low-frequency band, and performing small-scale or local fusion in the high-frequency band, and finally reconstructing a seamless stitching result. This method decomposes the overlapping region image at multiple scales and employs differentiated fusion strategies in different frequency bands to achieve a smooth transition in low-frequency brightness and preservation of high-frequency details, thereby obtaining a stitched image with no obvious seams, consistent brightness, and clear details. This method has at least the following effects: The multi-band fusion mechanism effectively eliminated the problem of inconsistent brightness at the splicing seams; While ensuring a smooth transition in low-frequency brightness, it preserves high-frequency detail information and reduces blur and ghosting; It is more robust to minor registration errors and local motion, and can adapt to complex scenarios; It can be widely used in panoramic image stitching, industrial visual inspection, large field of view imaging and other scenarios.

[0071] Example 2: Please refer to Figure 12 , Figure 12 This invention provides an architecture diagram of a multi-band fusion processing system for seamless stitching of overlapping images, comprising: Image input module 10 is used to acquire the original image to be stitched; The overlapping area analysis module 20 and the electrical connection image input module 10 are used to find the best seam path on the overlapping area image of the original image to be stitched and obtain the overlapping area seam image. The multi-band decomposition module 30 is electrically connected to the overlapping region analysis module 20, which is used to perform multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands. The fusion module 40 is electrically connected to the overlapping region analysis module 20, which is used to set the fusion weight of the sampled images based on the differences in the seam images of the overlapping regions; and is also electrically connected to the multi-band decomposition module 30, which is used by the probe to perform weighted fusion of the sampled images of the two overlapping regions based on the set fusion weight, and to reconstruct the fusion result to generate a fused image of the two overlapping regions. The stitching module 50 and the electrical connection fusion module 40 are used to replace the overlapping area image with the fused image, and to stitch the two original images to be stitched together to obtain the stitched image.

[0072] The brightness uniformity processing module 60 and the electrical connection splicing module 50 are used to perform local brightness uniformity processing on the splicing boundary of the spliced ​​image.

[0073] The above-described device can realize the multi-band fusion processing method for seamless stitching of overlapping images as described in Embodiment 1. Since Embodiment 1 has already described the multi-band fusion processing method in detail, it will not be repeated in this embodiment.

[0074] Example 3: This embodiment also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement a multi-band fusion processing method for seamless stitching of overlapping images as described in Embodiment 1.

[0075] Since Example 1 has already described in detail the multi-band fusion processing method for seamless stitching of overlapping images, it will not be repeated in this example.

[0076] Example 4: The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements a multi-band fusion processing method for seamless stitching of overlapping images as described in Embodiment 1.

[0077] Since Example 1 has already described in detail the multi-band fusion processing method for seamless stitching of overlapping images, it will not be repeated in this example.

[0078] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0079] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0080] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-band fusion processing method for seamless stitching of overlapping images, characterized in that, include: Find the optimal seam path on the overlapping area image of the original images to be stitched, and obtain the overlapping area seam image; Multi-band decomposition is performed on the overlapping region image to obtain sampled images of K different frequency bands; Based on the differences in the seam images of the overlapping region, the fusion weights of the sampled images are set; Based on the set fusion weights, the sampled images of the two overlapping regions are weighted and fused, and the fusion result is reconstructed to generate a fused image of the two overlapping regions. Replace the overlapping area image with the fused image, and then stitch the two original images to be stitched together to obtain the stitched image.

2. The multi-band fusion processing method for seamless stitching of overlapping images according to claim 1, characterized in that, The step of finding the optimal seam path on the overlapping area image of the original images to be stitched, and obtaining the overlapping area seam image, includes: The overlapping regions of the two original images to be stitched are segmented to obtain two overlapping region images; let the two original images be the first original image and the second original image, and let the overlapping region image of the first original image be the first overlapping region image, and the overlapping region image of the second original image be the second overlapping region image. Calculate the overlap region error between two overlapping region images to obtain an overlapping region difference map; the formula for calculating the overlap region error is: ； In the formula, e 直 The overlapping region error is represented by p and q, which represent two horizontally adjacent pixels in the overlapping region image, respectively. I1 represents the gray value of the first overlapping region image, I2 represents the gray value of the second overlapping region image, and ||| represents the L2 norm calculation. The minimum cumulative error is calculated for all paths on the difference map of the overlapping region using the minimum error objective function, resulting in the minimum cumulative error map; the minimum error objective function is as follows: ； In the formula, (i,j) represents the pixel coordinates on the difference map of the overlapping region, and e i,j E represents the error of the current pixel. i-1,j-1 E i-1,j and E i-1,j+1 E represents the minimum cumulative error value on the top left, top, and top right sides of the current pixel, respectively. i,j This represents the minimum cumulative error across all paths of the current pixel. Finally, the optimal splicing seam path is obtained by backtracking from the minimum cumulative error map, and the seam image of the overlapping area is obtained.

3. The multi-band fusion processing method for seamless stitching of overlapping images according to claim 2, characterized in that, The method for obtaining the optimal splicing seam path is as follows: In the last row of the overlapping region image, y=H, the pixel with the smallest error is taken as the seam endpoint; the seam endpoint (y end x end This can be represented as: , This indicates the position where the function value is minimized; Starting from the end of the seam, backtrack upwards, subtracting 1 from the row coordinate and taking the column coordinate of the pixel with the smallest error in the previous row, until backtracking to the first row, to obtain the optimal seam path: The coordinates of the pixels on the optimal seam path can be expressed as: 。 4. The multi-band fusion processing method for seamless stitching of overlapping images according to claim 3, characterized in that, The coordinates of the pixels on the optimal seam path are as follows: 。 5. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 4, characterized in that, The step of performing multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands specifically involves constructing a Laplacian pyramid structure on the overlapping region image to obtain K Laplacian pyramid components.

6. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 5, characterized in that, The construction of a Laplacian pyramid structure from the overlapping region image to obtain K Laplacian pyramid components specifically includes: Gaussian filtering and downsampling are performed layer by layer on the overlapping region image to construct a K-layer Gaussian pyramid; The Laplacian pyramid components of the corresponding level are obtained by performing a difference operation on adjacent Gaussian pyramids. The method is as follows: Let G be the k-th level of the Gaussian pyramid. k , for G k+1 Perform upsampling to make its size consistent with G. k Consistent, resulting in Expand(G) k+1 ), G k With Expand(G) k+1 Subtracting pixel by pixel, we obtain the k-th layer Laplacian pyramid component L. k ,Right now: 。 7. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 6, characterized in that, The method for setting fusion weights for sampled images based on the differentiation of seam images in overlapping regions is as follows: Let the overlapping region seam image obtained based on the first overlapping region image be the first overlapping region seam image, and the overlapping region seam image obtained based on the second overlapping region image be the second overlapping region seam image; Differential Gaussian smoothing is applied to the seam images of the first and second overlapping regions according to the pyramid level to obtain the fusion weight w of the k-th Laplacian pyramid component. 1k and w 2k ; Among them, w 1k +w 2k =1; The method for weighted fusion of sampled images from two overlapping regions based on a set fusion weight is as follows: We perform a weighted fusion of the k-th level Laplace pyramid components to obtain the k-th level merged pyramid component R. k The calculation method is as follows: R k =L 1k* w 1k +L 2k* w 2k L 1k and L 2k These represent the Laplacian components of the first and second overlapping region images at the k-th layer, respectively. The method for reconstructing the fusion result to generate a fused image is as follows: The merged pyramid components are upsampled and stacked layer by layer from top to bottom to complete the reconstruction of the fused image; the reconstruction relationship can be expressed as: S k =R k +expand(S k+1 In the formula, S k This represents the fusion of pyramids.

8. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 7, characterized in that, After obtaining the stitched image, the following is also included: Local brightness homogenization is performed on the stitching boundaries of the stitched images.

9. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 8, characterized in that, The method for performing local brightness homogenization processing on the splicing boundary of the spliced ​​image is as follows: Using the stitching boundary of the stitched image as the center reference, expand the area of ​​interest (ROI) to both sides by a preset pixel width to obtain the ROI region: Extract a first ROI image corresponding to the ROI region from the fused image, and extract a second ROI image corresponding to the ROI region from the original image; Perform secondary multi-band fusion processing on the first ROI image and the second ROI image to generate a smoothed fused ROI image; The image of the fused ROI is used to replace the corresponding region in the stitched image to obtain the final stitched image.

10. A multi-band fusion processing method for seamless stitching of overlapping images according to claim 2, characterized in that, Before finding the optimal seam path on the overlapping area image of the original images to be stitched and obtaining the overlapping area seam image, the process also includes: Acquire at least two original images that have overlapping regions; The method for segmenting the overlapping regions of the two original images to be stitched together to obtain two overlapping region images is as follows: Using the geometric registration parameters between the two original images to be stitched, the overlapping regions in the two original images are segmented to obtain the first overlapping region image and the second overlapping region image.

11. A multi-band fusion processing system for seamless stitching of overlapping images, characterized in that, include: The overlapping region analysis module is used to find the optimal seam path on the overlapping region image of the original images to be stitched, and to obtain the overlapping region seam image. The multi-band decomposition module is electrically connected to the overlapping region analysis module and is used to perform multi-band decomposition on the overlapping region image to obtain sampled images of K different frequency bands. The fusion module is electrically connected to the overlapping region analysis module and is used to set the fusion weight of the sampled images based on the differences in the seam images of the overlapping regions; and is also electrically connected to the multi-band decomposition module, which is used by the probe to perform weighted fusion of the sampled images of the two overlapping regions based on the set fusion weight, and to reconstruct the fusion result to generate a fused image of the two overlapping regions. The stitching module, electrically connected to the fusion module, is used to replace the overlapping area image with the fused image, and to stitch the two original images to be stitched together to obtain the stitched image.

12. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 11, characterized in that, The overlapping region analysis module is specifically used to: segment out the overlapping regions of the two original images to be stitched together, and obtain two overlapping region images; Let the two original images be the first original image and the second original image, and let the overlapping region of the first original image be the first overlapping region image and the overlapping region of the second original image be the second overlapping region image. The overlapping region analysis module is also used to calculate the overlapping region error between two overlapping region images, obtaining an overlapping region difference map; wherein, the formula for calculating the overlapping region error is: ； In the formula, e 直 The overlapping region error is represented by p and q, which represent two horizontally adjacent pixels in the overlapping region image, respectively. I1 represents the gray value of the first overlapping region image, I2 represents the gray value of the second overlapping region image, and ||| represents the L2 norm calculation. The overlapping region analysis module is also used to calculate the minimum cumulative error for all paths on the overlapping region difference map using a minimum error objective function, thereby obtaining a minimum cumulative error map; wherein, the minimum error objective function is: ； In the formula, (i,j) represents the pixel coordinates on the difference map of the overlapping region, and e i,j E represents the error of the current pixel. i-1,j-1 E i-1,j and E i-1,j+1 E represents the minimum cumulative error value on the top left, top, and top right sides of the current pixel, respectively. i,j This represents the minimum cumulative error across all paths of the current pixel. The overlapping region analysis module is also used to backtrack from the minimum cumulative error map to obtain the optimal splicing seam path and obtain the overlapping region seam image.

13. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 12, characterized in that, The method for obtaining the optimal splicing seam path is as follows: In the last row of the overlapping region image, y=H, the pixel with the smallest error is taken as the seam endpoint; the seam endpoint (y end x end This can be represented as: , This indicates the position where the function value is minimized; Starting from the end of the seam, backtrack upwards, subtracting 1 from the row coordinate and taking the column coordinate of the pixel with the smallest error in the previous row, until backtracking to the first row, to obtain the optimal seam path: The coordinates of the pixels on the optimal seam path can be expressed as: 。 14. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 13, characterized in that, The coordinates of the pixels on the optimal seam path are as follows: 。 15. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 14, characterized in that, The multi-band decomposition module is specifically used to: construct a Laplacian pyramid structure for the overlapping region image to obtain K Laplacian pyramid components.

16. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 15, characterized in that, The construction of a Laplacian pyramid structure from the overlapping region image to obtain K Laplacian pyramid components specifically includes: Gaussian filtering and downsampling are performed layer by layer on the overlapping region image to construct a K-layer Gaussian pyramid; The Laplacian pyramid components of the corresponding level are obtained by performing a difference operation on adjacent Gaussian pyramids. The method is as follows: Let G be the k-th level of the Gaussian pyramid. k , for G k+1 Perform upsampling to make its size consistent with G. k Consistent, resulting in Expand(G) k+1 ), G k With Expand(G) k+1 Subtracting pixel by pixel, we obtain the k-th layer Laplacian pyramid component L. k ,Right now: 。 17. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 16, characterized in that, The method for setting fusion weights for sampled images based on the differentiation of seam images in overlapping regions is as follows: Let the overlapping region seam image obtained based on the first overlapping region image be the first overlapping region seam image, and the overlapping region seam image obtained based on the second overlapping region image be the second overlapping region seam image; Differential Gaussian smoothing is applied to the seam images of the first and second overlapping regions according to the pyramid level to obtain the fusion weight w of the k-th Laplacian pyramid component. 1k and w 2k ; Among them, w 1k +w 2k =1; The method for weighted fusion of sampled images from two overlapping regions based on a set fusion weight is as follows: We perform a weighted fusion of the k-th level Laplace pyramid components to obtain the k-th level merged pyramid component R. k The calculation method is as follows: R k =L 1k* w 1k +L 2k* w 2k L 1k and L 2k These represent the Laplacian components of the first and second overlapping region images at the k-th layer, respectively. The method for reconstructing the fusion result to generate a fused image is as follows: The merged pyramid components are upsampled and stacked layer by layer from top to bottom to complete the reconstruction of the fused image; the reconstruction relationship can be expressed as: S k =R k +expand(S k+1 In the formula, S k This represents the fusion of pyramids.

18. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 17, characterized in that, It also includes a brightness uniformity processing module, which is electrically connected to the stitching module and is used to perform local brightness uniformity processing on the stitching boundary of the stitched image.

19. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 18, characterized in that, The method for performing local brightness homogenization processing on the splicing boundary of the spliced ​​image is as follows: Using the stitching boundary of the stitched image as the center reference, expand the area of ​​interest (ROI) to both sides by a preset pixel width to obtain the ROI region: Extract a first ROI image corresponding to the ROI region from the fused image, and extract a second ROI image corresponding to the ROI region from the original image; Perform secondary multi-band fusion processing on the first ROI image and the second ROI image to generate a smoothed fused ROI image; The image of the fused ROI is used to replace the corresponding region in the stitched image to obtain the final stitched image.

20. A multi-band fusion processing system for seamless stitching of overlapping images according to claim 12, characterized in that, It also includes an image input module, which is electrically connected to the overlapping region analysis module, and is used to acquire at least two original images that have overlapping regions. The method for segmenting the overlapping regions of the two original images to be stitched together to obtain two overlapping region images is as follows: Using the geometric registration parameters between the two original images to be stitched, the overlapping regions in the two original images are segmented to obtain the first overlapping region image and the second overlapping region image.

21. A computer-readable storage medium storing at least one instruction, characterized in that, The instructions are loaded and executed by the processor to implement a multi-band fusion processing method for seamless stitching of overlapping images as described in any one of claims 1-10.

22. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the multi-band fusion processing method for seamless stitching of overlapping images as described in any one of claims 1-10.

Images