Image stitching methods, devices and electronic equipment

Abstract

This application provides an image stitching method, apparatus, and electronic device, relating to the field of image stitching technology. The method includes: determining a first image to be stitched and a second image to be stitched from a set of images to be stitched; generating a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched; performing erosion processing on the second mask image to obtain a first shrinking mask image; determining an exclusive region image corresponding to the second mask image based on the first mask image and the first shrinking mask image; performing erosion and Gaussian blur processing on the first shrinking mask image and inverting it to obtain a gradient mask image; performing dilation and constraint processing on the exclusive region image to obtain a target expansion mask image; and stitching the first image to be stitched and the second image to be stitched based on the gradient mask image and the target expansion mask image. This not only eliminates stitching gaps but also improves the processing efficiency of large-scale image stitching.

Description

Technical Field

[0001] This application relates to the field of image stitching technology, and more specifically, to an image stitching method, apparatus, and electronic device. Background Technology

[0002] Image stitching is a method that stitches together multiple partially overlapping images into a wide-view, high-resolution panoramic image containing all image information, and it has been widely used in various fields.

[0003] However, commonly used image stitching methods suffer from problems such as poor fusion results, high computational complexity, and low efficiency. Summary of the Invention

[0004] In order to at least overcome the above-mentioned shortcomings in the prior art, the purpose of this application is to provide an image stitching method, apparatus and electronic device.

[0005] In a first aspect, embodiments of this application provide an image stitching method, the image stitching method comprising: A set of images to be stitched is obtained, and a first image to be stitched and a second image to be stitched are determined from the set of images to be stitched. The set of images to be stitched includes multiple adjacent images that have overlapping areas, and the first image to be stitched and the second image to be stitched are adjacent. A first mask image and a second mask image are generated based on the first image to be stitched and the second image to be stitched, respectively. The second mask image is eroded according to the first preset radius to obtain the first shrinkage mask image; The exclusive region image corresponding to the second mask image is determined based on the first mask image and the first shrink mask image; The first shrinkage mask image is eroded and Gaussian blurred according to the second preset radius to obtain the second shrinkage mask image, and the second shrinkage mask image is inverted to obtain the gradient mask image; The exclusive region image is dilated according to the second preset radius, and the dilated exclusive region image is constrained by the second shrinking mask image to obtain the target expanded mask image. The first image to be stitched and the second image to be stitched are stitched together based on the gradient mask image and the target extended mask image to obtain the target stitched image.

[0006] In one possible implementation, the step of generating a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched, respectively, includes: The first image to be stitched and the second image to be stitched are subjected to coordinate transformation to obtain a first transformed image and a second transformed image; A first mask image and a second mask image are generated based on the first transformed image and the second transformed image, respectively.

[0007] In one possible implementation, the step of performing coordinate transformation on the first image to be stitched and the second image to be stitched to obtain a first transformed image and a second transformed image includes: Feature points are extracted and matched on the first image to be stitched and the second image to be stitched to determine target feature matching point pairs; each target feature matching point pair includes a first target feature point of the first image to be stitched and a second target feature point of the second image to be stitched. Determine the homography matrix based on the target feature matching point pairs; Based on the homography matrix, coordinate transformation is performed on each of the images to be stitched in the image set to be stitched, and the maximum and minimum values ​​corresponding to the horizontal and vertical coordinates of each image to be stitched are determined respectively. A translation transformation matrix is ​​constructed based on the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates, respectively, and a geometric transformation matrix is ​​determined based on the translation transformation matrix and the homography matrix. Based on the geometric transformation matrix, the first image to be stitched and the second image to be stitched are geometrically transformed to obtain a first transformed image and a second transformed image.

[0008] In one possible implementation, the step of stitching the first image to be stitched and the second image to be stitched together based on the gradient mask image and the extended mask image to obtain the target stitched image includes: The gradient mask image is updated based on the target extended mask image to obtain the weight mask image; The first image to be stitched is updated based on the weighted mask image to obtain the first stitched image; The first stitched image is updated based on the extended mask image to obtain the target stitched image.

[0009] In one possible implementation, the first stitched image is calculated in the following way:

[0010] Where Ia_w represents the first stitched image; Ia_w0 represents the first transformed image; Ib_w0 represents the second transformed image; and Ib_m_grad represents the weight mask image.

[0011] In one possible implementation, the exclusive region image corresponding to the second mask image is obtained in the following way:

[0012] Where Ib_ex represents the exclusive region image corresponding to the second mask image; Ib_m1 represents the first contracted mask image; & represents logical AND; ~ represents logical NOT; Ia_m0 represents the first mask image.

[0013] In one possible implementation, the steps of dilating the exclusive region image according to the second preset radius and constraining the dilated exclusive region image using the second shrinking mask image to obtain the target expanded mask image include: The exclusive region image is dilated according to the second preset radius to obtain the first expanded mask image; The first expanded mask image is subjected to logical AND constraint processing using the second shrunken mask image to obtain the target expanded mask image.

[0014] Secondly, embodiments of this application also provide an image stitching device, comprising: An acquisition module is used to acquire a set of images to be stitched together, and to determine a first image to be stitched together and a second image to be stitched together from the set of images to be stitched together, wherein the set of images to be stitched together includes multiple adjacent images to be stitched together that have overlapping areas, and the first image to be stitched together and the second image to be stitched together are adjacent; The generation module is used to generate a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched, respectively. The first erosion module is used to perform erosion processing on the second mask image according to a first preset radius to obtain a first shrinkage mask image; The determining module is used to determine the exclusive region image corresponding to the second mask image based on the first mask image and the first shrinking mask image; The second erosion module is used to erode and Gaussian blur the first shrinkage mask image according to the second preset radius to obtain the second shrinkage mask image, and to invert the second shrinkage mask image to obtain the gradient mask image; The dilation module is used to dilate the exclusive region image according to the second preset radius, and to constrain the dilated exclusive region image by the second shrinking mask image to obtain the target expanded mask image. The stitching module is used to stitch the first image to be stitched and the second image to be stitched together based on the gradient mask image and the target extended mask image to obtain the target stitched image.

[0015] Thirdly, embodiments of this application also provide an electronic device, including: Memory, used to store one or more programs; A processor, when the one or more programs are executed by the processor, implements the image stitching method provided in the first aspect above.

[0016] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image stitching method provided in the first aspect above.

[0017] Based on any of the above aspects, the image stitching method, apparatus, and electronic device provided in this application can effectively remove unstable edge regions and improve the geometric reliability of the fusion region by performing erosion processing on the second mask image; by performing erosion and Gaussian blur processing on the first shrinking mask image, the boundary transition can be made to have a gradient distribution, avoiding brightness or color jump problems; by performing dilation processing on the exclusive region image and constraining the dilated exclusive region image through the second shrinking mask image, the expansion of the exclusive region can be guaranteed to be safe and not cover the effective area of ​​the image. In this way, not only can the stitching gaps at the stitching position of the target stitching image be eliminated, achieving smooth transition and continuous fusion of the seam area, but also the morphological operators can be directly used to achieve adaptive fusion of the gaps in the overlapping area of ​​the image, significantly improving the processing efficiency of large-scale image stitching while ensuring fusion quality. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the image stitching method provided in this embodiment; Figure 2a This is a schematic diagram of the first image to be stitched together provided in this embodiment; Figure 2b This is a schematic diagram of the second image to be stitched together provided in this embodiment; Figure 3a A schematic diagram of the first mask image provided in this embodiment; Figure 3b This is a schematic diagram of the second mask image provided in this embodiment; Figure 4 This is a schematic diagram of the first shrinkage mask image provided in this embodiment; Figure 5 This is a schematic diagram of the exclusive region image corresponding to the second mask image provided in this embodiment; Figure 6 This is a schematic diagram of the gradient mask image provided in this embodiment; Figure 7 This is a schematic diagram of the target extended mask image provided in this embodiment; Figure 8 This is a schematic diagram of the sub-steps of step S200 provided in this embodiment; Figure 9a A schematic diagram of the first transformed image provided in this embodiment; Figure 9b This is a schematic diagram of the second transformed image provided in this embodiment; Figure 10 This is a schematic diagram of the sub-steps of step S210 provided in this embodiment; Figure 11 This is a schematic diagram of the sub-steps of step S700 provided in this embodiment; Figure 12 This is a schematic diagram of the weighted mask image provided in this embodiment; Figure 13 A schematic diagram of the first stitched image provided in this embodiment; Figure 14 This is a schematic diagram of the target stitched image provided in this embodiment; Figure 15 This is a schematic diagram of the sub-steps of step S600 provided in this embodiment; Figure 16 This is a schematic structural block diagram of the electronic device provided in this embodiment; Figure 17 This is a schematic diagram of the functional modules of the image stitching device provided in this embodiment.

[0020] Icons: 700 - Electronic device; 710 - Processor; 720 - Computer-readable storage medium; 730 - Image stitching device; 731 - Acquisition module; 732 - Generation module; 733 - First erosion module; 734 - Determination module; 735 - Second erosion module; 736 - Expansion module; 737 - Stitching module. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] In the description of this application, it should be noted that the terms "upper," "lower," etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] It should be noted that, where there is no conflict, different features in the embodiments of this application can be combined with each other.

[0026] The inventors discovered that existing technologies typically employ alpha blending, multi-band blending, and graph-cut seam finding methods for image stitching. However, when using alpha blending to handle complex seam shapes, such as right-angle inflection points, its reliance on linear weights for transitions makes it ill-suited for complex boundary shapes, easily resulting in noticeable stitching artifacts and reduced fusion quality. Multi-band blending requires extensive convolution and interpolation operations, leading to high computational complexity, low processing efficiency, and significant system resource consumption. Graph-cut seam finding requires numerous iterative calculations and irregular memory accesses, making it difficult to fully utilize the parallel computing advantages of Compute Unified Device Architecture (CUDA) and failing to meet the demands for rapid processing of large-scale images.

[0027] In view of this, this embodiment provides a solution that can solve the above problems. The solution provided in this embodiment will be described in detail below.

[0028] This application provides an image stitching method. Please refer to the following embodiments. Figure 1 , Figure 1 Example: A flowchart of the image stitching method provided in this embodiment is shown. The image stitching method may include the following steps.

[0029] Step S100: Obtain a set of images to be stitched, and determine a first image to be stitched and a second image to be stitched from the set of images to be stitched, wherein the set of images to be stitched includes multiple adjacent images to be stitched with overlapping areas, and the first image to be stitched and the second image to be stitched are adjacent.

[0030] In this embodiment, any two adjacent images with overlapping areas can be selected as the first image to be stitched and the second image to be stitched. Please refer to [reference needed]. Figure 2a and Figure 2b , Figure 2a Example: A schematic diagram of the first image to be stitched provided in this embodiment. Figure 2b Example: A schematic diagram of the second image to be stitched provided in this embodiment. The image set to be stitched may include multiple adjacent images to be stitched, and there are overlapping areas between the multiple images to be stitched. The images to be stitched may be images taken by a camera at different times or angles. The multiple images to be stitched are the same size, with the same width and height, and the multiple images to be stitched are also in the same format. For example, the multiple images to be stitched may all be RGB format images.

[0031] Step S200: Generate a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched, respectively.

[0032] In this embodiment, a binarized first mask image and a second mask image can be generated based on the first image to be stitched and the second image to be stitched, respectively, to identify the effective pixel regions of the first image to be stitched and the second image to be stitched. Please refer to... Figure 3a and Figure 3b , Figure 3a Example: A schematic diagram of the first mask image provided in this embodiment. Figure 3b Example: A schematic diagram of the second mask image provided in this embodiment. White pixels (value 255) represent valid areas (i.e., pixels containing content in the first and second images to be stitched together), while black pixels (value 0) represent invalid or transparent areas.

[0033] Step S300: Erosion processing is performed on the second mask image according to the first preset radius to obtain the first shrinkage mask image.

[0034] In this embodiment, the second mask image obtained in verification step S200, with a radius of a first preset radius, can be eroded, and the erosion result can be reassigned to the second mask image to obtain the first shrinkage mask image. Please refer to... Figure 4 , Figure 4 Example: A schematic diagram of the first shrinkage mask image provided in this embodiment. The first preset radius ranges from 5 to 40.

[0035] In the above design, by performing erosion processing on the second mask image, unstable edge regions can be removed, and invalid pixels caused by geometric errors can be reduced to participate in the fusion, thus ensuring the reliability of the fusion region.

[0036] Step S400: Determine the exclusive region image corresponding to the second mask image based on the first mask image and the first shrink mask image.

[0037] In this embodiment, to clearly distinguish between image fusion regions and non-fusion regions, the exclusive region image corresponding to the second mask image can be calculated based on the first mask image obtained in step S200 and the first shrinking mask image obtained in step S300. Please refer to... Figure 5 , Figure 5 Example: A schematic diagram of the exclusive region image corresponding to the second mask image provided in this embodiment. The exclusive region image corresponding to the second mask image refers to the unique region of the second image to be stitched, that is, the region in the second image to be stitched other than the overlapping region of the first and second images to be stitched.

[0038] Specifically, the exclusive region image corresponding to the second mask image can be obtained in the following way:

[0039] Wherein, Ib_ex can represent the exclusive region image corresponding to the second mask image; Ib_m1 can represent the first shrink mask image; & can represent logical AND; ~ can represent logical NOT; Ia_m0 can represent the first mask image.

[0040] Step S500: Erosion and Gaussian blur processing are performed on the first shrinkage mask image according to the second preset radius to obtain the second shrinkage mask image, and the second shrinkage mask image is inverted to obtain the gradient mask image.

[0041] In this embodiment, the first shrinkage mask image obtained in the verification step S300 with a radius of the second preset radius can be used for erosion processing to further reduce the boundary based on the first erosion, so as to generate a smooth transition region in the future.

[0042] After the erosion process, the erosion result can be Gaussian blurred using a kernel with a second preset radius to obtain a second shrinkage mask image. The edges of the second shrinkage mask image exhibit a gradient distribution. Then, the second shrinkage mask image is inverted, so that white pixels (value 255) represent invalid or transparent areas, and black pixels (value 0) represent valid areas (i.e., pixels containing content in the first and second images to be stitched together), thus obtaining a gradient mask image. Please refer to [reference needed]. Figure 6 , Figure 6 This example illustrates a gradient mask image. This ensures a natural transition in overlapping areas and avoids noticeable seams. The first preset radius ranges from 10 to 30.

[0043] Step S600: Dilate the exclusive region image according to the second preset radius, and constrain the dilated exclusive region image by the second shrinking mask image to obtain the target expanded mask image.

[0044] In this embodiment, the exclusive region image obtained in the verification step S400 with a radius of the second preset radius can be dilated to expand the exclusive region image outward in space, thereby expanding the range of available pixels and enabling it to generate a certain overlap buffer with the overlapping areas of the first and second images to be stitched, which is beneficial to improving the fusion effect.

[0045] However, since the outward expansion of the exclusive region image in space is not restricted in the pixel coordinate system, it may lead to an excessively large dilation range, thus affecting the fusion effect. Therefore, the dilated exclusive region image can be constrained using the second shrinking mask image obtained in step S500 to obtain the target expanded mask image. Please refer to [reference needed]. Figure 7 , Figure 7 This embodiment provides a schematic diagram of the target expanded mask image. It avoids the expanded image from covering the effective area of ​​the existing image, while retaining the exclusive region image corresponding to the second image to be stitched, ensuring edge processing safety and improving fusion security.

[0046] Step S700: The first image to be stitched and the second image to be stitched are stitched together according to the gradient mask image and the target extended mask image to obtain the target stitched image.

[0047] In this embodiment, the first image to be stitched and the second image to be stitched can be stitched together using a weighted fusion formula based on the gradient mask image obtained in step S500 and the target extended mask image obtained in step S600, so as to achieve a natural transition in the overlapping and non-overlapping areas, while ensuring the geometric reliability of the edges and the security of the exclusive area.

[0048] After stitching the first image to be stitched and the second image to be stitched, other adjacent images to be stitched can be stitched using the methods in steps S100 to S700, thereby obtaining a seamlessly fused target stitched image.

[0049] As can be seen, based on the above design, the image stitching method provided in this application embodiment can effectively remove unstable edge regions and improve the geometric reliability of the fusion region by performing erosion processing on the second mask image; by performing erosion and Gaussian blur processing on the first shrinking mask image, the boundary transition can be made to have a gradient distribution, avoiding brightness or color jump problems; by performing dilation processing on the exclusive region image and constraining the dilated exclusive region image through the second shrinking mask image, the expansion of the exclusive region can be guaranteed to be safe and not cover the effective area of ​​the image. In this way, not only can the stitching gaps at the stitching position of the target stitching image be eliminated, achieving smooth transition and continuous fusion of the seam area, but also the morphological operators can be directly used to achieve adaptive fusion of the gaps in the overlapping area of ​​the image, significantly improving the processing efficiency of large-scale image stitching while ensuring fusion quality.

[0050] In one possible implementation, please refer to Figure 8 Step S200 may include the following sub-steps.

[0051] Step S210: Perform coordinate transformation on the first image to be stitched and the second image to be stitched to obtain a first transformed image and a second transformed image.

[0052] In this embodiment, coordinate transformation can be performed on all images to be stitched in the image set to be stitched, so that all images to be stitched are located in the same whole coordinate system, thereby ensuring that the target stitched image obtained by stitching all images to be stitched can be located in the positive coordinate system.

[0053] Specifically, by performing coordinate transformations on the first and second images to be stitched together, we can obtain the first transformed image and the second transformed image. Please refer to... Figure 9a and Figure 9b , Figure 9a Example: A schematic diagram of the first transformed image provided in this embodiment. Figure 9b Example: A schematic diagram of the second transformed image provided in this embodiment.

[0054] Step S220: Generate a first mask image and a second mask image based on the first transformed image and the second transformed image, respectively.

[0055] In this embodiment, a binarized first mask image and a second mask image can be generated based on the first transformed image and the second transformed image obtained in step S210, respectively, to distinguish the effective area and the blank area in the image, so as to facilitate the processing of the overlapping area of ​​the first transformed image and the second transformed image.

[0056] In one possible implementation, please refer to Figure 10 Step S210 may include the following sub-steps.

[0057] Step S211: Extract and match feature points for the first image to be stitched and the second image to be stitched to determine target feature matching point pairs; each target feature matching point pair includes a first target feature point of the first image to be stitched and a second target feature point of the second image to be stitched.

[0058] In this embodiment, the feature point coordinates and feature point description vectors of the first and second images to be stitched can be obtained by the Scale-Invariant Feature Transform (SIFT) algorithm. Each feature point corresponds to a 128-dimensional feature vector. Then, feature point matching is performed using the feature point coordinates and feature point description vectors of the first and second images to be stitched. The matched feature points are then purified to remove erroneous feature matching point pairs, resulting in a target feature matching point pair set.

[0059] Specifically, when performing feature point matching, a kd-tree (K-Dimensional Tree) index can be constructed based on the feature point coordinates and feature point description vectors of the first and second images to be stitched together. Then, candidate matching point pairs (i.e., possible corresponding feature matching point pairs in the first and second images to be stitched together) are searched in the kd-tree (K-Dimensional Tree) using the Fast Library for Approximate Nearest Neighbors (FLANN) method. Finally, erroneous feature matching point pairs are eliminated using the Lowe ratio test method, and valid feature matching point pairs are extracted to obtain the target feature matching point pair set.

[0060] Step S212: Determine the homography matrix based on the target feature matching point pairs.

[0061] In this embodiment, the homography matrix between the first image to be stitched and the second image to be stitched can be calculated using the Random Sample Consensus (RANSAC) algorithm based on the target feature matching point pairs in the target feature matching point pair set.

[0062] Step S213: Based on the homography matrix, perform coordinate transformation on each of the images to be stitched in the image set to be stitched, and determine the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates, respectively.

[0063] In this embodiment, coordinate transformation can be performed on all images to be stitched according to the homography matrix obtained in step S212, so that all images to be stitched are located in the same coordinate system. At the same time, the coordinates of the four corner points of each image to be stitched are obtained in the coordinate system, and the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates are calculated according to the corner point coordinates. Thus, the overall size of the target stitched image formed by stitching all images to be stitched is obtained according to the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates.

[0064] Step S214: Construct a translation transformation matrix based on the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates, respectively, and determine the geometric transformation matrix based on the translation transformation matrix and the homography matrix.

[0065] In this embodiment, a translation transformation matrix can be constructed based on the maximum and minimum values ​​of the horizontal and vertical coordinates of each image to be stitched obtained in step S213, and the translation transformation matrix can be multiplied by the homography matrix to obtain the corrected geometric transformation matrix.

[0066] Step S215: Perform geometric transformations on the first image to be stitched and the second image to be stitched based on the geometric transformation matrix to obtain a first transformed image and a second transformed image.

[0067] In this embodiment, all images to be stitched can be mapped to a new coordinate system based on the geometric transformation matrix obtained in step S214, ensuring that the coordinates of each image are non-negative. This guarantees the correct positioning of each image on the stitching canvas and avoids offset issues. During the process of mapping all images to the new coordinate system, the size of the output image can be determined by the overall size of the target stitched image obtained in step S213.

[0068] In one possible implementation, please refer to Figure 11 Step S700 may include the following sub-steps.

[0069] Step S710: Update the gradient mask image according to the target extended mask image to obtain the weight mask image.

[0070] In this embodiment, the gradient mask image can be updated using the target extended mask image. Specifically, in the overlapping area of ​​the target extended mask image and the gradient mask image, the pixel values ​​of the target extended mask image are assigned to the gradient mask image, so that the pixel value of the corresponding region of the target extended mask image in the gradient mask image is 255, thus obtaining the weighted mask image. Please refer to... Figure 12 , Figure 12 Example: A schematic diagram of the weighted mask image provided in this embodiment.

[0071] Step S720: Update the first image to be stitched based on the weighted mask image to obtain the first stitched image.

[0072] In this embodiment, the first image to be stitched can be updated based on the weighted mask image obtained in step S710, so that the overlapping areas of the first and second stitched images are smoothly blended to obtain the first stitched image. Please refer to... Figure 13 , Figure 13 This embodiment provides a schematic diagram of the first stitched image. Seamless stitching is achieved, avoiding abrupt changes in brightness or color.

[0073] Specifically, the first stitched image can be calculated in the following way:

[0074] Where Ia_w can represent the first stitched image; Ia_w0 can represent the first transformed image; Ib_w0 can represent the second transformed image; and Ib_m_grad can represent the weight mask image.

[0075] Step S730: Update the first stitched image based on the extended mask image to obtain the target stitched image.

[0076] In this embodiment, the first stitched image can be updated by extending the mask image. That is, in the region with a pixel value of 255 in the extended mask image, the pixel value of the second transformed image is directly used to update the first stitched image, thereby merging the exclusive region corresponding to the second mask image with the first stitched image to obtain the target stitched image. Please refer to... Figure 14 , Figure 14 Example: A schematic diagram of the target stitched image provided in this embodiment.

[0077] In one possible implementation, please refer to Figure 15 Step S600 may include the following sub-steps.

[0078] Step S610: Dilate the exclusive region image according to the second preset radius to obtain the first extended mask image.

[0079] In this embodiment, if the exclusive area image corresponding to the second mask image is directly used for splicing at the edge of the overlapping area, it is easy to cause sudden changes in color or brightness and obvious splicing marks. Therefore, the exclusive area image corresponding to the second mask image can be expanded according to the second preset radius to expand the exclusive area so that it can generate a certain overlap buffer with the overlapping area, leaving a gradual transition space for the fusion edge.

[0080] Step S620: Perform logical AND constraint processing on the first expanded mask image using the second contracted mask image to obtain the target expanded mask image.

[0081] In this embodiment, to prevent the expansion of the exclusive region image from exceeding the original boundary of the second transformed image, or from expanding into the overlapping area with the first transformed image or the background area of ​​the first image to be stitched together, a logical AND operation can be performed between the second shrinking mask image and the first expanding mask image. This ensures that the boundary of the first expanding mask image after the expansion process is located within the boundary of the second transformed image, guaranteeing the safety of the exclusive region expansion and preventing it from covering the effective area of ​​the image.

[0082] Based on the same inventive concept, this embodiment also provides an electronic device 700, please refer to... Figure 16 , Figure 16 A block diagram of an example electronic device 700. The electronic device 700 includes an image stitching device 730, a computer-readable storage medium 720, and a processor 710.

[0083] The computer-readable storage medium 720 and the processor 710 are electrically connected directly or indirectly to each other to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The image stitching device 730 includes multiple software function modules that can be stored in the computer-readable storage medium 720 or embedded in the operating system (OS) of the image stitching device 730 in the form of software or firmware. The processor 710 is used to execute executable modules stored in the computer-readable storage medium 720, such as the software function modules and computer programs included in the image stitching device 730.

[0084] The computer-readable storage medium 720 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The computer-readable storage medium 720 is used to store a program, which the processor 710 executes after receiving an execution instruction.

[0085] The processor 710 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor 710 can be a general-purpose processor 710, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor 710 can be a microprocessor 710, or any conventional processor 710, etc.

[0086] Please refer to Figure 17 This application also provides an image stitching device 730. The image stitching device 730 includes multiple functional modules that can be stored in a computer-readable storage medium 720 in software form. Functionally, the image stitching device 730 may include an acquisition module 731, a generation module 732, a first erosion module 733, a determination module 734, a second erosion module 735, a dilation module 736, and a stitching module 737. Wherein: The acquisition module 731 can be used to acquire a set of images to be stitched together, and to determine a first image to be stitched together and a second image to be stitched together from the set of images to be stitched together, wherein the set of images to be stitched together includes multiple adjacent images to be stitched together that have overlapping areas, and the first image to be stitched together and the second image to be stitched together are adjacent.

[0087] In this embodiment, the acquisition module 731 can be used to perform... Figure 1 For a detailed description of the acquisition module 731, please refer to the description of step S100 shown in step S100.

[0088] The generation module 732 can be used to generate a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched, respectively.

[0089] In this embodiment, the generation module 732 can be used to execute Figure 1 For a detailed description of step S200, please refer to the description of step S200.

[0090] The first erosion module 733 can be used to erode the second mask image according to the first preset radius to obtain the first shrinkage mask image.

[0091] In this embodiment, the first corrosion module 733 can be used to perform... Figure 1 For a detailed description of the first corrosion module 733, please refer to the description of step S300 shown in step S300.

[0092] The determining module 734 can be used to determine the exclusive region image corresponding to the second mask image based on the first mask image and the first shrinking mask image.

[0093] In this embodiment, the determination module 734 can be used to perform... Figure 1 For a detailed description of the determination module 734, please refer to the description of step S400 shown.

[0094] The second erosion module 735 can be used to erode and Gaussian blur the first shrinkage mask image according to the second preset radius to obtain the second shrinkage mask image, and invert the second shrinkage mask image to obtain the gradient mask image.

[0095] In this embodiment, the second corrosion module 735 can be used to perform... Figure 1 For a detailed description of the second corrosion module 735, please refer to the description of step S500 shown in step S500.

[0096] The dilation module 736 can be used to dilate the exclusive region image according to the second preset radius, and to constrain the dilated exclusive region image by the second shrinking mask image to obtain the target expanded mask image.

[0097] In this embodiment, the expansion module 736 can be used to perform... Figure 1 For a detailed description of step S600, please refer to the description of step S600.

[0098] The stitching module 737 can be used to stitch the first image to be stitched and the second image to be stitched according to the gradient mask image and the target extended mask image to obtain the target stitched image.

[0099] In this embodiment, the splicing module 737 can be used to perform... Figure 1 For a detailed description of step S700, please refer to the description of step S700.

[0100] In summary, the embodiments of this application provide an image stitching method, apparatus, and electronic device. By performing erosion processing on the second mask image, unstable edge regions can be effectively removed, improving the geometric reliability of the fusion region. By performing erosion and Gaussian blur processing on the first shrinking mask image, the boundary transition can be made to have a gradient distribution, avoiding brightness or color jump problems. By performing dilation processing on the exclusive region image and constraining the dilated exclusive region image through the second shrinking mask image, the expansion of the exclusive region can be guaranteed to be safe and not cover the effective area of ​​the image. In this way, not only can the stitching gaps at the stitching position of the target stitching image be eliminated, achieving smooth transition and continuous fusion of the seam region, but also morphological operators can be directly used to achieve adaptive fusion of gaps in the overlapping area of ​​the image, significantly improving the processing efficiency of large-scale image stitching while ensuring fusion quality.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0102] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image stitching method, characterized in that, The method includes: A set of images to be stitched is obtained, and a first image to be stitched and a second image to be stitched are determined from the set of images to be stitched. The set of images to be stitched includes multiple adjacent images that have overlapping areas, and the first image to be stitched and the second image to be stitched are adjacent. The first image to be stitched and the second image to be stitched are subjected to coordinate transformation to obtain a first transformed image and a second transformed image; A first mask image and a second mask image are generated based on the first transformed image and the second transformed image, respectively; The second mask image is eroded according to the first preset radius to obtain the first shrinkage mask image; The exclusive region image corresponding to the second mask image is determined based on the first mask image and the first shrinking mask image; the first shrinking mask image is eroded and Gaussian blurred according to the second preset radius to obtain the second shrinking mask image, and the second shrinking mask image is inverted to obtain the gradient mask image; The exclusive region image is dilated according to the second preset radius, and the dilated exclusive region image is constrained by the second shrinking mask image to obtain the target expanded mask image. Specifically, this includes the following steps. The exclusive region image is dilated according to a second preset radius to obtain a first expanded mask image; a logical AND operation is performed between the second contracted mask image and the first expanded mask image, so that the boundary of the first expanded mask image after dilation is again located within the boundary of the second transformed image; the first image to be stitched and the second image to be stitched are stitched together according to the gradient mask image and the target expanded mask image to obtain a target stitched image. Specifically, this includes the following steps: In the overlapping area of ​​the target extended mask image and the gradient mask image, the pixel values ​​of the target extended mask image are assigned to the gradient mask image, so that the pixel values ​​of the corresponding areas of the target extended mask image in the gradient mask image are the pixel values ​​of the effective areas, thereby obtaining the weight mask image; The first image to be stitched is updated based on the weighted mask image, so that the overlapping areas of the first and second stitched images are smoothly blended to obtain the first stitched image. Specifically, the first stitched image is calculated in the following way: ； Wherein, Ia_w represents the first stitched image; Ia_w0 represents the first transformed image; Ib_w0 represents the second transformed image; and Ib_m_grad represents the weight mask image. In the region of the extended mask image where the pixel value is the effective region, the pixel value of the second transformed image is directly used to update the first stitched image, thereby merging the exclusive region corresponding to the second mask image with the first stitched image to obtain the target stitched image.

2. The image stitching method according to claim 1, characterized in that, The step of performing coordinate transformation on the first image to be stitched and the second image to be stitched to obtain a first transformed image and a second transformed image includes: Feature points are extracted and matched on the first image to be stitched and the second image to be stitched to determine target feature matching point pairs; each target feature matching point pair includes a first target feature point of the first image to be stitched and a second target feature point of the second image to be stitched. Determine the homography matrix based on the target feature matching point pairs; Based on the homography matrix, coordinate transformation is performed on each of the images to be stitched in the image set to be stitched, and the maximum and minimum values ​​corresponding to the horizontal and vertical coordinates of each image to be stitched are determined respectively. A translation transformation matrix is ​​constructed based on the maximum and minimum values ​​of each image to be stitched in the horizontal and vertical coordinates, respectively, and a geometric transformation matrix is ​​determined based on the translation transformation matrix and the homography matrix. Based on the geometric transformation matrix, the first image to be stitched and the second image to be stitched are geometrically transformed to obtain a first transformed image and a second transformed image.

3. The image stitching method according to claim 1, characterized in that, The exclusive region image corresponding to the second mask image is obtained in the following way: ； Where Ib_ex represents the exclusive region image corresponding to the second mask image; Ib_m1 represents the first contracted mask image; & represents logical AND; ~ represents logical NOT; Ia_m0 represents the first mask image.

4. An image stitching device, employing the image stitching method as described in any one of claims 1-3, characterized in that, include: An acquisition module is used to acquire a set of images to be stitched together, and to determine a first image to be stitched together and a second image to be stitched together from the set of images to be stitched together, wherein the set of images to be stitched together includes multiple adjacent images to be stitched together that have overlapping areas, and the first image to be stitched together and the second image to be stitched together are adjacent; The generation module is used to generate a first mask image and a second mask image based on the first image to be stitched and the second image to be stitched, respectively. The first erosion module is used to perform erosion processing on the second mask image according to a first preset radius to obtain a first shrinkage mask image; The determining module is used to determine the exclusive region image corresponding to the second mask image based on the first mask image and the first shrinking mask image; The second erosion module is used to erode and Gaussian blur the first shrinkage mask image according to the second preset radius to obtain the second shrinkage mask image, and to invert the second shrinkage mask image to obtain the gradient mask image; The dilation module is used to dilate the exclusive region image according to the second preset radius, and to constrain the dilated exclusive region image by the second shrinking mask image to obtain the target expanded mask image. The stitching module is used to stitch the first image to be stitched and the second image to be stitched together based on the gradient mask image and the target extended mask image to obtain the target stitched image.

5. An electronic device, characterized in that, include: Memory, used to store one or more programs; A processor, when the one or more programs are executed by the processor, implements the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-3.

Images