Image splicing method and system and readable storage medium

Abstract

The invention discloses an image splicing method and system and a readable storage medium. The method comprises the following steps: acquiring calibration plate images of a first visual field and a second visual field; obtaining a homography matrix between the first visual field image and the second visual field image, and taking the homography matrix as a conversion matrix between the two spliced images; acquiring a spliced image of the first visual field and the second visual field at the same visual angle as the calibration plate image, converting the spliced image by using the conversion matrix, and performing splicing according to a conversion result to obtain a preliminary spliced image; according to the preliminary spliced image, respectively calculating homography matrixes between the two spliced images and the preliminary spliced image; and transforming the two spliced images into the preliminary spliced image through the homography matrix, and carrying out pixel fusion on the transformed overlapped region to obtain a spliced image. The method disclosed by the invention has the characteristics of high precision, strong robustness, high speed and convenience in use.

Description

Technical Field

[0001] The present application relates to the technical field of image stitching, and in particular to an image stitching method, system and readable storage medium. Background Art

[0002] Image stitching technology is a technology that stitches multiple overlapping images into one image. It is very commonly used in the field of machine vision. Conventional image stitching methods mainly include feature matching-based methods and calibration plate conversion-based methods, but conventional methods have many problems and shortcomings.

[0003] The principle of the feature matching-based method is to match the feature points of the images to be stitched together, calculate the transformation matrix based on the matched feature points, and transform one of the images into the coordinate system of the other image through the transformation matrix for stitching. However, this method has high requirements for feature points. If there are a large number of similar feature points in the two images, it may cause feature point pairing errors and cause unstable stitching.

[0004] The principle of the method based on calibration plate conversion is to convert the image coordinate system of one of the images to the calibration plate coordinate system, and then convert it to the image coordinate system of another image for stitching, which can effectively avoid the instability of stitching caused by feature point matching errors. However, it requires multiple coordinate conversions, and errors are prone to occur during the conversion process. In addition, the requirements for the calibration plate are high. The calibration plate needs to satisfy the requirement that the calibration plate coordinate system can be directly read in the image, otherwise the conversion relationship from the image coordinate system to the calibration plate coordinate system and then to the image coordinate system cannot be calculated, and the conversion relationship calculation process is relatively cumbersome. At the same time, multiple conversions will bring more time-consuming in the actual stitching process, and affect the efficiency of use when the image is large. Summary of the invention

[0005] The purpose of the present application is to provide an image stitching method, system and readable storage medium. By designing a calibration plate composed of a black marker and a plurality of small holes and a conversion relationship calculation method, the coordinates of the center of mass of the small holes are extracted and matched one by one by using two images, and the conversion matrix is ​​directly calculated through multiple points to improve the accuracy and speed of image stitching. The first aspect of the present application provides an image stitching method, the method comprising: Acquire calibration plate images of the first field of view and the second field of view, wherein the calibration plate images are composed of a plurality of black rectangular markers with ordered circular holes distributed inside; Performing relational transformation on the calibration plate images of the first field of view and the second field of view to obtain a homography matrix between the first field of view and the second field of view images as a transformation matrix between the two stitched images; Acquire a stitched image of the first field of view and the second field of view at the same viewing angle as the calibration plate image, which are respectively the first stitched image and the second stitched image; The first stitched image and the second stitched image are transformed by using a transformation matrix, and stitched together according to the transformation result to obtain a preliminary stitched image; According to the preliminary stitching image, respectively calculating a homography matrix H1 between the first stitching image and the preliminary stitching image and a homography matrix H2 between the second stitching image and the preliminary stitching image; The first stitched image and the second stitched image are transformed into a preliminary stitched image through the homography matrix H1 and the homography matrix H2 respectively, and the transformed overlapping area is pixel-fused to obtain a stitched image.

[0006] Optionally, performing relational transformation on the calibration plate images of the first field of view and the second field of view to obtain a homography matrix between the first field of view and the second field of view images specifically includes: The calibration plate image is threshold segmented to obtain a first brightness area and a second brightness area of ​​the calibration plate image; the brightness value of the second brightness area is higher than the brightness value of the first brightness area Taking the connected domain of the first brightness area as the dark connected domain, performing hole filling on the dark connected domain of the calibration plate image; Filter the dark connected domains by area, and retain the dark connected domains whose area is larger than the set threshold; The connected domain with the highest rectangularity is selected from the dark connected domains whose area is larger than the set threshold, which is the marker; Extract the minimum bounding box of the marker and obtain the two corner points on the upper edge of the minimum bounding box of the marker; Calculate the straight line function of the upper edge based on the two corner points of the upper edge of the minimum bounding box; Interference shielding is performed on the second brightness area to obtain the bright connected domain inside the calibration plate image; Filter the connected domains of the circular holes according to their areas, and calculate the centroid of each connected domain of the circular holes to obtain the circular hole centroid set Set1; The vertical distance from the center of mass of each circular hole to the upper edge of the marker is calculated according to the straight line function of the upper edge, and the center of mass set of circular holes Set1 is sorted in ascending order according to the vertical distance to obtain the center of mass set of circular holes Set2; The circular hole mass center set Set2 is locally sorted according to the horizontal coordinate from small to large, and the final circular hole mass center set Set is obtained; Extract the circular hole coordinates from the calibration plate images of the first field of view and the second field of view, and obtain two sets of circular hole centroids that correspond to each other in sequence; The homography matrix H between the calibration plate image of the first field of view and the calibration plate image of the second field of view is calculated and saved through two one-to-one corresponding circular hole centroid sets.

[0007] Optionally, the connected region of the first brightness area of ​​the calibration plate image is a connected white area in the binary image of the calibration plate image.

[0008] Optionally, the calculation formula of the rectangularity is: , Among them, y is the rectangularity, is the minimum bounding box area, is the pixel area of ​​the connected domain of the marker.

[0009] Optionally, the interference shielding is specifically: extracting the marker binary image and the second brightness area binary image, and mapping the marker binary image to the second brightness area binary image, and setting the grayscale value of the area in the second brightness area binary image corresponding to the black pixel in the marker binary image to 255.

[0010] Optionally, the circular hole centroid set Set2 is locally sorted according to the horizontal coordinate from small to large, and the local sorting rule is: Determine whether the upper edge of the minimum bounding box is a long edge or a short edge, and determine the number of circular holes in each row of the upper edge of the minimum bounding box; Assume that the number of circular holes in each row on the upper edge of the minimum bounding box is X, and traverse the circular hole centroid set Set2 according to the number of circular holes, and sort every X holes in order from small to large.

[0011] Optionally, the homography matrix H1 from the first stitched image to the preliminary stitched image and the homography matrix H2 from the second stitched image to the preliminary stitched image are calculated respectively according to the preliminary stitched image, specifically: Calculate the corner points of the first and second stitched images: Assume the height of the first stitched image is , the width is ; The height of the second stitched image is , the width is ; Then the four corner points of the first stitched image are , , , ; The four corner points of the second stitched image are , , , ; The corner point coordinates of the second stitched image are transformed into the coordinate system of the first stitched image through the homography matrix H. The transformation formula is: P"=P'*H. Then P" is normalized to obtain the new four corner point coordinates. , , , ; Calculating the size of the initial stitched image after stitching by using the coordinates of the four corner points of the first stitched image and the coordinates of the four corner points of the second stitched image after conversion; The width of the initial stitched image is: ; The height of the initial stitched image is: ; Calculate the corresponding coordinates of the corner points of the first stitched image and the corner points of the second stitched image in the preliminary stitched image: The corner points of the first stitched image and the corner points of the second stitched image are subtracted , which are the corresponding coordinates in the preliminary stitching image; in, , , The homography matrix H1 between the first stitched image and the preliminary stitched image and the homography matrix H2 between the second stitched image and the preliminary stitched image are calculated respectively by using the corresponding image corner point coordinates before and after the conversion.

[0012] Optionally, the transformed overlapping area is subjected to pixel fusion, and the fusion mode includes: Mean fusion: take the mean of corresponding pixels in the first stitched image and the second stitched image; Maximum fusion: the corresponding pixels in the first stitched image and the second stitched image take the maximum value; Minimum value fusion: the corresponding pixels in the first stitched image and the second stitched image take the minimum value; End-to-end fusion: The overlapping area directly covers the second stitched image.

[0013] A second aspect of the present invention provides an image stitching system, comprising a memory and a processor, wherein the memory comprises an image stitching method program, and when the image stitching method program is executed by the processor, the steps of the image stitching method are implemented.

[0014] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes an image stitching method program, and when the image stitching method program is executed by a processor, the steps of the image stitching method are implemented.

[0015] As can be seen from the above, the image stitching method, system and readable storage medium provided by the present application have the characteristics of high precision, strong robustness, high speed and easy use.

[0016] In order to solve the problem of accuracy and speed, this application designs a calibration plate composed of black markers and many small holes and a conversion relationship calculation method. By extracting and matching the coordinates of the center of mass of the small holes between the calibration plate images of the two fields of view, the conversion matrix is ​​directly calculated through multiple points. The more points there are, the higher the accuracy will be. Compared with the conventional calibration plate conversion method, the splicing can be completed through only one transformation, and there is no loss of accuracy and speed in the middle.

[0017] In order to solve the problem of convenience, the present application proposes a coordinate pairing method, which can automatically sort and pair the extracted circular hole centroids without the need for manual matching by operators. This method is simpler than conventional calibration plate conversion methods.

[0018] In order to solve the robustness problem, the present application adopts multiple modes to select and process the overlapping areas of the splicing, which can overcome the influence of lighting to a certain extent and make the splicing boundaries more natural.

[0019] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 An overall flow chart of an image stitching method provided in an embodiment of the present application.

[0022] Figure 2 This is the calibration plate image provided in the embodiment of the present application.

[0023] Figure 3 This is the first calibration plate image provided in the embodiment of the present application.

[0024] Figure 4 This is the second calibration plate image provided in the embodiment of the present application.

[0025] Figure 5 This is the first brightness area binary image provided in the embodiment of the present application.

[0026] Figure 6 This is an image after the holes in the dark connected domain of the calibration plate image provided in the embodiment of the present application are filled.

[0027] Figure 7 This is an image after area filtering of the calibration plate image provided in the embodiment of the present application.

[0028] Figure 8 This is a binary image of a marker provided in an embodiment of the present application.

[0029] Fig. 9 This is a second brightness area binary image provided in an embodiment of the present application.

[0030] Fig.10 This is a binary image of the internal brightness of the marker provided in the embodiment of the present application.

[0031] Fig.11 This is a schematic diagram of the centroid of a circular hole provided in an embodiment of the present application.

[0032] Fig.12 A schematic diagram of the vertical distance from the centroid of the circular hole to the upper edge of the marker provided in an embodiment of the present application.

[0033] Fig.13 A block diagram of an image stitching system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0035] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0036] Please refer to Figure 1 , is an image stitching method in some embodiments of the present application, the method comprising: S102: Acquire a calibration plate image of the first field of view and the second field of view, wherein the calibration plate image is composed of a plurality of black rectangular markers with ordered circular holes distributed inside; It should be noted that the calibration plate is composed of a black rectangular marker with multiple orderly circular holes distributed inside. The number of circular holes is customized by the user and needs to follow the row and column distribution (the number of holes in each row is equal, and the number of holes in each column is equal). The more circular holes there are, the higher the accuracy. Figure 2 As shown, the black marker is located between the first field of view and the second field of view, and both the first field of view and the second field of view need to capture the entire black marker.

[0037] It should be noted that both the first field of view and the second field of view need to contain a complete rectangular marker.

[0038] If it is horizontal splicing, the markers need to be in vertical direction (short sides are at the top and bottom, long sides are at the left and right). If it is vertical splicing, the markers need to be in horizontal direction (short sides are at the left and right, long sides are at the top and bottom).

[0039] S104: performing relational transformation on the calibration plate images of the first field of view and the second field of view to obtain a homography matrix between the first field of view and the second field of view images as a transformation matrix between two stitched images; S106: Acquire a stitched image of the first field of view and the second field of view at the same viewing angle as the calibration plate image, which are respectively a first stitched image and a second stitched image; S108: transforming the first stitched image and the second stitched image using a transformation matrix, and stitching them together according to the transformation result to obtain a preliminary stitched image; S110: According to the preliminary stitching image, respectively calculating a homography matrix H1 between the first stitching image and the preliminary stitching image and a homography matrix H2 between the second stitching image and the preliminary stitching image; S112: transforming the first stitched image and the second stitched image into a preliminary stitched image through the homography matrix H1 and the homography matrix H2 respectively, and performing pixel fusion on the transformed overlapping area to obtain a stitched image.

[0040] According to an embodiment of the present invention, performing relational conversion on the calibration plate images of the first field of view and the second field of view to obtain a homography matrix between the first field of view and the second field of view images specifically includes: It should be noted that the present application includes the calibration plate images of the first field of view and the second field of view, wherein the first calibration plate image is as follows: Figure 3 As shown, the second calibration plate image is Figure 4 shown.

[0041] The calibration plate image is threshold segmented to obtain a first brightness area and a second brightness area of ​​the calibration plate image; the brightness value of the second brightness area is higher than the brightness value of the first brightness area The connected domain of the first brightness area of ​​the calibration plate image is the connected white area in the binary image of the calibration plate image.

[0042] It should be noted that in this application, the calibration plate image is binarized to obtain a calibration plate binary image, and the connected white areas in the binary image are connected domains. Figure 5 shown.

[0043] Taking the connected domain of the first brightness area as the dark connected domain, performing hole filling on the dark connected domain of the calibration plate image; It should be noted that in this application, the dark connected domain of the calibration plate image is filled with holes, specifically as follows Figure 6 shown.

[0044] Filter the dark connected domains by area, and retain the dark connected domains whose area is larger than the set threshold; Figure 7 shown.

[0045] The connected domain with the highest rectangularity is selected from the dark connected domains whose area is larger than the set threshold, which is the marker (black rectangular connected domain); Extract the minimum bounding box of the marker and obtain the two corner points on the upper edge of the minimum bounding box of the marker; It should be noted that this application extracts the minimum bounding box of the rectangular marker, obtains the four corner points of the marker, obtains the two corner points of the upper edge of the rectangular marker, and determines whether the upper edge is a short side or a long side based on the distance between the corner points. Since the number of circular holes on a long side and a short side is different, it provides a basis for subsequent operations.

[0046] Calculate the straight line function of the upper edge based on the two corner points of the upper edge of the minimum bounding box; Interference shielding is performed on the second brightness area to obtain the bright connected domain inside the calibration plate image; It should be noted that the present application performs threshold segmentation on the calibration plate image, that is, performs binarization processing on the marker, obtains a marker binary image, obtains a first brightness area and a second brightness area of ​​the calibration plate image, and the marker binary image is as follows: Figure 8 As shown, the marker binary image is then interfered and shielded to obtain the bright connected domain inside the marker, that is, the second brightness area binary image. The second brightness area binary image is as shown in Fig. 9 shown.

[0047] The shielding method is to map the marker binary image to the second brightness region binary image, and set the gray value of the area in the second brightness region binary image corresponding to the black pixel in the marker binary image to 255. Obtain the marker internal bright binary image, obtain the marker internal bright connected domain, the marker internal bright binary image is as follows Fig.10 shown.

[0048] The connected domains of the circular holes are screened according to their areas, and the centroid of each connected domain of the circular holes is calculated to obtain the circular hole centroid set Set1; the schematic diagram of the circular hole centroid is shown in Fig.11 shown.

[0049] The vertical distance from the center of mass of each circular hole to the upper edge of the marker is calculated according to the straight line function of the upper edge, and the center of mass set of circular holes Set1 is sorted in ascending order according to the vertical distance to obtain the center of mass set of circular holes Set2; It should be noted that the present application calculates the vertical distance from the center of mass of each circular hole to the upper edge of the marker, sorts the circular hole center of mass set Set1 from small to large according to the vertical distance, and obtains the circular hole center of mass set Set2. Fig.12 shown.

[0050] The circular hole mass center set Set2 is locally sorted according to the horizontal coordinate from small to large, and the final circular hole mass center set Set is obtained; Extract the circular hole coordinates from the calibration plate images of the first field of view and the second field of view, and obtain two sets of circular hole centroids that correspond to each other in sequence; It should be noted that both the first field of view and the second field of view have undergone the circular hole coordinate extraction in the above steps, and two sets of circular hole centroids corresponding to each other in sequence are obtained, which are assumed to be Set' (reference image) and Set" (image to be stitched).

[0051] The homography matrix H between the calibration plate image of the first field of view and the calibration plate image of the second field of view is calculated and saved through two one-to-one corresponding circular hole centroid sets.

[0052] According to an embodiment of the present invention, the calculation formula of the rectangularity is: , Among them, y is the rectangularity, is the minimum bounding box area, is the pixel area of ​​the connected domain of the marker.

[0053] According to an embodiment of the present invention, the interference shielding is specifically: extracting the marker binary image and the second brightness area binary image, and mapping the marker binary image to the second brightness area binary image, and setting all areas in the second brightness area binary image that correspond to the black pixels in the marker binary image to black.

[0054] According to an embodiment of the present invention, the circular hole mass center set Set2 is locally sorted from small to large according to the horizontal coordinate, and the local sorting rule is: Determine whether the upper edge of the minimum bounding box is a long edge or a short edge, and determine the number of circular holes in each row of the upper edge of the minimum bounding box; Assume that the number of holes in each row on the upper edge of the minimum bounding box is X, and traverse the hole mass center set Set2 according to the number of holes, and sort every X holes from small to large. For example, if there are 4 holes in each row, traverse the hole mass center set Set2, and sort every four holes from small to large.

[0055] According to an embodiment of the present invention, the homography matrix H1 from the first stitched image to the preliminary stitched image and the homography matrix H2 from the second stitched image to the preliminary stitched image are respectively calculated according to the preliminary stitched image, specifically: Calculate the corner points of the first and second stitched images: It should be noted that, as a specific embodiment, the present application uses the first stitched image and the second stitched image for stitching.

[0056] Assume the height of the first stitched image is , the width is ; The height of the second stitched image is , the width is ; Then the four corner points of the first stitched image are , , , ; The four corner points of the second stitched image are , , , ; The corner point coordinates of the second stitched image are transformed into the coordinate system of the first stitched image through the homography matrix H. The transformation formula is: P"=P'*H. Then P" is normalized to obtain the new four corner point coordinates. , , , ; Calculating the size of the initial stitched image after stitching by using the coordinates of the four corner points of the first stitched image and the coordinates of the four corner points of the second stitched image after conversion; The width of the initial stitched image is: ; The height of the initial stitched image is: ; Calculate the corresponding coordinates of the corner points of the first stitched image and the corner points of the second stitched image in the preliminary stitched image: The corner points of the first stitched image and the corner points of the second stitched image are subtracted , which are the corresponding coordinates in the preliminary stitching image; in, , , The homography matrix H1 between the first stitched image and the preliminary stitched image and the homography matrix H2 between the second stitched image and the preliminary stitched image are calculated respectively by using the corresponding image corner point coordinates before and after the conversion.

[0057] According to an embodiment of the present invention, the pixel fusion is performed on the transformed overlapping area, and the fusion mode includes: Mean fusion: take the mean of corresponding pixels in the first stitched image and the second stitched image; Maximum fusion: the corresponding pixels in the first stitched image and the second stitched image take the maximum value; Minimum value fusion: the corresponding pixels in the first stitched image and the second stitched image take the minimum value; End-to-end fusion: The overlapping area directly covers the second stitched image.

[0058] It should be noted that the first stitched image and the second stitched image are transformed into the preliminary stitched image through H1 and H2 respectively, and the transformed overlapping area needs to be specially processed. The special processing described in this application is to perform pixel fusion on the overlapping area, and the fusion modes that the user can select include mean fusion, maximum fusion, minimum fusion, tail fusion, etc. Pixel fusion is performed on the transformed overlapping area to obtain a stitched image.

[0059] like Fig.13 As shown, the second aspect of the present invention provides an image stitching system, the image stitching system 13 includes a memory 131 and a processor 132, the memory 131 includes an image stitching method program, and when the image stitching method program is executed by the processor 132, the steps of the image stitching method are implemented.

[0060] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes an image stitching method program, and when the image stitching method program is executed by a processor, the steps of the image stitching method are implemented.

[0061] As can be seen from the above, the image stitching method, system and readable storage medium provided by the present application have the characteristics of high precision, strong robustness, high speed and easy use.

[0062] In order to solve the problem of accuracy and speed, this application designs a calibration plate composed of black markers and many small holes and a conversion relationship calculation method. The coordinates of the center of mass of the small holes are extracted and matched one by one between the calibration plate images of the first field of view and the second field of view, and the conversion matrix is ​​directly calculated through multiple points. The more points there are, the higher the accuracy will be. Compared with the conventional calibration plate conversion method, the splicing can be completed through only one transformation, and there is no loss of accuracy and speed in the middle.

[0063] In order to solve the problem of convenience, the present application proposes a coordinate pairing method, which can automatically sort and pair the extracted circular hole centroids without the need for manual matching by operators. This method is simpler than conventional calibration plate conversion methods.

[0064] In order to solve the robustness problem, this application adopts multiple modes to select and process the overlapping areas of the splicing, which can overcome the influence of lighting to a certain extent and make the splicing boundaries more natural.

[0065] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0066] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0067] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0068] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, the aforementioned program can be stored in a readable storage medium, and when the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0069] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. An image stitching method, characterized in that: The method comprises: Acquire calibration plate images of the first field of view and the second field of view, wherein the calibration plate images are composed of a plurality of black rectangular markers with ordered circular holes distributed inside; Performing a relational transformation on the calibration plate images of the first field of view and the second field of view, obtaining a homography matrix between the first field of view and the second field of view images as a transformation matrix between the two stitched images; Acquire the same viewing angle as the calibration plate image to obtain a stitched image of the first field of view and a second field of view, which are respectively a first stitched image and a second stitched image; The first stitched image and the second stitched image are transformed by using a transformation matrix, and stitched together according to the transformation result to obtain a preliminary stitched image; According to the preliminary stitching image, respectively calculating a homography matrix H1 between the first stitching image and the preliminary stitching image and a homography matrix H2 between the second stitching image and the preliminary stitching image; The first stitched image and the second stitched image are transformed into a preliminary stitched image through the homography matrix H1 and the homography matrix H2 respectively, and the transformed overlapping area is pixel-fused to obtain a stitched image.

2. The image stitching method according to claim 1, characterized in that: The performing relationship conversion on the calibration plate images of the first field of view and the second field of view to obtain a homography matrix between the first field of view and the second field of view images specifically includes: Performing threshold segmentation on the calibration plate image to obtain a first brightness area and a second brightness area of ​​the calibration plate image, wherein a brightness value of the second brightness area is higher than a brightness value of the first brightness area; Taking the connected domain of the first brightness area as the dark connected domain, performing hole filling on the dark connected domain of the calibration plate image; Filter the dark connected domains by area, and retain the dark connected domains whose area is larger than the set threshold; The connected domain with the highest rectangularity is selected from the dark connected domains whose area is larger than the set threshold, which is the marker; Extract the minimum bounding box of the marker and obtain the two corner points on the upper edge of the minimum bounding box of the marker; Calculate the straight line function of the upper edge based on the two corner points of the upper edge of the minimum bounding box; Interference shielding is performed on the second brightness area to obtain the bright connected domain inside the calibration plate image; Filter the connected domains of the circular holes according to their areas, and calculate the centroid of each connected domain of the circular holes to obtain the circular hole centroid set Set1; The vertical distance from the center of mass of each circular hole to the upper edge of the marker is calculated according to the straight line function of the upper edge, and the center of mass set of circular holes Set1 is sorted in ascending order according to the vertical distance to obtain the center of mass set of circular holes Set2; The circular hole mass center set Set2 is locally sorted according to the horizontal coordinate from small to large, and the final circular hole mass center set Set is obtained; Extract the circular hole coordinates from the calibration plate images of the first field of view and the second field of view, and obtain two sets of circular hole centroids that correspond to each other in sequence; The homography matrix H between the calibration plate image of the first field of view and the calibration plate image of the second field of view is calculated and saved through two one-to-one corresponding circular hole centroid sets.

3. The image stitching method according to claim 2, characterized in that: The connected domain of the first brightness area of ​​the calibration plate image is the connected white area in the binary image of the calibration plate image.

4. The image stitching method according to claim 3, characterized in that: The calculation formula of the rectangularity is: , Among them, y is the rectangularity, is the minimum bounding box area, is the pixel area of ​​the connected domain of the marker.

5. The image stitching method according to claim 4, characterized in that: The interference shielding is specifically as follows: extracting the marker binary image and the second brightness area binary image, mapping the marker binary image to the second brightness area binary image, and setting the grayscale value of the area in the second brightness area binary image corresponding to the black pixel in the marker binary image to 255.

6. The image stitching method according to claim 5, characterized in that: The circular hole mass center set Set2 is locally sorted according to the horizontal coordinate from small to large, and the local sorting rule is: Determine whether the upper edge of the minimum bounding box is a long edge or a short edge, and determine the number of circular holes in each row of the upper edge of the minimum bounding box; Assume that the number of circular holes in each row on the upper edge of the minimum bounding box is X, and traverse the circular hole centroid set Set2 according to the number of circular holes, and sort every X holes in order from small to large.

7. The image stitching method according to claim 3, characterized in that: According to the preliminary stitching image, the homography matrix H1 from the first stitching image to the preliminary stitching image and the homography matrix H2 from the second stitching image to the preliminary stitching image are respectively calculated, specifically: Calculate the corner points of the first and second stitched images: Assume the height of the first stitched image is , the width is ; The height of the second stitched image is , the width is ; Then the four corner points of the first stitched image are , , , ; The four corner points of the second stitched image are , , , ; The corner point coordinates of the second stitched image are transformed into the coordinate system of the first stitched image through the homography matrix H. The transformation formula is: P"=P'*H. Then P" is normalized to obtain the new four corner point coordinates. , , , ; Calculating the size of the initial stitched image after stitching by using the coordinates of the four corner points of the first stitched image and the coordinates of the four corner points of the second stitched image after conversion; The width of the initial stitched image is: , The height of the initial stitched image is: , Calculate the corresponding coordinates of the corner points of the first stitched image and the corner points of the second stitched image in the preliminary stitched image: The corner points of the first stitched image and the corner points of the second stitched image are subtracted , which are the corresponding coordinates in the preliminary stitching image; in, , , The homography matrix H1 between the first stitched image and the preliminary stitched image and the homography matrix H2 between the second stitched image and the preliminary stitched image are calculated respectively by using the corresponding image corner point coordinates before and after the conversion.

8. The image stitching method according to claim 3, characterized in that: The pixel fusion is performed on the transformed overlapping area, and the fusion mode includes: Mean fusion: take the mean of corresponding pixels in the first stitched image and the second stitched image; Maximum fusion: the corresponding pixels in the first stitched image and the second stitched image take the maximum value; Minimum value fusion: the corresponding pixels in the first stitched image and the second stitched image take the minimum value; End-to-end fusion: The overlapping area directly covers the second stitched image.

9. An image stitching system, characterized in that: The invention comprises a memory and a processor, wherein the memory comprises an image stitching method program, and when the image stitching method program is executed by the processor, the steps of an image stitching method as claimed in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an image stitching method program. When the image stitching method program is executed by a processor, the steps of an image stitching method according to any one of claims 1 to 8 are implemented.

Images