UAV image stitching method and device based on seam line and semi-projection shape preservation

By using suture and semi-projection conformal algorithms in drone image stitching, the problems of large parallax and shape distortion are solved, high-quality image stitching is achieved, and the stitching efficiency and visual effect are improved.

CN114283102BActive Publication Date: 2024-12-03CHINA UNIV OF GEOSCIENCES (WUHAN)
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111447842.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-12-03
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

When existing drone image stitching algorithms deal with large parallax or moving objects, they are prone to ghost misalignment, distortion and shape distortion problems. The suture search algorithm is complex and fails to effectively combine constraints, resulting in poor quality of stitching images. high.

Method used

A method of image stitching based on suture and semi-projection conformal is proposed. By defining the energy function, calculating the threshold of the suture, determining the search range, finding the suture with the smallest sum of energy, and combining semi-projection transformation to shape the image. Reduce parallax and shape distortion.

Benefits of technology

The splicing results without ghosting and small parallax are achieved, which improves the visual effect and quality of the splicing images, reduces the possibility of sutures passing through structural objects, and improves splicing efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114283102B_ABST
    Figure CN114283102B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for stitching drone images based on stitching lines and semi-projection conformal, the method comprising: obtaining two source images collected by drones and registering them to obtain two registered images, including a reference image and an image to be stitched; defining an energy function; under the condition that the stitching line can be searched, calculating the threshold of the energy function; determining the search range of the stitching line according to the threshold; within the search range, obtaining the minimum energy sum from the start point to the end point, and taking the path with the minimum energy sum as the best stitching line; cutting the image to be stitched according to the best stitching line to obtain a partial source image; taking the partial source image as a new source image, and performing a semi-projection transformation with the reference image to obtain a transformed image; cutting and reorganizing the transformed image according to the best stitching line to obtain a stitched image. The method of the present invention can produce a stitching result without ghosting and small parallax, and the non-overlapping area can keep more image details of the source image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and a device for stitching unmanned aerial vehicle images based on seam lines and semi-projection conformal. Background Art

[0002] As a new image acquisition method, drone images fill the gap in aerial photogrammetry. Compared with manned aerial surveying, drones have lower flight altitudes, lower costs, and higher flexibility. Therefore, drones have been widely used in remote sensing applications, such as photogrammetry, urban construction planning, and unmanned patrols. Due to the low flight altitude and small coverage of drones, it is difficult to capture the full picture of the area to be detected. Therefore, the application demand for drone image stitching is of great significance.

[0003] There are three main image stitching methods: global homography-based registration, local homography-based registration, and seam-based stitching. The first two have always been the mainstream algorithms, and they both achieve the purpose of stitching by registering the two images based on their geometric relationship. However, due to the influence of the drone acquisition environment and the rotation of the camera lens, adjacent drone images will have parallax or shooting angles that differ too much. The corresponding stitched panoramas of the first two algorithms will have ghosting, dislocation, distortion, and other phenomena.

[0004] The seam stitching algorithm is a method for image stitching that solves problems with large parallax or moving objects. It finds a seam in the overlapping area of ​​two images and cuts and reassembles the two images. Compared with the previous two algorithms, the seam stitching algorithm has two advantages: (1) This method does not rely on precise registration and alignment, which greatly reduces the image registration time. (2) The stitched images are all directly derived from the source images, so there will be no ghosting problem, and it can also solve the impact of moving objects.

[0005] The existing seam stitching algorithms have made some progress, but there are still some areas for improvement. First, the route of the seam depends largely on the corresponding constraints. The existing seam algorithms do not address issues such as whether the seam will pass through some structural objects, resulting in the two parts of some objects in the final stitched image coming from different source images, greatly increasing the possibility of parallax. Secondly, the existing seam finding algorithm is too complicated and does not combine the constraints well for seam finding. Thirdly, the seam algorithm only divides and reassembles the two registered images. The shape distortion problem of the registered image still exists in the stitched image, and it is impossible to obtain a stitched image with good visual effects. Summary of the invention

[0006] In order to solve the above-mentioned technical problems existing in the existing seam stitching algorithm, the technical solution adopted by the present invention is to provide a new seam stitching algorithm, and combine it with the semi-projection conformal algorithm to perform shape correction on the stitched image, thereby obtaining a drone image stitching method and device based on seam and semi-projection conformal.

[0007] According to one aspect of the present invention, a method for stitching drone images based on seam lines and half-projection conformal is provided, comprising the following steps:

[0008] Acquire two source images captured by a drone, and register them to obtain two registered images, wherein the registered images include a reference image and an image to be stitched;

[0009] Define the energy function;

[0010] Under the condition that the suture line can be found, a threshold value of the energy function is calculated;

[0011] Determine the search range of the suture line according to the threshold value;

[0012] In the search range, the minimum energy sum from the starting point to the end point is obtained, and the path with the minimum energy sum is taken as the optimal suture line;

[0013] Cutting the image to be stitched according to the optimal stitching line to obtain a partial source image;

[0014] Taking the partial source image as a new source image, and performing a half-projection transformation with the reference image to obtain a transformed image;

[0015] The transformed image is cut and reassembled according to the optimal stitching line to obtain a spliced ​​image.

[0016] Preferably, the energy function includes: color difference, structure difference and line segment difference.

[0017] Preferably, the calculation formula for the color difference is:

[0018]

[0019]

[0020] R=C 1,R -C 2,R

[0021] G=C 1,G -C 2,G

[0022] B=C 1,B -C 2,B

[0023] Among them, C1,R , C 2,R Represents the values ​​of the two images in the R channel, C 1,G , C 2,G Represents the values ​​of the two images in the G channel, C 1,B , C 2,B represents the value of the two images in the B channel, r represents the average value of the two images in the R channel, and R, G, and B represent the difference between the two images in the R, G, and B channels respectively.

[0024] Preferably, the step of obtaining the minimum energy sum from the starting point to the end point within the search range and taking the path with the minimum energy sum as the best suture line includes:

[0025] Within the search range, starting from the starting point, the energy values ​​of the pixels adjacent to the starting point are updated, and the updated energy value is the minimum energy sum of the path from the pixel point to the starting point, and so on, continuously updated to the end point, to obtain the minimum energy sum from the end point to the starting point;

[0026] Then, starting from the end point, move forward along the direction of the point with the smallest energy value in the eight adjacent directions until it reaches the starting point. The path of advancement is the route of the optimal suture line.

[0027] Preferably, the step of using part of the source image as a new source image and performing a half-projection transformation with the reference image to obtain a transformed image comprises:

[0028] For the new source image and the reference image, a projection transformation is performed in the overlapping area, and a similarity transformation is performed in the non-overlapping area. There is a transition area between the two transformations for guiding the image to transition from the projection transformation to the similarity transformation.

[0029] Preferably, after the step of cutting and reassembling the transformed image according to the optimal stitching line to obtain a spliced ​​image, the method further includes:

[0030] A quantitative index is selected to evaluate the quality of the stitched image.

[0031] Preferably, the calculation formula of the quantitative index is:

[0032]

[0033] Where N is the number of pixels on the seam line, p i The SSIM index is used to measure the difference between the current pixel point in the two images. seam The lower the value, the smaller the pixel difference on the stitching line and the higher the quality.

[0034] According to another aspect of the present invention, a drone image stitching device based on seam lines and semi-projection conformal is provided, comprising the following modules:

[0035] An image acquisition and registration module is used to acquire two source images collected by a drone and register them to obtain two registered images, wherein the registered images include a reference image and an image to be stitched;

[0036] Energy function definition module, used to define energy function;

[0037] An energy threshold calculation module, used to calculate the threshold of the energy function under the condition that the suture line can be found;

[0038] A search range determination module, used to determine the search range of the suture line according to the threshold value;

[0039] An optimal suture line acquisition module is used to acquire the minimum energy sum from the starting point to the end point within the search range, and take the path with the minimum energy sum as the optimal suture line;

[0040] A cutting module, used for cutting the image to be stitched according to the optimal stitching line to obtain a partial source image;

[0041] A half-projection transformation module is used to use the part of the source image as a new source image and perform a half-projection transformation on the reference image to obtain a transformed image;

[0042] The stitching module is used to cut and reassemble the transformed image according to the optimal stitching line to obtain a stitched image.

[0043] The technical solution provided by the present invention has the following beneficial effects:

[0044] The present invention proposes a new seam stitching algorithm process, and combines the semi-projection conformal algorithm to correct the shape of the stitched image. In order to obtain the difference in the overlapping area of ​​the two images, a new energy function is constructed, which is composed of color difference, structure difference and line segment difference. In order to increase the constraints on the energy function, an energy threshold is calculated, and the energy function is constrained under the condition that the seam can be searched, so as to increase the fault tolerance of the seam search. In combination with the above constraints, a seam search method based on the minimum total energy is proposed. After that, the image is cut and reorganized to obtain a preliminary stitched image. In order to solve the problem of shape distortion, a semi-projection deformation alignment is performed on the source image in combination with the image information after cutting, so that more areas can maintain the shape of the source image and improve the visual effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0046] Figure 1 It is an overall framework diagram of a UAV image stitching method based on stitching lines and semi-projection conformal of the present invention;

[0047] Figure 2 It is a flow chart of a method for stitching drone images based on seam lines and semi-projection conformal according to the present invention;

[0048] Figure 3 It is the energy function definition diagram of the present invention;

[0049] Figure 4 It is a definition diagram of the structural differences of the present invention;

[0050] Figure 5 is the energy function constraint diagram of the present invention;

[0051] Figure 6 is a schematic diagram of suture line search of the present invention;

[0052] Figure 7 is a schematic diagram of a suture cutting image of the present invention;

[0053] Figure 8 It is a schematic diagram of half-projection transformation of the present invention;

[0054] Fig. 9 It is a flow chart of half-projection transformation of the present invention;

[0055] Fig.10 It is a quantitative experimental result diagram of the present invention;

[0056] Fig.11 It is a structural diagram of a UAV image stitching device based on stitching lines and semi-projection conformal according to the present invention. DETAILED DESCRIPTION

[0057] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0058] The key to the stitching method is how to find a stitching line with the smallest difference between the two registered images and try not to pass through the structural objects. Specifically, it is to find the stitching line based on the difference information between the overlapping parts of the two registered images. However, most existing methods do not constrain the difference information of the endpoint method, resulting in the stitching line passing through some unfriendly places. In view of this, the overall framework of the method of the present invention is as follows Figure 1As shown. It mainly consists of two steps: seam stitching (SeamCut) and half-projection conformal (Half_projective). Given two source images, the role of SeamCut is to align and cut the two source images. According to the information after cutting, half-projective is used to perform semi-conformal projection deformation to reduce the shape distortion of objects in non-overlapping areas. Through the above steps, a stitching result with no ghosting and small parallax can be produced. In addition, the non-overlapping area can maintain more image details of the source image.

[0059] like Figure 2 As shown, the present invention provides a method for stitching drone images based on seam lines and semi-projection conformal, comprising the following steps:

[0060] S1: Acquire two source images captured by a drone, and register them to obtain two registered images, wherein the registered images include a reference image and an image to be stitched;

[0061] S2: define energy function;

[0062] The first step in seam stitching is to define an energy function, which measures the difference between the overlapping parts of the two registered images and is used for subsequent seam finding. Figure 2 As shown in FIG. 1 , the energy function proposed in the present invention mainly consists of three differences: color difference, structure difference and line segment difference. Through the superposition of three different differences, it is effective for the difference task of accurately defining the overlapping part.

[0063] Color Difference color Used to measure the color similarity of the overlapping parts of two images:

[0064]

[0065] R=C 1,R -C 2,R (2)

[0066] G=C 1,G -C 2,G (3)

[0067] B=C 1,B -C 2,B (4)

[0068]

[0069] Among them, C 1,R , C 2,R Represents the values ​​of the two images in the R channel, C 1,G , C 2,G Represents the values ​​of the two images in the G channel, C 1,B , C 2,Brepresents the value of the two images in the B channel, r represents the average value of the two images in the R channel, and R, G, and B represent the difference between the two images in the R, G, and B channels respectively.

[0070] Defining structural differences gradient When suturing, we must consider what factors the suture route must meet. In addition to preferably passing through the area with the smallest structural difference, we must also consider allowing the suture to avoid passing through structural objects and let the suture pass through flat areas as much as possible. Flat areas are mostly areas with small gradient differences. Therefore, the registration image can be filtered to reduce the information constraints of flat areas in structural differences and guide the suture to search in flat areas. The specific process of structural differences is as follows: Figure 4 As shown in the figure, the following operations are performed on the two registered images: the image is Gaussian filtered once, and the filtered image is converted into a grayscale image; the grayscale image is Gaussian filtered with different parameters to obtain two different filtered images and subtract them to obtain the structural information image of the image; finally, the structural information images corresponding to the two registered images are subtracted to obtain the final structural difference.

[0071] Structural objects also contain a large amount of line segment information. In order to better ensure that the suture line does not pass through the structural object, the LSD line segment detection algorithm is used in this embodiment to detect the line segment information of the overlapping images, and the difference E of the line segment information is calculated. line is also superimposed on the energy function.

[0072] Therefore, the energy function E = E color +E gradient +E line .

[0073] S3: Under the condition that the suture line can be found, the threshold of the energy function is calculated;

[0074] S4: determining a search range of a suture line according to the threshold value;

[0075] After defining the energy function, the energy function is constrained so that all pixels on a complete seam line are in the same eight-connected region. An eight-connected region means that starting from each pixel in the region, you can reach any pixel in the region without going out of the region by moving in a combination of eight directions, namely up, down, left, right, upper left, upper right, lower left, and lower right. The starting point and end point of the seam line are at the intersection of the two images. Therefore, under the condition that the starting point and the end point are in the same eight-connected region, that is, they can be connected to form an uninterrupted seam line, a threshold e of the energy function is calculated, and energy regions greater than this threshold are not included in the search range of the seam line, and other pixels that are not in the eight-connected region where the starting point and the end point are located are also excluded from the search range of the seam line. Figure 5As shown, by constraining the energy function, the difference information on the energy function is richer, which can increase the error tolerance of subsequent suture line search.

[0076] S5: within the search range, obtaining the minimum energy sum from the start point to the end point, and taking the path with the minimum energy sum as the best suture line;

[0077] After defining the energy function, we get the positions of the start and end points, as well as a threshold value e of the energy function. All energy values ​​in the search area of ​​the seam line on the energy function will be less than the threshold value. The smaller the energy value, the smaller the difference between the two images. Therefore, combined with the defined energy function, a seam line search method based on the minimum sum of capabilities is designed. The central idea is to minimize the sum of energy values ​​corresponding to all pixels on the seam line.

[0078] Specifically, within the search range, starting from the starting point, the energy value of the pixel adjacent to the starting point is updated, and the updated energy value is the minimum energy sum of the path from the pixel point to the starting point, and so on, continuously updated to the end point, to obtain the minimum energy sum from the end point to the starting point;

[0079] Then, starting from the end point, move forward along the direction of the point with the smallest energy value in the eight adjacent directions until it reaches the starting point. The path of advancement is the route of the optimal suture line, such as Figure 6 shown.

[0080] S6: cutting the image to be stitched according to the optimal stitching line to obtain a partial source image; Figure 7 shown.

[0081] S7: taking the partial source image as a new source image and performing a half-projection transformation with the reference image to obtain a transformed image;

[0082] When the source images are registered, one of the images will be projected according to the geometric relationship of the other image, and a certain shape deformation will occur in the overlapping area of ​​the two images. If the viewing angles of the two images themselves are very different, the degree of deformation will be even greater. After searching and cutting the image through the stitching line, a part of the source image to be spliced ​​can be obtained. This part is used as the new source image and a semi-projection transformation is performed with another complete reference source image. The central idea of ​​the semi-projection transformation is to perform a projection transformation on the overlapping area of ​​the two images and a similarity transformation on the non-overlapping area. The similarity transformation will only change the size and direction of the image, and will not change the shape of the image. There is also a transition area between the two transformations, which guides the image to transition from the projection transformation to the similarity transformation, such as Figure 8 shown.

[0083] First, rotate the coordinate system (x, y) to form new coordinates (u, v):

[0084]

[0085] The corresponding homogeneous coordinate transformation is as follows:

[0086]

[0087]

[0088]

[0089] where θ is the rotation angle, and the rotated coordinates (u, v) are transformed into (x', y') using the homography matrix. is the homography matrix, and h1 to h6 are all parameters of the homography matrix. Formulas (8) and (9) represent the relationships between x' and y' and (u, v) after transformation. c is a constant. If u = u0 is a constant, then H x (u0, v) and H y (u0, v) are linear functions of v. Therefore, the parameters u1 and u2 can be determined, and the image is divided into R H = {(u, v)|u ≤ u1}, R T = {(u, v)|u1 < u < u2}, and R S = {(u, v)|u2 ≤ u}. For R H , projective transformation is adopted, for R S , similarity transformation is adopted, and R T serves as the transition region between the two transformations.

[0090] Next, the parameters u1 and u2 are determined through the following energy cost:

[0091]

[0092] where (x, y) ∈ Ω i , Ω i is the rectangular domain of the two images, J i (x, y; u1, u2) is the value of the Jacobian matrix of ω i at (x, y), ω i is the corresponding warping function of the two images composed of R H R T R S a i , b i are the coefficients of ω i . The minimum total energies E1(u1, u2) and E2(u1, u2) are obtained to derive the parameters u1 and u2.

[0093] S8: cutting and reassembling the transformed image according to the optimal stitching line to obtain a spliced ​​image.

[0094] Perform half-projection transformation on the image to obtain the transformed image, and then cut and reassemble the transformed image according to the position of the best stitching line to finally obtain the spliced ​​result. The overall process is as follows: Fig. 9 shown.

[0095] In this embodiment, a method using half-projection transformation is qualitatively compared with a method not using half-projection transformation. The results are as follows: Fig.10 As shown, it can be seen that the use of the half-projection transformation method retains more original information of the source image, solves the distortion problem that occurs on some objects, and obtains better visual effects.

[0096] In order to verify the beneficial effects of the UAV image stitching method based on seam lines and semi-projection conformal of the present invention, in this embodiment, quantitative indicators are also selected to evaluate the quality of the stitched images.

[0097] The calculation formula of quantitative indicators is:

[0098]

[0099] Where N is the number of pixels on the seam line, p i The SSIM index is used to measure the difference between the current pixel point in the two images. seam The lower the value, the smaller the pixel difference on the stitching line and the higher the quality.

[0100] Five groups of experimental results were selected to compare the suture quality with different suture methods SEAGULL and Seam-Driven. The quantitative experimental results are shown in Table 1. The quantitative index Q of the method of the present invention is higher than that of other methods. seam The value of is lower, indicating that the difference in the paths of the sutures of the present invention is smaller, the quality of the sutures is higher, and the quality of the spliced ​​images is better.

[0101] Table 1 Quantitative experimental comparison

[0102]

[0103] like Fig.11 As shown, this embodiment also provides a UAV image stitching device based on stitching lines and semi-projection conformal, including the following modules:

[0104] The image acquisition and registration module 1 is used to acquire two source images collected by the drone and register them to obtain two registered images, wherein the registered images include a reference image and an image to be spliced;

[0105] Energy function definition module 2, used to define energy function;

[0106] An energy threshold calculation module 3, used to calculate the threshold of the energy function under the condition that the suture line can be found;

[0107] A search range determination module 4, used to determine the search range of the suture line according to the threshold value;

[0108] An optimal suture line acquisition module 5 is used to acquire the minimum energy sum from the starting point to the end point within the search range, and take the path with the minimum energy sum as the optimal suture line;

[0109] A cutting module 6, used for cutting the image to be stitched according to the optimal stitching line to obtain a partial source image;

[0110] A half-projection transformation module 7 is used to use the part of the source image as a new source image and perform a half-projection transformation with the reference image to obtain a transformed image;

[0111] The stitching module 8 is used to cut and reassemble the transformed image according to the optimal stitching line to obtain a stitched image.

[0112] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0113] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0114] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A UAV image stitching method based on seam lines and half-projection conformal, characterized in that: The following steps are involved: Acquire two source images captured by a drone, and register them to obtain two registered images, wherein the registered images include a reference image and an image to be stitched; Define the energy function; Under the condition that the suture line can be found, a threshold value of the energy function is calculated; Determine the search range of the suture line according to the threshold value; In the search range, the minimum energy sum from the starting point to the end point is obtained, and the path with the minimum energy sum is taken as the optimal suture line; Cutting the image to be stitched according to the optimal stitching line to obtain a partial source image; Taking the partial source image as a new source image, and performing a half-projection transformation with the reference image to obtain a transformed image; Cutting and reassembling the transformed image according to the optimal stitching line to obtain a spliced ​​image; The energy function includes: color difference, structure difference and line segment difference; The calculation formula of the color difference is: R=C 1,R -C 2,R G=C 1,G -C 2,G B=C 1,B -C 2,B Among them, C 1,R , C 2,R Represents the values ​​of the two images in the R channel, C 1,G , C 2,G Represents the values ​​of the two images in the G channel, C 1,B , C 2,B represents the value of the two images in the B channel, r represents the average value of the two images in the R channel, and R, G, and B represent the difference between the two images in the R, G, and B channels respectively; The step of using part of the source image as a new source image and performing a half-projection transformation with the reference image to obtain a transformed image comprises: For the new source image and the reference image, a projection transformation is performed in the overlapping area, and a similarity transformation is performed in the non-overlapping area. There is a transition area between the two transformations for guiding the image to transition from the projection transformation to the similarity transformation.

2. The method for stitching drone images based on seam lines and half-projection conformal according to claim 1, characterized in that: The step of obtaining the minimum energy sum from the starting point to the end point within the search range and taking the path with the minimum energy sum as the best suture line comprises: Within the search range, starting from the starting point, the energy values ​​of the pixels adjacent to the starting point are updated, and the updated energy value is the minimum energy sum of the path from the pixel point to the starting point, and so on, continuously updated to the end point to obtain the minimum energy sum from the end point to the starting point; Then, starting from the end point, move forward along the direction of the point with the smallest energy value in the eight adjacent directions until it reaches the starting point. The path of advancement is the route of the optimal suture line.

3. The method for stitching drone images based on seam lines and half-projection conformal according to claim 1, characterized in that: After the step of cutting and reassembling the transformed image according to the optimal stitching line to obtain a spliced ​​image, the method further includes: A quantitative index is selected to evaluate the quality of the stitched image.

4. The method for stitching drone images based on seam lines and half-projection conformal according to claim 3, characterized in that: The calculation formula of the quantitative index is: Where N is the number of pixels on the seam line, p i The SSIM index is used to measure the difference between the current pixel point in the two images. seam The lower the value, the smaller the pixel difference on the stitching line and the higher the quality.

5. A drone image stitching device based on seam lines and semi-projection conformal, characterized in that: Includes the following modules: An image acquisition and registration module is used to acquire two source images collected by a drone and register them to obtain two registered images, wherein the registered images include a reference image and an image to be stitched; Energy function definition module, used to define energy function; An energy threshold calculation module, used to calculate the threshold of the energy function under the condition that the suture line can be found; A search range determination module, used to determine the search range of the suture line according to the threshold value; An optimal suture line acquisition module is used to acquire the minimum energy sum from the starting point to the end point within the search range, and take the path with the minimum energy sum as the optimal suture line; A cutting module, used for cutting the image to be stitched according to the optimal stitching line to obtain a partial source image; A half-projection transformation module is used to use the part of the source image as a new source image and perform a half-projection transformation on the reference image to obtain a transformed image; A stitching module, used for cutting and reassembling the transformed image according to the optimal stitching line to obtain a stitched image; The energy function includes: color difference, structure difference and line segment difference; The calculation formula of the color difference is: R=C 1,R -C 2,R G=C 1,G -C 2,G B=C 1,B -C 2,B Among them, C 1,R , C 2,R Represents the values ​​of the two images in the R channel, C 1,G , C 2,G Represents the values ​​of the two images in the G channel, C 1,B , C 2,B represents the value of the two images in the B channel, r represents the average value of the two images in the R channel, and R, G, and B represent the difference between the two images in the R, G, and B channels respectively; The step of using part of the source image as a new source image and performing a half-projection transformation with the reference image to obtain a transformed image comprises: For the new source image and the reference image, a projection transformation is performed in the overlapping area, and a similarity transformation is performed in the non-overlapping area. There is a transition area between the two transformations for guiding the image to transition from the projection transformation to the similarity transformation.

Citation Information

Patent Citations

  • Unmanned aerial vehicle aerial photographed image splicing method based on suture line

    CN107146213A

  • Adjacent image splicing method based on elastic registration and grid optimization

    CN112862683A

  • Panoramic image splicing method based on optimal suture line

    CN112991176A

  • Method for ensuring perfect stitching of a subject's images in a real-site image stitching operation

    US20180061006A1