Image splicing method and system, electronic equipment and storage medium

Abstract

The invention discloses an image stitching method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining an image test set which comprises a plurality of to-be-stitched images; performing feature matching on the plurality of to-be-spliced images to obtain a plurality of groups of feature points; determining a homography matrix based on the plurality of groups of feature points, and performing structure transformation on all the to-be-spliced images through the homography matrix to obtain a plurality of mapping images; splicing the plurality of mapping images to obtain a spliced image, and determining an overlapping region in the spliced image; carrying out region division on the overlapping region, and determining splicing gaps for dividing pixel points; and eliminating the splicing gap based on a preset image fusion algorithm to obtain a fused image. In the embodiment of the invention, the splicing precision can be improved, and the situation of ghosting or dislocation after image splicing can be avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image stitching method, system, electronic device and storage medium. Background Art

[0002] Images are the most direct way for humans to obtain information, and image stitching technology is an important topic in various types of image processing research. In research projects or engineering sciences, images with ultra-wide viewing angles and high resolutions are often required. In order to obtain panoramic images with a large field of view and high resolution, expensive and complex wide-angle lenses and scanning cameras are often required. Even so, the images obtained still have distortion and distortion caused by wide-angle lenses.

[0003] To address the above problems, image stitching technology has emerged. The essence of image stitching is to stitch multiple images with overlapping parts into an image with a wider viewing angle and higher resolution after image registration and other processing. However, existing image stitching technology often has a high error rate, resulting in ghosting and misalignment in the stitched image, resulting in low stitching accuracy and affecting the visual effect of the final image. Summary of the invention

[0004] The embodiments of the present application provide an image stitching method, system, electronic device and storage medium, which can improve the stitching accuracy and avoid ghosting or misalignment after image stitching.

[0005] In a first aspect, an embodiment of the present application provides an image stitching method, the method comprising:

[0006] Acquire an image test set, wherein the image test set includes a plurality of images to be stitched;

[0007] Performing feature matching on the plurality of images to be stitched to obtain a plurality of groups of feature points;

[0008] Determine a homography matrix based on the multiple groups of feature points, and perform structural transformation on all the images to be stitched using the homography matrix to obtain multiple mapping images;

[0009] splicing a plurality of the mapping images to obtain a spliced ​​image, and determining an overlapping area in the spliced ​​image;

[0010] Divide the overlapping area into regions to determine the splicing gaps of divided pixel points;

[0011] The stitching gap is eliminated based on a preset image fusion algorithm to obtain a fused image.

[0012] The image stitching method provided by the embodiment of the present application has at least the following beneficial effects: first, an image test set including multiple images to be stitched is obtained, and then feature matching is performed on the multiple images to be stitched, so that multi-level features in the image can be captured, the richness and diversity of the features are enhanced, and multiple groups of feature points are obtained. Then, a homography matrix is ​​determined based on the multiple groups of feature points, and structural transformation is performed on all the images to be stitched through the homography matrix, so as to reduce geometric distortion during image stitching, and further reduce stitching traces to obtain multiple mapped images. Then, the multiple mapped images are stitched to obtain a stitched image, and overlapping areas in the stitched image are determined to facilitate subsequent determination of pixel values ​​of the overlapping areas. Then, the overlapping areas are divided into regions to determine the stitching gaps of the divided pixels, so that the secant lines of the divided pixels can be found. Finally, the stitching gaps are eliminated based on a preset image fusion algorithm to obtain a fused image. By eliminating the stitching gaps, the stitched image is visually more natural and coherent, without obvious seams or unnatural transition traces, thereby improving the overall quality of the stitched image. The embodiment of the present application first performs structural transformation on the image to be stitched through the homography matrix, so as to reduce the geometric distortion during image stitching, provide a basis for subsequent image stitching, make the stitched image more natural, and then determine the stitching gap in the overlapping area to achieve accurate determination of the stitching gap. After finding the stitching gap, the embodiment of the present application uses an image fusion algorithm to eliminate gaps, ghosting, etc., improve stitching accuracy, and further improve image quality.

[0013] In some embodiments, after performing structural transformation on all the images to be stitched using the homography matrix to obtain a plurality of mapping images, the method further includes:

[0014] Selecting a first mapping image and a second mapping image from the stitched image, cutting the first mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of first sub-regions, and cutting the second mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of second sub-regions;

[0015] Extracting first feature points of the first sub-region and second feature points of the second sub-region;

[0016] Performing feature point matching on the first feature point and the second feature point to obtain a plurality of test point pairs;

[0017] Screening all the test point pairs through a preset iterative algorithm to obtain a target point pair set;

[0018] Performing matrix estimation on the target point pair set by using a preset direct linear transformation algorithm to obtain a perspective transformation matrix;

[0019] Distortion correction is performed on the stitched image according to the perspective transformation matrix to align feature points of the stitched image.

[0020] In some embodiments, dividing the overlapping area to determine the stitching gap of the divided pixel points includes:

[0021] A test route is set in the overlapping area, and image analysis is performed on the overlapping area to determine a source node and a sink node in the stitched image, wherein the source node is a node used to represent a foreground area of ​​the stitched image, and the sink node is a node used to represent a background area of ​​the stitched image;

[0022] Selecting a first pixel point and a second pixel point in the stitched image, calculating a first eigenvalue of the first pixel point at the source node and a second eigenvalue of the first pixel point at the sink node, and calculating a third eigenvalue of the second pixel point at the source node and a fourth eigenvalue of the second pixel point at the sink node, wherein the first eigenvalue and the third eigenvalue are used to characterize visual features at the source node, and the second eigenvalue and the fourth eigenvalue are used to characterize visual features at the sink node;

[0023] Determine a first distance between the first pixel and the second pixel at the source node according to the first eigenvalue and the third eigenvalue, and determine a second distance between the first pixel and the second pixel at the sink node according to the second eigenvalue and the fourth eigenvalue;

[0024] Calculate a secant weight according to the first distance and the second distance;

[0025] The splicing gap of the divided pixel points is determined according to the cut line weight.

[0026] In some embodiments, the step of eliminating the stitching gap based on a preset image fusion algorithm to obtain a fused image includes:

[0027] Performing Gaussian filtering on the mapping image, and downsampling the Gaussian filtered mapping image to construct a Laplacian pyramid;

[0028] performing a difference operation on two adjacent layers of the Laplacian pyramid to decompose the mapped image into a plurality of sub-images of different sizes;

[0029] For each of the sub-graphs, a weighted average operation is performed on the sub-graph to obtain a fusion result;

[0030] Determining the number of pyramid levels of the Laplacian pyramid, and performing a reverse reconstruction operation on the Laplacian pyramid based on the number of pyramid levels to obtain a reverse pyramid;

[0031] Performing weight assignment on the stitched image, and performing an upsampling operation on the stitched image after weight assignment through the reverse pyramid to obtain a sampled image;

[0032] The sampled image and the fusion result are fused to obtain a fused image.

[0033] In some embodiments, performing feature matching on the plurality of images to be stitched to obtain a plurality of sets of feature points includes:

[0034] For each of the images to be stitched, searching for the image position of the image to be stitched in a multi-scale Gaussian scale space to obtain an extreme point in the scale space;

[0035] Filtering the extreme points in the scale space by a preset three-dimensional quadratic function to determine a plurality of key points of the image to be stitched;

[0036] Assigning directions to all the key points, and describing the features of the key points after the direction assignment to obtain description information;

[0037] Based on the description information, feature matching is performed on all the images to be spliced ​​to obtain multiple groups of feature points.

[0038] In some embodiments, determining a homography matrix based on the multiple groups of feature points includes:

[0039] The feature points are screened by a preset iterative algorithm to obtain multiple pairs of matching points;

[0040] Calculating the matching points according to a preset direct linear transformation algorithm to obtain a test homography matrix;

[0041] Calculating the mapping errors of all the matching points by using the test homography matrix;

[0042] Comparing the mapping error with a preset error threshold to screen out all matching point pairs whose mapping error is greater than the error threshold to obtain an inlier set;

[0043] The test homography matrix is ​​updated according to the interior point set to obtain a homography matrix.

[0044] In some embodiments, performing structural transformation on all the images to be stitched by using the homography matrix to obtain multiple mapping images includes:

[0045] Determine a pixel coordinate set of all pixels on the image to be stitched;

[0046] Performing coordinate transformation on all pixel point coordinates in the pixel point coordinate set by using the homography matrix to obtain a target coordinate set;

[0047] Performing interpolation processing on the image to be stitched according to the target coordinate set to determine a target pixel set corresponding to the target coordinate set;

[0048] The target pixel set and the target coordinate set are fused to obtain a plurality of mapping images.

[0049] In a second aspect, an embodiment of the present application further provides an image stitching system, the image stitching system comprising:

[0050] A data set acquisition module, used to acquire an image test set, wherein the image test set includes a plurality of images to be stitched;

[0051] A feature matching module is used to perform feature matching on the multiple images to be stitched to obtain multiple groups of feature points;

[0052] A structural transformation module, used for determining a homography matrix based on the plurality of groups of feature points, and performing structural transformation on all the images to be stitched by using the homography matrix to obtain a plurality of mapping images;

[0053] An image stitching module, used for stitching a plurality of the mapping images to obtain a stitched image, and determining an overlapping area in the stitched image;

[0054] A region division module, used to divide the overlapping region into regions and determine the splicing gaps of the divided pixel points;

[0055] The gap elimination module is used to eliminate the stitching gap based on a preset image fusion algorithm to obtain a fused image.

[0056] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image stitching method as described in the first aspect is implemented.

[0057] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image stitching method as described in the first aspect.

[0058] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the examples of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0060] Figure 1 is a specific flow chart of an image stitching method provided by an embodiment of the present application;

[0061] Figure 2 is a specific flow chart of an image stitching method provided by another embodiment of the present application;

[0062] Figure 3 is a specific flow chart of step S105 provided in an embodiment of the present application;

[0063] Figure 4 is a specific flow chart of step S106 provided in an embodiment of the present application;

[0064] Figure 5 is a specific flow chart of step S102 provided in an embodiment of the present application;

[0065] Figure 6 It is a specific flow chart of determining a homography matrix based on multiple groups of feature points provided in an embodiment of the present application;

[0066] Figure 7 It is a specific flow chart of performing structural transformation on all images to be stitched by using a homography matrix provided in an embodiment of the present application;

[0067] Figure 8 is a system schematic diagram of an image stitching system provided in an embodiment of the present application;

[0068] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0072] An image stitching method provided in an embodiment of the present application can be applied to a terminal, can be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms; the software can be an application that implements the above method, etc., but is not limited to the above forms.

[0073] Embodiments of the present application can be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0074] Images are the most direct way for humans to obtain information, and image stitching technology is an important topic in various types of image processing research. In research projects or engineering sciences, images with ultra-wide viewing angles and high resolutions are often required. In order to obtain panoramic images with a large field of view and high resolution, expensive and complex wide-angle lenses and scanning cameras are often required. Even so, the images obtained still have distortion and distortion caused by wide-angle lenses.

[0075] To address the above problems, image stitching technology has emerged. The essence of image stitching is to stitch multiple images with overlapping parts into an image with a wider viewing angle and higher resolution after image registration and other processing. However, existing image stitching technology often has a high error rate, resulting in ghosting and misalignment in the stitched image, resulting in low stitching accuracy and affecting the visual effect of the final image.

[0076] In order to solve the above problems, the embodiments of the present application provide an image stitching method, system, electronic device and storage medium. First, an image test set including multiple images to be stitched is obtained, and then feature matching is performed on the multiple images to be stitched, so that multi-level features in the image can be captured, the richness and diversity of the features are enhanced, and multiple groups of feature points are obtained. Then, a homography matrix is ​​determined based on the multiple groups of feature points, and the structure of all the images to be stitched is transformed through the homography matrix to reduce geometric distortion during image stitching, and further reduce stitching traces to obtain multiple mapped images. Then, the multiple mapped images are stitched to obtain a stitched image, and the overlapping area in the stitched image is determined to facilitate subsequent determination of the pixel value of the overlapping area. Then, the overlapping area is divided into regions to determine the stitching gap of the divided pixel points, so that the secant line of the divided pixel points can be found. Finally, the stitching gap is eliminated based on a preset image fusion algorithm to obtain a fused image. By eliminating the stitching gap, the stitched image is visually more natural and coherent, without obvious seams or unnatural transition traces, thereby improving the overall quality of the stitched image. The embodiment of the present application first performs structural transformation on the image to be stitched through the homography matrix, so as to reduce the geometric distortion during image stitching, provide a basis for subsequent image stitching, make the stitched image more natural, and then determine the stitching gap in the overlapping area to achieve accurate determination of the stitching gap. After finding the stitching gap, the embodiment of the present application uses an image fusion algorithm to eliminate gaps, ghosting, etc., improve stitching accuracy, and further improve image quality.

[0077] Reference Figure 1 , Figure 1 1 is a specific flow chart of the image stitching method provided in the embodiment of the present application. In some embodiments, the image stitching method includes but is not limited to steps S101 to S106.

[0078] Step S101, obtaining an image test set.

[0079] It should be noted that the image test set includes multiple images to be stitched.

[0080] In step S101 of some embodiments, an image test set is first obtained, and the image test set includes multiple images to be stitched, wherein the images to be stitched in the embodiments of the present application are obtained by image acquisition through an image acquisition device, and the image acquisition device includes but is not limited to a mobile phone, a camera, a handheld camera, etc.

[0081] Step S102: performing feature matching on a plurality of images to be stitched to obtain a plurality of groups of feature points.

[0082] In step S102 of some embodiments, feature matching is performed on multiple images to be stitched, and low-contrast key points and unstable edge response points are removed through feature matching, and multiple key points of the images to be stitched are determined, thereby enhancing matching stability and improving noise resistance, and obtaining multiple groups of feature points, thereby increasing the probability of correct matching of feature points.

[0083] Step S103: determining a homography matrix based on the multiple groups of feature points, and performing structural transformation on all the images to be spliced ​​using the homography matrix to obtain multiple mapping images.

[0084] In step S103 of some embodiments, a homography matrix is ​​determined based on multiple groups of feature points, so that the homography matrix can be used to improve the effect of image transformation and reduce image distortion. The homography matrix is ​​used to perform structural transformation on all images to be stitched to obtain multiple mapping images, thereby realizing image mapping transformation. The converted image is a mapping image suitable for stitching, thereby further ensuring the accuracy of image stitching.

[0085] Step S104: stitching the multiple mapping images to obtain a stitched image, and determining an overlapping area in the stitched image.

[0086] In step S104 of some embodiments, multiple mapping images are stitched to obtain a stitched image, and an overlapping area in the stitched image, ie, an overlapping area between two images, is determined to facilitate subsequent processing of pixels in the overlapping area.

[0087] Step S105, dividing the overlapping area into regions and determining the stitching gaps of the divided pixels.

[0088] In step S105 of some embodiments, the overlapping area is divided into regions, the stitching gaps of the divided pixels are determined, and the minimum cut in the image is found based on the obtained maximum flow, thereby achieving image segmentation and accurately finding the stitching gaps. By reasonably determining the segmentation line, the pixel values ​​of the overlapping area can be smoothly transitioned, the stitching marks can be reduced, and obvious seams can be avoided.

[0089] Step S106, performing an operation of eliminating the stitching gaps based on a preset image fusion algorithm to obtain a fused image.

[0090] In step S106 of some embodiments, the stitching gaps are eliminated based on a preset image fusion algorithm, which can effectively retain the details and structural information of the image, and on this basis ensure the natural transition and high resolution of the stitched image, thereby obtaining a better stitching effect and obtaining a fused image, so that the transition of the stitched image is natural and the visual effect is better.

[0091] Reference Figure 2 , Figure 2 FIG. 2 is a specific flow chart of an image stitching method provided by another embodiment of the present application. In some embodiments, the image stitching method includes but is not limited to steps S201 to S206.

[0092] It should be noted that steps S201 to S206 of the embodiment of the present application occur after structural transformation of all images to be stitched is performed through the homography matrix to obtain multiple mapping images.

[0093] Step S201, select a first mapping image and a second mapping image from the mapping image, cut the first mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of first sub-regions, and cut the second mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of second sub-regions.

[0094] Step S202: extracting first feature points of the first sub-region and second feature points of the second sub-region.

[0095] Step S203: performing feature point matching on the first feature point and the second feature point to obtain a plurality of test point pairs.

[0096] Step S204: Screen all test point pairs using a preset iterative algorithm to obtain a target point pair set.

[0097] Step S205 , performing matrix estimation on the target point pair set by using a preset direct linear transformation algorithm to obtain a perspective transformation matrix.

[0098] Step S206: performing distortion correction on the stitched image according to the perspective transformation matrix to align feature points of the stitched image.

[0099] In steps S201 to S206 of some embodiments, after obtaining multiple mapping images, during the process of image stitching and fusion, due to the influence of objective factors, the stitched and fused images may have problems such as ghosting and excessive discontinuity between images. In order to solve the above problems, the embodiment of the present application also performs image registration processing on the mapping images. Specifically, the embodiment of the present application randomly selects a first mapping image and a second mapping image from the mapping images, that is, two images prepared for stitching, and cuts the first mapping image through a preset adaptive phase adjustment projection algorithm (Adaptive Phase Adjusted Projection Algor ithm, APAP) to obtain multiple first sub-areas. Similarly, the second mapping image is cut through the adaptive phase adjustment projection algorithm to obtain multiple second sub-areas. The first mapping image and the second mapping image are cut into countless small blocks, and each small block is transformed by a homography matrix. Specifically, the embodiment of the present application extracts multiple first feature points from the first sub-area and multiple second feature points from the second sub-area, and then performs feature point matching on the first feature points and the second feature points to obtain multiple test point pairs to achieve feature point matching, and then screens all the test point pairs through a preset iterative algorithm to obtain a target point pair set to exclude erroneous points, so that the filtered feature points can basically correspond one to one, which can ensure that the correct matching points are used to align the images, thereby improving the matching accuracy. After that, the target point pair set is matrix estimated through a preset direct linear transformation algorithm (Direct Linear Transform, DLT) to obtain a perspective transformation matrix, which is convenient for the subsequent conversion of points in the image from one perspective to another. Finally, the spliced ​​image is calibrated for distortion according to the perspective transformation matrix, so that the distortion in the image can be corrected to align the feature points of the spliced ​​image, and multiple image fragments can be aligned to a common vision to form a seamless panoramic image, thereby achieving image registration and avoiding ghosting or excessive discontinuity problems in the image.

[0100] It should be noted that the adaptive phase adjustment projection algorithm in the embodiment of the present application and the iterative algorithm can be a random sample consensus algorithm (RANSAC), a gradient descent algorithm, a conjugate gradient method, etc. In this embodiment, the RANSAC algorithm is used, and the selection of the iterative algorithm is not specifically limited in this embodiment.

[0101] Reference Figure 3 , Figure 3 It is a specific flow chart of step S105 provided in an embodiment of the present application. In some embodiments, step S105 specifically includes but is not limited to step S301 and step S305.

[0102] Step S301, setting a test route in the overlapping area, and performing image analysis on the overlapping area to determine the source node and the sink node in the spliced ​​image.

[0103] It should be noted that the source node is a node used to represent the foreground area of ​​the stitched image, and the sink node is a node used to represent the background area of ​​the stitched image.

[0104] Step S302, select a first pixel and a second pixel in the stitched image, calculate a first eigenvalue of the first pixel at a source node and a second eigenvalue at a sink node, and calculate a third eigenvalue of the second pixel at a source node and a fourth eigenvalue at a sink node.

[0105] It should be noted that the first eigenvalue and the third eigenvalue are used to characterize the visual features at the source node, and the second eigenvalue and the fourth eigenvalue are used to characterize the visual features at the sink node.

[0106] Step S303, determining a first distance between the first pixel and the second pixel at the source node according to the first eigenvalue and the third eigenvalue, and determining a second distance between the first pixel and the second pixel at the sink node according to the second eigenvalue and the fourth eigenvalue.

[0107] Step S304: Calculate the secant weight according to the first distance and the second distance.

[0108] Step S305, determining the splicing gap of the divided pixel points according to the cut line weight.

[0109] In steps S301 to S305 of some embodiments, when stitching two images, processing the overlapping part is a key step because it directly affects the quality and naturalness of the final stitched image. In the process of dividing the overlapping area in the embodiment of the present application, first, a test route is set in the overlapping area of ​​the image stitching, and the overlapping area is image analyzed to determine the source node and the sink node in the stitched image, wherein the source node is a node representing the foreground area of ​​the stitched image, and the sink node is a node representing the background area of ​​the stitched image, and then, the first pixel point and the second pixel point are selected in the stitched image, and the first eigenvalue of the first pixel point at the source node and the second eigenvalue at the sink node are calculated, and similarly, the third eigenvalue of the second pixel point at the source node and the fourth eigenvalue at the sink node are calculated. Then, a first distance between the first pixel and the second pixel at the source node is determined according to the first eigenvalue and the third eigenvalue, and a second distance between the first pixel and the second pixel at the sink node is determined according to the second eigenvalue and the fourth eigenvalue. After that, a secant weight is calculated according to the first distance and the second distance, so that the edge weight between the first pixel and the second pixel can be obtained, which is convenient for subsequent image segmentation by finding the minimum cut. Finally, the splicing gap of the divided pixel points is determined according to the secant weight, and the minimum cut in the image is found according to the obtained maximum flow, so as to realize image segmentation and realize accurate search for the splicing gap. By reasonably determining the segmentation line, the pixel values ​​in the overlapping area can be smoothly transitioned, the splicing marks can be reduced, and obvious seams can be avoided.

[0110] It should be noted that the secant weight in the embodiment of the present application is used to characterize the edge weight between the first pixel and the second pixel. The first eigenvalue and the third eigenvalue in the embodiment of the present application can be grayscale values, color values, texture features, etc. at the source node. Similarly, the second eigenvalue and the fourth eigenvalue in the embodiment of the present application can be grayscale values, color values, texture features, etc. at the sink node. Specifically, when the first eigenvalue and the third eigenvalue are grayscale values ​​at the source node, they are used to represent the brightness information of each pixel near the source node area; when the first eigenvalue and the third eigenvalue are color values ​​at the source node, they are used to represent the color information of each pixel near the source node area, for example, red, green, blue information, hue information, saturation information or brightness information. Similarly, the selection of the second eigenvalue and the fourth eigenvalue is similar to the selection of the first eigenvalue and the third eigenvalue, and the embodiment of the present application will not be repeated.

[0111] In some embodiments, the secant weight may be calculated by the following formula:

[0112] M(s,t,A,B)=‖A(s)-B(s)‖+‖A(t)-B(t)‖;

[0113] Among them, s represents the source node in the flow network, t represents the sink node in the flow network, A represents the first pixel point, B represents the second pixel point, A(s) represents the first eigenvalue, B(s) represents the third eigenvalue, A(t) represents the second eigenvalue, A(s) represents the fourth eigenvalue, ‖A(s)-B(s)‖ represents the distance between the eigenvalues ​​of pixels A and B at node s (the first distance), and ‖A(t)-B(t)‖ represents the distance between the eigenvalues ​​of pixels A and B at node t (the second distance).

[0114] Specifically, the formula M(s, t, A, B) = ‖A(s)-B(s)‖ + ‖A(t)-B(t)‖ is used to calculate the weight of the secant between pixels. It measures the similarity between two pixels by calculating the distance between the eigenvalues ​​of the pixels at the source node and the sink node. The larger the distance, the more different the eigenvalues ​​of the two pixels at the source node and the sink node, and the larger the weight of the secant. Conversely, the smaller the distance, the more similar the eigenvalues ​​of the two pixels are, and the smaller the weight of the secant.

[0115] In some embodiments, in the process of determining the stitching gaps for dividing pixel points based on the cut line weights, the embodiments of the present application can solve the maximum flow problem in the flow network through the Ford-Fu l kerson, FFA algorithm, and then find the minimum cut in the image based on the obtained maximum flow to achieve image segmentation.

[0116] It can be understood that in graph theory, the set of edges that can make a network flow graph no longer connected by removing all the edges is called the cut of the graph, and the smallest cut on a graph is called the minimum cut. Given a specified directed graph, there are two special point sources S (Sources) and sinks T (Sinks), and each edge has a specified capacity. Find the maximum flow from S to T that satisfies the conditions. A cut is a fixed-point partition in the network, which divides all vertices in the network into two vertex sets S and T, where the source point s∈S and the sink point t∈T. It is denoted as CUT(S,T), the minimum cut from S to T that satisfies the conditions. In a directed network flow with edge weights, the minimum cut is defined as the set of edges that can make the source and sink disconnected after cutting off all edges and have the smallest edge weight sum. According to the maximum flow minimum cut theorem, the minimum cut edge weight sum at this time is equal to the maximum flow that can flow from the source point to the sink point on the network, which is simply referred to as the minimum cut equals the maximum flow.

[0117] Reference Figure 4 , Figure 4 401 and 406. FIG. 402 is a specific flow chart of step S106 provided in an embodiment of the present application. In some embodiments, step S106 specifically includes but is not limited to step S401 and step S406.

[0118] Step S401 , performing Gaussian filtering on the mapped image, and downsampling the Gaussian filtered mapped image to construct a Laplacian pyramid.

[0119] Step S402 : performing a difference operation on two adjacent layers of the Laplacian pyramid to decompose the mapped image into a plurality of sub-images of different sizes.

[0120] Step S403: For each sub-image, a weighted average operation is performed on the sub-image to obtain a fusion result.

[0121] Step S404, determining the number of pyramid levels of the Laplacian pyramid, and performing an inverse reconstruction operation on the Laplacian pyramid based on the number of pyramid levels to obtain an inverse pyramid.

[0122] Step S405 , weights are assigned to the spliced ​​images, and an upsampling operation is performed on the weight-assigned spliced ​​images through a reverse pyramid to obtain a sampled image.

[0123] Step S406: Fuse the sampled image and the fusion result to obtain a fused image.

[0124] In steps S401 to S406 of some embodiments, after finding the joint seam, due to factors such as image noise, illumination, exposure, model matching error, etc., directly performing image synthesis will result in obvious edge traces at the joint of the image overlapping area, so the embodiment of the present application will eliminate these traces through an image fusion algorithm. Specifically, in the process of eliminating the joint seam based on the preset image fusion algorithm, first, Gaussian filtering is performed on the mapped image, and the mapped image after Gaussian filtering is downsampled to construct a Laplacian pyramid, wherein the scale of each downsampling is approximately half of the previous one, and then, a difference operation is performed on the Laplacian pyramid of two adjacent layers until the top layer of the Gaussian pyramid is reached, so as to decompose the mapped image into multiple sub-images of different sizes. For each sub-image, a weighted average operation is performed on the sub-image to obtain a fusion result, so that the fusion result of each layer can be obtained, and then the number of pyramid layers of the Laplacian pyramid is determined, and the Laplacian pyramid is reversely reconstructed based on the number of pyramid layers, and the reconstructed image is initialized from the bottom layer, and then reconstructed layer by layer to obtain a reverse pyramid. After the reverse reconstruction obtains the reverse pyramid, the embodiment of the patent application will also assign weights to the stitched image. In the stitching area, different weights are assigned according to the position of the stitching seam and the image content, wherein the center of the stitching area will have more weights, while the area far away from the stitching seam has a smaller weight. The stitching image after weight assignment is upsampled through the reverse pyramid, starting from the top layer, upsampling layer by layer and adding back the Laplacian layer of each layer to achieve layer-by-layer upsampling of the stitched image to obtain a sampled image. Finally, the sampled image and the fusion result are fused to obtain a fused image, so that the fused image after fusion can obtain a better stitching effect, thereby ensuring the natural transition and high resolution of the stitched image.

[0125] It can be understood that the embodiment of the present application performs image fusion through the Laplacian pyramid, which can effectively retain the details and structural information of the image and obtain a better stitching effect.

[0126] Reference Figure 5 , Figure 5 1 is a specific flow chart of step S102 provided in an embodiment of the present application. In some embodiments, step S102 specifically includes but is not limited to step S501 and step S504.

[0127] Step S501 : for each image to be stitched, search for the image position of the image to be stitched in a multi-scale Gaussian scale space to obtain an extreme point in the scale space.

[0128] Step S502 , filtering the extreme points in the scale space by using a preset three-dimensional quadratic function to determine a plurality of key points of the image to be stitched.

[0129] Step S503, assigning directions to all key points, and performing feature description on the key points after the direction assignment to obtain description information.

[0130] Step S504: performing feature matching on all the images to be stitched based on the description information to obtain multiple groups of feature points.

[0131] In steps S501 to S504 of some embodiments, the embodiments of the present application perform feature matching on multiple images to be stitched using a Scale-Invariant Feature Transform (SIFT) algorithm. Specifically, for each image to be spliced, the embodiment of the present application searches for the image position of the image to be spliced ​​in a multi-scale Gaussian scale space based on a preset image local feature extraction algorithm to obtain the scale space extreme point, and realizes the detection of the scale space extreme point. At this time, the scale space extreme point obtained is the extreme point of the discrete space. After that, the scale space extreme point is detected by a preset three-dimensional quadratic function to determine the position and size of each scale space extreme point, and the scale space extreme point is filtered to remove low-contrast key points and unstable edge response points, and multiple key points of the image to be spliced ​​are determined, thereby enhancing the matching stability and improving the anti-noise ability. Then, all key points are assigned directions, and one or more directions are assigned to each key point position. In this way, even after the image is rotated, the feature descriptors of the key points can be guaranteed to be consistent, and the key points after the direction assignment are characterized, that is, a descriptor is established for each key point, and the key point is described with a set of vectors to obtain description information so that it does not change with various changes, such as changes in illumination and viewing angle. Finally, feature matching is performed on all images to be stitched based on the description information to obtain multiple groups of feature points, thereby increasing the probability of correct matching of feature points.

[0132] The description information is used to characterize the relationship between the key point and the local image area around it.

[0133] It is worth noting that in the process of feature matching of all images to be spliced ​​based on the description information, the embodiment of the present application can use the KD (Kdimension tree) tree algorithm to match the descriptors of two groups of key points, and select the closest distance pairing as the matching point. And after feature matching, the embodiment of the present application can also perform matching verification on multiple groups of feature points, and remove false positive matches through additional verification, which can improve the accuracy of feature point matching.

[0134] In some embodiments, the key point main direction assignment is to assign one or more directions to each key point position based on the local gradient direction of the image. All subsequent operations on the image data are transformed relative to the direction, scale and position of the key point, so that the descriptor has rotation invariance.

[0135] It is understandable that the SI FT algorithm is invariant to rotation and scale, which means that no matter how the image is scaled, the SI FT algorithm can detect the same feature points. It is also robust to noise, perspective changes, and illumination changes, and can still identify the same feature points after the image is rotated, which can resist these interferences to a certain extent, thereby improving the accuracy of feature matching.

[0136] In some embodiments, the three-dimensional quadratic function of the embodiment of the present application is used to characterize the relationship between the position coordinates and scale of the key point.

[0137] Reference Figure 6 , Figure 6 is a specific flow chart of determining a homography matrix based on multiple groups of feature points provided by an embodiment of the present application. In some embodiments, the method includes but is not limited to step S601 and step S605.

[0138] Step S601, screening feature points by using a preset iterative algorithm to obtain multiple pairs of matching points.

[0139] Step S602, calculating the matching points according to a preset direct linear transformation algorithm to obtain a test homography matrix.

[0140] Step S603: Calculate the mapping errors of all matching points by testing the homography matrix.

[0141] Step S604: compare the mapping error with a preset error threshold to screen out all matching point pairs whose mapping errors are greater than the error threshold to obtain an inlier set.

[0142] Step S605: update the test homography matrix according to the interior point set to obtain the homography matrix.

[0143] In steps S601 to S605 of some embodiments, in the process of determining the homography matrix based on multiple groups of feature points, the feature points are first screened by a preset iterative algorithm to obtain multiple pairs of matching points, and then the matching points are calculated according to a preset direct linear transformation algorithm (DLT) to obtain a test homography matrix. Then, the mapping errors of all matching points are calculated by the test homography matrix. Specifically, the embodiment of the present application calculates the mapping error of each matching point by Euclidean distance, and then compares the mapping error with a preset error threshold to screen out all matching point pairs whose mapping errors are greater than the error threshold to obtain an inlier set. Finally, the test homography matrix is ​​updated according to the inlier set. Using the largest inlier set can ensure that the calculation of the homography matrix is ​​based on the most reliable data, thereby improving the accuracy of the transformation matrix and obtaining the homography matrix, thereby improving the effect of image transformation and reducing image distortion.

[0144] It should be noted that the iterative algorithm in this embodiment can be a random sample consensus algorithm (RANSAC), a gradient descent algorithm, a conjugate gradient method, etc. In this embodiment, the RANSAC algorithm is used, and this embodiment does not make any specific restrictions on the choice of iterative algorithm.

[0145] It is understood that RANSAC is an iterative algorithm that estimates the parameters of a mathematical model from a set of observation data containing outliers. Outliers refer to invalid data in the observation data, usually noise or erroneous data, such as mismatched points in image matching and outliers in curve fitting. Corresponding to the outliers are inliers, which are valid data used to estimate the model parameters.

[0146] Reference Figure 7 , Figure 7 It is a specific flow chart of performing structural transformation on all images to be spliced ​​by using homography matrix provided by the embodiment of the present application. In some embodiments, the method includes but is not limited to step S701 to step S704.

[0147] Step S701, determining a pixel coordinate set of all pixels on the image to be stitched.

[0148] Step S702: performing coordinate transformation on all pixel point coordinates in the pixel point coordinate set by using a homography matrix to obtain a target coordinate set.

[0149] Step S703: performing interpolation processing on the image to be stitched according to the target coordinate set, and determining a target pixel set corresponding to the target coordinate set.

[0150] Step S704: merge the target pixel set and the target coordinate set to obtain multiple mapping images.

[0151] In steps S701 to S704 of some embodiments, in the process of performing structural transformation on all images to be stitched through the homography matrix, the embodiment of the present application first determines a pixel point coordinate set of all pixels on the image to be stitched, and then performs coordinate transformation on all pixel point coordinates in the pixel point coordinate set through the homography matrix, that is, converting the coordinates of any pixel point in the image to be stitched into a new coordinate point to obtain a target coordinate set. Afterwards, since the transformed coordinates may not be integers, the embodiment of the present application needs to use an interpolation method to obtain the pixel value at the new coordinate point, for example, nearest neighbor interpolation, bilinear interpolation or bicubic interpolation, etc., that is, interpolation processing is performed on the image to be stitched according to the target coordinate set, and a target pixel set corresponding to the target coordinate set is determined, thereby obtaining the pixel value at the new coordinate point. Finally, the target pixel set and the target coordinate set are merged to obtain multiple mapping images, and the mapping transformation of the image is realized. The converted image is a mapping image suitable for stitching.

[0152] In some embodiments, in the process of performing coordinate conversion on all pixel point coordinates in a pixel point coordinate set through a homography matrix, the embodiment of the present application first converts the image coordinates in the pixel point coordinate set into homogeneous coordinates, and then transforms the aligned secondary coordinates through the homography matrix to dehomogenize the coordinates and obtain a target coordinate set.

[0153] Reference Figure 8 The embodiment of the present application further provides an image stitching system, the image stitching system comprising:

[0154] The data set acquisition module 801 is used to acquire an image test set, wherein the image test set includes a plurality of images to be stitched;

[0155] A feature matching module 802 is used to perform feature matching on multiple images to be stitched to obtain multiple groups of feature points;

[0156] The structural transformation module 803 is used to determine a homography matrix based on multiple groups of feature points, and perform structural transformation on all images to be spliced ​​using the homography matrix to obtain multiple mapping images;

[0157] An image stitching module 804 is used to stitch multiple mapping images to obtain a stitched image and determine an overlapping area in the stitched image;

[0158] The region division module 805 is used to divide the overlapping region into regions and determine the splicing gaps of the divided pixels;

[0159] The gap elimination module 806 is used to eliminate the stitching gaps based on a preset image fusion algorithm to obtain a fused image.

[0160] In some embodiments, the image stitching system also includes an image registration module, which is used to select a first mapping image and a second mapping image in the mapping image, cut the first mapping image through a preset adaptive phase adjustment projection algorithm to obtain multiple first sub-regions, and cut the second mapping image through a preset adaptive phase adjustment projection algorithm to obtain multiple second sub-regions; extract the first feature point of the first sub-region and the second feature point of the second sub-region; and perform feature point matching on the first feature point and the second feature point to obtain multiple test point pairs; screen all test point pairs through a preset iterative algorithm to obtain a target point pair set; and perform matrix estimation on the target point pair set through a preset direct linear transformation algorithm to obtain a perspective transformation matrix; and perform distortion correction on the stitched image according to the perspective transformation matrix to align the feature points of the stitched image.

[0161] The image stitching system of the embodiment of the present application is used to execute the image stitching method in the above embodiment. Its specific processing process is the same as the image stitching method in the above embodiment, and will not be described in detail here.

[0162] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is used to execute the image stitching method in the above embodiment of the present application.

[0163] See also Fig. 9 , Fig. 9 The hardware structure of the electronic device provided in the embodiment of the present application is illustrated, and the electronic device includes:

[0164] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0165] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the image stitching method of the embodiment of this application;

[0166] Input / output interface 903, used to implement information input and output;

[0167] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0168] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0169] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0170] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image stitching method is implemented.

[0171] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0173] It can be understood by those skilled in the art that Figure 1-9 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0174] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0175] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0176] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0177] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0179] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0182] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. An image stitching method, characterized in that: The method comprises: Acquire an image test set, wherein the image test set includes a plurality of images to be stitched; Performing feature matching on the plurality of images to be stitched to obtain a plurality of groups of feature points; Determine a homography matrix based on the multiple groups of feature points, and perform structural transformation on all the images to be stitched using the homography matrix to obtain multiple mapping images; splicing a plurality of the mapping images to obtain a spliced ​​image, and determining an overlapping area in the spliced ​​image; Divide the overlapping area into regions to determine the splicing gaps of divided pixel points; The stitching gap is eliminated based on a preset image fusion algorithm to obtain a fused image.

2. The image stitching method according to claim 1, characterized in that: After performing structural transformation on all the images to be stitched by using the homography matrix to obtain a plurality of mapping images, the method further includes: Selecting a first mapping image and a second mapping image from the mapping images, cutting the first mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of first sub-regions, and cutting the second mapping image by a preset adaptive phase adjustment projection algorithm to obtain a plurality of second sub-regions; Extracting first feature points of the first sub-region and second feature points of the second sub-region; Performing feature point matching on the first feature point and the second feature point to obtain a plurality of test point pairs; Screening all the test point pairs through a preset iterative algorithm to obtain a target point pair set; Performing matrix estimation on the target point pair set by using a preset direct linear transformation algorithm to obtain a perspective transformation matrix; Distortion correction is performed on the stitched image according to the perspective transformation matrix to align feature points of the stitched image.

3. The image stitching method according to claim 1, characterized in that: The step of dividing the overlapping area into regions and determining the splicing gaps of the divided pixel points includes: A test route is set in the overlapping area, and image analysis is performed on the overlapping area to determine a source node and a sink node in the stitched image, wherein the source node is a node used to represent a foreground area of ​​the stitched image, and the sink node is a node used to represent a background area of ​​the stitched image; Selecting a first pixel point and a second pixel point in the stitched image, calculating a first eigenvalue of the first pixel point at the source node and a second eigenvalue of the first pixel point at the sink node, and calculating a third eigenvalue of the second pixel point at the source node and a fourth eigenvalue of the second pixel point at the sink node, wherein the first eigenvalue and the third eigenvalue are used to characterize visual features at the source node, and the second eigenvalue and the fourth eigenvalue are used to characterize visual features at the sink node; Determine a first distance between the first pixel and the second pixel at the source node according to the first eigenvalue and the third eigenvalue, and determine a second distance between the first pixel and the second pixel at the sink node according to the second eigenvalue and the fourth eigenvalue; Calculate a secant weight according to the first distance and the second distance; The splicing gap of the divided pixel points is determined according to the cut line weight.

4. The image stitching method according to claim 1, characterized in that: The step of performing an operation of eliminating the stitching gap based on a preset image fusion algorithm to obtain a fused image includes: Performing Gaussian filtering on the mapping image, and downsampling the Gaussian filtered mapping image to construct a Laplacian pyramid; Performing a difference operation on two adjacent layers of the Laplacian pyramid to decompose the mapped image into a plurality of sub-images of different sizes; For each of the sub-graphs, a weighted average operation is performed on the sub-graph to obtain a fusion result; Determining the number of pyramid levels of the Laplacian pyramid, and performing a reverse reconstruction operation on the Laplacian pyramid based on the number of pyramid levels to obtain a reverse pyramid; Performing weight assignment on the stitched image, and performing an upsampling operation on the stitched image after weight assignment through the reverse pyramid to obtain a sampled image; The sampled image and the fusion result are fused to obtain a fused image.

5. The image stitching method according to claim 1, characterized in that: The performing feature matching on the plurality of images to be stitched to obtain a plurality of groups of feature points includes: For each of the images to be stitched, searching for the image position of the image to be stitched in a multi-scale Gaussian scale space to obtain an extreme point in the scale space; Filtering the extreme points in the scale space by a preset three-dimensional quadratic function to determine a plurality of key points of the image to be stitched; Assigning directions to all the key points, and describing the features of the key points after the direction assignment to obtain description information; Based on the description information, feature matching is performed on all the images to be spliced ​​to obtain multiple groups of feature points.

6. The image stitching method according to claim 1, characterized in that: The determining of the homography matrix based on the multiple groups of feature points comprises: The feature points are screened by a preset iterative algorithm to obtain multiple pairs of matching points; Calculating the matching points according to a preset direct linear transformation algorithm to obtain a test homography matrix; Calculating the mapping errors of all the matching points by using the test homography matrix; Comparing the mapping error with a preset error threshold to screen out all matching point pairs whose mapping error is greater than the error threshold to obtain an inlier set; The test homography matrix is ​​updated according to the interior point set to obtain a homography matrix.

7. The image stitching method according to claim 1, characterized in that: The step of performing structural transformation on all the images to be stitched by using the homography matrix to obtain a plurality of mapping images includes: Determine a pixel coordinate set of all pixels on the image to be stitched; Performing coordinate transformation on all pixel point coordinates in the pixel point coordinate set by using the homography matrix to obtain a target coordinate set; Performing interpolation processing on the image to be stitched according to the target coordinate set to determine a target pixel set corresponding to the target coordinate set; The target pixel set and the target coordinate set are fused to obtain a plurality of mapping images.

8. An image stitching system, characterized in that: The image stitching system comprises: A data set acquisition module, used to acquire an image test set, wherein the image test set includes a plurality of images to be stitched; A feature matching module is used to perform feature matching on the multiple images to be stitched to obtain multiple groups of feature points; A structural transformation module, used for determining a homography matrix based on the plurality of groups of feature points, and performing structural transformation on all the images to be stitched by using the homography matrix to obtain a plurality of mapping images; An image stitching module, used for stitching a plurality of the mapping images to obtain a stitched image, and determining an overlapping area in the stitched image; A region division module, used to divide the overlapping region into regions and determine the splicing gaps of the divided pixel points; The gap elimination module is used to eliminate the stitching gap based on a preset image fusion algorithm to obtain a fused image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the image stitching method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the image stitching method according to any one of claims 1 to 7.

Images