Image stitching method and device
By calculating the distance matrix and cost matrix in image stitching, the stitching line is searched and determined, thus solving the problem of artifacts in image stitching and generating a panoramic image with a natural transition.
Patent Information
- Application Number
- CN202111639255.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-29
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-12-29
AI Technical Summary
In existing technologies, image stitching often suffers from artifacts, resulting in unnatural fused images.
By determining the overlapping regions of the images to be stitched, a distance matrix and a cost matrix are calculated. Based on these matrices, suture lines are searched, and image stitching is performed based on the suture lines. The distance matrix and cost matrix are used to determine suture lines with shorter lengths and smaller differences to remove artifacts.
It effectively removes artifacts during the image stitching process, generating panoramic images with natural transitions.
Smart Images

Figure CN114549301B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image stitching method and device. Background Art
[0002] Due to the limitation of hardware equipment, a camera can only obtain images with a small viewing angle, and the field of view is not wide enough. To obtain a wide viewing angle image, it is generally necessary to move the camera to obtain a sequence of images with overlapping areas in space, and then use image stitching technology to generate a panoramic image. The process of image stitching technology mainly includes image preprocessing, image registration, image fusion and other steps. Image registration uses matching technology to obtain the mapping relationship between images, so that the same object captured by different images after mapping can overlap. Image fusion includes seam line search and further fusion processing of the seam line area, so as to obtain a more natural wide-view image.
[0003] In actual use, due to the complex and changeable shooting environment and the relatively strict requirements on the shooting equipment, there is a high probability that the fused results will have problems such as artifacts after the images shot from different perspectives are registered and mapped. Summary of the invention
[0004] The present application provides an image stitching method and device, which can effectively solve the artifact phenomenon generated during the image stitching process.
[0005] To achieve the above object, the present application provides an image stitching method, which comprises:
[0006] Determine an overlapping area of the first image and the second image to be stitched;
[0007] Calculating the distance matrix and cost matrix of the overlapping area;
[0008] Searching and determining a stitching line based on the distance matrix and the cost matrix;
[0009] The first image and the second image are stitched together based on the stitching line.
[0010] Wherein, the searching and determining the stitching line based on the distance matrix and the cost matrix comprises:
[0011] Taking one of the two intersection points of the first image and the second image as a starting point and the other intersection point as an end point;
[0012] Taking the starting point as the current center point;
[0013] Based on the distance matrix, selecting at least one first pixel point from the neighborhood pixel points of the current center point;
[0014] Selecting a pixel point with the smallest cost matrix value from the at least one first pixel point as a second pixel point;
[0015] In the case where the end point is not on the line between the current center point and the second pixel point, taking the second pixel point as the current center point, and performing the step of selecting at least one first pixel point from the neighborhood pixel points of the current center point based on the distance matrix;
[0016] In a case where the end point is on a line between the current center point and the second pixel point, the stitching line is determined based on the start point, all searched second pixel points, and the end point.
[0017] The value of each pixel in the distance matrix represents the distance between each pixel in the overlapping area and the target pixel.
[0018] The selecting at least one first pixel point from the neighboring pixel points of the current center point based on the distance matrix includes:
[0019] If the end point is the target pixel point, a neighboring pixel point whose distance matrix value is smaller than that of the current center point is selected from the neighboring pixel points of the current center point, and the selected neighboring pixel point is used as the first pixel point;
[0020] If the starting point is the target pixel point, a neighborhood pixel point whose distance matrix value is larger than that of the current center point is selected from the neighborhood pixel points of the current center point, and the selected neighborhood pixel point is used as the first pixel point.
[0021] Wherein, when the end point is not on the line between the current center point and the second pixel point, the second pixel point is used as the current center point, which includes:
[0022] Adding the second pixel point to the stitching line path set;
[0023] When there is no end point in the stitching line path set, it is confirmed that the end point is not on the line between the current center point and the second pixel point.
[0024] The step of taking the second pixel point as the current center point and performing the step of selecting at least one first pixel point from the neighborhood pixel points of the current center point based on the distance matrix comprises:
[0025] The values of the pixel points newly added to the stitching line path set in the distance cost and the cost matrix are increased.
[0026] Wherein, when there is no end point in the stitching line path set, confirming that the second pixel point is not the end point, and confirming that the end point is not on the line between the current center point and the second pixel point, comprises:
[0027] If there is at least one pixel point between the current center point and the second pixel point, the point between the current center point and the second pixel point is added to the stitching line path set.
[0028] The step of calculating the distance matrix and the cost matrix of the overlapping area includes:
[0029] A distance matrix of the overlapping area is calculated based on the distance transformation.
[0030] The step of calculating the distance matrix and the cost matrix of the overlapping area includes:
[0031] Based on the distance transformation and in combination with the gamma transformation result of the difference between the first image and the second image for each pixel point in the overlapping area, a distance matrix of the overlapping area is calculated.
[0032] The step of calculating the distance matrix and the cost matrix of the overlapping area includes:
[0033] The cost of each pixel in the overlapping area is calculated by using the color energy cost and / or the structural energy cost of each pixel in the overlapping area to obtain a cost matrix of the overlapping area.
[0034] To achieve the above objectives, the present application also provides an electronic device, which includes a processor; the processor is used to execute instructions to implement the above method.
[0035] To achieve the above objectives, the present application also provides a computer-readable storage medium, which is used to store instructions / program data, and the instructions / program data can be executed to implement the above method.
[0036] In the image stitching method of the present application, the overlapping area of the first image and the second image is first determined; then the distance matrix and the cost matrix of the overlapping area are calculated; then the stitching line is searched based on the distance matrix and the cost matrix, and the first image and the second image are stitched based on the stitching line, so that the stitching line with a shorter length and a smaller difference can be determined by the distance matrix and the cost matrix, so that the artifacts in the overlapping area of the two images can be effectively removed based on the stitching line with a shorter length and a smaller difference. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0038] Figure 1 It is a flowchart of an implementation method of the image stitching method of the present application;
[0039] Figure 2 is a schematic diagram of a mask used for distance matrix calculation in the image stitching method of the present application;
[0040] Figure 3 is a schematic diagram of artifacts in the overlapping area of two images;
[0041] Figure 4 The image stitching method of this application is based on Figure 3 Schematic diagram of the seam lines determined by the overlapping region search shown;
[0042] Figure 5 is a schematic diagram of image stitching results in the image stitching method of the present application;
[0043] Figure 6 It is a structural schematic diagram of an embodiment of the electronic device of the present application;
[0044] Figure 7 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present application. In addition, unless otherwise specified (for example, "or in addition" or "or in an alternative"), the term "or" as used herein refers to a non-exclusive "or" (that is, "and / or"). Moreover, the various embodiments described herein are not necessarily mutually exclusive, because some embodiments can be combined with one or more other embodiments to form new embodiments.
[0046] Specifically, Figure 1 As shown, the image stitching method of the present application includes the following steps. It should be noted that the following step numbers are only used to simplify the description and are not intended to limit the execution order of the steps. The execution order of each step of this embodiment can be changed arbitrarily without violating the technical idea of the present application.
[0047] S101: Determine an overlapping area of a first image and a second image to be stitched.
[0048] The overlapping area of the first image and the second image to be stitched may be determined first, so as to subsequently determine a stitching line in the overlapping area, and then stitch the first image and the second image based on the stitching line.
[0049] Optionally, the first image and the second image may be registered; and then an overlapping area of the registered first image and the second image is determined.
[0050] The manner of registering the first image and the second image is not limited. For example, the first image and the second image may be registered by a method based on the frequency domain (phase correlation method) or a method based on the time domain.
[0051] Specifically, during registration, feature points of the first image and feature points on the second image can be detected; a mapping matrix is constructed based on the feature points on the first image and the feature points on the second image; the first image is mapped to the coordinate system of the second image based on the mapping matrix, or the second image is mapped to the coordinate system of the first image based on the mapping matrix, or the first image and the second image are mapped to a preset coordinate system based on the mapping matrix to complete image registration.
[0052] The feature points of the above-mentioned image may be SIFT feature points, SURF feature points, ORB feature points or binary descriptors of the image.
[0053] In addition, before constructing the mapping matrix, the feature points on the first image and the feature points on the second image may be screened to filter out the outliers, and then the mapping matrix may be constructed using the screened feature points. Specifically, the RANSAC algorithm may be used to screen the feature points on the first image and the feature points on the second image.
[0054] Furthermore, the step of constructing a mapping matrix based on the feature points on the first image and the feature points on the second image may include: matching the feature points on the first image with the feature points on the second image; and constructing a mapping matrix based on the matching result. The constructed mapping matrix may be a homography matrix.
[0055] Optionally, the image may be mapped to a corresponding coordinate system by a reverse mapping bilinear interpolation algorithm, etc. For example, the first image may be mapped to a coordinate system of the second image by a reverse mapping bilinear interpolation algorithm, etc.
[0056] S102: Calculate the distance matrix and cost matrix of the overlapping area.
[0057] After determining the overlapping area of the first image and the second image based on step S101, the distance matrix and the cost matrix of the overlapping area can be calculated, so that the stitching line can be searched based on the distance matrix and the cost matrix, and the first image and the second image can be spliced based on the stitching line, so that the artifacts in the overlapping area after registration (for example, Figure 3 Artifacts shown in the box).
[0058] Optionally, a distance matrix of the overlapping area can be calculated by distance transformation. In this way, the distance matrix can be used to express the distance between the pixel point in the overlapping area and the target pixel point or area block, so that when any point between the two intersection points of the first image and the second image is used as the target pixel point, the distance matrix can be used to represent the distance between the pixel point in the overlapping area and any point between the two intersection points, so that the shorter stitching line can be determined by the distance matrix.
[0059] In addition, the distance matrix of the overlapping area can be calculated by combining the gamma transformation result of the difference between the first image and the second image in the overlapping area; so that when the stitching line is determined based on the distance matrix, the stitching point where the difference between the first image and the second image is small can be determined based on the distance matrix.
[0060] Specifically, the specific implementation process of the step of calculating the distance matrix of the overlapping area by combining the gamma transformation result of the difference between the first image and the second image may be:
[0061] Create a new matrix F with the same size as the overlapping area;
[0062] One of the two intersections of the first image and the second image is used as the target pixel, the value of the target pixel in the matrix is set to the first value, and the value of the points other than the target pixel in the matrix is set to the second value. The first value and the second value are not equal. The first value and the second value can be set according to actual conditions and are not limited here. For example, the first value can be 0 or 2, and the second value can be 100 or infinity.
[0063] In this embodiment, the value of the target pixel is set to 0, and the value of the remaining pixels is set to infinity.
[0064] use Figure 2 The mask 1 shown starts from the upper left corner, traverses the matrix F from left to right and from top to bottom in normal order, and updates the values corresponding to the point p in the mask and the matrix F to obtain the matrix F'.
[0065] The formula is as follows:
[0066] F′(p)=min q∈mask1 {F(p),D(p,q)+F(q)};
[0067] Where F′(p) represents the updated value of point p in the matrix F, F(p) represents the value of point p in the matrix F before the update, and F(q) represents the value of matrix F at point q; D(p,q) is the distance between point p and point q (which can be Euclidean distance or Hamming distance).
[0068] Using mask 2, starting from the lower right corner, traverse the matrix F' from right to left and from bottom to top, and update the values corresponding to the matrix F' and the point p in the mask to obtain the distance matrix.
[0069] The formula is as follows:
[0070] F″(p)=min q∈mask2 {F′(p),D(p,q)+F′(q)}*gama(p)*;
[0071] In the above formula, F″(p) represents the value of point p in the matrix F' after updating, F′(p) represents the value of point p in the matrix F' before updating, and F′(q) represents the value of the matrix F' at point q; D(p, q) is the distance between point p and point q (which can be Euclidean distance or Hamming distance), and gama(p) represents the gamma transform result of the difference at point p.
[0072] The calculation formula of the gamma transform result gama(p) of the difference at point p in the overlapping area is as follows:
[0073] gama(p)=(|I0(p)-I1(p)|)+esp) γ ;
[0074] Among them, |I0(p)-I1(p)| is the difference between the first image and the second image at pixel point p. The esp coefficient has a compensation effect. The value of the esp coefficient determines the contrast of different grayscale areas of the image: when the gamma value is greater than 1, the high grayscale area of the image will be contrast enhanced, when the gamma value is less than 1, the low grayscale area of the image will be contrast enhanced, and when the gamma value is equal to 1, the input and output images remain unchanged. In this way, the esp coefficient is set according to the actual situation to highlight the difference part of the overlapping area, so that the distance matrix obtained based on the gamma transform result value can not only reflect the distance between the pixel point and the target pixel point, but also reflect the difference between the first image and the second image. Specifically, the esp coefficient can be set to 3 or 2.
[0075] In addition, when the first image and the second image are RGB images,
[0076] |I0(p)-11(p)|=(|R0(p)-R1(p)|+|G0(p)-G1(p)|+|B0(p)-B1(p)|) / 3,
[0077] Wherein, R0(p) is the value of the R channel of point p on the first image, and R1(p) is the value of the R channel of point p on the second image; G0(p) is the value of the G channel of point p on the first image, and G1(p) is the value of the G channel of point p on the second image; B0(p) is the value of the B channel of point p on the first image, and B1(p) is the value of the B channel of point p on the second image;
[0078] In addition, when the first image and the second image are YUV images,
[0079] |I0(p)-I1(p)|=|Y0(p)-Y1(p)|,
[0080] Wherein, Y0(p) is the value of the Y channel of point p on the first image, and Y1(p) is the value of the Y channel of point p on the second image.
[0081] In addition, the cost matrix of the overlapping area may be calculated using the color energy cost and / or the structure energy cost.
[0082] Exemplarily, the cost of each pixel in the overlapping area may be calculated by using the color energy cost and the structural energy cost of each pixel in the overlapping area to obtain a cost matrix of the overlapping area.
[0083] In one example, the calculation formula of the cost E of the pixel points in the overlapping area is as follows:
[0084] E(p)=E color (p) 2 +E geometry (p);
[0085] Among them, E(p) is the cost of pixel p in the overlapping area, E color (p) is the color energy cost of pixel p in the overlapping area, E geometry (p) is the structural energy cost of pixel p in the overlapping area.
[0086] Optionally, the color energy cost of the overlapping area may be calculated by using the color difference between the first image and the second image in the overlapping area.
[0087] Optionally, the color energy cost of the overlapping area may be calculated using the following formula.
[0088] E color (p)=|I0(p)-I1(p)| / max(I0(p), I1(p));
[0089] Among them, E color (p) is the color energy cost of point p, I0(p) is the pixel value of point p on the first image, and I1(p) is the pixel value of point p on the second image.
[0090] In the case where the first image and the second image are RGB images,
[0091]
[0092] In addition, when the first image and the second image are YUV images,
[0093] max(I0(p), I1(p)=max(Y0(p), Y1(p)).
[0094] Optionally, the structural energy cost of the overlapping area may be calculated by using the gradient difference between the first image and the second image in the overlapping area.
[0095] The formula is as follows:
[0096] E geometry (p) = || T0(p) - T1(p) ||;
[0097] Among them, E geometry (p) is the structural energy cost of point p, T0(p) is the gradient value of point p on the first image, and T1(p) is the gradient value of point p on the second image.
[0098] The gradient values of the pixels in the respective overlapping regions of the first image and the second image may be calculated using the Sobel gradient calculation formula, the Scharr gradient calculation formula or the Laplacian gradient calculation formula.
[0099] S103: Search and determine the seam line based on the distance matrix and the cost matrix.
[0100] Optionally, after the distance matrix and the cost matrix of the overlapping area are determined based on the above steps, a stitching line may be determined based on the distance matrix and the cost matrix search, so that the first image and the second image are subsequently spliced based on the stitching line.
[0101] Optionally, one of the two intersections of the first image and the second image can be used as the starting point and the other intersection as the end point; the starting point is used as the current center point; based on the distance matrix, at least one first pixel is selected from the neighboring pixel points of the current center point; from at least one first pixel point, the pixel point with the smallest cost matrix value is selected as the second pixel point; when the second pixel point is not the end point and the end point is not on the line between the current center point and the second pixel point, the second pixel point is used as the current center point, and the step of selecting at least one first pixel point from the neighboring pixel points of the current center point based on the distance matrix is performed; when the second pixel point is the end point, or the end point is on the line between the current center point and the second pixel point, the stitching line is determined based on the starting point, all the searched second pixel points and the end point. In this way, the moving direction of the search center point can be determined by the distance matrix. When there are multiple directions that can be moved, the point with the smallest energy cost difference can be selected as the next search center point, so that a stitching line with a shorter distance and less change can be determined, and the stitching line can be determined at a faster speed.
[0102] Optionally, when the end point is the target pixel point in the distance matrix, the selected pixel points can select the neighboring pixel points whose distance matrix values are smaller than the current center point from the neighboring pixel points of the current center point, and use the selected neighboring pixel points as the first pixel point, that is, the value of the first pixel point in the distance matrix is smaller than the value of the current center point. In other alternative embodiments, when the starting point is the target pixel point in the distance matrix, the selected pixel points can select the neighboring pixel points whose distance matrix values are larger than the current center point from the neighboring pixel points of the current center point, and use the selected neighboring pixel points as the first pixel point, that is, the value of the first pixel point in the distance matrix is larger than the value of the current center point.
[0103] In one implementation, when the second pixel point is searched through the current center point in the above manner, the current center point and the second pixel point can be connected with a straight line; when the end point is not on the line between the current center point and the second pixel point, the second pixel point is used as the current center point, and the second pixel point is searched based on the updated current center point, and the updated current center point and the newly searched second pixel point are connected with a straight line, and this is repeated; until the end point is on the line between the updated current center point and the newly searched second pixel point, and a stitching line connecting the starting point and the end point is obtained.
[0104] In another implementation, when the second pixel point is searched through the current center point in the above manner, the second pixel point is added to the stitching line path set; if there is no end point in the updated stitching line path set, it can be confirmed that the second pixel point is not the end point, and it can be confirmed that the end point is not on the line between the current center point and the second pixel point. In this case, the second pixel point is used as the current center point, and the second pixel point is searched based on the updated current center point, and then the newly searched second pixel point is added to the stitching line path set, and this is repeated; until the end point is on the line between the current center point and the second pixel point, and finally the stitching line connecting the starting point and the end point can be obtained according to the stitching line path set.
[0105] To further speed up the search, the step size can be set to an integer greater than 1. For example, the step size is set to 2, that is, the current center point has [(2*2+1)2-1]——24 neighboring pixels. This can reduce the number of searches for the second pixel point and improve the efficiency of the seam line search, making the image stitching method of this application more suitable for scenes that need to run in real time. It can be understood that when the step size is 1, the current center point has only [(1*2+1)2-1]——8 neighboring pixels, so the second center point will only be searched from the 8 neighboring pixels of the current center point.
[0106] Furthermore, when the step size is an integer greater than 1 and the stitching line is determined by updating the stitching line path set, if there is at least one pixel between the current center point and the second pixel point searched with the current center point, the newly searched second pixel point and the pixel point between the current center point and the second pixel point can be added to the stitching line path set; in this way, by confirming whether there is an end point in the stitching line path set, it can be confirmed whether there is an end point on the line connecting the current center point and the second pixel point searched with the current center point.
[0107] In addition, in order to avoid the searched second pixel point and the pixel point between the second pixel point and the current center point from being repeatedly selected as stitching line points to reduce the stitching line length, the values of the second pixel point and the pixel point between the second pixel point and the current center point in the distance cost and the cost matrix can be increased (even set to positive infinity).
[0108] S104: stitching the first image and the second image based on the stitching line.
[0109] Optionally, after the stitching line is determined based on the above steps, the first image and the second image may be spliced based on the stitching line.
[0110] In one implementation, in step S104, the first image may be taken on one side and the second image may be taken on the other side of the suture line. Figure 4As shown, the first image I0 is taken on the left side of the stitching line, and the second image I1 is taken on the right side of the stitching line.
[0111] In order to make the transition at the stitching line natural, after the selected images on both sides of the stitching line are determined, the overlapping area can be fused based on the stitching line and through the first image and the second image.
[0112] Specifically, weighted averaging, feathering algorithm or Laplace pyramid fusion and other methods can be used to fuse the overlapping areas to eliminate the unnatural transition between the two images to be stitched due to exposure differences, and finally obtain the stitching result. Figure 5 The stitched images fused by the Laplacian pyramid fusion method are shown to be filtered naturally and without artifacts.
[0113] In this embodiment, the overlapping area of the first image and the second image is first determined; then the distance matrix and the cost matrix of the overlapping area are calculated; then the stitching line is searched based on the distance matrix and the cost matrix, and the first image and the second image are spliced based on the stitching line, so that the stitching line with a shorter length and a smaller difference can be determined by the distance matrix and the cost matrix, so that the artifacts in the overlapping area of the two images can be effectively removed based on the stitching line with a shorter length and a smaller difference.
[0114] In addition, the image stitching method of the present application can also be applied to the stitching of multiple images (for example, three images), and has a wide range of applications.
[0115] See also Figure 6 , Figure 6 The electronic device 20 of the present application includes a processor 22, which is used to execute instructions to implement the method of any of the above embodiments of the present application and any non-conflicting combination thereof.
[0116] The electronic device 20 may be a camera device, a server or other device, which is not limited here.
[0117] The processor 22 may also be referred to as a CPU (Central Processing Unit). The processor 22 may be an integrated circuit chip having signal processing capabilities. The processor 22 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor, or the processor 22 may also be any conventional processor, etc.
[0118] The electronic device 20 may further include a memory 21 for storing instructions and data required for the processor 22 to run.
[0119] See also Figure 7 , Figure 7 It is a schematic diagram of the structure of the computer-readable storage medium in the implementation mode of the present application. The computer-readable storage medium 30 of the embodiment of the present application stores instructions / program data 31, and when the instructions / program data 31 are executed, the method provided by any embodiment of the above-mentioned method of the present application and any non-conflicting combination is implemented. Among them, the instructions / program data 31 can form a program file and be stored in the above-mentioned storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) executes all or part of the steps of each implementation method of the present application. The aforementioned storage medium 30 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a computer, a server, a mobile phone, a tablet and other devices.
[0120] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0122] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0123] The above are only implementation methods of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. An image stitching method, It is characterized in that The method comprises: Determine an overlapping area of the first image and the second image to be stitched; Calculating a distance matrix and a cost matrix of the overlapping area, wherein the cost of each pixel in the cost matrix is calculated by using the color energy cost and / or the structural energy cost of each pixel in the overlapping area; Searching and determining a stitching line based on the distance matrix and the cost matrix; splicing the first image and the second image based on the stitching line; Wherein, the searching and determining the stitching line based on the distance matrix and the cost matrix comprises: Search out all second pixel points; each time searching, based on the distance matrix, select at least one first pixel point from the neighboring pixel points of the current center point, and select the pixel point with the smallest cost matrix value from the at least one first pixel point as the second pixel point; the current center point is initially the starting point, and the subsequent ones are the second pixel points that are most recently searched; The stitching line is determined based on the starting point, all the searched second pixel points and an end point; the starting point is one of the two intersection points of the first image and the second image, and the end point is the remaining one of the two intersection points.
2. The method according to claim 1, It is characterized in that The step of searching and determining a stitching line based on the distance matrix and the cost matrix comprises: Taking the starting point as the current center point; Based on the distance matrix, selecting at least one first pixel point from the neighborhood pixel points of the current center point; Selecting a pixel point with the smallest cost matrix value from the at least one first pixel point as a second pixel point; In the case where the end point is not on the line between the current center point and the second pixel point, taking the second pixel point as the current center point, and performing the step of selecting at least one first pixel point from the neighborhood pixel points of the current center point based on the distance matrix; In a case where the end point is on a line between the current center point and the second pixel point, the stitching line is determined based on the start point, all searched second pixel points, and the end point.
3. The method according to claim 2, It is characterized in that The value of each pixel in the distance matrix represents the distance between each pixel in the overlapping area and the target pixel. The selecting at least one first pixel point from the neighboring pixel points of the current center point based on the distance matrix includes: If the end point is the target pixel point, a neighboring pixel point whose distance matrix value is smaller than that of the current center point is selected from the neighboring pixel points of the current center point, and the selected neighboring pixel point is used as the first pixel point; If the starting point is the target pixel point, a neighborhood pixel point whose distance matrix value is larger than that of the current center point is selected from the neighborhood pixel points of the current center point, and the selected neighborhood pixel point is used as the first pixel point.
4. The method according to claim 2, It is characterized in that In the case where the end point is not on the line between the current center point and the second pixel point, taking the second pixel point as the current center point, the method comprises: Adding the second pixel point to the stitching line path set; When there is no end point in the stitching line path set, it is confirmed that the end point is not on the line between the current center point and the second pixel point.
5. The method according to claim 4, It is characterized in that The step of taking the second pixel point as the current center point and performing the step of selecting at least one first pixel point from the neighborhood pixel points of the current center point based on the distance matrix comprises: The values of the pixel points newly added to the stitching line path set in the distance cost and the cost matrix are increased.
6. The method according to claim 4 or 5, It is characterized in that In the case where there is no end point in the stitching line path set, confirming that the second pixel point is not an end point, and confirming that the end point is not on a line connecting the current center point and the second pixel point, the method comprises: If there is at least one pixel point between the current center point and the second pixel point, the point between the current center point and the second pixel point is added to the stitching line path set.
7. The method according to claim 1, It is characterized in that The step of calculating the distance matrix and the cost matrix of the overlapping area includes: A distance matrix of the overlapping area is calculated based on the distance transformation.
8. The method according to claim 7, It is characterized in that The step of calculating the distance matrix and the cost matrix of the overlapping area includes: Based on the distance transformation and in combination with the gamma transformation result of the difference between the first image and the second image for each pixel in the overlapping area, a distance matrix of the overlapping area is calculated.
9. The method according to claim 1, It is characterized in that The step of calculating the distance matrix and the cost matrix of the overlapping area includes: The cost of each pixel in the overlapping area is calculated by using the color energy cost and / or the structural energy cost of each pixel in the overlapping area to obtain a cost matrix of the overlapping area.
10. An electronic device, It is characterized in that The electronic device comprises a processor, and the processor is used to execute instructions to implement the steps of the method according to any one of claims 1-9.
11. A computer-readable storage medium having a program and / or instructions stored thereon, It is characterized in that When the program and / or instruction is executed, the steps of the method according to any one of claims 1 to 9 are implemented.
