A local minimum spanning tree path planning method supporting global coordinate updating

Abstract

The application discloses a local minimum spanning tree path planning method supporting global coordinate updating, and belongs to the field of image processing. The method comprises the following steps: grouping collected images according to an XY space coordinate system and an adjacency relation, calculating relative offset distances among the images and a normalized correlation index; confirming a starting group and a starting image of the starting group, constructing a minimum spanning tree by using a minimum spanning tree algorithm, and calculating global coordinates of the images in the starting group; in a current group and an adjacent group in which global coordinates have been solved, finding an image pair having a spatial adjacency relation with the images in the current group, determining a reference image and a starting image of the current group, calculating the global coordinates of the starting image of the current group, updating the global coordinates of the remaining adjacent images in the current group, and calculating the global coordinates of the remaining images in the current group along the minimum spanning tree; and images in the remaining groups are processed in the same way until the global coordinates of all the images in the groups are calculated, and a path planning graph is obtained. The application improves the image splicing precision and efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to a local minimum spanning tree path planning method supporting global coordinate updating. BACKGROUND

[0002] Image stitching refers to a scene of merging a series of local views with overlapping areas into a large field-of-view panoramic image. The core process of image stitching can be divided into three key links: image registration, path planning and image fusion. Among them, path planning is in the middle link and has a decisive influence on the final stitching quality. For an image set collected by a camera, first, the spatial correspondence between different images is established through an image registration algorithm, including: a normalized correlation coefficient (NCC) between images, used to represent the matching accuracy between adjacent images; and a relative offset distance (ROD) between images, used to represent the relative displacement amount between adjacent images. Then, reasonable path planning is performed on all images for calculating the global coordinates of the images, and a proper image fusion algorithm is used to stitch and fuse the aligned images into a seamless panoramic image.

[0003] In the field of image stitching, path planning methods can be mainly divided into two categories: sequential path planning and unordered path planning, and the sequential path planning algorithm can be further divided into two typical modes: S-shaped path and Z-shaped path. Through the image registration algorithm, the system has already obtained the spatial relationship parameters between images. In S-shaped path planning, image 00 is set as the starting image, and the global coordinates of subsequent images are calculated step by step according to the ROD between images. For example, the global coordinates of image 01 can be calculated through the ROD between image 00 and image 01. However, the matching accuracy between images is high or low, and errors will be generated in stitching, which will gradually accumulate along the path. When image 10 is calculated, the accumulated error may cause significant deviation in calculating the global coordinates of the image, resulting in stitching misplacement (such as image 00 and image 10). Similarly, when the Z-shaped path planning reaches image 13, the accumulated error will cause errors in the global coordinates of the cross-column node (such as image 01 and image 13), resulting in stitching misplacement. Moreover, the sequential path planning method only considers the NCC of adjacent images, ignoring the adjacency relationship of other images with higher accuracy. The unordered path planning method considers the NCC size of all images, and preferentially selects the edges with the highest matching accuracy to construct the path, and calculates the global coordinates step by step along the optimal path. Although this method reduces the influence of single matching error, the path is usually long, and the error accumulates step by step, and the global coordinate offset of the end node may be large, causing stitching misplacement (such as image 01 and image 02).

[0004] Although the existing path planning method shows good stability in the conventional image stitching task, in the large-scale image stitching process, with the significant increase in the number of images, the path length also increases, thereby causing the error to gradually accumulate, and eventually possibly causing the local adjacent images to appear obvious stitching misalignment.

[0005] In the prior art, based on the sequential path planning method, only the single NCC value between the adjacent images is considered when propagating the position along the first column, and the more optimal connection path with other images is easily ignored. In addition, the outermost image usually contains less information, and the matching accuracy is not high, thereby further increasing the error of the position calculation. The minimum spanning tree path planning method adopted by MIST (Microscopy Image Stitching Tool) needs to consider all images, resulting in a too long path, and the longer path causes a larger error accumulation, thereby causing a large deviation of the global coordinates of the terminal image, and further causing the stitching misalignment problem with the adjacent image. SUMMARY

[0006] The present application aims to solve the technical problems of large error accumulation in path propagation in the path planning method and low accuracy in large-scale image stitching task. The present application provides a local minimum spanning tree path planning method supporting global coordinate updating, and the technical scheme adopted is:

[0007] A local minimum spanning tree path planning method supporting global coordinate updating, comprising the steps of:

[0008] S1, establishing an XY space coordinate system and an adjacency relationship for a plurality of collected images, grouping the plurality of images according to the XY space coordinate system and the adjacency relationship, and calculating the relative offset distance and the normalized correlation index between the images;

[0009] S2, selecting a starting group and a starting image of the starting group, constructing a minimum spanning tree based on the relative offset distance and the normalized correlation index using a minimum spanning tree algorithm, and calculating the global coordinates of the images in the starting group along the minimum spanning tree;

[0010] S3, in the current group and the adjacent group whose global coordinates have been solved, finding the image pair having a spatial adjacency relationship with the images in the current group, selecting the image pair with the largest normalized correlation index, taking the image whose global coordinates have been solved as the reference image, and taking the image whose global coordinates have not been solved as the starting image of the current group, calculating the global coordinates of the starting image of the current group according to the global coordinates of the reference image and the relative offset distance, updating the global coordinates of the remaining adjacent images in the current group, and calculating the global coordinates of the remaining images in the current group along the minimum spanning tree;

[0011] S4, processing the images of the remaining groups according to step S3 until the global coordinates of all images in the groups are calculated, and obtaining a path planning map.

[0012] Optionally, step S1 comprises: calculating the relative offset distance between the acquired images using a phase correlation registration algorithm.

[0013] wherein, for two reference images and a to-be-registered image of the same size, the positional relationship between the to-be-registered image and the reference images is expressed by the coordinates of the overlapping region as the basis of the upper left corner of the respective images as:

[0014] , (1);

[0015] In formula (1), the coordinates of the overlapping region of the reference images are represented by , the coordinates of the overlapping region of the to-be-registered image are represented by , and and represent the positional relationship between the to-be-registered image and the reference images.

[0016] For each pixel point in the overlapping region of the to-be-registered image, a corresponding pixel point in the reference image is found, which is represented as:

[0017] (2);

[0018] In formula (2), represents the reference image, represents the to-be-registered image, represents the horizontal coordinate in the spatial domain, represents the vertical coordinate in the spatial domain.

[0019] According to the Fourier shift theorem, we have:

[0020] (3);

[0021] (4);

[0022] In formula (3), represents the Fourier transform of the reference image, represents the Fourier transform of the to-be-registered image, represents the Fourier transform of the Dirac impulse function, represents the frequency domain coordinate of the image in the horizontal direction, corresponding to the horizontal coordinate of the spatial domain, representing the frequency component in the horizontal direction; represents the frequency domain coordinate of the image in the vertical direction, corresponding to the vertical coordinate of the spatial domain, representing the frequency component in the vertical direction. denotes the imaginary unit; in equation (4), the normalized cross-power spectrum is obtained by the exponential part, denotes the complex conjugate of

[0023] The two-dimensional Dirac delta impulse response function is obtained by calculating the inverse Fourier transform of the normalized cross-power spectrum, and is expressed as:

[0024] (5);

[0025] In equation (5), denotes the inverse Fourier transform, CPS denotes the normalized cross-power spectrum, is the two-dimensional Dirac delta impulse response function;

[0026] The relative offset distance between the to-be-registered image and the reference image is obtained by finding the peak position of the two-dimensional Dirac delta impulse response function , and the relative offset distance in the corresponding direction is denoted by ROD_X.

[0027] Optionally, step S1 further comprises: calculating the normalized correlation index using a normalized cross-correlation method in the spatial domain, and the calculation method is expressed as:

[0028] (6)

[0029] In equation (6), denotes the normalized correlation index, and denote the overlapping area images extracted from the reference image and the to-be-registered image according to the relative position, respectively, and denote the average pixel values of and , respectively, denotes the summation of all corresponding pixel positions.

[0030] Optionally, step S2 comprises:

[0031] S21, confirming a starting group, selecting an image with the smallest spatial coordinate in the starting group as a starting image, and initializing the global coordinates of the starting image;

[0032] S22, performing path planning using a minimum spanning tree algorithm, selecting an edge with the largest normalized correlation index to establish a connection, constructing a minimum spanning tree, and accumulating the global coordinates of the images in the starting group in the direction of the minimum spanning tree according to the relative offset distance in the direction.

[0033] Optionally, step S22 comprises:

[0034] S221, constructing a graph structure for the images in the starting group, each image block corresponding to a node in the graph structure, and the normalized correlation index serving as the weight of the edge of the graph structure;

[0035] S222, sorting all edges by weight from large to small, and using a union-find set to determine whether the two nodes connected by the edge form a closed loop, and sequentially selecting the edge with the largest weight and not forming a loop to construct a minimum spanning tree;

[0036] S223, in the direction of the minimum spanning tree, accumulating and calculating the global coordinates of the images in the starting group according to the relative offset distance in the direction.

[0037] Optionally, step S3 comprises:

[0038] S31, among the current group and the adjacent group in which the global coordinates have been solved, finding image pairs having a spatial adjacency relationship with the images in the current group, selecting the image pair with the largest normalized correlation index, taking the image in which the global coordinates have been solved as the reference image, and taking the image in which the global coordinates have not been solved as the starting image of the current group;

[0039] S32, calculating the global coordinates of the starting image of the current group according to the global coordinates of the reference image and the relative offset distance;

[0040] S33, updating and judging the global coordinates of the remaining adjacent images in the current group;

[0041] S34, taking the global coordinates of the starting image of the current group as the starting point, in the direction of the minimum spanning tree, accumulating and calculating the global coordinates of the remaining images according to the relative offset distance in the direction, until the global coordinates of all images are calculated.

[0042] Optionally, step S33 comprises:

[0043] For the remaining adjacent images in the current group, finding the largest normalized correlation index in four directions, and judging whether the adjacent image in the direction corresponding to the largest normalized correlation index belongs to the adjacent completed group; if it belongs to the adjacent completed group, updating the global coordinates of the current image through the adjacent image in the direction corresponding to the largest normalized correlation index, and marking it as actived; otherwise, the global coordinates of the current image are not updated temporarily.

[0044] Optionally, step S34 comprises:

[0045] If the state of the image has been marked as actived, the existing global coordinates are retained and not overwritten, and the global coordinates of the images located after the image marked as actived are calculated from the global coordinates of the image marked as actived, until the global coordinates of all images are calculated.

[0046] Optionally, the method of the present application further comprises: step S5, calculating the minimum value of the global coordinates, if there is a negative number, translating all the global coordinates as a whole, putting the images into the splicing board according to the order of the global coordinates, and fusing the splicing seam.

[0047] Optionally, step S5 comprises:

[0048] Let represent the pixel value of the fused image, whose coordinates are ; and respectively represent the pixel values of the two images, whose pixel coordinates are, the feathering weighting function is used to gradually fade out the processing of the overlapping area, and transition fusion is realized;

[0049] According to the contribution of the two images in the fusion, the weight is adjusted, which is related to the pixel position and is represented as:

[0050] Region;

[0051] Overlapping area (7);

[0052] Region;

[0053] In formula (7), the weight is represented, when only horizontal fusion is performed, the weight is only related to , and the calculation formula is represented as:

[0054] (8);

[0055] When only vertical fusion is performed, the weight is only related to , and the calculation formula is represented as:

[0056] (9);

[0057] In order to unify the weight formula, the weights of the two directions are combined and represented as:

[0058] (10);

[0059] In formula (8) and formula (9), , represent the width and height of the image to be fused.

[0060] The present application has the following beneficial effects:

[0061] The local minimum spanning tree path planning method supporting global coordinate updating provided by the application not only comprehensively considers NCC of all directions of an image, selects a higher-precision matching adjacency relationship, and effectively controls the length of path propagation by grouping the image, thereby reducing the cumulative effect of errors in the propagation process, reducing the probability of misalignment of adjacent image splicing, and realizing efficient splicing of large-scale images. The application supports flexible and adjustable image grouping methods and is compatible with various path planning strategies, such as an optimized S-shaped path planning method and a minimum spanning tree path planning method. In addition, the application also introduces a global coordinate updating mechanism to dynamically adjust the global position of the image in a local range, further improving the stability and accuracy of the splicing result. The application can be widely applied to medical image processing, satellite remote sensing imaging, industrial detection and other application scenarios requiring large-area image splicing, and has high engineering practical value and popularization potential. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is a flowchart of a local minimum spanning tree path planning method supporting global coordinate updating provided by an embodiment of the application;

[0063] Figure 2 is a schematic diagram of an image coordinate system provided by an embodiment of the application;

[0064] Figure 3 is NCC and ROD of a space coordinate (1, 1) image and four adjacent images provided by an embodiment of the application;

[0065] Figure 4 is a schematic diagram of a reference image and a to-be-registered image provided by an embodiment of the application;

[0066] Figure 5 is a default grouping result of an image provided by an embodiment of the application;

[0067] Figure 6 is a local minimum spanning tree path planning diagram provided by an embodiment of the application;

[0068] Figure 7 is a diagram for confirming a starting image and coordinate updating provided by an embodiment of the application;

[0069] Figure 8 is a path planning diagram provided by an embodiment of the application;

[0070] Figure 9 is a comparison diagram of a local minimum spanning tree path planning spliced and fused image (left) based on global coordinate updating and a global minimum spanning tree path planning spliced and fused image (right) provided by an embodiment of the application;

[0071] Figure 10 is a comparison chart of a local minimum spanning tree path planning image based on global coordinate update (left), an image based on an optimized S-shaped path planning (middle), and an image based on a global minimum spanning tree path planning (right) provided by an embodiment of the present application;

[0072] Figure 11 is a splicing schematic diagram in a large-scale splicing scenario provided by an embodiment of the present application. DETAILED DESCRIPTION

[0073] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0074] To reduce error accumulation of path propagation and improve the accuracy of large-scale image splicing tasks, the present application provides a local minimum spanning tree path planning method supporting global coordinate update, with reference to Figure 1 The method comprises the following steps:

[0075] S1, establishing an XY spatial coordinate system and an adjacency relationship for the collected multiple images, grouping the multiple images according to the XY spatial coordinate system and the adjacency relationship, and calculating the relative offset distance and the normalized correlation index between the images;

[0076] S2, confirming a starting group and a starting image of the starting group, constructing a minimum spanning tree based on the relative offset distance and the normalized correlation index using a minimum spanning tree algorithm, and calculating the global coordinates of the images in the starting group along the minimum spanning tree;

[0077] S3, in the current group and the adjacent group in which the global coordinates have been solved, finding an image pair having a spatial adjacency relationship with the images in the current group, selecting the image pair with the largest normalized correlation index, taking the image in which the global coordinates have been solved as a reference image, taking the image in which the global coordinates have not been solved as a starting image of the current group, calculating the global coordinates of the starting image of the current group according to the global coordinates of the reference image and the relative offset distance, updating the global coordinates of the remaining adjacent images in the current group, and calculating the global coordinates of the remaining images in the current group along the minimum spanning tree;

[0078] S4, processing the images of the remaining groups according to step S3 until the global coordinates of all the images in the groups are calculated, and obtaining a path planning image.

[0079] The local minimum spanning tree path planning method provided by the present application comprehensively considers the normalized correlation indexes of all directions of the images, selects the adjacency relationship with higher matching accuracy, effectively controls the length of path propagation by grouping the images, thereby reducing the cumulative effect of errors in the propagation process while ensuring the accuracy of image matching, and reducing the probability of mispositioning of adjacent image splicing.

[0080] Specifically, the application groups images according to a spatial coordinate system and an adjacency relationship, and can divide all images into several groups (Group), with 2x2 (a total of 4 images) in each group by default, and the remaining less than 4 images are grouped separately. The grouping rule can be freely set according to actual needs. Generally, the number of images in each group does not exceed the total number of images, that is, the minimum is a single image for a group, and the maximum is all images for a group. As shown in Figure 2 , the XY spatial coordinates of the first image are (0, 0), and the adjacency relationship is the left direction and the lower direction, and there are adjacent images in the left direction and the lower direction.

[0081] The phase correlation registration algorithm is used to calculate the ROD of the adjacent images, and the normalized cross-correlation method in the spatial domain is used to calculate the NCC between images. As shown in Figure 3 , the NCC and ROD of the image with spatial coordinates (1, 1) and four adjacent images are calculated, where NCC X represents the normalized correlation index in the corresponding direction; ROD X represents the relative offset distance in the corresponding direction. Specifically, NCC L represents the normalized correlation index in the left direction, NCC R represents the normalized correlation index in the right direction, NCC U represents the normalized correlation index in the upper direction, and NCC D represents the normalized correlation index in the lower direction; ROD L represents the relative offset distance in the left direction, ROD R represents the relative offset distance in the right direction, ROD U represents the relative offset distance in the upper direction, and ROD D represents the relative offset distance in the lower direction.

[0082] The main purpose of image registration is to determine the spatial correspondence between two images or multiple images. The phase correlation registration algorithm is a frequency domain-based image registration method, which is widely used to calculate the translation distance between two images. The phase correlation registration algorithm is used to calculate the relative offset distance between images. For details, refer to the specific implementation process of step S1 as shown below:

[0083] For two reference images and images to be registered with the same size, taking the coordinates of the overlapping region in the upper left corner of each image as the reference, referring to Figure 4 , Figure 4 , the left side is the reference image and the right side is the image to be registered. The positional relationship between the image to be registered and the reference image is expressed as:

[0084] , (1);

[0085] In formula (1), the coordinates of the overlapping region of the reference image are represented by , the coordinates of the overlapping region of the image to be registered are represented by , , and represent the positional relationship between the image to be registered and the reference image.

[0086] For each pixel in the overlap region of the image to be registered, find the corresponding pixel in the reference image, denoted as:

[0087] (2);

[0088] In formula (2), denotes the reference image, denotes the image to be registered, denotes the horizontal coordinate in the spatial domain, denotes the vertical coordinate in the spatial domain.

[0089] Generalize it to all regions of the image, according to the Fourier shift theorem:

[0090] (3);

[0091] (4);

[0092] In formula (3), denotes the Fourier transform of the reference image, denotes the Fourier transform of the image to be registered, denotes the Fourier transform of the Dirac impulse function, denotes the frequency domain coordinate of the image in the horizontal direction, corresponding to the horizontal coordinate in the spatial domain , denotes the frequency component in the horizontal direction; denotes the frequency domain coordinate of the image in the vertical direction, corresponding to the vertical coordinate in the spatial domain , denotes the frequency component in the vertical direction; denotes the imaginary unit.

[0093] In order to obtain and , the exponential part can be obtained by calculating the normalized cross power spectrum (CPS), and the formula is:

[0094] In formula (4), the exponential part is obtained by calculating the normalized cross power spectrum, denotes the complex conjugate of .

[0095] Therefore, the two-dimensional Dirac impulse corresponding function can be calculated by calculating the inverse Fourier transform of the normalized cross power spectrum, and the formula is:

[0096] (5);

[0097] In formula (5),​​ denotes inverse Fourier transform, CPS denotes normalized cross power spectrum, is the two-dimensional Dirac impulse response function.

[0098] The relative offset distance between the two images to be registered and the reference image is obtained by finding the peak position of the two-dimensional Dirac impulse response function , and the relative offset distance in the corresponding direction is denoted by ROD_X.

[0099] The normalized correlation method in the spatial domain is used to calculate the normalized correlation index, which is used to calculate the similarity of two images and represents the matching accuracy of adjacent images. The value range is between (0, 1), and the value closer to 1 indicates a higher similarity, and the value closer to 0 indicates a lower similarity, which can be used to judge the accuracy of the relative position under various translation interpretations. The calculation method is represented as:

[0100] (6);

[0101] In formula (6), denotes the normalized correlation index, and denote the overlapping area images extracted from the reference image and the image to be registered according to the relative position, respectively, and denote the average pixel values of and , respectively, denotes the summation of all corresponding pixel positions.

[0102] In an embodiment of the present application, all images are grouped, and the default grouping result of the images is referred to Figure 5 . Specifically, step S2 of the present application comprises:

[0103] S21, confirm the starting group, select the image with the smallest spatial coordinates (X, Y) in the starting group as the starting image, and initialize the global coordinates (global_x, global_y) of the starting image, so that the global coordinates (global_x, global_y) of the starting image are (0, 0). In this embodiment, Group1 is selected as the starting group;

[0104] S22, adopt the minimum spanning tree algorithm for path planning, select the edge with the largest normalized correlation index to establish connection, construct the minimum spanning tree, and calculate the global coordinates of the images in the starting group according to the relative offset distance in the direction along the minimum spanning tree. Specifically, the minimum spanning tree path planning is performed on Group1, and the local minimum spanning tree path planning diagram is referred to Figure 6And according to the ROD between images, the global coordinates of all images in the group are solved. The local of the application means grouping, the minimum spanning tree path planning is carried out for the current group, and then the global coordinates of the remaining images in the current group are calculated from the starting image according to the path.

[0105] In order to obtain the relative position relationship of the image blocks in the global coordinate system, Kruskal minimum spanning tree algorithm is used for path planning in the present application. The method of path planning by the minimum spanning tree algorithm of the application will be specifically described as follows:

[0106] S221, the image in the starting group is constructed into a graph structure, each image block in the graph structure corresponds to a node, and the normalized correlation index is used as the weight of the edge of the graph structure;

[0107] S222, all edges are sorted in descending order of weight, and whether the two nodes connected by the edge form a closed loop is judged by using the union-find set, and the edge with the maximum weight and not forming a loop is selected to build the minimum spanning tree, so that the whole graph is connected and the total matching error is minimized;

[0108] S223, the global coordinates of the images in the starting group are calculated according to the relative offset distance in the direction of the minimum spanning tree.

[0109] The union-find set is used to process the merging (union) and finding (find) operations between disjoint sets. In image stitching, image blocks (nodes) are connected by edges, and the weight of the edge is usually the normalized correlation index. In order to have good stitching effect, it is hoped to build a loop-free minimum spanning tree (MST) to avoid repeated connection between image blocks and form a closed loop.

[0110] Specifically, the implementation process of this step is as follows:

[0111] Initialization: each image block is an independent set.

[0112] Traverse the edge (sort by matching degree): for an edge connecting image blocks A and B, use Find(A) to find the representative set of A, and use Find(B) to find the representative set of B; if Find(A) = Find(B), it means that A and B have been connected through other paths, and adding this edge will form a loop (closed loop), so it cannot be added. Otherwise, you can use Union(A, B) to merge the two sets (add the edge to the spanning tree).

[0113] The specific method can refer to the following algorithm process:

[0114]

[0115] The step S3 of the application comprises:

[0116] S31, find the image pair with spatial adjacency relationship with the current group in the current group and the adjacent group which has completed global coordinate solving, select the image pair with the maximum normalized correlation index, take the image which has completed global coordinate solving as the reference image, and take the image which has not completed global coordinate solving as the starting image of the current group. As shown in the following figure, Figure 7 Group 1 and Group 2 belong to adjacent images, and the NCC indexes are 0.95 and 0.99 respectively. Since the image with coordinates (2, 1) corresponds to the maximum NCC value 0.99, the image with coordinates (2, 1) is taken as the starting image of Group 2.

[0117] S32, calculate the global coordinates of the starting image of the current group according to the global coordinates of the reference image and the relative offset distance.

[0118] Since each group after grouping is basically in an isolated state, it needs to be processed respectively. For Group 1, set its starting image as the image with the smallest horizontal and vertical coordinates in the group, and take it as the origin (0, 0) of the spatial coordinate system, that is, provide the starting point for the minimum spanning tree path planning. Subsequently, according to the relative offset relationship between the images, the global coordinates of all images in the group are calculated along the minimum spanning tree path. However, when processing Group 2, the problem is that it lacks a clear starting image, so it cannot directly perform minimum spanning tree path planning. If a starting image is directly initialized for Group 2, its global coordinate system will be independent of Group 1, resulting in inconsistent overall coordinate system. In order to solve this problem, the global coordinates of Group 1 which have been solved are used to connect with Group 2. Since there is often an image matching relationship between adjacent groups, the starting image in Group 2 can be selected by matching accuracy. Specifically, select the image with the highest matching accuracy as the starting image of Group 2. Then, with the help of the global coordinates of the corresponding image in Group 1 and the offset relationship between the two images, the global coordinates of the starting image of Group 2 are calculated. Next, the minimum spanning tree path planning is performed according to the starting point, and the global coordinates of the remaining images in Group 2 are calculated in the order of the path, and updated according to whether the state is activated (actived).

[0119] Assuming that the global coordinates of image block 1 are (0, 0), according to the relative offset distance relationship between the image block to the right of image block 1 and image block 1, the global coordinates of the image block to the right of image block 1 can be solved as (0+ , 0+ ).

[0120] The purpose of steps S31 and S32 is to set a starting point for the minimum spanning tree path of the current group.

[0121] S33, update judgment is made on the global coordinates of the remaining neighboring images in the current group.

[0122] The specific method for updating the global coordinates of the remaining neighboring images is as follows: for the remaining neighboring images in the current group, the maximum normalized correlation index in four directions is found, and it is determined whether the neighboring image in the direction corresponding to the maximum normalized correlation index belongs to the neighboring completed group; if it belongs to the neighboring completed group, the global coordinate of the current image is updated through the neighboring image in the direction corresponding to the maximum normalized correlation index, and is marked as actived; otherwise, the global coordinate of the current image is not updated.

[0123] Specifically, global coordinate update judgment is made on the image pairs other than the image pair of Group 2. If the NCC index of an image is the maximum NCC index of the image in four directions, the global coordinate of the image should also be propagated through the connection relationship. As shown in Figure 7 , the images whose global coordinates are updated through the neighboring group are marked with dashed arrows.

[0124] S34, taking the global coordinate of the starting image of the current group as the starting point, the global coordinates of the remaining images are calculated according to the relative offset distance in the direction of the minimum spanning tree until the global coordinates of all images are calculated.

[0125] If the state of an image has been marked as actived, the existing global coordinate is retained and not overwritten, and the global coordinates of the images after the image marked as actived are calculated from the global coordinate of the image marked as actived until the global coordinates of all images are calculated.

[0126] The minimum spanning tree path planning is performed for Group 2. Since the image with spatial coordinates (2.0) has been updated, the global coordinate of the image is retained and not calculated according to the path.

[0127] The images of the remaining groups are processed according to step S3 until the global coordinates of all images in all groups are calculated, and a path planning graph is obtained. The operation of step S3 is performed for all groups, and the final local minimum spanning tree path planning graph is shown in Figure 8 . As can be seen from Figure 8 , the path length of the tree is effectively limited.

[0128] This is embodied in the following aspects: on the one hand, from Figure 8The spatial correlation between images in each group is enhanced by grouping processing, and each individual group performs minimum spanning tree path planning, which avoids the path breaking or misalignment problems caused by the traditional global minimum spanning tree method that may only expand along the optimal path and skip adjacent images. On the other hand, since each group of path planning is performed in a local range, the path length is limited, thereby limiting the cumulative propagation of errors. More importantly, between multiple image groups, a global coordinate updating mechanism is introduced: the starting image of each new group is no longer independently initialized, but is connected through the matching accuracy between the solved groups, and the initial coordinates of the current group are accurately derived by means of the known global coordinates and relative offset relationship. This mechanism ensures the consistency and continuity of multiple local paths in the global space, and realizes the path planning of the stitching that is not divided but united, accurate locally and unified globally. Therefore, while improving the robustness of stitching, the method effectively controls the complexity of the path structure and the range of error propagation, which is an important improvement and promotion of the traditional global path planning model.

[0129] Image fusion is the last step of image stitching. The main role of image fusion in the image stitching process is: one is to combine the same content of multiple images into one image, and the other is to make the transition of the image stitching more uniform and eliminate the obvious seams caused by image stitching. In an embodiment of the present application, the method of the present application further comprises step S5: calculating the minimum value of the global coordinates, if there is a negative number, translating all global coordinates as a whole, placing the images on the stitching board according to the order of the global coordinates, and fusing the stitching seam.

[0130] The purpose of step S5 is to ensure that all image blocks do not exceed the boundary when drawn to the stitching canvas. Since the global coordinates are calculated by accumulating the relative displacement, there may be negative values. If these coordinates are directly used to draw on the canvas, it will cause array out-of-bounds or image data writing to illegal areas, causing program crashes or stitching errors. Therefore, it is necessary to detect whether there are negative values in all image block global coordinates, and if there are, uniformly translate them as a whole to make all coordinates non-negative, ensuring that all image blocks can be correctly placed and fused on the canvas with the top left corner as the origin. The entire stitched image can be imagined as a large puzzle board, and each small image block is like a puzzle piece. The specific position of each puzzle piece on the entire puzzle board, i.e. the global coordinates, has been calculated by the registration algorithm. According to these positions, the puzzle pieces are placed one by one.

[0131] The present application adopts feathering weighting algorithm to fuse the stitching seam, and the specific method is as follows:

[0132] Let represent the pixel value of the fused image, and its coordinates are ; and represent the pixel values of the two images, respectively, For its pixel coordinates, the feathering weighting function is used to gradually fade out the overlapping area, realizing transition fusion.

[0133] According to the contribution of the two images in the fusion, the weight is adjusted according to the pixel position, which is expressed as:

[0134] Region;

[0135] Overlapping area (7)

[0136] Region;

[0137] In formula (7), The weight is only related to In the horizontal direction fusion only, the calculation formula is expressed as:

[0138] (8);

[0139] In the vertical direction fusion only, the weight is only related to The calculation formula is expressed as:

[0140] (9);

[0141] In order to unify the weight formula, the weights of the two directions Are combined and expressed as:

[0142] (10);

[0143] In formula (8) and formula (9), , Indicate the width and height of the image to be fused.

[0144] After the global coordinates of all images are calculated, the minimum values of global_x and global_y are calculated. If there is a negative number, the coordinate system is translated as a whole, so that the global coordinates are normalized to a unified coordinate system, and it is ensured that the global coordinates are all non-negative values. As shown in Figure 9 The image spliced and fused based on the local minimum spanning tree path planning method updated based on the global coordinates is compared with the image spliced and fused based on the global minimum spanning tree path planning, and the splicing effect is obviously improved. Figure 9In the figure, the red circle marked area is the image splicing seam position, when the global minimum spanning tree based path planning is used for splicing and fusion, the obvious misplacement phenomenon occurs at the splicing seam; compared with the prior art, after the method of the application is used, the splicing accuracy is improved significantly, and the misplacement problem is effectively alleviated.

[0145] The global minimum spanning tree path planning method is the main path planning strategy adopted in the microscopic image splicing tool developed by the National Institute of Standards and Technology (NIST). The technology builds a global graph structure for all image blocks, takes the matching accuracy between the blocks as the edge weight, and constructs a minimum spanning tree covering all the blocks. Then, the relative position information is propagated from the specified starting image according to the minimum spanning tree path, and the global coordinates of each block are calculated to complete the image splicing.

[0146] Unlike the prior art, the application proposes a local minimum spanning tree path planning method based on global coordinate updating. The core of the method is: first, the image set composed of multiple images is structurally grouped, a minimum spanning tree is constructed in each group of images, and local path planning is performed. Through grouping, the length of the path can be effectively limited. More importantly, during the inter-group connection process, the method does not directly initialize an independent global coordinate for the new group, but establishes a connection with the high matching accuracy images between the already positioned image groups, accurately transfers the known global coordinates to the starting image of the new group through the relative offset relationship, and expands the subsequent path planning and coordinate calculation based on this. This design effectively improves the accuracy and robustness of splicing, and is particularly suitable for weak texture, large size, and non-homogeneous image set splicing tasks.

[0147] Therefore, the application has substantial differences in path planning method, coordinate initialization mechanism, error control method, and inter-group connection strategy from the existing global MST (Minimum Spanning Tree) splicing method, and does not belong to the simple combination or equivalent replacement of the prior art, and has significant innovation and practical value.

[0148] In order to verify the robustness of the method of the application in complex scenes such as weak texture and low matching accuracy, a splicing comparison experiment as shown in Figure 10 is designed. Only three path planning methods are used for splicing test on weak texture images, and no fusion processing is performed. The experiment shows that the local minimum spanning tree path planning achieves comparable splicing effect to the minimum spanning tree in such scenes, which is significantly better than the optimized S-shaped path planning and the global minimum spanning tree path planning based on the global minimum spanning tree path planning, and the specific results are shown in Figure 10 . Figure 10 The left side is the result of splicing using the method of the application, Figure 10 the middle is the result of splicing based on the optimized S-shaped path planning method, Figure 10The right side is the result based on global minimum spanning tree (MST) splicing.

[0149] It can be observed from the experimental results that the present application has stronger robustness in weak texture areas, can significantly inhibit the splicing misplacement phenomenon caused by local error propagation, and the overall splicing accuracy is obviously improved compared with the S-shaped path planning method. It should be noted that in some extreme cases, even if the matching accuracy is low or the image grouping boundary is blurred, the worst-case splicing result of the present application can guarantee the accuracy comparable to the global minimum spanning tree method, avoiding the error accumulation and structure misplacement problems commonly seen in the S-shaped path planning method. This shows that the present application is not only superior to some existing methods in robustness, but also has good stability and practical value in complex scenes. And the present application still maintains high splicing accuracy in such cases, effectively inhibiting the splicing misplacement problem caused by error accumulation.

[0150] Experiments show that the method of the present application can still maintain high splicing accuracy in most splicing scenes. In order to verify the effectiveness of the method, a data set containing a large number of overlapping image blocks is selected for testing. The method proposed in the patent is used in the splicing process, and the final splicing result is shown in FIG. 2. Figure 11 Figure 11 The left image is the original image block distribution, and the right image is the final splicing result, which can be observed that there is no obvious misplacement or ghosting phenomenon between images, which can prove that the method still has good stability and accuracy performance under the condition of a large number of images and complex overlap.

[0151] The method of the present application can be used for large-scale image splicing tasks, improving the overall splicing accuracy and efficiency, supporting different sizes of image grouping, and having good flexibility. When the grouping size is large enough to cover all images, the path planning effect is equivalent to the global minimum spanning tree algorithm. The present application has wide application prospects and can be used in medical image processing, satellite remote sensing imaging, industrial detection and other fields with high requirements for large-scale image splicing.

[0152] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any modification, equivalent replacement and improvement made by any person skilled in the art within the technical scope disclosed by the present application shall be covered within the protection scope of the present application.​

Claims

1. A local minimum spanning tree path planning method supporting global coordinate updates, characterized in that, Including the following steps: S1. Establish an XY spatial coordinate system and adjacency relationship for multiple acquired images, group the multiple images according to the XY spatial coordinate system and adjacency relationship, and calculate the relative offset distance and normalized correlation index between the images; S2. Select the starting group and the starting image of the starting group. Based on the relative offset distance and the normalized correlation index, construct the minimum spanning tree using the minimum spanning tree algorithm, and calculate the global coordinates of the images in the starting group along the minimum spanning tree. S3. In the current group and the adjacent groups that have completed the global coordinate solution, find the image pair that has a spatial adjacency relationship with the image in the current group, select the image pair with the largest normalized correlation index, take the image that has completed the global coordinate solution as the reference image, and take the image that has not completed the global coordinate solution as the starting image of the current group. Calculate the global coordinates of the starting image of the current group based on the global coordinates of the reference image and the relative offset distance, update the global coordinates of the remaining adjacent images in the current group, and calculate the global coordinates of the remaining images in the current group along the minimum spanning tree. S4. Process the remaining images according to step S3 until the global coordinates of all images in all groups are calculated to obtain the path planning map.

2. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 1, characterized in that, Step S1 includes: calculating the relative offset distance between the acquired images using a phase correlation registration algorithm; For two reference images and images to be registered of the same size, the positional relationship between the image to be registered and the reference image is expressed by the coordinates of the overlapping region at the upper left corner of each image, based on the coordinates of the overlapping region: ， （1）； In formula (1), the coordinates of the overlapping region of the reference image are used. The coordinates of the overlapping region of the images to be registered are represented by... express, and This indicates the positional relationship between the image to be registered and the reference image; For each pixel within the overlapping region of the image to be registered, a corresponding pixel is found in the reference image, represented as follows: （2）； In formula (2), Indicates a reference image. This represents the image to be registered. The horizontal coordinate represents the spatial domain. The vertical coordinate represents the spatial domain; According to Fourier's displacement theorem: （3）； （4）； In formula (3), Represents the Fourier transform of the reference image. This represents the Fourier transform of the images to be registered. Represents the Fourier transform of the Dirac impulse function. This represents the frequency domain coordinates of the image in the horizontal direction, corresponding to the horizontal coordinates in the spatial domain. , representing the frequency component in the horizontal direction; This represents the frequency domain coordinates of the image in the vertical direction, corresponding to the ordinate in the spatial domain. , representing the frequency component in the vertical direction; The imaginary unit is represented; in formula (4), the normalized cross-power spectrum is obtained. The index section, express The complex conjugate; The inverse Fourier transform of the normalized cross-power spectrum yields the two-dimensional Dirac impulse response function, expressed as follows: （5）； In formula (5), This represents the inverse Fourier transform. CPS Represents the normalized cross-power spectrum. It is a two-dimensional Dirac impulse response function; By searching for the two-dimensional Dirac impulse response function The peak position is the relative offset distance between the image to be registered and the reference image. The relative offset distance in the corresponding direction is represented by ROD_X.

3. A local minimum spanning tree path planning method supporting global coordinate updates according to claim 2, characterized in that, Step S1 further includes: calculating the normalized correlation index using a normalized cross-correlation method in the spatial domain, the calculation method being expressed as: （6） In formula (6), Indicates the normalized correlation index. and These represent the overlapping region images extracted from the reference image and the image to be registered, respectively, based on their relative positions. and They represent and The average pixel value, This indicates that the summation is performed over all corresponding pixel positions.

4. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 1, characterized in that, Step S2 includes: S21. Confirm the starting group, select the image with the smallest spatial coordinates in the starting group as the starting image, and initialize the global coordinates of the starting image; S22. The minimum spanning tree algorithm is used for path planning. The edge with the largest normalized correlation index is selected to establish a connection and construct a minimum spanning tree. The global coordinates of the image in the initial group are calculated by accumulating the relative offset distance along the direction of the minimum spanning tree.

5. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 4, characterized in that, Step S22 includes: S221. Construct a graph structure for the images in the initial group, wherein each image block in the graph structure corresponds to a node, and the normalized correlation index is used as the weight of the edge of the graph structure. S222. Sort all edges in descending order of weight, and use disjoint-set data structure to determine whether the two nodes connected by the edge form a closed cycle. Select the edge with the largest weight that does not form a cycle to construct the minimum spanning tree. S223. Along the direction of the minimum spanning tree, calculate the global coordinates of the images in the initial group by accumulating the relative offset distance in that direction.

6. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 1, characterized in that, Step S3 includes: S31. In the current group and the adjacent groups that have completed the global coordinate solution, find the image pair that has a spatial adjacency relationship with the image in the current group, select the image pair with the largest normalized correlation index, take the image that has completed the global coordinate solution as the reference image, and take the image that has not completed the global coordinate solution as the starting image of the current group. S32. Calculate the global coordinates of the starting image of the current group based on the global coordinates of the reference image and the relative offset distance; S33. Update and determine the global coordinates of the remaining adjacent images in the current group; S34. Starting from the global coordinates of the initial image of the current group, along the direction of the minimum spanning tree, calculate the global coordinates of the remaining images by accumulating the relative offset distance in that direction until the global coordinates of all images have been calculated.

7. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 6, characterized in that, Step S33 includes: For the remaining adjacent images in the current group, find the largest normalized correlation index in four directions, and determine whether the adjacent images in the direction corresponding to the largest normalized correlation index belong to the adjacent completion group. If they belong to the adjacent completion group, update the global coordinates of the current image through the adjacent images in the direction corresponding to the largest normalized correlation index, and mark it as active. Otherwise, do not update the global coordinates of the current image for the time being.

8. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 7, characterized in that, Step S34 includes: If an image's state has been marked as active, its existing global coordinates are preserved and not overwritten. Images following the marked active image will use the global coordinates of that marked active image as a starting point to calculate their global coordinates, and so on, until the global coordinates of all images have been calculated.

9. The local minimum spanning tree path planning method supporting global coordinate updates according to claim 1, characterized in that, It also includes: step S5, calculating the minimum value of the global coordinates. If there is a negative value, all global coordinates are translated as a whole, the image is placed into the splicing plate according to the order of the global coordinates, and the splicing seams are merged.

10. A local minimum spanning tree path planning method supporting global coordinate updates according to claim 9, characterized in that, Step S5 includes: make The pixel values ​​of the merged image are represented by the coordinates of... ; and These represent the pixel values ​​of the two images, respectively. Using its pixel coordinates, a feathering weighted function is applied to the overlapping area to perform a gradual in-and-out process, achieving a smooth blending effect. The weights are adjusted according to the contributions of the two images in the fusion process. These weights are related to the pixel positions and are expressed as follows: area; Overlapping region (7); area; In formula (7), The weights represent the values ​​that, when merging only in the horizontal direction, are only related to the horizontal direction. The relevant calculation formula is expressed as follows: （8）； When merging only in the vertical direction, the weights are only related to... The relevant calculation formula is expressed as follows: （9）； To standardize the weighting formula, the weights in both directions are... Merging, represented as: （10）； In formulas (8) and (9), , This represents the width and height of the images to be merged.

Images