Image stitching method and device, computer device and storage medium

Abstract

The application relates to an image splicing method and device, computer equipment and a storage medium. An undirected graph is established based on the adjacency relationship between to-be-spliced images; a bidirectional splicing operation is performed on a splicing pair, edges between splicing pairs with splicing failure or a deviation of bidirectional splicing results exceeding a first threshold value are deleted; for a splicing pair with successful bidirectional splicing and a deviation of bidirectional splicing results within the first threshold value range, an offset vector between the splicing pair is obtained; for each edge of the undirected graph, a closed loop path containing the current edge is constructed, offset vector residuals of all closed loop paths containing the current edge are calculated, and a minimum value of the norm of the offset vector residuals is taken as the current edge weight; all connected subgraphs of the undirected graph are identified, optimal splicing paths corresponding to all connected subgraphs are extracted, and each connected subgraph is spliced according to the corresponding optimal splicing path; the connected subgraph with the largest number of nodes is taken as a main splicing graph, and other connected subgraphs are spliced onto the main graph, thereby improving the accuracy of image splicing.

Description

Technical Field

[0001] This application relates to the field of computer processing technology, and in particular to an image stitching method, apparatus, computer device, and storage medium. Background Technology

[0002] Cell morphology analysis is a crucial diagnostic method in medical testing, requiring physicians to diagnose diseases based on the size, shape, and distribution of cells in cell images. Traditional cell morphology examinations primarily rely on manual observation under a microscope, field by field. This method suffers from limitations in individual fields of view and strong subjectivity in slide interpretation, severely impacting the comprehensiveness and consistency of pathological assessment. In recent years, with the development of digital pathology and image processing technologies, whole-slide images (WSI), capable of large-scale information fusion and presenting continuous information across multiple fields of view, have become a key means to improve diagnostic efficiency, providing a data foundation for subsequent artificial intelligence analysis and assisted diagnosis. A crucial step in obtaining whole-slide images is stitching together multi-field cell images captured by an electronic scanning platform. Currently, the commonly used image stitching method is feature matching. However, cell images typically exhibit characteristics such as large numbers, irregular image overlap, uneven cell density, significant staining differences, and numerous blank fields. Feature matching suffers from poor stitching results when handling such complex image scenarios. Existing technologies lack image stitching methods for handling such complex image scenarios, limiting the feasibility of applying digital pathology technology to large-scale clinical data.

[0003] In existing technologies, feature matching methods for stitching cell images to obtain a full-view image suffer from poor stitching results, and no effective solution has yet been proposed. Summary of the Invention

[0004] Therefore, it is necessary to provide an image stitching method, apparatus, computer equipment, and storage medium to address the aforementioned technical problems, thereby resolving the issue of poor stitching results in cell image stitching using feature matching methods in related technologies.

[0005] In a first aspect, embodiments of this application provide an image stitching method, the method comprising:

[0006] Obtain the images to be stitched together, and establish an undirected graph based on the adjacency relationships between the images to be stitched together;

[0007] A bidirectional stitching operation is performed on the stitching pairs. In an undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from a first threshold are deleted. For stitching pairs that successfully stitch and whose bidirectional stitching results deviate from the first threshold, offset vectors between the stitching pairs are obtained. The stitching pairs are images to be stitched that are adjacent to each other. The offset vectors include horizontal offsets and vertical offsets.

[0008] For each edge of the undirected graph, construct a closed-loop path containing the current edge, calculate the offset vector residuals of all closed-loop paths containing the current edge, and take the minimum value of the offset vector residual norm as the weight of the current edge.

[0009] The depth-first search method is used to identify all connected subgraphs of the undirected graph. The minimum spanning tree algorithm is used to extract the optimal splicing path corresponding to all connected subgraphs. The connected subgraphs are then spliced ​​together according to the corresponding optimal splicing path.

[0010] The connected subgraph with the most nodes is used as the main graph for stitching. Other connected subgraphs are stitched onto the main graph based on the offset vector estimated by the scanning parameters of the electronic scanning platform or the offset vector estimated by the existing stitching results.

[0011] In some embodiments, performing a bidirectional splicing operation on the splicing pair includes the following steps:

[0012] A phase correlation algorithm based on the Fourier domain is used to perform a bidirectional splicing operation on the splicing pair;

[0013] If the confidence level of the splicing result is lower than the set second threshold, a template matching-based splicing method is used to perform a bidirectional splicing operation on the splicing pair.

[0014] In some embodiments, the extraction of the optimal concatenation path corresponding to all connected subgraphs based on the minimum spanning tree algorithm includes the following steps:

[0015] For each of the connected subgraphs, perform the Prim minimum spanning tree algorithm or the Kruskal minimum spanning tree algorithm to extract the optimal splicing path corresponding to the connected subgraph.

[0016] Secondly, this application also provides an image stitching device, which includes an image creation module, a filtering module, a calculation module, an extraction module, and a stitching module;

[0017] The establishment module is used to acquire images to be stitched together and establish an undirected graph based on the adjacency relationship between the images to be stitched together.

[0018] The filtering module is used to perform bidirectional stitching operations on stitching pairs. In an undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from a first threshold are deleted. For stitching pairs that successfully stitch and whose bidirectional stitching results deviate from the first threshold, the offset vector between the stitching pairs is obtained. The stitching pairs are images to be stitched that are adjacent to each other. The offset vector includes a horizontal offset and a vertical offset.

[0019] The calculation module is used to construct a closed-loop path containing the current edge for each edge of the undirected graph, calculate the offset vector residuals of all closed-loop paths containing the current edge, and take the minimum value of the offset vector residual norm as the weight of the current edge.

[0020] The extraction module is used to identify all connected subgraphs of the undirected graph using a depth-first search method, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path.

[0021] The stitching module is used to take the connected subgraph with the most nodes as the main stitching graph, and stitch other connected subgraphs onto the main graph based on the offset vector estimated by the scanning parameters of the electronic scanning platform or the offset vector estimated by the existing stitching results.

[0022] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0023] Fourthly, in this embodiment, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0024] The aforementioned image stitching method, apparatus, computer device, and storage medium acquire images to be stitched and establish an undirected graph based on the adjacency relationships between the images; perform bidirectional stitching operations on stitching pairs; in the undirected graph, delete edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate beyond a first threshold; for stitching pairs that successfully stitch bidirectionally and whose bidirectional stitching results deviate within the first threshold range, obtain offset vectors between the stitching pairs; the stitching pairs are images to be stitched that are adjacent to each other; the offset vectors include horizontal offsets and vertical offsets; for each edge of the undirected graph, construct a method that includes the offset vectors for each edge of the image stitching pair. For the preceding closed-loop paths, calculate the offset vector residuals of all closed-loop paths containing the current edge, and use the minimum norm of the offset vector residuals as the weight of the current edge; use a depth-first search method to identify all connected subgraphs of the undirected graph, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path; take the connected subgraph with the most nodes as the splicing main graph, and splice the other connected subgraphs onto the main graph based on the offset vectors estimated by the scanning parameters of the electronic scanning platform or based on the offset vectors estimated by the existing splicing results. This application establishes an undirected graph model for the images to be stitched, performs image registration verification based on bidirectional stitching, introduces the offset vector residual of the closed-loop path to constrain the undirected graph, and utilizes the depth-first search method and minimum spanning tree algorithm to extract the optimal stitching path. This constructs a widely applicable and highly scalable image stitching workflow framework, significantly improving the problems of low accuracy and severe error accumulation in traditional stitching methods for large-scale image processing. It effectively enhances the accuracy and robustness of image stitching, providing key technical support for constructing high-quality full-view images, further promoting the development of digital pathology and intelligent assisted diagnosis, and has broad prospects for clinical application and promotion. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0026] Figure 1 This is an application scenario diagram of the image stitching method provided in the embodiments of this application;

[0027] Figure 2 This is a flowchart of an image stitching method provided according to an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of an undirected graph established based on the adjacency relationship between images to be stitched in the image stitching method provided in the embodiments of this application;

[0029] Figure 4This is a schematic diagram of an undirected graph obtained after deleting the edges between stitching pairs that fail to stitch or whose deviation in bidirectional stitching results exceeds a first threshold, according to the image stitching method provided in the embodiments of this application.

[0030] Figure 5 A schematic diagram of the image stitching device provided in the embodiments of this application;

[0031] Figure 6 A schematic diagram of the structure of a computer device provided in the embodiments of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0033] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0034] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0035] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0036] Figure 1 This is an application scenario diagram of an image stitching method provided in one embodiment of this application. For example... Figure 1As shown, data transmission between server 101 and mobile terminal 102 can be performed via network. Mobile terminal 102 is used to acquire cell images and transmit them to server 101. After receiving the cell images, server 101 establishes an undirected graph based on the adjacency relationships between the images to be stitched; it performs bidirectional stitching operations on the stitching pairs; in the undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate beyond a first threshold are deleted; for stitching pairs that successfully stitch bidirectionally and whose bidirectional stitching results deviate within the first threshold range, the offset vector between the stitching pairs is obtained; the stitching pair consists of images that are adjacent to each other; the offset vector includes horizontal and vertical offsets; for each edge of the undirected graph, a closed-loop path containing the current edge is constructed, and the path containing the current edge is calculated. For offset vector residuals with closed-loop paths, the minimum norm of the offset vector residuals is used as the weight of the current edge. A depth-first search method is used to identify all connected subgraphs of the undirected graph. The optimal stitching path for each connected subgraph is extracted based on the minimum spanning tree algorithm, and the connected subgraphs are stitched together according to their corresponding optimal stitching paths. The connected subgraph with the most nodes is used as the main stitching graph. Other connected subgraphs are stitched onto the main graph based on the offset vectors estimated by the scanning parameters of the electronic scanning platform or based on the offset vectors estimated by existing stitching results. Server 101 can be implemented by a standalone server or a server cluster consisting of multiple servers, and mobile terminal 102 can be any type of image acquisition device.

[0037] This application provides an image stitching method, such as... Figure 2 As shown, the method includes the following steps:

[0038] Step S210: Obtain the images to be stitched and establish an undirected graph based on the adjacency relationships between the images to be stitched.

[0039] Assuming there are 25 images to be stitched together, arranged in a 5x5 pattern, then Figure 3 A schematic diagram of the undirected graph established based on the adjacency relationship between the images to be stitched in the image stitching method provided in this application is shown below. Figure 3 As shown, taking the image in the second row and third column, i.e., the image H with the index (1,2), as an example, the images adjacent to the image H in the second row and third column (1,2) include the image C in the first row and third column (0,2), the image G in the second row and second column (1,1), the image I in the second row and fourth column (1,3), and the image M in the third row and third column (2,2). Therefore, the image H in the second row and third column (1,2) has adjacent edges with the image C in the first row and third column (0,2), the image G in the second row and second column (1,1), the image I in the second row and fourth column (1,3), and the image M in the third row and third column (2,2).

[0040] Step S220: Perform bidirectional stitching operation on the stitching pairs. In the undirected graph, delete the edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from the first threshold. For stitching pairs that successfully stitch and whose bidirectional stitching results deviate from the first threshold, obtain the offset vector between the stitching pairs. The stitching pairs are images to be stitched that are adjacent to each other. The offset vector includes horizontal offset and vertical offset.

[0041] See also Figure 3 The images adjacent to image H in the second row and third column (1,2) include image C in the first row and third column (0,2), image G in the second row and second column (1,1), image I in the second row and fourth column (1,3), and image M in the third row and third column (2,2). Therefore, image H and image C are a stitching pair, image H and image G are a stitching pair, image H and image I are a stitching pair, image H and image M are a stitching pair, and so on, so that all stitching pairs can be determined. Assuming the stitching pair includes image ① and image ②, for any stitching pair, a bidirectional stitching operation is performed using existing stitching methods. First, image ① is stitched into image ② as a reference, then image ② is stitched into image ① as a reference. If stitching image ② with image ① as a reference fails, or stitching image ① with image ② as a reference fails, or the deviation between the stitching result of image ② with image ① as a reference and the stitching result of image ① with image ② as a reference exceeds a first threshold, then the edge between image ① and image ② in the undirected graph is deleted. Finally, only stitching pairs with high consistency in bidirectional stitching results are retained as valid registration results to improve the stability and accuracy of the stitching. Specifically, continuing with... Figure 3 Taking an undirected graph as an example, suppose the stitching pairs CG, PQ, VM, QR, RM, SN, and TO fail to stitch or the deviation of the bidirectional stitching results exceeds a first threshold. The resulting undirected graph after deleting the edges between stitching pairs that fail to stitch or whose bidirectional stitching results have a deviation exceeding the first threshold is as follows: Figure 4 As shown.

[0042] In addition, for stitching pairs where bidirectional stitching is successful and the deviation of the bidirectional stitching result is within the first threshold range, it is necessary to obtain the offset vector between the stitching pairs, that is, the offset vector (X) used to stitch image ② with image ① as the reference. ①② ,Y ①② ), and the offset vector (X) used to stitch image ① together with image ② as a reference. ②① ,Y ②① ).

[0043] As one implementation method, for each stitching pair containing image ① and image ②, a phase correlation algorithm based on the Fourier domain can be used to perform a bidirectional stitching operation on the stitching pair first; if the confidence of the stitching result is lower than a set second threshold, a template matching-based stitching method is used to perform a bidirectional stitching operation on the stitching pair.

[0044] Specifically, based on the scanning parameters of the electronic scanning platform, the overlapping area between each image and its neighboring images can be initially estimated. Within the allowable error range, the edge regions of the two images are cropped as registration regions, referencing the overlapping area. A phase correlation algorithm based on the Fourier domain is first used to perform a bidirectional stitching operation on the stitched pair. If the confidence level of the stitching result is lower than a set second threshold, a template matching-based stitching method is used to perform a bidirectional stitching operation on the stitched pair. To improve matching accuracy, the Canny edge detection algorithm can be used to extract regions with complex textures in the image, and the most discriminative sliding window region can be selected as a template based on image integration for high-precision matching.

[0045] Step S230: For each edge of the undirected graph, construct a closed-loop path containing the current edge, calculate the offset vector residuals of all closed-loop paths containing the current edge, and take the minimum value of the offset vector residual norm as the weight of the current edge.

[0046] See also Figure 4 Taking the edge between image A and image B as an example, closed-loop paths containing edge AB include ABGFA, ABGLKFA, ABGLQVUPKFA, ABCDIHGFA, ABCDINMHGFA, ABCDINMLGFA, ABCDINMLKFA, ABCDEJIHGFA, ABCDEJONIHGFA, ABCDEJONMHGFA, ABCDEJONMLGFA, ABCDEJONMLKFA, etc. Taking the closed-loop path ABGFA as an example, the offset vector residual of the closed-loop path ABGFA is equal to the offset vector (X) used to stitch image B with image A as the reference. AB ,Y AB Image B is the reference image G, and the offset vector (X) is used in the stitched image G. BG ,Y BG The image G is the offset vector (X) used to stitch the image F to the reference image G. GF ,Y GF ) + like F as the reference image A, the offset vector (X) used in the stitched image. FA ,Y FA For edge AB, calculate the offset vector residuals of all closed-loop paths containing the current edge, and use the minimum norm of the offset vector residuals as the weight of edge AB. Similarly, obtain the weights of all edges. Assuming the offset vector residual is (X, Y), then the norm of the offset vector residual is sqrt(X...).2 +Y 2 ).

[0047] Step S240: Use the depth-first search method to identify all connected subgraphs of the undirected graph, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path.

[0048] Continue with Figure 4 For example, the connected subgraphs identified using the depth-first search method include two: ABCDEJIHFKLMNOPQUV and RSTWXY. After knowing the weight of each edge through step S230, the optimal stitching path corresponding to all connected subgraphs can be extracted based on the minimum spanning tree algorithm. Then, each connected subgraph is stitched together according to the corresponding optimal stitching path to minimize the overall stitching error and improve the consistency and coherence of the overall image stitching.

[0049] As one implementation method, Prim's minimum spanning tree algorithm or Kruskal's minimum spanning tree algorithm can be applied to each connected subgraph to extract the optimal splicing path corresponding to the connected subgraph.

[0050] Step S250: The connected subgraph with the most nodes is used as the main graph for splicing. Other connected subgraphs are spliced ​​onto the main graph based on the offset vector estimated by the scanning parameters of the scanning platform or the offset vector estimated by the existing splicing results.

[0051] Continue with Figure 4 For example, if the connected subgraph with the most nodes is ABCDEJIHFKLMNOPQUV, then ABCDEJIHFKLMNOPQUV is used as the main image for stitching. Other connected subgraphs are then stitched onto the main image based on the offset vector estimated by the scanning parameters of the scanning platform or the offset vector estimated by existing stitching results. Taking images Q and R as examples, image R can be stitched onto the main image based on the offset vector estimated by the stitching offset vector of images PQ or RS, or the offset vector estimated by the scanning parameters of the scanning platform. Similarly, images W, S, and T are stitched onto the main image, thus completing the image stitching and outputting a full-view image.

[0052] In existing technologies, feature matching methods for stitching cell images to obtain a full-view image suffer from poor stitching results.

[0053] To address the aforementioned issues, this application provides an image stitching method. The method involves acquiring images to be stitched and establishing an undirected graph based on the adjacency relationships between these images. A bidirectional stitching operation is performed on the stitching pairs. In the undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate beyond a first threshold are deleted. For stitching pairs that successfully stitch bidirectionally and whose bidirectional stitching results deviate within the first threshold range, an offset vector between the stitching pairs is obtained. The stitching pairs are images to be stitched that are adjacent to each other. The offset vector includes a horizontal offset and a vertical offset. For each edge of the undirected graph, a method is constructed that includes the offset vectors of each edge. For the preceding closed-loop paths, calculate the offset vector residuals of all closed-loop paths containing the current edge, and use the minimum norm of the offset vector residuals as the weight of the current edge; use a depth-first search method to identify all connected subgraphs of the undirected graph, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path; take the connected subgraph with the most nodes as the splicing main graph, and splice the other connected subgraphs onto the main graph based on the offset vectors estimated by the scanning parameters of the electronic scanning platform or based on the offset vectors estimated by the existing splicing results. This application establishes an undirected graph model for the images to be stitched, performs image registration verification based on bidirectional stitching, introduces the offset vector residual of the closed-loop path to constrain the undirected graph, and utilizes the depth-first search method and minimum spanning tree algorithm to extract the optimal stitching path. This constructs a widely applicable and highly scalable image stitching workflow framework, significantly improving the problems of low accuracy and severe error accumulation in traditional stitching methods for large-scale image processing. It effectively enhances the accuracy and robustness of image stitching, providing key technical support for constructing high-quality full-view images, further promoting the development of digital pathology and intelligent assisted diagnosis, and has broad prospects for clinical application and promotion.

[0054] Figure 5 This is a schematic diagram of an image stitching device according to an embodiment of the present invention, such as... Figure 5 As shown, an image stitching device 30 is provided, which includes an establishment module 31, a filtering module 32, a calculation module 33, an extraction module 34, and a stitching module 35.

[0055] Module 31 is established to acquire the images to be stitched together and to establish an undirected graph based on the adjacency relationships between the images to be stitched together.

[0056] The filtering module 32 is used to perform bidirectional stitching operations on stitching pairs. In an undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from a first threshold are deleted. For stitching pairs that successfully stitch together and whose bidirectional stitching results deviate from the first threshold, the offset vector between the stitching pairs is obtained. The stitching pairs are images to be stitched that are adjacent to each other. The offset vector includes a horizontal offset and a vertical offset.

[0057] The calculation module 33 is used to construct a closed-loop path containing the current edge for each edge of the undirected graph, calculate the offset vector residuals of all closed-loop paths containing the current edge, and take the minimum value of the offset vector residual norm as the weight of the current edge.

[0058] Extraction module 34 is used to identify all connected subgraphs of an undirected graph using the depth-first search method, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path;

[0059] The stitching module 35 is used to use the connected subgraph with the most nodes as the main stitching graph, and to stitch other connected subgraphs onto the main graph based on the offset vector estimated by the scanning parameters of the electronic scanning platform or the offset vector estimated by the existing stitching results.

[0060] The image stitching device 30 described above acquires images to be stitched and establishes an undirected graph based on the adjacency relationships between the images to be stitched; performs bidirectional stitching operations on stitching pairs; in the undirected graph, deletes edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from a first threshold; for stitching pairs that successfully stitch bidirectionally and whose bidirectional stitching results deviate from the first threshold, acquires offset vectors between the stitching pairs; the stitching pairs are images to be stitched that are adjacent to each other; the offset vectors include horizontal and vertical offsets; for each edge of the undirected graph, constructs a closed loop containing the current edge. For each edge, the offset vector residuals of all closed-loop paths containing the current edge are calculated, and the minimum norm of the offset vector residuals is used as the weight of the current edge. A depth-first search method is used to identify all connected subgraphs of the undirected graph. The optimal splicing path corresponding to each connected subgraph is extracted based on the minimum spanning tree algorithm, and each connected subgraph is spliced ​​according to its corresponding optimal splicing path. The connected subgraph with the most nodes is used as the main splicing graph. Other connected subgraphs are spliced ​​onto the main graph based on the offset vectors estimated by the scanning parameters of the electronic scanning platform or based on the offset vectors estimated by existing splicing results. This application establishes an undirected graph model for the images to be stitched, performs image registration verification based on bidirectional stitching, introduces the offset vector residual of the closed-loop path to constrain the undirected graph, and utilizes the depth-first search method and minimum spanning tree algorithm to extract the optimal stitching path. This constructs a widely applicable and highly scalable image stitching workflow framework, significantly improving the problems of low accuracy and severe error accumulation in traditional stitching methods for large-scale image processing. It effectively enhances the accuracy and robustness of image stitching, providing key technical support for constructing high-quality full-view images, further promoting the development of digital pathology and intelligent assisted diagnosis, and has broad prospects for clinical application and promotion.

[0061] In one embodiment, the filtering module 32 is further configured to perform a bidirectional splicing operation on the splicing pair using a phase correlation algorithm based on the Fourier domain;

[0062] If the confidence level of the splicing result is lower than the set second threshold, a template matching-based splicing method is used to perform a bidirectional splicing operation on the splicing pair.

[0063] In one embodiment, the extraction module 34 is further configured to perform Prim's minimum spanning tree algorithm or Kruskal's minimum spanning tree algorithm on each connected subgraph to extract the optimal splicing path corresponding to the connected subgraph.

[0064] It should be noted that the above modules can be either functional modules or program modules, and can be implemented in software or hardware. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or they can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0065] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores a set of preset configuration information. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned image stitching method.

[0066] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements an image stitching method. The display screen of the computer device may be a liquid crystal display (LCD) or an electronic ink display (EIM). The input device of the computer device may be a touch layer covering the display screen, or buttons, a trackball, or a touchpad located on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0067] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0068] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0069] Obtain the images to be stitched, and build an undirected graph based on the adjacency relationships between the images to be stitched;

[0070] Perform bidirectional stitching on the stitching pairs. In an undirected graph, delete the edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from the first threshold. For stitching pairs that successfully stitch and whose bidirectional stitching results deviate from the first threshold, obtain the offset vector between the stitching pairs. The stitching pairs are images to be stitched that are adjacent to each other. The offset vector includes horizontal offset and vertical offset.

[0071] For each edge of the undirected graph, construct a closed-loop path containing the current edge, calculate the offset vector residuals of all closed-loop paths containing the current edge, and take the minimum value of the offset vector residual norm as the weight of the current edge.

[0072] The depth-first search method is used to identify all connected subgraphs of an undirected graph. The minimum spanning tree algorithm is used to extract the optimal splicing path corresponding to all connected subgraphs. The connected subgraphs are then spliced ​​together according to the corresponding optimal splicing path.

[0073] The connected subgraph with the most nodes is used as the main graph for stitching. Other connected subgraphs are stitched onto the main graph based on the offset vector estimated by the scanning parameters of the electronic scanning platform or the offset vector estimated by the existing stitching results.

[0074] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0075] A phase correlation algorithm based on the Fourier domain is used to perform bidirectional splicing operations on the spliced ​​pairs;

[0076] If the confidence level of the splicing result is lower than the set second threshold, a template matching-based splicing method is used to perform a bidirectional splicing operation on the splicing pair.

[0077] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0078] For each connected subgraph, perform Prim's minimum spanning tree algorithm or Kruskal's minimum spanning tree algorithm to extract the optimal splicing path corresponding to the connected subgraph.

[0079] The aforementioned storage medium acquires the images to be stitched and establishes an undirected graph based on the adjacency relationships between the images. It then performs a bidirectional stitching operation on the stitching pairs. In the undirected graph, edges between stitching pairs that fail to stitch or whose bidirectional stitching results deviate from a first threshold are deleted. For stitching pairs that successfully stitch bidirectionally and whose bidirectional stitching results deviate within the first threshold range, offset vectors between the stitching pairs are obtained. The stitching pairs are the images to be stitched that are adjacent to each other. The offset vectors include horizontal and vertical offsets. For each edge of the undirected graph, a closed-loop path containing the current edge is constructed. Calculate the offset vector residuals of all closed-loop paths containing the current edge, and use the minimum norm of the offset vector residuals as the weight of the current edge; use a depth-first search method to identify all connected subgraphs of the undirected graph, extract the optimal splicing path corresponding to all connected subgraphs based on the minimum spanning tree algorithm, and splice each connected subgraph according to the corresponding optimal splicing path; take the connected subgraph with the most nodes as the splicing master graph, and splice other connected subgraphs onto the master graph based on the offset vectors estimated by the scanning parameters of the electronic scanning platform or based on the offset vectors estimated by the existing splicing results. This application establishes an undirected graph model for the images to be stitched, performs image registration verification based on bidirectional stitching, introduces the offset vector residual of the closed-loop path to constrain the undirected graph, and utilizes the depth-first search method and minimum spanning tree algorithm to extract the optimal stitching path. This constructs a widely applicable and highly scalable image stitching workflow framework, significantly improving the problems of low accuracy and severe error accumulation in traditional stitching methods for large-scale image processing. It effectively enhances the accuracy and robustness of image stitching, providing key technical support for constructing high-quality full-view images, further promoting the development of digital pathology and intelligent assisted diagnosis, and has broad prospects for clinical application and promotion.

[0080] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0081] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0082] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0083] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An image stitching method, characterized by, The method comprises: obtaining to-be-stitched images, and establishing an undirected graph based on the adjacency relationship between the to-be-stitched images; performing bidirectional stitching on the stitching pairs, deleting edges between the stitching pairs in the undirected graph for which stitching fails or the deviation of the bidirectional stitching result exceeds a first threshold value, and obtaining the offset vector between the stitching pairs for the stitching pairs for which bidirectional stitching succeeds and the deviation of the bidirectional stitching result is within the first threshold value range; the stitching pairs are to-be-stitched images that are adjacent to each other; the offset vector comprises a horizontal offset and a vertical offset; for each edge of the undirected graph, constructing a closed loop path containing the current edge, calculating the offset vector residuals of all closed loop paths containing the current edge, and taking the minimum value of the norm of the offset vector residuals as the weight of the current edge; using a depth-first search method to identify all connected subgraphs of the undirected graph, extracting optimal stitching paths corresponding to all connected subgraphs based on a minimum spanning tree algorithm, and stitching each connected subgraph according to the corresponding optimal stitching path; taking the connected subgraph with the largest number of nodes as a main stitching graph, and stitching other connected subgraphs to the main graph based on the offset vector estimated based on the scanning parameters of the electronic scanning platform or the offset vector estimated based on the existing stitching result.

2. The image stitching method of claim 1, wherein, The method comprises: performing bidirectional stitching on the stitching pairs using a phase correlation algorithm based on the Fourier domain; if the confidence of the stitching result is lower than a second threshold value, performing bidirectional stitching on the stitching pairs using a stitching method based on template matching.

3. The image stitching method of claim 1, wherein, The method comprises: performing Prim's minimum spanning tree algorithm or Kruskal's minimum spanning tree algorithm on each connected subgraph to extract the optimal stitching path corresponding to the connected subgraph.

4. An image stitching apparatus characterized by comprising: The image stitching device comprises an establishing module, a screening module, a calculating module, an extracting module, and a stitching module; The establishing module is configured to obtain to-be-stitched images, and establish an undirected graph based on the adjacency relationship between the to-be-stitched images; The screening module is configured to perform bidirectional stitching on the stitching pairs, delete edges between the stitching pairs in the undirected graph for which stitching fails or the deviation of the bidirectional stitching result exceeds a first threshold value, and obtain the offset vector between the stitching pairs for the stitching pairs for which bidirectional stitching succeeds and the deviation of the bidirectional stitching result is within the first threshold value range; the stitching pairs are to-be-stitched images that are adjacent to each other; the offset vector comprises a horizontal offset and a vertical offset; The calculating module is configured to, for each edge of the undirected graph, construct a closed loop path containing the current edge, calculate the offset vector residuals of all closed loop paths containing the current edge, and take the minimum value of the norm of the offset vector residuals as the weight of the current edge; The extracting module is configured to use a depth-first search method to identify all connected subgraphs of the undirected graph, extract optimal stitching paths corresponding to all connected subgraphs based on a minimum spanning tree algorithm, and stitch each connected subgraph according to the corresponding optimal stitching path. The splicing module is configured to take the connected subgraph with the largest number of nodes as a main splicing graph, and splice other connected subgraphs to the main splicing graph based on an offset vector estimated based on a scanning parameter of the electronic scanning platform or an offset vector estimated based on an existing splicing result.

5. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 3.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 3.

Images