Graph alignment method and device based on gw distance, system and storage medium

Abstract

The application relates to the technical field of graph data processing, and discloses a graph alignment method and device based on GW distance, a system and a storage medium. The method first constructs an initial coupling matrix and obtains an optimization matrix for graph alignment from multiple rounds of global updating operations, and the construction and updating operations of the coupling matrix are based on a related algorithm of GW distance. Secondly, in the process of each round of global updating, a local selection block is first confirmed to ensure that an adaptive dynamic region selection strategy can be adopted in stages. Finally, the iterative updating operation in the current round of global updating is limited within the local selection block, and updating is performed according to a coupling submatrix obtained on the coupling matrix according to the local selection block, to obtain an updating submatrix, and then the coupling matrix is updated in a local range according to the updating submatrix, so that the updating range can be compressed to the local selection block range of each round under the condition that the total round number of direct updating is the same, thereby reducing the calculation complexity of the GW distance in the graph alignment task.

Description

Technical Field

[0001] This invention relates to the field of graph data processing technology, and in particular to a graph alignment method, apparatus, system, and storage medium based on GW distance. Background Technology

[0002] Gromov-Wasserstein (GW) distance is a powerful framework for aligning structured data, but its high computational cost has been a core bottleneck limiting its practical application. For a source and target domain each containing n samples (graph nodes), a complete comparison of the relationship patterns between them requires up to O(n^2) computational costs. 4 The computational complexity is O(n^2). Current mainstream methods typically introduce entropy regularization and utilize the Sinkhorn algorithm to reduce the computational complexity to O(n^2). 3 However, the computational complexity of cubes remains unacceptable when dealing with large-scale graph data. Summary of the Invention

[0003] This invention provides a graph alignment method based on GW distance to reduce the computational complexity of GW distance in graph alignment tasks.

[0004] The first aspect of this invention discloses a graph alignment method based on GW distance, the method comprising: Obtain the first and second images; Based on the first graph and the second graph, an initial coupling matrix is ​​constructed, wherein the initial coupling matrix includes multiple matching scores, the matching scores are used to predict the degree of matching between graph nodes between the first graph and the second graph, and the graph nodes are used to represent graph information; Perform multiple rounds of global update operations on the initial coupling matrix, and determine the coupling matrix after the last round of global update as the optimized matrix. The optimized matrix is ​​used to align the first graph and the second graph. The global update operation includes: Based on the GW distance objective function, a near-end mapping is performed on the coupling matrix to determine local selection blocks. The local selection blocks are obtained based on the changes in the matching scores in the coupling matrix during the near-end mapping process. A coupling sub-matrix is ​​obtained based on the local selection blocks and the coupling matrix. Multiple rounds of near-end mapping are performed on the coupling sub-matrix to obtain an updated sub-matrix. The coupling matrix is ​​updated based on the updated sub-matrix to obtain the updated coupling matrix.

[0005] As an optional implementation, in the first aspect of the present invention, the method for obtaining the local selection block in each round of the global update operation includes: Based on the GW distance objective function, the coupling matrix is ​​mapped to the near end to obtain the transition matrix; Based on the transition matrix and the coupling matrix, a change parameter is obtained, which represents the change in the matching score at each position in the transition matrix compared to the coupling matrix. When it is determined that the changed parameter is greater than a preset first threshold, the position corresponding to the changed parameter in the coupling matrix is ​​determined as a candidate sub-region; based on all the candidate sub-regions, a local selection block is determined.

[0006] As an optional implementation, in the first aspect of the present invention, the specific operation of each round of global update further includes: When it is determined that the changed parameter is less than a preset first threshold, the position of the changed parameter in the coupling matrix is ​​marked. The positions in the coupling matrix that are marked less than a preset second threshold are identified as candidate sub-regions. Based on all the candidate sub-regions, candidate blocks are determined. The candidate blocks are used to control the operating range of the coupling matrix when performing near-end mapping.

[0007] As an optional implementation, in the first aspect of the present invention, the specific operation of each round of global update further includes: The positions in the coupling matrix that are marked more than a preset second threshold and less than a preset third threshold are identified as secondary selection regions; Based on the changing parameters and the coupling matrix, the relative change rate is obtained. When it is determined that the relative change rate is less than a preset fourth threshold, the updated coupling matrix is ​​determined to be the optimization matrix. The runtime of the previous global update operation and the prediction time of the optimized matrix obtained by the global update are obtained. When it is determined that the runtime is less than a preset fifth threshold and the prediction time is less than a preset sixth threshold, the candidate block is updated according to the secondary selection sub-region to expand the operation range of the candidate block.

[0008] As an optional implementation, in the first aspect of the invention, during each round of the global update process, the transition matrix... The calculation methods include:

[0009] In the above formula, k represents the number of rounds of global update. For the coupling matrix of the current global update round, Used to solve for the coupling matrix that minimizes the function within the parentheses. and It is a discrete probability vector. The elements in the graph are used to represent the weights of the nodes in the first graph. The elements in the graph are used to represent the weights of the nodes in the second graph. Used to indicate and The joint distribution of marginal distributions Let be the coupling matrix. Let GW be the gradient of the objective function. To solve Temporary variables that change during the process, Used to represent the Frobenius inner product. Let be the step size for convergence during the proximal mapping process. Used to represent KL divergence; Wherein, the gradient of the GW distance objective function The calculation methods include:

[0010] as well as,

[0011]

[0012] In the above formula, The number of nodes in the first graph. The number of nodes in the second graph. For length is A completely one vector, Used to represent transpose, This is the distance matrix between nodes in the first graph. This is the distance matrix between nodes in the second graph. For Hadamard products.

[0013] As an optional implementation, in the first aspect of the invention, the local selection block is The rectangle, in each round of the global update process. The calculation methods include:

[0014] in,

[0015]

[0016]

[0017]

[0018] In the above formula, Used to find the maximum value. Used to represent the index of the graph node in the first graph. Used to represent the index of the graph node in the second graph. Two-dimensional indices are used to represent matrix subscripts. Used to represent the subset of matrix indices corresponding to candidate subregions For the changing parameters of the current global update round, This is the preset first threshold.

[0019] As an optional implementation, in the first aspect of the present invention, the calculation method for the intermediate value of the coupling submatrix during each round of near-end mapping includes:

[0020] in,

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] In the above formula, s represents the number of rounds in which the coupling submatrix performs near-end mapping. For the KL projection operator, It is a natural exponential function. for submatrix, for submatrices; After determining the coupling submatrix and performing multiple rounds of near-end mapping, the final value of the coupling submatrix is ​​the update submatrix.

[0027] A second aspect of the present invention discloses a graph alignment device based on GW distance, the device comprising: The image acquisition module is used to acquire the first image and the second image; A matrix construction module is used to construct an initial coupling matrix based on the first graph and the second graph. The initial coupling matrix includes multiple matching scores, which are used to predict the degree of matching between graph nodes in the first graph and the second graph. The graph nodes are used to represent graph information. The matrix update module is used to perform multiple rounds of global update operations on the initial coupling matrix, and determine the coupling matrix after the last round of global update as the optimized matrix. The optimized matrix is ​​used to align the first graph and the second graph. The global update operation includes: Based on the GW distance objective function, a near-end mapping is performed on the coupling matrix to determine local selection blocks. The local selection blocks are obtained based on the changes in the matching scores in the coupling matrix during the near-end mapping process. A coupling sub-matrix is ​​obtained based on the local selection blocks and the coupling matrix. Multiple rounds of near-end mapping are performed on the coupling sub-matrix to obtain an updated sub-matrix. The coupling matrix is ​​updated based on the updated sub-matrix to obtain the updated coupling matrix.

[0028] A third aspect of the present invention discloses another graph alignment system based on GW distance, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the graph alignment method based on GW distance according to any of the first aspects of the present invention.

[0029] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked by a processor, are used to execute some or all of the steps in the graph alignment method based on GW distance as described in any of the first aspects of the present invention.

[0030] Compared with the prior art, the present invention has the following beneficial effects: This invention first performs a near-end mapping on the coupling matrix during each round of global update. By analyzing the changes in matching scores between the first and second graphs during this mapping, local selection blocks are determined. Then, a coupling submatrix is ​​obtained by combining these local selection blocks with the coupling matrix. Next, multiple rounds of near-end mapping are performed on the coupling matrix to achieve local updates, resulting in an updated submatrix. Finally, multiple rounds of global update further update each local region of the coupling matrix, yielding a more accurate coupling matrix for graph alignment tasks. This reduces the update range to the selection area of ​​local selection blocks while maintaining the same total number of rounds as direct update, thereby lowering the computational complexity of GW distance in graph alignment tasks. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating a graph alignment method based on GW distance disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a graph alignment device based on GW distance disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of a graph alignment system based on GW distance disclosed in an embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0035] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0036] It should be noted that the Gromov–Wasserstein (GW) distance mentioned in this paper applies to general metric spaces. Therefore, the definition of "graph" in this paper is not limited to regular pixel grids like images, but should also include any structure that can be constructed using the relationships between nodes and edges, that is, various structural graphs that can be generated using such structures as elements. Therefore, the technical solutions adopted in the following embodiments can be applied to the fields of social network analysis (user alignment), bioinformatics (protein interaction network alignment, brain network alignment), chemical information (molecular graph matching), knowledge graphs (entity alignment), computer vision (scene graph matching, point cloud registration), natural language processing (syntactic tree alignment), transportation networks (road network matching), and finance (transaction object alignment), etc.

[0037] Furthermore, the graph alignment method described in this paper aims to establish a correspondence between nodes in two graphs, ensuring that the intrinsic structure of the nodes remains as consistent as possible under this correspondence. This correspondence includes matching and mapping; therefore, this graph alignment method can align nodes / structures of the same type / domain or perform cross-modal alignment, establishing such a correspondence at the semantic or structural level for information from different data formats. For example, given an image containing a target object and a text description containing the target object, feature vectors are extracted from the image using a CNN model and from the text using BERT. Matching / aligning these two feature vectors is then performed. The GW distance can be used to achieve alignment by comparing the consistency of the graph's internal structure.

[0038] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a graph alignment method based on GW distance disclosed in an embodiment of the present invention. Figure 1 The described GW distance-based graph alignment method can be applied to GW distance-based graph alignment and assignment devices. For example... Figure 1 As shown, this graph alignment method based on GW distance can include the following operations: Step 101: Obtain the first image and the second image.

[0039] In this embodiment of the invention, the first and second graphs can be the definitions of "graph" as described above, which will not be repeated in this embodiment. If the "graph" itself corresponds to an image, graph nodes can be divided according to a pixel grid, and Manhattan / Euclidean distance can be used as the spatial distance to apply the image to graph alignment operations. The selection / construction of nodes can also employ the method of obtaining superpixels through oversegmentation, using superpixels as nodes and the distance between superpixel centroids or the distance of region color / texture features as the spatial distance; alternatively, a graph structure can be constructed for some feature points, using the distance between feature points or the similarity between points as the spatial distance. If the "graph" corresponds to non-image data, a method of first defining a set of nodes and then defining the distance or relationship strength between nodes is adopted, which is not limited in this embodiment.

[0040] Step 102: Construct the initial coupling matrix based on the first and second diagrams.

[0041] In this embodiment of the invention, the initial coupling matrix corresponds to the coupling matrix (transportation plan) in the GW distance, and the operation of determining the initial coupling matrix may include: We define the graph nodes for the first and second graphs, where the first graph has n nodes and the second graph has m nodes. Here, we use [n]={1,2,3,...,n} as the index set.

[0042] make and Given two discrete probability vectors, where each vector is a single vector. and ,satisfy Determine if the initial coupling matrix satisfies the set of coupling matrices. This set represents all those that are... and It is a joint distribution of marginal distributions.

[0043] In this embodiment of the invention, the initial coupling matrix includes multiple matching scores. Specifically, each matrix coordinate corresponds to a matching score. In the above embodiment, and These are two discrete probability distributions, defined on the node sets of two graphs respectively. Therefore, Each element in the graph represents the quality / weight assigned to each node in the first graph. The elements in the matrix represent the quality of each node in the second graph. This quality can be understood as the importance or available matching budget of each node in the graph alignment task, used to control the capacity or priority of each node's participation in the alignment task. The coupling matrix, on the other hand, is... and Given the joint probability distribution under marginal constraints, the matching score represents the distribution... The quality transferred from a node in the first graph to the corresponding node in the second graph can be interpreted as the joint probability of selecting node i from the first graph and node j from the second graph, i.e., the probability that both nodes are selected simultaneously. While maintaining their respective marginal importance, the higher this probability, the more likely the two nodes are to pair up. Therefore, this matching score is used to predict the degree of matching between graph nodes in the first and second graphs, and these graph nodes are used to represent graph information.

[0044] Step 103: Perform multiple rounds of global update operations on the initial coupling matrix, and determine the coupling matrix after the last round of global update as the optimized matrix.

[0045] Global update operations can include: Based on the GW distance objective function, a near-end mapping is performed on the coupling matrix to determine local selection blocks; a coupling submatrix is ​​obtained based on the local selection blocks and the coupling matrix; multiple rounds of near-end mapping are performed on the coupling submatrix to obtain an updated submatrix; the coupling matrix is ​​updated based on the updated submatrix to obtain the updated coupling matrix. The local selection block is obtained based on the change in the matching score in the coupling matrix during the near-end mapping process; In this embodiment of the invention, proximal mapping is essentially Bregman proximal mapping. Proximal mapping / operation on the coupling matrix can be understood as an optimization step that maps the current coupling matrix to a new coupling matrix under the penalty of Bregman divergence. The purpose of this step is to reduce the objective function while satisfying marginal constraints. The standard definition of Bregman proximal mapping is:

[0046] In the formula, The objective function is the GW distance, which is the objective to be minimized. u is the new coupling matrix, and x is the coupling matrix in the initial stage. It is composed of potential function The generated Bregman divergence, when At this time, the proximal mapping is Euclidean. Preferably, the mirror descent method induced by KL (Kullback-Leibler) divergence is used, that is, KL proximal mapping is used. In this case, the Bregman divergence is the KL divergence. Furthermore, when using KL divergence, the update rule is Sinkhorn scaling; when using Euclidean distance squared, the update rule is Euclidean projection onto the set of coupling matrices (transmission polyhedra). In addition, Bregman divergence can also employ Itakura-Saito, α-divergence, etc., with different divergences corresponding to their respective update algorithms. This embodiment of the invention does not limit these algorithms.

[0047] In this embodiment of the invention, the local selection block is a local selection box generated based on the matrix coordinates of the coupling matrix. This local selection block can be obtained based on the changes in the matching score within the coupling matrix during the near-end mapping process. These changes are obtained by comparing the new coupling matrix with the initial coupling matrix, and can involve varying degrees of increase / decrease in the matching score, or it can remain unchanged. Preferably, the local selection block can be divided based on the locations where the matching score changes. Corresponding parameter thresholds can be set to filter out the locations of these changes, thus obtaining the local selection block. For example, based on whether the matching score at the coordinate position changes, or when the rate of change exceeds a certain threshold, a range encompassing these locations is selected, and this range corresponds to the local selection block. The process of obtaining the coupling submatrix based on the local selection block and the coupling matrix can be achieved by selecting a matrix within a specified range within the coupling matrix based on the local selection block.

[0048] Preferably, the local selection block and the method of obtaining the coupling submatrix in the above embodiments can also be to reconstruct the matrix on the remaining rows and columns after excluding those that do not meet the change conditions. In this case, the local selection block is used to extract the coordinates of the rows and columns that meet the change conditions or to provide information on the coordinates to be selected, and the reconstructed matrix is ​​the coupling submatrix.

[0049] In existing technologies, graph alignment tasks using GW distance acceleration methods mainly fall into two categories. The first category is constraint relaxation methods, which aim to improve efficiency by simplifying the Sinkhorn solution process or decoupling double stochastic constraints. However, this strategy only optimizes computationally less complex tasks with a computational complexity of O(n^2). 2 The projection step failed to overcome the complexity of O(n) 3 The first type of problem faces a significant bottleneck: gradient computation. Furthermore, it often fails to guarantee that the obtained coupling matrix strictly satisfies the marginal distribution constraints. The second type involves low-dimensional approximation methods, such as multi-scale coarsening, spectral truncation, or low-rank parameterization, to reduce the problem size. However, these methods are prone to error accumulation and heavily rely on strong prior assumptions about the data structure, typically at the expense of geometric fidelity. These limitations highlight the urgent need to develop a novel, scalable, improved GW method that must significantly enhance computational efficiency while strictly maintaining geometric alignment accuracy.

[0050] In view of the above-mentioned computing power bottleneck problem encountered in large-scale graph data alignment, embodiments of the present invention creatively provide a selection strategy involving local block coordinates. This strategy is based on an important observation in the iterative updating process of the coupling matrix: the gradient of the objective function limits significant numerical updates to a continuously shrinking subset of sample pairs, while most elements in the matrix quickly stabilize. This means that continuing to perform global calculations on these stabilized elements is largely redundant. Therefore, by constructing a local selection block to frame the coordinates of elements that are still in an updated state, the huge global calculation can be limited to a coupling submatrix with a size of r, and generally r<<n, so that in a single global update operation, the calculation complexity can be reduced from O(n 3 ) to O(r 3 ). It effectively reduces the computational complexity of GW distance in large-scale graph alignment tasks.

[0051] Wherein, the purpose of multiple rounds of global update operations is to update the local selection block, because after the local matching score in the coupling matrix changes, it may cause a change in the gradient of the objective function again, so that the increment of the matching score at the coordinate positions excluded before meets the framing condition and is reselected. Therefore, the local selection block changes dynamically in different global update rounds.

[0052] In the embodiments of the present invention, the method for updating the coupling matrix according to the update submatrix may be a method of directly replacing the matching score at the original position, or may be a method of approximate substitution, in which the approximate value is solved by using the matching scores before and after. The embodiments of the present invention do not limit this.

[0053] It can be seen that in the present invention, in each round of global update process, first, proximal mapping is performed on the coupling matrix, the local selection block is determined according to the change of the corresponding matching score between the first graph and the second graph in the proximal mapping process, and then the coupling submatrix is obtained through the local selection block and the coupling matrix. Then, through multiple rounds of proximal mapping on the coupling matrix, local update of the coupling matrix is realized, and an update submatrix is obtained. Finally, through multiple rounds of global updates, each local region of the coupling matrix is further updated to obtain a more accurate coupling matrix for the graph alignment task. Furthermore, under the condition that the total number of update rounds is the same as that of direct update, the update range is compressed within the selection range of the local selection block, thereby reducing the computational complexity of GW distance in the graph alignment task.

[0054] In order to refine the above process of determining the local selection block, in an alternative embodiment, in each of the above rounds of global update operations, the acquisition method of the local selection block may include: Performing proximal mapping on the coupling matrix based on the GW distance objective function to obtain a transition matrix; Based on the transition matrix and the coupling matrix, the change parameters are obtained. The change parameters are used to measure the change in the matching score obtained by comparing the transition matrix and the coupling matrix. When it is determined that the changing parameter is greater than the preset first threshold, the position of the changing parameter in the coupling matrix is ​​determined as a candidate sub-region; based on all candidate sub-regions, a local selection block is determined.

[0055] In this optional embodiment, the transition matrix can be the coupling matrix obtained after any round of near-end mapping before iteratively updating the optimization matrix to the optimal matching score. The purpose of obtaining the transition matrix is ​​to compare the transition matrix and the coupling matrix to obtain the change parameter. Therefore, the change parameter can be the change in the matching score obtained from this comparison. Specifically, the change parameter can be the absolute value of the change at each position in the matrix, or it can be a matrix used to describe the increment, which contains the change information of the matching score at all positions.

[0056] Because the coupling matrix undergoes conditional gradient, sinkhorn, and near-end mapping during iterative updates, even if the theoretical solution has precisely reached the fixed point, rounding errors in floating-point operations can cause the matching scores obtained from two iterations to be less than equal, with a difference potentially of 10. -16 10 -18 The parameters are extremely small. Meanwhile, due to the needs of the convergence process and to avoid noise interference, in this optional embodiment, a first threshold is set to limit the judgment conditions for the changing parameters. Furthermore, the first threshold can also be used to filter out significantly updated regions and set a higher threshold, or, depending on the computational scale of the dataset, a more lenient threshold can be applied to larger datasets; this optional embodiment is not limited in this regard.

[0057] As can be seen, this optional embodiment, by specifying the process of determining the local selection block and introducing the constraint of the first threshold, enables the local selection block to select the changing region more accurately, thereby improving the update accuracy of each round of global update.

[0058] In the above embodiments, a single threshold judgment may lead to an overly strict screening process due to an inappropriate threshold, resulting in inaccurate results. For example, a threshold that is too large may filter out positions that should continue to be updated, causing the iterative update process to enter a suboptimal solution. A threshold that is too small may include too many positions in the local selection block, failing to alleviate too much computational complexity in actual operation. Therefore, in another optional embodiment, a marking method is used to update the selected region. This can reselect candidate regions to supplement the selection set if the threshold is too large, thereby weakening the sensitivity of the threshold restriction. The specific operation method of each round of global update may also include: When it is determined that the changing parameter is less than a preset first threshold, the position of the changing parameter in the coupling matrix is ​​marked. The positions in the coupling matrix that are marked less than a preset second threshold are identified as candidate sub-regions. Based on all candidate sub-regions, candidate blocks are determined. Candidate blocks are used to control the operating range of the coupling matrix when performing near-end mapping.

[0059] In this optional embodiment, the positions of the changing parameters in the coupling matrix can be marked using a Boolean mask, a Boolean array with the same shape as the coupling matrix, where each element in the array indicates whether the changing parameter at the corresponding position is less than a first threshold; a coordinate list can also be used, including all row and column index pairs that meet the requirement of being less than the first threshold; a hash set can also be used, etc., and this optional embodiment does not limit the choice of which method is used.

[0060] In different global update rounds, the positions where the changing parameters satisfy the first threshold may differ. After multiple rounds of global updates, some positions may have been marked with matching scores but require updating. However, if the first threshold is set too high, these items that need updating may be missed during the filtering process. Therefore, this optional embodiment sets a second threshold to exclude items with minimal changes (which may have already converged) after multiple updates. It performs secondary filtering based on the number of times the items are marked, thereby weakening the sensitivity of the first threshold and enriching the selection set of positions after the first threshold filtering.

[0061] As can be seen, this optional embodiment marks the positions where the changing parameters do not meet the first threshold, and then performs a second screening on the marked positions using the second threshold. In this way, even if the update convergence is excluded, the positions that need to be updated due to the first threshold being too large can be supplemented, thereby reducing the sensitivity of the first threshold and making the division of the region to be updated more accurate. Furthermore, the update accuracy is improved while reducing the computational complexity.

[0062] In another optional embodiment, update convergence conditions and operations for adjusting the task volume according to task time requirements are set. The specific operation method for each round of global update may further include: The positions in the coupling matrix that are marked more than a preset second threshold and less than a preset third threshold are identified as secondary selection regions; Based on the changing parameters and the coupling matrix, the relative rate of change is obtained. When it is determined that the relative rate of change is less than the preset fourth threshold, the updated coupling matrix is ​​determined to be the optimization matrix. The runtime of the previous global update operation and the prediction time of the optimized matrix obtained from the global update are obtained. When it is determined that the runtime is less than the preset fifth threshold and the prediction time is less than the preset sixth threshold, the candidate block is updated according to the secondary selection sub-region to expand the operation range of the candidate block.

[0063] In this optional embodiment, a new filtering range is formed by setting a third threshold and the second threshold. The purpose is to expand the selection of candidate blocks in the above embodiments, and this selection is based on task time requirements. For large-scale graph matching or graph alignment tasks, both time and matching accuracy are factors to consider. Therefore, setting a fifth threshold is to detect whether the computation time of the previous round has delayed the overall task progress, because the previous round may have included using candidate blocks or updated candidate blocks to expand the selection area of ​​the update process. Increasing the selection area directly increases the computation time of the update process. Setting a sixth threshold is to predict the burden of updating secondary sub-regions on the total task. If this detailed operation affects the time required to output the optimized matrix and significantly increases the time requirement, then the update operation for secondary sub-regions is abandoned. If time requirements are not a concern, and the focus is on the quality of the graph alignment task, the selection range of candidate blocks can be readjusted to ensure the accuracy of the output results while reducing computational complexity.

[0064] In this optional embodiment, the relative rate of change can be obtained by comparing the incremental matrix and coupling matrix corresponding to the changing parameters. The relative rate of change indirectly reflects the progress of the update iteration. If the relative rate of change is too small, it indicates that the updates of matching scores at most positions tend to be stable. Although subsequent iterative update operations may still have matching scores greater than the first threshold, the resulting local selection blocks are too small or the overall matching scores fluctuate slightly. Although such small-scale updates can still continue, this results in very low efficiency / small benefits for continued updates. At this point, it is determined that the convergence condition is met, and the update is stopped. Because the fourth threshold is a hard constraint, in actual operation, there may be situations where the convergence condition has been met, but the relative rate of change is always higher than the fourth threshold because the fourth threshold is too low. In this case, an upper limit for the number of iterations can be set. This optional embodiment does not limit this.

[0065] As can be seen, this optional embodiment, by adjusting the candidate block in the secondary selection region, can freely adjust the fineness of the update process according to the time requirements of the task. At the same time, by setting the stopping condition for iterative updates, the update process can be reasonably converged, avoiding the waste of computing resources. Thus, while reducing computational complexity, the output time of the optimization matrix can be further controlled, improving the adaptability and flexibility of this optional embodiment in different tasks.

[0066] In yet another alternative embodiment, during each round of global update, the transition matrix... The calculation methods may include:

[0067] In the above formula, k represents the number of rounds of global update. For the coupling matrix of the current global update round, Used to solve for the coupling matrix that minimizes the function within the parentheses. and It is a discrete probability vector. The elements in the graph are used to represent the weights of the nodes in the first graph. The elements in the graph are used to represent the weights of the nodes in the second graph. Used to indicate and The joint distribution of marginal distributions Let be the coupling matrix. Let GW be the gradient of the objective function. For the solution process Temporary variables that change during the process, Used to represent the Frobenius inner product. Let be the step size for convergence during the proximal mapping process. Used to represent KL divergence; Wherein, the gradient of the GW distance objective function The calculation methods may include:

[0068] as well as,

[0069]

[0070] In the above formula, The number of nodes in the first graph. The number of nodes in the second graph. For length is A completely one vector, Used to represent transpose, This is the distance matrix between nodes in the first graph. This is the distance matrix between nodes in the second graph. For Hadamard products.

[0071] In yet another alternative embodiment, the local selection block is... A rectangle, in each round of the global update process, The calculation methods may include:

[0072] in,

[0073]

[0074]

[0075]

[0076] In the above formula, Used to find the maximum value. Used to represent the index of the graph node in the first graph. Used to represent the index of the graph node in the second graph. Two-dimensional indices are used to represent matrix subscripts. Used to represent the subset of matrix indices corresponding to candidate subregions The parameters for the current global update cycle are the changed parameters, and the first threshold is the preset threshold.

[0077] In another optional embodiment, during each round of near-end mapping of the coupling submatrix, the intermediate value of the coupling submatrix is ​​calculated in the following ways:

[0078] in,

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] In the above formula, s represents the number of rounds in which the coupling submatrix performs near-end mapping. For the KL projection operator, It is a natural exponential function. for submatrix, for submatrices; After determining the coupling submatrix and performing multiple rounds of near-end mapping, the final value of the coupling submatrix is ​​the update submatrix.

[0085] To facilitate the implementation of the technical solutions in the above embodiments, this optional embodiment further elaborates on the details and applications of the formulas appearing in the above embodiments: In the case of graph alignment, let and Let be the metric space of the two graphs. Where, and Let these represent the set of graph nodes in the first graph and the set of graph nodes in the second graph, respectively. The intra-domain distance matrices of the first graph and the second graph are respectively... and . and It is a discrete probability vector. The elements in the graph are used to represent the weights of the nodes in the first graph. The elements in the graph represent the weights of the nodes in the second graph. These weights can be determined based on task requirements, using node degree (centrality), node features (such as text or image features), or the importance of the matching task (such as highly influential users in a social network or highly conserved proteins in a biological network). For simplicity, we assume here that the nodes in the graph are uniformly distributed, i.e. The GW distance between the metric spaces of two graphs is defined as follows:

[0086]

[0087] in, and These represent the indices of the source graph node and the target graph node, respectively. The constraints in formula (1) above ensure that... It is an efficient joint distribution between the metric spaces of two graphs: "and" "Guaranteed" It conforms to the given marginal distribution, while " "This ensures that each element in the coupling matrix..." Non-negative, representing the source graph node With target graph nodes The matching score between them. Once the optimal coupling is found... ,make This represents the set of all feasible node-to-node (injective) mappings between the source and target graphs. The discrete alignment result can then be derived as follows: :

[0088] The quadruple summation form in formula (1) intuitively demonstrates the structural consistency of the GW objective, but its complex high-order indexes obscure the underlying computational efficiency of the algorithm. To clearly elucidate its algebraic structure, a fourth-order loss tensor is introduced. Based on squared Euclidean loss The elements of this tensor are defined as Furthermore, let This represents the contraction operation between tensors and matrices (defined as...). ), and order Let Frobenius inner product be the expression. Therefore, the GW objective can be equivalently rewritten as the following matrix inner product form:

[0089] in, Indicates coupling The structural cost.

[0090] In order to the coupling matrix Perform a first-order update and calculate the gradient of the objective function (2), which is expressed as follows:

[0091] in, and Let be a vector, where This represents the Hadamard product (element-wise multiplication) operation. Although it involves... and The item can be found Pre-calculation or updates are performed within a specified timeframe, but interactive items " The computation of "" remains the main bottleneck, with a time complexity of up to 100%. .

[0092] The core task of this optional embodiment is to design an efficient algorithm to minimize the GW objective function defined in equation (2). This algorithm aims to overcome the computational complexity bottleneck in a single iteration while ensuring the rigorous feasibility of the solution obtained without introducing any constraint relaxation. To achieve this goal, this optional embodiment first analyzes the optimization process. Based on this, a data-dependent local block coordinate selection strategy is designed to decompose the coupling matrix into multiple coordinate blocks, thereby significantly reducing the gradient computation complexity in equation (3).

[0093] To determine the local block coordinate selection strategy, this alternative embodiment first introduces a global optimization scheme for the GW objective in equation (2). Specifically, in the first... In the next iteration, the transition matrix is ​​further obtained using the gradient obtained in equation (3):

[0094] Given step size By solving the above equation, we can deduce that:

[0095] Here, Proj represents the projection of the coupling matrix set based on KL divergence, which can be efficiently implemented using the Sinkhorn algorithm. This optional embodiment will not be elaborated further.

[0096] In the optimization process defined by Equation (4), the global computation coupling matrix constitutes the main computational bottleneck in the gradient evaluation of Equation (3). To address this issue, this alternative embodiment proposes a local block coordinate selection strategy. Specifically, projecting the gradient restricts significant numerical updates to a continuously shrinking subset of sample pairs, while most elements in the matrix quickly stabilize. This means that continuing global computation on these stabilized elements is largely redundant.

[0097] set up The first threshold is used to distinguish between static regions and regions that have undergone significant updates due to gradient-driven processes. Based on different graph datasets from Select the optimal Parameters. In the first In the next iteration, we define the selected set for:

[0098] The changing parameter can be an increment matrix. This represents the change caused by a single projection mirror descent step:

[0099] Using the selected set described above, this alternative embodiment incorporates the update process into a local selection block. Specifically, the optimization process is strictly limited to the selected set. Within the range of coordinates it covers, and its complement. The elements in remain unchanged:

[0100] in," "This is the objective function defined in equation (3). It should be noted that the selected set..." It updates dynamically. If, during the update process of the selected set, interactive items are used... This caused a gradient The change caused the increase in previously excluded items to exceed the threshold again. These items will then be reselected in subsequent iterations. Let:

[0101]

[0102] At this point, the original global optimization problem has been effectively reduced in dimensionality, simplified into a problem consisting of... Composition Local subspace, where .

[0103] Then, this optional embodiment introduces a marginal compensation mechanism that maintains strict feasibility. This mechanism aims to ensure that global marginal constraints still hold when updating candidate submatrices within a local selection block. The core idea is to deduct the total mass already allocated to stationary elements outside the selected submatrix from the global marginal requirements, thereby reallocating local marginal requirements. Specifically, for candidate submatrices... The derived local edge quantity and as follows:

[0104] Thanks to the strict feasibility guarantee provided by the above formula, the iterative update (near-end mapping) within the local selection block can be safely restricted to execution within the local selection block:

[0105] Among them, the gradient of the local selection block Defined as:

[0106] Furthermore, interactive items It can be efficiently calculated by multiplying submatrices:

[0107] By combining the local block coordinate selection strategy with the aforementioned feasibility preservation mechanism, the complete framework is achieved. The condition for stopping the iteration is... or the number of iterations The number of iterations reaches 2000. In this alternative embodiment, the time complexity of a single iteration is effectively reduced to... The total complexity consists of two parts: the computational complexity required for the interaction items... Time, and the gradient required for local selection blocks Time, and meet the requirements .

[0108] The optional embodiment of the optimized GW distance-based graph alignment method can capture and maintain the global topology of the graph across the metric space by minimizing the second-order difference in the distance between node pairs between different graphs, and is theoretically naturally adapted to graph alignment tasks. To further verify the technical advantages of this optional embodiment, the following sets of experiments show its comparison results with a number of mainstream benchmark methods. These benchmark methods are mainly divided into the following two categories: (1) constraint relaxation methods: including eBPG, BPG and its single-step variable BPG-S, FW, KL-BAPG and JOENA; (2) low-dimensional approximation methods: including ScalaGW, SpecGW, LR and HOT.

[0109] To verify the effectiveness of this alternative embodiment, evaluation tests were conducted on five benchmark graph datasets. These datasets included a synthetic dataset (average number of nodes 1500, number of edges 56579) and four datasets with different topologies and sizes.

[0110]

[0111] As shown in Table 1, the above datasets cover the following two application areas: (1) Social network area including Reddit (average number of nodes 375.9, number of edges 499.3) and Douban (average number of nodes 234, number of edges 345.33); (2) Co-authored network area including ACM (average number of nodes 1000, number of edges 4163) and DBLP (average number of nodes 1000, number of edges 3747). This optional embodiment uses two metrics, accuracy and computational efficiency, for performance evaluation, specifically defined as follows: Accuracy ,in It is a set of real node correspondences. It is the set of predictions generated by the algorithm. Efficiency is defined as the average wall clock time (in seconds) required to complete each pair of graph alignment tasks, and is used to measure the computational cost of the algorithm.

[0112] This optional embodiment uses a symmetrically normalized and unweighted adjacency matrix as input. Step size exist The best alignment is obtained by selecting the optimal option from within.

[0113]

[0114] Table 2 summarizes the alignment accuracy (%) and runtime (seconds) of this alternative embodiment (the proposed method) on five benchmark datasets. Bold values ​​in the table indicate the best-performing results. Experimental results show that the method employed by this alternative embodiment achieves the highest accuracy on all test datasets and significantly outperforms existing comparative methods. Regarding computational efficiency, for large-scale graphs (Synthetic, ACM, and DBLP), the running speed of this alternative embodiment is tens of times faster than the existing best-performing method. For small-scale graphs (Douban and Reddit), although the marginal gain from the acceleration is relatively limited due to the overall computation time being reduced to less than 1 second, it still maintains extremely high real-time processing performance.

[0115] To further evaluate the algorithm's scalability, this optional embodiment generated a series of node sizes. The dataset includes a synthetic graph dataset with 20% noise, and the single-run time threshold for the experiment is set to 300 seconds. In terms of computational efficiency, this alternative embodiment maintains the lowest runtime across all node sizes. As the number of nodes increases, this alternative embodiment can achieve up to 160x speedup compared to the baseline method. This significant efficiency improvement is primarily due to the local block coordinate descent algorithm employed in this alternative embodiment; in contrast, the baseline method is limited by high complexity... The bottleneck limitation reaches a time limit when the number of nodes reaches 2000. Regarding memory efficiency, this alternative embodiment significantly alleviates the storage bottleneck in large-scale graph alignment. When the node size increases to 5000, its peak memory consumption is reduced by 4.9 times compared to the baseline method. The above experimental results fully demonstrate that this alternative embodiment has excellent scalability and practical application potential when handling large-scale graph tasks.

[0116] Example 2 Please see Figure 2 , Figure 2 This is a schematic diagram of a graph alignment device based on GW distance disclosed in an embodiment of the present invention. Figure 2 As shown, the graph alignment device based on GW distance may include: Image acquisition module 201 is used to acquire the first image and the second image; The matrix construction module 202 is used to construct an initial coupling matrix based on the first graph and the second graph. The initial coupling matrix includes multiple matching scores, which are used to predict the degree of matching between graph nodes in the first graph and the second graph. Graph nodes are used to represent graph information. The matrix update module 203 is used to perform multiple rounds of global update operations on the initial coupling matrix and determine the coupling matrix after the last round of global update as the optimized matrix. The optimized matrix is ​​used to align the first graph and the second graph. The global update operation includes: Based on the GW distance objective function, a near-end mapping is performed on the coupling matrix to determine local selection blocks. The local selection blocks are obtained based on the changes in the matching scores in the coupling matrix during the near-end mapping process. A coupling submatrix is ​​obtained based on the local selection blocks and the coupling matrix. Multiple rounds of near-end mapping are performed on the coupling submatrix to obtain an updated submatrix. The coupling matrix is ​​then updated based on the updated submatrix to obtain the updated coupling matrix.

[0117] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of a graph alignment system based on GW distance disclosed in an embodiment of the present invention. Figure 3 As shown, the graph alignment system based on GW distance may include: Memory 301 storing executable program code; Processor 302 coupled to memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute some or all of the steps in any of the graph alignment methods based on GW distance in Embodiment 1 of the present invention.

[0118] Example 4 This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute some or all of the steps in any of the graph alignment methods based on GW distance disclosed in Embodiment 1 of this invention.

[0119] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0121] Finally, it should be noted that the above embodiments are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A graph alignment method based on GW distance, characterized in that, The method includes: Obtain a first image and a second image, where the first image and the second image have different pixel grids; Based on the first image and the second image, an initial coupling matrix is ​​constructed, wherein the initial coupling matrix includes multiple matching scores, and each matrix coordinate corresponds to a matching score. The matching score is used to represent the probability that a graph node in the first image and a graph node in the second image are used for graph alignment at the same time. The graph node is obtained by dividing the pixel grid. Multiple rounds of global update operations are performed on the initial coupling matrix, and the coupling matrix after the last round of global update is determined as the optimized matrix. The optimized matrix is ​​used to align the first graph and the second graph, and the elements in the optimized matrix are used to control the priority of each graph node in the graph alignment task. The global update operation includes: Based on the GW distance objective function, the coupling matrix is ​​mapped to the near end to obtain the transition matrix; Based on the transition matrix and the coupling matrix, a change parameter is obtained, which is used to measure the change in the matching score obtained by comparing the transition matrix and the coupling matrix; When it is determined that the changing parameter is greater than a preset first threshold, the position corresponding to the changing parameter in the coupling matrix is ​​determined as a candidate sub-region; based on all the candidate sub-regions, a local selection block is determined; wherein, the local selection block is a local selection box generated based on the matrix coordinates of the coupling matrix; based on the local selection block and the coupling matrix, a coupling sub-matrix is ​​obtained, and multiple rounds of near-end mapping are performed on the coupling sub-matrix to obtain an updated sub-matrix; based on the updated sub-matrix, the coupling matrix is ​​updated to obtain the updated coupling matrix.

2. The graph alignment method based on GW distance according to claim 1, characterized in that, The specific operation method for each round of global update also includes: When it is determined that the changed parameter is less than a preset first threshold, the position of the changed parameter in the coupling matrix is ​​marked. The positions in the coupling matrix that are marked less than a preset second threshold are identified as candidate sub-regions. Based on all the candidate sub-regions, candidate blocks are determined. The candidate blocks are used to control the operating range of the coupling matrix when performing near-end mapping.

3. The graph alignment method based on GW distance according to claim 2, wherein the specific operation of each round of global update further includes: The positions in the coupling matrix that are marked more than a preset second threshold and less than a preset third threshold are identified as secondary selection regions; Based on the changing parameters and the coupling matrix, the relative change rate is obtained. When it is determined that the relative change rate is less than a preset fourth threshold, the updated coupling matrix is ​​determined to be the optimization matrix. The runtime of the previous global update operation and the prediction time of the optimized matrix obtained by the global update are obtained. When it is determined that the runtime is less than a preset fifth threshold and the prediction time is less than a preset sixth threshold, the candidate block is updated according to the secondary selection sub-region to expand the operation range of the candidate block.

4. The graph alignment method based on GW distance according to claim 3, characterized in that, In each round of the global update process, the transition matrix The calculation methods include: In the above formula, k represents the number of rounds of global update. For the current global update round's coupling matrix, Used to solve for the coupling matrix that minimizes the function within the parentheses. and It is a discrete probability vector. The elements in the graph are used to represent the weights of the nodes in the first graph. The elements in the graph are used to represent the weights of the nodes in the second graph. Used to indicate and The joint distribution of marginal distributions Let be the coupling matrix. Let GW be the gradient of the objective function. To solve Temporary variables that change during the process, Used to represent the Frobenius inner product. Let be the step size for convergence during the proximal mapping process. Used to represent KL divergence; Wherein, the gradient of the GW distance objective function The calculation methods include: as well as, In the above formula, The number of nodes in the first graph. The number of nodes in the second graph. For length is A completely one vector, Used to represent transpose, This is the distance matrix between nodes in the first graph. This is the distance matrix between nodes in the second graph. For Hadamard products.

5. The graph alignment method based on GW distance according to claim 4, characterized in that, The local selection block is The rectangle, in each round of the global update process. The calculation methods include: in, In the above formula, Used to find the maximum value. Used to represent the index of the graph node in the first graph. Used to represent the index of the graph node in the second graph. Two-dimensional indices are used to represent matrix subscripts. Used to represent the subset of matrix indices corresponding to candidate subregions For the changing parameters of the current global update round, This is the preset first threshold.

6. The graph alignment method based on GW distance according to claim 5, characterized in that, During each round of near-end mapping, the intermediate values ​​of the coupled submatrix are calculated in the following ways: in, In the above formula, s represents the number of rounds in which the coupling submatrix performs near-end mapping. For the KL projection operator, It is a natural exponential function. for submatrix, for submatrices; After determining the coupling submatrix and performing multiple rounds of near-end mapping, the final value of the coupling submatrix is ​​the update submatrix.

7. A graph alignment device based on GW distance, characterized in that, The device includes: The image acquisition module is used to acquire the first image and the second image; A matrix construction module is used to construct an initial coupling matrix based on the first graph and the second graph. The initial coupling matrix includes multiple matching scores, which are used to predict the degree of matching between graph nodes in the first graph and the second graph. The graph nodes are used to represent graph information. The matrix update module is used to perform multiple rounds of global update operations on the initial coupling matrix, and determine the coupling matrix after the last round of global update as the optimized matrix. The optimized matrix is ​​used to align the first graph and the second graph. The global update operation includes: Based on the GW distance objective function, a near-end mapping is performed on the coupling matrix to determine local selection blocks. The local selection blocks are obtained based on the changes in the matching scores in the coupling matrix during the near-end mapping process. A coupling sub-matrix is ​​obtained based on the local selection blocks and the coupling matrix. Multiple rounds of near-end mapping are performed on the coupling sub-matrix to obtain an updated sub-matrix. The coupling matrix is ​​updated based on the updated sub-matrix to obtain the updated coupling matrix.

8. A graph alignment system based on GW distance, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the graph alignment method based on GW distance as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked by a processor, execute the graph alignment method based on GW distance as described in any one of claims 1-6.

Images