System and method for second-order and third-order compatibility hypergraph matching based on CUR decomposition

Abstract

The hypergraph matching algorithm has better matching precision, but the cost is exponentially increased by computing resources. The compatibility between two hypergraphs is represented as a sparse tensor by an approximate nearest neighbor (ANN) method based on a kd tree, which still needs to perform exhaustion calculation on a large-scale graph matching problem. In order to reduce the calculation amount of a large-scale hypergraph matching problem, a CUR-based cascade framework is provided, and a sparse compatibility tensor is generated by reducing the compatibility between two calculated graphs. The framework comprises a CUR-based second-order graph matching algorithm, a Fiber-CUR-based tensor generation method and a PRL-based hypergraph matching algorithm specially suitable for sparse tensor. Experimental results show that the proposed cascade framework needs fewer computing resources, and can effectively process a larger-scale graph matching problem. According to the PRL-based hypergraph matching algorithm, higher matching precision can be realized by using a tensor which is more sparse than that of an existing method.

Description

Technical Field

[0001] The present disclosure relates to a system and method for hypergraph matching based on second-order and third-order compatibility of CUR decomposition. Background Art

[0002] Computer vision and pattern recognition tasks, finding correspondences between pairs of features have a wide range of applications, such as feature tracking, image classification, object detection, and gene-drug association identification. Given two point sets, the task is to find matching pairs based on the extracted features. Graph matching can effectively solve this problem by representing each point set as a graph. Each point in the point set is described as a node, and the edges describe the structural relationship between these points. The graph matching problem aims to minimize an objective function consisting of unary or pairwise terms. Previous related tasks aim to find correspondences between visual features such as keypoints, edges, or raw pixels. A common approach is to define a compatibility measure between two features, such as SIFT, and match the features in the first image with their nearest neighbors in the second image. This approach may not work well in practice due to ambiguities such as repeated patterns and textures.

[0003] To further improve the matching performance, some methods consider the geometric consistency between a pair of features, which is called second-order graph matching. In order to describe the compatibility between the edges of two feature sets, a prior art system uses the distance between the two features without considering the invariance of rotation, and another prior art method uses distance- and angle-based features to balance rotation and scale invariance. Another prior art method considers that adjacent points in the first image should also be close to each other in the target image. For pairwise matching, which is called second-order graph matching, the problem can be considered as a quadratic assignment problem (QAP), which is NP-hard. Therefore, various methods have been proposed to match pairs of features with soft constraint approximations. Some prior art systems are configured to calculate a rank-1 approximation of the compatibility matrix. In one prior art, a random walk view and a weighted jump scheme for graph matching are introduced. In another prior art system, more geometry is used to gain an in-depth understanding of the problem and further reduce the space and time complexity. However, it is difficult to define the similarity between pairs of features in the second-order matching problem. Therefore, second-order graph matching algorithms are generally invalid for patterns under rotation and scaling deformation.

[0004] In the past decade, researchers have proposed models with higher-order graphs, called hypergraphs. Traditional graphs consider edges as associations between a pair of nodes. Hypergraphs consider edges (called hyperedges) as associations between multiple nodes. Compared with second-order graph matching methods, higher-order graph matching methods have better geometric information integration and better ability to handle transformations such as scaling or rotation. The hypergraph matching problem considers the compatibility between two hypergraphs as a high-order tensor. The objective function aims to find the maximization of all permutations of features.

[0005] Recent hypergraph matching methods utilize higher-order similarity measures to obtain more accurate results. One prior art considers matching problems related to Kronecker products. Another prior art system represents the high-order compatibility between two hypergraphs as a sparse tensor based on the Approximate Nearest Neighbor (ANN) method, which is widely used to generate compatibility tensors in subsequent hypergraph matching work. Another prior art system is configured to reweight the walk hypergraph matching problem. Another prior art attempts to solve it through a sequence of linear or quadratic assignment problems. One prior art decomposes the problem of the alternating direction method of multipliers. Another prior art system considers the problem of non-cooperative multiplayer games and extracts stable strategies to find matching accuracy. In the past decade, there has been little research dedicated to optimizing the compatibility tensor for hypergraph matching problems. Although the ANN-based method can significantly reduce the time complexity during the matching process, it requires a large computational cost to calculate the compatibility tensor.

[0006] In the past few years, with the rapid development of deep learning, several learning-based algorithms have been proposed to learn parameters of graph matching problems in a data-driven manner. Many attempts have been made to design an end-to-end deep learning framework to solve the graph matching problem. One prior art uses a graph convolutional network (GCN) to project the graph structure to node features. Both methods learn graph properties and match nodes with the Sinkhorn algorithm. A prior art system transforms the second-order graph matching problem into a node classification problem on an association graph. In one prior art, a learning-based method directly learns the compatibility matrix / tensor through data. The graph / hypergraph matching problem is solved by learning different models on different datasets. Although state-of-the-art accuracy is achieved on the current test dataset, the models trained by the algorithm are not uniform. In addition, for large-scale graph matching problems, the GCN network requires huge computing resources, so the current learning-based hypergraph matching algorithm can only handle relatively small-scale problems. Summary of the invention

[0007] According to a first aspect of the present disclosure, a computer system for second-order graph matching is provided. The computer system comprises:

[0008] A memory storing at least two node sets A compatibility matrix (H) representing the similarity between the edges of the nodes;

[0009] A processing unit is configured to receive the compatibility matrix and execute machine instructions, the machine instructions comprising the following steps:

[0010] retrieving a subset of columns and rows from said compatibility matrix (H) in memory to form a first matrix (C);

[0011] Perform CUR decomposition on the tensor to generate an approximate matrix (U * );

[0012] Applying a soft-constrained second-order graph matching algorithm to the approximate matrix to generate an assignment matrix (X);

[0013] Identify the predetermined element (k) with the highest probability in the assignment matrix as the node matching set

[0014] According to the first aspect, each of the at least two groups of nodes has a cardinality greater than the columns and rows of the first matrix.

[0015] According to the first aspect, the compatibility matrix uses the first matrix (C) and the second matrix (U * ) decomposition, wherein the size of each of the first matrix and the second matrix is ​​smaller than the size of the compatibility matrix.

[0016] According to the first aspect, the second matrix is generate.

[0017] According to a first aspect, the first matrix (C) comprises randomly selected columns and rows retrieved from the compatibility matrix (H).

[0018] According to the first aspect, the machine instructions further include generating a node matching set The compatibility tensor of steps.

[0019] According to the first aspect, the compatibility tensor is a sparse tensor.

[0020] According to the first aspect, the compatibility tensor is generated by Fiber-CUR based tensor generation.

[0021] According to the first aspect, the Fiber-CUR based tensor generation comprises the following steps:

[0022] From the node matching set Select a hyperedge from one of them;

[0023] Combine the selected hyperedge with the hyperedge subset {{i 2 , j 2 ,:},{i 2 ,:,k 2},{:,j 2 , k 2}}∈ε 2 For comparison, and

[0024] For each hyperedge in the source graph, r nearest neighbors are selected to generate the compatibility tensor.

[0025] According to a first aspect, the machine instructions further comprise the step of generating a relaxed allocation matrix (X) with respect to the compatibility tensor.

[0026] According to a first aspect, the relaxation allocation matrix (X) is generated by a probabilistic relaxation labeling algorithm.

[0027] According to a first aspect, the probabilistic relaxed labeling algorithm comprises the steps of generating a probability of node i from one of the node matching sets, and generating a probability of node j from another of the node matching sets.

[0028] According to the first aspect, the probability that the node i is associated with the node j is given by express.

[0029] According to a first aspect, the probabilities are replaced by a vector x, where x is a column-wise flattened version of the soft constraint assignment matrix.

[0030] According to a first aspect, the vector x is adapted to be normalized as:

[0031]

[0032] According to the first aspect, when the relaxation allocation matrix X converges, the relaxation marking algorithm stops

[0033] According to the first aspect, the at least two node sets are corresponding points in the template with regular shape.

[0034] According to the first aspect, the templates with regular shapes come from different objects, including different Chinese characters.

[0035] According to the first aspect, the at least two node sets The corresponding points of the frame image sequence of two objects include two images or two frame image streams (video) of building images.

[0036] According to the first aspect, the at least two node sets Corresponding feature points and shapes for different 2D and 3D objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Embodiments of the present disclosure will be described below by way of examples with reference to the accompanying drawings.

[0038] Figure 1 FIG. 4 is a schematic diagram of a computer server according to an embodiment of the present disclosure, wherein the computer server is used to implement a system for analyzing an image.

[0039] Figure 2 Flowchart of a hypergraph matching framework according to a preferred embodiment of the present disclosure.

[0040] Figure 3 (a) is a diagram showing a comparison between a conventional ANN-based tensor generation method and a CUR-based tensor generation method according to an embodiment of the present disclosure.

[0041] Figure 3 (b) is a diagram showing a comparison between a conventional ANN-based tensor generation method and a CUR-based tensor generation method according to an embodiment of the present disclosure.

[0042] Figure 4 (a) is a diagram showing the comparison results between the ANN-based and CUR-based methods of an embodiment of the present disclosure.

[0043] Figure 4 (b) is a diagram showing the comparison results between the ANN-based and CUR-based methods of an embodiment of the present disclosure.

[0044] Figure 4 (c) is a diagram showing the comparison results between the ANN-based and CUR-based methods of an embodiment of the present disclosure.

[0045] Figure 5 FIG. 4 is a diagram showing examples of geometric deformation using rotation, scaling, adding noise and abnormal data on a dataset of a specific shape.

[0046] Figure 6 (a) is a diagram showing the matching result with the angle [-30°, 30°] rotation as deformation.

[0047] Figure 6 (b) shows the scaling factor of 1.1. β Illustration of the matching result as a deformation scaled on the x-coordinate of , where β∈[−5, 5].

[0048] Figure 6(c) is a diagram showing the matching result with noise added as a deformation, where σ / σ 0 =[0, 0.1].

[0049] Figure 6 (d) shows the addition of n l Anomaly data as a graphical representation of the deformed matching results, where the anomaly data ratio is n l / n 1 .

[0050] Figure 7 (a) is a diagram showing the comparison results of House and Hotel datasets using various matching algorithms.

[0051] Figure 7 (b) is a diagram showing the comparison results of House and Hotel datasets using various matching algorithms.

[0052] Figure 7 (c) is a diagram showing the comparison results of House and Hotel datasets using various matching algorithms.

[0053] Figure 7 (d) is a diagram showing the comparison results of House and Hotel datasets using various matching algorithms.

[0054] Figure 8 Graphs showing the results of ANN and CUR based tensor generation methods on car and motorcycle datasets using various prior art hypergraph matching algorithms.

[0055] Fig. 9 (a) is a diagram showing an example of car dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0056] Fig. 9 (b) is a diagram showing an example of car dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0057] Fig. 9 (c) is a diagram showing an example of car dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0058] Fig. 9 (d) is a diagram showing an example of car dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0059] Fig. 9 (e) is a diagram showing an example of car dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0060] Fig. 9 (f) is a diagram showing an example of car dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0061] Fig. 9 (g) is a diagram showing an example of car dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0062] Fig. 9 (h) is a diagram showing an example of car dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0063] Fig.10 (a) is a diagram showing a motorcycle dataset matching example using an example of an ANN-based third-order hypergraph matching algorithm.

[0064] Fig.10 (b) is a diagram showing a motorcycle dataset matching example using an example of an ANN-based third-order hypergraph matching algorithm.

[0065] Fig.10 (c) is a diagram showing an example of motorcycle dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0066] Fig.10 (d) is a diagram showing an example of motorcycle dataset matching using an example of an ANN-based third-order hypergraph matching algorithm.

[0067] Fig.10 (e) is a diagram showing a motorcycle dataset matching example of an example of a third-order hypergraph matching algorithm using the CUR-based tensor generation method.

[0068] Fig.10 (f) is a diagram showing an example of motorcycle dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0069] Fig.10 (g) is a diagram showing an example of motorcycle dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0070] Fig.10 (h) is a diagram showing an example of motorcycle dataset matching using an example of a third-order hypergraph matching algorithm based on the CUR tensor generation method.

[0071] Fig.11(a) is a graph showing the average hit rate of the 100-vs-100 synthetic dataset with σ=0.02 using the CUR-based second-order graph matching algorithm.

[0072] Fig.11 (b) is a graph showing the average precision of the (b) 20-vs-30 CMU House dataset.

[0073] Fig.12 (a) is a graph showing the results of the 100-vs-120 synthetic dataset with different noise levels using the CUR-based tensor generation method and the PRL-based hypergraph matching algorithm.

[0074] Fig.12 (b) is a graph showing the results of the 100-vs-120 synthetic dataset with different noise levels using the CUR-based tensor generation method and the PRL-based hypergraph matching algorithm.

[0075] Fig.13 (a) is a diagram showing the results of the PRL-based matching algorithm on the 100-vs-120 synthetic dataset.

[0076] Fig.13 (b) is a graph showing the results of the PRL-based matching algorithm on the 100-vs-120 synthetic dataset.

[0077] Fig.13 (c) is a graph showing the results of the PRL-based matching algorithm on the 100-vs-120 synthetic dataset. DETAILED DESCRIPTION

[0078] Embodiments of the present disclosure may be designed to be implemented in a digital computing system to improve the efficiency of processing graphs and hypergraph matching.

[0079] In an embodiment of the present disclosure, a cascaded second-order and third-order hypergraph matching framework based on CUR decomposition is provided for solving large-scale graph matching problems. The method provided in the present disclosure has lower computational complexity than the traditional ANN-based tensor generation method. The PRL-based hypergraph matching algorithm disclosed in the present disclosure has a fast convergence speed on sparse compatibility tensors. Experimental results show that the CUR-based tensor generation method and the PRL-based matching algorithm disclosed in the present disclosure can obtain higher matching accuracy under a sparser compatibility tensor. This method has less computational complexity and fewer non-zero elements, can solve large-scale graph matching problems under the same computing resources, and has higher compatibility in the big data era.

[0080] In a preferred embodiment of the present disclosure, there is provided Figure 1The computer system for second-order graph matching shown in FIG. The computer system includes: a memory storing at least two node sets A compatibility matrix (H) representing the similarity between nodes and edges representing the similarity between nodes; and a processing unit configured to receive the compatibility matrix and execute machine instructions.

[0081] The machine instruction comprises the following steps: retrieving a subset of columns and rows from the compatibility matrix (H) in the memory to form a first matrix (C); performing CUR decomposition on the tensor to generate a matrix consisting of the first matrix and the second matrix (U * ) formed by the approximate matrix; applying the soft-constrained second-order graph matching algorithm to the approximate matrix to generate an assignment matrix (X); and identifying the predetermined element (k) with the highest probability in the assignment matrix as a node matching set

[0082] Each of the at least two node sets has a cardinality greater than the columns and rows of the first matrix. The compatibility matrix uses the first matrix (C) and the second matrix (U * ), wherein the size of each of the first matrix and the second matrix is ​​smaller than the size of the compatibility matrix. The first matrix (C) includes randomly selected columns and rows retrieved from the compatibility matrix (H). The second matrix can be composed of generate.

[0083] The machine instructions also include generating a matching set about the node The compatibility tensor can be a sparse tensor and is generated by Fiber-CUR based tensor generation. Fiber-CUR based tensor generation includes the following steps: Select hyperedges; combine the selected hyperedges with the hyperedge subset {{i 2 , j 2 ,:},{i 2 ,:,k 2},{:,j 2 , k 2}}∈ε 2 For comparison, and And select r nearest neighbors for each hyperedge in the source graph to generate a compatibility tensor.

[0084] The machine instructions also include the step of generating a relaxed assignment matrix (X) for the compatibility tensor. The relaxed assignment matrix (X) is generated by a probabilistic relaxed labeling algorithm. The probabilistic relaxed labeling algorithm includes the steps of generating a probability of node i from one of the node matching sets, and generating a probability of node j from another of the node matching sets. The probability of node i being associated with node j is given by express.

[0085] Further, the probability is replaced by a vector x, where x is the column-wise flattening of the soft constraint assignment matrix. The vector x is suitable for being normalized as: The relaxed marking algorithm will continue to iterate on generating the relaxed allocation matrix X until the relaxed allocation matrix X converges.

[0086] The at least two node sets These may be corresponding points from a template with a regular shape (eg, Chinese characters), corresponding points or feature points from a frame image sequence of a building, and shapes from different 2D and 3D objects.

[0087] Hypergraph matching performs better than pairwise graph matching methods, but at the cost of exponentially increased computational complexity. Considering n 1 Node and target 2 The second-order matching method represents the compatibility between two graphs as a compatibility matrix, which has only a space complexity of O((n 1 n 2 ) 2 ). On the other hand, k th The tensor of order is O((n 1 n 2 ) k ). Therefore, it is difficult to handle large-scale graph matching problems with fully compatible tensors.

[0088] In order to reduce the space usage of the tensor, an existing technique uses an approximate nearest neighbor (ANN) algorithm based on a kd-tree to calculate the compatibility tensor by searching for the nearest neighbors between randomly sampled hyperedges in the source graph and all hyperedges in the fully connected target graph. The compatibility tensor generated by the ANN-based algorithm is a sparse tensor with only a small number of non-zero elements representing paired hypergraphs with the highest compatibility. Computing the compatibility with all hyperedges in the target graph requires exhaustive computation for large-scale problems. In addition, achieving higher matching accuracy requires as many nearest neighbors (highest compatibility) as possible to increase the probability of finding ground truth paired neighbors. Therefore, traditional ANN-based tensor generation methods limit the ability to handle large-scale hypergraph matching problems.

[0089] The embodiments of the present disclosure provide a framework based on cascaded second-order and third-order models, which uses CUR decomposition to reduce matching complexity. The framework includes a CUR-based second-order graph matching algorithm that provides coarse matching results, a sparse compatibility tensor generation method based on Fiber-CUR, and a hypergraph matching algorithm based on probabilistic relaxed labeling (PRL) that utilizes high tensor sparsity. Experimental results show that the method has high computational accuracy and can handle larger hypergraph problems less frequently.

[0090] In one embodiment of the present disclosure, it may be possible to derive:

[0091] A cascaded second-order and third-order hypergraph matching method for large-scale hypergraph matching problems. The framework can handle thousands of points, ten times larger than the state-of-the-art algorithms with the same computational resources.

[0092] Aiming at the large-scale graph matching problem, a CUR decomposition method for second-order graph matching is proposed. This method and the symmetric property of the compatibility matrix can further reduce the computational complexity.

[0093] The high-order compatible tensor generation method based on Fiber-CUR reduces the computational complexity by using the second-order graph matching results and effectively selects useful sparse tensor elements. This may be the first task to realize a computer system for generating compatible tensors based on high-order CUR decomposition.

[0094] The PRL-based tensor matching algorithm takes into account the high sparsity of the compatibility tensor and greatly improves the computational efficiency.

[0095] Experimental results demonstrate that the above approach provides state-of-the-art matching accuracy by efficiently finding essential neighbors in the compatibility tensor.

[0096] like Figure 1, a schematic diagram of a computer system or computer server 100 is shown. The computer system or computer server 100 is used to implement an example embodiment of a system for analyzing images. In this embodiment, the system includes the server 100. The server 100 includes appropriate components required to receive, store and execute appropriate computer instructions. These components may include: a processing unit 102, which includes a central processing unit (CPU), a math co-processing unit (Math Processor), a graphics processing unit (GPU), or a tensor processing unit (TPU) for tensor or multi-dimensional array calculation or manipulation operations; a read-only memory (ROM) 104; a random-access memory (RAM) 106; an input / output (I / O) device (e.g., a disk drive 108); an input device 110 (e.g., an Ethernet port, a USB port, etc.); a display 112, such as a liquid crystal display, a light-emitting display, or any other suitable display; and a communication link 114. The computing device 100 may include instructions that may be stored in the ROM 104, the RAM 106, or the disk drive 108 and executed by the processing unit 102. A plurality of communication links 114 may be provided that may be variously connected to one or more computing devices, such as servers, personal computers, terminals, wireless or handheld computing devices, IoT devices, smart devices, and edge computing devices. At least one of the plurality of communication links may be connected to an external computing network via a telephone line or other type of communication link.

[0097] The server 100 may include storage devices such as a disk drive 108, which may include a solid-state drive, a hard disk drive, an optical drive, a tape drive, or a remote or cloud-based storage device. The server 100 may use a single disk drive or multiple disk drives, or a remote storage service 120. The server 100 may also have a suitable operating system 116 resident on the disk drive or in ROM of the server 100.

[0098] The computer or computing device may also provide the necessary computing power to operate or interface with a machine learning network such as a neural network to provide various functions and outputs. The neural network may be implemented locally, or may be accessed or partially accessed through a server or cloud-based service. The machine learning network may also be untrained, partially trained, or fully trained, and / or may also be retrained, modified, or updated over time.

[0099] In this manual, the following symbols are used. A pair of matching diagrams is represented by and express. and The matching results can be obtained by the distribution matrix It indicates that, and Stored in computer memory for processing.

[0100] Since the graph matching problem is NP-hard, in order to obtain an approximate matching result, the elements of the assignment matrix are soft-constrained as Among them, X ij The value indicates in i th Node and Medium th The probability of node matching. In one embodiment, Each node in A point in Each node in matches at most A point in , that is, the matching point is and In the rest of the manual, It is called the source graph. It is called the target graph.

[0101] The compatibility matrix of the second-order graph matching problem, is a fourth-order tensor The expansion of , where Represents edge i 1 , j 1 and i 2 , j 2 The similarity between . The soft constraint assignment matrix is ​​flattened to x. The second-order graph matching problem can be expressed as the optimization of this function:

[0102]

[0103] Among them, ind i ={(i-1)n 2 +1,…,in 2} represents the i of X th The H high-order elements correspond to highly compatible edge pairs. The compatibility matrix is ​​symmetric, which means that {i 1 , j 1} and {i 2 , j 2} and {j 1 ,i 1} and {j 2 ,i2} are the same. Equation (1) becomes the classic Rayleigh quotient problem, where x is associated with the principal eigenvector of H.

[0104] k th The -order hypergraph matching problem can be expanded to:

[0105]

[0106] Among them, k th -rank compatibility tensor Represents compatibility between hyperedges from two graphs. is the mode-l product of a tensor and a vector, which is calculated as:

[0107]

[0108] is supersymmetric, similar to a second-order compatibility matrix, which means that all index permutations {i 1 ,…,i n} are equal.

[0109] All K th The rank-compatible tensor requires O((n 1 n 2 ) k ) space complexity. For the 100-vs-100 point matching problem, the second-order compatibility matrix requires 0.75GB of storage, while the third-order compatibility tensor requires more than 7TB of storage, which is 10,000 times larger than the former. Therefore, for complex applications such as medical imaging, it is impractical to process high-definition images. In particular, for real-time applications, response times within 10 seconds are usually required.

[0110] In addition, the time complexity of (2) is also 0((n 1 n 2 ) k ). In order to reduce the exhaustive space and time complexity, the ANN-based method first computes the complete compatibility tensor blocks between the t sampled hyperedges in the source graph and all hyperedges in the target. Then, the r1 largest element in each tensor block is selected as the element in the compatibility tensor. The time complexity of (2) will be reduced to O(tr 1 ), which is much smaller than O((n 1 n 2 ) k ). However, each tensor block still requires at least Space complexity.

[0111] CUR decomposition is used to compute a low-rank approximation of a matrix with some rows and columns selected from a large matrix. Consider the matrix By selecting c columns and r rows from A as and The low-rank approximation of A can be expressed as where is the pseudo-inverse of the matrix. can be represented as matrix U. It has been proven that if rank(A) < min{c, r}, U can be directly represented as its pseudo-inverse by the intersection of C and R.

[0112] The proposed randomized Fiber-CUR decomposition extends CUR decomposition to tensors. A fiber of a tensor is a one-dimensional vector. Along mode-1, located at the tensor (i 1 , …, i k ) of Fiberv can be expressed as:

[0113]

[0114] This method first samples c l Fibers on mode-l and extends the Fibers to matrix C along mode-l l . The intersection of all samples forms the tensor U l is the pseudo-inverse of the mode-l expansion of The Fiber-CUR decomposition of the tensor can be expressed as:

[0115]

[0116] where, × l denotes the tensor-mode matrix along l m mode.

[0117] The performance of CUR decomposition highly depends on the selection of C and R (or higher-order ). Key samples can achieve high-precision approximation. Therefore, one of the most important tasks of CUR decomposition is to find important samples with less computational effort.

[0118] The second-order compatibility matrix occupies n 1 n 2 × n 1 n 2Space complexity. For example, a 1000-vs-1000 graph matching problem requires the use of 7.27 TB of storage. It is not feasible to handle large-scale graph matching problems involving thousands of points because it requires terabytes of storage. In an embodiment of the present disclosure, a low-rank approximation of the compatibility matrix with CUR decomposition is estimated. Instead of directly computing the entire matrix with terabytes of storage, a small number of rows and columns are loaded into the system memory and processed. In addition, due to the symmetric nature of the compatibility matrix, only column sampling is sufficient to reduce the computational complexity. The sampled columns of the second-order compatibility matrix are used as matrices C and R T For higher low-rank estimation accuracy, more elements in the compatibility matrix will be randomly selected and processed. The detailed algorithm is shown in Algorithm 1, assuming that the c columns of C from H are calculated. The second-order compatibility matrix can be represented by two smaller matrices C and U * Decomposition. Matrix Calculated as:

[0119]

[0120] R Ω (·) represents the matrix elements, including random element selection and C and C T All intersection elements of . ||·|| F represents the Frobenius norm of the matrix. During the power iteration process, the multiplication result of Hx can be simplified to:

[0121] Hx≈CU * (C T x) (7)

[0122] This reduces the speed from O((n 1 n 2 ) 2 ) to O(cn 1 n 2 ) time and space complexity. For large-scale graph matching problems, c<<n 1 n 2 For large-scale graph matching problems, memory usage will be reduced from Terabytes to Gigabytes, or even to Megabytes.

[0123] By applying an arbitrary soft-constrained second-order graph matching algorithm, a rough second-order graph matching result X of the CUR decomposition of H is generated in a processing unit of the computing system. For H, select X i，： The k elements with the highest probability in are the best k matching sets Note that if only second-order matching is applied, the CUR-based method can lead to lower matching accuracy with fewer sampled rows and columns. However, it provides coarse results for subsequent high-order graph matching, which avoids the infeasibility of large-scale graph matching. All nodes in The k best matches in ) will be used to generate a high-rank compatibility tensor.

[0124]

[0125] In one embodiment of the present disclosure, the processing unit is configured to use the second-order graph matching result with the best k matches. Reduce calculation, where k<<n 2 For a third-order hyperedge {i 1 , j 1 , k 1}∈ε 1 , this method does not need to compare the hyperedge with all the hyperedges in the fully connected target hypergraph, but instead compares the smaller set of hyperedges {{i 2 , j 2 ,:},{i 2 ,:,k 2},{:,j 2 , k 2}}∈ε 2 ,in,

[0126] In one embodiment of the present disclosure, the processing unit of the computer system is adapted to select r nearest neighbors for each hyperedge in the source graph. Since each hyperedge is compared with much fewer hyperedges, the complexity of the algorithm can be significantly reduced to O(tk d-1 n 2 +tr). The overall algorithm can be summarized as Algorithm 2. The supersymmetric elements in the compatibility tensor will be removed, and each non-zero element will represent all its supersymmetric correspondences for the large-scale hypergraph matching problem.

[0127] Figure 2 The ANN-based tensor generation method commonly used in the hypergraph matching algorithm of the prior art and the Fiber-CUR-based method according to the embodiment of the present disclosure used in their mathematical notation are compared. Instead of computing the entire tensor block and extracting the nearest neighbors in the block, the processing unit of the embodiment of the present disclosure is programmed to compute a small number of block fibers and find fewer nearest neighbors in these fibers. (b) Fibers along one mode are shown in a neat view. Fibers are selected in all three modes.

[0128] For the third-order hypergraph matching problem, the total tensor size is (n 1 n2 ) 3 The compatibility between each hyperedge in the source graph and all hyperedges in the fully connected target graph can be expressed as The traditional ANN-based method randomly selects t tensor blocks and calculates the complete tensor block, which regards the time and space complexity as In each block, select r 1 The method disclosed in the present invention does not need to calculate all elements in each tensor block. Similar to the second-order graph matching based on CUR, the second-order graph matching results are used to The processing unit is configured to calculate the corresponding fibers in the tensor block and select the r highest order elements in these fibers. The processing unit of the present disclosure is suitable for generating each tensor block As a nearest neighbor problem only, no redundant computation of U and is required during tensor generation. Therefore, compared with the original random Fiber-CUR decomposition, the compatibility tensor requires less computation during tensor generation and requires less computational complexity during the hypergraph matching process in (2). Sampled Fiber is based on Rather than random sampling, it is closer to key sampling. Ground truth matching neighbors with higher probability can be selected, thus achieving higher matching accuracy. Compared with ANN-based methods, its computational complexity is reduced and the generated tensors are sparser. Therefore, it can handle larger-scale hypergraph matching problems.

[0129]

[0130] The Probabilistic Relaxed Labeling (PRL) algorithm aims to update the probability of assigning labels to objects with a specified set of compatibilities, which has been applied to various graph matching problems. and represents the probability of association between i and j, which can be updated according to the following equation:

[0131]

[0132] as well as

[0133]

[0134] Among them, d iL is the relationship between l and i and ∑ l d il =1 is the weight factor of the influence.

[0135] In one embodiment, r il (j, m) replaced by R il (j, m) = 0.5 (ril (j, m)+1, set Then, we get:

[0136]

[0137] in, It plays a weighting role. Equation (10) is similar to the standard power iteration. The weighting factor increases the larger probability to 1 and the smaller probability to 0.

[0138] Further replacing the weighting factors with the updated probabilities, (10) becomes:

[0139]

[0140] Get more consistent results and converge faster.

[0141] For the numerator of (11), the following terms are derived:

[0142]

[0143] Here, we define

[0144]

[0145] in, represents the corresponding value in the compatibility matrix H, Involved nodes With Node First-order compatibility between them. Since for all Therefore (12) can be updated as:

[0146]

[0147] make In order to increase the corresponding true matches during each iteration Must have This leads to

[0148]

[0149] In other words, if is guaranteed, the probability can converge quickly.

[0150] Now, the algorithm is extended to the third-order hypergraph matching problem. The steps are shown in Algorithm 3. The probability is replaced by the vector x, which is a column-by-column expansion of the soft constraint assignment matrix X. Using the normalization process, the processing unit can be reconfigured (14) as

[0151]

[0152] in,(·) 2 is element-by-element, α∈[0,1].⊙ represents the element-by-element multiplication of two vectors. By column-by-column expansion Get the first-order compatibility vector For tensors with high sparsity, The non-zero values ​​of are concentrated. Therefore, x (k) With fast convergence speed, the proposed PRL-based algorithm is suitable for compatible tensors with high sparsity and relatively low α.

[0153]

[0154]

[0155] The Fiber-CUR based compatibility tensor generation method (denoted as CUR) of the present disclosure is implemented in a computing system and compared with a series of prior art hypergraph matching algorithms based on ANN (denoted as ANN): Tensor Matching (TM), Hypergraph Matching by Weighted Random Walk (RRWHM), Third Order BCAGM (BCAGM3), and Alternating Directional Graph Matching (ADGM) with original implementation. The all-1 vector will be set as the starting point, and the Hungarian algorithm will convert the output into the appropriate match. The framework of the present disclosure is compared with the prior art learning-based hypergraph matching algorithm HNN-HM on the House and Hotel datasets, which has a relatively small scale. Because the prior art HNN-HM fails on large-scale datasets, it cannot even be used for comparison with the present disclosure. The experiments are applied to a dataset with Core TM Computer with i7-9700 CPU @ 3.00GHz and 16GB RAM.

[0156] Two pairs of nodes {i 1 , j 1} and {i 2 , j 2} compatibility is defined as

[0157]

[0158] Among them, v i，j represents the vector point from node i to node j, d i，j represents the distance between node i and node jA(·), A(·) represents the directional similarity between pairs of nodes, and D(·) represents the distance similarity:

[0159]

[0160]

[0161] Randomly process c columns into C with uniform distribution and process another c 2 The random elements in the compatibility matrix are used to calculate U in (6). The second-order PRL algorithm is used to calculate the soft constraint assignment matrix.

[0162] For each hyperedge, a feature vector f is generated based on the cosine of the angle of the triangle formed. 1 , j 1 , k 1 ) and (i 2 , j 2 , k 2 ) is defined as:

[0163]

[0164] Wherein, γ is the inverse of the average value of all generated tensor elements. The processing unit is configured to randomly select t tuples from the source graph. Using the same recursion, the CUR-based method generates a tensor with r nearest neighbors from the target graph, and the ANN-based method generates a tensor with r nearest neighbors. 1 A tensor of nearest neighbors.

[0165] 16) is calculated as:

[0166] M i，j =exp(-γ 0 ||f i -f j || 2 ) (twenty one)

[0167] Among them, f i and f j P 1 i th Point and P 2 J in th Normalized coordinates of the points. In this experiment, the processing unit is adapted to normalize the coordinates by subtracting the mean of the coordinates in each group. 0 It is from P 1 From point to P 2 The inverse of the average of all distances of the points in M ​​is then column-flattened into a first-order compatibility vector m. α was set to 0.2 throughout the experiments.

[0168] The processing unit of the computer system is configured to first evaluate the method of the disclosed embodiment on a large-scale synthetic data set. Approximately 10002D points, P, are drawn from a Gaussian distribution. By sampling the Gaussian deformation noise Add to P to generate the point set Q. During the experiment, n are selected from P 1points as the internal points of the source image, and select n corresponding matching points of the source image from Q 2 points and additional outlier points as target graph points. T tuples are selected from the source points. For the traditional tensor generation method, r1=300 best matches will be selected for each tuple. For the method according to the embodiment of the present disclosure, c and r are selected to be relatively small. For the illustrations in this section, each quantitative result is obtained by 50 trials.

[0169] The CUR-based framework is implemented on a synthetic dataset with increasing graph size n. 1 n 2 .σ is set to 0.005. The results are shown in Table 1. As a comparison, the results of the ADGM algorithm and the ANN-based method are shown, which are the highest among all existing hypergraph matching algorithms. It can be observed that with low c and k, the architecture according to the embodiment of the present disclosure can handle large-scale graph matching problems and achieve privileged matching accuracy. The CUR-based method is able to handle the 1000-vs-1000 matching problem with high accuracy. On the other hand, under the same memory constraint, when n 1 When >300, traditional ANN-based methods cannot produce tensors.

[0170] Table 1

[0171] Results on synthetic datasets using ANN-ADGM and CUR-PRL.

[0172]

[0173] *Tensor generation failed.

[0174] The algorithm was also implemented on a smaller dataset for comparison. The processing unit was first configured to set σ to 0.02 and n 1 =100, and n 2 From 100 to 300, this means that in the target map There are abnormal data ranging from 0 to 200. Figure 4 (a) and Figure 4 The results in (b) show that the CUR-based method of the disclosed embodiment can produce tensors that achieve much higher accuracy using about ten times the compatibility tensor of the ANN-based method. Then, the processing unit is configured to set n 2 is 100, and σ varies from 0 to 0.1. Figure 4 The results in (c) show that the matching algorithm based on the CUR method has higher matching accuracy than the matching algorithm based on the ANN method. The CUR-based method can produce a compatibility tensor with higher robustness.

[0175] The proposed CUR-based tensor generation method further uses the PRL-based matching algorithm of the embodiment of the present disclosure to evaluate some existing templates with regular shapes. Compared with the randomly generated synthetic dataset, it can better demonstrate the performance of different matching algorithms. Four 2D templates are selected: 150 points for whales, 105 points for Chinese characters, 98 points for UCF fish, and 91 points for tropical fish. The processing unit of the present disclosure is configured to create target points by rotating, scaling, adding noise, and adding abnormal data, such as Figure 5 As shown. The matching results generated by the embodiments of the present disclosure are compared with the hypergraph matching algorithm of the prior art. During the experiment, for the CUR-based method, c=100, k=5 and r=25. For the ANN-based method, r1 is set to 300.t=0.3n 1 n 2 , used for both methods.

[0176] In one embodiment, the processing unit is configured to add different deformations to all target points. For the case of rotation, the processing unit is configured to first rotate the source point by an angle θ∈[-30°, 30°] and add σ=0.02σ 0 Add noise, where σ 0 is the standard deviation of all source point coordinates. For the case of scaling, the processing unit is configured to scale with a factor of s = 1.1 β To scale the x-coordinate of a 2D point, where β∈[-5,5] and σ=0.02σ 0 Add noise. To add noise, the processor is configured to vary the noise, σ∈[0, 0.1σ 0 ]. To add abnormal data, the mean value of the source points is assumed to be μ 0 , the processing unit is configured to draw from a Gaussian distribution Randomly add n l abnormal data, among which the abnormal data ratio is

[0177] The results of the head test are shown below Figure 6Due to the rotation invariance of the hyperedge features, almost all algorithms show an average accuracy close to 1 under changes in rotational deformation, regardless of the ANN and CUR-based methods. The curves for the other three cases show that the matching algorithm using the method of the present invention is better than the ANN. Take the case of scaling as an example. Since the scaling process only scales the x-coordinate of the target point, the hyperedge feature is basically damaged. As the scale factor increases, the matching results of the ANN-based algorithm are significantly reduced. In contrast, the method of the present invention can produce a compatibility tensor with higher robustness. After integrating the method of the present invention, the performance of the compared matching algorithms is significantly improved and more competent for the job. In particular, in all cases, due to the unfavorable noise level, the results of RRWHM are worse than other algorithms. With the help of the compatibility tensor generated by the method of the present invention, the matching results of the same RRWHM algorithm are stabilized and show results comparable to those of other algorithms.

[0178] Previous work has also extensively used the CMU House and Hotel datasets to evaluate matching algorithms. The CMU House dataset has 30 manually labeled feature points on a rotated 3D House model tracked on a 111-frame image sequence. The CMUHotel dataset has 30 manually labeled points of a Hotel model tracked on a 101-frame image sequence. The processing unit is configured to match all possible pairs, where the baseline (interval between frames) of these pairs varies from 10 to 100 in intervals of 10 for the House dataset and from 10 to 90 in intervals of 10 for the Hotel dataset. The processing unit is configured to match 20 and 30 points in the first image with all 30 points in the second image. Select t=n from the first image 1 n 2 For ANN-based methods, select r for each super-sparse 1 = 200 nearest neighbors. For the method disclosed herein, r is set to 25 and c = 300.

[0179] The sequence matching results are as follows Figure 7 The dashed line represents the matching result on the compatibility tensor using the ANN-based method. The solid curve with the same color represents the matching accuracy on the tensor generated by the proposed CUR-based method with the same hypergraph matching algorithm.

[0180] In all cases except the BCAGM3 algorithm, all the matching algorithms based on the CUR method without learning have matching accuracy that exceeds that based on ANN. The BCAGM3 algorithm based on the CUR algorithm often shows reduced errors in experiments. When the compatibility tensor is too sparse, its matching accuracy is greatly reduced. When the abnormal data is 10, the PRL-based matching algorithm and the CUR-based matching algorithm have matching accuracy comparable to the matching algorithms based on learning without learning in the prior art, and have higher matching accuracy for the 30-vs-30 point problem on the House dataset. For the Hotel dataset, the method according to the embodiment of the present disclosure is configured to first consider the distance and rotation invariance during the second-order graph matching process, which therefore has a much higher matching accuracy than the ANN-based method. Compared with the HNN-HM based on deep learning, the method of the present disclosure lags behind the 20-vs-30 problems because the HNN-HM based on deep learning is specially trained on the House dataset. However, the method of the present disclosure still achieves the best matching results in the 30-vs-30 problem. The method disclosed in the present invention solves the matching problem of the unified model, while HNN-HM, on the other hand, must first train separate models in different datasets to obtain better matching results.

[0181] The present disclosure is used to process a car and motorcycle dataset, which includes 30 real-life car image pairs and 20 motorcycle image pairs with different scale feature points and shapes. In this experiment, the correct data points in both images are kept, and the abnormal data points in the second image are randomly selected, where the number changes from 0 to 30 in steps of 5. t=n is calculated in the first image 1 n 2 For the traditional method, r1 is set to 0.3n 1 n 2 According to the method of the embodiment of the present disclosure, r=50 nearest neighbors in each tensor block will be selected, and k will be set to 10. c=300 is used for the initial guess during CUR-based second-order graph matching.

[0182] Each image pair in the dataset is matched. The system reports the average precision using the ANN-based and CUR-based tensor generation methods. The system also reports the average number of non-zero elements in the tensors produced by the two methods.

[0183] Comparison results such as Figure 8 shown. Figure 8 The first column shows the results for the car dataset, and the second column shows the results for the motorcycle dataset.

[0184] It can be observed that for the same matching algorithm, the graph matching algorithm can achieve higher accuracy using the CUR-based method when the graph is more than ten times sparser than the ANN-based method. In most cases, the PRL-based matching algorithm of the present disclosure can achieve higher accuracy than the hypergraph matching algorithm of the prior art.

[0185] Fig. 9 and 10 Several matching examples are shown. Fig. 9 An example of matching the car dataset using various third-order hypergraph matching algorithms based on ANN and CUR tensor generation methods is shown. There are 30 correct data points in both images and 10 abnormal data points in the second image. The green line indicates a correct match and the red line indicates a mismatch. Fig.10 An example of matching a motorcycle dataset using various third-order hypergraph matching algorithms based on ANN and CUR tensor generation methods is shown. There are 52 correct data points in both images and 15 anomalous data points in the second image. The green line indicates a correct match and the red line indicates a mismatch.

[0186] Using the same hypergraph matching algorithm, there are fewer mismatches in the CUR-based method. The proposed PRL-based matching algorithm using the CUR-based tensor generation method has the highest matching accuracy.

[0187] To analyze the impact of the second-order CUR-based matching results, the processing unit is configured to first generate the entire compatibility matrix for the 100-vs-100 point synthetic data set, σ = 0.02. The processing unit is configured to calculate the CUR decomposition and relaxed assignment matrix with less than 100 columns using a second-order PRL-based matching algorithm. The hit rate with increasing k is calculated based on the assignment matrix and is defined as in

[0188]

[0189] The results are as follows Fig.11 (a). Each curve indicates how many columns are randomly selected during CUR decomposition. The more columns are selected, the higher the hit rate is, but the hit rate does not increase much when it exceeds 10 columns.

[0190] The impact of the second-order graph matching results is further analyzed by implementing the House dataset with a 20-vs-30 point problem. The processing unit is configured to set t=3n 1 n 2 and r = n 1 n 2, used for the ANN-based tensor generation method. Then, the processing unit is configured to directly select the corresponding fiber on the tensor block using the result of the CUR-based second-order matching algorithm to simulate the compatibility tensor generated by the CUR-based method. c is set to 10. The matching results with RRWHM and ADGM are as follows Fig.11 (b) shows that the matching results using the non-zero elements of the corresponding fiber utilize the matching results using all non-zero elements generated by the ANN-based method. Therefore, it can be proved that the fiber selected by the CUR-based second-order graph matching result is preferred.

[0191] In one embodiment of the present disclosure, the processing unit is configured to analyze how the selection of k and r affects the final matching accuracy. In this experiment, the processing unit is configured to use the proposed PRL-based hypergraph matching algorithm to analyze the impact of the compatibility tensor. The processing unit is configured to use a 100-vs-120 point synthetic dataset with various noise deviations σ of the dataset.

[0192] To analyze the effect of k, the processing unit was configured to first set r = 50 and then change k from 1 to 15 in steps of 2. Fig.12 (a) shown. Fig.12 (a) shows the matching accuracy with increasing number of points selected from the second-order graph matching algorithm. Fig.12 (b) shows the matching accuracy with increasing nearest neighbors in each tensor block.

[0193] For low noise levels, the matching accuracy first increases rapidly and then converges to 1, which has the same Fig.11 Similar curves in (a). It can be observed that higher k achieves higher matching accuracy. However, the computational effort will increase with the increase of k. For matching results with high noise levels, such as σ ≥ 0.02, the matching accuracy increases to its highest and gradually decreases. For σ = 0.03, the highest point is at k = 11. For σ = 0.05, it decreases due to k = 9. As σ increases, the maximum point drifts to fewer k. Therefore, if the target point is severely damaged, a larger k may reduce the matching accuracy.

[0194] By setting k = 5 and changing r from 1 to 181 in steps of 20, the effect of r is analyzed. The final result is as follows Fig.12 (b) is shown. Fig.12The results in (a) are different, and for low noise levels, the matching accuracy decreases from its peak. For denoised points, the second-order graph matching algorithm itself can have relatively high accuracy. If the processing unit is configured to select too many nearest neighbors on the tensor Fiber, these elements may interfere with the third-order matching and degrade the final results. For high noise levels, the super-sparseness of the true match may not have the highest similarity. Therefore, when r is about 20°, the matching accuracy reaches a peak and then gradually decreases. It can be noted that for σ = 0.07, the matching accuracy continues to increase. The maximum point drifts slowly to higher r, which is opposite to the effect of k.

[0195] In summary, higher k helps achieve higher matching accuracy. However, if the points are severely deformed, it may also reduce matching performance. Similarly, denser tensor blocks Fiber may reduce the performance of the graph matching algorithm. The best parameters k and r for the best matching results are different when dealing with different matching problems and can be configured by the processing unit.

[0196] Finally, the influence of parameters on the PRL-based matching algorithm is analyzed. The system of the present disclosure is configured to apply the ANN-based and CUR-based tensor generation methods to a 100-vs-120 point synthetic data set, σ∈[0,0.1]. For the ANN-based method, the processing unit is configured to set t=1200 and r1=200.

[0197] The PRL-based matching algorithm is evaluated on two compatibility tensors, α∈[0,1]. For the CUR-based method, the processing unit is configured to set k=5 and r=25. To analyze the convergence of the algorithm, the processing unit is configured to set the stopping criterion of the PRL-based algorithm to ||x k+1 -x k || 2 ≤10 -8 , set the maximum iterations to 1000. Fig.13 (a) shows the average matching accuracy of ANN-PRL and CUR-PRL using different α. Fig.13 (b) shows the corresponding average convergence iterations. For σ ≥ 0.01, ANN-PRL does not converge. Fig.13 (c) shows ||x during iteration k using CUR-PRL and ANN-PRL with different α settings. k+1 -x k || 2. Set σ to 0.1. With the compatibility tensor generated by the CUR-based method, the PRL algorithm shows equal matching results regardless of α. However, the matching results of the ANN-based method show almost zero matching accuracy. For the CUR-based method, a smaller α can produce fewer convergence iterations, while for the ANN-based method, the PRL-based algorithm hardly converges. Therefore, the PRL-based method is very suitable for the proposed CUR-based method.

[0198] Although not required, the embodiments described with reference to the accompanying drawings may be implemented as an application programming interface (API) or a series of libraries used by developers, or may be included in another software application, such as a terminal or personal computer operating system or a portable computing device operating system. Generally, since program modules include routines, programs, objects, components, and data files that help perform specific functions, those skilled in the art will understand that the functionality of a software application may be distributed among multiple routines, objects, or components to achieve the same functionality required herein.

[0199] It should also be understood that in the case where the methods and systems of the present disclosure are implemented in whole or in part by a computing system, any suitable computing system architecture may be used. The computing system may include tablet computers, wearable devices, smart phones, IoT devices, edge computing devices, stand-alone computers, network computers, cloud-based computing devices, and dedicated hardware devices. Where the terms "computing system" and "computing device" are used, these terms are intended to cover any suitable configuration of computer hardware capable of implementing the described functionality.

[0200] It will be appreciated by those skilled in the art that various changes and / or modifications may be made to the present disclosure as shown in the specific embodiments without departing from the spirit or scope of the present disclosure as broadly described. Therefore, the present embodiments are to be considered in all aspects as illustrative and not restrictive.

[0201] Unless otherwise indicated, any reference to prior art contained herein is not to be taken as an admission that the information is common general knowledge.

Claims

1. A computer system for second-order graph matching, characterized in that: The system comprises: A memory storing at least two node sets A compatibility matrix (H) representing the similarity between the edges of the nodes; A processing unit is configured to receive the compatibility matrix and execute machine instructions, the machine instructions comprising the following steps: retrieving a subset of columns and rows from said compatibility matrix (H) in memory to form a first matrix (C); Perform CUR decomposition on the tensor to generate an approximate matrix (U * ); Applying a soft-constrained second-order graph matching algorithm to the approximate matrix to generate an assignment matrix (X); Identify the predetermined element (k) with the highest probability in the assignment matrix as the node matching set 2. The system according to claim 1, characterized in that in, Each of the at least two node sets has a cardinality greater than the columns and rows of the first matrix.

3. The system according to claim 2, characterized in that The compatibility matrix uses the first matrix (C) and the second matrix (U * ) decomposition, wherein the size of each of the first matrix and the second matrix is ​​smaller than the size of the compatibility matrix.

4. The system according to claim 3, characterized in that in, The second matrix is ​​given by generate.

5. The system according to claim 4, wherein the first matrix (C) comprises randomly selected columns and rows retrieved from the compatibility matrix (H).

6. The system according to claim 5, characterized in that in, The machine instructions also include generating information about the node matching set The compatibility tensor of steps.

7. The system according to claim 6, characterized in that in, The compatibility tensor is a sparse tensor.

8. The system according to claim 7, characterized in that in, The compatibility tensor is generated by Fiber-CUR based tensor generation.

9. The system according to claim 8, characterized in that in, The Fiber-CUR based tensor generation includes the following steps: From the node matching set Select a hyperedge from one of them; Combine the selected hyperedge with the three subgraphs For comparison, and For each hyperedge in the source graph, r nearest neighbors are selected to generate the compatibility tensor.

10. The system according to claim 9, characterized in that in, The machine instructions also include the step of generating a relaxed allocation matrix (X) for the compatibility tensor.

11. The system according to claim 10, characterized in that in, The relaxation allocation matrix (X) is generated by a probabilistic relaxation labeling algorithm.

12. The system according to claim 11, characterized in that in, The probabilistic relaxed labeling algorithm includes the steps of generating a probability of node i from one of the node matching sets, and generating a probability of node i from another of the node matching sets.

13. The system according to claim 12, characterized in that in, The probability that node i is associated with node j is given by express.

14. The system according to claim 13, characterized in that in, The probabilities are replaced by a vector x, where x is the column-wise flattened version of the soft constraint assignment matrix.

15. The system according to claim 14, characterized in that in, The vector x is suitable to be normalized as:

16. The system according to claim 15, characterized in that in, When the relaxation allocation matrix X converges, the relaxation marking algorithm stops.

17. The system of claim 16, wherein: in, The at least two node sets are corresponding points in the template with regular shape.

18. The system according to claim 17, characterized in that in, The template with a regular shape is composed of features of the object to be matched.

19. The system according to claim 16, characterized in that in, The at least two node sets are corresponding points in two images of the same object taken under different conditions, including two objects to be matched in two images or two frames of a video.

20. The system according to claim 19, characterized in that in, The at least two node sets Corresponding feature points and shapes for different 2D and 3D objects.

Images