A graph matching method and system

Abstract

The present invention provides a graph matching method and system, which belongs to the field of computer vision, and can: input two attribute graphs to be matched: graph G 1 and Figure G 2 ; Construct graph G 1 Figure G 2 The affinity matrix W all ; Update the affinity matrix W based on neighbor node pairs all , get the updated affinity matrix W U ; Based on the updated affinity matrix W U , solve graph G 1 Figure G 2 The present invention is used to reduce mismatching and improve matching accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a graph matching method and system. Background Art

[0002] A graph is composed of points and edges. Points can represent specific objects, and edges can represent the relationship between specific objects. Graph matching is to find a correspondence between points in two graphs through a certain method. Since many things in reality can be abstracted as graphs and then analyzed, graph matching technology has applications in many fields, such as target recognition, feature tracking, binocular vision and other applications in the fields of computer vision and pattern recognition, protein structure analysis in the biological field, and in the process of discovering useful information in the field of data mining. Graph matching is often used.

[0003] In graph matching, the vertices of the graph represent the extracted image feature points, and the edges represent the information between the feature points of two images, which are used to measure the similarity between the point pairs. Specifically, when performing graph matching, traditional graph matching methods often construct a descriptor for each feature point, and then complete the matching of feature points by comparing the similarities between descriptors. The descriptor constructed for the feature point is a high-dimensional vector that is used to uniquely represent a feature point. However, when there are weak texture areas or repeated texture areas, the uniqueness of the descriptor is weakened, and it is easy to make false matches, affecting the matching accuracy. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a graph matching method and system to reduce mismatching and improve matching accuracy.

[0005] In a first aspect, the present invention provides a graph matching method, comprising the steps of:

[0006] S1: Input two attribute graphs to be matched: Graph G 1 and Figure G 2 , G 1 =(V 1 , E 1 , A 1 ), G 2 =(V 2 , E 2 , A 2 ), where V x 、E x , A x Represent the attribute graph G respectively x The node set (i.e. vertex set), edge set, and attribute set of x = 1, 2;

[0007] S2: Constructing graph G 1 Figure G 2 The affinity matrix Wall ;

[0008] S3: Update the affinity matrix W based on neighbor node pairs all , get the updated affinity matrix W U ;

[0009] S4: Based on the updated affinity matrix W U , solve graph G 1 Figure G 2 matching relationship.

[0010] Furthermore, the implementation method of step S2 includes the steps of:

[0011] S21: Constructing graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ;

[0012] S22: Constructing graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ;

[0013] S23: Constructing graph G 1 and Figure G 2 The global feature affinity matrix W global ;

[0014] S24: Constructing graph G 1 and Figure G 2 The intermediate feature affinity matrix W mid ;

[0015] S25: construct the local distance affinity matrix W local-distance , local angle affinity matrix W local -angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is:

[0016]

[0017] Where k = 0.6 or 0.7.

[0018] Furthermore, the local distance affinity matrix W constructed in step S21 local-distance =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the local distance affinity matrix Wlocal-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0019]

[0020] in: is the local distance affinity matrix W local-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, Representative Figure G 1 The i-th node in Representative Figure G 1 The jth node in Representative Figure G 2 The a-th node in Representative Figure G 2 The bth node in and Composition diagram G 1 and Figure G 2 A pair of candidate correspondences between Representation graph G 1 The i-th node in the graph G 1 The Euclidean distance of the jth node in, Representation graph G 2 Midpoint To Node Euclidean distance; i = 1, 2, ..., |V 1 |; j = 1, 2, ..., | V 1 |; a = 1, 2, ..., | V 2 |;b=1,2,...,|V2|;|V x |Representative Figure G x The number of nodes, x=1,2.

[0021] Furthermore, the local angle affinity matrix W constructed in step S22 local-angle =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the local angle affinity matrix W local-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0022]

[0023] in, is the local angle affinity matrix W local-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position,

[0024] Representation Node To Node The vector of Representation Node To Node The vector of Representation vector With vector The product of Representation vector The model, Representation vector Model.

[0025] Furthermore, the implementation method of step S23 is:

[0026] Construct attribute graph G 1 , G 2 The global distance affinity matrix W between global-distance ;

[0027] Construct attribute graph G 1 , G 2 The global angular affinity matrix W global-angle ;

[0028] Fusion of the above global distance affinity matrix W global-distance and the global angular affinity matrix W global-angle , get the global feature affinity matrix W global , the fusion formula is:

[0029] W global =(W global-distance +W global-angle ) / 2.

[0030] Furthermore, the constructed global distance affinity matrix W global-distance =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the global distance affinity matrix W global-distance In ((a-1)×|V 1|+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0031]

[0032] in, is the global distance affinity matrix W global-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, c 1 、c 2 Figure G 1 Figure G 2 The central node of and In turn, nodes To the central node c 1 The Euclidean distance and nodes To the central node c 1 The Euclidean distance of and In turn, nodes To the central node c 2 The Euclidean distance and nodes To the central node c 2 The Euclidean distance.

[0033] Furthermore, the constructed global angle affinity matrix W global-angle =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the global angle affinity matrix W global-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0034]

[0035] in, is the global angle affinity matrix W global-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, For Figure G 1 Middle node c 1 , The three internal angles of the triangle formed by the three sides between them, ∠θ1, ∠θ2 and ∠θ3 are the angles of the node c in graph G2. 2 , The three interior angles of the triangle formed by the three sides between them, With Node c 1 Correspondingly, ∠θ1, ∠θ2 and ∠θ3 are related to the nodes c 2 Correspond in sequence.

[0036] Furthermore, the implementation method of step S24 is:

[0037] In Figure G 1 In As the center of the circle, Draw a circle with radius , and the intersection of the two circles gives three areas distributed in sequence: the first area, the first central area, and the second area. Get the center points of the first area, the first central area, and the second area, and record them as Connect nodes sequentially and

[0038] In Figure G 2 In As the center of the circle, Draw a circle with radius , and the two circles intersect to obtain three areas distributed in sequence: the third area, the second central area, and the fourth right area; obtain the center points of the obtained third area, the second central area, and the fourth right area, and record them as Connect nodes sequentially and

[0039] Computational graph G 1 and Figure G 2 The intermediate distance affinity matrix W mid-distance ;

[0040] Computational graph G 1 and Figure G 2 The intermediate angle affinity matrix W mid-angle ;

[0041] The intermediate distance affinity matrix W obtained by fusion calculation mid-distance and the intermediate angle affinity matrix W mid-angle , and obtain the intermediate feature affinity matrix W mid , the fusion formula is: W mid =(W mid-distance +W mid-angle ) / 2.

[0042] Furthermore, the constructed intermediate distance affinity matrix W mid-distance=(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the intermediate distance affinity matrix W mid-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0043]

[0044] in: is the ((a-1)×|V in the mid-distance affinity matrix Wmid-distance 1 |+i) rows, ((b-1)×|V 2 |+j) are elements in the column position, distance1 and distance2 are row vectors,

[0045] and In turn, nodes To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of Node To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of ||distance1-distance2||1 represents the norm of vector distance1 and distance2;

[0046] The constructed intermediate angle affinity matrix W mid-angle =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the intermediate angle affinity matrix W mid-angleIn ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is:

[0047]

[0048] in: is the intermediate angle affinity matrix W mid-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) is the element in the column position, angle1 and angle2 are row vectors, angle1 = (∠α1, ∠α2, ∠α3, ∠α4), angle2 = (∠β1, ∠β2, ∠β3, ∠β4), ||angle1-angle2||1 represents the norm of vectors angle1 and angle2, ∠α1, ∠α2, ∠α3 and ∠α4 are nodes Connect nodes in sequence The corresponding internal angles in the formed polygon; ∠β1, ∠β2, ∠β3 and ∠β4 are nodes Connect nodes in sequence The corresponding interior angles of the formed polygon.

[0049] Furthermore, before step S25, the following steps are also included:

[0050] The constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The matrices are normalized respectively, and the normalized expressions are:

[0051] In the formula, max(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The maximum value, min(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The minimum value in mRepresents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid .

[0052] Furthermore, the implementation steps of step S3 are:

[0053] For Figure G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U :

[0054] S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows:

[0055]

[0056] in: Represents the affinity matrix W all ((a-1)|V 1 |+i) rows, ((b-1)|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W ：；jb Represented by the matrix W all The ((b-1)|V 2 |+j) columns, ∑rank(W ia；： , 4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb , 4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

[0057] In a second aspect, the present invention provides a graph matching system, comprising:

[0058] An input unit is configured to input two attribute graphs to be matched, wherein the two attribute graphs are graphs G 1 and Figure G 2 , G 1 =(V 1 , E 1 , A 1 ), G 2 =(V 2 , E 2 , A 2 ), where V x 、E x , A x Represent the attribute graph G respectively x The node set, edge set, and attribute set of x = 1, 2;

[0059] An affinity matrix construction unit is configured to construct the graph G 1 Figure G 2 The affinity matrix W all ;

[0060] The affinity matrix updating unit is configured to update the affinity matrix W based on the neighbor node pairs. all , get the updated affinity matrix W U ;

[0061] A matching solving unit is configured to match the updated affinity matrix W U , solve graph G 1 Figure G 2 matching relationship.

[0062] Furthermore, the affinity matrix construction unit includes:

[0063] The first module is configured to construct a graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ;

[0064] The second module is configured to construct the graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ;

[0065] The third module is configured to construct the graph G 1 and Figure G 2 The global feature affinity matrix W global ;

[0066] The fourth module is configured to construct the graph G1 and Figure G 2 The intermediate feature affinity matrix W mid ;

[0067] The fifth module is configured to construct the local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is:

[0068]

[0069] Where k = 0.6 or 0.7;

[0070] The affinity matrix updating unit is configured to perform the following steps:

[0071] For the graph G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U :

[0072] S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows:

[0073]

[0074] in: Represents the affinity matrix W all ((a-1)|V 1 |+i) rows, ((b-1)|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W：；jb Represented by the matrix W all The ((b-1)|V 2 |+j) columns, ∑rank(W ia；： , 4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb , 4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

[0075] The beneficial effects of the present invention are:

[0076] (1) The graph matching method and system provided by the present invention do not need to construct a descriptor, and can update and optimize the affinity matrix W based on neighbor node pairs. all Get the updated affinity matrix W U , then based on the updated affinity matrix W U Solve graph G 1 Figure G 2 The matching relationship helps to overcome the shortcomings of the traditional graph matching method recorded in the background technology, helps to reduce the affinity of the wrong matching relationship, and then reduces the wrong matching and improves the matching accuracy.

[0077] (2) The graph matching method and system provided by the present invention can be based on the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion to get the affinity matrix W all , the intermediate feature affinity matrix W is introduced mid , which helps to further reduce the affinity of false matching relationships and improve matching accuracy.

[0078] (3) The graph matching method and system provided by the present invention do not require the construction of high-dimensional descriptors, which helps to improve matching efficiency.

[0079] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0081] Figure 1is a schematic flow chart of a method according to an embodiment of the present invention.

[0082] Figure 2 is a candidate corresponding relationship in a method of an embodiment of the present invention Schematic diagram of the global angle feature map.

[0083] Figure 3 is a candidate corresponding relationship in a method of an embodiment of the present invention Schematic diagram of the global angle feature map.

[0084] Figure 4 G in the method of one embodiment of the present invention 1 A schematic diagram of an intermediate feature map.

[0085] Figure 5 G in the method of one embodiment of the present invention 2 A schematic diagram of an intermediate feature map.

[0086] Figure 6 It is a schematic structural block diagram of a system according to an embodiment of the present invention.

[0087] Among them: 100, first area, 200, first central area, 300, second area, 400, third area, 500, second central area, 600, fourth area, 700, graph matching system, 701, input unit, 702, affinity matrix construction unit, 703, affinity matrix update unit, 704, matching relation solving unit. DETAILED DESCRIPTION

[0088] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0089] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a computer terminal.

[0090] like Figure 1 As shown, a graph matching method of the present invention includes the following steps S1 to S4, specifically:

[0091] Step S1: Input two attribute graphs to be matched: Graph G 1 and Figure G 2 .

[0092] Among them, G 1 =(V 1 , E 1 , A 1 ), G 2 =(V 2 , E 2 , A 2 ), V x 、E x , A x Represent the attribute graph G respectively x The node set, edge set, and attribute set of , x=1,2.

[0093] In specific implementation, for example, in binocular vision matching, Figure G 1 and Figure G 2 The acquisition method is as follows: the same method is used to extract the feature points of the left view and the right view to be visually matched respectively, and the extracted feature points of the left and right views are used as vertices to construct the attribute graphs of the left and right views respectively. The attribute graphs of the left and right views constructed are the graphs G obtained in the binocular visual matching. 1 and Figure G 2 .

[0094] For example, remember G 1 is the attribute graph of the constructed left view, graph G 2 is the property graph of the constructed right view, then:

[0095] For G 1 =(V 1 , E 1 , A 1 ) 1 All nodes in E are all feature points extracted from the left view. 1 V 1 The set of edges formed by the connections between the midpoints, A 1 Characterization V 1 The relationship between the midpoints (i.e., geometric relationship);

[0096] For G 2 =(V 2 , E 2 , A 2 ) 2 All nodes in E represent the set of all feature points extracted from the right view. 2 V 2 The set of edges formed by the connections between the midpoints, A 2 Characterization V 2 All the relationships (i.e. geometric relationships) between the nodes.

[0097] Step S2: Construct graph G 1 Figure G 2 The affinity matrix W all .

[0098] Specifically, the implementation method of step S2 includes the following steps:

[0099] S21: Constructing graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ;

[0100] S22: Constructing graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ;

[0101] S23: Constructing graph G 1 and Figure G 2 The global feature affinity matrix W global ;

[0102] S24: Constructing graph G 1 and Figure G 2 The intermediate feature affinity matrix W mid ;

[0103] S25: construct the local distance affinity matrix W local-distance , local angle affinity matrix W local -angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is:

[0104]

[0105] Where k = 0.7.

[0106] In specific implementation, the value of k in step S25 may also be 0.6.

[0107] In step S2, W local-distance , W local-angle , W global and W mid The size of the four matrices is (|V 1 |×|V 2 |)×(|V 1 |×|V 2 |), where |V 1 |Representative Figure G 1 The number of nodes, |V 2 |Representative Figure G2 The number of nodes.

[0108] use Representative Figure G 1 The i-th node in Representative Figure G 1 The jth node in Representative Figure G 2 The a-th node in Representative Figure G 2 The bth node in and Composition diagram G 1 and Figure G 2 The two pairs of candidate correspondences between the above W local-distance , W local-angle , W global and W mid The ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) columns represent two pairs of candidate correspondences. and affinity, where i = 1, 2, ..., |V 1 |, j = 1, 2, ..., | V 1 |, a=1, 2, ..., |V 2 |, b = 1, 2, ..., | V 2 |.

[0109] The higher the affinity between two pairs of candidate correspondences, the more similar the two pairs of candidate correspondences are, and the higher the possibility that the candidate correspondences are correct matches.

[0110] Specifically:

[0111] (I) Constructing the local distance affinity matrix W in step S21 local-distance The method is:

[0112] The candidate correspondence pairs The affinity between them is in the local distance affinity matrix W local -distance In the expression:

[0113]

[0114] in, is the local distance affinity matrix W local-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, Representation graph G1 The i-th node in the graph G 1 The Euclidean distance of the jth node in, Representation graph G 2 Midpoint To Node Euclidean distance, i = 1, 2, ..., |V 1 |, j = 1, 2, ..., | V 1 |, a=1, 2, ..., |V 2 |, b = 1, 2, ..., | V 2 |;

[0115] Calculate graph G using formula ① 1 Figure G 2 The affinity of all candidate correspondences in completes the local distance affinity matrix W local-distance structure.

[0116] (ii) Constructing the local angle affinity matrix W in step S22 local-angle The method is:

[0117] The candidate correspondence pairs and The affinity between them is in the local angle affinity matrix W local -angle The expression in is:

[0118]

[0119] in, is the local angle affinity matrix W local-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position,

[0120] Representation Node To Node The vector of Representation Node To Node The vector of Representation vector With vector The product of Representation vector The model, Representation vector Model;

[0121] Calculate graph G using formula ② 1 Figure G 2The affinity of all candidate correspondences in completes the local distance affinity matrix W local-angle structure.

[0122] (III) Constructing the global feature affinity matrix W in step S23 global The method is:

[0123] Construct attribute graph G 1 , G 2 The global distance affinity matrix W between global-distance ;

[0124] Construct attribute graph G 1 , G 2 The global angular affinity matrix W global-angle ;

[0125] Fusion of the above global distance affinity matrix W global-distance and the global angular affinity matrix W global-angle , get the global feature affinity matrix W global , the fusion formula is:

[0126] W global =(W global-distance +W global-angle ) / 2.

[0127] The global distance affinity matrix W constructed above global-distance and the global angular affinity matrix W global -angle All are (|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix.

[0128] Among them, the global distance affinity matrix W global-distance The construction method is:

[0129] The candidate correspondence and The affinity between them is in the global distance affinity matrix W global -distance In the expression:

[0130]

[0131] In the formula, is the global distance affinity matrix W global-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, c 1 、c2 Figure G 1 Figure G 2 The central node of Representation Node To the central node c 1 The Euclidean distance of Representation Node To the central node c 1 The Euclidean distance of Representation Node To the central node c 2 The Euclidean distance of Representation Node To the central node c 2 The Euclidean distance of

[0132] Calculate graph G using formula ③ 1 Figure G 2 The affinity of all candidate correspondences in completes the global distance affinity matrix W global-distance structure.

[0133] Among them, Figure G 1 The central node c 1 , for graph G 1 The average value of all nodes in Graph G 2 The central node c 2 , for graph G 2 The average value of all nodes in .

[0134] The global angle affinity matrix W global-angle The construction method is:

[0135] The candidate correspondence and The affinity between them is in the global perspective affinity matrix W global -angle In the expression:

[0136]

[0137] In the formula, is the global angle affinity matrix W global-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, and For Figure G 1 Midpoint c 1 The three internal angles of the triangle formed by the three sides between them, ∠θ1, ∠θ2 and ∠θ3 are the nodes in graph G2 c2 The three interior angles of the triangle formed by the three sides between them, With Node c 1 Corresponding in sequence, as follows Figure 2 As shown, ∠θ1, ∠θ2 and ∠θ3 are related to the node c 2 Corresponding in sequence, as follows Figure 3 As shown;

[0138] Refer to the candidate correspondence and Computational graph G 1 Figure G 2 The affinity of all candidate correspondences in completes the global angle affinity matrix W global-angle structure.

[0139] (IV) The implementation method of step S24 is:

[0140] In Figure G 1 China-Israel Construct G for the center of the circle 1 Intermediate feature maps, such as Figure 4 As shown, the specific construction method is:

[0141] In Figure G 1 In As the center of the circle, Draw a circle with a radius of , and the two circles intersect to obtain three areas distributed in sequence: the first area 100, the first central area 200, and the second area 300; obtain the center points of the first area 100, the first central area 200, and the second area 300, and record them as Connect nodes sequentially and

[0142] In Figure G 2 China-Israel Construct G for the center of the circle 2 Intermediate feature maps, such as Figure 5 As shown, the specific construction method is: in Figure G 2 In As the center of the circle, A circle with a radius of , the two circles intersect to obtain three areas distributed in sequence: the third area 400, the second central area 500, and the fourth area 600; the center points of the third area 400, the second central area 500, and the fourth area 600 are obtained and recorded as Connect nodes sequentially and

[0143] Computational graph G 1 and Figure G2 The intermediate distance affinity matrix W mid-distance ;

[0144] Computational graph G 1 and Figure G 2 The intermediate angle affinity matrix W mid-angle ;

[0145] The intermediate distance affinity matrix W obtained by fusion calculation mid-distance and the intermediate angle affinity matrix W mid-angle , and obtain the intermediate feature affinity matrix W mid , the fusion formula is: W mid =(W mid-distance +W mid-angle ) / 2.

[0146] Among them, the constructed intermediate distance affinity matrix W mid-distance and the intermediate angle affinity matrix W mid-angle The size is (|V 1 |×|V 2 |)×(|V 1 |×|V 2 |).

[0147] Specifically, the intermediate distance affinity matrix W mid-distance The construction method is:

[0148] The candidate correspondence pairs and The affinity between the intermediate distances is the affinity matrix W mid -distance In the expression:

[0149]

[0150] In the formula, is the intermediate distance affinity matrix W mid-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column elements, distance1 and distance2 are both row vectors,

[0151] and In turn, nodes To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of Node To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of ||distance1-distance2||1 represents the norm of vector distance1 and distance2;

[0152] Calculate graph G using formula ⑤ 1 Figure G 2 The affinity of all candidate correspondences in the intermediate distance affinity matrix W is completed. mid-distance structure.

[0153] Specifically, the intermediate angle affinity matrix W mid-angle The construction method is:

[0154] The candidate correspondence pairs and The affinity between them is in the middle angle affinity matrix W mid -angle In the expression:

[0155]

[0156] In the formula, is the intermediate angle affinity matrix W mid-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) are column elements, angle1 and angle2 are row vectors, angle1 = (∠α1, ∠α2, ∠α3, ∠α4), angle2 = (∠β1, ∠β2, ∠β3, ∠β4), ||angle1-angle2||1 represents the norm of vectors angle1 and angle2, ∠α1, ∠α2, ∠α3 and ∠α4 are nodes Connect nodes in sequence The corresponding internal angles in the formed polygon (i.e. ∠α1, ∠α2, ∠α3 and ∠α4) are connected to the nodes in sequence. corresponding), such as Figure 4 As shown; ∠β1, ∠β2, ∠β3, ∠β4 are nodes Connect nodes in sequence The corresponding internal angles in the formed polygon (i.e. ∠β1, ∠β2, ∠β3, ∠β4) are connected to the nodes in sequence. corresponding), such as Figure 5 As shown;

[0157] Calculate the calculation graph G using formula ④ 1 Figure G 2 The affinity of all other candidate correspondences in the intermediate angle affinity matrix W is completed. mid-angle structure.

[0158] Step S3: Update the affinity matrix W based on neighbor node pairs all , get the updated affinity matrix W U .

[0159] In order to further improve the matching accuracy, the present application proposes the concept of neighbor nodes. In step S3, the original affinity is updated based on the affinity of neighbor node pairs that meet specific conditions.

[0160] In this application, the neighbor node pair refers to the candidate corresponding relationship. Specifically, the neighbor node pair refers to a node pair whose affinity with the candidate corresponding relationship is greater than 0.

[0161] by and Take these two candidate correspondences as examples to explain the concept of neighbor node pairs:

[0162] Candidate correspondence The neighbor node pair refers to the corresponding relationship with the candidate The affinity between the nodes is greater than 0, that is, there is a pair m and n are both positive integers. Correspondence with candidates The affinity between them is greater than 0, then the matching pair Candidate correspondence A neighbor node pair of ;

[0163] Candidate correspondence The neighbor node pair refers to the corresponding relationship with the candidate The affinity between the nodes is greater than 0, that is, there is a pair p and d are both positive integers. Correspondence with candidates The affinity between them is greater than 0, then the matching pair Candidate correspondence A neighbor node pair.

[0164] Furthermore, the present invention uses the affinity of neighbor node pairs to update matching pairs and Specifically, the present invention uses the top four neighbor node pairs in terms of affinity among all neighbor node pairs to update the original affinity.

[0165] Specifically, the implementation steps of step S3 are:

[0166] For Figure G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U :

[0167] S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows:

[0168]

[0169] in: Represents the affinity matrix W all ((a-1)|V 1 |+i) rows, ((b-1)|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W ：；jb Represented by the matrix W all The ((b-1)|V 2 |+j) columns, ∑rank(W ia；： , 4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb , 4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

[0170] In this embodiment, t=0.7.

[0171] Step S4: Based on the updated affinity matrix W U , solve graph G 1 Figure G 2 matching relationship.

[0172] In this embodiment, a reweighted random walk algorithm is used based on the updated affinity matrix W U , solve graph G 1 Figure G 2 matching relationship.

[0173] In specific implementation, those skilled in the art may also adopt other related existing technologies based on the affinity matrix W U , solve graph G 1 Figure G 2 matching relationship.

[0174] Preferably, before step S25, the method further includes the following steps:

[0175] The constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The matrices are normalized respectively, and the normalized expressions are:

[0176] In the formula, max(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The maximum value, min(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The minimum value in m Represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid .

[0177] like Figure 6 As shown, the present invention further provides a graph matching system 700, comprising:

[0178] The input unit 701 is configured to input two attribute graphs to be matched, wherein the two attribute graphs are graphs G 1 and Figure G 2 , G 1 =(V 1 , E 1 , A 1 ), G 2 =(V 2 , E 2 , A 2 ), where V x 、E x , A x Represent the attribute graph G respectively x The node set, edge set, and attribute set of x = 1, 2;

[0179] Affinity matrix construction unit 702 is configured to construct the graph G 1 Figure G 2 The affinity matrix W all ;

[0180] The affinity matrix updating unit 703 is configured to update the affinity matrix W based on the neighbor node pairs. all , get the updated affinity matrix W U ;

[0181] The matching solving unit 704 is configured to U , solve graph G 1 Figure G 2 matching relationship.

[0182] The affinity matrix construction unit 702 includes:

[0183] The first module is configured to construct a graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ;

[0184] The second module is configured to construct the graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ;

[0185] The third module is configured to construct the graph G 1 and Figure G 2 The global feature affinity matrix W global ;

[0186] The fourth module is configured to construct the graph G 1 and Figure G 2 The intermediate feature affinity matrix Wmid ;

[0187] The fifth module is configured to construct the local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is:

[0188]

[0189] Where k = 0.6 or 0.7.

[0190] The affinity matrix updating unit 703 is configured to perform the following steps:

[0191] For the graph G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U :

[0192] S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows:

[0193]

[0194] in: Represents the affinity matrix W all ((a-1)|V 1 |+i) rows, ((b-1)|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W ：；jb Represented by the matrix W allThe ((b-1)|V 2 |+j) columns, ∑rank(W ia；： , 4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb , 4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

[0195] In this embodiment, the value of t is 0.7.

[0196] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A graph matching method, characterized in that: The method is applied to binocular vision matching, and specifically includes the following steps: S1: Input two attribute graphs to be matched: Graph G 1 and Figure G 2 , G 1 =(V 1 ,E 1 ,A 1 ), G 2 =(V 2 ,E 2 ,A 2 ), where G 1 is the attribute graph of the left view, Figure G 2 is the property graph of the right view, V x 、E x , A x Represent the attribute graph G x The node set, edge set, attribute set, V x All nodes in E are all feature points of the extracted view. x V x The set of edges formed by the connections between the midpoints, A x Characterization E x The relationship between the nodes, x = 1, 2; S2: Constructing graph G 1 Figure G 2 The affinity matrix W all ; S3: Update the affinity matrix W based on neighbor node pairs all , get the updated affinity matrix w U ; S4: Based on the updated affinity matrix W U , solve graph G 1 Figure G 2 The matching relationship; The implementation method of step S2 comprises the following steps: S21: Constructing graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ; S22: Constructing Graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ; S23: Constructing graph G 1 and Figure G 2 The global feature affinity matrix W global ; S24: Constructing graph G 1 and Figure G 2 The intermediate feature affinity matrix W mid ; S25: construct the local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is: Where k = 0.6 or 0.7; The implementation method of step S24 is: In Figure G 1 In As the center of the circle, Draw a circle with radius , and the intersection of the two circles gives three areas distributed in sequence: the first area, the first central area, and the second area. Get the center points of the first area, the first central area, and the second area, and record them as Connect nodes sequentially and In Figure G 2 In As the center of the circle, Draw a circle with radius , and the two circles intersect to obtain three areas distributed in sequence: the third area, the second central area, and the fourth right area; obtain the center points of the obtained third area, the second central area, and the fourth right area, and record them as Connect nodes sequentially and Computational graph G 1 and Figure G 2 The intermediate distance affinity matrix W mid-distance ; Computational graph G 1 and Figure G 2 The intermediate angle affinity matrix W mid-angle ; The intermediate distance affinity matrix W obtained by fusion calculation mid-distance and the intermediate angle affinity matrix W mid-angle , get the intermediate feature affinity matrix W mid , the fusion formula is: W mid =(W mid-distance +W mid-angle ) / 2.

2. The graph matching method according to claim 1, characterized in that: The local distance affinity matrix W constructed in step S21 local-distance =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the local distance affinity matrix W local -distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in: is the local distance affinity matrix w local-distance ((a-1)×|v 1 |+i) rows, ((b-1)×|v 2 |+j) column position, Representative Figure G 1 The i-th node in Representative Figure G 1 The jth node in Representative Figure G 2 The a-th node in Representative Figure G 2 The bth node in and Composition diagram G 1 and Figure G 2 A pair of candidate correspondences between Representation graph G 1 The i-th node in the graph G 1 The Euclidean distance of the jth node in, Representation graph G 2 Midpoint To Node Euclidean distance; i = 1, 2, ..., | V 1 |; j = 1, 2, ..., | V 1 |; a=1,2,…,|V 2 |; b = 1, 2, ..., | V 2 |;|V x |Representation graph G x The number of nodes, x = 1, 2; The local angle affinity matrix W constructed in step S22 local-angle =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the local angle affinity matrix W local-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in, is the local angle affinity matrix W local-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, Representation Node To Node The vector of Representation Node To Node The vector of Representation vector With vector The product of Representation vector The model, Representation vector Model.

3. The graph matching method according to claim 2, characterized in that: The implementation method of step S23 is: Construct attribute graph G 1 , G 2 The global distance affinity matrix W between global-distance ; Construct attribute graph G 1 , G 2 The global angular affinity matrix W global-angle ; Fusion of the above global distance affinity matrix W global-distance and the global angular affinity matrix W global-angle , get the global feature affinity matrix W global , the fusion formula is: IN global =(In global-distance +W global-angle ) / 2; The constructed global distance affinity matrix W global-distance =(|V 1 |×|v 2 |)×(|V 1 |×|V 2 |) matrix, global distance affinity matrix w global-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in, is the global distance affinity matrix W global-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, c 1 、c 2 Figure G 1 Figure G 2 The central node of and In turn, nodes To the central node c 1 The Euclidean distance and nodes To the central node c 1 The Euclidean distance of and In turn, nodes To the central node c 2 The Euclidean distance and nodes To the central node c 2 The Euclidean distance of The constructed global angle affinity matrix W global-angle =(|V 1 |×|V 2 |)×(|V 1 |×|v 2 |) matrix, the global angle affinity matrix W global-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in, is the global angle affinity matrix W global-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column position, and For Figure G 1 Middle node c 1 , The three internal angles of the triangle formed by the three sides between them, ∠θ1, ∠θ2 and ∠θ3 are the angles of the node c in graph G2. 2 , The three interior angles of the triangle formed by the three sides between them, With Node c 1 Correspondingly, ∠θ1, ∠θ2 and ∠θ3 are related to the nodes c 2 Correspond in sequence.

4. The graph matching method according to claim 1, characterized in that: The constructed intermediate distance affinity matrix W mid -distance =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the intermediate distance affinity matrix W mid-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in: is the intermediate distance affinity matrix W mid-distance In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) are elements in the column position, distance1 and distance2 are row vectors, and In turn, nodes To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of Node To Node Euclidean distance, node To Node Euclidean distance, node To Node The Euclidean distance and nodes To Node The Euclidean distance of ||distance1-distance2||1 represents the norm of vector distance1 and distance2; The constructed intermediate angle affinity matrix W mid-angle =(|V 1 |×|V 2 |)×(|V 1 |×|V 2 |) matrix, the intermediate angle affinity matrix W mid-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 The calculation formula for the element at the |+j) column position is: in: is the intermediate angle affinity matrix W mid-angle In ((a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) is the element in the column position, angle1 and angle2 are row vectors, angle1 = (∠α1, ∠α2, ∠α3, ∠α4), angle2 = (∠β1, ∠β2, ∠β3, ∠β4), ||angle1-angle2||1 represents the norm of vector angle1 and angle2, ∠α1, ∠α2, ∠α3 and ∠α4 are nodes Connect nodes in sequence The corresponding internal angles in the formed polygon; ∠β1, ∠β2, ∠β3 and ∠β4 are nodes Connect nodes in sequence The corresponding interior angles of the formed polygon.

5. The graph matching method according to claim 4, characterized in that: Before step S25, the method further includes the following steps: The constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The matrices are normalized respectively, and the normalized expressions are: In the formula, max(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The maximum value, min(W m ) represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid The minimum value in m Represents the constructed local distance affinity matrix W local-distance , local angle affinity matrix W local--angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid .

6. The graph matching method according to any one of claims 1 to 5, characterized in that: The implementation steps of step S3 are: For Figure G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U : S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows: in: Represents the affinity matrix W all 9(a-1)×|V 1 |+i) rows, ((b-1)×|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W ：；jb Represented by the matrix W all The ((b-1)|V 2 |+j) columns, ∑rank(W ia；： ,4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb ,4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

7. A graph matching system, characterized in that: The system is used for binocular vision matching, including: An input unit is configured to input two attribute graphs to be matched, wherein the two attribute graphs are graphs G 1 and Figure G 2 , G 1 =(V 1 ,E 1 ,A 1 ), G 2 =(V 2 ,E 2 ,A 2 ), where G 1 is the attribute graph of the left view, Figure G 2 is the property graph of the right view, V x 、E x , A x Represent the attribute graph G x The node set, edge set, attribute set, V x All nodes in E are all feature points of the extracted view. x V x The set of edges formed by the connections between the midpoints, A x Characterization E x The relationship between the nodes, x = 1, 2; An affinity matrix construction unit is configured to construct the graph G 1 Figure G 2 The affinity matrix W all ; The affinity matrix updating unit is configured to update the affinity matrix W based on the neighbor node pairs. all , get the updated affinity matrix W U ; A matching solving unit is configured to match the updated affinity matrix W U , solve graph G 1 Figure G 2 The matching relationship; The affinity matrix construction unit comprises: The first module is configured to construct a graph G 1 and Figure G 2 The local distance affinity matrix W local-distance ; The second module is configured to construct the graph G 1 and Figure G 2 The local angle affinity matrix W local-angle ; The third module is configured to construct the graph G 1 and Figure G 2 The global feature affinity matrix W global ; The fourth module is configured to construct the graph G 1 and Figure G 2 The intermediate feature affinity matrix W mid ; The fifth module is configured to construct the local distance affinity matrix W local-distance , local angle affinity matrix W local-angle , global feature affinity matrix W global and the intermediate feature affinity matrix W mid Fusion is performed to obtain the affinity matrix W all , the fusion formula is: Where k = 0.6 or 0.7; The constructed graph G 1 and Figure G 2 The intermediate feature affinity matrix W mid The implementation method is: In Figure G 1 In As the center of the circle, Draw a circle with radius , and the intersection of the two circles gives three areas distributed in sequence: the first area, the first central area, and the second area. Get the center points of the first area, the first central area, and the second area, and record them as Connect nodes sequentially and In Figure G 2 In As the center of the circle, Draw a circle with radius , and the two circles intersect to obtain three areas distributed in sequence: the third area, the second central area, and the fourth right area; obtain the center points of the obtained third area, the second central area, and the fourth right area, and record them as Connect nodes sequentially and Computational graph G 1 and Figure G 2 The intermediate distance affinity matrix W nud-distance ; Computational graph G 1 and Figure G 2 The intermediate angle affinity matrix W mid-angle ; The intermediate distance affinity matrix W obtained by fusion calculation mid-distance and the intermediate angle affinity matrix W mid-angle , get the intermediate feature affinity matrix W mid , the fusion formula is: IN mid =9W mid-distance +W mid-angle ) / 2。 8. The graph matching system according to claim 7, characterized in that: The affinity matrix updating unit is configured to perform the following steps: For the graph G 1 and Figure G 2 Each candidate correspondence between and Execute the following steps S31 respectively, and finally obtain the updated affinity matrix W U : S31, based on the candidate corresponding and The affinity of the neighbor node pair is updated accordingly. and In W all The affinity in is updated as follows: in: Represents the affinity matrix W all ((a-1)|V 1 |+i) rows, ((b-1)|V 2 |+j) column, which is the candidate corresponding relationship and The affinity between them before the update; for The corresponding updated affinity represents the candidate correspondence and The updated affinity between ia；： Represented by the matrix W all The ((a-1)|V 1 |+i) rows, W ：；jb Represented by the matrix W all The ((b-1)|V 2 |+j) columns, ∑rank(W ia；： ,4) represents the vector W ia；： The values ​​of all the elements in the top four median rankings are added together, ∑rank(W ：；jb ,4) represents the vector W ：；jb The values ​​of all the elements before the median are added together, and t represents the weight.

Images