Semantic point set matching method based on triangle construction adjoint graph

Abstract

The invention discloses a semantic point set matching method for constructing an adjoint graph based on triangles, and belongs to the technical field of geographic information and unmanned aerial vehicle visual positioning processing. The method comprises the following steps: constructing a candidate precise point set; obtaining a to-be-matched image, identifying a target, and obtaining a to-be-matched semantic point set; screening candidate precise point subsets in the candidate precise point set, and constructing an irregular triangular network; constructing an irregular triangulation network based on all points in the to-be-matched semantic point set; constructing a triangular adjoint graph; calculating the similarity probability of any two node edges in the triangular adjoint graph; solving the weight of any node; obtaining a matched triangle pair; searching a matching point set of any point in the semantic point set to be matched according to a matching rule of triangles in the matched triangle pair; and according to a maximum probability principle, searching a matching point of any point in the semantic point set to be matched. According to the method, the matching graph can be constructed by utilizing the spatial relationship of the point locations in the database, and an optimized adjoint graph solving method is introduced, so that quick matching of the points in the image is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of geographic information and unmanned aerial vehicle visual positioning processing, and in particular relates to a semantic point set matching method based on triangle-based accompanying graph construction. Background Art

[0002] With the development of Internet of Things technology, more terminals are connected, and it is more convenient to obtain information about the surrounding environment. As a very important mobile node, drones are widely used as information sharing platforms between other nodes. However, in urban environments with densely populated high-rise buildings, it is often impossible to obtain the position of drones in real time during actual applications due to reasons such as obstruction of satellite positioning signals, signal interference, and a small number of visible stars, which brings great difficulties to the dynamic sharing and communication of information in the Internet of Things. Therefore, the real-time positioning technology of drones based on visual cameras to obtain images has become a hot topic in current research, among which the image matching algorithm with strong robustness has become a key difficulty in positioning technology.

[0003] Most of the current real-time image matching algorithms are based on low-level image features. These algorithms mainly extract the grayscale, texture, and structural features of the target in the image, but they are sensitive to different lighting conditions, different seasonal climates, and slight changes in the interior of the object, which often leads to matching failures. As an important feature of the object, semantic attributes have strong robustness characteristics and are very adaptable to changes in the object, as well as seasonal and lighting conditions. With the development of edge computing, the computing power of the terminal has been greatly improved, and it has the ability to extract semantic information of complex environments in real time, providing a technical basis for the widespread application of object semantics. Summary of the invention

[0004] The purpose of the present invention is to provide a semantic point set matching method based on constructing an accompanying graph based on triangles, which can construct a matching graph using the spatial relationship of points in a database, introduce an optimized method for solving the accompanying graph, and achieve fast matching of points in an image.

[0005] The technical solution adopted by the present invention is:

[0006] A semantic point set matching method based on triangle-based companion graph construction includes the following steps:

[0007] Step 1: Construct a candidate precise point set with category attributes, geographic location information and multiple attributes

[0008] Step 2: Get the image to be matched, identify the target from the image to be matched, and obtain the semantic point set to be matched.

[0009] Step 3: In the candidate precise point set Filter out the semantic point set to be matched Candidate precise point subsets with consistent semantic information of point sets And according to The location information of all points is used to construct an irregular triangle network TIN r1 ;

[0010] Step 4: Based on the semantic point set to be matched The location information of all points is used to construct an irregular triangle network TIN d ;

[0011] Step 5: Combine the two irregular triangle networks TIN d and TIN r1 The triangles in are used as elements to construct the triangular adjoint graph AG;

[0012] Step 6: Calculate any edge in the triangle adjoint graph AG The similarity probability

[0013] Step 7: Use the random walk algorithm to calculate any point in the triangle adjoint graph AG Weight

[0014] Step 8: Obtain the set of matching triangle pairs DT according to the weights of each point in the triangle companion graph AG and the matching similarity threshold e. r1～d ;

[0015] Step 9: Construct a set of matching triangle pairs DT r1～d The matching rules of the triangle in the figure are used to find the semantic point set to be matched. Any point in Matching rule set

[0016] Step 10: Obtain the semantic point set to be matched based on the maximum probability principle Any point in matching points Update Point attributes and location information.

[0017] Furthermore, the candidate precise point set in step 1 for:

[0018]

[0019]

[0020] in, Represent the candidate precise point sets The i-th point in The longitude and latitude coordinates of Indicate point Category information, Indicate point The property of n r Represents the candidate precise point set The total number of points in .

[0021] Furthermore, in step 2, the YOLOv5 model is used to identify the target from the image to be matched, and the semantic point set to be matched is obtained. for:

[0022]

[0023] in, Represent the candidate precise point sets The kth point in The image pixel coordinates, Indicate point Category information, n d Represents the candidate precise point set The total number of points in .

[0024] Furthermore, the candidate precise point subset in step 3 for:

[0025]

[0026] in, is the candidate precise point subset The jth point in for The corresponding semantic category information, Represents the semantic point set to be matched The category information of the kth point in n r1 Represents a subset of candidate precise points The total number of points in ;

[0027] Irregular Triangular Network TIN r1 for:

[0028]

[0029] in, Indicates irregular triangle network TIN r1 The vth triangle of Represents irregular triangle network TIN r1 The total number of triangles in Represents a triangle The three vertices of .

[0030] Furthermore, the triangle adjoint graph AG in step 5 is:

[0031] AG=(N dr1 ,E dr1 )

[0032] N dr1 =TIN d ×TIN r1

[0033]

[0034] Among them, N dr1 represents a node set, N dr1 For TIN d and TIN r1 The Cartesian product of dr1 The nodes in the TIN are d and TIN r1 The two triangles of m N N dr1 The number of nodes contained in E dr1 represents the set of edges, N dr1 The ath node in and the bth node The edge between.

[0035] Furthermore, the edge in the triangle adjoint graph AG in step 6 The similarity probability for:

[0036]

[0037] Among them, dis(a e ,b v ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle similarity distance; dis(a v ,b e ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle The similarity distance, Is a node and nodes The edge between e ,b v ) and dis(a v ,b e ) is calculated as follows:

[0038]

[0039]

[0040] Among them, β * and a * They represent the maximum and minimum interior angles of the triangle, respectively. * ,q * and u * They represent the side lengths of the triangles and are arranged from small to large.

[0041] Furthermore, in step 8, the set of matching triangle pairs DT r1～d for:

[0042]

[0043] in, is the jth node in the triangular adjoint graph AG The weights of je and jv represent the irregular triangulated network TIN. d and TIN r1 where j is the triangle number in , and j is the node number in the triangle companion graph AG.

[0044] Furthermore, the specific method of step 9 is:

[0045] According to the set of matching triangle pairs DT r1～d The matching relationship of the triangles in There is a unique triangle Matching it, where Represents triangles The three internal angles in the figure are reduced in sequence, Represents triangles The three internal angles in the figure are reduced in sequence to the corresponding vertices;

[0046] Construct the following matching rules:

[0047]

[0048] In the formula, Indicates that point A and point B establish a corresponding matching relationship;

[0049] Get the semantic point set to be matched Any point in Matching rule set

[0050]

[0051] The elements in can appear repeatedly, and v0,v1,...,vn represent the numbers.

[0052] Furthermore, in step 10, the semantic point set to be matched Any point in matching points for:

[0053]

[0054] in, Indicate point In the matching rule set The frequency of occurrence in, Max() means finding the maximum value, v0, v1, ..., vn means the number;

[0055] Update Point The properties and location information are:

[0056]

[0057] The present invention has the following beneficial effects:

[0058] (1) The present invention proposes a semantic point set matching method based on triangle-based companion graph construction. The triangle-oriented companion graph construction method realizes fast matching of different points and can effectively locate the midpoints in the image and assign attributes.

[0059] (2) The method of the present invention can effectively match the three-dimensional coordinates of points in drone images, providing effective technical support for multi-source data fusion and drone positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the principle of the semantic point set matching method based on triangle-based companion graph construction. DETAILED DESCRIPTION

[0061] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0062] A semantic point set matching method based on triangle construction adjoint graph, such as Figure 1 As shown, the following steps are included:

[0063] Step 1: Construct a candidate precise point set with category attributes, geographic location information and multiple attributes The calculation is as follows:

[0064]

[0065]

[0066] in, Represent the candidate precise point sets The i-th point The longitude and latitude coordinates of Indicate point Category information, Indicate point The property of n r Represents the candidate precise point set The total number of midpoints.

[0067] Step 2: Get the image to be matched and use the YOLOv5 model to identify the target to obtain the semantic point set to be matched The calculation is as follows:

[0068]

[0069]

[0070] in, Represent the candidate precise point sets The kth point The image pixel coordinates, Indicate point Category information, n d Represents the candidate precise point set The total number of midpoints.

[0071] Step 3: In the candidate precise point set Filter out the semantic point set to be matched Candidate precise point subsets with consistent semantic information of point sets It can be expressed as:

[0072]

[0073] in, is the candidate precise point subset The jth point of is the corresponding semantic category information, Represents the semantic point set to be matched The category information of the kth point in .

[0074] and will The location information of all points builds an irregular triangle network TIN r1 , which is expressed as follows:

[0075]

[0076] in, Indicates irregular triangle network TIN r1 The vth triangle of Represents irregular triangle network TIN r1 The number of Represents a triangle The three vertices of .

[0077] Step 4: Similarly, based on the semantic point set to be matched All points can be used to construct irregular triangulated network TIN d , is calculated as follows:

[0078]

[0079] in, Indicates irregular triangle network TIN d The e-th triangle of Represents irregular triangle network TIN d The number of Represents a triangle The three vertices of .

[0080] Step 5: Combine the two irregular triangulated networks (TIN) d and TIN r1 The triangles in are used as elements to construct the triangular adjoint graph AG, which can be expressed as:

[0081] AG=(N dr1 ,E dr1 )

[0082] N dr1 =TIN d ×TIN r1

[0083]

[0084] Among them, N dr1 represents a node set, N dr1 For TIN d and TIN r1 The Cartesian product of dr1 The nodes in the TIN are d and TIN r1 The two triangles of m N N dr1 The number of nodes contained in E dr1 represents the set of edges, N dr1 The ath node in and the bth node The edge between.

[0085] Step 6: Next, construct the edges between any two nodes in the triangle adjoint graph AG. The similarity probability It can be expressed as:

[0086]

[0087] Among them, dis(a e ,b v ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle similarity distance; dis(a v ,b e ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle The similarity distance, Is a node and nodes The edge between e ,b v ) and dis(a v ,b e ) is calculated as follows:

[0088]

[0089]

[0090] Among them, β * and a * They represent the maximum and minimum interior angles of the triangle, respectively. * ,q * and u * They represent the side lengths of the triangles and are arranged from small to large.

[0091] Step 7: Find any node using the random walk algorithm based on the triangle companion graph AG Weight

[0092] Step 8: Based on the weight of each node And the matching similarity threshold e is used to obtain the matching triangle pair set DT r1～d , which can be expressed as:

[0093]

[0094] Among them, je and jv represent irregular triangulated network TIN respectively. d and TIN r1 where j is the triangle number in , and j is the node number of the triangle adjoint graph AG.

[0095] Step 9: According to the matching triangle pair set DT r1～d From the triangle matching relationship, we can see that for any Each consists of a unique triangle Matching this, assuming and The three vertices of can be represented as:

[0096]

[0097]

[0098] in, Represents triangles The three internal angles in the figure are reduced in sequence, Represents triangles The three internal angles decrease in sequence along the corresponding vertices.

[0099] Therefore, construct the following matching rules:

[0100]

[0101] In the formula, Indicates that point A and point B establish a corresponding matching relationship.

[0102] Then the semantic point set to be matched can be obtained Any point in Matching rule set

[0103]

[0104] in, The elements in can appear repeatedly, and v0,v1,...,vn represent the numbers.

[0105] Step 10: According to the maximum probability principle, the semantic point set to be matched Any point in matching points It can be expressed as:

[0106]

[0107] in, Indicate point In the matching rule set The frequency of occurrence in , Max() means finding the maximum value, v0, v1, ..., vn means the number.

[0108] Finally, update point The properties and location information are:

[0109]

[0110] Updated set of points to be matched for:

[0111]

[0112] The present invention utilizes the semantic information of ground objects obtained from drone images to construct a matching strategy between semantic points, thereby achieving high-precision matching between drone images and database environment images.

[0113] The present invention can construct a matching graph using the spatial relationship of points in a database, introduce an optimized method for solving the adjoint graph, and achieve rapid matching of points in an image.

[0114] In summary, the semantic point set matching method based on triangle-based companion graph construction proposed in the present invention can realize fast matching of different points based on the triangle-based companion graph construction method, can effectively locate points in images and assign attributes, and perform three-dimensional coordinate matching of points in images of drones, etc., providing effective technical support for multi-source data fusion and drone positioning.

Claims

1. A semantic point set matching method based on triangle-based companion graph construction, characterized in that: The following steps are involved: Step 1: Construct a candidate precise point set with category attributes, geographic location information and multiple attributes Step 2: Get the image to be matched, identify the target from the image to be matched, and obtain the semantic point set to be matched. Step 3: In the candidate precise point set Filter out the semantic point set to be matched Candidate precise point subsets with consistent semantic information of point sets And according to The location information of all points is used to construct an irregular triangle network TIN r1 ; Step 4: Based on the semantic point set to be matched The location information of all points is used to construct an irregular triangle network TIN d ; Step 5: Combine the two irregular triangle networks TIN d and TIN r1 The triangles in are used as elements to construct the triangular adjoint graph AG; Step 6: Calculate any edge in the triangle adjoint graph AG The similarity probability Step 7: Use the random walk algorithm to calculate any point in the triangle adjoint graph AG Weight Step 8: Obtain the set of matching triangle pairs DT according to the weights of each point in the triangle companion graph AG and the matching similarity threshold e. r1～d ; Step 9: Construct a set of matching triangle pairs DT r1～d The matching rules of the triangle in the figure are used to find the semantic point set to be matched. Any point in Matching rule set Step 10: Obtain the semantic point set to be matched based on the maximum probability principle Any point in matching points Update Point attributes and location information.

2. The semantic point set matching method based on triangle-based companion graph construction according to claim 1, characterized in that: The candidate precise point set in step 1 for: in, Represent the candidate precise point sets The i-th point in The longitude and latitude coordinates of Indicate point Category information, Indicate point The property of n r Represents the candidate precise point set The total number of points in .

3. The semantic point set matching method based on triangle-based companion graph construction according to claim 2, characterized in that: In step 2, the YOLOv5 model is used to identify the target from the image to be matched, and the semantic point set to be matched is obtained. for: in, Represent the candidate precise point sets The kth point in The image pixel coordinates, Indicate point Category information, n d Represents the candidate precise point set The total number of points in .

4. The semantic point set matching method based on triangle-based companion graph construction according to claim 3, characterized in that: The candidate precise point subset in step 3 for: in, is the candidate precise point subset The jth point in for The corresponding semantic category information, Represents the semantic point set to be matched The category information of the kth point in n r1 Represents a subset of candidate precise points The total number of points in ; Irregular Triangular Network TIN r1 for: in, Indicates irregular triangle network TIN r1 The vth triangle of Represents irregular triangle network TIN r1 The total number of triangles in Represents a triangle The three vertices of .

5. The semantic point set matching method based on triangle-based companion graph construction according to claim 4, characterized in that: The triangle adjoint graph AG in step 5 is: AG=(N dr1 ,E dr1 ) N dr1 NEWS d ×NEWS r1 Among them, N dr1 represents a node set, N dr1 For TIN d and TIN r1 The Cartesian product of dr1 The nodes in the TIN are d and TIN r1 The two triangles of m N N dr1 The number of nodes contained in E dr1 represents the set of edges, N dr1 The ath node in and the bth node The edge between.

6. The semantic point set matching method based on triangle-based companion graph construction according to claim 5, characterized in that: The edge of the triangle adjoint graph AG in step 6 The similarity probability for: Among them, dis(a e ,b v ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle similarity distance; dis(a v ,b e ) represents irregular triangulated network TIN d Medium triangle and irregular triangulated network TIN r1 Medium triangle The similarity distance, Is a node and nodes The edge between e ,b v ) and dis(a v ,b e ) is calculated as follows: Among them, β * and a * They represent the maximum and minimum interior angles of the triangle, respectively. * ,q * and u * They represent the side lengths of the triangles and are arranged from small to large.

7. The semantic point set matching method based on triangle-based companion graph construction according to claim 6, characterized in that: In step 8, match the triangle pair set DT r1～d for: in, is the jth node in the triangular adjoint graph AG The weights of je and jv represent the irregular triangulated network TIN. d and TIN r1 where j is the triangle number in , and j is the node number in the triangle companion graph AG.

8. The semantic point set matching method based on triangle-based companion graph construction according to claim 7, characterized in that: The specific method of step 9 is: According to the set of matching triangle pairs DT r1～d The matching relationship of the triangles in There is a unique triangle Matching it, where Represents triangles The three internal angles in the figure are reduced in sequence, Represents triangles The three internal angles in the figure decrease in sequence, and the corresponding vertices; Construct the following matching rules: In the formula, Indicates that point A and point B establish a corresponding matching relationship; Get the semantic point set to be matched Any point in Matching rule set The elements in can appear repeatedly, and v0,v1,...,vn represent the numbers.

9. The semantic point set matching method based on triangle-based companion graph construction according to claim 8, characterized in that: In step 10, the semantic point set to be matched Any point in matching points for: in, Indicate point In the matching rule set The frequency of occurrence in, Max() means finding the maximum value, v0, v1, ..., vn means the number; Update Point The properties and location information are:

Images