An Image Matching Method Based on the Affine Invariance of Points and Lines

Abstract

The present invention provides an image matching method based on point-line affine invariance. Based on point-line affine invariance, by using the combination of point features and line feature descriptors in an image, and relying on the stability of the geometric relationship of the same point-line elements between image transformations, it can achieve stable and robust line segment matching under scenarios such as rotation, and ensure a relatively fast matching speed.

Description

Technical Field

[0001] The present invention relates to the technical fields of image matching, computer vision, and digital image processing, and particularly relates to an image matching method based on the affine invariance of points and lines. Background Art

[0002] Feature matching of images is a very important basic technology in computer vision and has currently been widely applied in three-dimensional scene reconstruction, object recognition, and simultaneous localization and mapping.

[0003] Feature matching of images refers to the process of using two captured pictures for matching and recognition. These two pictures are referred to as the reference picture and the retrieval picture. Due to the different shooting positions of the reference picture and the retrieval picture, there are certain changes in the image content. However, there are still some features that appear in both pictures and can be used for feature matching.

[0004] Features are some special geometric elements in an image. According to the geometric attributes of the features used, features can be divided into point features, line features, and surface features. Through feature matching between images, the common geometric elements in the reference picture and the retrieval picture can be found. Combining triangulation, epipolar geometry, and other contents, basic environmental information can be provided for various directions in the field of computer vision.

[0005] Currently, most feature matching methods are based on feature point matching methods. This method generally describes the point features in an image by first constructing local feature descriptors, and then compares the approximation degree of the descriptors between pictures to determine the matching point pairs. The advantage of using point features is high precision. When the scene contains stable feature points, there are already mature algorithms to achieve stable feature point extraction. Relatively speaking, when using point features, all information except feature points will be ignored. For an image with hundreds of thousands of pixels, the number of extracted feature points may only be a few hundred, that is, point features may discard some useful map information. Therefore, such matching methods will show poor matching effects in scenes with less texture and in cases of large-scale image changes.

[0006] Compared with point features, the picture information contained in line features is significantly more than that of point features, and it is more stable and robust for scenes such as rotation. The disadvantages of line features are the incomplete detectability of lines and the sensitivity to partial occlusion. Due to problems such as inaccurate endpoint position estimation and blurred image edges, the current research on using line features for matching is not as in-depth as that of point features. Summary of the Invention

[0007] The present invention provides an image matching method based on point-line affine invariance, aiming to utilize the combination of point features and line feature descriptors to solve the accuracy problem in the process of line feature matching while ensuring a relatively fast matching speed. To achieve the purpose of the present invention, the present invention provides an image line segment matching method combining point features and line feature descriptors, and the method adopted is as follows:

[0008] (1) Extract and match point features and line features at multiple scales

[0009] Build five-layer Gaussian scale pyramids for the template image and the real-time image respectively, extract SIFT feature points and use the EDlines algorithm to extract line segments, which can reduce the influence of extracted broken line segments and enrich the features under scale changes. For the line features on different layers, they are matched by position, angle and length. The line segments matched on different layers will be regarded as the same line segment in space, and the same line segment at different scales will be projected as the same line segment on the first layer.

[0010] Build descriptors for the extracted feature points, and use the brute-force matching method to obtain candidate matching point pairs. Then use the KNN ratio screening and RANSAC algorithm to eliminate the mismatched points to obtain the finally matched feature point pairs.

[0011] (2) Build a rotation histogram and calculate the global rotation angle

[0012] The line segments in the image often maintain a relatively consistent rotation angle with the image. By constructing a rotation histogram, the global approximate rotation angle can be obtained, and this angle can be used to further screen the candidate matching line segment pairs. When constructing the rotation histogram, the direction of the line segment is divided into 18 rotation intervals within the range of 360°, that is, one interval every 20°. Normalize the direction angles of the template image and the real-time image into the rotation intervals to obtain the rotation histogram.

[0013] The global rotation angle is the rotation angle that can minimize the difference between the rotation histograms of the template image and the real-time image under this rotation, that is:

[0014]

[0015] where h l (x) and h r (x) represent the intervals of the rotation histograms of the template image and the real-time image respectively. To obtain a more accurate global rotation angle, in addition to constraining the direction of the line segment, the length of the line segment can also be used for screening. Record the cumulative lengths of all line segments in the rotation intervals of the template image and the real-time image. Only when both the change in angle and the change in length are less than a certain threshold can it be considered that the correct global rotation angle has been obtained.

[0016] (3) Determine the line segment support domain

[0017] Affine invariance means that certain relationships between points and lines in space remain unchanged. Specifically, for a straight line and two points that are coplanar with it and not on this line, the ratio of the distances from the two points to the line remains unchanged under affine transformation. The area on the left and right sides of the straight line is divided into two regions. The region pointed by the average line gradient direction is denoted as the right region, and the positive direction of the line is orthogonal to the average gradient direction and rotates counterclockwise. Thus, the left and right regions and directions of the line segments extracted in this paper are defined accordingly.

[0018] Since the line segments detected in the image rarely correspond to isolated 3D lines in the real world, they usually correspond to the edges of surfaces. Therefore, feature points within a specific range at least on the left or right side region of the line can be considered coplanar.

[0019] To determine the feature points coplanar with the line segment, it is necessary to construct its support domain, which contains all the matching feature points that may be coplanar with it. As Figure 2 shown, the support domain is defined by a rectangular region with a length of 2αl and a width of 2βl, where l is the length of the line segment, and α and β are the scale factors of the support domain. Based on this length and width, the support domain can be determined around the line segment.

[0020] For the feature points falling into the support domain of the line segment, determine whether they belong to the left support domain or the right support domain, calculate the cosine value of the angle between the direction from the midpoint of the line segment to the point to be detected and the direction of the line segment, and the falling position can be determined by comparing the positive and negative values.

[0021] (4) Construct affine invariant point-line pairs

[0022] Among the set of matching feature points obtained from the support region, the feature points that are not coplanar with the current line segment must be screened out. As Figure 4 shown, since coplanar points should maintain a relatively consistent rotation angle of the image, some outliers can be screened out by the angle. First, calculate the rotation angles of all feature point pairs in the left and right support domains respectively, and then eliminate the outliers with significantly inconsistent rotation angles with this region. These points can be considered as outliers that are not coplanar with the line segment.

[0023] Since every two points and a line segment can form an affine invariant point-line group, for a line segment p with N feature points in the left support domain, it can form groups of affine invariant point-line pairs. The ratio of the distances from the two feature points of each group of affine invariant point-line pairs to the line segment is called the affine invariant ratio (the ratio is not greater than 1). The calculation formula of the affine invariant ratio is:

[0024]

[0025] And the similarity calculation of the affine invariant point-line pair is:

[0026]

[0027] where p and q represent two line segments in the template image and the real-time image, and (X i , X j ) and (Y i , Y j ) represent two pairs of matching points within the support regions of p and q. dis is a distance calculation function, and its return value is the distance from a point to a line. The similarity of the affine invariant point-line pair describes the geometric similarity between two sets of points and lines and is used to calculate the similarity of the two line segments later.

[0028] (5) Calculate the similarity

[0029] Taking the left support region as an example, the line segments to be detected in the template image and the real-time image are p and q respectively. If there are N common feature point pairs within the support regions of the two lines, first take a feature point pair (X i , Y i ) as the base pair, and then sequentially select the remaining N - 1 feature point pairs (X k , Y k ) as the reference pairs. Calculate the affine invariant ratios D(X i , X k , p) and D(Y i , Y k , q) formed by these two point pairs and the line segment. Then, based on this, calculate the point-line pair similarity Affsim(D(X i , X k , p), D(Y i , Y k , q)). After the calculation, represent the median of the similarities of these N - 1 point-line pairs as the similarity Sim i (p, q) of this base pair.

[0030] Loop to calculate the similarities of N base pairs, and according to the ratio of the distances of the affine invariant point-line groups selected by this base pair, put them into a ratio histogram composed of five intervals [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), [0.8, 1.0]. Calculate the maximum value m 1 , m 2 , m 3 , m 4 , m 5 of the similarities in each interval respectively. Finally, the similarity of these two line segments in the left support region is expressed as:

[0031]

[0032] Similarly, calculate the affine invariant similarity Sim right(p, q), and finally take the maximum value of the affine invariant similarities of the left and right support regions as the affine invariant similarity Sim of the current line segment pair:

[0033] Sim(p, q) = max(Sim left (p, q), Sim right (p, q))

[0034] By looping the above calculations, the similarities of all line segment pairs can be obtained.

[0035] (6) Obtain candidate matching line segment pairs

[0036] Since the rotation angle change of the matching line segments should be relatively consistent with the global angle change, this consistency can be used to filter out some mis-matched line segments. Calculate the rotation angle of the line segment pair and compare it with the previously calculated global rotation angle, and eliminate the candidate matching line segment pairs whose difference exceeds the threshold. For the line segment pairs that pass the angle screening, if the similarity is greater than the given threshold, then this line segment pair can be used as a candidate matching line segment pair.

[0037] (7) Construct geometric pairwise constraints

[0038] After the previous screening, the candidate matches may still not match or there may be a non-one-to-one correspondence. Considering the affine invariance of the geometric relationship between two lines in the same image, that is, the crossing angle, the crossing ratio, and the projection ratio of the two lines should satisfy affine invariance, which is called pairwise constraint. As Figure 3 , for two line segments li and lj, S and E are the starting point and ending point of the line segment respectively. The crossing angle is the intersection angle of the extended lines of the two line segments, and the value range of the crossing angle is between -π and π. The crossing ratio is given by the following formula:

[0039]

[0040] (8) Construct the global consistency score

[0041] Include all candidate matching pairs into the adjacency matrix representing pairwise constraints. N pairs of candidate matches can establish an N×N-dimensional adjacency matrix, and this matrix is denoted as the global consistency score matrix A. Then construct the global consistency scoring, and each value of the matrix represents the consistency score of the corresponding line segment pair:

[0042] A ij = 3 - α 1 P ij - α 2 D ij - α 3 Θ ij + α 4 S ij

[0043] where α1 , α 2 , α 3 and α 4 are the proportionality coefficients for each score, with values between 0 and 1, P ij , S ij and Θ ij are the pairwise geometric relationship scores, S ij is the affine invariance score. They are defined as:

[0044] Cross - ratio deviation:

[0045] Projection - ratio deviation:

[0046] Angle - deviation:

[0047] Similarity sum:

[0048] (9) Solving for the final matching pairs

[0049] After constructing the matrix A of the global consistency score, the problem of solving for the final matching pairs is transformed into the problem of maximizing the global consistency score, that is, finding a set of matching pairs X=(x 1 , x 2 , … x N ) T from the candidate matching pairs such that X T AX is maximized, which can be obtained by calculating the dominant eigenvector of the adjacency matrix. Brief Description of the Drawings

[0050] The drawings herein are incorporated into and form a part of this specification, showing embodiments consistent with the present invention and, together with the specification, are used to explain the present invention.

[0051] Figure 1 is the implementation flowchart of the present invention.

[0052] Figure 2 is a schematic diagram of a support domain shown according to an exemplary embodiment of the present invention.

[0053] Figure 3 is a schematic diagram of geometric pairwise constraints shown according to an exemplary embodiment of the present invention.

[0054] Figure 4 is a schematic diagram of screening non - coplanar points using the local rotation angle shown according to an exemplary embodiment of the present invention. Detailed Description of the Invention

[0055] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims; the terms used in the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention; the singular forms "a", "the", and "said" used in the present invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more of the associated listed items.

[0056] To achieve the object of the present invention, the present invention provides an image line segment matching method combining point features and line feature descriptors. The method adopted is as follows:

[0057] Step 1: After inputting the reference image and the retrieval image, extract and match the SIFT feature points of the reference image and the retrieval image respectively. The matching results are screened by the KNN ratio and optimized by RANSAC;

[0058] Step 2: Use a five-layer Gaussian pyramid for the reference image and the retrieval image respectively. Extract line segments through the EDLine algorithm for each layer, and match the line segments in different layers by position, length, and direction. The matched line segments are considered to correspond to the same line segment in reality and are put into the same vector;

[0059] Step 3: Utilize the line segments that have been extracted from the reference image and the retrieval image to construct a rotation histogram and a length histogram, and attempt to calculate the global rotation angle between the two images. If the global rotation angle cannot be calculated, the global rotation angle is set to an illegal value. Among them, the global rotation angle is defined as:

[0060]

[0061] where h l (x) and h r (x) represent the intervals of the rotation histograms of the template image and the real-time image respectively. To obtain a more accurate global rotation angle, in addition to constraining the direction of the line segments, the lengths of the line segments can also be used for screening. Record the cumulative lengths of all line segments in the rotation intervals of the template image and the real-time image. Only when both the change in angle and the change in length are less than a certain threshold can it be considered that the correct global rotation angle has been obtained;

[0062] Step 4: Traverse all the extracted line segments in the reference graph and the retrieval graph, and count whether the feature points extracted in step 1) fall into the support domain of the line segment. The support domain is defined by a rectangular area with a length of 2αl and a width of 2βl, where l is the length of the line segment, and α and β are the proportionality factors of the support domain. The feature points falling into the support domain should satisfy the following formula:

[0063]

[0064]

[0065]

[0066] where the starting point and the ending point of the line segment l are (X s , Y s )(X e , Y e ) respectively, the coordinates of the point to be detected are (X, Y), length(l) represents the length of the line segment, and the calculated dis1 and dis2 represent the distance from the point to be detected to the line represented by the line segment and the distance from the point to be detected to the perpendicular bisector of the line represented by the line segment respectively. When the line represented by the line segment is parallel to the x-axis, that is, the perpendicular bisector is parallel to the y-axis, the calculation of dis2 should become:

[0067]

[0068] If the point to be detected falls into the support domain, it should be detected whether the support domain it falls into is the left support domain or the right support domain, that is, calculate the cosine value of the angle between the direction from the midpoint of the line segment to the point to be detected and the direction of the line segment. The formula is:

[0069]

[0070] By comparing the positive and negative of toward, it can be judged which support domain it is in. If it is positive, it means it is in the left support domain, otherwise it is in the right support domain;

[0071] Step 5: Extract a line segment from each of the reference graph and the retrieval graph to form a line segment pair to be detected. For the two lines in the line segment pair, first detect whether there are matching feature points in the left (right) support domain of the line segment in the retrieval graph for the feature points falling into the left (right) support domain of the reference graph line segment, and count the number of them. If the number of matching feature points in both the left support domain and the right support domain is less than 2, it is considered that the current line segment pair is illegal, and the next line segment pair is recalculated. Otherwise, go to step 6);

[0072] Step 6: For the current line segment pair, use the regional rotation angle of points to screen points, calculate the change value of the direction angles of the two line segments, that is, the regional rotation angle, denoted as Θ. Detect the matching feature point pairs in the left and right support regions. If the difference between the angle change of the points and Θ exceeds the threshold, it is considered that the current feature point pair is not coplanar with the line segment, and this feature point pair is discarded. After detecting all feature point pairs in all support regions, if the number of matching feature point pairs in both the left support region and the right support region is less than 2, it is considered that the current line segment pair is illegal, and return to Step 5 to recalculate the next line segment pair; otherwise, enter Step 7).

[0073] Step 7: The line segments to be detected in the template image and the real-time image are p and q respectively. First, calculate the affine invariant similarity of the left support region. If there are N common feature point pairs in its support region, first take a feature point pair (X i , Y i ) in the support region as the base pair, and then sequentially select the remaining N - 1 feature point pairs (X k , Y k ) as the reference pairs, and calculate the affine invariant ratios D(X i , X k , p) and D(Y i , Y k , q), as well as the point-line pair similarity Affsim(D(X i , X k , p), D(Y i , Y k , q)). Among them:

[0074]

[0075]

[0076] Based on this, calculate the similarity Sim i (p, q):

[0077]

[0078] Loop to calculate the similarities of N base pairs, and according to the ratio of the affine invariant point-line group distances selected by this base pair, put them into the ratio histogram composed of five intervals [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), [0.8, 1.0], and calculate the maximum value m 1 , m 2 , m 3 , m 4 , m 5 of the similarity in each interval, and calculate the similarity of the left support region:

[0079]

[0080] Calculate the affine invariant similarity Sim of the right support region in the same way right (p, q), and then take the maximum value of the affine invariant similarities of the left and right support regions as the similarity Sim(p, q) of the current line segment pair:

[0081] Sim(p, q) = max(Sim left (p, q), Sim right (p, q))

[0082] If the similarities of all line segment pairs have been calculated, go to step 8); otherwise, go back to step 5) to calculate the next line segment pair;

[0083] Step 8: Traverse all line segment pairs, check the similarities of the line segment pairs, and discard the line segment pairs if the similarity is less than the threshold;

[0084] Step 9: Traverse the remaining line segment pairs. If the global rotation angle was calculated in step 3), check the difference between the angle change of the line segment pair and the global rotation angle. If it is less than the angle threshold, it is considered qualified and saved as a candidate matching line segment pair. If the global rotation angle cannot be calculated in step 3), directly consider the line segment pair as a candidate matching line segment pair.

[0085] The candidate matching line segment pairs are stored in a separate vector, denoted as CM, and CM can be expressed as:

[0086]

[0087] Each element in CM represents a candidate matching line segment pair;

[0088] Step 10: According to the number N of candidate matching line segment pairs, construct a two-dimensional array of size N×N, denoted as matrix A, for storing the global consistency scores. Since the geometric relationship between two line segments in the same graph is stable under affine transformation, construct Figure 3 the extended lines as shown, and use the generated intersection angles and projection distances to construct the descriptive quantity cross ratio p i and projection ratio d i :

[0089]

[0090] For two line segments in two graphs, the cross ratio and projection ratio can be calculated in the same way. In addition, denote the angle formed by the two extended lines as θ, and the angles in the two graphs are respectively and Using the above descriptive quantities, three items in the global consistency score can be obtained:

[0091] Cross-ratio deviation:

[0092] Projection ratio deviation:

[0093] Angle deviation:

[0094] These three quantities are used to describe the relative geometric relationship between two line segments. In addition, the similarity is taken as the fourth item:

[0095] Sum of similarities:

[0096] Calculate the global consistency score:

[0097] A ij = 3 - α 1 P ij - α 2 D ij - α 3 Θ ij + α 4 S ij

[0098] where α 1 , α 2 , α 3 and α 4 respectively represent four weights, which are used to measure the influence degree of the corresponding description quantity on the global consistency score. After calculating the global consistency score, it is stored in the corresponding position of the two-dimensional array for the next step of screening;

[0099] Step 11: Initialize the vector LM as an empty vector to store the final matching result, and use ARPACK to calculate the dominant eigenvector X of matrix A;

[0100] Step 12: Assume the dimension of the vector is k. Each time, find the largest element in vector X, record its position in X as a, and the value at this position is X(a), that is:

[0101]

[0102] If X(a) is not zero, then remove the line segment pair l corresponding to position a in CM and include it in LM, and modify X(a) to 0:

[0103]

[0104]

[0105] X(a) = 0

[0106] After detecting CM, delete other line segment pairs j that conflict with a from CM, and modify X(j) to 0:

[0107]

[0108] X(j) = 0

[0109] Repeat the current step until the newly obtained X(a) is zero or CM becomes an empty vector. At this time, it is considered that all matching line segment pairs have been screened, and the final matching results are stored in LM.

Claims

1. An image matching method based on the affine invariance of points and lines, characterized in that, the method comprises the following steps: (1) Extract point and line features from the template image and the real-time image at multiple scales: Construct Gaussian pyramids for the template image and the real-time image, extract feature points and extract line segments. Match the line features of different layers, and the matched line segments of different layers will be regarded as the same line segment in space; match the extracted feature points to obtain candidate matching point pairs, and then eliminate the wrong matches to obtain feature point pairs; (2) Construct a rotation histogram and calculate the global rotation angle: The direction of the line segment is divided into 18 rotation intervals within the range of 360°. Normalize the direction angles of the template image and the real-time image into the rotation intervals to obtain a rotation histogram. At the same time, record the cumulative length of all line segments in the rotation intervals of the template image and the real-time image. Combine the angle change and the length change to comprehensively calculate the global rotation angle; (3) Determine the line segment support region: Divide the two regions on the left and right sides of the straight line. The region pointed by the average line gradient direction is denoted as the right region. The positive direction of the straight line whose average gradient direction is orthogonal and rotates counterclockwise. Denote the region within a certain range on the left side of the straight line as the left support region, and the region within a certain range on the right side as the right support region. For the feature point line segment midpoint falling into the support region, detect whether the cosine value of the angle between the direction to the point to be detected and the line segment direction falls into the left support region or the right support region; (4) Construct affine invariant point-line pairs: Use the regional rotation angle to screen out some outlier points that are not coplanar with the line segment in the support region of the line segment pair to be detected. After screening, if the number of feature point pairs in the left and right support regions is less than 2, the line segment pair is considered illegal and cannot form an affine invariant point-line group. For the formed affine invariant point-line group, calculate the affine invariant ratio of the feature points, and calculate the similarity of the affine invariant point-line pairs based on this; (5) Calculate the similarity: For the line segment pair to be detected, calculate the similarity of all base pairs in the left support region, and use the ratio histogram to calculate the similarity of the left support region. The right support region uses the same calculation method, and the maximum value of the similarities of the left and right support regions is taken as the similarity of the line segment pair. Use this calculation method to calculate the similarities of all line segment pairs; (6) Obtain candidate matching line segment pairs: If the global rotation angle can be calculated, screen out the line segment pairs whose deviation of the line segment rotation angle from the global rotation angle exceeds the threshold, and the remaining line segment pairs are used as candidate matching line segment pairs; If the global rotation angle cannot be calculated, directly use all line segment pairs as candidate matching line segment pairs; (7) Construct geometric pairwise constraints: For a pair of candidate matching line segment pairs, use pairwise constraints to construct a cross ratio; (8) Construct a global consistency score: Calculate the sum of the cross ratio deviation, projection ratio deviation, included angle deviation and similarity of the candidate matching line segment pairs to obtain a consistency score; For all candidate matching line segment pairs, construct a global consistency score matrix; (9) Solve the final matching pairs: Find the main eigenvector of the similarity matrix, extract the maximum value of the main eigenvector in turn, and screen out the line segment pairs corresponding to the maximum value position and put them into the set. The elements in the final set are the final matching pairs.

2. The image matching method according to claim 1, It is characterized in that In step (1), point-line features are extracted from the template image and the real-time image and matched. Specifically: Build a five-layer Gaussian pyramid for the template image and the real-time image, extract SIFT feature points as point features, use the EDLine algorithm to extract line segments at each layer. For line segments in different layers, match them using position, length, and direction. The successfully matched line segments are regarded as the same line segment in space; The extracted point features obtain candidate matching point pairs through brute-force matching. The KNN ratio screening and the RANSAC algorithm are used to filter out incorrect matches for the candidate matching point pairs to obtain the matched feature point pairs.

3. The image matching method according to claim 1, It is characterized in that After obtaining the rotation histogram in step (2), calculate the global rotation angle. The global rotation angle to be found is the rotation angle that can minimize the difference between the rotation histograms of the template image and the real-time image under this rotation, that is: where h l (x) and h r (x) represent the intervals of the rotation histograms of the template image and the real-time image respectively. When jointly constraining the global rotation angle using the line segment length and angle, it should be ensured that both the change in angle and the change in length are less than a certain threshold before the obtained global rotation angle is considered legal.

4. The image matching method according to claim 1, It is characterized in that In step (3), the division and judgment of the support region are specifically as follows: The support region is defined by a rectangular region with a length of 2αl and a width of 2βl, where l is the length of the line segment, and α and β are the scale factors of the support region. Based on this length and width, the support region can be determined around the line segment; The support region is defined by a rectangular region with a length of 2αl and a width of 2βl, where l is the length of the line segment, and α and β are the scale factors of the support region. Based on this length and width, the support region is determined around the line segment; Denote the starting point and the ending point of the line segment l as (X s , Y s )(X e , Y e ), and the coordinates of the point to be detected are (X, Y). Then, the feature points falling within the support region should satisfy: When the line represented by the line segment is parallel to the x-axis, that is, the perpendicular bisector is parallel to the y-axis, the calculation of dis2 should become: To judge whether it falls into the support region, the cosine value of the angle between the direction from the midpoint of the line segment to the point to be detected and the direction of the line segment should be calculated, that is, calculate: Judge the support region where it is located by comparing the sign of toward. If it is positive, it means it is in the left support region, otherwise it is in the right support region.

5. The image matching method according to claim 1, It is characterized in that In step (4), the similarity of the affine invariant point-line pairs is specifically as follows: Use the regional rotation angle to filter out non-coplanar points. The regional rotation angle refers to the direction angle change of the line segment pairs to be matched. The direction change of the outliers to be filtered out should be close to the regional rotation angle. If the deviation between the two exceeds the threshold, it is considered that the feature point and the line segment are non-coplanar and regarded as outliers; In the process of calculating the similarity of the affine invariant point-line pair, denote the line segments to be detected in the template image and the real-time image as p and q respectively. First, take a feature point pair (X i , Y i ) within its support domain as the base pair, and then sequentially select the remaining N - 1 feature point pairs (X k , Y k ) as the reference pairs to calculate the affine invariant ratios D(X i , X k , p) and D(Y i , Y k , q), as well as the point-line pair similarity Affsim(D(X i , X k , p), D(Y i , Y k , q)), where: 。 6. The image matching method according to claim 1, It is characterized in that In step (5), the calculation of the similarity of the line segment pairs is specifically as follows: The similarity of the point-line pair calculated in step 4), based on which the similarity Sim of the base pair is calculated i (p, q): After that, the similarity of N base pairs is calculated cyclically, and it is placed into a ratio histogram composed of five intervals [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), [0.8, 1.0] according to the ratio of the distances of the selected affine invariant point-line groups of the base pairs. The maximum value m of the similarity in each interval is calculated respectively 1 , m 2 , m 3 , m 4 , m 5 , and the similarity of the left support domain is calculated as follows: Calculate the affine invariant similarity Sim of the right support region in the same way right (p, q), and finally take the maximum value of the affine invariant similarities of the left and right support regions as the similarity Sim(p, q) of the current line segment pair: Sim(p,q) = max(Sim left (p,q), Sim right (p,q)).

7. The image matching method according to claim 1, It is characterized in that In step (6), for the candidate matching line segment pairs, use the vector CM to store the candidate matching line segment pairs. CM can be expressed as: Each element in CM represents a candidate matching line segment pair.

8. The image matching method according to claim 1, It is characterized in that In step (7), the pairwise constraints are specifically as follows: The geometric relationship between two lines in the same image is affine invariant, that is, the crossing angle, the crossing ratio, and the projection ratio of the two lines should satisfy affine invariance, which is called pairwise constraint. Among them, the crossing angle is the intersection angle of the extended lines of the two line segments, and the value range of the crossing angle is between -π and π. The crossing ratio is given by the following formula: 。 9. The image matching method according to claim 1, It is characterized in that In step (8), the construction process of the similarity matrix is specifically as follows: Initialize the global consistency score matrix according to the size of CM. Denote the size of CM as N, then the scale of the matrix is N×N. Each value in the matrix represents the consistency score of the corresponding line segment pair. The score is defined as: A ij = 3 - α 1 P ij -α 2 D ij -α 3 Θ ij +α 4 S ij where α 1 , α 2 , α 3 and α 4 are the proportionality coefficients of each score, with values between 0 and 1, P ij , D ij and Θ ij are the paired geometric relationship scores, S ij is the affine invariance score; they are defined as: Cross-ratio deviation: Projection ratio deviation: Angle deviation: Similarity sum: .

10. The image matching method according to claim 1, characterized in that, the process of solving the final matching pairs in step (9) is specifically as follows: The initialization vector LM is an empty vector, which is used to store the final matching result. The ARPACK is used to calculate the principal eigenvector X of matrix A. Assume that the dimension of the vector is k. Each time, the largest element in vector X is found, and the position where it is located in X is denoted as a, and the value at this position is X(a), that is: If X(a) is not zero, remove the line segment pair l corresponding to the position a in CM and incorporate it into LM, and modify X(a) to 0: X(a) = 0 After that, detect CM, delete other line segment pairs j conflicting with a from CM, and modify X(j) to 0: X(j) = 0 Repeat the current step until the newly obtained X(a) is zero or CM becomes an empty vector. At this time, it is considered that all matching line segment pairs have been screened, and the final matching result is stored in LM.

Images