A method for matching non-coded landmarks in stereo image pairs

Abstract

The present invention discloses a method for matching non-coding marker points in a stereo pair, including: inputting a stereo pair; identifying marker points; solving corresponding points; excluding isolated marker points; constructing a shape context; calculating a similarity measure matrix; solving the optimal match; eliminating mismatches; and outputting a matching result. The present invention takes into account the spatial topological relationship of non-coding marker points and the epipolar geometric constraints in the stereo pair: before constructing the shape context, epipolar constraints are used to exclude isolated marker points in the stereo pair and reduce the influence of interference factors; a similarity measure matrix calculation formula is proposed to accelerate the iterative convergence process of solving the optimal match and reduce the probability of iterative non-convergence; after the shape context is used to obtain the optimal match, the mismatched point pairs are eliminated through epipolar constraints to improve the reliability of the matching result. The present invention is not only suitable for matching discretely distributed non-coding marker points, but also can achieve correct matching of regularly distributed non-coding marker points.

Description

Technical Field

[0001] The present invention belongs to the field of binocular stereo vision technology, relates to a stereo matching method, and further relates to a non-coding landmark point matching method in a stereo image pair combining shape context and epipolar constraints. Background Art

[0002] Binocular stereo vision is an important part of computer vision. It uses a binocular camera to simultaneously collect left and right images from different perspectives, namely stereo pairs, and restore the three-dimensional spatial information of an object through the principle of parallax. When using binocular stereo vision for precision measurement, some artificial markers are usually pasted on the surface of the object to be measured. During measurement, the artificial markers in the stereo pair are located and matched with sub-pixel accuracy, and finally a high-precision three-dimensional reconstruction result is obtained, thereby meeting the needs of high-precision industrial measurement.

[0003] The matching of the same-name landmarks in a stereo pair (i.e., the imaging of the same artificial landmark in the left and right images) is the basis of three-dimensional reconstruction. For coded landmarks, automatic matching can be performed by the landmark number (ID); but for non-coded landmarks, a special algorithm is required for matching. Chinese invention patent CN201810008643.9 proposes a binocular stereo image matching method based on a minimum spanning tree, but the color, grayscale or gradient of the pixel point is required to calculate the matching cost, while only the coordinate information of the non-coded landmark point is available, so it is not suitable for the matching of non-coded landmark points. Chinese invention patent CN202010847540.9 discloses a stereo matching method based on deep learning, the disadvantage of which is that there is a lack of high-quality data sets in the field of stereo matching, and it is difficult to obtain a network with generalization ability through simple training, resulting in poor applicability of the stereo matching method. Chinese invention patent CN201710615418.7 provides a stereo matching method based on adaptive weights and local entropy. This method calculates weights based on the grayscale similarity and spatial proximity of pixel points, and the grayscale changes around non-coded markers are small, so this method is not suitable for matching non-coded markers. Chinese invention patent CN201510253385.7 provides a method that combines shape context features that can reflect the distribution relationship of point positions for coarse matching and gradient direction histograms that can reflect the gradient attributes of the points themselves for precise matching. However, this method can only be used for matching the contour of the workpiece, and cannot be used for matching non-coded markers in stereo pairs.

[0004] In summary, the existing stereo matching methods mainly use the similarity of pixel color, grayscale, gradient or space to achieve the matching of pixels with the same name. However, non-coding markers are generally white dots on a black background or black dots on a white background with monotonous color and obvious contrast. The color, grayscale, gradient and other feature information of the markers in the image are relatively consistent and cannot be used to distinguish different markers. Therefore, the existing stereo matching methods are difficult to apply to the stereo matching of non-coding markers. However, it is easy to produce mismatches when matching non-coding markers in stereo pairs by relying solely on epipolar constraints, especially for non-coding markers distributed in rows and columns. Multiple markers may be located on the same epipolar line, so that a non-coding marker in the left image corresponds to multiple non-coding markers in the right image (one to many), or multiple non-coding markers in the left image correspond to a non-coding marker in the right image (many to one), resulting in mismatches such as misalignment. Therefore, it is urgent to study more robust non-coding marker matching methods. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a method for matching non-coded landmark points in a stereo pair. Shape context is used as a feature descriptor based on shape contours. According to the relationship between the target contour sampling points (such as distance, gradient, etc.), matching between targets with similar outlines in different images is achieved. However, the distribution of non-coded landmark points in a stereo pair is arranged according to the measurement requirements and does not necessarily reflect the outline of the object. In addition, the number of landmark points is often small and is greatly affected by interference factors such as noise. Therefore, when directly using shape context to solve the optimal matching of non-coded landmark points in a stereo pair, it is very easy to encounter the problem of iterative non-convergence, resulting in matching failure. To this end, the present invention combines shape context with polar constraints to overcome the problems of the two methods and improve the accuracy and robustness of matching. First, before constructing the shape context, the epipolar constraint is used to exclude isolated landmarks in the stereo pair and reduce the influence of interference factors; second, when calculating the similarity measure matrix, the one-to-one corresponding non-encoding landmarks in the stereo pair are determined by the epipolar constraint, and a higher similarity measure value is assigned to accelerate the optimal matching solution process and reduce the probability of non-convergence of iteration; finally, after the shape context is used to obtain the optimal match, the epipolar constraint is used to eliminate the mismatched point pairs, thereby improving the reliability of the matching results. Compared with the existing non-encoding landmark matching technology, the method of the present invention has the advantages of high accuracy and strong robustness.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for matching non-coded landmarks in a stereo image pair comprises the following steps:

[0008] The first step is to input the stereo image pair. Input the stereo image pair to be matched and its internal and external orientation elements.

[0009] The second step is to identify the landmark points. The center coordinates of the non-encoded landmark points in the stereo image pair input in the first step are extracted.

[0010] The third step is to solve the corresponding points. Based on the epipolar constraints, the corresponding points of the non-encoded landmark points in the stereo image pair identified in the second step are solved.

[0011] The fourth step is to exclude isolated landmark points. The corresponding points solved in the third step are counted, isolated points are marked, and they are removed from the set of non-encoded landmark points so that they do not participate in the subsequent shape context matching.

[0012] Step 5: Shape context construction: A shape context is established for each non-encoded landmark point in the stereo image pair remaining after excluding isolated points in step 4.

[0013] Step 6: Calculate the similarity measure matrix. Based on the correspondence between the non-coding landmark points in the left and right images solved in step 3 and the shape context of the non-coding landmark points constructed in step 5, calculate the similarity measure matrix.

[0014] Step 7: Find the best match. Based on the similarity measure matrix obtained in step 6, the bipartite graph matching algorithm is used to iteratively find the best match.

[0015] Step 8: Eliminate mismatches. Use epipolar constraints to eliminate mismatched point pairs from the non-coding landmark point pairs successfully matched in step 7, thereby improving the reliability of the matching results.

[0016] Step 9: Output the matching results. Output the correspondence between the non-encoded landmarks in the left and right images of the stereo pair to complete the matching of the stereo pair.

[0017] Furthermore, the method for eliminating mismatches in the eighth step is: for a pair of non-coding marker points p successfully matched in the seventh step, i With q π(i) First, calculate p according to the epipolar solution method described in step 3 i The epipolar line f in the right view ri , and q π(i) to f ri The distance d ri ; Secondly, calculate q π(i) The epipolar line f in the left view li , and p i to f li The distance d li ; Let d i =(d li +d ri ) / 2, if di >D T , then we think that p i With q π(i) is a mismatched point pair, and this pair of matches is deleted from the optimal match π calculated in step 7. Where π represents the correspondence between the non-encoded marker points in the left and right images obtained in step 7, and π(i) represents the i-th non-encoded marker point p in the left image. i Corresponding to the π(i)th non-encoded landmark point q in the right image π(i) , 1≤i≤n1, 1≤j≤m1, n1 and m1 represent the number of non-encoded landmark points remaining in the left image and the right image after eliminating isolated points in the fourth step, respectively, D T is the distance threshold described in the third step.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) The method of the present invention takes into account the spatial topological relationship of the landmarks and the epipolar geometric constraints in the stereo image pair, overcoming the respective shortcomings of the traditional epipolar constraint and shape context methods, and achieving higher matching accuracy and stronger robustness.

[0020] (2) Before constructing the shape context, the method of the present invention adopts epipolar constraints to exclude isolated landmarks in the stereo image pair and reduce the influence of interference factors, so that the shape context algorithm can also be applied to the matching of discretely distributed non-encoded landmarks.

[0021] (3) The method of the present invention proposes a new similarity measurement matrix calculation formula. The similarity measurement values ​​corresponding to the one-to-one corresponding non-coding landmark point pairs in the stereo image pair determined by the epipolar line constraint are assigned larger constant weights, which accelerates the iterative convergence process of the optimal match and reduces the probability of iterative non-convergence.

[0022] (4) After the shape context is used to obtain the optimal match, the mismatched point pairs are eliminated through the epipolar constraint, thereby improving the reliability of the matching results.

[0023] (5) The method of the present invention can be applied to the matching of non-coding landmarks that present a certain distribution pattern, such as the matching of non-coding landmarks distributed in rows and columns. Therefore, the method is more robust than the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the specific operation steps of the present invention.

[0025] Figure 2 The figure shows a pair of input stereo images to be matched. Wherein, a is the image captured by the left camera, that is, the left image in the input stereo image pair, and b is the image captured by the right camera, that is, the right image in the input stereo image pair.

[0026] Figure 3 for Figure 2 The recognition result of the center coordinates of the non-encoded landmark points in the stereo image pair, where a is Figure 2 The recognition result of non-coding landmarks in a and b is Figure 2 b Non-coding landmark point recognition results.

[0027] Figure 4 For detection Figure 2 Isolated landmark points in Figure b.

[0028] Figure 5 The result of matching the input stereo image using the traditional epipolar constraint, where a is the left image matching result and b is the right image matching result.

[0029] Figure 6 It is the result of matching the input stereo image pair using the method of the present invention, wherein a is the left image matching result, and b is the right image matching result. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0031] The present invention proposes a method for matching non-encoded landmark points in stereo image pairs by combining shape context and epipolar constraints. Figure 1 As shown, it mainly includes the following steps:

[0032] The first step is to input the stereo pair.

[0033] Input the stereo image pair to be matched and its internal and external orientation elements. In this embodiment, the left and right images in the input stereo image pair are required to contain at least 3 pairs of non-encoded landmarks with the same name; at the same time, the internal and external orientation elements of the stereo image pair, i.e., the internal and external parameters of the binocular camera that shoots the stereo image pair, are also required to be input. Figure 2 The following is an input stereo pair to be matched. Figure 2 a is the image captured by the left camera, Figure 2 b is the image captured by the right camera.

[0034] The second step is landmark point recognition.

[0035] Extract the center coordinates of the non-coded markers in the stereo image pair input in the first step. Common non-coded markers include circular markers, square markers, vertex markers, and cross markers. These markers define the center of their contours or edge intersections as the characteristic positions of the markers (i.e., the center coordinate positions). The characteristic position of a circular marker is the center of the circle, and its identification and extraction is relatively simple. The edge extraction algorithm is used to find the sub-pixel edge contour, and then the sub-pixel contour is fitted with the least squares ellipse to obtain the center coordinates of the circular marker. The characteristic positions of the vertex markers and the cross markers are the edge intersections. The center coordinates of the markers are obtained by extracting two edges and fitting a straight line to find the intersection. The characteristic position of the square marker is the center position of the inner square. When identifying, first extract the four vertices of the inner square, and then calculate the intersection of the diagonals as the center coordinates of the marker. Figure 3 Shown Figure 2 The recognition results of the center coordinates of the non-encoded landmark points in the stereo image pair in , such as Figure 3 As shown in a and b in the figure, the intersection of the white cross lines indicates the center of the identified landmark point, and the landmark point numbers are all set to -1.

[0036] The third step is to solve the corresponding points.

[0037] Based on the epipolar constraints, the corresponding points of the non-encoded landmark points in the stereo image pair extracted in the second step are solved. The method for solving the corresponding points is as follows:

[0038] 1) For any non-coding landmark point p in the left image extracted in the second step i , construct the basic matrix F according to the internal and external orientation elements of the stereo image pair input in the first step, and then solve the non-coding landmark point p i The epipolar line f in the right image ri Among them, p i ∈P,P={p 1 ,p 2 ,...,p n0} represents the set of non-coding landmark points in the left image identified in the second step, 1≤i≤n0, n0 represents the number of non-coding landmark points.

[0039] 2) Calculate each non-coding landmark point q in the right image extracted in the second step j To the outer polar line f solved in step 1) ri The distance d rij , if d rij Less than a given distance threshold D T , that is, d rij <D T , then the non-encoded landmark point q in the right image j Marked as the non-encoded landmark point p in the left image i The corresponding point ofj ∈Q, Q={q 1 ,q 2 ,...,q m0} represents the set of non-encoded landmark points in the right image identified in the second step, 1≤j≤m0, m0 represents the number of non-encoded landmark points. Distance threshold D T The value is generally 3 to 6 pixels.

[0040] 3) For any non-coding landmark point q in the right image extracted in the second step j , according to the basic matrix F constructed in step 1), solve the non-coding landmark point q j The epipolar line f in the left image lj .

[0041] 4) Calculate each non-coding landmark point p in the left image extracted in the second step i To the outer polar line f solved in step 3) lj The distance d lji , if d lji Less than a given distance threshold D T , that is, d lji <D T , then the non-encoded landmark point p in the left image i Marked as the non-encoded landmark point q in the right image j The corresponding point of .

[0042] The fourth step is to exclude isolated landmark points. The corresponding points solved in the third step are counted, isolated points are marked, and they are removed from the set of non-encoded landmark points so that they do not participate in the subsequent shape context matching.

[0043] In an embodiment of the present invention, the specific method for excluding isolated marker points is as follows: the number of corresponding points of the non-coding marker points in the stereo image pair solved in the third step is counted. If the number of corresponding points of a non-coding marker point is 0, it is marked as an isolated point and removed from the set of non-coding marker points so that it does not participate in the subsequent shape context matching, thereby eliminating its interference with the subsequent shape context matching and improving the reliability and robustness of the matching. The set of non-coding marker points in the left image after excluding the isolated points is denoted as P = {p 1 ,p 2 ,...,p n1}, n1≤n0 represents the number of non-coding landmark points remaining in the left image after excluding isolated points; the set of non-coding landmark points in the right image after excluding isolated points is recorded as Q={q 1 ,q 2 ,...,q m1}, m1≤m0 represents the number of non-encoded landmark points remaining in the right image after excluding isolated points. Figure 4The non-encoded landmark points circled in the figure are excluded isolated landmark points. It can be seen from the figure that these landmark points are noise points in the background. These noise points have a great influence on the subsequent shape context matching. If they are not eliminated, it may lead to the failure of subsequent matching.

[0044] Step 5: Shape context construction. A shape context is established for each non-encoded landmark point in the stereo image pair remaining after excluding isolated points in step 4. The specific process is as follows:

[0045] 1) For any non-coding landmark point p in the left image after excluding isolated points in the fourth step i , 1≤i≤n1, establish p i A polar coordinate system with the pole as the center and R as the radius is created. Next, the circle is divided into k equal parts in the polar direction according to the logarithmic distance. 1 parts, divided equally into k parts along the polar angle direction 2 The area is divided into k parts. 1 ×k 2 sub-regions. Among them, R = log(max(p j -p i )), 1≤j≤n1, max() represents the maximum value function, log() represents the logarithmic function; k 1 Generally, 5 to 7 k 2 Generally, the value is 10 to 12.

[0046] 2) Count the number of other non-coding landmarks in the left image in each sub-region divided in step 1) to obtain the histogram h i (k), 1≤k≤k 1 ×k 2 , called non-coding landmark point p i The shape context.

[0047] 3) Repeat steps 1) and 2) to calculate the shape context of the remaining non-encoded landmark points in the left image.

[0048] 4) For the remaining non-coding landmark points in the right image after excluding isolated points in the fourth step, the shape context of each non-coding landmark point is constructed according to the process described in steps 1) to 3).

[0049] Step 6: Calculate the similarity measure matrix. Based on the correspondence between the non-coding landmarks in the left and right images solved in step 3 and the shape context of the non-coding landmarks constructed in step 5, calculate the similarity measure matrix. The formula for calculating the element in the i-th row and j-th column of the similarity measure matrix is ​​as follows:

[0050]

[0051] Among them, Cij represents the element in the i-th row and j-th column of the similarity matrix, C(p i ,q j ) represents the non-encoded landmark point p in the left image i and the non-encoded landmark point q in the right image j similarity measure, W is a constant weight, generally ranging from 0.7 to 0.9; h i (k) and h j (k) represent the non-encoded landmark point p in the left image of the stereo pair constructed in the fifth step. i and the non-encoded landmark point q in the right image j shape context, 1≤i≤n1, 1≤j≤m1, n1 and m1 represent the number of non-encoded landmark points remaining in the left image and the right image after excluding isolated points in the fourth step.

[0052] If the non-encoded landmark point p in the left image solved in the third step i There is only one corresponding point q in the right image j , and the non-encoded landmark point q in the right image j There is also only one corresponding point p in the left image. i , then p i With q j is a one-to-one corresponding point, otherwise, p i With q j It's not a one-to-one correspondence.

[0053] Step 7: Find the best match. Based on the similarity measure matrix obtained in step 6, the bipartite graph matching algorithm is used to iteratively find the best match.

[0054] In one embodiment of the present invention, based on the similarity measurement matrix calculated in the fifth and sixth steps, a bipartite graph matching algorithm is used to solve the optimal match π. The optimization objective function for solving the optimal match π is:

[0055]

[0056] Among them, π represents the correspondence between the non-coding landmarks in the left and right images, and π(i) represents the i-th non-coding landmark p in the left image. i Corresponding to the π(i)th non-encoded landmark point q in the right image π(i) The π corresponding to the minimum value of the objective function H(π) is the optimal match.

[0057] Step 8: Eliminate mismatches. Use epipolar constraints to eliminate mismatched point pairs from the non-coding landmark point pairs successfully matched in step 7, thereby improving the reliability of the matching results.

[0058] In one embodiment of the present invention, for a pair of non-coding marker points p successfully matched in step 7, i With q π(i) First, calculate p according to the epipolar solution method described in step 3 i The epipolar line f in the right view ri , and q π(i) to f ri The distance d ri ; Secondly, calculate q π(i) The epipolar line f in the left view li , and p i to f li The distance d li ; Let d i =(d li +d ri ) / 2, if d i >D T , then we think that p i With q π(i) For a mismatched point pair, delete this pair of matches from the optimal match π calculated in step 7. T is the distance threshold, and its value is as described in the third step.

[0059] Step 9: Output the optimal matching result. Output the correspondence between the non-encoded landmarks in the left and right images of the stereo pair to complete the matching of the stereo pair.

[0060] In one embodiment of the present invention, after eliminating the mismatches, the correspondence between the non-coding points in the left view and the non-coding points in the right view that are finally retained is output to complete the matching. Figure 5 Figures a and b show the matching results when only epipolar constraints are used. Figure 6 a and b in the figure are the matching results of the method of the present invention. After the matching is completed, the numbers of the corresponding points in the left and right images are modified to the same number greater than 100. Figure 5 As can be seen from a and b, when only epipolar line constraint matching is used, all the marker points coded 137 to 150 are matched incorrectly. These marker points are arranged in rows, so their epipolar lines are almost on the same straight line, and the number of marker points on the epipolar line in the left and right images is the same, so they can match each other. However, in fact, due to the shooting angle, the marker points in the image are misplaced (one marker point is misplaced), which leads to incorrect matching results. In addition, the marker points numbered 130, 131, and 133 are also matched incorrectly. All the matching results that rely solely on epipolar line constraint matching Figure 5 The landmark points in the stereo pair are not feasible. Figure 6 It can be seen from a and b that when the matching method of the present invention is used, the corresponding landmark points in the stereo image pair can be correctly matched.

[0061] In summary, the method of the present invention takes into account the spatial topological relationship of non-coding landmarks and the epipolar geometric constraints in the stereo image pair: before constructing the shape context, the epipolar constraint is used to exclude isolated landmarks in the stereo image pair and reduce the influence of interference factors; a new similarity measure matrix calculation formula is proposed to accelerate the iterative convergence process of solving the optimal match and reduce the probability of iterative non-convergence; after the shape context is used to obtain the optimal match, the epipolar constraint is used to eliminate the mismatched point pairs and improve the reliability of the matching results. The method of the present invention overcomes the respective shortcomings of the traditional epipolar constraint and shape context methods. It is not only suitable for matching discretely distributed non-coding landmarks, but also can achieve correct matching of regularly distributed (such as distributed in rows and columns) non-coding landmarks. It is more robust and reliable than traditional methods.

Claims

1. A method for matching non-coded landmarks in a stereo image pair, characterized in that: The steps include: The first step is to input the stereo pair Input the stereo image pair to be matched and its internal and external orientation elements; The second step is landmark recognition Extract the center coordinates of the non-encoded landmarks in the stereo image pair input in the first step; The third step is to solve the corresponding points Solve the corresponding points of the non-encoded landmark points in the stereo image pair identified in the second step based on the epipolar constraints; Step 4: Eliminate isolated landmarks Count the corresponding points solved in the third step, mark the isolated points, and remove them from the set of non-encoding landmark points so that they do not participate in the subsequent shape context matching; Step 5: Shape Context Construction Establish a shape context for each non-encoded landmark point in the stereo image pair remaining after eliminating isolated points in the fourth step; Step 6: Similarity measurement matrix calculation Calculate the similarity measurement matrix based on the correspondence between the non-encoded landmark points in the left and right images solved in the third step and the shape context of the non-encoded landmark points constructed in the fifth step; Step 7: Find the optimal match Based on the similarity measurement matrix obtained in the sixth step, the bipartite graph matching algorithm is used to iteratively solve the optimal match; Step 8: Eliminate false matches The polar line constraint is used to eliminate the mismatched point pairs in the non-coding landmark point pairs successfully matched in the seventh step, thereby improving the reliability of the matching results; Step 9: Output the matching results Output the correspondence between the non-coded landmarks in the left and right images of the stereo pair to complete the matching of the stereo pair.

2. The method for matching non-coded landmark points in a stereo image pair according to claim 1, characterized in that: When the stereo image pair is input in the first step, the left and right images in the input stereo image pair contain at least 3 pairs of non-encoded marker points with the same name.

3. The method for matching non-coded landmarks in a stereo image pair according to claim 1, characterized in that: The method for solving the corresponding points in the third step is as follows: 1) For any non-coding landmark point p in the left image identified in the second step i , construct the basic matrix F according to the internal and external orientation elements of the stereo image pair input in the first step, and then solve the non-coding landmark point p i The epipolar line f in the right image ri ; Among them, p i ∈P,P={p1,p2,...,p n0 } represents the set of non-coding landmark points in the left image identified in the second step, 1≤i≤n0, n0 represents the number of non-coding landmark points; 2) Calculate each non-coding landmark point q in the right image extracted in the second step j To the outer polar line f solved in step 1) ri The distance d rij , if d rij Less than a given distance threshold D T , that is, d rij <D T , then the non-encoded landmark point q in the right image j Marked as the non-encoded landmark point p in the left image i The corresponding point of j ∈Q, Q={q1,q2,...,q m0 } represents the set of non-coding landmark points in the right image identified in the second step, 1≤j≤m0, m0 represents the number of non-coding landmark points; the distance threshold D T The value ranges from 3 to 6 pixels; 3) For any non-coding landmark point q in the right image extracted in the second step j , according to the basic matrix F constructed in step 1), solve the non-coding landmark point q j The epipolar line f in the left image lj ; 4) Calculate each non-coding landmark point p in the left image extracted in the second step i To the outer polar line f solved in step 3) lj The distance d lji , if d lji Less than a given distance threshold D T , that is, d lji <D T , then the non-encoded landmark point p in the left image i Marked as the non-encoded landmark point q in the right image j The corresponding point of .

4. The method for matching non-coded landmark points in a stereo image pair according to claim 1, characterized in that: The fourth step of the isolated point elimination method is: count the number of corresponding points of the non-coding marker points in the stereo image pair solved in the third step, if the number of corresponding points of a non-coding marker point is 0, mark it as an isolated point, and remove it from the set of non-coding marker points so that it does not participate in the subsequent shape context matching, thereby eliminating its interference with the subsequent shape context matching and improving the reliability and robustness of the matching; the set of non-coding marker points in the left image after excluding the isolated points is recorded as P = {p1, p2, ..., p n1 }, n1≤n0 represents the number of non-coding landmark points remaining in the left image after excluding isolated points; the set of non-coding landmark points in the right image after excluding isolated points is recorded as Q={q1,q2,...,q m1 }, m1≤m0 represents the number of non-encoded landmark points remaining in the right image after excluding isolated points.

5. The method for matching non-coded landmark points in a stereo image pair according to claim 1, characterized in that: The calculation formula of the element in the i-th row and j-th column of the similarity measurement matrix in the sixth step is as follows: Among them, C ij represents the element in the i-th row and j-th column of the similarity matrix, 1≤i≤n1, 1≤j≤m1, n1 and m1 represent the number of non-encoded landmark points remaining in the left image and the right image after eliminating isolated points in the fourth step; C(p i ,q j ) represents the non-encoded landmark point p in the left image i and the non-encoded landmark point q in the right image j similarity measure; W is a constant weight, ranging from 0.7 to 0.9; h i (k) and h j (k) represent the non-encoded landmark point p in the left image of the stereo pair constructed in the fifth step. i and the non-encoded landmark point q in the right image j The shape context; if the non-encoded landmark point p in the left image solved in the third step i There is only one corresponding point q in the right image j , and the non-encoded landmark point q in the right image j There is also only one corresponding point p in the left image. i , then p i With q j is a one-to-one corresponding point, otherwise, p i With q j It's not a one-to-one correspondence.

6. The method for matching non-coded landmarks in a stereo image pair according to claim 1, characterized in that: The method for eliminating mismatches in the eighth step is: i With q π(i) First, calculate p according to the epipolar solution method described in step 3. i The epipolar line f in the right view ri , and q π(i) to f ri The distance d ri ; Secondly, calculate q π(i) The epipolar line f in the left view li , and p i to f li The distance d li ; Let d i =(d li +d ri ) / 2, if d i >D T , then we think that p i With q π(i) is a mismatched point pair, and this pair of matches is deleted from the optimal match π calculated in step 7; where π represents the correspondence between the non-encoded marker points in the left and right images obtained in step 7, and π(i) represents the i-th non-encoded marker point p in the left image. i Corresponding to the π(i)th non-encoded landmark point q in the right image π(i) , 1≤i≤n1, 1≤j≤m1, n1 and m1 represent the number of non-encoded landmark points remaining in the left image and the right image after eliminating isolated points in the fourth step, respectively, D T is the distance threshold described in the third step.

Images