Pseudo matching point pair removal method and device, and electronic equipment

Abstract

The application provides a pseudo matching point pair removal method and device and electronic equipment. The method comprises the following steps: determining a matching point pair between a registration image and a verification image; in the case that the number of the matching point pair exceeds a preset number threshold, generating a first minimum spanning tree according to matching points in a first image of the registration image and the verification image, and generating a second minimum spanning tree corresponding to the first minimum spanning tree according to matching points in a second image; for any edge in the first minimum spanning tree and the second minimum spanning tree, determining the error of the two edges according to the length error and the angle error of the two edges; for any matching point, determining the error of the matching point according to the error of the edge with the matching point as a vertex; and removing pseudo matching point pairs from the matching point pair according to the error of each matching point. The method can improve the feature point matching accuracy.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, apparatus and electronic device for removing pseudo-matching point pairs. Background Technology

[0002] In the field of image recognition, the common approach is to extract image feature points and use feature point matching to represent image matching to calculate the similarity between two images.

[0003] Feature point matching describes a feature point by describing its local region, generating a local feature vector. By calculating the similarity between the feature vectors of the feature points, matching feature point pairs can be determined.

[0004] However, due to factors such as differences in image acquisition equipment, changes in lighting environment, image rotation and translation distortion, and noise interference, incorrect matching feature point pairs (which can be called pseudo-matching point pairs) are easily generated during the feature matching stage, ultimately increasing the probability of misidentification in image recognition. Therefore, it is necessary to screen the matched feature point pairs and remove pseudo-matching point pairs during the feature matching stage. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus and electronic device for removing pseudo-matching point pairs.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] According to a first aspect of the embodiments of this application, a method for removing pseudo-matching point pairs is provided, comprising:

[0008] Identify matching point pairs between the registered image and the verification image;

[0009] If the number of matching point pairs exceeds a preset threshold, a first minimum spanning tree is generated based on the matching points in the first image of the registered image and the verification image, and a second minimum spanning tree corresponding to the first minimum spanning tree is generated based on the matching points in the second image; wherein, the first image is the registered image and the second image is the verification image; or, the first image is the verification image and the second image is the registered image.

[0010] For any pair of edges in the first minimum spanning tree and the second minimum spanning tree, the error between the two edges is determined based on the length error and the angle error between the two edges;

[0011] For any matching point, the error of the matching point is determined based on the error of the edge with that matching point as the vertex;

[0012] Based on the error of each matching point, pseudo-matching point pairs are removed from the matching point pairs.

[0013] According to a second aspect of the embodiments of this application, a pseudo-matching point pair removal apparatus is provided, comprising:

[0014] The first determining unit is used to determine the matching point pairs between the registered image and the verification image;

[0015] The generation unit is configured to, when the number of matching point pairs exceeds a preset threshold, generate a first minimum spanning tree based on matching points in a first image of the registered image and the verification image, and generate a second minimum spanning tree corresponding to the first minimum spanning tree based on matching points in a second image; wherein, the first image is the registered image and the second image is the verification image; or, the first image is the verification image and the second image is the registered image.

[0016] The second determining unit is used to determine the error of two edges based on the length error and angle error of any pair of edges in the first minimum spanning tree and the second minimum spanning tree; and to determine the error of any matching point based on the error of the edge with the matching point as the vertex.

[0017] The removal unit is used to remove pseudo-matching point pairs from the matching point pairs based on the error of each matching point.

[0018] According to a third aspect of the present application, an electronic device is provided, including a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method provided in the first aspect.

[0019] According to a fourth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when the machine-executable instructions are executed by a processor, the method provided in the first aspect is implemented.

[0020] The pseudo-matching point pair removal method of this application embodiment determines matching point pairs between the registered image and the verification image. When the number of matching point pairs exceeds a preset threshold, a first minimum spanning tree is generated based on the matching points of the first image in the registered image and the verification image. A second minimum spanning tree corresponding to the first minimum spanning tree is generated based on the matching points in the second image. For any pair of edges in the first minimum spanning tree and the second minimum spanning tree, the error of the two edges is determined based on the length error and angle error of the two edges. For any matching point, the error of the matching point is determined based on the error of the edge with the matching point as the vertex. Then, based on the error of each matching point, pseudo-matching point pairs are removed from the matching point pairs. By applying the spanning tree structure to the matching point screening based on the minimum spanning tree, the pseudo-matching point pairs in the matching point pairs are determined based on the overall topological structure of the feature points, which improves the accuracy of pseudo-matching point pair removal and thus improves the feature point matching accuracy. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a pseudo-matching point pair removal method according to an exemplary embodiment of this application;

[0022] Figure 2 This is a schematic diagram illustrating an initial matching point pair of a registration image and a verification image, as shown in an exemplary embodiment of this application.

[0023] Figure 3 This is a schematic diagram illustrating the angle difference between a pair of matching points, as shown in an exemplary embodiment of this application.

[0024] Figure 4 This is a schematic diagram illustrating the remaining matching point pairs after stage one filtering, as shown in an exemplary embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating a minimum spanning tree in a registration image and a verification image, as shown in an exemplary embodiment of this application;

[0026] Figure 6 This is a schematic diagram illustrating a minimum spanning tree for generating a registered image, as shown in an exemplary embodiment of this application;

[0027] Figure 7 This is a schematic diagram illustrating an adjacency matrix according to an exemplary embodiment of this application;

[0028] Figure 8 This is a schematic diagram illustrating the final minimum spanning tree in a registration image and a verification image, as shown in an exemplary embodiment of this application;

[0029] Figure 9 This is a schematic diagram illustrating a final matching point pair in a registration image and a verification image, as shown in an exemplary embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the structure of a pseudo-matching point pair removal device shown in an exemplary embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the hardware structure of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0034] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0035] Please see Figure 1 This is a flowchart illustrating a pseudo-matching point pair removal method provided in an embodiment of this application. For example, this pseudo-matching point pair removal method can be applied to systems with high image recognition accuracy requirements but low real-time requirements, such as the backend server of a security authentication system. Figure 1 As shown, the method for removing pseudo-matching point pairs may include the following steps:

[0036] Step S100: Determine the matching point pair between the registration image and the verification image.

[0037] In this embodiment of the application, during the process of image recognition of the verification image based on the registration image, for any registration image and any verification image, feature points can be extracted from the registration image and the verification image respectively, and the feature points can be described based on the local regions of the feature points to generate local feature vectors of the feature points. By calculating the similarity between the feature vectors of the feature points, the matching feature point pairs (which can be called matching point pairs) can be determined.

[0038] Step S110: If the number of matching point pairs exceeds a preset threshold, generate a first minimum spanning tree based on the matching points in the first image of the registration image and the verification image, and generate a second minimum spanning tree corresponding to the first minimum spanning tree based on the matching points in the second image.

[0039] For example, the first image is a registration image and the second image is a verification image; or, the first image is a verification image and the second image is a registration image.

[0040] For ease of understanding, the first image will be used as the registration image and the second image as the verification image in the following explanation.

[0041] In this embodiment of the application, in order to improve the reliability of the matching point pairs between the determined registration image and the verification image, the pseudo matching point pairs in the matching point pairs in step S100 can be removed based on the position and topology of the matching points in the image and the overall structural information of each matching point pair.

[0042] Furthermore, considering that the number of matching point pairs between the registered image and the verification image is too small, it may be impossible to accurately identify false matching point pairs based on the overall structural information of the matching point pairs.

[0043] Therefore, once the matching point pairs between the registration image and the verification image are determined, the number of matching point pairs can be counted. If the number of matching point pairs exceeds a preset threshold, a minimum spanning tree (referred to as the first minimum spanning tree in this paper) can be generated based on the matching points in the registration image, and a minimum spanning tree (referred to as the second minimum spanning tree in this paper) corresponding to the first minimum spanning tree can be generated based on the matching points in the verification image.

[0044] For example, for matching points in the registration image, their positions can be extracted, points can be connected pairwise, and the distance of each connection line (such as the Euclidean distance between two points) can be calculated to form an undirected graph. The Prim algorithm can then be used to calculate the minimum spanning tree of the undirected graph, resulting in the first minimum spanning tree. For matching points in the verification image, the corresponding matching points in the verification image can be connected according to the connection relationships between the matching points in the first minimum spanning tree to obtain the second minimum spanning tree corresponding to the first minimum spanning tree.

[0045] For example, suppose the first minimum spanning tree includes edge AB, and the matching point of point A in the registration image is A' in the verification image, and the matching point of point B in the registration image is B' in the verification image. Then, edge A'B' can be obtained from A' and B' in the verification image. And so on, to obtain the second minimum spanning tree corresponding to the first minimum spanning tree.

[0046] In one example, the preset quantity threshold mentioned above can be 3.

[0047] For example, in the embodiments of this application, the minimum number of point pairs that can be processed is 3. That is, when there are more than or equal to 3 pairs of matching point pairs between the extracted registration image and the verification image, the technical solution provided in the embodiments of this application can be used to remove pseudo-matching point pairs, thereby reducing the limitation on the implementation of the solution due to the number of extracted matching point pairs.

[0048] Step S120: For any pair of edges in the first minimum spanning tree and the second minimum spanning tree, determine the error between the two edges based on the length error and angle error between the two edges.

[0049] In this embodiment of the application, given the first minimum spanning tree and the second minimum spanning tree, the error of each pair of edges in the first minimum spanning tree and the second minimum spanning tree can be determined respectively.

[0050] For example, for any edge in the first minimum spanning tree, in the second minimum spanning tree, the edge whose two vertices match the points in the verification image is a pair of edges with the edge in the first minimum spanning tree.

[0051] For example, suppose the first minimum spanning tree includes edge AB, and point A in the registration image has a matching point A' in the verification image, and point B in the registration image has a matching point B' in the verification image. Then edge AB in the first minimum spanning tree and A'B' in the second minimum spanning tree are a pair of edges.

[0052] In this embodiment of the application, considering that in real-world scenarios, for any pair of edges in the first minimum spanning tree and the second minimum spanning tree, the length and angle of the pair of edges should be consistent if both vertices of the pair are correct matching points. Therefore, the existence of false matching point pairs can be determined based on the consistency of the length and angle of each pair of edges in the first minimum spanning tree and the second minimum spanning tree.

[0053] Accordingly, for any pair of edges in the first minimum spanning tree and the second minimum spanning tree, the length error of the two edges can be determined based on the length of the two edges in the pair, and the angle error of the two edges can be determined based on the angle of the two edges in the pair. The total error of the two edges can be determined based on the length error and the angle error.

[0054] It should be noted that for any pair of sides, if the errors of the two sides in that pair are the same, then the error of that pair of sides is the same as the error of the two sides in that pair.

[0055] Step S130: For any matching point, determine the error of the matching point based on the error of the edge with the matching point as the vertex.

[0056] In this embodiment of the application, given that the errors of each pair of edges have been determined, for any matching point, the error of the matching point can be determined based on the error of the edge with the matching point as the vertex.

[0057] For example, for any matching point, the average error of the edges with that matching point as the vertex can be determined as the error of that matching point, or the maximum error of the edges with that matching point as the vertex can be determined as the error of that matching point.

[0058] For example, for any pair of matching points, the errors of the two matching points in the pair are the same.

[0059] Step S140: Based on the error of each matching point, remove pseudo-matching point pairs from the matching point pairs.

[0060] In this embodiment of the application, after determining the error of each matching point in the manner described above, the false matching points in each matching point can be determined based on the error of each matching point, and the false matching point pairs can be removed based on the determined false matching points.

[0061] It should be noted that since the errors of the two matching points in the same matching point pair are the same, for any matching point pair, it is possible to determine whether the matching point is a pseudo-matching point based on the error of one of the matching points. If the matching point is determined to be a pseudo-matching point, the matching point pair can be determined to be a pseudo-matching point pair.

[0062] For example, matching points with excessively large errors (which can be determined by threshold comparison) can be identified as pseudo-matching points and pseudo-matching point pairs can be removed.

[0063] It can be seen that, in Figure 1 In the illustrated method flow, matching point pairs between the registered image and the verification image are determined. When the number of matching point pairs exceeds a preset threshold, a first minimum spanning tree is generated based on the matching points of the first image in the registered image and the verification image. A second minimum spanning tree corresponding to the first minimum spanning tree is generated based on the matching points in the second image. For any pair of edges in the first minimum spanning tree and the second minimum spanning tree, the error of the two edges is determined based on the length error and angle error of the two edges. For any matching point, the error of the matching point is determined based on the error of the edge with the matching point as the vertex. Then, based on the error of each matching point, pseudo-matching point pairs are removed from the matching point pairs. By applying the spanning tree structure to the matching point filtering based on the minimum spanning tree, the pseudo-matching point pairs in the matching point pairs are determined based on the overall topological structure of the feature points, which improves the accuracy of pseudo-matching point pair removal and thus improves the feature point matching accuracy.

[0064] In some embodiments, determining the error of the two sides based on the length error and angle error of the two sides may include:

[0065] Based on the lengths of the two sides and the scaling factor, determine the length error of the two sides; and,

[0066] Based on the angles of the two sides and the rotation angle of the second image relative to the first image, the angle error of the two sides is determined; wherein, for any side, the angle of the side is the angle between the side and the horizontal direction.

[0067] For example, considering that there may be scale transformation and rotation transformation between the registration image and the verification image in a real scenario, for any pair of edges, when determining the length error and angle error of the two edges in the pair of edges, it is necessary to consider scale and rotation invariance, that is, to ensure that the length error and angle error of the pair of edges are determined while the scale and angle of the registration image and the verification image are consistent.

[0068] Accordingly, for any pair of sides, when determining the length error of the two sides in the pair, the length error of the two sides can be determined based on the length of the two sides and the scaling factor.

[0069] For example, the scaling factor can be used to characterize the scaling ratio of the verification image relative to the registration image.

[0070] In one example, determining the length error of the two sides based on their lengths and the scaling factor could include:

[0071] The length error of the two edges is determined based on the length of the first type edge and the corrected length of the second type edge. The first type edge is the edge in the first minimum spanning tree of the two edges, and the second type edge is the edge in the second minimum spanning tree of the two edges. The corrected length of the second type edge is obtained by correcting the length of the second type edge using a scaling factor.

[0072] For example, for any pair of edges, the edges belonging to the second minimum spanning tree (i.e., the minimum spanning tree in the second image) (referred to as second-type edges in this paper) can be corrected using a scale transformation factor.

[0073] For example, multiplying the length of the second type of edge by the scaling factor yields the corrected length of the second type of edge.

[0074] The length error between the two edges can be determined by comparing the length of the edge belonging to the first minimum spanning tree (i.e., the minimum spanning tree in the first image) (referred to as the first type edge in this paper) with the corrected length of the second type edge.

[0075] For example, the absolute value of the difference between the length of the first type of edge and the corrected length of the second type of edge is determined as the length error between the two edges.

[0076] For example, for any pair of sides, the angle error between the two sides can be determined based on the angles of the two sides and the rotation angle of the verification image relative to the registered image.

[0077] For example, for any edge, the angle of that edge is the angle between that edge and the horizontal direction (such as the horizontal to the right).

[0078] For example, for side AB, the angle of side AB can be the angle between ray AB and the horizontal rightward direction.

[0079] In one example, determining the angle error of the two sides based on their angles and the rotation angle of the second image relative to the first image can include:

[0080] Determine the included angle between the two sides based on the angle difference between them;

[0081] The angle error of the two sides is determined based on the included angle between the two sides and the rotation angle of the second image relative to the first image.

[0082] For example, for any pair of sides, the angle difference between the two sides can be determined based on the angles of the two sides in the pair, and the included angle between the two sides can be determined based on the angle difference.

[0083] For example, if the included angle of two sides is set to be between 0 and 180°, then when the absolute value of the angle difference between the two sides is between 0 and 180°, the absolute value of the angle difference between the two sides can be determined as the included angle of the two sides; when the absolute value of the angle difference between the two sides is between 180° and 360°, the difference between 360° and the absolute value of the angle difference between the two sides can be determined as the included angle of the two sides.

[0084] In one example, the rotation angle of the second image relative to the first image is determined based on the angle difference between each pair of matching points between the second and first images; for any pair of matching points, the angle difference is the angle between the directions of the two matching points in the pair.

[0085] For example, the rotation angle of the verification image relative to the registration image can be determined based on the angle difference between each matching point pair between the verification image and the registration image.

[0086] For example, the average angle difference between each pair of matching points between the verification image and the registration image can be used to determine the rotation angle of the verification image relative to the registration image.

[0087] It should be noted that, in practical applications, the rotation angle of the second image relative to the first image is not limited to the angle difference between each pair of matching points between the second and first images; it can also be determined based on the included angle between each pair of opposite sides between the second and first images.

[0088] For example, the angles between all opposite edges in the first and second minimum spanning trees can be counted, such as by using a histogram to count the angles between all opposite edges, to determine the intervals where the angles of each opposite edge are located, to determine the interval with the most opposite edges, and to determine the average value of the angles between all opposite edges in that interval as the rotation angle of the second image relative to the first image.

[0089] In some embodiments, for any matching point, the error of the matching point is the maximum value of the error of the edge with the matching point as the vertex;

[0090] The above-mentioned process of removing pseudo-matching pairs based on the error of each matching point can include:

[0091] Traverse the unstable edges; where an unstable edge is an edge whose error is greater than a first error threshold.

[0092] For any unstable edge traversed, if the error of the unstable edge is greater than or equal to the second error threshold, a third minimum spanning tree is generated based on the first vertex of the unstable edge and each stable point in the set of stable points of the image to which the unstable edge belongs; and a fourth minimum spanning tree is generated based on the second vertex of the unstable edge and each stable point in the set of stable points of the image to which the unstable edge belongs; wherein, the set of stable points includes matching points with an error less than or equal to the first error threshold; and the second error threshold is greater than the first error threshold.

[0093] The error of the first vertex is determined based on the third minimum spanning tree and the corresponding fifth minimum spanning tree in another image; and the error of the second vertex is determined based on the fourth minimum spanning tree and the corresponding sixth minimum spanning tree in another image.

[0094] Based on the errors of the unstable edge, the first vertex, and the second vertex, the stability scores of the first and second vertices are updated.

[0095] Once the unstable edge traversal is complete, pseudo-matching point pairs are removed based on the stability scores of each matched point that has not been added to the stable point set.

[0096] For example, the error of the matching point can be taken as the maximum value of the error of the edge with that matching point as the vertex.

[0097] Having determined the error of each matching point in the manner described above, the error of each matching point can be compared with a preset error threshold (which can be called the first error threshold). Matching points with errors less than the first error threshold are determined as correct matching points (also called stable points) and added to the set of stable points.

[0098] Considering that points with high error values ​​are not necessarily false matching points, as the error in the edge connecting the correct matching point and the false matching point may be too large, which in turn leads to a larger error in the determined correct matching point, matching points with errors greater than the first error threshold can be further judged and filtered.

[0099] For example, since the error of a matching point is the maximum value of the error of the edge with that matching point as its vertex, if the error of a matching point is greater than the first error threshold, there is at least one edge with that matching point as its vertex whose error is greater than the first error threshold, and the error of the other vertex of that edge will also be greater than the first error threshold. Therefore, by traversing the edges whose errors are greater than the first error threshold, the vertices of the edges whose errors are greater than the first error threshold can be further filtered to determine the final pseudo-matching point.

[0100] For example, edges with errors greater than a first error threshold (referred to as unstable edges in this paper) can be traversed.

[0101] For any unstable edge that has been traversed, the error of the unstable edge can be compared with another error threshold (referred to as the second error threshold in this paper), and the second error threshold is greater than the first error threshold.

[0102] If the error of the unstable edge is greater than or equal to the second error threshold, it can be determined that there may be pseudo-matching points among the vertices of the unstable edge, and further judgment and screening are required.

[0103] In this case, for the vertices of the unstable edge (which can be referred to as the first vertex and the second vertex, respectively), a minimum spanning tree (which can be referred to as the third minimum spanning tree) can be generated based on the first vertex and each stable point in the stable point set of the image to which the unstable edge belongs; and a minimum spanning tree (which can be referred to as the fourth minimum spanning tree) can be generated based on the second vertex and each stable point in the stable set of the image to which the unstable edge belongs.

[0104] Since all points in the third minimum spanning tree except the first vertex are stable points, and all points in the fourth minimum spanning tree except the second vertex are stable points, the error in determining the first vertex based on the third minimum spanning tree can avoid the influence of the edge formed by connecting the first vertex and the pseudo-matching point on the error of the first vertex. Similarly, the error in determining the second vertex based on the fourth minimum spanning tree can also avoid the influence of the edge formed by connecting the second vertex and the pseudo-matching point on the error of the second vertex.

[0105] For example, if the third minimum spanning tree and the fourth minimum spanning tree are generated in the manner described above, the error of the first vertex can be determined based on the third minimum spanning tree and the corresponding minimum spanning tree in another image (referred to as the fifth minimum spanning tree in this document). The specific implementation method can be found in the relevant description in the above process, and will not be repeated here in the embodiments of this application.

[0106] For example, for any unstable edge, if the unstable edge belongs to the registered image and the third minimum spanning tree of the first vertex is generated in the manner described above, the error of the first vertex can be determined based on the third minimum spanning tree and the minimum spanning tree corresponding to the third minimum spanning tree in the verification image (i.e., the fifth minimum spanning tree mentioned above).

[0107] Similarly, the error of the second vertex can be determined based on the fourth minimum spanning tree and the corresponding minimum spanning tree in another image (referred to as the sixth minimum spanning tree in this paper). The specific implementation method can be found in the relevant description in the above process, and will not be repeated here in the embodiments of this application.

[0108] For example, the stability scores of the first vertex and the second vertex can be updated based on the error of the unstable edge, the error of the first vertex, and the error of the second vertex.

[0109] After traversing all unstable edges in the manner described above, pseudo-matching point pairs can be removed based on the stability scores of each matched point that has not been added to the stable point set.

[0110] In one example, updating the stability scores of the first and second vertices based on the errors of the unstable edge, the first vertex, and the second vertex can include:

[0111] If the error of the unstable edge is greater than or equal to a third error threshold, the smaller of the error of the first vertex and the error of the second vertex is compared with the second error threshold. If the smaller of the error of the first vertex and the error of the second vertex is greater than or equal to the second error threshold, a preset value is subtracted from the stability scores of the first vertex and the second vertex of the unstable edge, respectively. If the smaller of the error of the first vertex and the error of the second vertex is less than the second error threshold, the preset value is subtracted from the stability score of the larger error between the first vertex and the second vertex, and the preset value is added to the stability score of the smaller error between the first vertex and the second vertex. The third error threshold is greater than the second error threshold.

[0112] If the error of the unstable edge is greater than or equal to the second error threshold and less than the third error threshold, the stability score of the vertex with the error greater than or equal to the first error threshold is reduced by a preset value, and the stability score of the vertex with the error less than the first error threshold is increased by a preset value.

[0113] The above-mentioned removal of pseudo-matching point pairs based on the stability scores of each matched point not included in the stable point set can include:

[0114] For any matched point that is not added to the set of stable points, if the stability score of the matched point is less than or equal to the initial value, the matched point is identified as a pseudo-matched point; if the stability score of the matched point is greater than the initial value, the matched point is added to the set of stable points.

[0115] Based on the identified pseudo-matching points, pseudo-matching point pairs are removed.

[0116] For example, after determining the error of the first vertex and the error of the second vertex in the manner described above, the error of the unstable edge can be compared with another error threshold (referred to herein as the third error threshold). The third error threshold is greater than the second error threshold.

[0117] If the error of the unstable edge is greater than or equal to the third error threshold, the error of the first vertex and the error of the second vertex can be compared, the smaller one can be determined, and the smaller one can be compared with the second error threshold.

[0118] If the smaller of the error of the first vertex and the error of the second vertex is greater than or equal to the second error threshold, the probability that the first vertex and the second vertex are pseudo-matching points can be determined to be increased. In this case, a preset value can be subtracted from the stability scores of the first vertex and the second vertex, respectively.

[0119] For example, for any matching point, the lower the stability score of the matching point, the higher the probability that the matching point is a false matching point.

[0120] If the smaller of the error of the first vertex and the error of the second vertex is less than the second error threshold, a preset value can be subtracted from the stability score of the larger error between the first vertex and the second vertex, and a preset value can be added to the stability score of the smaller error between the first vertex and the second vertex.

[0121] For example, if the error of the stable edge is greater than or equal to the second error threshold and less than the third error threshold, a preset value can be subtracted from the stability score of the vertices in the first and second vertices whose errors are greater than or equal to the first error threshold, and a preset value can be added to the stability score of the vertices in the first and second vertices whose errors are less than the first error threshold.

[0122] It should be noted that for any unstable edge traversed, if the first unstable edge is less than the second error threshold, a preset value can be added to the stability score of the first vertex and the second vertex of the unstable edge, respectively.

[0123] For example, after the unstable edge traversal is completed, it is possible to determine whether each matched point that has not been added to the stable point set is a pseudo-matched point based on the instability score of each matched point.

[0124] For any matched point that is not added to the set of stable points, if the stability score of the matched point is less than or equal to the initial value, the matched point is identified as a pseudo-matched point; if the stability score of the matched point is greater than the initial value, the matched point is added to the set of stable points.

[0125] In some embodiments, determining the matching point pair between the registration image and the verification image may include:

[0126] Determine the initial matching point pairs between the registered image and the verification image;

[0127] If the number of initial matching point pairs exceeds a preset threshold, pseudo-matching point pairs are removed from the initial matching point pairs based on the angle difference between each initial matching point pair, thus obtaining matching point pairs between the registered image and the verification image.

[0128] For example, considering any pair of matching points between the registration image and the verification image, if the pair of matching points is correct and there is no rotation angle between the registration image and the verification image, the directions of the two matching points in the pair of matching points should be consistent.

[0129] Furthermore, considering that the verification image may have a certain rotation angle relative to the registration image, the angles of the two matching points in the same matching point pair in the registration image and the verification image may have a certain difference. However, since the effect of this rotation angle is consistent on all matching point pairs, the angle difference of the matching point pairs should be consistent for correct matching point pairs.

[0130] For example, assuming there are matching point pairs AA' and BB' between the registration image and the verification image, if both are correct matching point pairs, the angle between matching point pair AA' (i.e., the angle between the direction of matching point A and the direction of matching point A') and the angle between matching point pair BB' should be the same. That is, the difference between the angle between matching point pair AA' and the angle between matching point pair BB' is less than a preset angle difference.

[0131] Therefore, based on the consistency of the angle difference between matching point pairs (which can be called direction consistency), the existence of pseudo-matching point pairs can be determined by using the angle difference between matching point pairs.

[0132] Furthermore, considering that when the number of matching point pairs is too small, it may be impossible to accurately identify false matching point pairs based on the consistency of the matching point directions.

[0133] Therefore, for the matching point pairs (which can be called initial matching point pairs) in the registration image and verification image determined by feature point extraction and matching, if the number of initial matching point pairs exceeds a preset threshold (such as 3 pairs), the angle difference of each initial matching point pair can be determined separately.

[0134] For example, for any pair of matching points, the angle difference between the pair of matching points can be the angle between the directions of the two matching points, that is, the absolute value of the difference between the angles of the two matching points.

[0135] For example, the initial matching point pairs can be screened for pseudo-matching point pairs based on the angle difference between each initial matching point pair to obtain matching point pairs between the registered image and the verification image. Then, according to the method described in the above embodiments, the pseudo-matching point pairs are screened again based on the minimum spanning tree. That is, the matching point pairs after initial screening are used as input to the minimum spanning tree to remove pseudo-matching point pairs again, thereby reducing the workload of removing pseudo-matching point pairs based on the minimum spanning tree.

[0136] In one example, the above method of removing pseudo-matching point pairs based on the angle difference between each initial matching point pair can include:

[0137] Based on the angle difference of each initial matching point pair, determine the maximum and minimum values ​​of the angle difference among the initial matching point pairs;

[0138] If the difference between the maximum and minimum values ​​is greater than or equal to a preset angle difference threshold, the range between the maximum and minimum values ​​is divided into multiple intervals based on the difference between the maximum and minimum values.

[0139] Based on the interval where the angle difference of each initial matching point pair is located, determine the number of initial matching point pairs in each interval;

[0140] Based on the number of initial matching point pairs in each interval, and the interval in which the angle difference of each initial matching point pair is located, pseudo-matching point pairs are removed from the initial matching point pairs.

[0141] For example, given the angle difference between each initial matching point pair, the maximum and minimum angle differences between each initial matching point pair can be determined based on the angle difference between each initial matching point pair, and the difference between the maximum and minimum values ​​can be compared with a preset angle difference threshold.

[0142] For example, if the difference between the maximum and minimum values ​​is less than a preset angle difference threshold, it can be determined that each initial matching point pair satisfies the direction consistency. In this case, each initial matching point pair can be used as a matching point pair between the registered image and the verification image, and pseudo matching point pairs can be screened out again based on the minimum spanning tree.

[0143] If the difference between the maximum and minimum values ​​is greater than or equal to the preset angle difference threshold, the range between the maximum and minimum values ​​can be divided into multiple intervals based on the difference between the maximum and minimum values. Based on the angle difference of each initial matching point pair, the interval in which the angle difference of each initial matching point pair is located can be determined, and the number of initial matching point pairs in each interval can be counted.

[0144] Furthermore, based on the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located, pseudo-matching point pairs can be removed from the initial matching point pairs.

[0145] As an example, based on the difference between the maximum and minimum values, the range between the maximum and minimum values ​​can be divided into multiple intervals, which may include:

[0146] The number of intervals is determined based on the difference between the maximum and minimum values, and the number of initial matching point pairs;

[0147] Based on the number of intervals, the range between the maximum and minimum values ​​is divided into intervals equal to the number of intervals.

[0148] For example, to avoid having too many or too few intervals, which would prevent us from intuitively understanding the distribution of the angle difference of each initial matching point pair by the number of initial matching point pairs in different intervals, we can determine the number of intervals to be divided based on the difference between the maximum and minimum values ​​of the angle difference of the initial matching point pairs, as well as the number of initial matching point pairs. Based on the number of intervals, we can divide the range between the maximum and minimum values ​​into the number of intervals.

[0149] As an example, the above-mentioned process of removing pseudo-matching points from initial matching point pairs based on the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located can include:

[0150] Based on the number of initial matching point pairs in each interval, determine the target interval with the largest number of initial matching point pairs;

[0151] Based on the number of initial matching point pairs in the target interval, the intervals in which the number of initial matching point pairs is less than the product of the number of initial matching point pairs in the target interval and a preset ratio are defined as pseudo-matching point intervals.

[0152] Initial matching point pairs whose angle difference lies within the pseudo-matching point interval are identified as pseudo-matching point pairs and then removed.

[0153] For example, considering that in real-world scenarios, the angle differences of correct matching point pairs in the initial matching point pairs are usually quite close, and they are likely to be in the same or adjacent intervals; while for pseudo-matching point pairs, their angle differences are usually irregular and they are likely to be scattered in different intervals. Therefore, pseudo-matching point pairs can be determined based on the number of initial matching point pairs in the interval.

[0154] For example, the interval with the most initial matching point pairs can be determined based on the number of initial matching point pairs in each interval (which can be called the target interval), and the interval with the number of initial matching point pairs in the target interval that is less than the product of the number of initial matching point pairs in the target interval and a preset ratio can be determined as a pseudo-matching point interval.

[0155] For example, all intervals outside the target interval can be designated as pseudo-matching intervals (i.e., the preset ratio is 100%), or intervals with a number of pseudo-matching pairs less than 80% of the number of pseudo-matching pairs in the target interval can be designated as pseudo-matching intervals (i.e., the preset ratio is 80%).

[0156] For example, initial matching point pairs whose angle difference is located in the pseudo-matching point interval can be identified as pseudo-matching point pairs and then removed.

[0157] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, the technical solutions provided in the embodiments of this application are described below with reference to specific examples.

[0158] In this embodiment, taking the fingerprint recognition scenario as an example, the similarity between two images (i.e., the registration image and the verification image) can be calculated by extracting image feature points and using feature point matching to represent image matching.

[0159] Feature point matching describes the feature point using local regions, generating local feature vectors for the feature point. By calculating the similarity between the feature vectors of the feature points, the matching feature point pairs (i.e., the initial matching point pairs mentioned above) can be determined.

[0160] For example, the initial matching point pairs determined in the manner described above can be found in [reference needed]. Figure 2 .like Figure 2 As shown, the left image is the registration image and the right image is the verification image. To show the matching relationship, the feature points are connected in pairs with lines. It can be seen that there are many false matching lines in the image, and the feature points at both ends are not the same point on the fingerprint image.

[0161] In addition, this solution can also be used for palm print and palm vein recognition.

[0162] When matching feature points, the location, orientation, and neighborhood information of a point are typically used individually, or only the location and orientation information of a point pair. The location of all points in the image and the topological structure they form are not considered; that is, only local information is considered, and overall structural information is not taken into account.

[0163] Based on this, this application proposes a scheme to remove pseudo-matching point pairs using overall structural information, thereby further improving matching accuracy.

[0164] For example, the overall scheme may include two stages: Stage 1, removal of pseudo-matching point pairs based on directional consistency; Stage 2, removal of pseudo-matching point pairs based on minimum spanning tree.

[0165] The implementation of these two stages will be explained below.

[0166] Phase 1: Removal of False Matching Point Pairs Based on Directional Consistency

[0167] For example, for initial matching point pairs, the angle difference between each pair can be calculated, and a histogram can be used to analyze the angle differences. Matching point pairs that do not meet the direction consistency requirement can be removed. The specific implementation process is as follows:

[0168] 1.1 In the initial set of matching point pairs, count the number of initial matching point pairs. If the number of matching point pairs is less than a preset threshold T... 11 (e.g. T) 11 If the number of initial matching point pairs in the initial matching point pair set is small, the result of subsequent judgment of pseudo-matching points based on direction consistency will be unreliable, and the current process can be terminated; otherwise, the removal of pseudo-matching point pairs can continue.

[0169] 1.2 Calculate the angle difference between each initial matching point pair.

[0170] For example, such as Figure 3 As shown, the set of angle differences θ = {θ1, θ2, ... θ} for each initial matching point pair can be calculated. n}, θ∈[0, 180°], n is the number of initial matching point pairs.

[0171] 1.3 Calculate the difference between the maximum and minimum angle differences of each initial matching point pair; if the difference is less than the preset difference threshold, then each initial matching point pair is determined to meet the direction consistency requirement; otherwise, go to 1.4.

[0172] For example, the maximum value θ of the angle difference between each initial pair of matching points can be calculated. max With minimum value θ minThe difference θ range =θ max -θ min And compare θ range and preset difference threshold T 12 (e.g. T) 12 =5).

[0173] 1.4, Based on θ range θ min to θ max The range of values ​​between them is divided into N equal intervals, and the angle difference of the initial matching point pairs is statistically analyzed using the intervals as scales.

[0174] For example, the value of N is determined based on the number of initial matching point pairs num and θ. range The size is determined. For example:

[0175]

[0176] Among them, 3, 10, 36 and T in the above formula 12 All are empirical values.

[0177] 1.5. Based on the histogram statistics, if the angle difference of the initial matching point pair is located in the interval greater than 80% of the highest frequency, then the initial matching point pair satisfies the direction consistency requirement; otherwise, it does not satisfy the direction consistency requirement and should be removed.

[0178] For example, for all pairs of matching points that satisfy directional consistency, the mean θ of the angle difference between each pair of matching points is calculated. main The rotation angle of the image is calculated as follows:

[0179]

[0180] Where n is the number of matching point pairs that satisfy directional consistency.

[0181] Furthermore, considering that the angle difference between pseudo-matching point pairs may exactly satisfy the orientation consistency of the image, the matching point pairs that satisfy the orientation consistency should be further screened in the next stage.

[0182] For example, after the screening in stage one, the remaining matching point pairs can be as follows: Figure 4 As shown.

[0183] Phase 2: Removal of False Matching Pairs Based on Minimum Spanning Tree

[0184] For example, for matching point pairs that satisfy directional consistency, the topological structure of feature points can be utilized to remove pseudo-matching point pairs using a minimum spanning tree-based algorithm.

[0185] For example, for a registration image, matching points are connected pairwise to form a graph, and a minimum spanning tree of the graph is established; for a verification image, a corresponding spanning tree is generated, the schematic diagram of which can be as follows: Figure 5 As shown. The specific implementation process for performing error statistics on the two minimum spanning trees and filtering the nodes (i.e., matching points) of the trees can be as follows:

[0186] 2.1 Establish the minimum spanning tree for the registration image and the spanning tree for the verification image.

[0187] For example, for matching point pairs that satisfy directional consistency, the positions are extracted, the points are connected pairwise, and the distance of each connection line (such as the Euclidean distance between the two points) is calculated to form an undirected graph. A schematic diagram of this graph can be shown below. Figure 6 As shown, circles represent the indexes of points, and numbers on line segments represent the distances between two points. The minimum spanning tree of the undirected graph is calculated using Prim's algorithm. Figure 6 The part marked by the bold line segment is the same as... Figure 5 The image in the middle left corresponds to this.

[0188] For example, for a verification image, the corresponding points are connected to form a spanning tree according to the connection method of the minimum spanning tree generated from the registration image.

[0189] For example, in Figure 5 In the given information, the minimum spanning tree connection method for the registration image is [(1, 5), (5, 7), (1, 6), (6, 3), (6, 2), (1, 8), (7, 4)]. Therefore, the spanning tree connection method for the verification image is [(1', 5'), (5', 7'), (1', 6'), (6', 3'), (6', 2'), (1', 8'), (7', 4')]. Figure 5 The middle right image corresponds to this.

[0190] 2.2 Calculate the error value of each edge of the spanning tree.

[0191] For each edge of the spanning tree, we use length and angle to represent it, where the length is the Euclidean distance of the vertex and the angle is the angle between the edge and the horizontal axis (taking the horizontal direction to the right as an example).

[0192] Suppose that the two vertices of an edge are point 1 and point 2, with coordinates (x1, y1) for point 1 and (x2, y2) for point 2, and the length of the edge is... Angle is Where θ∈[0, 360]. Similarly, the length d′ and angle θ′ of the edges connecting points 1' and 2' in the matching image can be obtained. Where:

[0193]

[0194] For example, considering that there may be scale and rotation transformations between the registration image and the verification image, the error value needs to take into account scale and rotation invariance.

[0195] For example, the error value for length can be: loss d =Δd=|d-δ·d′|

[0196] Where δ is the scaling factor.

[0197] The angle error can be calculated in the following way:

[0198] 2.2.1 Calculate the angle Δθ between angles θ and θ′, where Δθ∈[0, 180]:

[0199]

[0200] 2.2.2 The rotation angle of the image is θ, calculated in the first stage. main .

[0201] 2.2.3, Angle error value loss θ =|Δθ-θ main |

[0202] For example, the error value for each edge can be represented using length error values ​​and angle error values. For instance, loss... d +α·loss θ .

[0203] For example, α can be 0.5.

[0204] 2.3 Calculate the error value of each point in the spanning tree.

[0205] For example, an adjacency matrix of points can be constructed, and the matrix can be filled with the error value of each edge calculated in step 2.2, wherein the matrix is ​​diagonally symmetric.

[0206] like Figure 7 As shown, based on the error value of each edge of the minimum spanning tree, we can obtain... Figure 7 The adjacency matrix shown.

[0207] For example, for the error value of each point, the maximum value of the row (or column) of the adjacency matrix containing that point can be taken.

[0208] For example, the error value of point 1 is: MAX(0, 0, 0, 0, 127.07, 119.61, 0, 41.75) = 127.07.

[0209] 2.4 Further screening and evaluation of points with high error values.

[0210] For example, points with small error values ​​as determined in 2.3, such as points with errors less than or equal to the threshold T0 (i.e., the first error threshold mentioned above), can be identified as correct matching points. However, points with high errors are not necessarily false matching points, because the error may be due to the connection between the correct matching point and the false matching point. Therefore, it is necessary to further judge and filter points with high error values.

[0211] For example, points with errors less than or equal to T0 are designated as stable points, forming a set of stable points. Each edge of the current spanning tree is then evaluated sequentially; if the error value of an edge is less than or equal to T0... If the value is greater than T0, then for the two vertices P corresponding to that edge... A and P B Further assessment is needed.

[0212] Among them, for P A and P B Two new spanning trees (i.e., the third and fourth spanning trees mentioned above) are then constructed using the points in the stable point set, and P is calculated. A and P B error value and

[0213] in accordance with To determine P, we need to make judgments based on different situations. A Whether it is a spurious match point.

[0214] For example, the judgment process can be as follows:

[0215] Two thresholds, T1 and T2 (i.e., the second and third error thresholds mentioned above), are set; where T0 < T1 < T2, for example, T0 = 11.5, T1 = 13, and T2 = 20. The initial value of the stability score is 0, and the preset value is 1.

[0216] 2.4.1 In In the case of, if Then it is P A and P B The stability score is reduced by the preset value; otherwise, it is P. A and P B The stability score with a larger error is reduced by the preset value, while the stability score with a smaller error is increased by the preset value.

[0217] 2.4.2, in In the case of P A and P B For stability scores with a mean error greater than or equal to T0, the preset value is subtracted; for stability scores with an error less than T0, the preset value is added.

[0218] 2.4.3, in In the case of P A and P B The stability scores are all added with preset values.

[0219] It is possible to traverse edges with errors exceeding T0 (i.e., the unstable edges mentioned above) and update the stability scores of the vertices of the unstable edges in the manner described above.

[0220] When the unstable edge traversal is complete, for any matching point that has not been added to the stable point set, if its stability score is greater than or equal to the initial value, the matching point is added to the stable point set; otherwise, it is treated as a pseudo-matching point and pseudo-matching point pairs are removed.

[0221] by Figure 7 For example, based on the threshold T0, the set of stable points can be determined as {2, 3, 4, 7}. The edges are then evaluated sequentially, due to the loss... (1，5) =127.07>T0, therefore, further determine the two vertices corresponding to edge (1, 5): point 1 and point 5. Build a minimum spanning tree between point 1 and the four points in the stable point set (points 2, 3, 4, and 7), and calculate loss1 = 114.55. Similarly, build a minimum spanning tree between point 5 and the four points in the stable point set, and calculate loss5 = 1.85. Based on the above determination method, subtract the preset value from the stability score of point 1, and increase the preset value from the stability score of point 5.

[0222] The algorithm described above can remove false matching points, thereby improving matching accuracy. Figure 5 For example, points 1 and 8 ultimately scored less than their initial values ​​in stability, and were therefore removed as pseudo-matching points. The final spanning tree result is as follows: Figure 8 As shown, the matching results are as follows: Figure 9 As shown.

[0223] The method provided in this application has been described above. The apparatus provided in this application is described below:

[0224] Please see Figure 10 This is a schematic diagram of a pseudo-matching point pair removal device provided in an embodiment of this application, as shown below. Figure 10 As shown, the pseudo-matching point pair removal device may include:

[0225] The first determining unit 1010 is used to determine the matching point pairs between the registered image and the verification image;

[0226] The generation unit 1020 is configured to, when the number of matching point pairs exceeds a preset threshold, generate a first minimum spanning tree based on matching points in a first image of the registered image and the verification image, and generate a second minimum spanning tree corresponding to the first minimum spanning tree based on matching points in a second image; wherein, the first image is the registered image and the second image is the verification image; or, the first image is the verification image and the second image is the registered image.

[0227] The second determining unit 1030 is used to determine the error of two edges based on the length error and angle error of any pair of edges in the first minimum spanning tree and the second minimum spanning tree; and to determine the error of any matching point based on the error of the edge with the matching point as the vertex.

[0228] The removal unit 1040 is used to remove pseudo-matching point pairs from the matching point pairs based on the error of each matching point.

[0229] In some embodiments, the second determining unit 1030 determines the error of the two sides based on the length error and angle error of the two sides, including:

[0230] Based on the lengths of the two sides and the scaling factor, determine the length error of the two sides; and,

[0231] Based on the angles of the two sides and the rotation angle of the second image relative to the first image, the angle error of the two sides is determined; wherein, for any side, the angle of the side is the angle between the side and the horizontal direction.

[0232] In some embodiments, the scaling factor is used to characterize the scaling ratio of the second image relative to the first image;

[0233] The second determining unit 1030 determines the length error of the two sides based on their lengths and the scaling factor, including:

[0234] The length error of the two edges is determined based on the length of the first type edge and the corrected length of the second type edge. The first type edge is the edge in the first minimum spanning tree of the two edges, and the second type edge is the edge in the second minimum spanning tree of the two edges. The corrected length of the second type edge is obtained by correcting the length of the second type edge using the scaling factor.

[0235] In some embodiments, the second determining unit 1030 determines the angle error of the two sides based on the angles of the two sides and the rotation angle of the second image relative to the first image, including:

[0236] Determine the included angle between the two sides based on the angle difference between them;

[0237] The angle error between the two sides is determined based on the included angle between the two sides and the rotation angle of the second image relative to the first image.

[0238] In some embodiments, the rotation angle of the second image relative to the first image is determined based on the angle difference between each pair of matching points between the second image and the first image; for any pair of matching points, the angle difference is the angle between the directions of the two matching points in the pair.

[0239] In some embodiments, for any matching point, the error of the matching point is the maximum value of the error of the edge with the matching point as the vertex;

[0240] The removal unit 1040 removes pseudo-matching point pairs from the matching point pairs based on the error of each matching point, including:

[0241] Traverse the unstable edges; wherein, the unstable edges are edges whose errors are greater than a first error threshold;

[0242] For any unstable edge traversed, if the error of the unstable edge is greater than or equal to the second error threshold, a third minimum spanning tree is generated based on the first vertex of the unstable edge and each stable point in the set of stable points of the image to which the unstable edge belongs; and a fourth minimum spanning tree is generated based on the second vertex of the unstable edge and each stable point in the set of stable points of the image to which the unstable edge belongs; wherein, the set of stable points includes matching points with an error less than or equal to the first error threshold; and the second error threshold is greater than the first error threshold.

[0243] Based on the third minimum spanning tree and the corresponding fifth minimum spanning tree in another image, the error of the first vertex is determined; and based on the fourth minimum spanning tree and the corresponding sixth minimum spanning tree in another image, the error of the second vertex is determined.

[0244] Based on the errors of the unstable edge, the first vertex, and the second vertex, the stability scores of the first and second vertices are updated.

[0245] Once the unstable edge traversal is complete, pseudo-matching point pairs are removed based on the stability scores of each matched point that has not been added to the stable point set.

[0246] In some embodiments, the removal unit 1040 updates the stability scores of the first vertex and the second vertex based on the error of the unstable edge, the error of the first vertex, and the error of the second vertex, including:

[0247] If the error of the unstable edge is greater than or equal to a third error threshold, the smaller of the error of the first vertex and the error of the second vertex is compared with the second error threshold. If the smaller of the errors of the first vertex and the second vertex is greater than or equal to the second error threshold, a preset value is subtracted from the stability scores of the first vertex and the second vertex of the unstable edge, respectively. If the smaller of the errors of the first vertex and the second vertex is less than the second error threshold, the preset value is subtracted from the stability score of the vertex with the larger error, and the preset value is added to the stability score of the vertex with the smaller error. The third error threshold is greater than the second error threshold.

[0248] If the error of the unstable edge is greater than or equal to the second error threshold and less than the third error threshold, the preset value is subtracted from the stability score of the vertex whose error is greater than or equal to the first error threshold in the first vertex and the second vertex, and the preset value is added to the stability score of the vertex whose error is less than the first error threshold in the first vertex and the second vertex.

[0249] The removal unit 1040 removes pseudo-matching point pairs based on the stability scores of each matching point that is not included in the stable point set, including:

[0250] For any matched point that is not added to the set of stable points, if the stability score of the matched point is less than or equal to the initial value, the matched point is identified as a pseudo-matched point; if the stability score of the matched point is greater than the initial value, the matched point is added to the set of stable points.

[0251] Based on the identified pseudo-matching points, pseudo-matching point pairs are removed.

[0252] In some embodiments, the removal unit 1040 is further configured to, for any unstable edge traversed, increase the stability score of the first vertex and the second vertex of the unstable edge by a preset value when the unstable edge is less than the second error threshold.

[0253] In some embodiments, the first determining unit 1010 is specifically used to determine an initial matching point pair between the registration image and the verification image;

[0254] The removal unit 1040 is further configured to remove pseudo-matching point pairs from the initial matching point pairs based on the angle difference between each initial matching point pair when the number of initial matching point pairs exceeds a preset threshold, thereby obtaining matching point pairs between the registered image and the verification image.

[0255] In some embodiments, the removal unit 1040 removes pseudo-matching point pairs from the initial matching point pairs based on the angle difference between each initial matching point pair, including:

[0256] Based on the angle difference of each initial matching point pair, determine the maximum and minimum values ​​of the angle difference among the initial matching point pairs;

[0257] If the difference between the maximum and minimum values ​​is greater than or equal to a preset angle difference threshold, the range between the maximum and minimum values ​​is divided into multiple intervals based on the difference between the maximum and minimum values.

[0258] Based on the interval where the angle difference of each initial matching point pair is located, determine the number of initial matching point pairs in each interval;

[0259] Based on the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located, pseudo-matching point pairs are removed from the initial matching point pairs.

[0260] In some embodiments, the removal unit 1040 divides the range between the maximum and minimum values ​​into multiple intervals based on the difference between the maximum and minimum values, including:

[0261] The number of intervals is determined based on the difference between the maximum and minimum values, and the number of the initial matching point pairs;

[0262] Based on the number of intervals, the range between the maximum and minimum values ​​is divided into the number of intervals.

[0263] In some embodiments, the removal unit 1040 removes pseudo-matching point pairs from the initial matching point pairs based on the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located, including:

[0264] Based on the number of initial matching point pairs in each interval, determine the target interval with the largest number of initial matching point pairs;

[0265] Based on the number of initial matching point pairs in the target interval, the intervals in which the number of initial matching point pairs is less than the product of the number of initial matching point pairs in the target interval and a preset ratio are defined as pseudo-matching point intervals.

[0266] Initial matching point pairs whose angle difference lies within the pseudo-matching point interval are identified as pseudo-matching point pairs and then removed.

[0267] In some embodiments, the preset quantity threshold is 3.

[0268] Please see Figure 11This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor 1101 and a memory 1102 storing machine-executable instructions. The processor 1101 and the memory 1102 can communicate via a system bus 1103. Furthermore, by reading and executing the machine-executable instructions in the memory 1102 corresponding to the pseudo-matching point pair removal control logic, the processor 1101 can execute the pseudo-matching point pair removal method described above.

[0269] The memory 1102 mentioned in this document can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0270] In some embodiments, a machine-readable storage medium, such as Figure 11 The memory 1102 in the machine-readable storage medium stores machine-executable instructions, which, when executed by a processor, implement the pseudo-matching point pair removal method described above. For example, the machine-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0271] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0272] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for removing pseudo matching point pairs, characterized in that, The method comprises: determining matching point pairs between a registration image and a verification image; in a case where a number of the matching point pairs exceeds a preset number threshold, generating a first minimum spanning tree according to matching points in a first image of the registration image and the verification image, and generating a second minimum spanning tree corresponding to the first minimum spanning tree according to matching points in a second image; wherein the first image is the registration image, and the second image is the verification image; or, the first image is the verification image, and the second image is the registration image; for any edge pair in the first minimum spanning tree and the second minimum spanning tree, determining an error of the edge pair according to a length error and an angle error of the two edges; for any matching point, determining an error of the matching point according to errors of edges with the matching point as a vertex; performing pseudo matching point pair removal on the matching point pairs according to the errors of the matching points; wherein the determining the error of the edge pair according to the length error and the angle error of the two edges comprises: determining the length error of the two edges according to lengths of the two edges and a scale transformation factor; and determining the angle error of the two edges according to angles of the two edges and a rotation angle of the second image relative to the first image; wherein for any edge, the angle of the edge is an included angle between the edge and a horizontal direction.

2. The method of claim 1, wherein, the scale transformation factor is used to represent a scaling ratio of the second image relative to the first image; the determining the length error of the two edges according to the lengths of the two edges and the scale transformation factor comprises: determining the length error of the two edges according to a length of a first type edge and a corrected length of a second type edge in the two edges; wherein the first type edge is an edge in the first minimum spanning tree in the two edges, the second type edge is an edge in the second minimum spanning tree in the two edges, and the corrected length of the second type edge is obtained by correcting a length of the second type edge by using the scale transformation factor.

3. The method of claim 1, wherein, the determining the angle error of the two edges according to the angles of the two edges and the rotation angle of the second image relative to the first image comprises: determining an included angle of the two edges according to an angle difference of the two edges; determining the angle error of the two edges according to the included angle of the two edges and the rotation angle of the second image relative to the first image.

4. The method of claim 1, wherein, the rotation angle of the second image relative to the first image is determined according to angle differences of matching point pairs between the second image and the first image; for any matching point pair, the angle difference of the matching point pair is an included angle between directions of two matching points in the matching point pair.

5. The method of claim 1, wherein, for any matching point, the error of the matching point is a maximum value of errors of edges with the matching point as a vertex; the performing the pseudo matching point pair removal on the matching point pairs according to the errors of the matching points comprises: traversing unstable edges; wherein the unstable edges are edges with errors greater than a first error threshold. For any unstable edge traversed, if an error of the unstable edge is less than a second error threshold, stability scores of first and second vertices of the unstable edge are increased by a preset value respectively; if the error of the unstable edge is greater than or equal to the second error threshold, a third minimum spanning tree is generated according to the first vertex of the unstable edge and each stable point in a stable point set of an image to which the unstable edge belongs, and a fourth minimum spanning tree is generated according to the second vertex of the unstable edge and each stable point in the stable point set; the stable point set comprises matching points with errors less than or equal to a first error threshold; the second error threshold is greater than the first error threshold; errors of the first and second vertices are determined according to the third minimum spanning tree and a corresponding fifth minimum spanning tree in another image, and errors of the first and second vertices are determined according to the fourth minimum spanning tree and a corresponding sixth minimum spanning tree in another image; the stability scores of the first and second vertices are updated according to the error of the unstable edge, the errors of the first and second vertices; if the unstable edge is traversed completely, pseudo matching point pairs are removed according to stability scores of matching points not added to the stable point set.

6. The method of claim 5, wherein, The updating of the stability scores of the first and second vertices according to the error of the unstable edge, the errors of the first and second vertices comprises: if the error of the unstable edge is greater than or equal to a third error threshold, a smaller one of the errors of the first and second vertices is compared with the second error threshold, if the smaller one of the errors of the first and second vertices is greater than or equal to the second error threshold, the stability scores of the first and second vertices are decreased by the preset value respectively, if the smaller one of the errors of the first and second vertices is less than the second error threshold, the stability score of the vertex with the greater error is decreased by the preset value, and the stability score of the vertex with the smaller error is increased by the preset value; the third error threshold is greater than the second error threshold; if the error of the unstable edge is greater than or equal to the second error threshold and less than the third error threshold, the stability score of the vertex with the error greater than or equal to the first error threshold is decreased by the preset value, and the stability score of the vertex with the error less than the first error threshold is increased by the preset value; The pseudo matching point pair removal according to the stability scores of the matching points not added to the stable point set comprises: for any matching point not added to the stable point set, if the stability score of the matching point is less than or equal to an initial value, the matching point is determined as a pseudo matching point; if the stability score of the matching point is greater than the initial value, the matching point is added to the stable point set; the pseudo matching point pair removal is performed according to the determined pseudo matching points.

7. The method according to any one of claims 1 to 6, characterized in that, The method comprises the following steps: determining the matching point pairs between the registration image and the verification image; determining the initial matching point pairs between the registration image and the verification image; 8. The method of claim 7, wherein, in the case that the number of the initial matching point pairs exceeds a preset number threshold, removing the pseudo matching point pairs from the initial matching point pairs according to the angle difference of each initial matching point pair, to obtain the matching point pairs between the registration image and the verification image. The method comprises the following steps: determining the maximum value and the minimum value of the angle difference of each initial matching point pair according to the angle difference of each initial matching point pair; in the case that the difference between the maximum value and the minimum value is greater than or equal to a preset angle difference threshold, dividing the value interval between the maximum value and the minimum value into multiple intervals according to the difference between the maximum value and the minimum value; determining the number of the initial matching point pairs in each interval according to the interval in which the angle difference of each initial matching point pair is located; 9. The method of claim 8, wherein, removing the pseudo matching point pairs from the initial matching point pairs according to the number of the initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located. The method comprises the following steps: determining the number of intervals according to the difference between the maximum value and the minimum value and the number of the initial matching point pairs; 10. The method of claim 8, wherein, equally dividing the value interval between the maximum value and the minimum value into intervals according to the number of intervals. The method comprises the following steps: determining the target interval with the largest number of initial matching point pairs according to the number of the initial matching point pairs in each interval; determining the pseudo matching point interval as the interval with a number of initial matching point pairs less than the product of the number of initial matching point pairs in the target interval and a preset proportion; 11. A device for removing pseudo-matching point pairs, characterized in that, determining the initial matching point pairs with the angle difference in the pseudo matching point interval as the pseudo matching point pairs and removing the pseudo matching point pairs. The method comprises the following steps: a first determining unit is configured to determine the matching point pairs between the registration image and the verification image; a generating unit is configured to, in the case that the number of the matching point pairs exceeds a preset number threshold, generate a first minimum spanning tree according to the matching points in a first image of the registration image and the verification image and generate a second minimum spanning tree corresponding to the first minimum spanning tree according to the matching points in a second image; wherein the first image is the registration image and the second image is the verification image, or the first image is the verification image and the second image is the registration image; a second determining unit is configured to, for any pair of edges in the first minimum spanning tree and the second minimum spanning tree, determine the error of the two edges according to the length error and the angle error of the two edges, and determine the error of any matching point according to the error of the edge with the matching point as the vertex. The removing unit removes the pseudo matching point pairs according to the error of each matching point. The second determining unit determines the error of the two edges according to the length error and the angle error of the two edges, comprising: determining the length error of the two edges according to the length of the two edges and a scale transformation factor; and determining the angle error of the two edges according to the angle of the two edges and a rotation angle of the second image relative to the first image; wherein for any edge, the angle of the edge is the included angle between the edge and the horizontal direction.

12. The apparatus of claim 11, wherein, The scale transformation factor is used to represent the scaling ratio of the second image relative to the first image. The second determining unit determines the length error of the two edges according to the length of the two edges and a scale transformation factor, comprising: determining the length error of the two edges according to the length of the first type edge and the corrected length of the second type edge; wherein the first type edge is the edge in the first minimum spanning tree among the two edges, the second type edge is the edge in the second minimum spanning tree among the two edges, and the corrected length of the second type edge is obtained by correcting the length of the second type edge by using the scale transformation factor; The second determining unit determines the angle error of the two edges according to the angle of the two edges and a rotation angle of the second image relative to the first image, comprising: determining the included angle of the two edges according to the angle difference of the two edges; determining the angle error of the two edges according to the included angle of the two edges and a rotation angle of the second image relative to the first image; The rotation angle of the second image relative to the first image is determined according to the angle difference of each matching point pair between the second image and the first image; for any matching point pair, the angle difference of the matching point pair is the included angle between the directions of the two matching points in the matching point pair; and / or For any matching point, the error of the matching point is the maximum value of the errors of the edges with the matching point as the vertex; The removing unit removes the pseudo matching point pairs according to the error of each matching point, comprising: traversing unstable edges; wherein the unstable edge is an edge with an error greater than a first error threshold; for any unstable edge traversed, in the case that the unstable edge is less than a second error threshold, increasing the stability score of the first vertex and the second vertex of the unstable edge by a preset value, respectively; in the case that the error of the unstable edge is greater than or equal to the second error threshold, generating a third minimum spanning tree according to the first vertex of the unstable edge and each stable point in the stable point set of the image to which the unstable edge belongs, and generating a fourth minimum spanning tree according to the second vertex of the unstable edge and each stable point in the stable point set of the image to which the unstable edge belongs; wherein the stable point set comprises matching points with an error less than or equal to the first error threshold; the second error threshold is greater than the first error threshold. determine an error of the first vertex according to the third minimum spanning tree and a fifth minimum spanning tree corresponding to another image, and determine an error of the second vertex according to the fourth minimum spanning tree and a sixth minimum spanning tree corresponding to another image; update the stability scores of the first vertex and the second vertex according to the error of the unstable edge, the error of the first vertex, and the error of the second vertex; remove the pseudo matching point pairs according to the stability scores of the matching points not added to the stable point set when the unstable edge is traversed completely; wherein the removing unit updates the stability scores of the first vertex and the second vertex according to the error of the unstable edge, the error of the first vertex, and the error of the second vertex, including: when the error of the unstable edge is greater than or equal to a third error threshold, comparing the smaller one of the error of the first vertex and the error of the second vertex with a second error threshold, when the smaller one of the error of the first vertex and the error of the second vertex is greater than or equal to the second error threshold, subtracting a preset value from the stability scores of the first vertex and the second vertex respectively, when the smaller one of the error of the first vertex and the error of the second vertex is less than the second error threshold, subtracting the preset value from the stability score of the vertex with the greater error and adding the preset value to the stability score of the vertex with the smaller error, and the third error threshold is greater than the second error threshold; when the error of the unstable edge is greater than or equal to the second error threshold and less than the third error threshold, subtracting the preset value from the stability score of the vertex with the error greater than or equal to a first error threshold and adding the preset value to the stability score of the vertex with the error less than the first error threshold; the removing unit removes the pseudo matching point pairs according to the stability scores of the matching points not added to the stable point set, including: for any matching point not added to the stable point set, when the stability score of the matching point is less than or equal to an initial value, the matching point is determined as a pseudo matching point, and when the stability score of the matching point is greater than the initial value, the matching point is added to the stable point set; the removing unit removes the pseudo matching point pairs according to the determined pseudo matching points; and / or the first determining unit is specifically configured to determine initial matching point pairs between the registration image and the verification image; the removing unit is further configured to remove pseudo matching point pairs from the initial matching point pairs according to the angle differences of the initial matching point pairs when the number of the initial matching point pairs exceeds a preset number threshold, to obtain the matching point pairs between the registration image and the verification image; wherein the removing unit removes the pseudo matching point pairs from the initial matching point pairs according to the angle differences of the initial matching point pairs, including: determining the maximum value and the minimum value of the angle differences of the initial matching point pairs according to the angle differences of the initial matching point pairs; In a case where the difference between the maximum value and the minimum value is greater than or equal to a preset angle difference threshold, the value interval between the maximum value and the minimum value is divided into multiple intervals according to the difference between the maximum value and the minimum value; The number of initial matching point pairs in each interval is determined according to the interval in which the angle difference of each initial matching point pair is located; The initial matching point pairs are removed according to the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located; The removing unit divides the value interval between the maximum value and the minimum value into multiple intervals according to the difference between the maximum value and the minimum value, and includes: The number of initial matching point pairs in each interval is determined according to the interval in which the angle difference of each initial matching point pair is located; The value interval between the maximum value and the minimum value is equally divided into intervals according to the number of intervals; The removing unit removes the initial matching point pairs according to the number of initial matching point pairs in each interval and the interval in which the angle difference of each initial matching point pair is located, and includes: The target interval with the largest number of initial matching point pairs is determined according to the number of initial matching point pairs in each interval; The interval with a number of initial matching point pairs less than the product of the number of initial matching point pairs in the target interval and a preset proportion is determined as a pseudo matching point interval according to the number of initial matching point pairs in the target interval; The initial matching point pairs with angle differences in the pseudo matching point interval are determined as pseudo matching point pairs and are removed.

13. An electronic device, comprising: The processor and the memory are included, the memory stores machine executable instructions which can be executed by the processor, and the processor is used to execute the machine executable instructions to implement the method in any one of claims 1-10.

Images