Feature matching method and device based on multi-neighborhood guided Kendall rank correlation coefficient

Through the feature matching method of Kendall's level correlation coefficient, combined with the Kendall's level correlation coefficient and multiple neighborhood strategy, the problem of mismatch under complex transformation and high outlier rate in the existing technology is solved, and efficient and reliable The image feature matching of the stick is suitable for a variety of visual tasks.

CN114937156BActive Publication Date: 2024-11-26CHINA UNIV OF GEOSCIENCES (WUHAN)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210466626.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-29
Publication Date
2024-11-26
Estimated Expiration
2042-04-29

AI Technical Summary

Technical Problem

The existing image matching method is poor in the face of complex non-rigid transformations or high outlier point rates, making it difficult to effectively remove mismatched points, and has high computational complexity, which limits its universality in multiple scene visual tasks.

Method used

The feature matching method of multi-neighborhood-guided Kendall rank correlation coefficient is adopted, and the generalized matching Kendall rank correlation coefficient, neighborhood common element guidance strategy and multiple neighborhood strategy are used to calculate the multiple neighborhood guidance Kendall rank of feature points. Correlation coefficient, set a threshold to remove wrong matching points, and improve the accuracy and robustness of image matching.

Benefits of technology

Effectively deal with high outlier rate in the image under projection transformation, non-rigid transformation, etc., improves the accuracy and efficiency of feature matching, and can handle thousands of initial matches in tens of milliseconds, which is suitable for real-time visual tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114937156B_ABST
    Figure CN114937156B_ABST
Patent Text Reader

Abstract

The present invention provides a feature matching method and device of multi-neighborhood guided Kendall rank correlation coefficient, the method comprising: obtaining two images to be matched; extracting feature points in the two images by SIFT algorithm, and establishing feature descriptors; constructing a set of initial matching sets according to the similarity of feature descriptors in the two images; filtering out disordered error matching sets from the initial matching sets by Kendall rank correlation coefficient; obtaining neighborhood common elements of corresponding feature points, and calculating the guided Kendall rank correlation coefficient of each feature point by neighborhood common element guiding strategy; calculating the guided Kendall rank correlation coefficient of multiple neighborhoods of feature points by multiple neighborhood strategies and taking the average, and then setting a threshold to obtain the final matching result. The beneficial effects of the present invention are: introducing Kendall rank correlation coefficient to remove mismatches, and designing neighborhood common element guiding strategy and multi-neighborhood strategy to improve the robustness of the algorithm, with high accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision image processing, in particular to image matching technology, and proposes a feature matching method and device for multi-neighborhood guided Kendall rank correlation coefficient. Background Art

[0002] Image feature matching refers to establishing a reliable feature correspondence between two or more images that are identical or similar. It is a basic and important problem in the field of computer vision and image processing. Improving the accuracy of image matching is of great significance to visual tasks such as image registration, SLAM (Simultaneous Localization and Mapping, SLAM), 3D reconstruction, target detection, image stitching and fusion. Taking SLAM as an example, the accuracy of the match between the query image observed by the system and the database (i.e. the currently drawn map) will directly affect the effect of the robot's visual position recognition, which is crucial to achieving precise navigation. Therefore, a robust and efficient feature matching algorithm can provide a stable foundation and guarantee for practical visual tasks.

[0003] However, images captured in real life always have some interference and changes to varying degrees, which brings great challenges to the task of image matching. For example, satellite remote sensing images often contain nonlinear radiation differences, speckle noise and rigid transformations, wide baseline images captured by multi-view stereo often have large viewpoint changes and projection distortions, and natural images captured by drones at low altitudes contain local distortions and non-rigid changes. Since local feature descriptors are ambiguous and fuzzy, they are very sensitive to data degradation and have poor robustness. When faced with images with complex changes, there will be a large number of serious mismatches in the initial matches generated based on feature descriptors (e.g., scale-invariant feature transform, SIFT), making the distribution of feature point sets more chaotic and complex. As a result, the problem of removing mismatches becomes increasingly difficult, and the complexity of the calculations inevitably increases.

[0004] In the neighborhood of feature matching, the basic idea of ​​existing methods for removing mismatches is to estimate an underlying geometric transformation model, which imposes geometric constraints on matching and can remove mismatched points that do not satisfy the transformation model. Before executing the algorithm, a transformation model needs to be predefined, which is divided into parametric models and non-parametric models. The parametric model is used to deal with the rigidity of the image, affine, homography transformation and epipolar geometry, and the non-parametric model is used to deal with the non-rigid transformation of the image. However, when the data contains multiple complex structures, the transformation model cannot be determined in advance, such as the recognition of deformable objects and dynamic scene matching. This severely limits the universality of this type of algorithm in the face of a variety of different scene vision tasks. Secondly, this method has a heavy computational burden, especially in the face of complex non-rigid transformations or high outlier rates. Summary of the invention

[0005] In order to solve the problem that the existing image matching methods cannot effectively deal with complex non-rigid transformations or high outlier rates, the technical solution adopted by the present invention is to provide a feature matching method and device based on multi-neighborhood guided Kendall rank correlation coefficient.

[0006] According to one aspect of the present invention, a feature matching method based on multi-neighborhood guided Kendall rank correlation coefficient is provided, which combines the generalized matching-oriented Kendall rank correlation coefficient, the neighborhood common element guiding strategy and the multiple neighborhood strategy, and specifically comprises the following steps:

[0007] Get two images to be matched;

[0008] Detect feature points in the two images using the SIFT algorithm and establish feature descriptors;

[0009] Constructing an initial set of matches based on the similarity of feature descriptors in the two images;

[0010] According to the definition of the generalized Kendall rank correlation coefficient, the Kendall rank correlation coefficient of each initial match is calculated, and a disordered erroneous match set is obtained by screening from the initial match set according to the Kendall rank correlation coefficient;

[0011] Obtain the neighborhood common elements of each initial matching corresponding feature point to obtain a neighborhood common element set;

[0012] Based on the set of false matches and the set of neighborhood common elements, a guided Kendall rank correlation coefficient of each feature point is calculated by a neighborhood common element guided strategy;

[0013] The guided Kendall rank correlation coefficient of multiple neighborhoods of the feature point is calculated using the multiple neighborhood strategy and the average is taken to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point.

[0014] Set the threshold δ according to The relationship with the threshold δ removes false matches and obtains the final image feature matching result.

[0015] Preferably, the step of calculating the Kendall rank correlation coefficient of each initial match according to the definition of the generalized Kendall rank correlation coefficient, and screening out-of-order erroneous match sets from the initial match sets according to the Kendall rank correlation coefficient comprises:

[0016] According to the definition of the generalized Kendall rank correlation coefficient, the Kendall rank correlation coefficient τ of each initial match is calculated i ;

[0017] Set a larger threshold ε and compare ε with τ i By comparison, we can filter out the wrong matching points with disordered neighborhood elements and obtain the disordered wrong matching set F:

[0018] F={(x i ,y i )∈S|τ i >ε,i∈[1,N]}

[0019] Among them, S represents the initial matching set, i represents the sequence number of the initial matching, (x i ,y i ) represents the i-th initial match, τ i Indicates i The Kendall rank correlation coefficient corresponding to the initial matches, N represents the size of the initial matching set.

[0020] Preferably, the step of obtaining the neighborhood common elements of each initial matching corresponding feature point to obtain a neighborhood common element set includes:

[0021] Get the neighborhood point set of each initially matched feature point and

[0022] Get and The ordered indexes of all feature points in the initial matching set and

[0023] According to the ordered index and Calculate the index Ind of the common element in the neighborhood c , and get the neighborhood common element set M:

[0024]

[0025] Among them, cis the index of the common element of the neighborhood;

[0026] M={(x i ,y i )∈S|i∈Ind c}

[0027] Among them, S represents the initial matching set, i represents the sequence number of the initial matching, (x i ,y i ) represents the i-th initial match, Ind c is the index of the common element of the neighborhood.

[0028] Preferably, the step of calculating the guided Kendall rank correlation coefficient of each feature point by a neighborhood common element guiding strategy based on the set of false matches and the set of neighborhood common elements comprises:

[0029] According to the disordered mismatch set F and the neighborhood common element set M, the intersection of F and M is calculated to obtain the cunning mismatch set

[0030]

[0031] Among them, S represents the initial matching set, i represents the sequence number of the initial matching, (x i ,y i ) represents the i-th initial match, Ind c is the index of the common element of the neighborhood;

[0032] definition Represents the mismatch set The cardinality of Where n k Neighborhood point set and Size, similarly define |M|=m, m represents the cardinality of the neighborhood common element set M, and also satisfies m≤n k ;

[0033] according to The Kendall rank correlation coefficient τ guided by the common elements of the neighborhood is calculated by g , referred to as the bootstrapped Kendall rank correlation coefficient τ g :

[0034]

[0035] Among them, n k -m represents the number of non-common elements in the neighborhood, Represents the number of mismatched points hidden in common points.

[0036] Preferably, the step of calculating the guided Kendall rank correlation coefficients of multiple neighborhoods of the feature point using a multiple neighborhood strategy and taking the average to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point includes:

[0037] For any initial match, multiple neighborhoods of different scales are constructed, and the size of each neighborhood is Where B is the number of multiple neighborhoods;

[0038] Calculate the bootstrapped Kendall rank correlation coefficient for each neighborhood And take the average to get the bootstrapped Kendall rank correlation coefficient of multiple neighborhoods for:

[0039]

[0040] Where B is the number of multiple neighborhoods, is the size of the jth neighborhood, m j is the cardinality of the neighborhood common element set of the jth neighborhood, is the mismatch set of the jth neighborhood The base number, is the bootstrapped Kendall rank correlation coefficient of the jth neighborhood, and j∈[1,B].

[0041] Preferably, the threshold δ is established according to The steps of removing false matches based on the relationship with the threshold δ and obtaining the final image feature matching result include:

[0042] Set threshold δ and add indicator variable according to The relationship with the threshold δ is the indicator variable Assignment:

[0043]

[0044] Where i represents the i-th initial match, For the i-th matching

[0045] According to the indicator variable The value of determines the correct set of matches:

[0046]

[0047] Where I is the correct matching set and N is the size of the initial matching set.

[0048] According to another aspect of the present invention, a feature matching device based on multi-neighborhood guided Kendall rank correlation coefficient is provided, comprising the following modules:

[0049] An image acquisition module, used for acquiring two images to be matched;

[0050] Feature point detection and descriptor creation module, used to detect feature points in two images through SIFT algorithm and create feature descriptors;

[0051] An initial matching set construction module, used to construct a set of initial matching sets according to the similarity of feature descriptors in two images;

[0052] A Kendall rank correlation coefficient calculation module is used to calculate the Kendall rank correlation coefficient of each initial match according to the definition of the generalized Kendall rank correlation coefficient, and to screen out-of-order erroneous matching sets from the initial matching sets according to the Kendall rank correlation coefficient;

[0053] A neighborhood common element acquisition module is used to acquire the neighborhood common elements of each initial matching corresponding feature point to obtain a neighborhood common element set;

[0054] A neighborhood common element guiding strategy module, used to calculate the guided Kendall rank correlation coefficient of each feature point through a neighborhood common element guiding strategy based on the error matching set and the neighborhood common element set;

[0055] The multiple neighborhood strategy module is used to calculate the guided Kendall rank correlation coefficient of multiple neighborhoods of each feature point using the multiple neighborhood strategy and take the average to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point

[0056] Feature matching module, used to The relationship with the threshold δ removes false matches and obtains the final image feature matching result.

[0057] The present invention starts from a new statistical perspective, and its basic basis is that the neighborhood elements of corresponding feature points will maintain level consistency in a certain attribute. First, the feature descriptor is used to construct an initial matching set, and the n k The nearest neighbors are used as neighborhood elements, and a ranking table is generated according to a certain stable attribute. Then the Kendall rank correlation coefficient between the corresponding feature point neighborhood ranking tables is calculated to judge the correlation between them, so as to distinguish and remove the wrong matching points. In order to cope with the interference of a large number of wrong matching points caused by various unfavorable factors, a neighborhood common element guidance strategy is designed to improve the robustness of the method. In addition, a multiple neighborhood strategy is used to cope with the fluctuation of accuracy caused by the initial feature point set with different densities, so as to obtain a more accurate correct matching set.

[0058] The technical solution provided by the present invention has the following beneficial effects:

[0059] (1) It can deal with the problem of high outlier rate caused by projection transformation, non-rigid transformation, rigid transformation, affine transformation, etc. that often appear in images or external noise.

[0060] (2) It can effectively deal with multimodal images and images for classification or object recognition.

[0061] (3) The present invention has linear complexity and can process thousands of initial matches within tens of milliseconds, and can be well applied to various real-time visual tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The specific effects of the present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

[0063] Figure 1 It is a flow chart of a feature matching method of multi-neighborhood guided Kendall rank correlation coefficient of the present invention;

[0064] Figure 2 : is the overall framework diagram of the multi-neighborhood guided Kendall rank correlation coefficient algorithm of the present invention, in which the * points in the third row are wrong matching points, and the · points are correct matching points;

[0065] Figure 3 Schematic diagram of four different distribution states of neighborhood points of the present invention, the dotted line represents an incorrect match, and the solid line represents a correct match;

[0066] Figure 4 is a schematic diagram of four different sorting schemes;

[0067] Figure 5 It is a schematic diagram of the experimental results of the hyperparameter setting of the present invention;

[0068] Figure 6 is the qualitative experimental result of the feature matching of the present invention, the dotted line represents an incorrect match, and the solid line represents a correct match;

[0069] Figure 7 is the quantitative experimental result of feature matching of the present invention;

[0070] Figure 8 It is a structural diagram of a feature matching device of a multi-neighborhood guided Kendall rank correlation coefficient according to the present invention. DETAILED DESCRIPTION

[0071] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0072] refer to Figure 1-2 This embodiment provides a feature matching method of multi-neighborhood guided Kendall rank correlation coefficient, which mainly includes the following steps:

[0073] S1: Get two images to be matched;

[0074] Step S1 specifically includes:

[0075] The two images to be matched must be of the same size and contain the same or similar objects or scenes to be used for matching. The image categories that can be processed by the present invention are relatively wide, taking the data set used in the experiment as an example. The method of the present invention can be applied to:

[0076] (1) Cars and Motorbikes dataset: This dataset contains two image sequences taken with a common camera, one of which is a car image and the other is a motorcycle image.

[0077] (2) CIAP dataset: This dataset contains 40 pairs of color infrared aerial photographs with a size of 700*700 taken by satellites, all of which are from the eastern part of Illinois, USA.

[0078] (3) DTU dataset: This dataset contains 130 pairs of images taken by a high-precision camera, including two indoor scenes (a frustum and a house model);

[0079] (4) Multimodality_image dataset: This dataset contains 52 image pairs, covering all typical multimodal image pairs in the fields of medicine, remote sensing, and computer vision. The devices used to collect images are also diverse.

[0080] (5) SAR dataset: This dataset contains 67 image pairs captured by synthetic aperture radar over Nantong City, Jiangsu Province, China;

[0081] (6) SUIRD dataset: This dataset contains 60 pairs of low-altitude remote sensing images captured by small drones;

[0082] (8) UAV dataset: This dataset contains 35 pairs of color images captured by a drone over a farmland.

[0083] S2: Detect feature points in the two images using the SIFT algorithm and establish feature descriptors;

[0084] Step S2 obtains the feature descriptor through the SIFT algorithm, which can be briefly summarized as the following process:

[0085] (1) First, construct the scale space of the image and use the scale space filter to detect extreme points to obtain scale invariance;

[0086] (2) Use the Taylor series expansion of the scale space to obtain the exact position of the extreme value. If the gray value of the extreme point is less than the threshold (usually 0.03 or 0.04), it can be removed. Through this step, the key points with low contrast and boundaries are removed, and accurate feature points are filtered and accurately located;

[0087] (3) Assign scale values ​​and orientation values ​​to the feature points, and obtain an orientation reference based on the neighborhood structure of the Gaussian scale image where the detected key points are located.

[0088] S3: construct an initial set of matches based on the similarity of feature descriptors in the two images;

[0089] Through the S2 step, each key point is assigned position, scale and direction information. Then the corresponding descriptor vector is generated on the Gaussian scale image where the key point is located. A descriptor is established for each key point. The descriptor is both distinguishable and invariant to certain variables, such as illumination, perspective, etc. Moreover, the descriptor not only contains the key point, but also includes the pixels around the key point that contribute to it. The main idea is to divide the image area around the key point into blocks, calculate the gradient histogram in the block, generate a feature vector, abstract the image information, and then obtain the descriptor. Finally, the initial matching set is obtained based on the similarity of the descriptors.

[0090] S4: according to the definition of the generalized Kendall rank correlation coefficient, the Kendall rank correlation coefficient of each initial match is calculated, and a disordered erroneous match set is obtained by screening from the initial match set according to the Kendall rank correlation coefficient;

[0091] Step S4 specifically includes:

[0092] (1) Kendall rank correlation coefficient

[0093] In statistics, the Kendall rank correlation coefficient is a nonparametric statistic used to measure the degree of ordinal association between two observations. In other words, it measures the degree of similarity between the two sets of sorted lists.

[0094] First, after obtaining the local neighborhood of each feature point, sort the neighborhood elements according to a certain sorting standard to obtain the corresponding sorting table. Suppose there is a pair of corresponding feature points x and y, and the sorting tables of their neighborhoods are σ x and σ y , their sizes are both n k For σ x and σ y The elements in (x i ,x j ) and (yi ,y j ), where i<j, if their rankings are the same, then (x i ,y i ) and (x j ,y j ) are consistent. Otherwise they are inconsistent. In this way, we can get the traditional definition of Kendall's rank correlation coefficient:

[0095]

[0096] Where n c is the number of identical pairs, n d is the number of non-identical pairs. According to the need of feature matching, the present invention improves the traditional τ original After processing, the variant definition of Kendall rank correlation coefficient was obtained:

[0097]

[0098] However, for convenience, it is still called Kendall rank correlation coefficient in the following.

[0099] In order to more simply find the permutation σ x and σ y The variant Kendall rank correlation coefficient between them can be calculated first to get the Kendall tau distance between them. Kendall tau distance is a metric that calculates the number of pairwise inconsistencies between two sorted lists. The larger the distance, the greater the difference between the two sorted lists.

[0100] Sorting table σ x and σ y The inverse number k of elements i and j in i,j (σ x ,σ y ) is defined as:

[0101] k i,j (σ x ,σ y )=|{(i,j):i<j,[σ x (i)<σ x (j)∩σ y (i)>σ y (j)]∪[σ x (i)>σ x (j)∩σ y (i)<σ y (j)]}| (3)

[0102] Among them, σ x (i) and σ y(i) are the index numbers of element i in the two sorted lists.

[0103] It is known that the Kendall tau distance is the sum of the inversion numbers between two arrangements, so the Kendall tau distance K d (σ x ,σ y ) can be defined as:

[0104]

[0105] Where U is σ x and σ y The set of random pairs of different elements in . If the two sorted lists are exactly the same, then K d (σ x ,σ y )=0; if one of the sorting tables is completely opposite to the other, then

[0106] According to the Kendall tau distance, the arrangement σ can be obtained x and σ y The Kendall rank correlation coefficient τ(σ x ,σ y )for:

[0107]

[0108] Where n k is the size of the two sequences. It can be seen that the Kendall rank correlation coefficient is actually the normalized Kendall tau distance, so τ∈[0,1].

[0109] However, the validity of formula (5) requires a condition: the sorting table σ x and σ y The same elements have the same number, but the order of arrangement may be different. However, due to the non-rigid transformation of the image or the influence of external noise, a large number of mismatched points will be generated. Due to the existence of outliers, the n in the neighborhood of feature points x and y will be k The elements are not necessarily exactly the same, there will always be some non-common elements. Therefore, it is necessary to generalize the Kendall rank correlation coefficient.

[0110] Define the set M(σ x ,σ y ), Z(σ x ,σ y ), T(σ x ,σ y ) are:

[0111] M(σ x ,σ y )=σx ∩σ y

[0112] Z(σ x ,σ y )=σ x \σ y

[0113] T(σ x ,σ y )=σ y \σ x (6)

[0114] Among them, the set M(σ x ,σ y ) represents σ x and σ y The common element set in x and m∈σ y ; Set Z(σ x ,σ y ) indicates that it belongs to σ x But not σ y The element The set T(σ x ,σ y ) indicates that it belongs to σ y But not σ x The element of y and

[0115] Next, the set M(σ x ,σ y ), Z(σ x ,σ y ), T(σ x ,σ y ) are referred to as M, Z and T, and the sorting table σ x and σ y There are four cases in , according to which the generalized Kendall rank correlation coefficient is defined as:

[0116]

[0117] These four formulas correspond to Case 1, Case 2, Case 3, and Case 4 from top to bottom.

[0118] In order to distinguish between correct matches and incorrect matches, theoretically, the value of τ will be larger for incorrect matches and smaller for correct matches. Figure 3 As shown in (a), the neighborhood of the correct match (x, y) is also a correct match, and their arrangement is exactly the same (note that Figure 3The sorting criteria for all images in the image are Euclidean distance). According to the formula of case 1, the Kendall rank correlation coefficient τ = 0. However, from the solution formula in case 2, it can be seen that if a wrong match (i', j') in the neighborhood of the correct match (x, y) is in the sorting table σ x or y When the sequence number is smaller, this may return a larger τ, causing it to be judged as a wrong match. Figure 3 As shown in (b), there is a wrong match with a relatively high ranking in the neighborhood of the correct match (x, y) (the sequence number is 3 in the left figure and the sequence number is 0 in the right figure). According to the calculation formula of case 2, τ = 0.467 can be calculated. Figure 3 (c) shows the wrong match hidden in the correct match. Its neighborhood is all correct matches. According to the calculation formula of case 1, we can calculate τ = 0.4. If we change Figure 3 Comparing the τ values ​​of (b) and (c), it is easy to mistake them for correct matches. It can be seen that the calculation formulas for τ in cases 1, 2, and 3 all contain the term (n k -n m ), which shows that the Kendall rank correlation coefficient τ is sensitive to the number of non-common items in the arrangement. Figure 3 As shown in (c), half of the correctly matched neighborhoods have incorrect matches (non-common items). At this time, according to the formula in case 3, τ = 0.3 can be calculated, which is the same as Figure 3 Compared with the τ value in (c), it does not affect the judgment of it as a correct match. However, when the number of non-public items is too large, for example, if n m =4, then τ = 0.533, and misjudgment will occur. Figure 3 The following conclusions can be drawn from the values ​​of τ for the four cases in : (1) If there are too many outliers or wrong matches in the neighborhood of an inlier, the number of non-common items will increase, which can easily lead to misjudgment; (2) When the outlier is hidden inside the inlier, it is also easy to cause misjudgment. (3) Because the arrangements taken from the matching (x, y) neighborhood come from two images respectively, i and j cannot appear in the same sequence, so the situation described in case 4 does not exist and can be ignored.

[0119] In actual matching tasks, images may be affected by environmental noise or undergo non-rigid transformations, so there will inevitably be a large number of outliers and false matches in the neighborhood arrangement of feature points, but the arrangement order of correct matches in the neighborhood remains basically unchanged in the two images. Therefore, the following assumptions are made: (1) The correctly matched feature points in image I1 will retain a certain spatial arrangement order in image I2; (2) The spatial arrangement of incorrect matches is random and irregular, and is also randomly distributed; (3) The arrangement of correct matches in the neighborhood of feature points is uniformly distributed. Based on the above analysis and assumptions, the solution formula (5) for case 2 is unreasonable, but the formula for case 3 has good robustness to a certain number of false matches in the neighborhood. Therefore, in order to better apply it to feature matching tasks, the formula is modified:

[0120]

[0121] According to the modified formula (8), we can calculate Figure 3 The τ values ​​of (a), (b), (c) and (d) are 0, 0.033, 0.4 and 0.3 respectively. It can be seen that the new Kendall rank correlation coefficient formula (8) can be more robust to outliers and mismatches and can be better used for matching tasks.

[0122] (2) Sorting criteria scheme

[0123] The modified Kendall rank correlation coefficient formula has been obtained to measure the difference between the two sorting tables. The greater the difference, the greater the τ value. According to assumptions (1) and (2), the correct matches will retain a certain spatial order, while the mismatches and outliers are randomly distributed. In order to better distinguish between correct matches and mismatches, the current goal is to find a more robust sorting criterion, so as to find the sorting table that best reflects the geometric information of the inlier neighborhood, so as to more effectively avoid the influence of noise and non-rigid transformation.

[0124] Four sorting criteria are proposed for selection, and the experimental results of different sorting criteria are compared. For details, see the experimental section. The principle of each scheme is as follows:

[0125] Solution 1: Sort by Euclidean distance, such as Figure 4 As shown in (a). Assuming there is a match (x, y), first calculate the neighborhood of the corresponding points x and y, the size of the neighborhood is n k , and then calculate the Euclidean distances from the neighboring points of x and y to themselves. Sort the neighboring points according to the size of the Euclidean distance, and get the sorting tables of x and y as σ x and σ y Finally, according to formula (8), we can get σ xand σ y The Kendall rank correlation coefficient τ(σ x ,σ y ).

[0126] Solution 2: Introduce neighborhood points into polar coordinates and add angle information, such as Figure 4 (b) Assume there is a match (x, y), now take feature point x as an example. First, get the neighborhood point set N of feature point x x , then establish a polar coordinate system with x as the pole, taking x and N x The line connecting the average coordinates of all points is used as the polar axis. Each neighborhood point can be represented by polar coordinates (ρ, θ), where ρ represents the polar radius of the neighborhood point and θ represents the polar angle of the neighborhood point.

[0127] We can use the neighborhood point x i and x to represent ρ, using The angle between the line connecting x and the polar axis is expressed as θ. First, the neighborhood point x is calculated based on the polar radius ρ. i Sort to get a sorted table Similarly, we can get the sorting table of feature points y According to formula (9), we can get and Kendall rank correlation coefficient τ ρ :

[0128]

[0129] Then sort the neighborhood points of x and y according to the polar angle θ to get the sorting table and Similarly, the Kendall rank correlation coefficient τ between them is calculated θ :

[0130]

[0131] Finally, take τ ρ and τ θ The final Kendall rank correlation coefficient τ is taken as the average value between:

[0132]

[0133] Solution 3: Establish the polar coordinate system of the corresponding feature points x and y in the same way as Solution 2, and obtain the coordinates (ρ, θ) of each neighborhood point, such as Figure 4 (c) However, here, the cosine φ of the polar diameter ρ is used as the sorting criterion, that is:

[0134] φ=ρ*cos(θ) (12)

[0135] The sorting tables obtained according to φ are and Finally, the Kendall rank correlation coefficient between them was calculated

[0136] Solution 4: In Solution 4, the concept of neighborhood vectors is introduced, and the sorting is performed by constructing vector differences. Figure 4 As shown in (d), taking the feature point x and its neighborhood as an example, assuming that the coordinates of the feature point x are (x 0 ,y 0 ), take any point a in the neighborhood 1 (x 1 ,y 1 ). Start with x, a 1 Make vector xa for the tail 1 , with x a 1 is the base vector, where xa 1 The coordinates of (x 1 -x 0 ,y 1 -y 0 ). Similarly, the neighborhood vectors xa of other neighborhood points can also be constructed. i (x i -x 0 ,y i -y 0 ), and obtain the neighborhood vector xa i and the reference vector xa 1 The magnitude of the vector difference between :

[0137]

[0138] Then, according to the modulus of this vector difference, all neighborhood points can be sorted to obtain a sorted table Similarly, we can also get the neighborhood point sorting table of feature point y Finally, using formula (8), we can get the sorting table and Kendall rank correlation coefficient

[0139] Through experimental testing, it was decided to adopt Scheme 4 as the best sorting criterion.

[0140] (3) Obtaining the out-of-order error matching set

[0141] For images captured with a wide baseline, or even images that are affected by noise or undergo non-rigid deformation, a large proportion and a large number of mismatches and outliers will be uniformly and randomly distributed in the initial matching. In the previous analysis, it can be seen that when the neighborhood of a feature point contains an excessive number of non-common points, the Kendall rank correlation coefficient calculated according to formula (9) can easily judge the feature point as a mismatch point. In order to eliminate the adverse effects of these non-common points, a neighborhood element preservation strategy is introduced to guide the Kendall rank correlation coefficient.

[0142] First, we can use the Kendall rank correlation coefficient τ to preliminarily identify the mismatches in the initial matching set S. It is known that the τ value of mismatches is larger. In this way, we can get the mismatch set F:

[0143]

[0144] in Represent the corresponding points x i ,y i The size of the initial matching set S is N. Define |F|=f to represent the cardinality of F, and f≤N. In order to avoid mistakenly deleting correct matches with a large number of mismatches in some neighborhoods, a larger threshold ε is set to retain more correct matches.

[0145] S5: Obtain the neighborhood common elements corresponding to the feature points to obtain a neighborhood common element set;

[0146] Step S5 specifically includes:

[0147] The KNN (K-Nearest Neighbor, KNN) structure of the feature points of the image is very stable, that is, the neighborhood elements of the feature point x of the image I1 will be retained in the neighborhood of its corresponding point y in the image I2. Assume that x i ,y i ∈S, their neighborhood point sets are marked as and Assumptions and The indices of all feature points in the ordered initial matching set S are and Then the index of the common element in the neighborhood is Ind c It can be defined as:

[0148]

[0149] According to formula (15), we can determine and The common element subset M in is:

[0150] M={(x i,y i )∈S|i∈Ind c} (16)

[0151] In the calculation of Kendall's rank correlation coefficient, M is actually the common element set M (σ x ,σ y ). Define |M|=m to represent the cardinality of M, and m≤n k .

[0152] S6: The guided Kendall rank correlation coefficient of each feature point is calculated by the neighborhood common element guidance strategy;

[0153] Step S6 specifically includes:

[0154] There are still several mismatched points hidden in the set M, which also exist in the neighborhood N x and N y However, their neighborhood rankings are usually very different, so these mismatches can be easily found through the mismatch set F. The set of these cunning mismatches is defined as

[0155]

[0156] definition represent The cardinality of

[0157] Based on the characteristics of maintaining the neighborhood structure of feature points and the orderliness of neighborhood elements, the neighborhood common element guided Kendall rank correlation coefficient τ is defined, which is referred to as the guided Kendall rank correlation coefficient τ. g :

[0158]

[0159] Among them, the first two terms in the above formula (n k -m) represents the number of non-public points, the third item represents the number of mismatched points hidden in common points. It not only penalizes the mismatched points that are not common to the neighborhood, but also penalizes the cunning mismatched points that are randomly and disorderly distributed and hidden inside the correct matching points. In addition, The item avoids misjudgment of correct matches because the correct matches The value is very small. g The range of variation is τ g ∈[0,1]. The best case is to match (x i ,y i ) and the matches in its neighborhood are all correct matches, and then τ g=0; the worst case is And τ g =1. It can be seen that the match (x i ,y i ) g The smaller the value, the more likely it is a correct match. You can adjust ε in formula (14) to control The weight of , thereby controlling its importance in the formula.

[0160] S7: Use the multiple neighborhood strategy to calculate the guided Kendall rank correlation coefficient of multiple neighborhoods of the feature point and take the average to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point

[0161] Step S7 specifically includes:

[0162] In the previous analysis, only a size of n is constructed for each matching point. k The neighborhood of , and then get the sorting table according to the corresponding sorting criteria, and finally solve the guided Kendall rank correlation coefficient τ g However, experiments on real datasets have found that the distribution and mismatch rate of mismatch points vary in images of different types and scenes. Some images have dense distribution of mismatch points, while others have sparse distribution. k The optimal value of will vary with different images, so n k The choice of n will have a great impact on the experimental results. However, there is no fixed n k It will adapt to all matching scenarios. In order to avoid k The uncertainty of the experimental results can reduce the uncontrollable problems caused by the uncertainty of the experimental results. k sensitivity.

[0163] The multiple neighborhood strategy is to construct a sorted table of multiple neighborhoods of different scales (assuming the number of neighborhoods is B), and calculate multiple corresponding guided Kendall rank correlation coefficients, and finally find the average. Define the guided Kendall rank correlation coefficient of multiple neighborhoods for:

[0164]

[0165] Where B is the number of neighborhoods of different scales, is the size of the jth neighborhood, and similarly m j , and They are also the m, and τ g value, j∈[1,B].

[0166] S8: Set a threshold δ, based on The relationship with the threshold δ removes false matches and obtains the final image feature matching result.

[0167] Step S8 specifically includes:

[0168] The threshold δ is set, and the criteria for distinguishing between false matches and correct matches are:

[0169]

[0170] The correct matching set I is redefined as:

[0171]

[0172] B and The choice of is directly related to the time cost of the method, B and The larger the value, the more time it takes. In actual matching tasks, in order to strike a balance between time and effectiveness, it is necessary to select the best B and

[0173] In order to test the performance and efficiency of the method, comparative experiments were carried out on 8 datasets, namely Cars and Motorbikes, CIAP, Corresp_Dataset, DTU, Multimodality_image, SAR, SUIRD and UAV.

[0174] Except for the Corresp_Dataset dataset, 30 pairs of images were randomly selected for the experiments in the other datasets. All the images in the Corresp_Dataset dataset were used because it only has 24 pairs of images. From top to bottom, Figure 7 The first column represents the cumulative distribution curve of the inlier rate of the data used in this experiment in the Cars and Motorbikes, CIAP, Corresp_Dataset, DTU, Multimodality_image, SAR, SUIRD and UAV datasets. These datasets include projection transformation (UAV), non-rigid transformation (SUIRD, DTU), rigid transformation (SAR, CIAP), affine transformation (Corresp_Dataset), multimodal images (Multimodality_image) and images of objects of the same category (Cars and Motorbikes), which have certain matching difficulties.

[0175] (1) Parameter setting experiment

[0176] It is known that the algorithm of the present invention has four hyperparameters: B, ε and δ. Their meanings have been explained in S7 and S8 above and will not be repeated here. In order to ensure the universality of the parameters, 5 groups of image pairs are randomly selected from the 8 data sets to form a mixed image data set of size 40 for experiments. The average inlier rate of the mixed data set is 40.45%, which is difficult and representative.

[0177] This experiment and the following experiments will use three commonly used evaluation indicators: precision (P), recall (R) and F-score (F).

[0178] The definitions of precision and recall are:

[0179]

[0180] Where TP, FP and FN are the number of true positives, false positives and false negatives respectively. Precision refers to the ratio of correct matches to all matches identified by the algorithm. Recall refers to the ratio of correct matches identified by the algorithm to all correct matches.

[0181] F-score is a comprehensive evaluation of precision and recall, and is defined as:

[0182]

[0183] According to the definition of ε and δ, the smaller ε and δ are, the higher the precision and the lower the recall, and vice versa. First, experiments are conducted on the mixed data set to find the best parameters ε and δ. The default values ​​B = 3 and Set ε,δ∈[0.4,0.5,0.6,0.7,0.8], that is, use 25 different sets of [ε,δ] to obtain the F-score, and the results are as follows Figure 5 As shown in (a). The results show that when ε=0.7 and δ=0.6, the optimal F-score can be obtained. Therefore, in the subsequent experiments, [ε,δ]=[0.7,0.6] is set.

[0184] Next, we fix [ε,δ] = [0.7,0.6] and use 6 different We conducted experiments to determine the best combination of multiple neighborhoods. We collected the accuracy, recall, F-score, and running time of each group of experiments for comparison. The results are as follows: Figure 5 As shown in (b), we can know from the results that The best F-score can be obtained. Based on the above experimental results, the default algorithm parameters are set to [ε,δ] = [0.7, 0.6].

[0185] (2) Feature matching qualitative test

[0186] In order to visualize the performance of the present invention, a pair of representative images are selected from each of the eight datasets for display. Due to the diversity of the datasets, the results can intuitively reflect the universality of the method to a certain extent. Figure 6 As shown, the image pairs in the first row come from the datasets Cars and Motorbikes, CIAP, Corresp_Dataset, and DTU from left to right, and the image pairs in the second row come from the datasets Multimodality_image, SAR, SUIRD, and UAV from left to right. Figure 6 Each pair of images in the figure intuitively shows the correspondence between feature points. The two points connected by a straight line are the corresponding points determined by the algorithm. For visualization, each pair of images only shows 100 random initial matches, where the blue lines represent true positive correspondences, the red lines represent false positive correspondences, and the green lines represent false negative correspondences. There is no straight line to show true negative correspondences.

[0187] From left to right and from top to bottom, the inlier rates of the selected 8 pairs of images are 0.52308, 0.78696, 0.50737, 0.40108, 0.3072, 0.97753, 0.78109 and 0.43005 respectively. The precision, recall and F-score (precision, recall, fscore) obtained by the method of the present invention for these 8 pairs of images after removing the mismatches are (91.667%, 97.059%, 94.286%), (100%, 100%, 100%), (98.468%, 97.787%, 98.1263%), (98.6577%, 99.3243%, 98.9899%), (97.4409%, 98.3118%, 97.8744%), (99.7135%, 100%, 99.8565%), (99.1903%, 100%, 99.5935%) and (98.9744%, 99.6558%, 99.3139%). It can be seen from the qualitative experimental results that the method of the present invention has excellent effects in all data sets. Moreover, even for images with low inlier rates caused by non-rigid changes and noise, the method of the present invention can still maintain high precision and recall rate. This shows that the method of the present invention can effectively deal with images of various scenes and different sizes of inlier rates, and has robustness and universality.

[0188] (3) Feature matching quantitative experiment

[0189] In order to quantitatively evaluate the feature matching performance of the method of the present invention, a comparative test was conducted directly with 6 other advanced algorithms in the 8 data sets mentioned above and the experimental data were recorded. Other algorithms involved in the comparison include EAS (Efficient Approximate Search, EAS), GLOF (Guided Local Outlier Factor, GLOF), LAP (Local Affine Preservation, LAP), LLT (Locally Linear Transforming, LLT), LPM (Locality Preserving Matching, LPM) and mTopKRP (multiscale Top K Rank Preservation, mTopKRP). All the compared algorithms are from the official publications of the original authors, belong to the prior art, and the parameters of all algorithms are set according to the prompts of the original text and remain unchanged throughout the experiment.

[0190] The quantitative experimental results are as follows Figure 7 As shown in the figure, from top to bottom, each row is the results of Cars and Motorbikes, CIAP, Corresp_Dataset, DTU, Multimodality_image, SAR, SUIRD and UAV datasets. From left to right, each column is the cumulative distribution curve of the internal point rate of each dataset, the cumulative distribution curve of the precision, recall rate, F-score and running time of all algorithms. From top to bottom, the average inlier rate and average initial match number of these 8 datasets are Cars and Motorbikes (40.06%, 88.1), CIAP (40.9%, 1582.4), Corresp_Dataset (32.26%, 3739.13), DTU (63.23%, 758.37), Multimodality_image (22.83%, 2359.93), SAR (59.04%, 1325.8), SUIRD (60.08, 1151.5), and UAV (53.81%, 863.43).

[0191] The images in the SAR and CIAP datasets mainly undergo simple rigid transformations such as translation and rotation, but the CIAP dataset has a lower inlier rate, and the images in the SAR dataset contain serious nonlinear radiation differences and speckle noise, so they are still somewhat difficult. Achieving matching of these images will benefit many remote sensing applications, such as image stitching and change detection. All methods can generate relatively accurate matches in these two datasets because the rigid transformation is relatively simple. The EAS algorithm has low precision and recall rates for some images in the CIAP dataset, because their low inlier rate causes the EAS loss function model to be severely degraded. Similarly, for these low-overlap or low-quality images, the accuracy of the LPM and LLT algorithms has decreased. The LAP algorithm has a low recall rate in both datasets, probably because the topological structure of the neighborhood becomes chaotic due to noise pollution and low-quality images, and many correct matches are mistakenly removed.

[0192] The SUIRD dataset contains extreme viewpoint changes and non-rigid transformations. The SUIRD dataset contains many exterior images such as roads, buildings, and rivers. For this dataset, except for the EAS and LAP algorithms, other algorithms have good F-scores. This is because the local geometric distortion and distortion in the image lead to serious outliers. The estimated geometric model established by the EAS algorithm and the local topological constraints of the LAP algorithm have been disturbed to varying degrees.

[0193] Both the DTU and UAV datasets have obvious projection distortions due to viewpoint changes. The DTU dataset includes wide baseline images of two objects (i.e., Frustum and House) taken from multiple locations, while the UAV dataset is a color farmland image taken by a drone. Figure 7 It can be seen that the accuracy of LPM in the UAV dataset is lower than that of other methods. This is because LPM only considers the common elements in the neighborhood of feature points, but ignores the specific position differences between adjacent elements, which is no longer effective when facing severe projection distortion. The EAS algorithm has good accuracy for wide baseline images such as DTU, but the recall rate is relatively low.

[0194] The images in the Corresp_Dataset dataset mainly undergo simple affine transformations, but the inlier rate is very low, only 32.26%, so there is a certain degree of matching difficulty. It can be seen that the indicators of all methods in this dataset have been reduced. However, the decline of the EAS algorithm is more serious because it is difficult to estimate the accurate transformation model at a very large outlier rate.

[0195] The images in the Multimodality_image dataset come from specific multimodal scenarios in different research fields such as medicine, remote sensing, and computer vision. There are significant modal or nonlinear appearance differences between images of different modalities. Due to the nonlinear modal variance and geometric deformation in the image, the performance of the LPM and local density-based GLOF algorithms have both declined significantly.

[0196] The Cars and Motorbikes dataset contains images of two different categories, cars and motorcycles. It is a difficult dataset that contains images of different objects of the same category for target recognition and classification. The only features that are matched in this dataset are the same. The LAP algorithm has the highest accuracy in this dataset because their local neighborhood topology is relatively stable for objects of the same category. Better matching should more comprehensively cluster images showing objects of the same category, while separating more object images from different categories. For image classification and target recognition, a higher recall rate is more meaningful, so the method of the present invention is obviously more suitable.

[0197] The algorithm of the present invention and mTopKRP have good performance in all data sets, while the indicators of EAS and LAP algorithms are not very good due to the limitations of their own principles. The performance of LLT, LPM and GLOF algorithms will decline in some data sets. LLT requires data priors, its model must be set manually, and has cubic complexity, which is very time-consuming and not suitable for real-time tasks. In many tasks of computer vision, the processing time of feature matching is as important as performance. In terms of running time, the common element guidance strategy of multiple neighborhoods of the method of the present invention has a faster speed than the iterative strategy of mTopKRP. Although LPM has the fastest speed, the method of the present invention obviously achieves the best trade-off between precision-recall and time.

[0198] refer to Figure 8 In some embodiments, a multi-neighborhood guided Kendall rank correlation coefficient feature matching device is also provided, comprising the following modules:

[0199] Image acquisition module 1, used to acquire two images to be matched;

[0200] The feature point detection and descriptor establishment module 2 is used to detect feature points in two images through SIFT algorithm and establish feature descriptors;

[0201] An initial matching set construction module 3, used for constructing a set of initial matching sets according to the similarity of feature descriptors in two images;

[0202] Kendall rank correlation coefficient calculation module 4, used for calculating the Kendall rank correlation coefficient of each initial match according to the definition of the generalized Kendall rank correlation coefficient, and screening out-of-order erroneous matching sets from the initial matching sets according to the Kendall rank correlation coefficient;

[0203] A neighborhood common element acquisition module 5 is used to acquire the neighborhood common elements of each initial matching corresponding feature point to obtain a neighborhood common element set;

[0204] A neighborhood common element guiding strategy module 6 is configured to calculate a guided Kendall rank correlation coefficient of each feature point through a neighborhood common element guiding strategy based on the error matching set and the neighborhood common element set;

[0205] Multiple neighborhood strategy module 7, used to calculate the guided Kendall rank correlation coefficient of multiple neighborhoods of each feature point using the multiple neighborhood strategy and take the average to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point

[0206] Feature matching module 8, used according to The relationship with the threshold δ removes false matches and obtains the final image feature matching result.

[0207] The present invention uses a new statistical perspective, the basic basis of which is that the neighborhood elements of the corresponding feature points will maintain level consistency in a certain attribute. First, the feature descriptor is used to construct an initial matching set, and the n of a certain feature point is searched. k The nearest neighbors are used as neighborhood elements, and a ranking table is generated according to a certain robust attribute. Then the Kendall rank correlation coefficient between the corresponding feature point neighborhood ranking tables is calculated to judge the correlation between them, so as to distinguish and remove the mismatched points. In order to cope with the interference of a large number of mismatched points caused by various unfavorable factors, a neighborhood common element guidance strategy is designed to improve the robustness of the method. In addition, a multiple neighborhood strategy is used to cope with the fluctuation of accuracy caused by feature point sets with different densities.

[0208] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0209] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0210] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A feature matching method based on multi-neighborhood guided Kendall rank correlation coefficient, characterized in that: The image feature matching method comprises the following steps: Get two images to be matched; Detect feature points in the two images using the SIFT algorithm and establish feature descriptors; Constructing an initial set of matches based on the similarity of feature descriptors in the two images; According to the definition of the generalized Kendall rank correlation coefficient, the Kendall rank correlation coefficient of each initial match is calculated, and a disordered erroneous match set is obtained by screening from the initial match set according to the Kendall rank correlation coefficient; Get the neighborhood point set of each initially matched feature point and ; Get and The ordered indexes of all feature points in the initial matching set and ; According to the ordered index and Calculate the index of common elements in the neighborhood , and get the neighborhood common element set : in is the index of the common element of the neighborhood; in, represents the initial matching set, Represents the sequence number of the initial match, Indicates Initial matches, is the index of the common element of the neighborhood; According to the out-of-order error matching set and the neighborhood common element set , calculated and The intersection of the two sets yields a tricky mismatch set : in, represents the initial matching set, Represents the sequence number of the initial match, Indicates Initial matches, is the index of the common element of the neighborhood; definition , represents the mismatch set The cardinality of ,in Neighborhood point set and Size, similarly defined , Represents the neighborhood common element set The cardinality of ; according to and Calculate the Kendall rank correlation coefficient guided by the neighborhood common elements , referred to as the bootstrapped Kendall rank correlation coefficient : in, represents the number of non-common elements in the neighborhood, Represents the number of mismatched points hidden in common points; The guided Kendall rank correlation coefficient of multiple neighborhoods of the feature point is calculated using the multiple neighborhood strategy and the average is taken to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point. ; Setting thresholds ,according to With threshold The relationship between them is used to remove false matches and obtain the final image feature matching result.

2. The feature matching method of multi-neighborhood guided Kendall rank correlation coefficient as claimed in claim 1, characterized in that: The step of calculating the Kendall rank correlation coefficient of each initial match according to the definition of the generalized Kendall rank correlation coefficient, and screening out-of-order erroneous match sets from the initial match sets according to the Kendall rank correlation coefficient comprises: According to the definition of the generalized Kendall rank correlation coefficient, the Kendall rank correlation coefficient of each initial match is calculated ; Set a larger threshold ,Will and Compare and filter out the wrong matching points with disordered neighborhood elements to obtain the disordered wrong matching set : in, represents the initial matching set, Represents the sequence number of the initial match, Indicates Initial matches, Indicates The Kendall rank correlation coefficient corresponding to the initial matches is Indicates the size of the initial matching set.

3. The feature matching method of multi-neighborhood guided Kendall rank correlation coefficient as claimed in claim 1, characterized in that: The multi-neighborhood strategy is used to calculate the guided Kendall rank correlation coefficients of multiple neighborhoods of the feature point and take the average to obtain the multi-neighborhood guided Kendall rank correlation coefficient of each feature point. The steps include: For any initial match, multiple neighborhoods of different scales are constructed, and the size of each neighborhood is ,in is the number of multiple neighbors; Calculate the bootstrapped Kendall rank correlation coefficient for each neighborhood , and take the average to get the bootstrapped Kendall rank correlation coefficient of multiple neighborhoods for: in is the number of multiple neighbors, For the The size of the neighborhood, For the The cardinality of the neighborhood common element set of the neighborhoods, For the Neighborhood mismatch set The base number, For the The bootstrapped Kendall rank correlation coefficient of the neighbors, and .

4. The feature matching method of multi-neighborhood guided Kendall rank correlation coefficient as claimed in claim 1, characterized in that: The threshold value ,according to With threshold The steps of removing false matches based on the relationship of and obtaining the final image feature matching result include: Setting the Threshold , add indicator variables ,according to With threshold The relationship between Assignment: in Representative Initial matches, Representative Matching ; According to the indicator variable The value of determines the correct set of matches: in is the correct matching set, and N is the size of the initial matching set.

5. A multi-neighborhood guided Kendall rank correlation coefficient feature matching device, characterized in that: Includes the following modules: An image acquisition module, used for acquiring two images to be matched; Feature point detection and descriptor creation module, used to detect feature points in two images through SIFT algorithm and create feature descriptors; An initial matching set construction module, used to construct a set of initial matching sets according to the similarity of feature descriptors in two images; A Kendall rank correlation coefficient calculation module is used to calculate the Kendall rank correlation coefficient of each initial match according to the definition of the generalized Kendall rank correlation coefficient, and to screen out-of-order erroneous matching sets from the initial matching sets according to the Kendall rank correlation coefficient; Neighborhood common element acquisition module, used to obtain the neighborhood point set of each initial matching feature point and ; Get and The ordered indexes of all feature points in the initial matching set and ; According to the ordered index and Calculate the index of common elements in the neighborhood , and get the neighborhood common element set : in is the index of the common element of the neighborhood; in, represents the initial matching set, Represents the sequence number of the initial match, Indicates Initial matches, is the index of the common element of the neighborhood; A neighborhood common element guiding strategy module is used to guide the mismatching set according to the disordered and the neighborhood common element set , calculated and The intersection of the two sets yields a tricky mismatch set : in, represents the initial matching set, Represents the sequence number of the initial match, Indicates Initial matches, is the index of the common element of the neighborhood; definition , represents the mismatch set The cardinality of ,in Neighborhood point set and Size, similarly defined , Represents the neighborhood common element set The cardinality of ; according to and Calculate the Kendall rank correlation coefficient guided by the neighborhood common elements , referred to as the bootstrapped Kendall rank correlation coefficient : in, represents the number of non-common elements in the neighborhood, Represents the number of mismatched points hidden in common points; The multiple neighborhood strategy module is used to calculate the guided Kendall rank correlation coefficient of multiple neighborhoods of each feature point using the multiple neighborhood strategy and take the average to obtain the multiple neighborhood guided Kendall rank correlation coefficient of each feature point ; Feature matching module, used to With threshold The relationship between them is used to remove false matches and obtain the final image feature matching result.