Image uniform matching method considering texture-based voronoi constraint

Abstract

The application relates to a texture-considered image uniform matching method of Voronoi constraint, which comprises the following steps: inputting two images with an overlapping area, namely a left image and a right image; extracting feature points of the left image and the right image, and constructing feature descriptors; calculating Euclidean distances of the feature descriptors, searching for candidate matching points according to the nearest neighbor distance criterion, and performing initial matching according to the ratio of the distances of the nearest neighbor and the second nearest neighbor feature points to obtain initial matching points; constructing a Thiessen polygon Voronoi graph for the initial matching points and the matching points corresponding to the initial matching points respectively, constructing a texture-based geometric constraint relationship for all the Voronoi nodes, realizing coarse error elimination, and dynamically updating; calculating the affine relationship of each corresponding Thiessen polygon face sheet; performing grid division through the Thiessen polygon Voronoi graph constructed for the initial matching points; and realizing matching of the grid encryption points by using an affine model and a normalized correlation coefficient. The application improves the uniformity of the homonymic matching points and the robustness of the matching algorithm.

Description

Technical Field

[0001] This invention relates to an image uniform matching method that takes into account texture Voronoi constraints. Background Technology

[0002] Image matching, as a key technology in photogrammetry and computer vision, has wide applications in multi-source image registration, aerial triangulation, image change detection, and 3D building reconstruction. The purpose of image matching is to find the corresponding connection points between two or more overlapping images. In recent years, in addition to traditional gray-level correlation matching methods, feature matching algorithms have been extensively studied. Their main steps include feature extraction, feature matching, and gross error removal. However, because descriptors are constructed using only local image information, the initial feature matching results often contain a large number of mismatched points. Furthermore, factors such as repetitive textures and large tilt angles can further cause mismatches. This results in uneven distribution of matching points on the image, leading to subsequent aerial triangulation failures. Traditional global constraint gross error removal algorithms based on random sampling consistency rely on sampling points, cannot effectively remove erroneous points, and sometimes even remove correct points. Moreover, their computational complexity increases exponentially with the increase in erroneous points. Summary of the Invention

[0003] This invention addresses the problem that traditional image matching algorithms cannot effectively remove erroneous points and obtain uniform matching points. This invention combines local texture features and local neighborhood geometric constraints to remove matching erroneous points. Based on the initial matching points, a Voronoi diagram is constructed, and the Voronoi geometric constraints are used to perform encrypted matching of the image, thereby obtaining uniformly distributed homonymous connection points.

[0004] The technical solution of this invention is as follows:

[0005] A texture-constrained Voronoi image uniform matching method includes the following steps:

[0006] Input two overlapping images, a left image and a right image; extract feature points from the left and right images and construct feature descriptors based on the extracted feature points; calculate the Euclidean distance between the feature descriptors, search for candidate matching points using the nearest neighbor distance criterion, and perform initial matching based on the ratio of the distances between the nearest and second nearest neighbor feature points to obtain initial matching points; construct Thiessen polygon Voronoi diagrams, V1 and V2, for the initial matching points and their corresponding matching points; construct texture-based geometric constraints for all Voronoi nodes in V1 and V2 to achieve gross error removal, and dynamically update V1 and V2; calculate the affine relation for each corresponding Thiessen polygon patch, and use the Thiessen polygon Voronoi diagram constructed from the initial matching points, divide it into a 10*10 grid, and use the affine model and normalized correlation coefficient to achieve matching of grid-refined points.

[0007] The feature points of the image are extracted using a Hessian-Affine elliptic domain detector and are known as HA feature points; the feature descriptors are constructed using the SIFT algorithm and are known as SIFT feature descriptors.

[0008] The specific implementation process of the Hessian-Affine elliptic domain detector is as follows: The Hessian-Affine algorithm obtains affine invariance of position and scale through adaptive iteration in the LoG multi-scale space. The initial feature points of the Hessian-Affine are determined by the second-order differential Hessian matrix H(L) of the image point and its row and column DoH (Determinant of Hessian).

[0009]

[0010]

[0011] In the above formula, L x (x) and L y (x) represents the derivative results in the x and y directions after applying Gaussian smoothing g(σ) to the target image. and Let σ represent the second-order differential results in the x and y directions, respectively, and σ be the Gaussian scale value.

[0012] The specific implementation process of constructing the SIFT feature descriptor is as follows: For each HA feature point, firstly, rotate the coordinate axes to the direction of the HA feature point; divide the 16*16 pixel range centered on the HA feature point in the scale space into 4*4 sub-regions, and calculate the gradient histogram for each sub-region; then arrange the 8-direction gradient histograms of the 4*4 sub-regions in order of position to form a 128-dimensional feature vector; finally, normalize the length of the feature vector.

[0013] The specific implementation process of constructing texture-based geometric constraints for all Voronoi nodes in V1 and V2 is as follows: taking the Voronoi node as the center, compare it clockwise with 8 points in its circular neighborhood; use the pixel value of the Voronoi node as a threshold; then compare the gray values ​​of the 8 neighboring pixels with the threshold. If the value of a surrounding pixel is greater than the value of the center pixel, the pixel is marked as 1; otherwise, the pixel is marked as 0, thus generating an 8-bit unsigned number, which gives the LBP value of the Voronoi node.

[0014] The specific process for implementing coarse insertion culling is as follows: For the matching point Pi to be culled, its adjacent Voronoi nodes are Pn (n = 1, 2, ..., n), with a quantity of n; the matching point Qi to be culled corresponding to Pi has adjacent Voronoi nodes Qn (n = 1, 2, ..., n), with a quantity of n, as follows... Figure 2 and Figure 3 As shown; define local polar coordinate systems with the pole Pi(Qi) and the vertical polar axis pointing to the right. Using polar coordinates, sequentially connect and combine the LBP texture features of neighboring Voronoi nodes to obtain new n*8 dimensional feature vectors X = (x1, x2, ... xn) and Y = (y1, y2, ... yn). Calculate the Euclidean distance between the two feature vectors; the smaller the Euclidean distance, the more similar the two feature vectors are.

[0015]

[0016] x k y represents the LBP feature combination vector of the left image node k. k Let d represent the LBP feature combination vector of the right image node k; if d is greater than the threshold t, t = 10, then the point pair is retained; otherwise, the point pair is discarded, and the Thiessen polygon is updated. and

[0017] The specific process of matching mesh refinement points using an affine model and normalized correlation coefficients is as follows: For the updated V1 and V2, calculate the affine coefficient F of their corresponding Thiessen polygon faces. n; Create a 10x10 grid for the left image; For the Thiessen polygon falling into V1 Grid point N in i Using the affine coefficient F i M will be predicted on the right image. i The normalized correlation coefficient is used for matching to obtain the encrypted set of matching points.

[0018]

[0019] In the formula, ρ(c,r) is the correlation coefficient at pixel (c,r), c is the image row number, r is the image column number, g is the gray value of the left image, and g′ is the gray value of the right image.

[0020] The technical effects of this invention are as follows:

[0021] This invention comprehensively utilizes image grayscale and texture features, and employs Voronoi topological relationships to construct geometrically constrained texture features. By fully leveraging texture features and geometric constraints, it overcomes the shortcomings of traditional methods in removing erroneous matching points, effectively extracts and removes erroneous matching points, and effectively extracts corresponding connection points between two images. The corresponding connection points are evenly distributed, improving the uniformity of corresponding matching points and the robustness of the matching algorithm. Attached Figure Description

[0022] Figure 1 A flowchart for uniformly encrypted image matching to take into account the constraints of irregular triangular meshes in texture.

[0023] Figure 2 This is the Voronoi facet corresponding to the left image.

[0024] Figure 3 This is the Voronoi facet corresponding to the right image.

[0025] Figure 4 It is the Voronoi diagram V1 constructed from the initial matching points on the left image.

[0026] Figure 5 It is the Voronoi diagram V2 constructed from the initial matching points of the right image.

[0027] Figure 6 Preview image of the matching point file.

[0028] Figure 7 This is a preview of the initial matching points.

[0029] Figure 8 This is a preview of the final matching points after encryption. Detailed Implementation

[0030] A Voronoi constraint-based image uniform matching method considering texture is presented in this embodiment, taking the influence of two images with overlapping areas as an example; the specific process is as follows: Figure 1 As shown; including the following steps:

[0031] (1) Input two images with overlapping areas, namely the left image and the right image;

[0032] (2) Extract HA feature points from the left and right images respectively using the Hessian-Affine elliptic domain detector;

[0033] The specific implementation process of the Hessian-Affine elliptic domain detector is as follows: The Hessian-Affine algorithm obtains affine invariance of position and scale through adaptive iteration in the LoG multi-scale space. The initial feature points of the Hessian-Affine are determined by the second-order differential Hessian matrix H(L) of the image point and its row and column DoH (Determinant of Hessian).

[0034]

[0035]

[0036] In the above formula, L x (x) and L y (x) represents the derivative results in the x and y directions after applying Gaussian smoothing g(σ) to the target image. and Let σ represent the second-order derivatives in the x and y directions, respectively, and σ be the Gaussian scale value.

[0037] (3) Construct SIFT feature descriptors for the HA feature points extracted above;

[0038] The specific implementation process of constructing the SIFT feature descriptor is as follows: For each HA feature point, the coordinate axis is first rotated to the direction of H-A feature point to ensure rotation invariance; within a 16*16 pixel range centered on the HA feature point in the scale space, it is divided into 4*4 sub-regions, and a gradient histogram (the histogram is evenly divided into 8 directions) is calculated for each sub-region; then, the 8-direction gradient histograms of the 4*4 sub-regions are arranged sequentially according to their positions to form a 128-dimensional feature vector, which enhances the algorithm's noise resistance and fault tolerance; finally, the length of the feature vector is normalized to remove the influence of illumination changes.

[0039] (4) Calculate the Euclidean distance for the SIFT feature descriptor constructed above, and use the ratio of the distance between the nearest neighbor and the second nearest neighbor feature points to perform initial matching and obtain the initial matching point; Figure 2 yes Figure 4 Enlarged view of the part Figure 3 yes Figure 5 A magnified view of a portion, Figure 2 and Figure 3 They are a pair of corresponding Thiessen polygonal facets;

[0040] (5) Construct the Voronoi diagram of the Thiessen polygons for the initial matching points and the Voronoi diagram of the Thiessen polygons for the corresponding matching points, which are V1 and V2, respectively. The corresponding Thiessen polygons are respectively... and

[0041] (6) Construct texture features (LBP) for all Voronoi nodes in V1 and V2;

[0042] The specific implementation process is as follows: Taking the Voronoi node as the center, compare it clockwise with the 8 points in its circular neighborhood; use the pixel value of the Voronoi node as the threshold; then compare the gray values ​​of the 8 pixels in the neighborhood with the threshold. If the value of a surrounding pixel is greater than the value of the center pixel, the pixel is marked as 1; otherwise, the pixel is marked as 0, thus generating an 8-bit unsigned number, which gives the LBP value of the Voronoi node.

[0043] (7) For a matching point Pi to be removed, all initial matching points are points to be removed, and its adjacent Voronoi nodes are Pn (n = 1, 2, ..., n), with a total number of n; the matching point Qi to be removed corresponding to Pi has adjacent Voronoi nodes Qn (n = 1, 2, ..., n), with a total number of n, such as... Figure 2 and Figure 3 As shown, local polar coordinate systems are defined, with the pole being Pi (Qi) and the polar axis pointing vertically to the right. Using polar coordinates, the texture features (LBPs) of neighboring nodes are sequentially concatenated to obtain new n*8 dimensional feature vectors X = (x1, x2, ..., xn) and Y = (y1, y2, ..., yn). The Euclidean distance between the two feature vectors is then calculated. The smaller the Euclidean distance, the more similar the two feature vectors are.

[0044]

[0045] x k y represents the LBP feature combination vector of the left image node k. k Let d represent the LBP feature combination vector of the right image node k; if d is greater than the threshold t, t = 10, then the point pair is retained; otherwise, the point pair is discarded, and the Thiessen polygon is updated. and

[0046] (8) Regarding the above-updated Thiessen polygon and Calculate the affine coefficient F of the corresponding polygonal facet. n ; Create a 10x10 grid for the left image; For the Thiessen polygon falling into V1 Grid point N in i Using the affine coefficient F i M will be predicted on the right image. i The normalized correlation coefficient is used for matching to obtain the encrypted matching results.

[0047]

[0048] In the formula, ρ(c,r) is the correlation coefficient at pixel (c,r), c is the image row number, r is the image column number, g is the gray value of the left image, and g′ is the gray value of the right image.

[0049] (9) Input the final matching point into a text file.

Claims

1. A Voronoi constraint-based image uniform matching method that takes into account texture, characterized in that: The method comprises the following steps: inputting two images with an overlapping area, namely a left image and a right image; extracting feature points of the left image and the right image, and constructing a feature descriptor based on the extracted feature points; calculating the Euclidean distance of the feature descriptor, searching for candidate matching points based on the nearest neighbor distance criterion, and performing initial matching based on the ratio of the distance between the nearest neighbor and the second nearest neighbor feature points, to obtain initial matching points; constructing Voronoi diagrams V1 and V2 for the initial matching points and the matching points corresponding to the initial matching points respectively; constructing a texture-based geometric constraint relationship for all Voronoi nodes in V1 and V2 to realize gross error elimination, and dynamically updating V1 and V2; calculating the affine relationship of each corresponding Voronoi polygon patch, constructing a Voronoi diagram based on the initial matching points, and performing 10*10 grid division, and realizing the matching of the grid encryption points by using the affine model and the normalized correlation coefficient; the specific implementation process of constructing a texture-based geometric constraint relationship for all Voronoi nodes in V1 and V2 is as follows: taking the Voronoi node as the center, comparing the 8 points in the ring neighborhood of the Voronoi node in a clockwise manner; taking the pixel value of the Voronoi node as a threshold; then comparing the gray values of the 8 pixel points in the neighborhood with the threshold, if a certain pixel value in the neighborhood is greater than the center pixel value, the position of the pixel point is marked as 1; otherwise, the pixel point is marked as 0, thereby obtaining an 8-bit unsigned number, that is, the LBP value of the Voronoi node; the specific process of realizing gross error elimination is as follows: for a to-be-eliminated matching point Pi and its adjacent Voronoi nodes Pn (n=1, 2, …n), the number of which is n; for a to-be-eliminated matching point Qi corresponding to Pi and its adjacent Voronoi nodes Qn (n=1, 2, …n), the number of which is n; local polar coordinates are defined respectively, the pole points are Pi and Qi respectively, and the polar axis is vertical to the right; in the polar coordinate mode, the texture features LBP of the adjacent Voronoi nodes are connected and combined in a clockwise manner to obtain new n*8-dimensional feature vectors X=(x1, x2, …xn) and Y=(y1, y2, …yn), and the Euclidean distance of the two feature vectors is calculated; the smaller the Euclidean distance is, the more similar the two feature vectors are; x k LBP feature combination vector of left image node k, y k LBP feature combination vector of right image node k; if d is greater than threshold t, t = 10, the point pair is reserved, otherwise the point pair is rejected, and the Thiessen polygon is updated and 2. The texture-aware, Voronoi-constrained image homomorphic matching method of claim 1, wherein: the feature points of the image are extracted by a Hessian-Affine elliptical region detector, which are H-A feature points; the construction of the feature descriptor is realized by a SIFT algorithm, which is a SIFT feature descriptor.

3. The texture-aware, Voronoi-constrained image homomorphic matching method of claim 2, wherein: the specific implementation process of the Hessian-Affine elliptical region detector is as follows: the Hessian-Affine algorithm obtains the affine invariance of position and scale through adaptive iteration in the LoG multi-scale space; the initial feature points of Hessian-Affine are obtained by judging the second-order differential Hessian matrix H(L) of the image point x and the determinant of Hessian DoH; In the above formula, L x (x) and L y (x) represents the result of respectively differentiating x and y directions after the target image is subjected to g(σ) Gaussian smoothing, and represents the result of respectively differentiating x and y directions, and σ is a Gaussian scale value.

4. The texture-aware, Voronoi-constrained image homomorphic matching method of claim 3, wherein: The specific implementation process of constructing the SIFT feature descriptor is as follows: for each H-A feature point, first, the coordinate axis is rotated to the direction of the H-A feature point; in the 16*16 pixel range of the H-A feature point in the scale space, 4*4 sub-regions are divided, and the gradient histogram of each sub-region is calculated; then, the 8-direction gradient histograms of the 4*4 sub-regions are sequentially arranged according to the positions, so as to form a 128-dimensional feature vector; finally, the length of the feature vector is normalized.

5. The texture-aware, Voronoi-constrained image homomorphic matching method of claim 4, wherein: The specific process of matching mesh refinement points using an affine model and normalized correlation coefficients is as follows: For the updated V1 and V2, calculate the affine coefficient F of their corresponding Thiessen polygon faces. n ; Create a 10x10 grid for the left image; For the Thiessen polygon T falling into V1 i 1 N grid points in i Using the affine coefficient F i M will be predicted on the right image. i The normalized correlation coefficient is used for matching to obtain the encrypted set of matching points. In the formula, p(c, r) is a correlation coefficient at a pixel (c, r), c is an image row number, r is an image column number, g is a left image gray value, and g' is a right image gray value.

Images