Difference chart registration method based on feature point matching

Through the difference graph registration method based on feature point matching, image feature points are extracted and corrected, false matching points are removed, and accurate image transformation matrix is ​​generated, which solves the problem of low registration accuracy of images with large color differences, and significantly improves registration Accuracy and effect.

CN119090926AActive Publication Date: 2024-12-06BEIJING DAHENG IMAGE VISION CO LTD

Patent Information

Application Number
CN202411152637.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-06
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

When registering images with large color differences, the prior art has problems with mismatching feature points, resulting in inaccurate transformation matrix, especially in the conditions of noise, brightness changes or geometric deformation, the registration effect is poor.

Method used

Using the difference graph registration method based on feature point matching, image feature points and descriptors are extracted through feature extraction algorithms, Euro-type distance is calculated to judge feature point similarity, analyze the geometric transformation relationship of matching point pairs, generate image transformation matrix, and pass Scale, rotate and translate to correct feature points, use clustering methods to remove mismatch points, and improve registration accuracy.

Benefits of technology

It significantly improves the accuracy of image registration, reduces registration errors, improves the accuracy of the transformation matrix, and improves the registration effect, especially in complex conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090926A_ABST
    Figure CN119090926A_ABST
Patent Text Reader

Abstract

The invention discloses a difference image registration method based on feature point matching, and relates to the technical field of image matching, and the method comprises the steps: S1, extracting feature points and descriptors of a first image and a second image through employing an SIFT algorithm; s2, the Euclidean distance of SIFT descriptors between the two images is calculated, the similarity of feature points is judged according to the Euclidean distance, and feature point pairs in the two images are matched; s3, analyzing the geometric transformation relation of the matching point pairs, and generating an image transformation matrix; s4, transforming the second image into an image aligned with the first image content through the image transformation matrix; according to the method, the mismatched feature point pairs are effectively removed, so that registration errors are reduced, the precision of a transformation matrix is improved, and the registration effect is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image matching, and in particular to a difference map registration method based on feature point matching. Background Art

[0002] Image matching refers to matching similar or identical content between two images through a certain matching algorithm. This correspondence is usually represented by an image transformation matrix. Commonly used image matching algorithms are mainly divided into grayscale-based matching and feature-based matching. Grayscale-based matching measures the color difference between two images. The smaller the difference, the higher the matching degree between the two. Feature-based matching first extracts features from the image, and then establishes a matching correspondence between the features of the two images. Commonly used feature matching primitives include prominent features such as points, lines, and regions.

[0003] For the registration of difference images, since there is a big difference in color between the two images, color information cannot be used in the registration. Only the texture information or feature point information of the image can be used for matching. The industry usually adopts the method of calculating affine transformation by feature point matching. First, feature points are extracted, feature descriptors are generated, and then feature point matching is performed, and the image transformation matrix is ​​calculated by the least squares method. When using feature point matching, due to the limited information of feature points, there will inevitably be feature point mismatches. The current scheme is to calculate the transformation matrix with the minimum error while allowing for erroneous matching points. When there are many erroneously matched feature points or the number of feature points used for matching is small, an inaccurate transformation matrix will be generated. Summary of the invention

[0004] The purpose of the present invention is to design an image registration algorithm that can accurately register images with greatly different colors.

[0005] The technical solution of the present invention is to provide a difference map registration method based on feature point matching, the method comprising:

[0006] S1, extracting feature points and descriptors of the first image and the second image using a feature extraction algorithm;

[0007] S2, calculating the Euclidean distance between the descriptors of the first image and the second image, thereby determining the similarity of the feature points and matching the feature point pairs in the two images;

[0008] S3, analyzing the geometric transformation relationship of the matching point pairs and generating an image transformation matrix;

[0009] S4. Transform the second image into an image aligned with the content of the first image using an image transformation matrix.

[0010] In any of the above technical solutions, further, step S3 includes:

[0011] S31, calculating the scaling ratio S of the first image relative to the content on the second image x and S y , where S x is the horizontal scaling value, S y is the vertical scaling value; according to the transformation relationship between the two known images, there is the following relationship between any two pairs of matching points:

[0012] D1 2 =(X1 i -X1 j ) 2 +(Y1 i -Y1 j ) 2 ;

[0013] D1 2 =S x 2 *(X2 i -X2 j ) 2 +S y 2 *(Y2 i -Y2 j ) 2 ;

[0014] (X1 i , Y1 i )、(X1 j , Y1 j ) are the positions of any two feature points in the first image; (X2 i , Y2 i )、(X2 j , Y2 j ) are the positions of any two feature points in the second image; (X1 i , Y1 i ) and (X2 i , Y2 i ) is a pair of matching feature points, (X1 j , Y1 j ) and (X2 j , Y2 j ) is a pair of matching feature points; D1 is the two feature points in the first image (X1 i , Y1 i ) and (X1 j , Y1 j), transform the feature points according to the scaling ratio so that the distance between the two feature points in the second image is equal to D1, and combine the above two equations to solve the linear fitting problem of slope k and intercept b:

[0015]

[0016] The matching feature point pairs in the first image and the second image are used to calculate the point (x, y) on the fitting line as follows:

[0017]

[0018] The combination of all two feature point pairs on the first image and the second image constitutes a lot of points on the fitting line; the fitting line obtains k and b, and thus obtains the scaling ratio S x and S y , and excludes the characteristic point pairs that are not on the fitting line;

[0019] S32, according to the scaling ratio of the image calculated in step S31, the point (X2 i , Y2 i ) is scaled and corrected to (X2 i *S x , Y2 i *S y ), (X2 j , Y2 j ) becomes (X2 j *S x , Y2 j *S y ); Calculate the rotation angle A of each pair of two feature points based on the corrected points;

[0020] In the first image, calculate (X1 i , Y1 i ) as the starting point, with (X1 j , Y1 j ) as the end point of the straight line angle A1, calculate the angle of (X2 i *S x , Y2 i *S y ) as the starting point, with (X2 j *S x , Y2 j *S y ) is the straight line angle A2 of the end point; thus the angle difference between the two lines is calculated as A=A1-A2;

[0021] Calculate the rotation angle A of all two pairs of feature points in the two images. The rotation angle range is (-π,π]. Calculate the histogram of all rotation angles according to a certain step size. The angle corresponding to the highest peak of the histogram is the rotation angle to be calculated, and exclude the feature point pairs corresponding to the angles outside the peak.

[0022] S33, according to the scaling value and rotation angle calculated in S31 and S32, correct the feature point in the second image to the position (X2 i ′,Y2 i ′):

[0023] X2 i ′=X2 i *S x *cos(A)-Y2 i *S y *sin(A);

[0024] Y2 i ′=X2 i *S x *sin(A)+Y2 i *S y *cos(A);

[0025] Calculate the translation value (T ix , T iy ):

[0026] T ix =X2 i ′-X1 i ;

[0027] T iy =Y2 i ′-Y1 i ;

[0028] The two-dimensional vectors of the translation values ​​are divided into different clusters according to the spatial distance using the clustering method, so that the distance between the vectors in the same cluster is as close as possible, and the distance between the vectors in different clusters is as far as possible, that is, the vectors in the same cluster are clustered together, and the vectors in different clusters are separated as much as possible; then the average value of the group of vectors with the largest number of vectors is the translation position of the image (T x , T y );

[0029] S34, calculate the image transformation matrix M, the formula is:

[0030]

[0031] In any of the above technical solutions, further, step S2 includes:

[0032] Suppose the descriptor of a feature point in the first image is:

[0033] R i =(r i1 , r i2 , ..., r i128 );

[0034] The descriptor of a feature point in the second image is:

[0035] S i =(s i1 ,s i2 , ..., s i128 );

[0036] The Euclidean distance d(R i , S i ) is calculated as:

[0037]

[0038] Euclidean distance is the metric for feature point matching;

[0039] For each feature point, find the most similar matching point in the other image, and for each feature point descriptor R in the first image i , find the two descriptors S with the smallest Euclidean distance in the second image i and S p , these two points are R i Preliminary matching points in the second image;

[0040] The matching points are screened by the nearest neighbor ratio test. The specific steps are as follows:

[0041] Calculate the distance ratio between the nearest neighbor matching point and the second nearest neighbor matching point A ratio threshold is set. If the ratio is less than the threshold, the matching point is considered reliable. The lower the ratio, the greater the distance difference between the matching point and the secondary matching point, and the more reliable the matching result. If the ratio is higher than the threshold, the matching point may be a mismatch and should be removed. In this way, feature point matching pairs with higher reliability are finally left.

[0042] In any of the above technical solutions, further, step S1 includes:

[0043] S11, constructing the scale space of the image and detecting extreme points therein;

[0044] The construction of scale space is usually carried out using Gaussian pyramid. The image is smoothed at different scales to generate a series of images with different resolutions. After each layer of image is Gaussian blurred, the size is reduced to form a pyramid structure. The expression of Gaussian pyramid scale space L(x, y, σ) is:

[0045] L(x,y,σ)=G(x,y,σ)*I(x,y);

[0046] Among them, G(x, y, σ) is a Gaussian kernel function with variable scale, I(x, y) is the original image, and σ represents the scale;

[0047] On the generated Gaussian pyramid, the Gaussian blurred images of adjacent scales are differentially operated to obtain the Gaussian difference image (DoG) to form a Gaussian difference pyramid; DoG can be approximated as the Laplacian operator in the scale space and is used to detect extreme points in the image;

[0048] In each layer of the Gaussian difference pyramid, each pixel is compared with its 26 neighboring pixels in the scale space and the two-dimensional image space; if the point is the maximum or minimum value in all these neighborhoods, it is considered to be a potential feature point;

[0049] S12, after detecting the potential feature points, further accurately positioning them;

[0050] This step includes eliminating unstable feature points and determining the sub-pixel position of each feature point. The specific method is as follows:

[0051] Through Taylor series expansion, the extreme points are interpolated and fitted at the sub-pixel level to obtain a more accurate position, significantly improving the positioning accuracy of feature points;

[0052] Extreme points with low contrast are easily disturbed by noise, so they need to be removed. By comparing the contrast threshold of feature points, those key points that are not stable enough can be removed.

[0053] The principal curvature of the DoG function at the edge is large, which easily generates unstable feature points. The key points are screened using the ratio of the principal curvatures to remove those low-stability feature points that cross the edge.

[0054] S13, assigning a direction to each key point so that the extracted feature points have rotation invariance;

[0055] In the neighborhood of the feature point, the gradient magnitude and direction of each pixel are calculated, and the gradient direction in the neighborhood of the feature point is counted into a histogram. The horizontal axis of the gradient direction histogram is the gradient direction, and the vertical axis is the cumulative value of the gradient magnitude.

[0056] The gradient direction histogram is divided into 36 intervals of 10°. The peak of the histogram is the main direction of the feature point. If there are multiple peaks, multiple directions can be assigned to the feature point to enhance the robustness of the algorithm.

[0057] S14, after determining the position, scale and direction of the feature point, finally generating a descriptor for each feature point, the descriptor is used to characterize the local image information of the feature point, so that it has stable matching performance in different images;

[0058] According to the main direction of the feature point, the coordinate axis of its neighborhood is rotated to make the descriptor rotationally invariant; the neighborhood of the feature point is divided into 4×4 grids, and the gradient histograms of 8 directions are calculated in each grid to form a 128-dimensional feature vector, that is, a 128-dimensional descriptor.

[0059] The beneficial effects of the present invention are:

[0060] The technical solution in the present invention can significantly improve the accuracy of image registration, especially under complex conditions such as noise, brightness changes or geometric deformation. Compared with the prior art, the present invention reduces registration errors and improves the accuracy of the transformation matrix by effectively removing mismatched feature point pairs, thereby greatly improving the registration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The advantages of the above and additional aspects of the present invention will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0062] Figure 1 is a flow chart of a difference map registration method based on feature point matching according to an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of matching feature point pairs of a difference map registration method based on feature point matching according to an embodiment of the present invention;

[0064] Figure 3 is a histogram of key point allocation directions of a difference map registration method based on feature point matching according to an embodiment of the present invention;

[0065] Figure 4 is a histogram of gradients within a calculated block according to a difference map registration method based on feature point matching according to an embodiment of the present invention;

[0066] Figure 5 is an original image according to an embodiment of a difference map registration method based on feature point matching according to an embodiment of the present invention;

[0067] Figure 6According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Figure 5 Unscreened feature point pair matching results;

[0068] Figure 7 According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Figure 5 Image alignment using unfiltered feature point pairs;

[0069] Figure 8 According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Figure 5 Image alignment effect using filtered feature point pairs;

[0070] Fig. 9 According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Figure 5 The matching results of the selected feature point pairs;

[0071] Fig.10 is an original image according to another embodiment of a difference map registration method based on feature point matching according to one embodiment of the present invention;

[0072] Fig.11 According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Fig.10 Matching results of feature point pairs before and after screening;

[0073] Fig.12 According to an embodiment of the present invention, a difference map registration method based on feature point matching is provided. Fig.10 Image alignment using filtered before and after feature point pairs. DETAILED DESCRIPTION

[0074] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0075] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0076] like Figure 1 As shown, this embodiment provides a difference map registration method based on feature point matching, the method comprising:

[0077] S1. Use SIFT algorithm to extract feature points of two images.

[0078] The SIFT algorithm is also called the scale-invariant feature transform algorithm. It completes feature extraction through the following steps:

[0079] S11. Construct a scale space of the image and detect extreme points in it.

[0080] The construction of scale space is usually carried out using Gaussian pyramid. The image is smoothed at different scales to generate a series of images with different resolutions. After each layer of image is Gaussian blurred, the size is reduced to form a pyramid structure. The expression of Gaussian pyramid scale space L(x, y, σ) is:

[0081] L(x,y,σ)=G(x,y,σ)*I(x,y);

[0082] Among them, G(x,y,σ) is a Gaussian kernel function with variable scale, I(x,y) is the original image, and σ represents the scale.

[0083] On the generated Gaussian pyramid, the Gaussian blurred images of adjacent scales are differentially operated to obtain the Gaussian difference image (DoG) to form a Gaussian difference pyramid. DoG can be approximated as the Laplacian operator in the scale space and is used to detect extreme points in the image.

[0084] In each layer of the Gaussian difference pyramid, each pixel is compared with its 26 neighboring pixels in the scale space and the two-dimensional image space; if the point is the maximum or minimum value in all these neighbors, it is considered to be a potential feature point.

[0085] S12. After detecting the potential feature points, further accurately locate them.

[0086] This step includes eliminating unstable feature points and determining the sub-pixel position of each feature point. The specific method is as follows:

[0087] Through Taylor series expansion, the extreme points are interpolated and fitted at the sub-pixel level to obtain a more accurate position, significantly improving the positioning accuracy of the feature points.

[0088] Extreme points with low contrast are easily disturbed by noise, so they need to be removed; by comparing the contrast threshold of feature points, those key points that are not stable enough are removed.

[0089] The principal curvature of the DoG function at the edge is large, which easily generates unstable feature points. The key points are screened using the ratio of the principal curvatures to remove those low-stability feature points that cross the edge.

[0090] S13. Assign a direction to each key point so that the extracted feature points are rotation invariant.

[0091] In the neighborhood of the feature point, the gradient magnitude and direction of each pixel are calculated, such as Figure 3 As shown, the gradient directions in the neighborhood of the feature point are counted into a histogram, the horizontal axis of the gradient direction histogram is the gradient direction, and the vertical axis is the cumulative value of the gradient amplitude.

[0092] The gradient direction histogram is divided into 36 intervals (10° per interval), and the peak of the histogram is the main direction of the feature point. If there are multiple peaks, multiple directions can be assigned to the feature point to enhance the robustness of the algorithm.

[0093] S14. After determining the position, scale and direction of the feature point, a descriptor is finally generated for each feature point. The descriptor is used to characterize the local image information of the feature point so that it has stable matching performance in different images.

[0094] According to the main direction of the feature point, the coordinate axis of its neighborhood is rotated to make the descriptor rotationally invariant; Figure 4 As shown in FIG. 1 , the neighborhood of the feature point is divided into 4×4 grids, and the gradient histograms in 8 directions are calculated in each grid to form a 128-dimensional SIFT feature vector, that is, a 128-dimensional descriptor.

[0095] In order to reduce the impact of illumination changes, the generated descriptor vector is normalized. The resulting descriptor is invariant and can match the same feature points in different images.

[0096] S2. Calculate the Euclidean distance of the SIFT descriptors between the two images to determine the similarity of the feature points and match the feature point pairs in the two images.

[0097] Suppose the descriptor of a feature point in the first image is:

[0098] R i =(r i1 ,r i2 ,…,r i128 );

[0099] The descriptor of a feature point in the second image is:

[0100] S i =(s i1 ,s i2 ,…,s i128 );

[0101] The Euclidean distance d(R i ,S i ) is calculated as:

[0102]

[0103] The smaller the Euclidean distance is, the higher the similarity between the two feature points in the description subspace. Therefore, this distance can be used as a metric for feature point matching.

[0104] For each feature point, find the most similar matching point in the other image, and for each feature point descriptor R in the first image i , find the two descriptors S with the smallest Euclidean distance in the second image i and S p , these two points are R i Preliminary matching points in the second image.

[0105] In order to reduce mismatching, the matching points are screened through the nearest neighbor ratio test. The specific steps are as follows:

[0106] Calculate the distance ratio between the nearest neighbor matching point and the second nearest neighbor matching point A ratio threshold is set. If the ratio is less than the threshold, the matching point is considered reliable. The lower the ratio, the greater the distance difference between the matching point and the secondary matching point, and the more reliable the matching result. If the ratio is higher than the threshold, the matching point may be a mismatch and should be removed. In this way, the feature point matching pairs with higher reliability are finally left.

[0107] There may still be a small number of mismatched points in the preliminary matching pairs. Therefore, before applying the final matching point pairs, it is necessary to further verify the global consistency of these matching point pairs under geometric transformation: use the matching point pairs to fit the global geometric transformation model, such as affine transformation or perspective transformation model. Use algorithms such as RANSAC (random sampling consistency) to process; check whether the matching point pairs meet the fitted transformation model, and remove the matching pairs that do not meet the geometric consistency to further improve the matching accuracy; after the above steps, the feature point pairs that are finally retained are the matching point pairs between the two images. These matching point pairs can be used for further image registration or other subsequent processing operations.

[0108] S3. Generate image transformation matrix using improved algorithm.

[0109] like Figure 2 As shown, (X1 i ,Y1 i )、(X1 j ,Y1 j ) are the positions of any two feature points in the first image; (X2 i ,Y2 i )、(X2 j ,Y2 j ) are the positions of any two feature points in the second image; (X1 i,Y1 i ) and (X2 i ,Y2 i ) is a pair of matching feature points, (X1 j ,Y1 j ) and (X2 j ,Y2 j ) is a pair of matching feature points.

[0110] S31, calculating the scaling ratio S of the first image relative to the content on the second image x and S y , where S x is the horizontal scaling value, S y is the vertical scaling value. According to the transformation relationship between the two known images, there is the following relationship between any two pairs of matching points:

[0111] D1 2 =(X1 i -X1 j ) 2 +(Y1 i -Y1 j ) 2 ;

[0112] D1 2 =S x 2 *(X2 i -x2 j ) 2 +S y 2 *(Y2 i -Y2 j ) 2 ;

[0113] D1 is the distance between two feature points in the first image, and the feature point (X1 i , Y1 i ) and (X1 j , Y1 j ), transform the feature points according to the scaling ratio so that the distance between the two feature points in the second image is equal to D 1, and combine the above two equations to solve the linear fitting problem of slope k and intercept b:

[0114]

[0115] The matching feature point pairs in the first image and the second image are used to calculate the point (x, y) on the fitting line as follows:

[0116]

[0117] The combination of all two feature point pairs on the first image and the second image constitutes a lot of points on the fitting line. The fitting line obtains k and b, and thus obtains the scaling ratio S x and S y , and excludes those feature point pairs that are not on the fitting line.

[0118] S32, according to the scaling ratio of the image calculated in step S31, the point (X2 i , Y2 i ) is scaled and corrected to (X2 i *S x , Y2 i *S y ), (X2 j , Y2 j ) becomes (X2 j *S x , Y2 j *S y ). Calculate the rotation angle A of each pair of feature points based on the corrected points.

[0119] In the first image, calculate (X1 i , Y1 i ) as the starting point, with (X1 j , Y1 j ) as the end point of the straight line angle A1, calculate the angle of (X2 i *S x , Y2 i *S y ) as the starting point, with (X2 j *S x , Y2 j *S y ) is the straight line angle A2 of the end point; thus the angle difference between the two lines is calculated as A=A1-A2.

[0120] Calculate the rotation angle A of all two pairs of feature points in the two images. The rotation angle range is (-π,π]. Calculate the histogram of all rotation angles according to a certain step size. The angle corresponding to the highest peak of the histogram is the rotation angle to be calculated, and exclude the feature point pairs corresponding to the angles outside the peak.

[0121] S33, according to the scaling value and rotation angle calculated in S31 and S32, correct the feature point in the second image to the position (X2 i ′,Y2 i ′):

[0122] X2 i ′=X2 i *S x *cos(A)-Y2 i *Sy *sin(A);

[0123] Y2 i ′=X2 i *S x *sin(A)+Y2 i *S y *cos(A);

[0124] Calculate the translation value (T ix , T iy ):

[0125] T ix =X2 i ′-X1 i ;

[0126] T iy =Y2 i ′-Y1 i ;

[0127] The two-dimensional vectors of the translation values ​​are divided into different clusters according to the spatial distance using the clustering method, so that the distance between the vectors in the same cluster is as close as possible, and the distance between the vectors in different clusters is as far as possible, that is, the vectors in the same cluster are clustered together, and the vectors in different clusters are separated as much as possible; then the average value of the group of vectors with the largest number of vectors is the translation position of the image (T x , T y ).

[0128] S34, calculate the image transformation matrix M, the formula is:

[0129]

[0130] S4. Transform the second image into an image aligned with the content of the first image using an image transformation matrix.

[0131] In another embodiment of the present invention, two images are selected for matching to show the matching effect:

[0132] like Figure 5 The two selected images are shown in FIG. 1 , the first image is on the left, and the second image is on the right. Figure 6 As shown, the feature points with higher matching degree in the two images are taken, and the endpoints of each line segment are the matching feature points in the two images, which are called feature point pairs.

[0133] Figure 6The red boxes in the figure mark the positions of 8 feature points. The feature points in the red boxes with the same number are a matching pair. It can be seen that although the matching degree of these 4 pairs of feature points meets the requirements, they are mismatched. If the image transformation matrix is ​​generated using the feature point pairs containing mismatches, and the second image is geometrically transformed according to the transformation matrix and aligned with the first image, it will appear Figure 7 In the case of non-alignment, Figure 7 The image on the left is the first image. Figure 7 The image on the right is Figure 5 The image after the transformation of the second image.

[0134] According to the algorithm of the present invention, based on the known conditions of image transformation, the transformation matrix is ​​calculated after removing the wrong matching feature point pairs, and then the transformation Figure 5 The second image is Figure 5 The first image is aligned, the effect is as follows Figure 8 shown; Figure 7 In comparison, the transformation matrix generated by using the feature point pairs screened by the present invention has a better alignment effect.

[0135] The feature points selected by the algorithm of the present invention are as follows Fig. 9 As shown, Figure 6 By comparison, we can see Figure 6 The incorrectly matched point pairs marked by the red boxes are deleted, thus improving the accuracy of the transformation matrix.

[0136] use Fig.10 The two images provided are matched again, such as Fig.11 As shown, the left side is the unscreened feature point pair, and the right side is the feature point pair screened by the method provided by the present invention; different feature point pairs are used to generate transformation matrices, and the following are generated: Fig.12 Comparison of alignment effects shown.

[0137] In summary, the present invention proposes a difference map registration method based on feature point matching, comprising:

[0138] S1. Use SIFT algorithm to extract feature points and descriptors of the first image and the second image.

[0139] S2. Calculate the Euclidean distance of the SIFT descriptors between the two images to determine the similarity of the feature points and match the feature point pairs in the two images.

[0140] S3. Analyze the geometric transformation relationship of the matching point pairs and generate an image transformation matrix.

[0141] S4. Transform the second image into an image aligned with the content of the first image using an image transformation matrix.

[0142] The steps in the present invention can be adjusted in order, combined or deleted according to actual needs.

[0143] The units in the device of the present invention can be combined, divided and deleted according to actual needs.

[0144] Although the present invention is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present invention. The scope of protection of the present invention is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present invention.

Claims

1. A difference map registration method based on feature point matching, characterized in that: The method comprises: S1, extracting feature points and descriptors of the first image and the second image using a feature extraction algorithm; S2, calculating the Euclidean distance between the descriptors of the first image and the second image, thereby determining the similarity of the feature points and matching the feature point pairs in the two images; S3, analyzing the geometric transformation relationship of the matching point pairs and generating an image transformation matrix; S4. Transform the second image into an image aligned with the content of the first image using an image transformation matrix.

2. The difference map registration method based on feature point matching according to claim 1, characterized in that: The step S3 comprises: S31, calculating the scaling ratio S of the first image relative to the content on the second image x and S y , where S x is the horizontal scaling value, S y is the vertical scaling value; according to the transformation relationship between the two known images, there is the following relationship between any two pairs of matching points: D1 2 =(X1 i -X1 j ) 2 +(Y1 i -Y1 j ) 2 ; D1 2 =S x 2 *(X2 i -X2 j ) 2 +S y 2 *(Y2 i -Y2 j ) 2 ; (X1 i ,Y1 i )、(X1 j ,Y1 j ) are the positions of any two feature points in the first image; (X2 i ,Y2 i )、(X2 j ,Y2 j ) are the positions of any two feature points in the second image; (X1 i ,Y1 i ) and (X2 i ,Y2 i ) is a pair of matching feature points, (X1 j ,Y1 j ) and (X2 j ,Y2 j ) is a pair of matching feature points; D1 is the two feature points in the first image (X1 i ,Y1 i ) and (X1 j ,Y1 j ), transform the feature points according to the scaling ratio so that the distance between the two feature points in the second image is equal to D1, and combine the above two equations to solve the linear fitting problem of slope k and intercept b: The matching feature point pairs in the first image and the second image are used to calculate the point (x, y) on the fitting line as follows: The combination of all two feature point pairs on the first image and the second image constitutes a lot of points on the fitting line; the fitting line obtains k and b, and thus obtains the scaling ratio S x and S y , and excludes the characteristic point pairs that are not on the fitting line; S32, according to the scaling ratio of the image calculated in step S31, the point (X2 i ,Y2 i ) is scaled and corrected to (X2 i *S x ,Y2 i *S y ), (X2 j ,Y2 j ) becomes (X2 j *S x ,Y2 j *S y ); Calculate the rotation angle A of each pair of two feature points based on the corrected points; In the first image, the calculation is (X1 i ,Y1 i ) as the starting point, with (X1 j ,Y1 j ) as the end point of the straight line angle A1, calculate the angle of (X2 i *S x ,Y2 i *S y ) as the starting point, with (X2 j *S x ,Y2 j *S y ) is the straight line angle a2 of the end point; thus the angle difference between the two lines is calculated as A=A1-A2; Calculate the rotation angle A of all two pairs of feature points in the two images. The rotation angle range is (-π,π]. Calculate the histogram of all rotation angles according to a certain step size. The angle corresponding to the highest peak of the histogram is the rotation angle to be calculated, and exclude the feature point pairs corresponding to the angles outside the peak. S33, according to the scaling value and rotation angle calculated in S31 and S32, correct the feature point in the second image to the position (X2 i ′,Y2 i ′): X2 i ′=X2 i *S x *cos(A)-Y2 i *S y *sin(A); Y2 i ′=X2 i *S x *sin(A)+Y2 i *S y *cos(A); Calculate the translation value (T ix ,T iy ): T ix =X2 i ′-X1 i ; T iy =Y2 i ′-Y1 i ; The two-dimensional vectors of the translation values ​​are divided into different clusters according to the spatial distance using the clustering method, so that the distance between the vectors in the same cluster is as close as possible, and the distance between the vectors in different clusters is as far as possible, that is, the vectors in the same cluster are clustered together, and the vectors in different clusters are separated as much as possible; then the average value of the group of vectors with the largest number of vectors is the translation position of the image (T x ,T y ); S34, calculate the image transformation matrix M, the formula is:

3. The difference map registration method based on feature point matching according to claim 1, characterized in that: The step S2 comprises: Suppose the descriptor of a feature point in the first image is: R i =(r i1 ,r i2 ,…,r i128 ); The descriptor of a feature point in the second image is: S i =(s i1 ,s i2 ,…,s i128 ); The Euclidean distance d(R i ,S i ) is calculated as: Euclidean distance is the metric for feature point matching; For each feature point, find the most similar matching point in the other image, and for each feature point descriptor R in the first image i , find the two descriptors S with the smallest Euclidean distance in the second image i and S p , these two points are R i Preliminary matching points in the second image; The matching points are screened by the nearest neighbor ratio test. The specific steps are as follows: Calculate the distance ratio between the nearest neighbor matching point and the second nearest neighbor matching point A ratio threshold is set. If the ratio is less than the threshold, the matching point is considered reliable. The lower the ratio, the greater the distance difference between the matching point and the secondary matching point, and the more reliable the matching result. If the ratio is higher than the threshold, the matching point may be a mismatch and should be removed. In this way, feature point matching pairs with higher reliability are finally left.

4. The difference map registration method based on feature point matching according to claim 1, characterized in that: The step S1 comprises: S11, constructing the scale space of the image and detecting extreme points therein; The construction of scale space is usually carried out using Gaussian pyramid. The image is smoothed at different scales to generate a series of images with different resolutions. After each layer of image is Gaussian blurred, the size is reduced to form a pyramid structure. The expression of Gaussian pyramid scale space L(x, y, σ) is: L(x,y,σ)=G(x,y,σ)*I(x,y); Among them, G(x,y,σ) is a Gaussian kernel function with variable scale, I(x,y) is the original image, and σ represents the scale; On the generated Gaussian pyramid, the Gaussian blurred images of adjacent scales are differentially operated to obtain the Gaussian difference image (DoG) to form a Gaussian difference pyramid; DoG can be approximated as the Laplacian operator in the scale space and is used to detect extreme points in the image; In each layer of the Gaussian difference pyramid, each pixel is compared with its 26 neighboring pixels in the scale space and the two-dimensional image space; if the point is the maximum or minimum value in all these neighborhoods, it is considered to be a potential feature point; S12, after detecting the potential feature points, further accurately positioning them; This step includes eliminating unstable feature points and determining the sub-pixel position of each feature point. The specific method is as follows: Through Taylor series expansion, the extreme points are interpolated and fitted at the sub-pixel level to obtain a more accurate position, significantly improving the positioning accuracy of feature points; Extreme points with low contrast are easily disturbed by noise, so they need to be removed. By comparing the contrast threshold of feature points, those key points that are not stable enough can be removed. The principal curvature of the DoG function at the edge is large, which easily generates unstable feature points. The key points are screened using the ratio of the principal curvatures to remove those low-stability feature points that cross the edge. S13, assigning a direction to each key point so that the extracted feature points have rotation invariance; In the neighborhood of the feature point, the gradient magnitude and direction of each pixel are calculated, and the gradient direction in the neighborhood of the feature point is counted into a histogram. The horizontal axis of the gradient direction histogram is the gradient direction, and the vertical axis is the cumulative value of the gradient magnitude. The gradient direction histogram is divided into 36 intervals of 10°. The peak of the histogram is the main direction of the feature point. If there are multiple peaks, multiple directions can be assigned to the feature point to enhance the robustness of the algorithm. S14, after determining the position, scale and direction of the feature point, finally generating a descriptor for each feature point, the descriptor is used to characterize the local image information of the feature point, so that it has stable matching performance in different images; According to the main direction of the feature point, the coordinate axis of its neighborhood is rotated to make the descriptor rotationally invariant; the neighborhood of the feature point is divided into 4×4 grids, and the gradient histograms of 8 directions are calculated in each grid to form a 128-dimensional feature vector, that is, a 128-dimensional descriptor.

Citation Information

Patent Citations

  • Heterologous image registration method based on shape context and HOG features

    CN110796691A

  • Planetary landing collaborative navigation feature matching method

    CN113435495A

  • Multi-source remote sensing image registration method based on improved PIIFD feature description

    CN117115214A

  • Registration method and device of point cloud data and electronic equipment

    CN117934573A

  • Image registration method

    US20180150962A1

Cited By

  • Intelligent electronic equipment program control test system based on quality monitoring information

    CN119780589A