Robust region adaptive different-source image matching method

Abstract

The invention discloses a robust region self-adaptive different-source image matching method in the technical field of microwave remote sensing. The robust region self-adaptive different-source image matching method comprises the following steps of extracting region self-adaptive feature points based on VE-FAST; generating a similarity graph matched with the regional features; and carrying out matching point screening based on SK-NMS. According to the robust region adaptive different-source image matching method, image texture features are combined, two links of feature point extraction and matching point screening are started, weak feature regions can be effectively identified, and rapid, robust and high-precision matching of SAR / optical images in the weak feature regions is realized.

Description

Technical Field

[0001] The invention relates to the field of microwave remote sensing technology, and in particular to a robust regional adaptive heterogeneous image matching method. Background Art

[0002] Current remote sensing technology can achieve system-level geometric correction of images through platform positioning parameters, effectively eliminating the rotation and scale differences between SAR (synthetic aperture radar) images and optical images. Cutting-edge algorithms tend to combine region-based and feature-based methods to develop feature-based template matching algorithms. This type of template matching algorithm usually uses similar local structural features between heterogeneous images to extract a set of matching point pairs to construct a geometric transformation model. Studies have shown that compared with region-based methods, methods based on the combination of template regions and features can achieve higher-precision SAR / heterogeneous image matching. This type of method usually includes three main steps: feature point extraction, regional feature matching, and matching point pair screening. Depending on the feature descriptors used, common classic algorithms include: the directional gradient feature proposed by Ye Y et al., and AWOG, which improves the accuracy of gradient calculation on this basis; LNIFT using ORB and HOG-like, SRIF with scale invariance based on LNIFT, and LPHOG that combines line features and point features and uses HOG-like descriptors; the main registration algorithms for SAR and optical images that combine the SIFT algorithm in the field of computer vision include PSO-SIFT, OS-SIFT, and RTV-SIFT; the main registration algorithms using phase consistency include HOPC, FED-HOPC, etc.

[0003] When the observation area is rich in objects and has significant structural and texture features, algorithms such as CFOG can effectively achieve high-precision matching of SAR / optical heterogeneous images. However, in practical applications, it is often necessary to process images of weak feature areas (sparse structural features) such as ports and islands with large water areas and deserts with sparse structural features. At this time, traditional matching algorithms will generate redundant and meaningless image description points in these weak feature areas, and even common outlier feature point screening algorithms may not be able to effectively remove these points, resulting in a significant decrease in the performance of traditional algorithms. Differences in sensor imaging mechanisms, observation angles, and scattering characteristics of objects are key factors that limit the matching performance of SAR / optical heterogeneous images. The block Harris algorithm is the most commonly used feature point extraction algorithm in heterogeneous image matching algorithms. It can evenly extract a large number of feature points. However, the algorithm only relies on the significance of structural features and ignores the regional differences of weak feature objects. It will also extract a certain number of feature points on the water surface and desert Gobi. These feature points are usually randomly distributed. Even if they are mostly eliminated after regional feature matching and traditional non-maximum matching point screening, there are still a small number of false same-name point pairs in weak feature areas such as water surfaces. Feature point extraction algorithms that do not use block strategies, such as FAST, will not extract feature points in weak feature areas, but are prone to clustering, reducing the uniformity of feature point distribution. These factors are likely to reduce the accuracy of the geometric transformation model. The robustness of traditional matching algorithms is greatly reduced in weak feature areas such as ports, islands, and desert Gobi with large areas of water. Summary of the invention

[0004] In order to solve the above problems, the purpose of the present invention is to provide a robust regional adaptive heterogeneous image matching method to solve the technical problems in the prior art that the robustness of traditional matching algorithms is large in weak feature areas such as ports, islands and desert Gobi with large water areas, and the accuracy of the full-image geometric transformation model is low. Combining image texture features from the two links of feature point extraction and matching point screening, the weak feature areas can be effectively identified, and fast, robust and high-precision matching of SAR / optical images in weak feature areas can be achieved.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows:

[0006] The present invention provides a robust region-adaptive heterogeneous image matching method, comprising the following steps:

[0007] S1. Regional adaptive feature point extraction based on VE-FAST;

[0008] S2. Similarity graph generation based on regional feature matching;

[0009] S3. Matching point screening based on SK-NMS.

[0010] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, S1 includes the following steps:

[0011] S11. Introduce a block strategy to divide the image into N×N sub-image blocks, record the FAST feature point score of each sub-image block, and arrange them in descending order;

[0012] S12. According to the information theory method, according to the information entropy H I Calculation formula: For each sub-image block, the information entropy is greater than the set threshold λ e The first m u points as candidate points, and if they do not meet the set information entropy threshold, m points are extracted. w candidate points;

[0013] S13. Determine the template area and the search area based on each candidate point, and calculate the variance product Q of the template area and the search area corresponding to each candidate point p ;

[0014] S14. Based on the variance product Q p and variance product threshold λ v The size relationship is based on the extraction of feature points that are adaptive and evenly distributed in the region.

[0015] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, H in S12 I The calculation formula is as follows:

[0016]

[0017] Among them, p i is the probability of the i-th level grayscale appearing, i∈[1,k].

[0018] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, the variance product Q of the template region corresponding to each candidate point in S13 and the search region is p The calculation formula is as follows:

[0019]

[0020] Among them, V o is the variance of the template area, is the minimum variance of all template regions, is the maximum variance of the template area, V s is the variance of the search area, is the minimum variance of all search areas, is the maximum variance of the search area;

[0021] For size M I ×N I For an image I(x,y), the variance V is defined as follows:

[0022]

[0023] Among them, μ I is the pixel average value of the image, which is obtained by the following calculation formula (4):

[0024]

[0025] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, S2 includes the following steps:

[0026] S21. Referring to the template area of ​​the optical image and the search area of ​​the SAR image to be matched, construct feature descriptors respectively using MOGF;

[0027] S22. Generate a similarity graph based on the literature on the research of UAV SAR heterogeneous image matching and application technology.

[0028] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, S21 includes the following steps:

[0029] S211. Use the filter to obtain the gradient size IG and direction OG of the image in the x direction and y direction;

[0030] S212. Considering the robustness of the algorithm, the Sobel operator considering comprehensive factors is used to extract gradient information for optical images, and the ROEWA operator is used to consider the influence of multiplicative speckle noise for SAR images;

[0031] S213. Use the following calculation formula to obtain the gradient vector of each pixel:

[0032]

[0033] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, S3 includes the following steps:

[0034] S31. Through the ratio of primary to secondary peaks R and the skewness S of the similarity graph k The calculation formula is used to calculate the ratio of the primary and secondary peaks of the similarity graph R and the skewness S of the similarity graph respectively. k , and set the threshold λ r and λ s ;

[0035] S32. When the ratio R of the primary and secondary peaks of the similarity graph is greater than λ r And the skewness Sk Greater than λ s When , it is considered that the maximum value of the similarity graph is suitable for calculating the matching points, so as to obtain a reliable matching point pair;

[0036] S33. By processing and screening the similarity graph, a set of matching point pairs is obtained;

[0037] S34. Starting from the geometric transformation relationship model of matching point pairs, the FSC method is used to eliminate outlier matching points.

[0038] As one aspect of a robust region-adaptive heterogeneous image matching method of the present invention, the ratio R of the primary and secondary peaks in S31 and the skewness S of the similarity graph are k The calculation formula is as follows:

[0039]

[0040] Among them, IoU is the intersection-over-union ratio of the main peak window and the secondary peak window, ε is the machine zero introduced to prevent the intersection-over-union ratio from being 0, which is a number close to 0 but greater than 0, and P p is the main peak intensity, P s is the sub-peak intensity;

[0041]

[0042] in, is the mean, x i is the i-th element, and n is the number of samples.

[0043] By adopting the above technical solution, the present invention has the following advantages:

[0044] The present invention provides a robust regional adaptive heterogeneous image matching method, which effectively identifies weak feature areas through the dual threshold criteria of regional information entropy and variance product, appropriately controls the number of feature points in weak feature areas, avoids extracting feature points in weak feature areas as much as possible, and protects a small number of effective areas therein as much as possible. The present invention combines the matching point pair screening strategy of skewness and non-maximum suppression, combines the non-maximum suppression method, introduces the skewness concept of the similarity graph, effectively eliminates false matching points distributed in weak feature areas, and improves the accuracy of the geometric transformation model. Introducing a regional adaptive processing mechanism and a more comprehensive matching point screening strategy in the heterogeneous image matching method can determine whether a local area is a weak feature area such as water area, desert, Gobi, etc., so as to control the number of feature points and matching point pairs in such land feature areas. At the same time, since islands, scattered villages and roads are often scattered in large areas of weak feature images, and such land features are usually effective areas for extracting high-reliability feature points and matching to obtain matching points, the present invention can retain such small-area effective areas and obtain matching point pairs, thereby achieving the matching accuracy of the entire image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the robust region-adaptive heterogeneous image matching method of the present invention;

[0046] Figure 2 It is a flow chart of the VE-FAST feature point extraction algorithm of the present invention;

[0047] Figure 3 A flow chart of matching point screening based on SK-NMS of the present invention;

[0048] Figure 4 The measured data of airborne SAR are typical weak feature areas with a small number of islands scattered in the waters;

[0049] Figure 5 The figure shows the comparison of matching effects of a typical weak feature image before and after correction using the method of the present invention. DETAILED DESCRIPTION

[0050] The technical solution of the present invention is described in detail below in conjunction with the drawings in the specification. It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0051] Figure 1 The flowchart of the robust region adaptive heterogeneous image matching method is shown in Figure 1. It mainly includes three parts: (i) region adaptive feature point extraction; (ii) region feature matching (obtaining similarity graph); (iii) matching point pair screening. Figure 1 As shown in the figure. For a pair of SAR-optical heterogeneous images, the proposed VE-FAST feature point detection algorithm is first used to extract evenly distributed key points. Then, the template area and search area are determined with the extracted key points as the center, and the feature descriptor is constructed to obtain a similarity graph. Finally, the proposed SK-NMS (Skewness-Non-maximal suppression) same-name point screening algorithm is used to filter out mismatched point pairs and establish a correct geometric transformation model.

[0052] The present invention provides a robust region-adaptive heterogeneous image matching method, comprising the following steps:

[0053] S1. Regional adaptive feature point extraction based on VE-FAST;

[0054] Figure 2 The flowchart of the feature point extraction algorithm of VE-FAST (Variance and Entropy-Features from Accelerated Segment Test) is shown.

[0055] S1 includes the following steps: Figure 2 As shown:

[0056] S11. Introduce a block strategy to divide the image into N×N sub-image blocks, record the FAST feature point score of each sub-image block, and arrange them in descending order;

[0057] S12. According to the information theory method, according to the information entropy H I The calculation formula, the size of the information entropy basically reflects the texture transformation of each sub-image, and the information entropy of the weak feature area is usually small. For each sub-image block, the information entropy is greater than the set threshold λ e The first m u points as candidate points. In order to avoid the small effective feature area being ignored, m points are extracted from the sub-image blocks that do not meet the set information entropy threshold. w (m u >m w ) candidate points; in order to increase the information entropy threshold λ e To avoid the deviation caused by the large difference in the values ​​of different features, step S12 calculates the information entropy H of the sub-image block. I When , the image is first normalized to a grayscale image within [0,1].

[0058]

[0059] Among them, p i is the probability of the i-th level grayscale appearing, i∈[1,k].

[0060] S13. Determine the template area and the search area based on each candidate point, and calculate the variance product Q of the template area and the search area corresponding to each candidate point p ;

[0061] Among them, the variance product Q of the template area and the search area corresponding to each candidate point in S13 p The calculation formula is as follows:

[0062]

[0063] Among them, V o is the variance of the template area, is the minimum variance of all template regions, is the maximum variance of the template area, V s is the variance of the search area, is the minimum variance of all search areas, is the maximum variance of the search area;

[0064] For size M I ×N I For an image I(x,y), the variance V is defined as follows:

[0065]

[0066] Among them, μ I is the pixel average value of the image, which is obtained by the following calculation formula (4):

[0067]

[0068] S14. Based on the variance product Q p and variance product threshold λ v The size relationship is that when Q p Greater than λ v , the candidate point is considered to be located in a feature-rich area, and thus is determined as a feature point; the variance product Q p Less than the variance product threshold λ v The candidate points are eliminated, and the feature points that are region-adaptive and evenly distributed are extracted, thereby achieving region adaptation.

[0069] S2. Similarity graph generation based on regional feature matching;

[0070] Wherein S2 includes the following steps:

[0071] S21. After the feature points are extracted, the template matching method needs to construct feature descriptors for the template area of ​​the reference optical image and the search area of ​​the SAR image to be matched. Considering that the scale and rotation differences between the images have been basically eliminated. The template area of ​​the reference optical image and the search area of ​​the SAR image to be matched are constructed using MOGF, which has better computational efficiency and matching performance;

[0072] Wherein S21 comprises the following steps:

[0073] S211. Use the filter to obtain the gradient size IG and direction OG of the image in the x direction and y direction;

[0074] S212. Considering the robustness of the algorithm, the Sobel operator considering comprehensive factors is used to extract gradient information for optical images, and the ROEWA operator is used to consider the influence of multiplicative speckle noise for SAR images;

[0075] S213. Use the following calculation formula to obtain the gradient vector of each pixel:

[0076]

[0077] S22. According to the third chapter of the method in "Research on UAV SAR Heterogeneous Image Matching and Application Technology, Tong Wenhao, National University of Defense Technology, 2024", a similarity graph can be generated. By locating the maximum value in the similarity graph, the matching point pairs can be determined, and then the geometric transformation relationship model can be derived to complete the image matching. .

[0078] Figure 3 A flow chart of matching point screening based on SK-NMS of the present invention is shown.

[0079] S3. Matching point screening based on SK-NMS.

[0080] S3 includes the following steps: Figure 3 As shown:

[0081] S31. Through the ratio of primary to secondary peaks R and the skewness S of the similarity graph k The calculation formula is used to calculate the ratio of the primary and secondary peaks of the similarity graph R and the skewness S of the similarity graph respectively. k , and set the threshold λ r and λ s ;

[0082] Among them, the ratio of the primary and secondary peaks in S31 R and the skewness S of the similarity graph k The calculation formula is as follows:

[0083]

[0084] Among them, IoU is the intersection over union ratio of the main peak window and the secondary peak window, and ε is the machine epsilon introduced to prevent the intersection over union ratio from being 0. It is a number close to 0 but greater than 0. p is the main peak intensity, P s is the sub-peak intensity.

[0085]

[0086] in, is the mean, x i is the i-th element, and n is the number of samples.

[0087] S32. When the ratio R of the primary and secondary peaks of the similarity graph is greater than λ r And the skewness S k Greater than λ sWhen , it is considered that the maximum value of the similarity graph is suitable for calculating the matching points, so as to obtain a reliable matching point pair;

[0088] S33. By processing and screening the similarity graph, a set of matching point pairs is obtained;

[0089] S34. Starting from the geometric transformation relationship model of matching point pairs, the FSC method is used to eliminate outlier matching points.

[0090] The measured data used in the present invention comes from airborne SAR, specifically Figure 4 As shown, Figure 4 (a) is a typical weak feature area of ​​airborne SAR measured data 1. Figure 4 (b) is a schematic diagram of the airborne SAR measured data 2, showing a small number of islands scattered in the water. Figure 4 It can be seen that a small number of islands are scattered in the measured data in a large area with weak features. Figure 5 To compare the matching effects of typical weak feature images before and after correction using the method of the present invention, Figure 5 It can be seen that the SAR image corrected by this method has an accurate correspondence with the optical image.

[0091] Finally, it should be pointed out that although the present invention has been described with reference to the current specific embodiments, ordinary technicians in this technical field should realize that the above embodiments are only used to illustrate the present invention and are not used as a limitation of the present invention. Various equivalent changes or substitutions can be made without departing from the concept of the present invention. Therefore, as long as the changes and modifications to the above embodiments are within the essential spirit of the present invention, they will fall within the scope of the claims of the present invention.

Claims

1. A robust region-adaptive heterogeneous image matching method, It is characterized in that The following steps are involved: S1. Regional adaptive feature point extraction based on VE-FAST; S2. Similarity graph generation based on regional feature matching; S3. Matching point screening based on SK-NMS.

2. A robust region-adaptive heterogeneous image matching method according to claim 1, It is characterized in that The S1 comprises the following steps: S11. Introduce a block strategy to divide the image into N×N sub-image blocks, record the FAST feature point score of each sub-image block, and arrange them in descending order; S12. According to the information theory method, according to the information entropy H I Calculation formula: For each sub-image block, the information entropy is greater than the set threshold λ e The first m u points as candidate points, and if they do not meet the set information entropy threshold, m points are extracted. w candidate points; S13. Determine the template area and the search area based on each candidate point, and calculate the variance product Q of the template area and the search area corresponding to each candidate point p ; S14. Based on the variance product Q p and variance product threshold λ v The size relationship is based on the extraction of feature points that are adaptive and evenly distributed in the region.

3. A robust region-adaptive heterogeneous image matching method according to claim 2, It is characterized in that The H in S12 I The calculation formula is as follows: Among them, p i is the probability of the i-th level grayscale appearing, i∈[1,k].

4. A robust region-adaptive heterogeneous image matching method according to claim 2, It is characterized in that The variance product Q of the template area and the search area corresponding to each candidate point in S13 p The calculation formula is as follows: Among them, V o is the variance of the template area, is the minimum variance of all template regions, is the maximum variance of the template area, V s is the variance of the search area, is the minimum variance of all search areas, is the maximum variance of the search area; For size M I ×N I For an image I(x,y), the variance V is defined as follows: Among them, μ I is the pixel average value of the image, which is obtained by the following calculation formula (4):

5. A robust region-adaptive heterogeneous image matching method according to claim 1, It is characterized in that The S2 comprises the following steps: S21. Referring to the template area of ​​the optical image and the search area of ​​the SAR image to be matched, construct feature descriptors respectively using MOGF; S22. Generate a similarity graph based on the literature on the research of UAV SAR heterogeneous image matching and application technology.

6. A robust region-adaptive heterogeneous image matching method according to claim 5, It is characterized in that The S21 comprises the following steps: S211. Use the filter to obtain the gradient size IG and direction OG of the image in the x direction and y direction; S212. Considering the robustness of the algorithm, the Sobel operator considering comprehensive factors is used to extract gradient information for optical images, and the ROEWA operator is used to consider the influence of multiplicative speckle noise for SAR images; S213. Use the following calculation formula to obtain the gradient vector of each pixel:

7. A robust region-adaptive heterogeneous image matching method according to claim 1, It is characterized in that The S3 comprises the following steps: S31. Through the ratio of primary to secondary peaks R and the skewness S of the similarity graph k The calculation formula is used to calculate the ratio of the primary and secondary peaks of the similarity graph R and the skewness S of the similarity graph respectively. k , and set the threshold λ r and λ s ; S32. When the ratio R of the primary and secondary peaks of the similarity graph is greater than λ r And the skewness S k Greater than λ s When , it is considered that the maximum value of the similarity graph is suitable for calculating the matching points, so as to obtain a reliable matching point pair; S33. By processing and screening the similarity graph, a set of matching point pairs is obtained; S34. Starting from the geometric transformation relationship model of matching point pairs, the FSC method is used to eliminate outlier matching points.

8. A robust region-adaptive heterogeneous image matching method according to claim 7, It is characterized in that The ratio R of the primary and secondary peaks in S31 and the skewness S of the similarity graph k The calculation formula is as follows: Among them, IoU is the intersection-over-union ratio of the main peak window and the secondary peak window, ε is the machine zero introduced to prevent the intersection-over-union ratio from being 0, which is a number close to 0 but greater than 0, and P p is the main peak intensity, P s is the sub-peak intensity; in, is the mean, x i is the i-th element, and n is the number of samples.

Images