Different-source scene matching point feature preliminary screening method fusing navigation information

Abstract

The invention discloses a different-source scene matching point feature preliminary screening method fusing navigation information, and the method comprises the steps: obtaining a real-time image, and carrying out the rough matching of feature points of the real-time image and a reference image; for the corrected real-time image and the reference image, calculating a corresponding first-order gradient amplitude image, and calculating a corresponding first-order gradient directional diagram; pixel-by-pixel mapping is carried out on the first-order gradient directional diagram; for the real-time image, determining a radius based on a preset radius parameter and a radius coefficient; constructing a histogram for the position coordinates of the feature points on the corrected real-time image; determining a main direction of each feature point, constructing main direction arrays of three regions, and determining a unique main direction array of the feature points in the real-time image and the reference image; determining a main direction difference absolute value; and constructing an index array through the absolute value of the main direction difference value and a preset matching effect index, sorting the index array, and selecting a corresponding feature point pair according to the number of required feature point pairs.

Description

Technical Field

[0001] The present invention relates to the technical field of scene matching, and in particular to a method for preliminarily screening features of heterogeneous scene matching points that fuse navigation information. Background Art

[0002] Currently, point features are commonly used in traditional heterogeneous scene matching. Scene matching in this field is typically accomplished using a three-step process: point feature extraction, coarse matching, and fine matching. First, point features are extracted, and modality-independent descriptors are extracted from these points. Then, a brute-force matching method is used to select the point with the closest descriptor distance as the matching point for coarse matching. Finally, based on the coarse matching results, fine matching is performed using methods such as RANSAC (Random Sample Consensus Algorithm) and FSC (Fast Sample Consensus Algorithm). However, due to the significant differences in descriptor information between heterogeneous images, the proportion of correctly matched point pairs in the coarse matching results is typically low, which reduces the matching accuracy of the back-end fine matching process. Furthermore, the time consumption of the fine matching process based on cyclic convergence algorithms such as RANSAC and FSC is also greatly increased in order to select the correct matching point pairs from the coarse matching results with a low proportion of correct point pairs. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for initially screening features of heterogeneous scene matching points that fuses navigation information, so as to improve the positioning accuracy of the terrain matching navigation method while having high computational efficiency.

[0004] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0005] A method for initially screening features of heterogeneous scene matching points by fusing navigation information, comprising:

[0006] Obtain a real-time image, perform rough matching of feature points between the real-time image and the reference image, and obtain a feature point set of the real-time image and the reference image;

[0007] Correcting the geographic information of the real-time map to make it consistent with the reference map; calculating the corresponding first-order gradient amplitude map and the corresponding first-order gradient direction map for the corrected real-time map and the reference map;

[0008] The first-order gradient direction maps of the real-time image and the reference image are mapped pixel by pixel to obtain the corresponding partition maps;

[0009] For the real-time image, the radius is determined based on a preset radius parameter and radius coefficient; for the position coordinates of the feature points on the corrected real-time image, the position coordinates whose distance from the position coordinates of the feature points is less than the radius are traversed and a histogram is constructed; the main direction of each feature point is determined based on the histogram and recorded in the first regional main direction array; the radius scale coefficient is adjusted twice to obtain the second and third regional main direction arrays; based on the three regional main direction arrays, a unique main direction array of the feature points in the real-time image is calculated; and the unique main direction array of the reference image is determined using the same method;

[0010] Using the unique main direction array of the real-time image and the reference image, the absolute value of the main direction difference is determined;

[0011] An index array is constructed and sorted by the absolute value of the main direction difference and the preset matching effect index, and the corresponding feature point pairs are selected according to the required number of feature point pairs.

[0012] Furthermore, the first-order gradient direction maps of the real-time image and the reference image are mapped pixel by pixel to obtain corresponding partition maps, including:

[0013] Corrected real-time graph I rk , Reference Map I b Calculate the first-order gradient direction map I ar , I ab ;

[0014] For two first-order gradient patterns I ar with I ab Perform pixel-by-pixel mapping and calculate the partition maps respectively The first-order gradient pattern I ar Mapping to get partition map The process is:

[0015] For the first-order gradient pattern I ar Each value in is multiplied by the parameter O and the result is divided by 360 and rounded; where O is an adjustable threshold.

[0016] Furthermore, for the real-time graph, the radius is determined based on a preset radius parameter and a radius coefficient, including:

[0017] Set the radius parameter R0 and the radius coefficient r1, then the radius R = R0·t1.

[0018] Furthermore, traversing the position coordinates whose distance from the position coordinates of the feature point is less than the radius and constructing a histogram includes:

[0019] Initialize the qth element of the histogram hist_r1 hist_r1[q]={0},q∈[1,O], and the histogram is an array of size 1*O;

[0020]

[0021] in, Indicates the position coordinates (x, y) in the real-time graph corresponding to the distinction map The upper position, Express Perform statistical histogram operations, I gr (x,y) represents the first-order gradient amplitude map I corresponding to the position coordinate (x,y) in the real-time map gr The upper position, Indicates the histogram hist_r1 The element corresponding to the position.

[0022] Furthermore, the main direction of each feature point is determined based on the histogram and recorded in the first region main direction array, including:

[0023] Find the indexes O1 and O2 of the highest and second highest values ​​in all array elements of the histogram hist_r1, and then calculate the main direction of the i-th feature point in the real-time image as follows:

[0024]

[0025] Among them, A r1 (i) represents the main direction array A r1 The i-th element in represents the main direction of the i-th feature point.

[0026] Furthermore, the radius proportional coefficient is adjusted twice, wherein the value of each adjustment is larger than the previous value.

[0027] Furthermore, the unique main direction array A of the feature points in the real-time image and the reference image r ,A B It is expressed as follows:

[0028]

[0029] Among them, the three main direction arrays of the feature points in the real-time image are A r1 、A r2 、A r3 ; The three main direction arrays of feature points on the reference image are A B1 、A B2 、A B3 ; 0.56, 0.77, and 1 are the three values ​​of the radius coefficient.

[0030] Furthermore, according to the unique area main direction array A of the feature point r ,A B , the absolute value of the calculated main direction difference of the i-th group of feature point pairs:

[0031] E(i)=|A B (i)-A r (i)|

[0032] Among them, A B (i),A r (i) represents the unique area main direction array A r ,A B The value of the i-th element in .

[0033] Furthermore, an index array is constructed and sorted using the absolute value of the main direction difference and a preset matching effect index, including:

[0034] Determine the absolute value of the main direction difference of the i-th group of feature point pairs and construct the index array M:

[0035]

[0036] Where m is an adjustable threshold, and M(i) is the value of the i-th element in the indicator array M;

[0037] First, the feature point pairs corresponding to the 0 element value in the indicator array M are eliminated, and then the remaining feature point pairs are sorted from large to small according to the element value M(i); finally, according to the number of required feature point pairs, they are selected from the sorting from front to back to achieve coarse matching.

[0038] A terminal device comprises a processor, a memory and a computer program stored in the memory; when the processor executes the computer program, a method for initially screening features of heterogeneous scene matching points of fused navigation information is implemented.

[0039] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method for initially screening features of heterogeneous scene matching points of fused navigation information is implemented.

[0040] Compared with the prior art, the present invention has the following technical features:

[0041] 1. The present invention provides a novel method and index for preliminary screening of point feature rough matching results, which can complete the preliminary screening or sorting of the rough matching steps and improve the matching accuracy of the rough matching process by 42%.

[0042] 2. Compared with the traditional method of determining feature point response values, the accuracy of the initial screening matching points will be higher, reducing the difficulty of back-end fine matching and shortening the image matching process by 14% to 33%.

[0043] 3. The method proposed in the present invention is applicable to all point features and has wide applicability, and can introduce a rough matching step for all point features.

[0044] 4. The present invention has a better effect on heterogeneous image matching. The number of correct matching point pairs after rough matching of heterogeneous images is generally low. When relying on traditional descriptor matching response values ​​for screening, there will be a large number of false matches. The initial screening method proposed in the present invention can improve the accuracy after screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the process of the present invention;

[0046] Figure 2 A set of input example images in one embodiment of the method of the present invention (rough matching results of feature points between the real-time image and the reference image);

[0047] Figure 3 For Figure 2 The primary screening results obtained after processing using the method of the present invention. DETAILED DESCRIPTION

[0048] The present invention designs a method for initial screening of heterogeneous scene matching point features that integrates navigation information for the coarse matching process based on point features. This method can improve the accuracy of matching point pairs in the coarse matching step of heterogeneous scene matching, thereby significantly improving the accuracy of the subsequent fine matching step while reducing the time consumption of the subsequent fine matching step. Affected by factors such as scene, time of day, weather, and information source differences, the descriptor information of heterogeneous scene matching point features usually differs greatly, resulting in a low accuracy rate in the coarse matching results. To address this problem, the algorithm integrates navigation information to design a method for initial screening of point features. First, the attitude and relative height information in the navigation information are used to complete the coordinate system of the real-time image and the reference image. Furthermore, a method for calculating the weighted main direction of the matching point pairs is designed. Finally, considering that the difference between the correct main directions of the two is 0 after the fusion of navigation information, the present invention designs a matching effect indicator for the coarse matching point pairs based on the weighted main direction.

[0049] Algorithm input: First, the coordinate system of the real-time image and the reference image is completed using the posture and relative height information in the navigation information, and the feature point pairs after matching the real-time image and the reference image are obtained through rough matching of feature points.

[0050] The method of the present invention operates after the coarse matching step and before the fine matching step in the feature matching algorithm. In this solution, the two matched images are referred to as the real-time image and the reference image. After the coarse matching, the feature point set of the real-time image is ptRt, and the feature point set of the reference image is ptBenc. ptRt[i] is the i-th feature point in the feature point set ptRt, and ptBenc[i] is the i-th feature point in the feature point set ptBenc. N is the number of feature point pairs in the coarse matching. Based on this, the specific steps of the algorithm are as follows:

[0051] Step 1: Correction of real-time image and reference image.

[0052] Based on the inertial navigation attitude and relative flight altitude of the image acquisition platform when acquiring the real-time image, the real-time image can be corrected for geographic information so that the angle and scale of the corrected real-time image are consistent with the reference image; the corrected real-time image is denoted as I rk .

[0053] This step is a mature step in this field, as detailed in the article "A Geographic Information Correction Method Based on Inertial / Altimeter / Vision Fusion" in Issue 10 of the Journal of Chinese Inertial Technology.

[0054] Step 2: Calculate the gradient magnitude map.

[0055] Corrected real-time graph I rk Calculate the first-order gradient amplitude map, denoted as I gr . For reference image I b Calculate the first-order gradient magnitude map I gb .

[0056] Step 3: Calculate the gradient direction map.

[0057] Corrected real-time graph I rk Calculate the first-order gradient direction map, denoted as I ar ; For reference image I b Calculate the first-order gradient direction map I ab ; For two first-order gradient patterns I ar with I ab Perform pixel-by-pixel mapping and calculate the partition maps respectively The mapping process is as follows:

[0058]

[0059] The calculation process can be described as follows: ar Each value in is multiplied by parameter O and the result is divided by 360 and then rounded; where O is an adjustable threshold, which is set to 24 in this embodiment, and rand represents the rounding process.

[0060] Step 4: Calculate the weighted main direction of each set of feature point pairs.

[0061] For the i The feature point pair ptRt[i], ptBenc[i], which is in real-time graph I after correction rk With reference figure I b The position coordinates on are recorded as Initialize the main direction arrays of the three regions of the feature points on the real-time map as A r1 、A r2 、Ar3 ; Initialize the three main direction arrays of the feature points on the reference image, respectively A B1 、A B2 、A B3 ;The array size is 1*N, and all array values ​​are initialized to zero.

[0062] Set the radius parameter R0 = 70, and the steps for calculating the main direction of the i-th feature point on the real-time image are as follows:

[0063] Step 4.1, initialize the qth element of the histogram hist_r1 hist_r1[q]={0},q∈[1,O], the histogram is an array of size 1*O, that is, all elements of the array are initialized to 0.

[0064] Step 4.2: Set the radius coefficient t1 = 0.56 and calculate the radius based on the radius parameter R0:

[0065] R=R0·t1

[0066] Step 4.3, for the corrected real-time image I rk The position coordinates of the upper feature point Traversal distance The position coordinates (x, y) less than the radius R, statistical histogram:

[0067]

[0068] in, Indicates the position coordinates (x, y) in the real-time graph corresponding to the distinction map The upper position, Express Perform statistical histogram operations, I gr (x,y) represents the first-order gradient amplitude map I corresponding to the position coordinate (x,y) in the real-time map gr The upper position, Indicates the histogram hist_r1 The element corresponding to the position.

[0069] Step 4.4: Find the indexes O1 and O2 of the highest and second highest values ​​in all array elements of the histogram hist_r1. The main direction of the i-th feature point in the real-time image is calculated as follows:

[0070]

[0071] Among them, A r1 (i) represents the main direction array A r1 The i-th element in represents the main direction of the i-th feature point.

[0072] Step 4.5 Modify the radius coefficient t1 in step 4.2 to 0.77, and repeat steps 4.1 to 4.4 to calculate the new main direction A of the i-th feature point r2 (i); Similarly, let the radius ratio coefficient t1 = 1.00 and calculate the new main direction A of the i-th feature point r3 (i)

[0073] Through the above steps, by traversing i from the coarsely matched feature point pairs 1 to N, three regional main direction arrays of all feature points in the real-time image can be obtained.

[0074] Step 4.6: Replace the real-time image with the reference image. Use the same method as steps 4.1-4.5 to calculate the three main direction arrays A of all feature points on the reference image. B1 、A B2 、A B3 ; The histogram corresponding to the real-time image is initialized in step 4.1; when calculating in step 4.3, the distinction image corresponding to the reference image is used And the first-order gradient amplitude map I corresponding to the reference map gb .

[0075] Step 4.7: Weight the main direction arrays of the three regions of the real-time image and the reference image respectively to calculate the unique main direction array A of the feature points in the real-time image and the reference image. r ,A B :

[0076]

[0077] Different from the traditional point feature main direction calculation rules such as SURF, SIFT, SAR-SIFT, etc., the present invention takes the feature point position as the center, uses circular areas of different radii to calculate multiple main directions, and further performs weighted fusion on multiple main directions to construct a new main direction calculation rule.

[0078] Step 5: Calculate the main direction difference of the matching point pair.

[0079] The unique regional main direction array A of the feature points calculated according to step 4 r ,A B , the absolute value of the calculated main direction difference of the i-th group of feature point pairs:

[0080] E(i)=|A B (i)-A r (i)|

[0081] Among them, A B (i),A r (i) represents the unique area main direction array A r ,A BThe value of the i-th element in .

[0082] Step 6: Calculate the matching effect index.

[0083] Determine the absolute value of the main direction difference of the i-th group of feature point pairs and construct the index array M:

[0084]

[0085] Where m is an adjustable threshold value, which is set to 60 in this embodiment, and M(i) is the value of the i-th element in the indicator array M; that is, this scheme uses the auxiliary navigation information to correct the input image as prior information, and the prior information is specifically described as: the absolute value of the main direction difference of the feature point pair is 0.

[0086] Step 7: Rough matching to complete the initial screening of feature point pairs.

[0087] The indicators in step 6 are used as the indicators for determining whether the matching is successful or not; first, the feature point pairs corresponding to the 0 element value in the indicator array M are eliminated, and then the remaining feature point pairs are sorted from large to small according to the element value M(i); finally, the number of feature point pairs required is selected from the sorting from front to back to achieve a rough match.

[0088] Combine Figure 2 and Figure 3 The method of the present invention can perform preliminary screening of the rough matching results, which will greatly improve the accuracy of the matching point pairs obtained in the rough matching process, thereby reducing the complexity of the back-end fine matching process; at the same time, the time consumption of the method proposed in the present invention is very low, which greatly improves the image matching time.

[0089] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for initially screening features of heterogeneous scene matching points by integrating navigation information, characterized in that: include: Obtain a real-time image, perform rough matching of feature points between the real-time image and the reference image, and obtain a feature point set of the real-time image and the reference image; Correct the geographic information of the real-time map to make it consistent with the reference map; For the corrected real-time image and the reference image, calculating the corresponding first-order gradient amplitude image and the corresponding first-order gradient direction image; The first-order gradient direction maps of the real-time image and the reference image are mapped pixel by pixel to obtain the corresponding partition maps; For real-time graphs, the radius is determined based on the preset radius parameters and radius coefficient; For the position coordinates of the feature points on the corrected real-time image, traverse the position coordinates whose distance from the position coordinates of the feature points is less than the radius, and construct a histogram; Based on the histogram, the main direction of each feature point is determined and recorded in the first regional main direction array. The radius scale factor is adjusted twice to obtain the second and third regional main direction arrays. Based on the three regional main direction arrays, a unique main direction array of the feature point in the real-time image is calculated. The same method is used to determine the unique main direction array of the reference image. Using the unique main direction array of the real-time image and the reference image, the absolute value of the main direction difference is determined; An index array is constructed and sorted by the absolute value of the main direction difference and the preset matching effect index, and the corresponding feature point pairs are selected according to the required number of feature point pairs.

2. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: The first-order gradient direction maps of the real-time image and the reference image are mapped pixel by pixel to obtain the corresponding partition maps, including: Corrected real-time graph I rk , Reference Map I b Calculate the first-order gradient direction map I ar , I ab ; For two first-order gradient patterns I ar with I ab Perform pixel-by-pixel mapping and calculate the partition maps respectively The first-order gradient pattern I ar Mapping to get partition map The process is: For the first-order gradient pattern I ar Each value in is multiplied by the parameter O and the result is divided by 360 and rounded; where O is an adjustable threshold.

3. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: For real-time graphs, the radius is determined based on preset radius parameters and radius coefficients, including: Set the radius parameter R0 and the radius coefficient r1, then the radius R = R0·t1.

4. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: Traversing the position coordinates whose distance from the position coordinates of the feature point is less than the radius, and constructing a histogram, including: Initialize the qth element of the histogram hist_r1 hist_r1[q]={0},q∈[1,O], and the histogram is an array of size 1*O; in, Indicates the position coordinates (x, y) in the real-time graph corresponding to the distinction map The upper position, Express Perform statistical histogram operations, I gr (x,y) represents the first-order gradient amplitude map I corresponding to the position coordinate (x,y) in the real-time map gr The upper position, Indicates the histogram hist_r1 The element corresponding to the position.

5. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: Determine the main direction of each feature point based on the histogram and record it in the first region main direction array, including: Find the indexes O1 and O2 of the highest and second highest values ​​in all array elements of the histogram hist_r1, and then calculate the main direction of the i-th feature point in the real-time image as follows: Among them, A r1 (i) represents the main direction array A r1 The i-th element in represents the main direction of the i-th feature point.

6. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: The radius scale factor is adjusted twice, with each adjustment value being larger than the previous one.

7. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: Unique main direction array A of feature points in real-time image and reference image r ,A B It is expressed as follows: Among them, the three main direction arrays of the feature points in the real-time image are A r1 、A r2 、A r3 ; The three main direction arrays of feature points on the reference image are A B1 、A B2 、A B3 ; 0.56, 0.77, and 1 are the three values ​​of the radius coefficient.

8. The method for initially screening features of heterogeneous scene matching points for fused navigation information according to claim 1, characterized in that: Construct an indicator array and sort it using the absolute value of the main direction difference and the preset matching effect indicator, including: Determine the absolute value of the main direction difference of the i-th group of feature point pairs and construct the index array M: Where m is an adjustable threshold, and M(i) is the value of the i-th element in the indicator array M; First, the feature point pairs corresponding to the 0 element value in the indicator array M are eliminated, and then the remaining feature point pairs are sorted from large to small according to the element value M(i); finally, according to the number of required feature point pairs, they are selected from the sorting from front to back to achieve coarse matching.

9. A terminal device comprising a processor, a memory, and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the method for initially screening features of heterogeneous scene matching points of fused navigation information according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program; wherein: When the computer program is executed by a processor, the method for initially screening features of heterogeneous scene matching points of fused navigation information according to any one of claims 1 to 8 is implemented.

Images