An automatic selection method of gradient amplitude threshold for heterogeneous remote sensing image matching

Abstract

The present invention discloses a method for automatically selecting a gradient amplitude threshold for matching heterogeneous remote sensing images. The method selects the optimal gradient amplitude threshold parameter based on the distribution of significant contours in the image scene: first, the gradient amplitude is calculated; second, the gradient is thresholded within a set gradient amplitude threshold range; then, sub-pixel contours are extracted, and the average contour length and distribution range are statistically analyzed, with the average contour length and distribution range used as threshold evaluation indicators; finally, the gradient amplitude threshold corresponding to the highest evaluation indicator is selected as the optimal threshold parameter. The present invention is applicable to the field of computer vision. The optimal gradient amplitude threshold parameter can be automatically obtained by simply setting the image to be matched, reducing the user's professional technical requirements, improving the automation level of the matching system, and further enhancing the stability and accuracy of the matching.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, in particular to a method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching, which can be used for applications such as image fusion and visual navigation. Background Art

[0002] Optical, synthetic aperture radar (SAR), and infrared imaging modalities are commonly used in space remote sensing. The image information they provide about the same observation scene is complementary, making image fusion of these two modalities valuable. A prerequisite for image fusion is the alignment of the two images to be fused. Using heterogeneous image matching, we can calculate the alignment mapping between corresponding points in the two images to be fused.

[0003] Due to the large contrast differences between heterogeneous images, matching can usually be performed using features such as gradients that have a weak correlation with the imaging modality. Heterogeneous image matching methods based on gradient features require setting key parameters such as gradient thresholds and gradient direction intervals. These parameters are closely related to the scene of the image to be matched, and the correctness of parameter selection has a significant impact on the matching results. Existing matching methods usually require manual testing and selection of optimal parameters based on different scenes to ensure the accuracy and stability of the matching. Manual parameter selection places high demands on the operator and is not suitable for image matching tasks in scenes that are not known in advance. There is an urgent need for an automatic parameter selection method to improve the degree of automation of heterogeneous image matching fusion. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching, which can effectively improve the degree of automation and heterogeneous image matching performance.

[0005] To achieve the above object, the present invention provides a method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching, comprising the following steps:

[0006] Step 1: Select one of the two remote sensing images to be matched as a template image, and determine the upper and lower limits of the gradient amplitude threshold based on the template image. max 、T min and step length T step ;

[0007] Step 2: Calculate the gradient magnitude of each pixel in the template image to obtain a first gradient magnitude image;

[0008] Step 3, with T step The step size is in the gradient amplitude threshold interval [T min , T max], sequentially selecting gradient amplitude thresholds, and performing threshold processing on the gradient amplitude image based on the selected gradient amplitude thresholds, setting the gradient amplitudes of pixels in the gradient amplitude image whose gradient amplitudes are less than the gradient amplitude thresholds to 0, and obtaining a second gradient amplitude image corresponding to each gradient amplitude threshold;

[0009] Step 4: extracting sub-pixel center lines in each second gradient magnitude image, and calculating the average length and average circumscribed rectangular area of the sub-pixel center lines in each second gradient magnitude image to obtain a threshold score index for each second gradient magnitude image, and further fitting a threshold score association curve with the gradient magnitude threshold as a variable;

[0010] Step 5: Select the gradient amplitude threshold corresponding to the maximum value of the threshold score indicator in the threshold score association curve as the optimal gradient amplitude threshold.

[0011] In one embodiment, in step 1, the upper and lower limits of the gradient amplitude threshold T are set based on the template image. max 、T min and step length T step , specifically:

[0012] The maximum contrast value of the template image is used as the upper limit of the gradient amplitude threshold T max , the minimum contrast value of the template image is used as the lower limit of the gradient amplitude threshold;

[0013] Step length T step =(T max -T min ) / k, where k is the step size selection parameter.

[0014] In one embodiment, in step 4, extracting the sub-pixel center lines in each second gradient magnitude image is specifically:

[0015] Perform the following operations on each second gradient magnitude image:

[0016] Step 4.1: first smooth the second gradient magnitude image using a Gaussian function, then generate a first-order Gaussian derivative kernel and a second-order Gaussian derivative kernel through the Gaussian function, and convolve them with the second gradient magnitude image to obtain the Hessian matrix at each pixel in the second gradient magnitude image;

[0017] Step 4.2: The maximum eigenvalue obtained from the Hessian matrix is used as the linear saliency of the corresponding pixel, and the eigenvector corresponding to the maximum eigenvalue is used as the linear normal of the corresponding pixel;

[0018] Step 4.3: Based on the linear saliency and linear normal of each pixel, the two-dimensional Taylor expansion of the local image region, and the condition that the first-order derivative at the center of the linear structure is zero, the linear center points in the second gradient magnitude image are screened, and the linear center points with the highest linear saliency among all the linear center points are selected as seed points.

[0019] Step 4.4, determine whether the normal difference between the seed point and the neighborhood linear center point is less than

[0020] If yes, connect the line center point that meets the conditions with the seed point to form a line, and use the line center point that meets the conditions as the new seed point and repeat step 4.4;

[0021] Otherwise, the remaining linear center point with the largest linear saliency is used as the seed point and step 4.4 is repeated until all linear center points are traversed to obtain multiple sub-pixel center lines.

[0022] In one embodiment, in step 4.4, after obtaining the plurality of sub-pixel center lines, sub-pixel center lines with a length less than 5 are filtered out to obtain all sub-pixel center lines corresponding to the second gradient magnitude image.

[0023] In one embodiment, in step 4, the threshold score indicator is:

[0024]

[0025] Where f(T) is the threshold score index corresponding to the second gradient amplitude image when the gradient amplitude threshold is T, L m is the average length of the sub-pixel center line in the second gradient magnitude image when the gradient magnitude threshold is t, A m is the average circumscribed rectangular area of the sub-pixel center lines in the second gradient magnitude image when the gradient magnitude threshold is T.

[0026] In one embodiment, in step 4, the area of the rectangle circumscribed by the sub-pixel center line is calculated as follows:

[0027] Get the minimum row and column coordinates on the sub-pixel center line (E minx , E miny ) and the maximum row and column coordinates (E maxx , E maxy ), the area of the circumscribed rectangle of the sub-pixel center line is calculated to be A i =(E maxx -E minx )(E maxy -E miny ).

[0028] In one embodiment, in steps 3-4, T stepThe step size is in the gradient amplitude threshold interval [T min , T max ] select the gradient amplitude threshold from small to large in sequence, and calculate the threshold score index corresponding to each gradient amplitude threshold;

[0029] If the threshold score index corresponding to the currently selected gradient amplitude threshold is less than half of the current maximum threshold score index, the selection of the gradient amplitude threshold and the calculation of the threshold score index are stopped.

[0030] In one embodiment, in step 5, after smoothing and filtering the threshold score association curve, the gradient amplitude threshold corresponding to the maximum value of the threshold score indicator in the threshold score association curve is selected as the optimal gradient amplitude threshold.

[0031] Compared with the prior art, the present invention has the following beneficial technical effects:

[0032] 1. Improved automation of heterogeneous image matching: Since the present invention automatically obtains the gradient amplitude threshold parameter, the dependence of parameter setting on the user's professional skills is reduced;

[0033] 2. Improved heterogeneous image matching performance: Since the present invention selects the optimal gradient amplitude threshold parameter based on the distribution of significant contours in the image scene, the accuracy and stability of matching are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0035] Figure 1 Flowchart of a method for automatically selecting a gradient amplitude threshold value according to an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of a threshold score association curve in an embodiment of the present invention.

[0037] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0039] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0040] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0041] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0042] The following takes visible light and SAR image matching as an example to further explain the present invention in detail through specific implementation methods and in conjunction with the accompanying drawings.

[0043] Since gradient features are often used in heterogeneous image matching, selecting gradient features of salient contours through adaptive gradient amplitude thresholding is a key step in improving the accuracy and stability of image matching. The saliency of contours in an image can be characterized by the average length of the contours and the area of the maximum circumscribed rectangle of the contours. The longer the average length and the larger the coverage area, the more conducive it is to achieving correct matching. Figure 1 In this embodiment, a method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching specifically includes the following steps:

[0044] Step 1: Select a visible light image as a template image and determine the upper and lower limits of the gradient amplitude threshold T based on the program characteristics of the template image. max 、T min and step length T step , the process is:

[0045] The maximum contrast value of the template image is used as the upper limit of the gradient amplitude threshold Tmax , the minimum contrast value of the template image is used as the lower limit of the gradient amplitude threshold;

[0046] Step length T step =(T max -T min ) / k, where k is the step size selection parameter.

[0047] For example, based on the imaging characteristics of the visible light image, a suitable gradient amplitude threshold screening range is set to [0, 4000] and a step size of 20. Although a larger range is also feasible, by setting a suitable range, the number of iterative calculations can be reduced and the calculation speed can be improved.

[0048] Step 2: Use the Sobel operator to obtain the horizontal component G of the gradient of each pixel on the template image. x and the vertical component G y , thereby calculating the gradient amplitude of each pixel in the template image and obtaining the first gradient amplitude image. The process is:

[0049] First, the mean filter is used to denoise the visible light image, and the filter kernel is:

[0050]

[0051] Then use the following two convolution operations to obtain the gradient component G x and G y :

[0052]

[0053]

[0054] Where I is the template image;

[0055] Finally, through the horizontal component G x and the vertical component G y That is, the gradient amplitude corresponding to each pixel point can be obtained. The calculation process is a conventional technical means in the relevant field, so it will not be described in detail in this embodiment.

[0056] Step 3, with T step The step size is in the gradient amplitude threshold interval [T min , T max ], sequentially selecting gradient amplitude thresholds, and performing threshold processing on the gradient amplitude image based on the selected gradient amplitude thresholds, setting the gradient amplitudes of pixels in the gradient amplitude image whose gradient amplitudes are less than the gradient amplitude thresholds to 0, and obtaining a second gradient amplitude image corresponding to each gradient amplitude threshold;

[0057] Step 4: Extract the sub-pixel center lines in each second gradient amplitude image, and calculate the average length and average circumscribed rectangular area of the sub-pixel center lines in each second gradient amplitude image to obtain the threshold score index of each second gradient amplitude image, and further fit the threshold score association curve with the gradient amplitude threshold as the variable.

[0058] In this embodiment, the process of extracting the sub-pixel center line in each second gradient magnitude image is to perform the following operations on each second gradient magnitude image:

[0059] Step 4.1, first use a Gaussian function with standard deviation σ = 1.1 The second gradient magnitude image is smoothed, and then the first-order Gaussian derivative kernel and the second-order Gaussian derivative kernel are generated by the Gaussian function. The Hessian matrix at each pixel in the second gradient magnitude image is obtained by convolving it with the second gradient magnitude image, which is:

[0060]

[0061] Where H(x,y) is the Hessian matrix of the pixel point (x,y), g xx is the second derivative in the x-axis direction, g xy is the second-order derivative in the x-axis direction and then in the y-axis direction, g yy is the second-order derivative in the y-axis direction;

[0062] Step 4.2: The maximum eigenvalue obtained from each Hessian matrix is used as the linear saliency of the corresponding pixel point, and the eigenvector corresponding to the maximum eigenvalue is used as the linear normal of the corresponding pixel point (n x ,n y );

[0063] Step 4.3, based on the linear saliency and linear normal of each pixel point, combined with the two-dimensional Taylor expansion of the local area of the image and the condition that the first-order derivative at the center of the linear structure is zero, the linear center point in the second gradient amplitude image is screened out, that is, when (p x ,p y ) When the following conditions are met, the pixel point (x, y) is the center point of the line:

[0064]

[0065]

[0066] Where, (p x ,p y )=(tn x ,tn y ) represents the sub-pixel coordinates of the center line, that is, assuming the coordinates of the center point are (p x ,p y )=(tn x,tn y ) is substituted into the Taylor expansion formula and the constraint that the first-order derivative is 0 to solve for t, thereby obtaining the sub-pixel coordinate value;

[0067] Finally, the point with the largest linear significance among all linear center points is selected as the seed point;

[0068] Step 4.4, determine whether the normal difference between the seed point and the neighborhood linear center point is less than

[0069] If yes, connect the line center point that meets the conditions with the seed point to form a line, and use the line center point that meets the conditions as the new seed point and repeat step 4.4;

[0070] Otherwise, the linear center point with the largest linear significance among the remaining linear center points is used as the seed point and step 4.4 is repeated until all linear center points are traversed to obtain multiple sub-pixel center lines. Sub-pixel center lines with a length less than 5 are filtered out to obtain all sub-pixel center lines corresponding to the second gradient amplitude image.

[0071] In a specific implementation process, the calculation process of the threshold score index for a second gradient amplitude image is:

[0072] First, calculate the average length L of all sub-pixel center lines in the second gradient magnitude image m ,for:

[0073]

[0074] Where, l i is the length of the i-th sub-pixel centerline in the second gradient magnitude image (i.e., the number of linear center points in the sub-pixel centerline), and N is the number of sub-pixel centerlines in the second gradient magnitude image;

[0075] Secondly, calculate the average area A of the circumscribed rectangles of all sub-pixel center lines in the second gradient magnitude image m ,for:

[0076]

[0077] Where A i is the area of the rectangle circumscribed by the i-th sub-pixel centerline in the second gradient magnitude image. In this embodiment, for each sub-pixel centerline circumscribed rectangle, its length and width are respectively along the x and y directions, and its area is calculated as follows:

[0078] Get the minimum row and column coordinates on the sub-pixel center line (E minx , E miny ) and the maximum row and column coordinates (E maxx , E maxy), the area of the circumscribed rectangle of the sub-pixel center line is calculated to be A=(E maxx -E minx )(E maxy -E miny );

[0079] Finally, the Sigmoid function is used to adjust the two indicators L m 、A m Perform scoring and obtain the threshold scoring index as follows:

[0080]

[0081] Where f(t) is the threshold score index corresponding to the second gradient amplitude image when the gradient amplitude threshold is T, L m is the average length of the sub-pixel center line in the second gradient magnitude image when the gradient magnitude threshold is T, A m is the average circumscribed rectangular area of the sub-pixel center lines in the second gradient amplitude image when the gradient amplitude threshold is t. This threshold scoring indicator consists of two sigmoid functions with different weight values, 0.4 and 0.6, respectively, which are obtained experimentally. The Sigmoid function maps the input variable values of the average length of the sub-pixel center lines and the average circumscribed rectangular area to a range between 0 and 1. The longer the average length or the larger the area, the closer the sigmoid function output value is to 1. The larger the value, the more matchable elements there are in the scene, which is more conducive to matching. The Sigmoid function expression is: In order to better distinguish the differences in indicators corresponding to different lengths and areas, α and β are added as variable control parameters in the evaluation index function. These two parameters can be determined by experience or experiments and are conventional means in the field.

[0082] As a preferred embodiment, in step 3-4, T step The step size is in the gradient amplitude threshold interval [T min , T max ] select the gradient amplitude threshold from small to large in sequence, and calculate the threshold score index corresponding to each gradient amplitude threshold;

[0083] If the threshold score index corresponding to the currently selected gradient amplitude threshold is less than half of the current maximum threshold score index, the selection of the gradient amplitude threshold and the calculation of the threshold score index are stopped to save calculation time.

[0084] When the selection of the gradient amplitude threshold and the calculation of the threshold score index are stopped, a number of threshold score index discrete points with the gradient amplitude threshold as the variable will be obtained. By fitting all the discrete points, the threshold score association curve can be obtained.

[0085] Step 5: Smoothing the threshold score correlation curve to obtain:

[0086]

[0087] Then the threshold score association curve The gradient amplitude threshold T corresponding to the maximum value of the medium threshold score index i , as the optimal threshold parameter, that is Figure 2 shown.

[0088] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching, characterized in that: The steps include: Step 1: Select one of the two remote sensing images to be matched as a template image, and determine the upper and lower limits of the gradient amplitude threshold based on the template image. max 、T min and step length T step ; Step 2: Calculate the gradient magnitude of each pixel in the template image to obtain a first gradient magnitude image; Step 3, with T step The step size is in the gradient amplitude threshold interval [T min , T max ], sequentially selecting gradient amplitude thresholds, and performing threshold processing on the gradient amplitude image based on the selected gradient amplitude thresholds, setting the gradient amplitudes of pixels in the gradient amplitude image whose gradient amplitudes are less than the gradient amplitude thresholds to 0, and obtaining a second gradient amplitude image corresponding to each gradient amplitude threshold; Step 4: extracting sub-pixel center lines in each second gradient magnitude image, and calculating the average length and average circumscribed rectangular area of the sub-pixel center lines in each second gradient magnitude image to obtain a threshold score index for each second gradient magnitude image, and further fitting a threshold score association curve with the gradient magnitude threshold as a variable; The extracting of the sub-pixel center line in each second gradient magnitude image is specifically as follows: Perform the following operations on each second gradient magnitude image: Step 4.1: first smooth the second gradient magnitude image using a Gaussian function, then generate a first-order Gaussian derivative kernel and a second-order Gaussian derivative kernel through the Gaussian function, and convolve them with the second gradient magnitude image to obtain the Hessian matrix at each pixel in the second gradient magnitude image; Step 4.2: The maximum eigenvalue obtained from the Hessian matrix is used as the linear saliency of the corresponding pixel, and the eigenvector corresponding to the maximum eigenvalue is used as the linear normal of the corresponding pixel; Step 4.3: Based on the linear saliency and linear normal of each pixel, the two-dimensional Taylor expansion of the local image region, and the condition that the first-order derivative at the center of the linear structure is zero, the linear center points in the second gradient magnitude image are screened, and the linear center points with the highest linear saliency among all the linear center points are selected as seed points. Step 4.4, determine whether the normal difference between the seed point and the neighborhood linear center point is less than If yes, connect the line center point that meets the conditions with the seed point to form a line, and use the line center point that meets the conditions as the new seed point and repeat step 4.4; Otherwise, take the remaining linear center point with the highest linear significance as the seed point and repeat step 4.4 until all linear center points are traversed to obtain multiple sub-pixel center lines; Step 5: Select the gradient amplitude threshold corresponding to the maximum value of the threshold score indicator in the threshold score association curve as the optimal gradient amplitude threshold.

2. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to claim 1, characterized in that: In step 1, the upper and lower limits of the gradient amplitude threshold T are set based on the template image. max 、T min and step length T step , specifically: The maximum contrast value of the template image is used as the upper limit of the gradient amplitude threshold T max , the minimum contrast value of the template image is used as the lower limit of the gradient amplitude threshold; Step length T step =(T max -T min ) / k, where k is the step size selection parameter.

3. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to claim 1, characterized in that: In step 4.4, after obtaining multiple sub-pixel center lines, sub-pixel center lines with a length less than 5 are filtered out to obtain all sub-pixel center lines corresponding to the second gradient magnitude image.

4. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to any one of claims 1 to 3, characterized in that: In step 4, the threshold scoring index is: Where f(T) is the threshold score index corresponding to the second gradient amplitude image when the gradient amplitude threshold is T, L m is the average length of the sub-pixel center line in the second gradient magnitude image when the gradient magnitude threshold is T, A m is the average circumscribed rectangular area of the sub-pixel center lines in the second gradient magnitude image when the gradient magnitude threshold is t.

5. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to any one of claims 1 to 3, characterized in that: In step 4, the area of the circumscribed rectangle of the sub-pixel centerline is calculated as follows: Get the minimum row and column coordinates on the sub-pixel center line (E minx , E miny ) and the maximum row and column coordinates (E maxx , E maxy ), the area of the circumscribed rectangle of the sub-pixel center line is calculated to be A i =(E maxx -E minx )(E maxy -E miny ).

6. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to any one of claims 1 to 3, characterized in that: In steps 3-4, use T step The step size is in the gradient amplitude threshold interval [T min , T max ] select the gradient amplitude threshold from small to large in sequence, and calculate the threshold score index corresponding to each gradient amplitude threshold; If the threshold score index corresponding to the currently selected gradient amplitude threshold is less than half of the current maximum threshold score index, the selection of the gradient amplitude threshold and the calculation of the threshold score index are stopped.

7. The method for automatically selecting a gradient amplitude threshold for heterogeneous remote sensing image matching according to any one of claims 1 to 3, characterized in that: In step 5, after smoothing and filtering the threshold score association curve, the gradient amplitude threshold corresponding to the maximum value of the threshold score indicator in the threshold score association curve is selected as the optimal gradient amplitude threshold.

Images