Interest point detection method based on SAR and visible light remote sensing image matching

Through the point-of-interest detection method based on texture richness, the problem of low multimodal image matching accuracy in non-uniform texture distribution scenes is solved, and higher quality SAR and visible remote sensing image matching is achieved, which is suitable for a variety of noise scenarios.

CN114998617BActive Publication Date: 2024-12-06NAT UNIV OF DEFENSE TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210669461.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-14
Publication Date
2024-12-06
Estimated Expiration
2042-06-14

AI Technical Summary

Technical Problem

The existing multimodal remote sensing image matching method is difficult to distinguish between effective texture and invalid texture areas in non-uniform texture distribution scenarios, resulting in a reduced matching accuracy.

Method used

Using the texture richness-based point of interest detection method, the texture richness feature map on the SAR image is obtained, the points of interest are detected in blocks, and the center of the texture dense area is used instead of corner points that are susceptible to noise interference is extracted, and the template matching is combined with the template. The strategy is to match SAR with visible light remote sensing images.

Benefits of technology

The matching accuracy of SAR and visible remote sensing images in non-uniform texture distribution scenes is improved, the calculation complexity is reduced, and it is robust in other significant noise scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114998617B_ABST
    Figure CN114998617B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting points of interest for matching SAR and visible light remote sensing images, comprising: obtaining the texture richness of each pixel point on the SAR image to obtain a texture richness feature map; performing block interest point detection on the texture richness feature map to obtain a set of interest points of the SAR image; using the set of interest points as the center of a matching window to determine a matchable template to obtain corresponding points on the visible light remote sensing image. The present invention is applied to the field of image processing, and is specifically used to improve the matching accuracy of SAR and visible light remote sensing images in scenes with non-uniform texture distribution. By taking the texture richness of the scene as a feature, using the center of a texture-dense area instead of the corner point extraction susceptible to noise, and other interest point extraction strategies, the problem of poor matching of existing SAR and visible light multimodal image matching methods based on interest point feature extraction in scenes with non-uniform texture distribution is effectively solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a multimodal remote sensing image matching method, in particular to an interest point detection method based on texture richness-based SAR and visible light remote sensing image matching. Background Art

[0002] High-precision multimodal remote sensing image matching technology has important application prospects in many fields, such as remote sensing image fusion, change detection, and aircraft visual navigation. Remote sensing images cover a wide area and the scenes are complex and diverse, which brings challenges to high-precision multimodal remote sensing image matching. SAR and visible light image matching is a typical problem of multimodal image matching. In common non-uniform texture distribution scenes in remote sensing (ports, islands, etc.), the interference of invalid texture areas in SAR images (such as sea surface speckle noise) will significantly reduce the accuracy of commonly used multimodal image matching methods.

[0003] One of the commonly used multimodal matching methods is the method based on feature map matching. Feature maps usually need to be constructed based on image interest points. Heinrich et al. (Heinrich MP, Jenkinson M, Bhushan M, et al. MIND: Modality independent neighborhood descriptor for multi-modal deformable registration [J]. Medical image analysis, 2012, 16 (7): 1423-1435.) proposed a method called modality-independent neighborhood descriptor (MIND) to extract the unique structure in the local neighborhood of interest points (discrete equidistant sampling in the image) to generate a description vector. This method converts images of different modalities into the MIND feature description domain and guides the matching process by measuring the difference between images based on the sum of squared differences (SSD) of the feature description domain. However, this descriptor cannot completely suppress the influence of strong contrast changes between multimodalities and has a very high computational complexity.

[0004] Li Z et al. (Li Z, Mahapatra D, Tielbeek JAW, et al. Image registration based on autocorrelation of local structure [J]. IEEE transactions on medical imaging, 2015, 35 (1): 63-75.) proposed a local structure autocorrelation (ALOST) structure descriptor. This method uses phase consistency and average phase information to replace the intensity information in the MIND structure descriptor. The phase information has been shown to be insensitive to contrast changes. The image registration performance based on this descriptor is improved under strong spatial intensity distortion and large spatial deformation, but the computational complexity is also high.

[0005] Ye Y et al. (Ye Y, Shan J, Bruzzone L, et al. Robust registration of multimodal remote sensing images based on structural similarity [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55 (5): 2941-2958.) proposed a structural feature descriptor called HOPC that integrates phase consistency with the directional histogram strategy. First, the interest points are detected based on the block-based Harris operator and the top-k selection, and then a fast template matching scheme is applied around these interest points. The NCC of the HOPC descriptor is defined as a HOPC NCC The similarity measure is used to guide the matching process. Unreliable matches are eliminated by global constraints in the projection transformation model. Although this method has high matching accuracy, the computational complexity of the structure descriptor is high.

[0006] In principle, the above descriptors for multimodal image matching are designed for nonlinear intensity differences and contrast changes between images, and can be used in SAR and visible light image matching. However, the above methods do not take into account the severe challenges brought by the common non-uniform texture distribution in remote sensing images. The commonly used multimodal image matching methods based on image interest point feature extraction do not distinguish between valid texture and invalid texture areas. When invalid texture areas are involved in matching calculations, the matching accuracy of the entire image will be significantly reduced. Summary of the invention

[0007] In view of the deficiencies in the above-mentioned prior art, the present invention provides a method for detecting points of interest based on texture richness matching of SAR and visible light remote sensing images, which is used to improve the matching accuracy of SAR and visible light remote sensing images in scenes with non-uniform texture distribution. The method uses the texture richness of the scene as a feature, and uses interest point extraction strategies such as the center of a texture-dense area instead of corner points that are susceptible to noise interference, effectively solving the problem of poor matching of existing SAR and visible light multimodal image matching methods based on interest point feature extraction in scenes with non-uniform texture distribution.

[0008] To achieve the above object, the present invention provides a method for detecting points of interest by matching SAR and visible light remote sensing images, comprising the following steps:

[0009] Step 1, obtaining the texture richness of each pixel on the SAR image, and obtaining a texture richness feature map characterized by the texture richness;

[0010] Step 2, performing block interest point detection on the texture richness feature map to obtain an interest point set of the SAR image;

[0011] Step 3: Use the interest point set of the SAR image as the center of the matching window to determine the matching template, and obtain the corresponding points on the visible light remote sensing image based on the template matching strategy.

[0012] In one embodiment, in step 1, obtaining the texture richness of each pixel on the SAR image specifically includes:

[0013] Step 1.1: SAR image I SAR Perform speckle suppression to obtain the filtered image I SR ;

[0014] Step 1.2, for image I SR Perform convolution and obtain image I from the amplitude and phase of the convolution response SR Phase consistency response of each pixel in multiple directions;

[0015] Step 1.3, calculate image I SR The covariance matrix of the phase consistency response of each pixel point is obtained, and the maximum moment of each pixel point is obtained based on the covariance matrix;

[0016] Step 1.4, based on image I SR The maximum moment of each pixel on the grid is used to obtain the texture richness corresponding to each pixel.

[0017] In one embodiment, step 1.2 is specifically:

[0018] The image I is processed by using Log-Gabor wavelet with multiple scales and directions. SRFiltering is performed, passing the real part of the filter and the imaginary part For image I SR Convolution obtains the response e at scale n and direction o no (x,y) and o no (x, y), the amplitude and phase in direction o are calculated as:

[0019]

[0020]

[0021] In the formula, A o (x,y) is image I SR The amplitude of the pixel point (x, y) in direction o, φ no (x,y) is image I SR The phase of the upper pixel (x, y) in direction o;

[0022] The phase consistency response is calculated based on the amplitude and phase, which is:

[0023]

[0024] In the formula, PC o (x,y) image I SR The phase consistency response of the pixel point (x, y) in direction o, W o (x, y) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no (x,y) is a more sensitive phase deviation and ε is a small constant to avoid division by zero.

[0025] In one embodiment, step 1.3 is specifically:

[0026] First, I SR The phase consistency response of the pixel point (x, y) in each direction is projected into the x and y components as follows:

[0027]

[0028]

[0029] In the formula, PC x (x,y), PC y (x, y) are the components of the phase consistency response of the pixel point (x, y) in the x and y directions, respectively, θ o is the angle corresponding to direction o, θ o ∈θ, θ is the set of angles corresponding to all directions;

[0030] Secondly, based on PC x (x,y), PC y (x, y) to obtain the covariance matrix of the phase consistency response of the pixel point (x, y), which is:

[0031]

[0032] Where G(x,y) is the covariance matrix of the phase consistency response of the pixel point (x,y);

[0033] Finally, the maximum moment corresponding to the maximum singular value of the covariance matrix G(x,y) is the maximum moment M(x,y) of the pixel point (x,y).

[0034] In one embodiment, step 1.4 is specifically:

[0035] The intensity value of the maximum moment is used as the weight of the edge feature, and its intensity value is the weight of the edge. The texture richness is calculated in a local area of ​​size N×N, which is:

[0036]

[0037] Where TR(x,y) is the texture richness of the pixel (x,y), and M(x+i,y+j) is the maximum moment of the pixel (x+i,y+j).

[0038] In one embodiment, step 2 specifically includes:

[0039] Step 2.1: Divide the texture richness feature map into N′×N′ non-overlapping blocks, apply non-maximum suppression with a radius of r in each block to retain k extreme points, and obtain a texture map containing k·N′ 2 A set of candidate interest points

[0040] Step 2.2, use the threshold TR_THRESH and the number of interest points N to be retained p Two condition pairs The candidate interest points in are filtered to obtain the final high-quality interest point set MP, which is the interest point set of the SAR image, as follows:

[0041]

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention analyzes the characteristics of scenes with non-uniform texture distribution, designs the texture richness of the scene as the feature of interest point detection, and uses the center of the texture-dense area to replace the corner point extraction that is susceptible to noise interference, etc., to effectively solve the problem of poor matching of the existing SAR and visible light multimodal image matching methods based on interest point feature extraction in scenes with non-uniform texture distribution;

[0044] 2. The SAR and visible light image matching method involved in the present invention can adopt different similarity metrics and can be improved according to the actual matching task;

[0045] 3. The present invention has a simple principle and is easy to implement. It is not only suitable for scenes with non-uniform texture distribution, but also can be applied to other point of interest detection tasks containing significant noise scenes due to the robustness of texture richness to noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0047] Figure 1 Schematic diagram of SAR and visible light remote sensing images with non-uniform texture distribution in an embodiment of the present invention, wherein: (a) is a high-resolution spaceborne SAR image of a port, (b) is a high-resolution spaceborne SAR image of an island, (c) is a high-resolution spaceborne SAR image with missing information, (a′) is a high-resolution optical remote sensing image of a port, (b′) is a high-resolution optical remote sensing image of an island, and (c′) is a high-resolution optical remote sensing image with missing information;

[0048] Figure 2 is a flow chart of a method for detecting points of interest in an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of the effect of speckle suppression in this embodiment, wherein: (a) is a SAR image before speckle suppression, and (b) is a SAR image after speckle suppression;

[0050] Figure 4Schematic diagram of the effect of phase consistency response in this embodiment, wherein: (a) is a schematic diagram of phase consistency response in the first direction, (b) is a schematic diagram of phase consistency response in the second direction, (c) is a schematic diagram of phase consistency response in the third direction, (d) is a schematic diagram of phase consistency response in the fourth direction, (e) is a schematic diagram of phase consistency response in the fifth direction, and (f) is a schematic diagram of phase consistency response in the sixth direction;

[0051] Figure 5 is the maximum moment diagram in this embodiment;

[0052] Figure 6 is the texture richness feature map in this embodiment;

[0053] Figure 7 This is a schematic diagram of interest points extracted from a SAR image before interest point filtering in this embodiment;

[0054] Figure 8 Schematic diagram of interest point filtering in this embodiment, wherein: (a) is a texture richness mask map, and (b) is a schematic diagram of interest points extracted from a SAR image after interest point filtering;

[0055] Fig. 9 is the feature map in the example, where: (a) and (b) are two SAR images of ports, (a′) and (b′) are the corresponding texture richness feature maps; (c) and (d) are two SAR images of islands, (c′) and (d′) are the corresponding texture richness feature maps; (e) and (f) are two SAR images with missing information, (e′) and (f′) are the corresponding texture richness feature maps;

[0056] Fig.10 Schematic diagram for comparing the points of interest extracted by block Harris and the method of the present invention in SAR images with non-uniform texture distribution in the example, wherein: (a) and (b) are schematic diagrams of the points of interest extracted by block Harris in two port SAR images, (a′) and (b′) are schematic diagrams of the points of interest extracted by the method of the present invention in the same two port SAR images; (c) and (d) are schematic diagrams of the points of interest extracted by block Harris in two island SAR images, (c′) and (d′) are schematic diagrams of the points of interest extracted by the method of the present invention in the same two island SAR images; (e) and (f) are schematic diagrams of the points of interest extracted by block Harris in two information-missing SAR images, (e′) and (f′) are schematic diagrams of the points of interest extracted by the method of the present invention in the same two information-missing SAR images.

[0057] Fig.11Schematic diagram of the relationship between CMR and texture richness threshold TR_THRESH calculated on the image pair with non-uniform texture distribution in the example;

[0058] Fig.12 Schematic diagram of the relationship between RMSE and texture richness threshold TR_THRESH calculated on the image pair with non-uniform texture distribution in the example;

[0059] Fig.13 Schematic diagram of the relationship between the running time and the texture richness threshold TR_THRESH on the image pair with non-uniform texture distribution in the example.

[0060] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0063] In the description of the present invention, “plurality” means at least two, for example, two, three, etc., unless otherwise clearly and specifically defined.

[0064] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0066] This embodiment discloses a method for detecting points of interest based on texture richness in matching SAR and visible light remote sensing images, which is used to improve the matching accuracy of SAR and visible light remote sensing images with non-uniform texture distribution. Figure 1 As shown. Among them, Figure 1 (a) Figure 1 (b) Figure 1 (c) are high-resolution spaceborne SAR images, Figure 1 (a′), Figure 1 (b′), Figure 1 (c′) are high-resolution optical remote sensing images; the resolution is resampled to the same 5m, and the SAR and visible light remote sensing image pairs have been aligned using geo-registration technology and large-size template matching methods. Three typical scenes with non-uniform texture distribution are shown, namely ports, islands, and information loss. All three types of images have the characteristics of uneven texture distribution. In the specific application process, the method in this embodiment uses the texture richness of the scene as a feature, and uses interest point extraction strategies such as the center of the texture-dense area instead of the corner point extraction that is susceptible to noise interference, which effectively solves the problem of poor matching of existing SAR and visible light multimodal image matching methods based on interest point feature extraction in scenes with non-uniform texture distribution.

[0067] refer to Figure 2 In this embodiment, the method for detecting points of interest based on texture richness matching between SAR and visible light remote sensing images specifically includes the following steps:

[0068] Step 1: Obtain the texture richness of each pixel on the SAR image and obtain a texture richness feature map characterized by texture richness. The process is as follows:

[0069] Step 1.1: SAR image I SAR Perform speckle suppression to obtain the filtered image I SR ,Right now Figure 3 As shown in Figure 2, SAR image speckle suppression can remove coherent images while retaining details as much as possible. SAR The multiplicative noise in the image is removed to obtain a SAR image with better quality. SR, in preparation for the next step of extracting texture features, the filter window w can be adjusted according to the SAR image quality. If the speckle noise is strong, the window size can be increased. If you want to retain more texture details, the window size can be reduced. Finally, the filtered image I is obtained. SR for:

[0070] I SR =speckle_reduction(I SAR ,w,params)

[0071] Where speckle_reduction(·) represents any existing SAR image speckle reduction method, and params represents the adjustable parameters of the speckle reduction method.

[0072] Step 1.2, for image I SR Perform convolution and obtain image I from the amplitude and phase of the convolution response SR The phase consistency response of each pixel in multiple directions is calculated. In this embodiment, the phase consistency response in 6 directions is calculated respectively, that is, Figure 4 As shown, Figure 4 (a) is a schematic diagram of the phase consistency response in the first direction. Figure 4 (b) is a schematic diagram of the phase consistency response in the second direction. Figure 4 (c) is a schematic diagram of the phase consistency response in the third direction. Figure 4 (d) is a schematic diagram of the phase consistency response in the fourth direction. Figure 4 (e) is a schematic diagram of the phase consistency response in the fifth direction. Figure 4 (f) is a schematic diagram of the phase consistency response in the sixth direction. In the specific implementation process, the calculation process of the phase consistency response in each direction is as follows:

[0073] The filtered image I SR It is convolved as the input of a set of Log-Gabor wavelets, and the phase consistency response is further calculated by the amplitude and phase of the convolved response;

[0074] Among them, the real part of the filter and the imaginary part For image I SR Convolution obtains the response e at scale n and direction o no (x,y) and o no (x, y), the amplitude and phase in direction o are calculated as:

[0075]

[0076] φ no (x,y)=arctan(eno (x,y),o no (x,y)

[0077] In the formula, A o (x,y) is image I SR The amplitude of the pixel point (x, y) in direction o, φ no (x,y) is image I SR The phase of the upper pixel (x, y) in direction o;

[0078] The phase consistency response is then calculated based on the amplitude and phase, which is:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] In the formula, PC o (x,y) image I SR The phase consistency response of the pixel point (x, y) in direction o (o = 1 to 6), W o (x, y) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no (x,y) is a more sensitive phase deviation, ε is a small constant to avoid division by zero, and E(x,y) is the local energy function.

[0085] Step 1.3, calculate image I SR The covariance matrix of the phase consistency response of each pixel on the graph is obtained, and the maximum moment of each pixel is obtained based on the covariance matrix, that is, Figure 5 The specific implementation process is as follows:

[0086] First, I SR The phase consistency response of the pixel point (x, y) in each direction is projected into the x and y components as follows:

[0087]

[0088]

[0089] In the formula, PC x (x,y), PC y (x, y) are the components of the phase consistency response of the pixel point (x, y) in the x and y directions, respectively, θo is the angle corresponding to direction o, θ o ∈θ, θ is the set of angles corresponding to all directions;

[0090] Secondly, based on PC x (x,y), PC y (x, y) constructs the covariance matrix of the phase consistency response of each pixel on the image, which is:

[0091]

[0092] Where G(x,y) is the covariance matrix of the phase consistency response of the pixel point (x,y);

[0093] Finally, the maximum moment corresponding to the maximum singular value of the covariance matrix G(x,y) is the maximum moment M(x,y) of the pixel point (x,y).

[0094] Step 1.4, based on image I SR The maximum moment of each pixel point on the grid is used to obtain the texture richness corresponding to each pixel point. The specific implementation method is as follows:

[0095] According to the design idea of ​​edge density, the intensity value of the maximum moment is used as the weight of the edge feature, and its intensity value is the weight of the edge. The texture richness is calculated in a local area of ​​size N×N, which is:

[0096]

[0097] Wherein, TR(x,y) is the texture richness of the pixel point (x,y), M(x+i,y+j) is the maximum moment of the pixel point (x+i,y+j), and in this embodiment, the default value of N is set to 21;

[0098] After getting the image I SR After calculating the texture richness TR of each pixel, we finally get the image I SR Texture richness feature maps of the same size, i.e. Figure 6 shown.

[0099] Step 2: Perform block interest point detection on the texture richness feature map to obtain the interest point set of the SAR image. The process is as follows:

[0100] Step 2.1, divide the texture richness feature map into 10×10 non-overlapping blocks, apply non-maximum suppression with a radius of r in each block to retain 4 extreme points, and obtain Figure 7 The CIP contains 400 candidate interest points. 400 , that is, N′=10, k=4;

[0101] Step 2.2, reference Figure 8, the candidate interest point set CIP 400 By threshold TR_THRESH and retaining the number of interest points N p The two conditions are filtered to obtain the final high-quality interest point set MP, which is the interest point set of the SAR image, as follows:

[0102]

[0103] Step 3, for the task of matching SAR and visible light images with non-uniform texture distribution, the interest point set MP of the SAR image is used as the center of the matching window to determine the matching template, and based on the template matching strategy, the corresponding point set CP on the visible light remote sensing image is obtained. Finally, the transformation matrix between the SAR and visible light images can be determined through the point set MP and CP.

[0104] It should be noted that, because the present invention is a method for detecting points of interest in template matching, it needs to be integrated into the existing multimodal image matching method before it can be applied to the actual SAR and visible light image matching task. In addition, the present invention can be tested on SAR and visible light remote sensing datasets containing scenes with non-uniform texture distribution.

[0105] The method for detecting points of interest based on texture richness matching between SAR and visible light remote sensing images in the present invention is further described below with reference to specific examples.

[0106] First, the feature map of the interest point detection method proposed in the present invention is shown, that is, the visualization result of texture richness on the image with non-uniform texture distribution. Fig. 9 As shown, where: Fig. 9 (a) Fig. 9 (b) shows the SAR images of two ports. Fig. 9 (a′), Fig. 9 (b′) is the corresponding texture richness feature map; Fig. 9 (c) Fig. 9 (d) shows the SAR images of two islands. Fig. 9 (c′), Fig. 9 (d′) is the corresponding texture richness feature map; Fig. 9 (e) Fig. 9 (f) shows two SAR images with missing information. Fig. 9 (e′), Fig. 9 (f′) is the corresponding texture richness feature map.

[0107] Depend on Fig. 9It can be seen that the proposed texture richness feature can effectively divide the scene into texture-dense and texture-free areas (sea surface or information missing). The point of interest detection method proposed in the present invention is integrated into the HOPC template matching, denoted as HOPC-TR. The following is a comparison of the results of the method provided by the present invention (implementation) and the block Harris method (comparative example) in the original HOPC method. The comparative experiment uses high-resolution spaceborne SAR images collected by the TerraSAR-X satellite in Hainan and Google Maps optical images with non-uniform texture distribution. The comparison of points of interest extracted by the block Harris and the method proposed in the present invention in SAR images with non-uniform texture distribution is shown as follows. Fig.10 As shown, Fig.10 (a) Fig.10 (b) is a schematic diagram of the points of interest extracted by Harris block in two port SAR images. Fig.10 (a′), Fig.10 (b′) is a schematic diagram of points of interest extracted by the method of the present invention from the same two port SAR images; Fig.10 (c) Fig.10 (d) is a schematic diagram of the points of interest extracted by Harris block in two island SAR images. Fig.10 (c′), Fig.10 (d′) Schematic diagram of points of interest extracted by the method of the present invention from the same two island SAR images; Fig.10 (e) Fig.10 (f) is a schematic diagram of the points of interest extracted by Harris block in two SAR images with missing information. Fig.10 (e′), Fig.10 (f′) is a schematic diagram of interest points extracted by the method of the present invention from the same two SAR images with missing information.

[0108] Depend on Fig.10 It can be seen that most of the interest points extracted by the block Harris method fall in the invalid texture area; the block interest point detection method proposed in this invention has the interest points fall in the area with dense texture distribution and retains the uniform distribution characteristics, providing high-quality matching points for the subsequent matching steps. The correct match rate (CMR), root mean square error (RMSE) and running time commonly used in the field of image matching research are used as evaluation indicators, which are specifically defined as follows:

[0109] CMR=NCM / C

[0110]

[0111] In the formula, NCM represents the number of correctly matched point pairs, C is the total number of matched point pairs, (x i ,y i ) is the coordinate of the corresponding point calculated for the i-th interest point, (x o,y o ) represents the true value of the coordinates of the corresponding points.

[0112] Figure 11-13 represents the comparison of the matching performance of interest points obtained by using the block Harris method and the method proposed in the present invention, where: Fig.11 is the relationship between CMR and texture richness threshold TR_THRESH calculated on the image pair with non-uniform texture distribution, Fig.12 is the relationship between RMSE calculated on the image pair with non-uniform texture distribution and the texture richness threshold TR_THRESH, Fig.13 It is the relationship between the running time and the texture richness threshold TR_THRESH on the image pair with non-uniform texture distribution. The matching accuracy of the method can be compared by CMR and RMSE indicators, and the running time can be used to compare the computational efficiency of the method. Compared with other methods, the matching accuracy of the method provided by the present invention is higher, the CMR value is improved by 14 percentage points compared with the block Harris method, the RMSE value is reduced by more than 0.25 pixels of deviation, and the running time is reduced by about 20%, indicating that the matching accuracy of the points of interest detected by the method proposed in the present invention is higher and the calculation speed is faster. In the results of the method provided by the present invention, the CP matching of the points of interest is more accurate and faster, so it can be proved that the present invention can provide points of interest with better matching quality.

[0113] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for detecting points of interest by matching SAR and visible light remote sensing images, characterized in that: The steps include: Step 1, obtaining the texture richness of each pixel on the SAR image, and obtaining a texture richness feature map characterized by the texture richness, wherein obtaining the texture richness of each pixel on the SAR image specifically includes: Step 1.1: SAR image I SAR Perform speckle suppression to obtain the filtered image I SR ; Step 1.2, for image I SR Perform convolution and obtain image I from the amplitude and phase of the convolution response SR The phase consistency response of each pixel in multiple directions is as follows: The image I is processed by using Log-Gabor wavelet with multiple scales and directions. SR Filtering is performed, passing the real part of the filter and the imaginary part For image I SR Convolution obtains the response e at scale n and direction o no (x,y) and o no (x, y), the amplitude and phase in direction o are calculated as: In the formula, A o (x,y) is image I SR The amplitude of the pixel point (x, y) in direction o, φ no (x,y) is image I SR The phase of the upper pixel (x, y) in direction o; The phase consistency response is calculated based on the amplitude and phase, which is: In the formula, PC o (x,y) image I SR The phase consistency response of the pixel point (x, y) in direction o, W o (x, y) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no (x,y) is a more sensitive phase deviation, and ε is a small constant to avoid division by zero; Step 1.3, calculate image I SR The covariance matrix of the phase consistency response of each pixel is obtained, and the maximum moment of each pixel is obtained based on the covariance matrix, which is specifically: First, I SR The phase consistency response of the pixel point (x, y) in each direction is projected into the x and y components as follows: In the formula, PC x (x,y), PC y (x, y) are the components of the phase consistency response of the pixel point (x, y) in the x and y directions, respectively, θ o is the angle corresponding to direction o, θ o ∈θ, θ is the set of angles corresponding to all directions; Secondly, based on PC x (x,y), PC y (x, y) to obtain the covariance matrix of the phase consistency response of the pixel point (x, y), which is: Where G(x,y) is the covariance matrix of the phase consistency response of the pixel point (x,y); Finally, the maximum moment corresponding to the maximum singular value of the covariance matrix G(x,y) is the maximum moment M(x,y) of the pixel point (x,y); Step 1.4, based on image I SR The maximum moment of each pixel on the grid is used to obtain the texture richness corresponding to each pixel, which is: The intensity value of the maximum moment is used as the weight of the edge feature, and its intensity value is the weight of the edge. The texture richness is calculated in a local area of ​​size N×N, which is: Where TR(x,y) is the texture richness of the pixel (x,y), and M(x+i,y+j) is the maximum moment of the pixel (x+i,y+j); Step 2, performing block interest point detection on the texture richness feature map to obtain an interest point set of the SAR image; Step 3: Use the interest point set of the SAR image as the center of the matching window to determine the matching template, and obtain the corresponding points on the visible light remote sensing image based on the template matching strategy.

2. The method for detecting points of interest by matching SAR and visible light remote sensing images according to claim 1, characterized in that: Step 2 specifically includes: Step 2.1: Divide the texture richness feature map into N′×N′ non-overlapping blocks, apply non-maximum suppression with a radius of r in each block to retain k extreme points, and obtain a texture map containing k·N′ 2 A set of candidate interest points Step 2.2, use the threshold TR_THRESH and the number of interest points N to be retained p Two condition pairs The candidate interest points in are filtered to obtain the final high-quality interest point set MP, which is the interest point set of the SAR image, as follows: