A similarity measurement method for matching SAR and visible light remote sensing images

By analyzing texture richness in SAR images and distinguishing texture dense and textureless areas, the problem of low matching accuracy in non-uniform texture distribution scenes in multimodal remote sensing image matching is solved, and a more efficient image matching effect is achieved.

CN115100446BActive Publication Date: 2024-11-29NAT UNIV OF DEFENSE TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210726979.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-24
Publication Date
2024-11-29
Estimated Expiration
2042-06-24

AI Technical Summary

Technical Problem

The existing multimodal remote sensing image matching method is difficult to distinguish between texture-intensive and textureless areas in non-uniform texture distribution scenarios, resulting in a reduced matching accuracy.

Method used

By obtaining the texture richness of each pixel point on the SAR image, a texture richness feature map is generated, and the threshold is reduced to a binary mask, distinguishing texture dense and textureless areas, collecting feature descriptors for texture dense areas, and calculating similarity metrics , remove interference information from textureless areas.

Benefits of technology

The accuracy of matching SAR with visible light remote sensing images is improved, and the calculation complexity is reduced, which is suitable for matching tasks in non-uniform texture distribution scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115100446B_ABST
    Figure CN115100446B_ABST
Patent Text Reader

Abstract

The present invention discloses a similarity measurement method for matching SAR and visible light remote sensing images, including: obtaining a texture richness feature map of a SAR image; thresholding the texture richness feature map into a binary mask to distinguish between texture-intensive areas and texture-free areas; uniformly collecting unit feature descriptors in the corresponding areas of the binary mask on the SAR and visible light remote sensing images to obtain a first feature vector and a second feature vector; and obtaining a similarity measurement between the SAR and visible light remote sensing images based on the first feature vector and the second feature vector. The present invention is applied to the field of image processing, generates a mask based on the texture richness feature of a scene, calculates a similarity measurement by using effective information of texture-intensive areas in the scene and eliminating interference information of texture-free areas, and effectively solves the problem of poor performance of feature descriptors constructed by existing SAR and visible light heterogeneous image matching methods based on similarity measurement in scenes with non-uniform texture distribution.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a multimodal remote sensing image matching method, in particular to a similarity measurement method for matching SAR and visible light remote sensing images based on texture richness. Background Art

[0002] High-precision multimodal remote sensing image matching technology has important application prospects in many fields, such as remote sensing image fusion, change detection, and aircraft visual navigation. Remote sensing images cover a wide area and the scenes are complex and diverse, which brings challenges to high-precision multimodal remote sensing image matching. SAR and visible light image matching is a typical problem of multimodal image matching. In common non-uniform texture distribution scenes in remote sensing (ports, islands, etc.), the interference of texture-free areas in SAR images (such as sea surface speckle noise) will significantly reduce the accuracy of commonly used multimodal image matching methods.

[0003] One of the commonly used multimodal matching methods is the method based on feature map matching. After defining the template window in the image, the corresponding window on other images is searched by using a similarity measure based on the feature map. Heinrich et al. (Heinrich MP, Jenkinson M, Bhushan M, et al. MIND: Modality independent neighborhood descriptor for multi-modal deformable registration [J]. Medical image analysis, 2012, 16 (7): 1423-1435.) proposed a method called modality-independent neighborhood descriptor (MIND) to extract unique structures in the local neighborhood to generate a description vector. This method converts images of different modalities into the MIND feature description domain and guides the matching process by measuring the difference between images based on the sum of squared differences (SSD) of the feature description domain. However, this descriptor cannot completely suppress the influence of strong contrast changes between multimodal images and has a very high computational complexity.

[0004] Li Z et al. (Li Z, Mahapatra D, Tielbeek JAW, et al. Image registration based on autocorrelation of local structure [J]. IEEE transactions on medical imaging, 2015, 35 (1): 63-75.) proposed a local structure autocorrelation (ALOST) structure descriptor. This method uses phase consistency and average phase information to replace the intensity information in the MIND structure descriptor. The phase information has been shown to be insensitive to contrast changes. The image registration performance based on this descriptor is improved under strong spatial intensity distortion and large spatial deformation, but the computational complexity is also high.

[0005] Ye Y et al. (Ye Y, Shan J, Bruzzone L, et al. Robust registration of multimodal remote sensing images based on structural similarity [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55 (5): 2941-2958.) proposed a structural feature descriptor called HOPC that integrates phase consistency with the directional histogram strategy. First, interest points are detected based on the block Harris operator and top k selection, and then a fast template matching scheme is applied around these interest points. The NCC of the HOPC descriptor is defined as a similarity measure called HOPCncc to guide the matching process. Unreliable matches are eliminated by global constraints in the projection transformation model. This method has high matching accuracy, but the computational complexity of the structural descriptor is high.

[0006] In principle, the above descriptors for multimodal image matching are designed for nonlinear intensity differences and contrast changes between images, and can be used in SAR and visible light image matching. However, the above methods do not consider the severe challenges brought by the common non-uniform texture distribution in remote sensing images. The commonly used multimodal image matching methods based on similarity metrics do not distinguish between texture-dense and texture-free areas. When the descriptors constructed in texture-free areas are involved in the matching calculation, the matching accuracy of the entire image will be significantly reduced. Summary of the invention

[0007] In view of the deficiencies in the above-mentioned prior art, the present invention provides a similarity measurement method for matching SAR and visible light remote sensing images, which is used to improve the matching accuracy of SAR and visible light remote sensing images in scenes with non-uniform texture distribution. The method generates a mask based on the texture richness feature of the scene, uses the effective information of the texture-dense area in the scene and eliminates the interference information of the texture-free area to calculate the similarity measurement, and effectively solves the problem of poor performance of the feature descriptor constructed by the existing SAR and visible light heterogeneous image matching method based on similarity measurement in scenes with non-uniform texture distribution.

[0008] To achieve the above object, the present invention provides a similarity measurement method for matching SAR and visible light remote sensing images, comprising the following steps:

[0009] Step 1, obtaining the texture richness of each pixel on the SAR image, and obtaining a texture richness feature map characterized by the texture richness;

[0010] Step 2: threshold the texture richness feature map into a binary mask to distinguish between texture-dense areas and texture-free areas in the SAR image scene;

[0011] Step 3, uniformly collecting unit feature descriptors in the texture-dense area on the SAR image and merging them into a first feature vector; uniformly collecting unit feature descriptors in the corresponding area of ​​the binary mask on the visible light remote sensing image and merging them into a second feature vector;

[0012] Step 4: Based on the first eigenvector and the second eigenvector, a similarity measure between the SAR and visible light remote sensing images is obtained, and a transformation matrix between the SAR and visible light remote sensing images is determined using a template matching strategy.

[0013] In one embodiment, in step 2, the thresholding of the texture richness feature map into a binary mask is specifically:

[0014] Normalize the texture richness of each pixel on the texture richness feature map to the range of [0,1] through the maximum and minimum values;

[0015] The threshold TR_THRESH is used to perform threshold segmentation on the normalized texture richness feature map:

[0016] The texture richness of the pixels whose texture richness is greater than or equal to the threshold TR_THRESH in the normalized texture richness feature map is set to 1;

[0017] The texture richness of the pixels whose texture richness in the normalized texture richness feature map is less than the threshold TR_THRESH is set to 0;

[0018] Among them, the area where the texture richness is set to 1 in the texture richness feature map is the texture-dense area, and the area where the texture richness is set to 0 is the texture-free area.

[0019] In one embodiment, in step 3, the process of obtaining the first feature vector and the second feature vector is:

[0020] At intervals of d pixels, the unit feature descriptors feat_desp_sar are uniformly collected in the texture-dense areas of the SAR image and merged into the final first feature vector V A ,for:

[0021]

[0022] At intervals of d pixels, the unit feature descriptors feat_desp_opt are uniformly collected in the corresponding areas of the binary mask on the visible light remote sensing image and merged into the final second feature vector V B ,for:

[0023]

[0024] Where TR_MASK is the binary mask after thresholding the texture richness feature map.

[0025] In one embodiment, in step 4, the similarity measurement between the SAR and visible light remote sensing images is specifically:

[0026]

[0027] Where TRncc is the similarity measure between SAR and visible light remote sensing images, V A (k) is the k-th unit feature descriptor in the first feature vector, is the mean of all unit feature descriptors in the first eigenvector, V B (k) is the k-th unit feature descriptor in the second feature vector, is the mean of all unit feature descriptors in the second feature vector, and n is the total number of unit feature descriptors in the first feature vector and the second feature vector.

[0028] In one embodiment, in step 1, obtaining the texture richness of each pixel on the SAR image specifically includes:

[0029] Step 1.1: SAR image I SAR Perform speckle suppression to obtain the filtered image I SR ;

[0030] Step 1.2, for image I SR Perform convolution and obtain image I from the amplitude and phase of the convolution responseSR Phase consistency response of each pixel in multiple directions;

[0031] Step 1.3, calculate image I SR The covariance matrix of the phase consistency response of each pixel point is obtained, and the maximum moment of each pixel point is obtained based on the covariance matrix;

[0032] Step 1.4, based on image I SR The maximum moment of each pixel on the grid is used to obtain the texture richness corresponding to each pixel.

[0033] In one embodiment, step 1.2 is specifically:

[0034] The image I is processed by using Log-Gabor wavelet with multiple scales and directions. SR Filtering is performed, passing the real part of the filter and the imaginary part For image I SR Convolution obtains the response e at scale n and direction o no (x,y) and o no (x, y), the amplitude and phase in direction o are calculated as:

[0035]

[0036] φ no (x,y)=arctan(e no (x,y),o no (x,y)

[0037] In the formula, A o (x,y) is image I SR The amplitude of the pixel point (x, y) in direction o, φ no (x,y) is image I SR The phase of the upper pixel (x, y) in direction o;

[0038] The phase consistency response is calculated based on the amplitude and phase, which is:

[0039]

[0040] In the formula, PC o (x,y) image I SR The phase consistency response of the pixel point (x, y) in direction o, W o (x, y) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no (x,y) is a more sensitive phase deviation and ε is a small constant to avoid division by zero.

[0041] In one embodiment, step 1.3 is specifically:

[0042] First, I SR The phase consistency response of the pixel point (x, y) in each direction is projected into the x and y components as follows:

[0043]

[0044]

[0045] In the formula, PC x (x,y), PC y (x, y) are the components of the phase consistency response of the pixel point (x, y) in the x and y directions, respectively, θ o is the angle corresponding to direction o, θ o ∈θ, θ is the set of angles corresponding to all directions;

[0046] Secondly, based on PC x (x,y), PC y (x, y) to obtain the covariance matrix of the phase consistency response of the pixel point (x, y), which is:

[0047]

[0048] Where G(x,y) is the covariance matrix of the phase consistency response of the pixel point (x,y);

[0049] Finally, the maximum moment corresponding to the maximum singular value of the covariance matrix G(x,y) is the maximum moment M(x,y) of the pixel point (x,y).

[0050] In one embodiment, step 1.4 is specifically:

[0051] The intensity value of the maximum moment is used as the weight of the edge feature, and its intensity value is the weight of the edge. The texture richness is calculated in a local area of ​​size N×N, which is:

[0052]

[0053] Where TR(x,y) is the texture richness of the pixel (x,y), and M(x+i,y+j) is the maximum moment of the pixel (x+i,y+j).

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The present invention analyzes the characteristics of non-uniform texture distribution scenes, designs texture richness thresholds to mark texture-dense and texture-free areas in the scenes, and uses the effective information of texture-dense areas in the scenes and removes the interference information of texture-free areas to calculate the similarity metric, which effectively solves the problem of poor performance of feature descriptors constructed by existing SAR and visible light heterogeneous image matching methods based on similarity metrics in non-uniform texture distribution scenes;

[0056] 2. The similarity measure of the SAR and visible light image matching method involved in the present invention can be calculated based on different feature descriptors and can be improved according to the actual matching task;

[0057] 3. The principle of the present invention is simple and easy to implement. Due to the effectiveness of texture richness in dividing the texture-dense and texture-free areas in the scene, the present invention can be applied to the matching tasks of various scenes with non-uniform texture distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0059] Figure 1 Schematic diagram of SAR and visible light remote sensing images with non-uniform texture distribution in an embodiment of the present invention, wherein: (a) is a high-resolution spaceborne SAR image of a port, (b) is a high-resolution spaceborne SAR image of an island, (c) is a high-resolution spaceborne SAR image with missing information, (a′) is a high-resolution optical remote sensing image of a port, (b′) is a high-resolution optical remote sensing image of an island, and (c′) is a high-resolution optical remote sensing image with missing information;

[0060] Figure 2 is a flow chart of a similarity measurement method in an embodiment of the present invention;

[0061] Figure 3 Schematic diagram of the effect of speckle suppression in this embodiment, wherein: (a) is a SAR image before speckle suppression, and (b) is a SAR image after speckle suppression;

[0062] Figure 4Schematic diagram of the effect of phase consistency response in this embodiment, wherein: (a) is a schematic diagram of phase consistency response in the first direction, (b) is a schematic diagram of phase consistency response in the second direction, (c) is a schematic diagram of phase consistency response in the third direction, (d) is a schematic diagram of phase consistency response in the fourth direction, (e) is a schematic diagram of phase consistency response in the fifth direction, and (f) is a schematic diagram of phase consistency response in the sixth direction;

[0063] Figure 5 is the maximum moment diagram in this embodiment;

[0064] Figure 6 is the texture richness feature map in this embodiment;

[0065] Figure 7 This is a schematic diagram of collecting unit feature descriptors in this embodiment;

[0066] Figure 8 Collect schematic diagrams for unit feature descriptors of conventional methods;

[0067] Fig. 9 is the feature map in the example, where: (a) and (b) are two SAR images of ports, (a′) and (b′) are the corresponding texture richness feature maps; (c) and (d) are two SAR images of islands, (c′) and (d′) are the corresponding texture richness feature maps; (e) and (f) are two SAR images with missing information, (e′) and (f′) are the corresponding texture richness feature maps;

[0068] Fig.10 is the mask image in the example, where: (a), (b), (c) are SAR images of ports, islands, and missing information, respectively; (a-1), (b-1), (c-1) are the mask images corresponding to (a), (b), (c) when TR_THRESH=0.1, respectively; (a-2), (b-2), (c-2) are the mask images corresponding to (a), (b), (c) when TR_THRESH=0.25, respectively; (a-3), (b-3), (c-3) are the mask images corresponding to (a), (b), (c) when TR_THRESH=0.5, respectively;

[0069] Fig.11 Schematic diagram of the relationship between CMR and texture richness threshold TR_THRESH calculated on the image pair with non-uniform texture distribution in the example;

[0070] Fig.12 Schematic diagram of the relationship between RMSE and texture richness threshold TR_THRESH calculated on the image pair with non-uniform texture distribution in the example;

[0071] Fig.13 Schematic diagram of the relationship between the running time and feature vector length and the texture richness threshold TR_THRESH on the image pair with non-uniform texture distribution in the example.

[0072] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0075] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0076] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0077] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0078] This embodiment discloses a similarity measurement method for matching SAR and visible light remote sensing images based on texture richness, which is used to improve the matching accuracy of SAR and visible light remote sensing images with non-uniform texture distribution. Figure 1 As shown. Among them, Figure 1 (a) Figure 1 (b) Figure 1 (c) are high-resolution spaceborne SAR images, Figure 1 (a′), Figure 1 (b′), Figure 1 (c′) are high-resolution optical remote sensing images; the resolution is resampled to the same 5m, and the SAR and visible light remote sensing image pairs have been aligned using geo-registration technology and large-size template matching methods. Three typical scenes with non-uniform texture distribution are shown, namely ports, islands, and information loss. All three types of images have scene characteristics of uneven texture distribution. In the specific application process, the method in this embodiment generates a mask based on the texture richness features of the scene, calculates the similarity metric by using the effective information of the texture-dense areas in the scene and eliminating the interference information of the texture-free areas, and effectively solves the problem of poor performance of feature descriptors constructed by the existing SAR and visible light heterogeneous image matching methods based on similarity metrics in scenes with non-uniform texture distribution.

[0079] refer to Figure 2 In this embodiment, the similarity measurement method for matching SAR and visible light remote sensing images based on texture richness specifically includes the following steps:

[0080] Step 1: Obtain the texture richness of each pixel on the SAR image and obtain a texture richness feature map characterized by texture richness. The process is as follows:

[0081] Step 1.1: SAR image I SAR Perform speckle suppression to obtain the filtered image I SR ,Right now Figure 3 As shown in Figure 2, SAR image speckle suppression can remove coherent images while retaining details as much as possible. SAR The multiplicative noise in the image is removed to obtain a SAR image with better quality. SR , in preparation for the next step of extracting texture features, the filter window w can be adjusted according to the SAR image quality. If the speckle noise is strong, the window size can be increased. If you want to retain more texture details, the window size can be reduced. Finally, the filtered image I is obtained. SR for:

[0082] I SR =speckle_reduction(I SAR ,w,params)

[0083] Where speckle_reduction(·) represents any existing SAR image speckle reduction method, and params represents the adjustable parameters of the speckle reduction method.

[0084] Step 1.2, for image I SR Perform convolution and obtain image I from the amplitude and phase of the convolution response SR The phase consistency response of each pixel in multiple directions is calculated. In this embodiment, the phase consistency response in 6 directions is calculated respectively, that is, Figure 4 As shown, Figure 4 (a) is a schematic diagram of the phase consistency response in the first direction. Figure 4 (b) is a schematic diagram of the phase consistency response in the second direction. Figure 4 (c) is a schematic diagram of the phase consistency response in the third direction. Figure 4 (d) is a schematic diagram of the phase consistency response in the fourth direction. Figure 4 (e) is a schematic diagram of the phase consistency response in the fifth direction. Figure 4 (f) is a schematic diagram of the phase consistency response in the sixth direction. In the specific implementation process, the calculation process of the phase consistency response in each direction is as follows:

[0085] The filtered image I SR It is convolved as the input of a set of Log-Gabor wavelets, and the phase consistency response is further calculated by the amplitude and phase of the convolved response;

[0086] Among them, the real part of the filter and the imaginary part For image I SR Convolution obtains the response e at scale n and direction o no (x,y) and o no (x, y), the amplitude and phase in direction o are calculated as:

[0087]

[0088] φ no (x,y)=arctan(e no (x,y),o no (x,y)

[0089] In the formula, A o (x,y) is image I SR The amplitude of the pixel point (x, y) in direction o, φ no (x,y) is image I SR The phase of the upper pixel (x, y) in direction o;

[0090] The phase consistency response is then calculated based on the amplitude and phase, which is:

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] In the formula, PC o (x,y) image I SR The phase consistency response of the pixel point (x, y) in direction o (o = 1 to 6), W o (x, y) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no (x,y) is a more sensitive phase deviation, ε is a small constant to avoid division by zero, and E(x,y) is the local energy function.

[0097] Step 1.3, calculate image I SR The covariance matrix of the phase consistency response of each pixel on the graph is obtained, and the maximum moment of each pixel is obtained based on the covariance matrix, that is, Figure 5 The specific implementation process is as follows:

[0098] First, I SR The phase consistency response of the pixel point (x, y) in each direction is projected into the x and y components as follows:

[0099]

[0100]

[0101] In the formula, PC x (x,y), PC y (x, y) are the components of the phase consistency response of the pixel point (x, y) in the x and y directions, respectively, θ o is the angle corresponding to direction o, θ o ∈θ, θ is the set of angles corresponding to all directions;

[0102] Secondly, based on PC x (x,y), PC y (x, y) constructs the covariance matrix of the phase consistency response of each pixel on the image, which is:

[0103]

[0104] Where G(x,y) is the covariance matrix of the phase consistency response of the pixel point (x,y);

[0105] Finally, the maximum moment corresponding to the maximum singular value of the covariance matrix G(x,y) is the maximum moment M(x,y) of the pixel point (x,y).

[0106] Step 1.4, based on image I SR The maximum moment of each pixel point on the grid is used to obtain the texture richness corresponding to each pixel point. The specific implementation method is as follows:

[0107] According to the design idea of ​​edge density, the intensity value of the maximum moment is used as the weight of the edge feature, and its intensity value is the weight of the edge. The texture richness is calculated in a local area of ​​size N×N, which is:

[0108]

[0109] Wherein, TR(x,y) is the texture richness of the pixel (x,y), M(x+i,y+j) is the maximum moment of the pixel (x+i,y+j), and in this embodiment, the default value of N is set to 21, which is insensitive to noise under the premise of being able to characterize the local texture distribution. In addition, the noise can be further filtered out by adding Gaussian weights;

[0110] After getting the image I SR After calculating the texture richness TR of each pixel, we finally get the image I SR Texture richness feature maps of the same size, i.e. Figure 6 shown.

[0111] Step 2: Threshold the texture richness feature map into a binary mask TR_MASK to distinguish between texture-dense areas and texture-free areas in the SAR image scene. The process is as follows:

[0112] Normalize the texture richness of each pixel on the texture richness feature map to the range of [0,1] through the maximum and minimum values;

[0113] The threshold TR_THRESH is used to perform threshold segmentation on the normalized texture richness feature map:

[0114] The texture richness of the pixels whose texture richness is greater than or equal to the threshold TR_THRESH in the normalized texture richness feature map is set to 1;

[0115] The texture richness of the pixels whose texture richness in the normalized texture richness feature map is less than the threshold TR_THRESH is set to 0;

[0116] Among them, the area where the texture richness is set to 1 in the texture richness feature map is the texture-dense area, and the area where the texture richness is set to 0 is the texture-free area.

[0117] Step 3, uniformly collect unit feature descriptors in the texture-dense area on the SAR image and merge them into the first feature vector; uniformly collect unit feature descriptors in the corresponding area where the binary mask is 1 on the visible light remote sensing image and merge them into the second feature vector. The specific implementation process is:

[0118] The unit feature descriptors feat_desp are uniformly collected at intervals of several pixels d (usually the unit radius of the dense feature descriptor) in the area marked as valid texture by TR_MASK, and merged into the final feature vector V, as Figure 7 As shown, compared to Figure 8 The conventional feature descriptor extraction method shown in FIG. 1 can effectively remove the interference information of the texture-free area. Therefore, the acquisition process of the first feature vector and the second feature vector is as follows:

[0119] At intervals of d pixels, the unit feature descriptors feat_desp_sar are uniformly collected in the texture-dense areas of the SAR image and merged into the final first feature vector V A ,for:

[0120]

[0121] At intervals of d pixels, the unit feature descriptors feat_desp_opt are uniformly collected in the corresponding areas of the binary mask on the visible light remote sensing image and merged into the final second feature vector V B ,for:

[0122]

[0123] Where TR_MASK is the binary mask after thresholding the texture richness feature map.

[0124] It should be noted that the types of the unit feature descriptors in this embodiment can be replaced according to actual needs, and the types include but are not limited to MIND descriptors, ALOST descriptors and HOPC descriptors.

[0125] Step 4: Based on the first eigenvector and the second eigenvector, a similarity measure between the SAR and visible light remote sensing images is obtained, and a transformation matrix between the SAR and visible light remote sensing images is determined using a template matching strategy.

[0126] The similarity measurement between SAR and visible light remote sensing images is as follows:

[0127]

[0128] Where TRncc is the similarity measure between SAR and visible light remote sensing images, V A (k) is the k-th unit feature descriptor in the first feature vector, is the mean of all unit feature descriptors in the first eigenvector, V B (k) is the k-th unit feature descriptor in the second feature vector, is the mean of all unit feature descriptors in the second feature vector, n is the total number of unit feature descriptors in the first feature vector and the second feature vector;

[0129] After obtaining the similarity measure TRncc between SAR and visible light remote sensing images, TRncc is used as the similarity measure between template windows, and the template matching strategy is used to detect corresponding points in the search area, and then the transformation matrix between the two images can be determined.

[0130] It should be noted that, because the present invention is a similarity measurement method in template matching, it needs to be integrated into the existing multimodal image matching method before it can be applied to the actual SAR and visible light image matching task. And the present invention can be tested on SAR and visible light remote sensing datasets containing scenes with non-uniform texture distribution.

[0131] The similarity measurement method for matching SAR and visible light remote sensing images based on texture richness in the present invention is further described below with reference to specific examples.

[0132] First, the texture richness features and the generated masks proposed in this invention are visualized on images with non-uniform texture distribution. Figure 9-10 As shown, where: Fig. 9 (a) Fig. 9 (b) shows the SAR images of two ports. Fig. 9 (a′), Fig. 9 (b′) is the corresponding texture richness feature map; Fig. 9 (c) Fig. 9 (d) shows the SAR images of two islands. Fig. 9 (c′), Fig. 9 (d′) is the corresponding texture richness feature map; Fig. 9 (e) Fig. 9 (f) shows two SAR images with missing information. Fig. 9 (e′), Fig. 9 (f′) is the corresponding texture richness feature map. Fig.10 (a) Fig.10 (b) Fig.10 (c) SAR images of ports, islands, and missing information, respectively. Fig.10 (a-1), Fig.10 (b-1), Fig.10 (c-1) When TR_THRESH = 0.1 Fig.10 (a) Fig.10 (b) Fig.10 (c) The corresponding mask image, Fig.10 (a-2), Fig.10 (b-2), Fig.10 (c-2) When TR_THRESH = 0.25 Fig.10 (a) Fig.10 (b) Fig.10 (c) The corresponding mask image, Fig.10 (a-3), Fig.10 (b-3), Fig.10 (c-3) When TR_THRESH = 0.5 Fig.10 (a) Fig.10 (b) Fig.10 (c) The corresponding mask image.

[0133] Depend on Fig.10 It can be seen that the proposed texture richness mask can effectively divide the scene into texture-dense and texture-free areas (sea surface or image missing). The similarity measurement method proposed in the present invention is integrated into the HOPC template matching, denoted as HOPC-TRncc. The following is a comparison of the results of the similarity measurement HOPCncc (comparative example) in the method (implementation) provided by the present invention and the original HOPC method. The comparative experiment uses high-resolution spaceborne SAR images collected by the TerraSAR-X satellite in Hainan and image pairs with non-uniform texture distribution of Google map optical images. The correct match rate (CMR), root mean square error (RMSE) and running time commonly used in the field of image matching research are used as evaluation indicators, and the specific definitions are as follows:

[0134] CMR=NCM / C

[0135]

[0136] In the formula, NCM represents the number of correctly matched point pairs, C is the total number of matched point pairs, (x i ,y i ) is the coordinate of the corresponding point calculated for the i-th interest point, (x o ,y o ) represents the true value of the coordinates of the corresponding points.

[0137] Figure 11-13 It represents the matching performance comparison between HOPCncc and the similarity metric HOPC-TRncc proposed in the present invention, where: Fig.11Schematic diagram of the relationship between CMR calculated on an image pair with non-uniform texture distribution and the texture richness threshold TR_THRESH. Fig.12 Schematic diagram of the relationship between RMSE calculated on an image pair with non-uniform texture distribution and the texture richness threshold TR_THRESH. Fig.13 The figure is a schematic diagram of the relationship between the running time and the feature vector length on the image pair with non-uniform texture distribution and the texture richness threshold TR_THRESH. The matching accuracy of the method can be compared by CMR and RMSE indicators, and the running time can be used to compare the computational efficiency of the method. Compared with other methods, the matching accuracy of the method provided by the present invention is higher, the CMR value is improved by 11 percentage points compared with HOPCncc, and the RMSE value is reduced by more than 0.16 pixels. The running time is reduced by more than 50% due to the significant reduction in the feature vector length, indicating that the similarity metric proposed in the present invention has higher accuracy and faster calculation speed. In the results of the method provided by the present invention, the use of similarity metric to detect CP is more accurate and faster, so it can be proved that the present invention can provide a more effective similarity metric for non-uniform texture scenes.

[0138] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A similarity measurement method for matching SAR and visible light remote sensing images, characterized in that: The steps include: Step 1, obtaining the texture richness of each pixel on the SAR image, and obtaining a texture richness feature map characterized by the texture richness; Step 2: threshold the texture richness feature map into a binary mask to distinguish between texture-dense areas and texture-free areas in the SAR image scene; Step 3, uniformly collecting unit feature descriptors in the texture-dense area on the SAR image and merging them into a first feature vector; uniformly collecting unit feature descriptors in the corresponding area of ​​the binary mask on the visible light remote sensing image and merging them into a second feature vector; Step 4, based on the first eigenvector and the second eigenvector, obtain a similarity measure between the SAR and visible light remote sensing images, and use a template matching strategy to determine a transformation matrix between the SAR and visible light remote sensing images; In step 2, the texture richness feature map is thresholded into a binary mask, specifically: Normalize the texture richness of each pixel on the texture richness feature map to the range of [0,1] through the maximum and minimum values; Using threshold TR _ THRESH Perform threshold segmentation on the normalized texture richness feature map: The texture richness in the normalized texture richness feature map is greater than or equal to the threshold TR _ THRESH The texture richness of the pixel is set to 1; The texture richness in the normalized texture richness feature map is less than the threshold TR _ THRESH The texture richness of the pixel is set to 0; Among them, the area where the texture richness is set to 1 in the texture richness feature map is the texture-dense area, and the area where the texture richness is set to 0 is the texture-free area; In step 3, the process of obtaining the first feature vector and the second feature vector is: by d The unit feature descriptors are uniformly collected in the dense texture area of ​​the SAR image at an interval of pixels. feat_desp_ sar , merged into the final first eigenvector V A ,for: by d The unit feature descriptors are uniformly collected in the corresponding area of ​​the binary mask on the visible light remote sensing image with an interval of pixels. feat_desp_opt , merged into the final second eigenvector V B ,for: In the formula, TR_MASK is the binary mask after thresholding the texture richness feature map; The similarity measurement between the SAR and visible light remote sensing images is specifically: Where TRncc is the similarity measure between SAR and visible light remote sensing images, V A (k) is the k-th unit feature descriptor in the first feature vector, is the mean of all unit feature descriptors in the first eigenvector, V B (k) is the k-th unit feature descriptor in the second feature vector, is the mean of all unit feature descriptors in the second eigenvector, n is the total number of unit feature descriptors in the first feature vector and the second feature vector.

2. The similarity measurement method for matching SAR and visible light remote sensing images according to claim 1, characterized in that: In step 1, the step of obtaining the texture richness of each pixel on the SAR image specifically includes: Step 1.1: SAR image I SAR Perform speckle suppression to obtain a filtered image I SR ; Step 1.2, image I SR Perform convolution and obtain the image from the amplitude and phase of the convolution response I SR Phase consistency response of each pixel in multiple directions; Step 1.3, calculate the image I SR The covariance matrix of the phase consistency response of each pixel point is obtained, and the maximum moment of each pixel point is obtained based on the covariance matrix; Step 1.4, Image-based I SR The maximum moment of each pixel on the grid is used to obtain the texture richness corresponding to each pixel.

3. The similarity measurement method for matching SAR and visible light remote sensing images according to claim 2, characterized in that: Step 1.2 is as follows: The image is processed using Log-Gabor wavelet with multiple scales and directions. I SR Filtering is performed, passing the real part of the filter and the imaginary part For images I SR Convolution gets scale n and direction o Response on e no ( x , y )and o no ( x , y ), calculated in the direction o The amplitude and phase of are: In the formula, A o ( x , y ) is the image I SR Upper pixel ( x , y ) in the direction The amplitude on ( x , y ) is the image I SR Upper pixel ( x , y ) in the direction The phase on; The phase consistency response is calculated based on the amplitude and phase, which is: In the formula, PC o ( x , y )image I SR Upper pixel ( x , y ) in the direction The phase consistency response on W o ( x , y ) is the weighting function constructed by applying the sigmoid function to the filter response expansion value, T is the noise threshold, ΔΦ no ( x , y ) is a more sensitive phase deviation, ε A small constant to avoid division by zero.

4. The similarity measurement method for matching SAR and visible light remote sensing images according to claim 3 is characterized in that: Step 1.3 is as follows: First, I SR Upper pixel ( x , y )The phase consistency response in each direction is projected onto and The amount is: In the formula, PC x ( x , y ), PC y ( x , y ) are pixel points ( x , y ) has a phase consistency response in x , y The weight in direction, θ o For direction o The corresponding angle, θ o ∈ θ , θ is the set of angles corresponding to all directions; Secondly, based on PC x ( x , y ), PC y ( x , y ) to get the pixel point ( x , y )The covariance matrix of the phase-consistent response is: In the formula, is a pixel ( x , y )The covariance matrix of the phase-consistent response; Finally, the covariance matrix The maximum moment corresponding to the maximum singular value of is the pixel point ( x , y ) .

5. The similarity measurement method for matching SAR and visible light remote sensing images according to claim 2, characterized in that: Step 1.4 is as follows: The strength value of the maximum moment is used as the weight of the edge feature, and its strength value is the weight of the edge. N × N The texture richness is calculated in the local area of ​​​​the In the formula, TR ( x , y ) is the pixel point ( x , y )’s texture richness, M ( x + i , y + j ) is the pixel point ( x + i , y + j )’s maximum moment.

Citation Information

Patent Citations

  • Heterogenous remote sensing image registration method based on structural similarity

    CN113643334A

  • Fast and robust multimodal remote sensing images matching method and system

    US20200226413A1