Optical and SAR image matching method based on improved RTV model and structural saliency

Abstract

The invention relates to an optical and SAR image matching method based on an improved RTV model and structural saliency, and the method comprises the following steps: obtaining an optical image and an SAR image, inputting the optical image and the SAR image into the improved RTV model, and constructing a nonlinear scale space; a Sobel operator is adopted to extract gradient information from the optical image, and an ROEWA edge detection operator is adopted to extract gradient information from the SAR image; extracting feature points from each layer of image in the non-linear scale space, and calculating a structural information image according to edge saliency information and the filtering graph, so as to obtain a main direction of each feature point; further performing feature description on the extracted feature points by adopting an improved HOG descriptor to obtain a feature descriptor, merging gradient statistical information in the improved HOG descriptor, and performing normalization processing; and matching the feature descriptors of the optical image and the SAR image and eliminating mismatching points. Compared with the prior art, the registration effect can be further improved.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image intelligent processing, and in particular relates to an optical and SAR image matching method based on an improved RTV model and structural significance. Background Art

[0002] In recent years, earth observation technology has made great progress, and the imaging quality and mapping accuracy of spaceborne optical images and synthetic aperture radar (SAR) images have become increasingly higher. Since optical images contain rich spectral information and SAR images reflect structural information well, the combination of optical remote sensing images and SAR images has become the mainstream trend in image data applications. However, due to the differences in imaging methods, imaging conditions, and sensors between optical and SAR remote sensing images, the two types of images have significant differences, resulting in the registration of optical and SAR images becoming a difficult problem in the field of photogrammetry and remote sensing applications.

[0003] At present, there are mainly three ways to solve the problem of matching optical and SAR images: First, region-based matching methods, such as Normalized Cross Correlation (NCC) and Mutual Information (MI), but these methods require that the image has certain prior information to eliminate the rotation and scale differences between images, and the application scenarios are very limited; second, feature-based matching methods, which do not directly rely on grayscale information, but look for common significant features between images based on grayscale information, such as SIFT, SURF, ORB and RIFT, etc. Feature-based matching methods usually have scale and rotation invariance, but the matching effect between heterogeneous images is extremely poor; third, matching methods based on deep learning, which introduce convolutional neural networks (CNN) and generative adversarial networks (GANs). The deep learning model has a poor generalization ability due to the difficulty in collecting remote sensing image data sets and the complex and changeable environment, which limits the application of this type of method. Therefore, it is necessary to redesign a highly robust optical and SAR image matching method to effectively deal with the challenges of nonlinear radiation differences, rotation differences, scale differences, and geometric distortions in the optical and SAR image matching process, and further improve the matching effect. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an optical and SAR image matching method based on an improved RTV model and structural saliency, so as to effectively cope with the challenges of nonlinear radiation difference, rotation difference, scale difference and geometric distortion in the optical and SAR image matching process, and further improve the matching effect.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides an optical and SAR image matching method based on an improved RTV model and structural saliency, comprising the following steps:

[0007] Obtain optical images and SAR images, input them into the improved RTV model, and construct optical nonlinear scale space and SAR nonlinear scale space;

[0008] A Sobel operator is used to extract gradient information from the optical image, and a ROEWA edge detection operator is used to extract gradient information from the SAR image;

[0009] Extracting feature points on the filtered images in the optical nonlinear scale space and the SAR nonlinear scale space respectively, and calculating the structural information image according to the edge saliency information and the filtered image, thereby obtaining the main direction of each feature point, wherein the edge saliency information is the edge confidence of the filtered image;

[0010] The extracted feature points are further described using an improved HOG descriptor to obtain feature descriptors of optical images and SAR images, wherein the improved HOG descriptor is normalized after merging the gradient statistical information;

[0011] The feature descriptors of the optical image and the SAR image are matched, and the mismatched points are eliminated to finally obtain the matching results of the optical and SAR images.

[0012] Furthermore, the improved RTV model is specifically as follows:

[0013]

[0014] Among them, I represents the input image, S represents the output structural image, p is the index of the pixel, ε is a positive value to avoid the denominator being zero, and λ is the weight parameter. and represents the potential energy in the x direction, and represents the potential energy in the y direction;

[0015]

[0016]

[0017] Among them, q is the index of all pixels in a square area centered on p, g is the Gaussian kernel function,

[0018]

[0019] Among them, (x p ,y p ) is the position coordinate of pixel p, (x q ,y q ) is the position coordinate of pixel point q, and σ is the standard deviation of the Gaussian kernel function.

[0020] Furthermore, a FAST detector is used to extract feature points on the original images in the optical nonlinear scale space and the SAR nonlinear scale space, respectively. The specific process is as follows:

[0021] In each layer of the nonlinear scale space, select the top N feature points in the response value ranking;

[0022] The local non-maximum suppression algorithm is used to eliminate the redundant feature information of the selected feature points.

[0023] Furthermore, the edge saliency information is obtained through a machine learning algorithm.

[0024] Furthermore, the calculation formula of the structure information image is as follows:

[0025]

[0026] Among them, BW(x,N) is the structural information image, ⊙ represents the Hadamard product of two matrices, I(x,y) is the filtered image, is the edge saliency information of the image.

[0027] Furthermore, the process of obtaining the main direction of each feature point is as follows:

[0028] Based on the position information and scale information of the feature points, local structural information is extracted from the structural information image, and local normalization processing is performed to obtain a normalized local image. Direction information statistics are performed on the normalized local image to obtain the main direction of the feature points.

[0029] Furthermore, after performing statistics on the direction information on the normalized local image, the main direction angle θ is obtained. i ′, for θ i ′ performs the following operation, and finally obtains the main direction θ of the feature point i :

[0030] θ i =mod(θ i′,180).

[0031] Furthermore, the specific process of using the improved HOG descriptor for feature description is as follows:

[0032] S401, obtaining gradient information Ix and Iy of the original image;

[0033] S402: Extract the gradient information Ix in the neighborhood of the i-th feature point of the original image i and Iy i ;

[0034] S403, according to the main direction θ of the feature point i Calculate the gradient information after rotation and And calculate the gradient amplitude IG i and gradient direction OG i , the calculation formula is as follows:

[0035]

[0036]

[0037] S404, dividing the local image into multiple units, and counting the direction histogram of each unit;

[0038] S405, combining multiple units according to a set size to obtain multiple unit blocks;

[0039] S406 , normalizing the histogram statistical results in each unit block, and combining the statistical results of each unit block in order to obtain a feature descriptor.

[0040] Furthermore, in step S405, normalization processing is performed using a Z-score model.

[0041] Furthermore, the NNDR algorithm is first used to match the feature descriptors of the optical image and the SAR image, and then the FSC algorithm is used to eliminate the mismatched points.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention proposes an optical and SAR image matching method based on an improved RTV model and structural saliency. First, an optical image and a SAR image are obtained, and the improved RTV model is input respectively to construct an optical nonlinear scale space and a SAR nonlinear scale space, so that structural data can be more effectively extracted from a structural texture image, and the time complexity can be reduced, and the efficiency of structural feature extraction can be improved. In addition, the influence of complex noise can be weakened. Then, a Sobel operator is used to extract gradient information from the optical image, and a ROEWA edge detection operator is used to extract gradient information from the SAR image. The Sobel operator is a first-order differential operator. Combined with Gaussian smoothing and differential derivation, it can effectively resist images with additive noise in optical images. The ROEWA edge detection operator calculates the ratio of the horizontal neighborhood and the vertical field with the current pixel as the center, and then uses the ratio result as the horizontal gradient and vertical gradient of the point. At the same time, an exponential weighting method is used to increase the contribution rate of the central pixel, which has a good inhibitory effect on the multiplicative speckle noise of the SAR image. Compared with the use of a single gradient operator, this step can preserve the image quality to a greater extent. The consistency of the gradient information of optical and SAR images is verified; secondly, feature points are extracted on the filtered images in the optical nonlinear scale space and the SAR nonlinear scale space respectively, and the structural information image is calculated based on the edge saliency information and the filtered image, so as to obtain the main direction of each feature point. The edge saliency information is the edge confidence of the filtered image. The introduction of edge saliency information helps to suppress the speckle noise of the SAR image, overcome the difficulty of image matching caused by rotation, and weaken the influence of image nonlinear intensity distortion and image extreme values; then, the improved HOG descriptor is used to further describe the extracted feature points to obtain the feature descriptors of the optical image and the SAR image. In the improved HOG descriptor, the gradient statistical information is merged and normalized, which can improve the comparability of the data; finally, the feature descriptors of the optical image and the SAR image are matched, and the mismatched points are eliminated to obtain the matching results of the optical and SAR images; the above method can effectively deal with the challenges of nonlinear radiation difference, rotation difference, scale difference and geometric distortion in the matching process of optical and SAR images, and further improve the matching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 Schematic diagram of the experimental results of qualitative analysis using different methods.

[0046] Among them, (2a) is the method of the present invention, (2b) is OS-SIFT, (2c) is LPSO, (2d) is HOWP, and (2e) is RIFT2;

[0047] Figure 3Schematic diagram of experimental results of quantitative analysis using different methods.

[0048] Among them, (3a) is the number of correct matches NCM, and (3b) is the success rate SR (%). DETAILED DESCRIPTION

[0049] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0050] Example:

[0051] This embodiment provides an optical and SAR image matching method (WSROS) based on an improved RTV model and structural saliency. The main process is as follows: Figure 1 As shown in the figure: For a given optical image and SAR image, the improved RTV (Relative Total Variation) model is first used to construct a nonlinear scale space, and the Sobel operator is used to extract gradient information from the optical image, and the ROEWA operator is used to extract gradient information from the SAR image; then, feature points are extracted on the gradient image, and the main direction of the feature points is calculated based on the edge significance information; finally, the improved HOG descriptor is used to construct a feature descriptor for the feature points, and feature matching and mismatch point removal are performed accordingly. The specific descriptions of each step are as follows:

[0052] S1. Use the improved RTV model to construct the optical nonlinear scale space and SAR nonlinear scale space.

[0053] The improved RTV model is as follows:

[0054]

[0055] Among them, I represents the input image, S represents the output structural image, p is the index of the pixel, ε is a positive value to avoid the denominator being zero, and λ is the weight parameter. and represents a potential energy in the x direction, and represents some potential energy in the y direction:

[0056]

[0057] Among them, q is the index of all pixels in a square area centered on p, g is the Gaussian kernel function,

[0058]

[0059] Among them, (xp ,y p ) is the position coordinate of pixel p, (x q ,y q ) is the position coordinate of pixel point q, and σ is the standard deviation of the Gaussian kernel function.

[0060] The improved RTV model can more effectively extract structural data from structural texture images, while reducing time complexity and improving the efficiency of structural feature extraction.

[0061] S2. The Sobel operator is used to extract gradient information from optical images, and the ROEWA edge detection operator is used to extract gradient information from SAR images.

[0062] The Sobel operator is a first-order differential operator. It can effectively resist additive noise in optical images by combining Gaussian smoothing and differential derivation. However, there are large nonlinear radiation differences between optical images and SAR images. At the same time, due to the influence of the imaging mechanism, SAR images are subject to large multiplicative noise. It is difficult to accurately express the gradient information of SAR images using the Sobel operator.

[0063] The weighted exponential average ratio (ROEWA) edge detection operator calculates the ratio of the horizontal neighborhood and the vertical area with the current pixel as the center, and then uses the ratio result as the horizontal gradient and vertical gradient of the point. At the same time, the exponential weighting method is used to increase the contribution rate of the central pixel, which has a good inhibitory effect on the multiplicative speckle noise of the SAR image. This embodiment uses the Sobel operator and the ROEWA edge detection operator to calculate the gradient information of the optical and SAR images respectively. Compared with using a single gradient operator, the consistency of the gradient information of the optical and SAR images can be guaranteed to a greater extent.

[0064] S3. Extract feature points on the filtered images in the optical nonlinear scale space and the SAR nonlinear scale space respectively, and calculate the structural information image according to the edge saliency information and the filtered image, so as to obtain the main direction of each feature point.

[0065] The FAST detector is used to extract feature points on the original images in the optical nonlinear scale space and the SAR nonlinear scale space respectively. The specific process is as follows: in each layer of the image in the nonlinear scale space, the top N feature points with the highest response value are selected; the local non-maximum suppression algorithm is used to eliminate the redundant feature information of the selected feature points.

[0066] The edge saliency information is the edge confidence of the original image (i.e., the probability of whether a pixel is on the edge, 0 to 1), which is obtained through machine learning methods, specifically calculated using the existing open source toolbox Structured Edge Detection Toolbox (Matlab).

[0067] Assume that the original image is I(x,y), and its edge saliency information is The calculation formula of the structural information image BW(x,y) is as follows:

[0068]

[0069] Where ⊙ represents the Hadamard product of two matrices.

[0070] The above method can improve the contribution of edge information in the calculation of the centroid. Since the extreme value in the edge confidence is small, this method helps to suppress the speckle noise of SAR images and remove the influence of the extreme value of the connected area. In addition, edge and structural information have higher similarity in heterogeneous images, so giving higher weight to edge structural information when calculating the centroid has a good effect.

[0071] Considering that BW(x, y) is based on the extraction of structural information of the entire image, and this embodiment hopes that local information will dominate the direction calculation, therefore, a local image is extracted at the centroid of the feature point, and local normalization is performed, and the direction information is counted on the normalized local image to obtain the main direction of the feature point.

[0072] It is particularly important to note that after performing statistics on the direction information on the normalized local image, the main direction angle θ is obtained. i ′, considering the gradient reversal phenomenon of the same type of objects in the image, so θ i ′ performs the following operation, and finally obtains the main direction θ of the feature point i :

[0073] θ i =mod(θ i ′,180)

[0074] That is, the main direction θ i The angles are constrained to [0,180).

[0075] S4. Use the improved HOG descriptor (HOG-like) to further describe the extracted feature points and obtain feature descriptors of optical images and SAR images.

[0076] Since the time and space complexity of the Histogram of Oriented Gradient (HOG) descriptor is large, the memory usage is large and the time consumption is high in both feature description and feature matching. Therefore, this embodiment proposes to use a HOG-like descriptor for feature description.

[0077] In the improved HOG descriptor, the gradient statistics are merged and normalized. The specific process of using the improved HOG descriptor for feature description is as follows:

[0078] S401: Obtain gradient information Ix and Iy of the original image.

[0079] S402: Extract the gradient information Ix in the neighborhood of the i-th feature point of the original image i and Iy i .

[0080] S403, according to the main direction θ of the feature point i Calculate the gradient information after rotation and And calculate the gradient amplitude IG i and gradient direction OG i , the calculation formula is as follows:

[0081]

[0082] S404, dividing the local image into units of 8×8 pixel size, determining the number of bins for histogram statistics, and counting the direction histogram of each cell unit in turn.

[0083] S405. Multiple units are combined into a 2×2 cell size to obtain multiple unit blocks. The unimproved HOG descriptor usually has overlaps between two blocks. The overlap will cause the descriptor to be too large and have a large amount of redundancy. The improved HOG descriptor no longer has overlaps between adjacent blocks, and activity statistics are performed directly. Assuming radius = 48, cellsize = [8, 8], blocksize = [2, 2], bins = 4, then length (HOG) = 1936, length (HOG_like) = 576.

[0084] S406. After all blocks are combined, the histogram statistics in the unit block need to be normalized in consideration of local illumination changes. This embodiment uses the Z-score model to normalize the histogram statistics in each unit block, and then combines the statistics of each unit block in order to obtain a feature descriptor. It can be verified through experiments that the effect of Z-score normalization is better than the L2 norm normalization model in HOG.

[0085] S5. The NNDR algorithm is used to match the feature descriptors of the optical image and the SAR image, and then the FSC algorithm is used to eliminate the mismatched points.

[0086] The NNDR (Nearest Neighbor Distance Ratio) algorithm is the full name of the nearest neighbor distance ratio, which is used to determine whether two feature vectors are correct matching pairs (initial matching pairs). Since there are a large number of mismatching points in the initial matching pairs calculated by NNDR, other algorithms are needed to eliminate the wrong matching point pairs from the initial matching results.

[0087] The FSC algorithm is an improvement on the RANSAC algorithm. It determines the correct matches in the initial matching point pairs in a random manner and eliminates the wrong matching point pairs. After the FSC algorithm eliminates the wrong matching pairs, it is the matching result of the entire algorithm.

[0088] Based on the feature descriptor constructed by HOG-like, the NNDR (Nearest Neighbor Distance Ratio) algorithm is used to calculate the initial matching point pair. For example, there are 5,000 feature points on the optical image, each of which corresponds to a descriptor; there are 3,000 feature points and corresponding descriptors on the SAR image. For each feature point in the optical image, the feature point with the closest feature descriptor needs to be found among all the feature points on the SAR image as the initial matching point pair.

[0089] The feature points include position information (x, y), scale information σ and main direction θ i , the position information (x, y) is used to determine the position on the image; the scale information σ is used to determine the position (which layer) of the feature point in the scale pyramid, so as to obtain the local gradient information of the position information (x, y) of the corresponding layer to ensure the scale invariance of the algorithm; the main direction θ i Used to rotate the gradient information counterclockwise by θ i This returns the center of mass to 0 and ensures rotation invariance.

[0090] Since there are a large number of mismatched points in the initial matching pairs calculated by NNDR, other algorithms are needed to remove the wrong matching point pairs from the initial matching results. The FSC algorithm is an improvement on the RANSAC algorithm. It determines the correct matches in the initial matching point pairs in a random way and removes the wrong matching point pairs. After the FSC algorithm removes the wrong matching pairs, it is the matching result of the entire algorithm.

[0091] To verify the effectiveness of the above method, this embodiment is tested on the open source OPT-SAR dataset. The OPT-SAR dataset is produced and provided by the Key Laboratory of Space Information Processing and Application System Technology of the Chinese Academy of Sciences. The SAR images in this dataset are collected from China's C-band sensor Gaofen-3, imaging mode focus mode; optical images are collected from Google Earth with a resolution of 1m. The dataset consists of 2673 non-overlapping patch pairs of size 512*512 and 10692 non-overlapping patch pairs of size 256*256. This embodiment compares the matching effects of different algorithms from both qualitative and quantitative aspects:

[0092] (1) Qualitative analysis

[0093] The results of qualitative comparison between the method proposed in this embodiment and other algorithms are as follows Figure 2 As shown in the figure, (2a) corresponds to the method of the present invention, (2b) corresponds to OS-SIFT, (2c) corresponds to LPSO, (2d) corresponds to HOWP, and (2e) corresponds to RIFT2. It can be seen that the patented algorithm still obtains a large number of correct matching pairs when the image has severe nonlinear intensity distortion and large rotation, and the effect is better than OS-SIFT, LPSO, HOWP and RIFT2 algorithms.

[0094] (2) Quantitative analysis

[0095] In order to better evaluate the difference between the algorithm proposed in this embodiment and other algorithms, the results of quantitative evaluation will be shown below. In order to better show the results of quantitative analysis, during the experiment, the 424 pairs of optical-SAR image pairs in the test data set were randomly divided into 8 groups, each with 53 pairs. At the same time, the optical images in the image pairs were rotated and scaled. Figure 3 (3a) and (3b) show the quantitative comparison results of the WSROS algorithm (the method of this embodiment), OS-SIFT algorithm, HOWP algorithm, LPSO algorithm and RIFT2 algorithm in terms of the number of correct matches and success rate respectively.

[0096] from Figure 3 As can be seen from (3a), the number of feature points extracted by the method (WSROS) proposed in this embodiment is much greater than that of RIFT2, LPSO and OS-SIFT algorithms, and slightly higher than that of HOWP algorithm. The average number of correct matches on 8 pairs of test sets is greater than 100, and the number of correct matches on each test set is greater than that of HOWP algorithm. Figure 3As can be seen from (3b), the method (WSROS) proposed in this embodiment has a success rate of 100% on the five pairs of data sets (2), (3), (4), (5), and (7). In the data sets (1), (6), and (8), only one pair of images has a mismatch problem. This is because the structural information in these three pairs of images is not rich enough and is seriously affected by noise images. The other algorithms also fail to match correctly. Figure 3 It can also be seen from (3b) that the HOWP algorithm has also achieved good results in terms of success rate, while the other three algorithms performed very poorly. The success rate of RIFT2 is about 75%, while the SR of LPSO and OS-SIFT is less than 60%. Figure 3 It can be seen from (3a) and (3b) that the WSROS algorithm proposed in this embodiment is ahead of other existing algorithms in terms of correct matching number and success rate (slightly better than the HOWP algorithm, but far ahead of the OS-SIFT, LPSO and RIFT2 algorithms).

[0097] In summary, compared with existing methods, our method achieves better results in terms of matching accuracy, number of correct matches and success rate.

[0098] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0099] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for matching optical and SAR images based on an improved RTV model and structural saliency, characterized in that: The following steps are involved: Obtain optical images and SAR images, input them into the improved RTV model, and construct optical nonlinear scale space and SAR nonlinear scale space; A Sobel operator is used to extract gradient information from the optical image, and a ROEWA edge detection operator is used to extract gradient information from the SAR image; Extracting feature points on the filtered images in the optical nonlinear scale space and the SAR nonlinear scale space respectively, and calculating the structural information image according to the edge saliency information and the filtered image, thereby obtaining the main direction of each feature point, wherein the edge saliency information is the edge confidence of the filtered image; The extracted feature points are further described using an improved HOG descriptor to obtain feature descriptors of optical images and SAR images, wherein the improved HOG descriptor is normalized after merging the gradient statistical information; The feature descriptors of the optical image and the SAR image are matched, and the mismatched points are eliminated to finally obtain the matching results of the optical and SAR images.

2. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The improved RTV model is as follows: Among them, I represents the input image, S represents the output structural image, p is the index of the pixel, ε is a positive value to avoid the denominator being zero, and λ is the weight parameter. and represents the potential energy in the x direction, and represents the potential energy in the y direction; Among them, q is the index of all pixels in a square area centered on p, g is the Gaussian kernel function, Among them, (x p ,y p ) is the position coordinate of pixel p, (x q ,y q ) is the position coordinate of pixel point q, and σ is the standard deviation of the Gaussian kernel function.

3. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The FAST detector is used to extract feature points on the original images in the optical nonlinear scale space and the SAR nonlinear scale space, respectively. The specific process is as follows: In each layer of the nonlinear scale space, select the top N feature points in the response value ranking; The local non-maximum suppression algorithm is used to eliminate the redundant feature information of the selected feature points.

4. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The edge saliency information is obtained through a machine learning algorithm.

5. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The calculation formula of the structural information image is as follows: Among them, BW(x,y) is the structural information image, ⊙ represents the Hadamard product of two matrices, I(x,y) is the filtered image, is the edge saliency information of the image.

6. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The process of obtaining the main direction of each feature point is as follows: Based on the position information and scale information of the feature points, local structural information is extracted from the structural information image, and local normalization processing is performed to obtain a normalized local image. Direction information statistics are performed on the normalized local image to obtain the main direction of the feature points.

7. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 6, characterized in that: After performing statistics on the direction information on the normalized local image, the main direction angle θ is obtained. i ′, for θ i ′ performs the following operation, and finally obtains the main direction θ of the feature point i : i i =mod(θ i ′,180).

8. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: The specific process of using the improved HOG descriptor for feature description is as follows: S401, obtaining gradient information Ix and Iy of the original image; S402: Extract the gradient information Ix in the neighborhood of the i-th feature point of the original image i and Iy i ; S403, according to the main direction θ of the feature point i Calculate the gradient information after rotation and And calculate the gradient amplitude IG i and gradient direction OG i , the calculation formula is as follows: S404, dividing the local image into multiple units, and counting the direction histogram of each unit; S405, combining multiple units according to a set size to obtain multiple unit blocks; S406 , normalizing the histogram statistical results in each unit block, and combining the statistical results of each unit block in order to obtain a feature descriptor.

9. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 8, characterized in that: In step S405, normalization processing is performed using a Z-score model.

10. The optical and SAR image matching method based on the improved RTV model and structural saliency according to claim 1, characterized in that: Firstly, the NNDR algorithm is used to match the feature descriptors of the optical image and the SAR image, and then the FSC algorithm is used to eliminate the mismatched points.

Images