An automatic registration method for optical and SAR images based on bidirectional style transfer

Abstract

The present invention discloses an automatic registration method for optical and SAR images based on bidirectional style transfer, and belongs to the technical field of remote sensing image intelligent processing. The present invention first converts optical and SAR images into pseudo SAR and pseudo optical images through a CycleGAN style transfer network to achieve homogenization of heterogeneous images, and then uses SIFT to extract feature point sets from the optical and SAR images and the converted pseudo SAR and pseudo optical images, respectively, to obtain a feature point set of the optical image to be registered and a feature point set of the SAR image to be registered, then extracts an optical hybrid descriptor and a SAR hybrid descriptor, calculates the approximation of the hybrid descriptor, screens the feature point set, and finally obtains matching feature point pairs through RANSAC outlier filtering to complete optical and SAR registration. The present invention can realize automatic registration of optical and SAR images.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image intelligent processing, and in particular refers to a method for realizing automatic registration of optical and SAR heterogeneous images. Background Art

[0002] With the rapid development of remote sensing technologies such as constellations, it is possible to obtain multiple remote sensing data from the same area, which makes it possible to improve target detection, object segmentation and other applications by fusing multi-source remote sensing data. The premise for realizing the automated intelligent application of multi-source remote sensing data is to realize automated multi-source remote sensing data registration.

[0003] Existing registration methods, such as SIFT feature point matching, have good application effects on the registration of similar images. The principle of such methods is to extract corner points as feature points from the two images to be registered based on pixel features, and then calculate the same-name points in the two images through feature descriptor matching, and further calculate the transformation matrix to complete the registration. However, the meanings of pixels in heterogeneous images, especially optical and SAR images, are completely different. For example, the pixel values ​​of optical images refer to the spectral reflectance characteristics of their respective objects, while the pixel values ​​of SAR images represent the scattering characteristics of the objects, which leads to the pixel values ​​of the two images representing completely different meanings. Therefore, existing methods cannot obtain same-name points by calculating corner points and matching them to achieve registration between optical and SAR images. Summary of the invention

[0004] In order to solve the problem that optical and SAR images are difficult to automatically align due to differences in image styles caused by different imaging mechanisms, the present invention proposes an automatic registration method for optical and SAR images based on bidirectional style transfer, which can homogeneously transform heterogeneous optical and SAR images, extract hybrid features in the optical domain and SAR domain respectively, establish feature matching relationships between images, and realize automatic registration of optical and SAR images.

[0005] The object of the present invention is achieved in that:

[0006] An automatic registration method for optical and SAR images based on bidirectional style transfer includes the following steps:

[0007] (1) Perform BM3D denoising on the SAR image to obtain the SAR image to be registered; then perform homogenization processing on the optical image to be registered and the SAR image to be registered; use the CycleGAN network to transfer the style of the optical image to be registered and the SAR image to be registered into a pseudo-optical image and a pseudo-SAR image, respectively, to obtain an optical image homogeneous pair consisting of an optical image and a pseudo-optical image, and a SAR image homogeneous pair consisting of a SAR image and a pseudo-SAR image;

[0008] (2) Perform SIFT feature point extraction on homogeneous pairs of optical images and SAR images respectively, and obtain the feature point set of the optical image to be registered by taking the union of the feature points extracted from the optical image and the feature points extracted from the pseudo-SAR image; obtain the feature point set of the SAR image to be registered by taking the union of the feature points extracted from the SAR image and the feature points extracted from the pseudo-optical image;

[0009] (3) For the feature point set of the optical image to be registered, SIFT descriptors are extracted from the optical image and the pseudo-SAR image respectively, and the SIFT descriptors extracted from the optical image and the SIFT descriptors extracted from the pseudo-SAR image are concatenated end to end to obtain an optical hybrid descriptor; for the feature point set of the SAR image to be registered, SIFT descriptors are extracted from the pseudo-optical image and the SAR image respectively, and the SIFT descriptors extracted from the pseudo-optical image and the SIFT descriptors extracted from the SAR image are concatenated end to end to obtain a SAR hybrid descriptor;

[0010] (4) Using the optical hybrid descriptor of each feature point in the feature point set of the optical image to be registered and the SAR hybrid descriptor of each feature point in the feature point set of the SAR image to be registered, the approximation is calculated one by one, and the feature points in the feature point set of the optical image to be registered that do not have an approximate relationship with the feature point set of the SAR image to be registered are eliminated to obtain the feature point set of the optical image to be registered after screening; the feature points in the feature point set of the SAR image to be registered that do not have an approximate relationship with the feature point set of the optical image to be registered are eliminated to obtain the feature point set of the SAR image to be registered after screening;

[0011] (5) The RANSAC method is used to eliminate errors in the feature point set of the screened optical image to be registered and the feature point set of the screened SAR image to be registered, remove the external points that cannot be registered, and obtain the final registration result.

[0012] Compared with the background technology, the present invention has the following advantages:

[0013] 1. The method of the present invention converts the styles of optical and SAR images into pseudo-SAR and pseudo-optical images respectively, and performs joint registration in the optical field and the SAR field, thereby solving the problem of difficulty in registration of optical and SAR images due to differences in image styles caused by different imaging mechanisms.

[0014] 2. The present invention adopts a hybrid descriptor method of computational optical images and SAR images to describe feature points in the optical field and the SAR field respectively, and effectively screens the feature point set by calculating the approximation between feature points, thereby improving the registration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the present invention. Implementation

[0016] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0017] An automatic registration method for optical and SAR images based on bidirectional style transfer is firstly used to convert optical and SAR images into pseudo SAR and pseudo optical images through the CycleGAN style transfer network to achieve the homogenization of heterogeneous images. Then, SIFT is used to extract feature point sets from the optical and SAR images and the converted pseudo SAR and pseudo optical images respectively to obtain the feature point sets of the optical images to be registered and the feature point sets of the SAR images to be registered. Then, optical hybrid descriptors and SAR hybrid descriptors are extracted, the approximation of the hybrid descriptors is calculated, the feature point sets are screened, and finally, the matching feature point pairs are obtained through RANSAC outlier filtering to complete the registration of optical and SAR images.

[0018] like Figure 1 As shown, the method specifically comprises the following steps:

[0019] (1) First, the SAR image needs to be denoised by BM3D to obtain the SAR image to be registered. Then, the optical image to be registered and the SAR image to be registered are homogenized. The style of the optical image to be registered and the SAR image to be registered are transferred to pseudo-SAR image and pseudo-optical image respectively through the CycleGAN network, and the homogeneous pairs of optical images composed of optical and pseudo-optical images and the homogeneous pairs of SAR images composed of SAR and pseudo-SAR are obtained.

[0020] (2) Perform SIFT feature point extraction on homogeneous pairs of optical images and SAR images respectively. Take the union of the feature points extracted from the optical image and the feature points extracted from the pseudo-SAR image to obtain the feature point set of the optical image to be registered. Take the union of the feature points extracted from the SAR image and the feature points extracted from the pseudo-optical image to obtain the feature point set of the SAR image to be registered.

[0021] (3) Extract SIFT descriptors from the optical and pseudo-SAR images for the feature point set of the optical image to be registered, and concatenate the SIFT descriptors extracted from the optical image with the SIFT descriptors extracted from the pseudo-SAR image to obtain an optical hybrid descriptor. Extract SIFT descriptors from the pseudo-optical and SAR images for the feature point set of the SAR image to be registered, and concatenate the SIFT descriptors extracted from the pseudo-optical image with the SIFT descriptors extracted from the SAR image to obtain a SAR hybrid descriptor.

[0022] (4) The optical hybrid descriptor of each feature point in the feature point set of the optical image to be registered and the SAR hybrid descriptor of each feature point in the feature point set of the SAR image to be registered are used to calculate the approximation one by one. After calculation, it is found that there is an approximate relationship between some feature points in the feature point set of the optical image to be registered and some feature points in the feature point set of the SAR image to be registered, and there is no approximate relationship between some feature points. The feature points in the feature point set of the optical image to be registered that do not have an approximate relationship with the feature point set of the SAR image to be registered are eliminated to obtain the feature point set of the optical image to be registered after screening. Similarly, the feature point set of the SAR image to be registered after screening can be obtained.

[0023] (5) The RANSAC method is used to eliminate errors in the feature point set of the screened optical image to be registered and the feature point set of the screened SAR image to be registered, remove the external points that cannot be registered, and obtain the final registration result.

[0024] The present invention can realize automatic registration of optical and SAR heterogeneous images, and align the same objects and targets in optical and SAR images, thereby providing technical support for automatic image interpretation.

[0025] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for automatic registration of optical and SAR images based on bidirectional style transfer, characterized in that: The following steps are involved: (1) Perform BM3D denoising on the SAR image to obtain the SAR image to be registered; then perform homogenization processing on the optical image to be registered and the SAR image to be registered; use the CycleGAN network to transfer the style of the optical image to be registered and the SAR image to be registered into a pseudo-optical image and a pseudo-SAR image, respectively, to obtain an optical image homogeneous pair consisting of an optical image and a pseudo-optical image, and a SAR image homogeneous pair consisting of a SAR image and a pseudo-SAR image; (2) Perform SIFT feature point extraction on homogeneous pairs of optical images and SAR images respectively, and obtain the feature point set of the optical image to be registered by taking the union of the feature points extracted from the optical image and the feature points extracted from the pseudo-SAR image; obtain the feature point set of the SAR image to be registered by taking the union of the feature points extracted from the SAR image and the feature points extracted from the pseudo-optical image; (3) For the feature point set of the optical image to be registered, SIFT descriptors are extracted from the optical image and the pseudo-SAR image respectively, and the SIFT descriptors extracted from the optical image and the SIFT descriptors extracted from the pseudo-SAR image are concatenated end to end to obtain an optical hybrid descriptor; for the feature point set of the SAR image to be registered, SIFT descriptors are extracted from the pseudo-optical image and the SAR image respectively, and the SIFT descriptors extracted from the pseudo-optical image and the SIFT descriptors extracted from the SAR image are concatenated end to end to obtain a SAR hybrid descriptor; (4) Using the optical hybrid descriptor of each feature point in the feature point set of the optical image to be registered and the SAR hybrid descriptor of each feature point in the feature point set of the SAR image to be registered, the approximation is calculated one by one, and the feature points in the feature point set of the optical image to be registered that do not have an approximate relationship with the feature point set of the SAR image to be registered are eliminated to obtain the feature point set of the optical image to be registered after screening; the feature points in the feature point set of the SAR image to be registered that do not have an approximate relationship with the feature point set of the optical image to be registered are eliminated to obtain the feature point set of the SAR image to be registered after screening; (5) The RANSAC method is used to eliminate errors in the feature point set of the screened optical image to be registered and the feature point set of the screened SAR image to be registered, remove the external points that cannot be registered, and obtain the final registration result.

Images