CPD-based synthetic aperture radar image registration method and storage medium

Abstract

The invention discloses a CPD-based synthetic aperture radar image registration method and a storage medium, and belongs to the technical field of signal processing. The method comprises the following steps: constructing a segmentation data set, and training a segmentation network to extract point cloud features of an SAR image so as to overcome the influence of coherent noise and scattering difference of the SAR image on feature extraction; registering the feature point cloud by using a point cloud matching algorithm robust to an abnormal value, and estimating a registration relationship between SAR images; and performing spatial transformation and resampling on the SAR image according to the estimated registration relation to finish registration. Through the segmentation technology based on deep learning, the influence of coherent noise and scattering difference of the SAR image is overcome, fine image point cloud features can be extracted, and the precision of SAR image registration is effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of signal processing, and in particular relates to the registration of synthetic aperture radar (SAR) images, and specifically is an image registration method and a storage medium based on consistent point drift (CPD). Background Art

[0002] Synthetic aperture radar (SAR) is an active microwave remote sensing imaging system. Due to its unique advantages of all-day and all-weather detection, it is widely used in military and civilian fields. In various applications of SAR, image registration is a key technology, and the accuracy of image registration directly affects the effect of subsequent processing. SAR image registration is to convert images of the same scene detected at different times and angles into the same coordinate system to achieve accurate alignment of the images. For example, when monitoring changes in the surface environment, it is necessary to compare images of the same surface area detected at multiple time points to detect changes. However, since the flight trajectory and detection angle of each SAR detection may be different, there are angle or displacement differences between the images obtained. They need to be converted to the same coordinate system, that is, to perform accurate registration, so as to detect the differences at different times at the same location on the surface. Therefore, achieving accurate registration of SAR images is crucial to the effect of subsequent processing.

[0003] Traditional image registration methods can be roughly divided into two categories: intensity-based registration methods and feature-based registration methods. Feature-based registration methods have the advantages of strong adaptability and high registration accuracy, making them the most widely used methods in image registration. Among them, the scale-invariant feature transform (SIFT) descriptor is invariant to changes in scale, rotation, and illumination, and has become a representative algorithm for optical image registration. The rotation invariance of the SIFT algorithm is achieved by assigning a main direction to the key points. However, the coherent noise in SAR images greatly affects the calculation of the dominant direction, which will seriously affect the registration performance. In addition, the SIFT algorithm often has a high computational complexity when faced with a large number of feature points. Therefore, it is of great significance to study high-precision SAR image registration methods. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a synthetic aperture radar image registration method based on CPD, which overcomes the influence of coherent noise and scattering difference on image registration and can effectively improve the image registration accuracy.

[0005] The technical solution adopted by the present invention is:

[0006] A synthetic aperture radar image registration method based on CPD includes: firstly, constructing a segmentation data set and training a segmentation network to extract point cloud features of SAR images, thereby overcoming the influence of coherent noise and scattering difference of SAR images on feature extraction. In order to improve the registration accuracy, a point cloud matching algorithm that is robust to outliers is used to register the feature point cloud, and the registration relationship between SAR images is estimated. Finally, the SAR images are spatially transformed and resampled according to the estimated registration relationship to complete the registration.

[0007] Furthermore, the method of the present invention specifically comprises the following steps:

[0008] S1: Construct SAR image training set.

[0009] S2: Use the dataset generated in step S1 to train the U-Net network.

[0010] S3: Input the SAR image to be registered into the trained U-Net network to obtain the segmented mask.

[0011] S4: Extract the coordinate points of all non-zero value elements in the mask, generate a two-dimensional point cloud of image features, and randomly downsample it.

[0012] S5: Select one of all the SAR images to be registered as a reference image, i.e., the basis for registration, and refer to the corresponding two-dimensional point cloud generated in step S4 as the reference point cloud; the remaining images are called floating images, and the corresponding two-dimensional point clouds generated in step S4 are called floating point clouds. The floating point cloud and the reference point cloud are registered using the CPD method to estimate the registration relationship between the point clouds.

[0013] S6: According to the estimated registration relationship between the floating point cloud and the reference point cloud, the floating image is spatially transformed and resampled to complete the registration between the images.

[0014] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of a synthetic aperture radar image registration method based on CPD described in the present invention are implemented.

[0015] The beneficial effects of the present invention are:

[0016] (1) Through deep learning-based segmentation technology, the influence of coherent noise and scattering differences in SAR images can be overcome, and fine image point cloud features can be extracted, effectively improving the accuracy of SAR image registration;

[0017] (2) The method of extracting features is relatively easy;

[0018] (3) Feature point cloud registration method The CPD algorithm is robust to outliers and noise, which overcomes the influence of outliers in the point cloud and further improves the registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a flowchart of the method.

[0020] Figure 2 The SAR images to be registered provided by the embodiment of the present invention, wherein (a) is a reference image and (b) is a floating image.

[0021] Figure 3 The feature point cloud of the image to be registered provided by the embodiment of the present invention, wherein (a) is the reference point cloud and (b) is the floating point cloud.

[0022] Figure 4 This is a diagram of the registration results of the reference point cloud and the floating point cloud provided in an embodiment of the present invention.

[0023] Figure 5 A chessboard mosaic of the registration results of the reference image and the floating image provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0024] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0025] like Figure 1 The synthetic aperture radar image registration method based on CPD of the present invention comprises the following steps:

[0026] S1: Construct a SAR image segmentation training set. Manually mark the typical area edges in the SAR image, generate segmentation labels, and use the SAR image and its corresponding label to form a pair of training data. Here, it is necessary to ensure that the annotated areas of all SAR images are similar, which will serve as the features for subsequent registration steps. In order to reduce the workload of annotation, data augmentation is performed. The specific method is: randomly generate a set of rigid transformation relations {R, t}, expressed as:

[0027]

[0028] Where R represents the rotation matrix; t represents the translation vector; θ represents the rotation angle; t x is the translation in the x direction, t y is the translation in the y direction. The rigid transformation relationship {R, t} is applied to the SAR image and its corresponding label at the same time to generate a pair of training data.

[0029] By using the labeling method and data augmentation method, a total of 1,000 pairs of training data are generated to form the training set.

[0030] S2: Use the data set generated in step 1 to train the image segmentation network U-Net. Since the U-Net network structure is simple and can still maintain good performance when there are fewer training samples, the present invention selects U-Net as the segmentation network (reference: Ronneberger O, Fischer P, Brox TU-net: convolutional networks for biomedical image segmentation [C]. International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham, Switzerland, 2015: 234–241.).

[0031] During the training process, the Dice coefficient is used as the loss function L, which is expressed as:

[0032]

[0033] Among them, X represents the true label of the segmentation; Y represents the predicted value of the output. |X∩Y| represents the sum of the dot products of the label X and the predicted value Y; |X|+|Y| represents the direct summation of all elements of the label X and the predicted value Y.

[0034] S3: All SAR images to be registered are passed through the U-Net network trained in step S2 for segmentation to obtain corresponding segmentation masks.

[0035] S4: Extract the coordinates corresponding to all non-zero elements in the mask and generate the feature point cloud P corresponding to each SAR image, which can be expressed as

[0036] P={(x i ,y i )|i=1,2,...,I} (3)

[0037] Among them, I represents the number of points in the feature point cloud.

[0038] In order to reduce the amount of calculation for point cloud registration, the feature point cloud P is randomly downsampled so that the number of points I of each feature point cloud is controlled at about 2000.

[0039] S5: Select one of the images to be registered as the reference for registration, which is called the reference image. The remaining images are called floating images, that is, the remaining floating images will be registered to the reference image. The feature point cloud generated by the reference image in step S4 is called the reference point cloud, and the corresponding feature point cloud generated by the floating image in step S4 is called the floating point cloud. The floating point cloud and the reference point cloud are matched using the point cloud matching method CPD algorithm (reference: Myronenko A, Song X. Point set registration: coherent point drift [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (12): 2262-2275.) to estimate the registration relationship between the point clouds {R P ,t P}, where R P is the rotation matrix, t P is the translation vector.

[0040] S6: Based on the registration relationship between the reference point cloud and the floating point cloud estimated in step S5 {R P ,t P}, perform spatial transformation and resample the floating image corresponding to the floating point cloud to obtain the image after floating image registration, and complete the image registration.

[0041] like Figure 2 As shown in the figure, the SAR images to be registered have large coherent noise and there are scattering differences between the images.

[0042] like Figure 3 As shown in the figure, the extracted feature point cloud better reflects the edge of the image area to be registered, and the feature point cloud has fewer outliers, reducing the impact of image scattering differences. In addition, the floating point cloud and the reference point cloud have a high degree of similarity, overcoming the impact of coherent noise on feature extraction.

[0043] like Figure 4 As shown in the figure, the blue point cloud represents the reference point cloud, the red point cloud represents the floating point cloud, and the green point cloud represents the point cloud after the floating point cloud is matched. As can be seen from the figure, the green point cloud almost completely covers the blue point cloud, and the registration result is good.

[0044] Figure 5 A chessboard mosaic showing the registration result of the floating image and the reference image, where the brightness of the reference image is adjusted and the zoomed-in portion shows good agreement between the blocks.

[0045] It can be seen from the above results that the present invention can effectively overcome the influence of SAR image coherent noise and scattering differences between images, realize fine registration of SAR images, and has good registration performance.

[0046] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A synthetic aperture radar image registration method based on CPD, characterized in that: The method includes: Firstly, a segmentation dataset is constructed and a segmentation network is trained to extract the point cloud features of SAR images, which is used to overcome the influence of coherent noise and scattering differences of SAR images on feature extraction. Secondly, the feature point clouds are registered using a point cloud matching algorithm that is robust to outliers, and the registration relationship between SAR images is estimated; Finally, the SAR images are spatially transformed and resampled according to the estimated registration relationship to complete the registration.

2. The CPD-based synthetic aperture radar image registration method according to claim 1, characterized in that: The steps include: S1, construct SAR image training set; S2, training a U-Net network using the SAR image training set; S3, input the SAR image to be registered into the trained U-Net network to obtain the segmented mask; S4, extract the coordinate points of all non-zero value elements in the mask, generate a two-dimensional point cloud of image features, and randomly downsample it; S5, selecting one of all the SAR images to be registered as a reference image as a basis for registration, and referring to the corresponding two-dimensional point cloud generated in step S4 as the reference point cloud; referring to the remaining images as floating images, and referring to the corresponding two-dimensional point cloud generated in step S4 as the floating point cloud; registering the floating point cloud and the reference point cloud using the CPD method, and estimating the registration relationship between the point clouds; S6, according to the estimated registration relationship between the floating point cloud and the reference point cloud, the floating image is spatially transformed and resampled to complete the registration between the images.

3. The CPD-based synthetic aperture radar image registration method according to claim 2, characterized in that: In step S1, the method for constructing the SAR image training set is: manually marking typical area edges in the SAR image, generating segmentation labels, and forming a pair of training data with the SAR image and its corresponding label.

4. The CPD-based synthetic aperture radar image registration method according to claim 3, characterized in that: Also includes: Data augmentation is performed to reduce the workload of annotation. The specific method is: randomly generate a set of rigid transformation relations {R, t}, expressed as: t=(t x t y ) T Where: R represents the rotation matrix; t represents the translation vector; θ represents the rotation angle; t x is the translation in the x direction, t y is the translation in the y direction; Apply the rigid transformation relationship {R, t} to the SAR image and its corresponding label at the same time to generate a pair of training data; By using the labeling method and data augmentation method, several pairs of training data are generated to form a training set.

5. The CPD-based synthetic aperture radar image registration method according to claim 2, characterized in that: Step S2, during the training of the U-Net network, the Dice coefficient is used as the loss function L, which is expressed as: Where: X represents the true label of the segmentation; Y represents the predicted value of the output; |X∩Y| represents the sum of the dot products of the label X and the predicted value Y; |X|+|Y| represents the direct summation of all elements of the label X and the predicted value Y.

6. The CPD-based synthetic aperture radar image registration method according to claim 2, characterized in that: The two-dimensional point cloud representation of the image feature is: P={(x i ,y i )|i=1,2,...,I} Where: P represents the two-dimensional point cloud, and I represents the number of points in the feature point cloud.

7. The CPD-based synthetic aperture radar image registration method according to any one of claims 2 to 6, characterized in that: In step S5, the registration relationship between the point clouds is expressed as: {R P ,t P } Where: R P is the rotation matrix, t P is the translation vector.

8. The CPD-based synthetic aperture radar image registration method according to claim 7, characterized in that: In step S6, according to the registration relationship {R P ,t P }, perform spatial transformation and resample the floating image corresponding to the floating point cloud to obtain the image after floating image registration, and complete the image registration.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a synthetic aperture radar image registration method based on CPD as described in any one of claims 1 to 8 are implemented.

Images