An Emergency Water Body Extraction Method for Polarimetric SAR Time Series Images Based on Transfer Learning

Abstract

The present invention relates to a method for extracting emergency water bodies from polarimetric SAR time series images based on transfer learning, comprising the following steps: filtering the polarimetric SAR time series images; then extracting representative samples of the images; evaluating the transferability of the time series images to divide the source domain images and the target domain images; then selecting labeled samples in the source domain, high-information samples in the target domain, and unlabeled samples in the target domain; using transfer learning to label the unlabeled samples in the target domain and combining them with the high-information samples to obtain a labeled sample set; training a random forest classifier based on the labeled sample set to classify the images to obtain a water body distribution map, and finally combining the water body distribution maps of all the images to obtain a time series water body distribution map. The present invention can not only obtain the spatio-temporal distribution information of surface water bodies, but also significantly reduce the required human and time costs, and can be used for emergency response to flood disasters.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to an emergency water body extraction method for polarimetric SAR time series images based on transfer learning. Background Art

[0002] Water is an important resource for human survival and has an important impact on human survival and development. Water body extraction based on remote sensing technology can comprehensively monitor the distribution and changes of water bodies from macro to micro, and has played an important role in urban planning, industrial and agricultural production, water resource management and protection, and flood disaster prevention and control.

[0003] Polarimetric SAR time series images have the characteristics of wide coverage, large amount of information, no weather and time restrictions, etc., and can continuously and stably monitor surface water bodies, playing an irreplaceable role in disaster emergency response. However, to obtain the water body distribution map on each scene of the image, currently, the relatively accurate processing method requires first manually annotating a large number of training samples on each scene of the image, and then separately extracting the water body on each scene of the image, without sharing the knowledge between images to improve the efficiency of water body extraction. Therefore, the existing water body extraction process for polarimetric SAR time series images requires a large amount of human and time costs for training sample annotation, and it is difficult to meet the requirement of processing timeliness in flood disaster emergency response. Summary of the Invention

[0004] Transfer learning technology can transfer knowledge between different images, can effectively improve the reuse rate of existing data, save the resources and human costs required for constructing a discriminant model, and has good application prospects in the water body extraction of polarimetric SAR time series images. The present invention fully considers the characteristics of polarimetric SAR images. Based on transfer learning technology, for the water body extraction of polarimetric SAR time series images, by organically combining technologies such as polarimetric SAR image speckle filtering, Wishart distance measurement, active learning, transfer learning, and machine learning classification, the innovation of key technologies, processing processes, and application modes is studied, and the efficient water body extraction of polarimetric SAR time series images is realized by sharing the knowledge between images to meet the requirements of flood disaster emergency response.

[0005] The technical solution of the present invention is an emergency water body extraction method for polarimetric SAR time series images based on transfer learning, including the following steps:

[0006] Step 1, using the Mean-Shift filtering algorithm to filter each scene of the polarimetric SAR time series image to suppress the interference of speckle noise on water body extraction;

[0007] Step 2: Based on the H / A / Alpha-Wishart clustering algorithm and the strategy of maximizing the within-class Wishart distance, select a set of representative sample sets from each image scene.

[0008] Step 3: Conduct a temporal image transferability evaluation based on the representative sample sets. Automatically select one image scene from the polarimetric SAR time series images as the most suitable source domain image based on the evaluation results, and use the remaining images as the target domain images.

[0009] Step 4: Extract a sufficient number of samples from the representative sample set of the source domain image and perform manual annotation to obtain the labeled sample set of the source domain.

[0010] Step 5: For each set of representative sample sets of the target domain, based on the labeled sample set of the source domain and the active learning technique, extract a subset of samples with relatively high classification uncertainty and perform manual annotation to obtain the high-information sample set of the corresponding target domain.

[0011] Step 6: Randomly extract a sufficient number of samples from the representative sample set of each target domain image to obtain the unlabeled sample set of the target domain.

[0012] Step 7: For each unlabeled sample set of the target domain, based on the transfer learning technique, use the labeled sample set of the source domain and the corresponding high-information sample set of the target domain to perform transfer learning, assign class label information to the unlabeled sample set of the target domain, and then merge it with the high-information sample set to obtain the labeled sample set of the target domain.

[0013] Step 8: For each image, train a random forest classifier using all the labeled samples of this image, then use the trained random forest classifier to classify this image, distinguish two types of ground objects, water bodies and non-water bodies, according to the classification results to obtain the water body distribution map of this image. Finally, combine the water body distribution maps of all images to obtain the time series water body distribution map of the corresponding ground area of the images.

[0014] Moreover, the specific implementation of Step 2 is as follows: For each image scene, use the classical H / A / Alpha-Wishart clustering algorithm to perform clustering, divide the samples on this image into at most 16 clusters, and on this basis, use the strategy of maximizing the within-class Wishart distance to select a certain number of samples from each cluster to obtain a set of representative sample sets of this image.

[0015] Moreover, the specific implementation of the within-class Wishart distance maximization strategy in step 2 is as follows. For the samples within each cluster, assuming that N samples are to be extracted from each cluster, using the symmetric Wishart distance formula as the distance metric criterion, first extract 3 samples: the sample closest to the cluster center, the sample farthest from the cluster center, and the sample farthest from the first extracted sample. Then iteratively extract the remaining samples, each time extracting a sample with the largest sum of distances from the already extracted samples until the total number of extracted samples equals N.

[0016] Moreover, in step 2, the calculation formula of the symmetric Wishart distance is as follows:

[0017]

[0018] where A and B are the polarization covariance matrices of two samples (matrix dimension is q×q), and d(A, B) is the symmetric Wishart distance between these two samples.

[0019] Moreover, the specific implementation of step 3 is as follows. For the representative sample set of each scene of the image, calculate the maximum mean discrepancy (MMD) between every two sample sets, and then count the sum of the MMDs of each representative sample set with all other representative sample sets. Based on the statistical results, the image corresponding to the representative sample set with the smallest sum of MMD values is used as the source domain image, and the other images are used as the target domain images.

[0020] Moreover, the calculation formula of the maximum mean discrepancy in step 3 is:

[0021]

[0022] where X S and X T are two different sample sets, n S and n T are the number of samples in X S and X T respectively, and ψ(·) represents the kernel function. The larger the MMD, the greater the distribution difference between the two sets of data. When the distributions of the two sets of data are exactly the same, the value of MMD is equal to 0.

[0023] Moreover, the specific implementation of step 5 is as follows: First, a deep forest classifier is trained using the labeled sample set in the source domain. Then, the deep forest classifier is used to classify each group of representative sample sets in the target domain, and the classification uncertainty of each sample is calculated based on the classification results. Assuming that the number of high-information sample sets to be extracted in the target domain is N, then the N samples with the highest classification uncertainty in each group of representative sample sets in the target domain are extracted as the corresponding high-information sample sets in the target domain. Finally, the high-information sample sets in the target domain are manually labeled to assign them class labels.

[0024] Moreover, the specific implementation of calculating the classification uncertainty of each sample in step 5 is as follows: Suppose a sample has n possible classes (C1, C2,..., C n ), and the following formula is used to calculate its information entropy (Entropy):

[0025]

[0026] where P(C i ) is the probability that the sample is classified as class C i calculated by the deep forest classifier. The value range of the information entropy is [0, 1]. When the information entropy of a sample is higher, the sample is more difficult to be correctly classified by the classifier (the higher the classification uncertainty), indicating that the sample contains rich information. Adding it to the classifier training helps to improve the performance of the classifier.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows: The processing method of the present invention is clear and highly operable. It fully combines the characteristics of polarimetric SAR images following the complex Wishart distribution and the idea of transfer learning, designs a fully automatic representative sample selection method and a temporal image transferability evaluation method, which is highly targeted, helps to accurately transfer knowledge between time series images, can significantly reduce the number of training samples required for water body extraction from polarimetric SAR time series images, solves the problem that the prior art overly relies on training samples, can meet the needs of flood disaster emergency response, and has important application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is the algorithm flowchart of the embodiment of the present invention.

[0029] Figure 2 is the schematic diagram of the Mean-Shift filtering effect of the embodiment of the present invention, Figure 2 A is before filtering, Figure 2 B is after filtering.

[0030] Figure 3 is the schematic diagram of the evaluation result of the transferability of temporal images in the embodiment of the present invention.

[0031] Figure 4 This is the time-series water body distribution map of the embodiment of the present invention. Detailed implementation manners

[0032] A method for extracting emergency water bodies from polarimetric SAR time-series images based on transfer learning provided by the present invention is as follows: for the input geometrically registered polarimetric SAR time-series images, perform speckle filtering; based on the filtered images, extract respective representative sample sets from each image; conduct an evaluation of the transferability of the time-series images, and select source domain images and target domain images; then collect a source domain labeled sample set from the representative sample sets; based on the source domain labeled sample set and active learning techniques, extract a target domain high-information sample set; collect a target domain unlabeled sample set from the representative sample sets; then, based on the transfer learning method, use the source domain labeled sample set and the target domain high-information sample set to perform transfer learning on the target domain unlabeled sample set, assign each sample a class label, and obtain a target domain labeled sample set; then use the labeled sample sets of each image to train a random forest classifier respectively, perform binary classification on the images, and finally output the time-series water body distribution map.

[0033] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0034] As Figure 1 shown, the process of the embodiment of the present invention specifically includes the following steps:

[0035] Step 1, use the Mean-Shift filtering algorithm to filter each image in the polarimetric SAR time-series images respectively, suppress the speckle noise on the images, and improve the distinguishability of water bodies on the images. The effect of the filtering process is as Figure 2 shown.

[0036] Step 2, based on the H / A / Alpha-Wishart clustering algorithm and the strategy of maximizing the within-class Wishart distance, extract a set of representative sample sets from each image respectively.

[0037] The implementation method is as follows: First, perform H / A / Alpha polarization decomposition on each image respectively, divide all samples of the image into 16 categories according to the values of H, A, and Alpha obtained from the decomposition as the initial clustering result; then, based on the initial clustering result, use the Wishart clustering algorithm to perform precise clustering on all samples to obtain the precise clustering result; then, assuming that the number of clusters in the precise clustering result is N1 and the total number of samples in the representative sample set is set to N2, select N2 / N1 samples from each cluster based on the strategy of maximizing the within-class Wishart distance to form the representative sample set of the image.

[0038] The Wishart distance maximization strategy within a class is as follows: For each sample within a cluster, assuming that N samples are to be extracted from each cluster, using the symmetric Wishart distance formula as the distance metric, first extract 3 samples: the sample closest to the cluster center, the sample farthest from the cluster center, and the sample farthest from the first extracted sample; then iteratively extract the remaining samples, each time extracting the sample with the largest sum of distances from the already extracted samples until N samples are extracted. This strategy can ensure that the extracted samples are sufficiently representative of all samples within the cluster.

[0039] The symmetric Wishart distance formula is as follows:

[0040]

[0041] where A and B are the polarization covariance matrices of two samples (matrix dimension is q×q), and d(A, B) is the symmetric Wishart distance between these two samples.

[0042] Step 3, conduct the temporal image transferability evaluation based on the representative sample set. The evaluation results are as Figure 3 shown. Then, based on the evaluation results, automatically select one image from the polarimetric SAR time series images as the source domain image, and use the remaining images as the target domain images to ensure the highest overall transfer accuracy of the temporal images.

[0043] The implementation method is as follows: For all representative sample sets, first, based on the difference metric of Maximum Mean Discrepancy (MMD), calculate the MMD between every two representative sample sets pairwise; then, for each representative sample set, calculate the sum of the MMDs between it and other representative sample sets as the transferability of this sample set; next, based on the statistical results, use the image corresponding to the representative sample set with the smallest sum of MMD values as the source domain image, and this image is the image with the best global transferability; finally, use the images other than the source domain image as the target domain images.

[0044] The calculation formula for the maximum mean difference is as follows:

[0045]

[0046] where X S and X T are two different sample sets, n S and n T are the numbers of samples in X S and X TThe number of samples in it, ψ(·) represents the kernel function. The larger the MMD, the greater the distribution difference between the two groups of data. When the distributions of the two groups of data are exactly the same, the value of MMD is equal to 0.

[0047] Step 4: Uniformly sample a sufficient number of samples from the representative sample set of the source domain images and perform manual annotation to obtain the labeled sample set of the source domain.

[0048] Step 5: For each group of representative sample sets of the target domain, based on the labeled sample set of the source domain and active learning technology, extract a subset of samples with relatively high classification uncertainty and perform manual annotation to obtain the high-information sample set of the corresponding target domain.

[0049] The implementation method is as follows: First, train a deep forest classifier using the labeled sample set of the source domain. Then, use the deep forest classifier to classify each group of representative sample sets of the target domain, and calculate the classification uncertainty of each sample based on the classification results. Assume that the number of high-information sample sets of the target domain to be extracted is N. Then, extract the N samples with the highest classification uncertainty from each group of representative sample sets of the target domain as the corresponding high-information sample set of the target domain. Finally, perform manual annotation on the high-information sample set of the target domain and assign it a class label.

[0050] The specific method for calculating the classification uncertainty of each sample is as follows: Assume that a sample has n possible classes (C1, C2,..., C n ), and calculate its information entropy (Entropy) using the following formula:

[0051]

[0052] where P(C i ) is the probability that the sample is classified as class C i calculated by the deep forest classifier. The value range of the information entropy is [0, 1]. When the information entropy of a sample is higher, the sample is more difficult to be correctly classified by the classifier (the higher the classification uncertainty), indicating that the sample contains rich information. Adding it to the classifier training helps to improve the performance of the classifier.

[0053] Step 6: Uniformly extract a sufficient number of samples from the representative sample set of each scene of the target domain images as the unlabeled sample set of the target domain;

[0054] Step 7: For each unlabeled sample set of the target domain, based on transfer learning technology, use the labeled sample set of the source domain and the corresponding high-information sample set of the target domain to perform transfer learning, and assign class label information to the unlabeled sample set of the target domain to obtain the labeled sample set of the target domain.

[0055] The implementation method is as follows: The labeled sample set in the source domain, the high-information sample set in the target domain, and the unlabeled sample set in the target domain are simultaneously input into the transfer learning model. The transfer learning model is used to predict the unlabeled samples in the target domain, output their class labels, and then combine them with the high-information sample set in the target domain to obtain the labeled sample set in the target domain. Specifically, the transfer learning model can adopt the transfer Bagging algorithm, the integrated transfer learning algorithm based on Bagging, the semi-supervised transfer component analysis algorithm, or the semi-supervised maximum independence domain adaptation algorithm.

[0056] Step 8: For each scene of images (including source domain images and target domain images), use the labeled sample set of this image to train a random forest classifier, and then use the trained random forest classifier to classify this image. According to the classification results, distinguish two types of ground objects, water bodies and non-water bodies, to obtain the water body distribution map of this image. Combine the water body distribution maps of all images to obtain the time series water body distribution map of the ground area corresponding to the images, as Figure 4 shown.

[0057] The implementation method is as follows: For each scene of images, according to the class labels, divide the samples in the labeled sample set into two types of samples, water body class and non-water body class, and then use the K-fold cross-validation method to automatically select the best parameters of the random forest model; then train the random forest model based on the labeled sample set and the best parameters to obtain a water body extraction classifier; use the water body extraction classifier to perform binary classification on the image to distinguish the water body pixels and non-water body pixels on the image, and finally output the water body distribution map of this image.

[0058] Specifically in implementation, the above process can be realized by computer software technology to run a semi-automatic process.

[0059] The specific embodiments described in this article are only examples to illustrate the spirit of the present invention. Those skilled in the technical field to which the present invention belongs can make various modifications or supplements to the described specific embodiments or use similar methods to replace them, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for extracting emergency water bodies from polarimetric SAR time series images based on transfer learning, characterized in that, It includes the following steps: Step 1: Use the Mean-Shift filtering algorithm to filter each image in the polarimetric SAR time series images, suppressing the interference of speckle noise on water body extraction; Step 2: Based on the H / A / Alpha-Wishart clustering algorithm and the strategy of maximizing the within-class Wishart distance, select a set of representative sample sets from each image; Step 3: Conduct a temporal image transferability evaluation based on the representative sample sets. Based on the evaluation results, automatically select one image from the polarimetric SAR time series images as the most suitable source domain image, and regard the remaining images as target domain images; Step 4: Extract a sufficient number of samples from the representative sample set of the source domain image and perform manual annotation to obtain the labeled sample set of the source domain; Step 5: For each set of representative sample sets of the target domain, based on the labeled sample set of the source domain and active learning technology, extract a subset of samples with relatively high classification uncertainty and perform manual annotation to obtain the corresponding high-information sample set of the target domain; Step 6: Randomly extract a sufficient number of samples from the representative sample set of each target domain image as the unlabeled sample set of the target domain; Step 7: For each unlabeled sample set of the target domain, based on transfer learning technology, use the labeled sample set of the source domain and the corresponding high-information sample set of the target domain to perform transfer learning, assign class label information to the unlabeled sample set of the target domain, and then combine it with the high-information sample set to obtain the labeled sample set of the target domain; Step 8: For each image, train a random forest classifier using all the labeled samples of this image, and then use the trained random forest classifier to classify this image. According to the classification results, distinguish two types of ground objects, water bodies and non-water bodies, to obtain the water body distribution map of this image. Finally, combine the water body distribution maps of all images to obtain the time series water body distribution map of the corresponding ground area of the images.

2. The method for extracting emergency water bodies from polarimetric SAR time series images based on transfer learning according to claim 1, characterized in that: The specific implementation of Step 2 is as follows: First, perform H / A / Alpha polarization decomposition on each image respectively. According to the values of H, A, and Alpha obtained from the decomposition, divide all samples of this image into 16 categories as the initial clustering result; then, based on the initial clustering result, use the Wishart clustering algorithm to perform precise clustering on all samples to obtain the precise clustering result; next, assume that the number of clusters in the precise clustering result is N1 and the total number of samples in the representative sample set is set to N2. Then, based on the strategy of maximizing the within-class Wishart distance, select N2 / N1 samples from each cluster to form the representative sample set of this image.

3. The emergency water body extraction method for polarimetric SAR time series images based on transfer learning according to claim 2, wherein: The specific implementation of the within-class Wishart distance maximization strategy in step 2 is as follows. For each sample within a cluster, assuming that N samples are to be extracted from each cluster, using the symmetric Wishart distance formula as the distance metric, first extract 3 samples: one sample closest to the cluster center, one sample farthest from the cluster center, and one sample farthest from the first extracted sample. Then iteratively extract the remaining samples, each time extracting one sample with the largest sum of distances from the already extracted samples until N samples are extracted. This strategy can ensure that the extracted samples are sufficiently representative of all samples within the cluster.

4. The emergency water body extraction method for polarimetric SAR time series images based on transfer learning according to claim 3, wherein: The specific implementation of step 3 is as follows. For all representative sample sets, first calculate the Maximum Mean Discrepancy (MMD) between every two representative sample sets pairwise based on this difference metric. Then for each representative sample set, calculate the sum of the MMDs between it and other representative sample sets as the transferability of this sample set. Next, based on the statistical results, the image corresponding to the representative sample set with the smallest sum of MMD values is used as the source domain image, which is the image with the best global transferability. Finally, the images other than the source domain image are used as the target domain images.

5. The method for extracting emergency water bodies from polarimetric SAR time series images based on transfer learning according to claim 4, wherein: The specific implementation of step 5 is as follows. First, train a deep forest classifier using the source domain labeled sample set, and then use the deep forest classifier to classify each group of target domain representative sample sets respectively, and calculate the classification uncertainty of each sample based on the classification results. Assuming that the number of target domain high-information sample sets to be extracted is N, then extract the N samples with the highest classification uncertainty from each group of target domain representative sample sets as the corresponding target domain high-information sample sets. Finally, manually annotate the target domain high-information sample sets and assign them class labels.

Images