Sample expansion method and device for polarimetric SAR image change detection

Abstract

The invention discloses a sample expansion method and device for polarimetric SAR image change detection, and the method comprises the steps: obtaining a dual-time-phase original polarimetric SAR image set, carrying out the registration of the dual-time-phase original polarimetric SAR image set, and obtaining a dual-time-phase registration polarimetric SAR image set; based on the dual-time-phase registration polarimetric SAR image set, generating a dual-time-phase polarimetric total power diagram set, and performing logarithmic ratio operator operation on the dual-time-phase polarimetric total power diagram set to generate a difference diagram; generating a pseudo label set according to the difference chart, and dividing the dual-time-phase polarization total power chart set into three types of pseudo label samples according to the pseudo label set; and expanding the initial change class sample by using DDPM to obtain an expanded change class sample, so as to realize polarized SAR image change detection based on the expanded change class sample, the invariant class sample and the uncertain class sample, thereby improving the detection precision of the polarized SAR image change detection method.

Description

Technical Field

[0001] The present invention belongs to the field of SAR image change detection, and particularly relates to a method and device for sample augmentation for polarimetric SAR image change detection. Background Art

[0002] Synthetic Aperture Radar (SAR) has the characteristics of all-weather and all-day operation and can provide imaging at any time. As an extended system of SAR, Polarimetric Synthetic Aperture Radar (PolSAR) obtains four kinds of scattered echo signals through the ways of alternating transmission and simultaneous reception and vertical transmission and reception, so as to record the full polarization information of the target scattered echo. Therefore, compared with SAR, polarimetric SAR can obtain rich target information, avoid the uncertainty problem of target information, and enhance clutter suppression and anti-interference capabilities. Polarimetric SAR image change detection refers to the technology of locating the changed areas caused by human or natural factors in multi-temporal polarimetric SAR data, which plays a great role in monitoring deforestation, natural disaster assessment, land use, urban development, farmland detection and other fields. According to whether the manually marked change map is used in the detection process, the change detection process can be divided into two categories: supervised and unsupervised. The supervised model has achieved satisfactory results, but due to the influence of image noise and complex ground object types, manually annotating the changes on the ground during the change detection process is a time-consuming and laborious task. The existing unsupervised methods are widely used because they can automatically detect the changes occurring on the ground without relying on manually marked change areas.

[0003] There are many traditional polarimetric SAR image change detection processing methods. For example, the method based on scattering characteristics uses the geometric scattering characteristics of dual-temporal polarimetric SAR images to measure the difference between feature vectors, so as to complete the change detection task. This kind of method mainly uses polarization information for processing, and the principle is relatively simple; the polarimetric SAR change detection method based on statistical models is based on the fact that dual-temporal polarimetric SAR images follow a certain distribution, further estimates the parameters of the statistical distribution model, and conducts change detection by measuring the difference between statistical distributions; the method based on polarimetric distance refers to the physical quantity that measures the similarity between the scattering data of two multi-temporal polarimetric SAR images, and can be used to reflect the difference between the corresponding pixels of the two-temporal polarimetric SAR images.

[0004] Although the above traditional methods have achieved success in practical applications, due to the limitations of manually constructed features, the traditional methods cannot achieve the optimal detection effect. In recent years, with the rapid development of deep learning, methods for change detection of polarimetric SAR images using deep learning theory have been continuously proposed. Deep learning is an effective feature learning method that can automatically extract abstract features of complex ground objects without the need for manually designing complex feature extractors. Applying deep learning methods to change detection not only makes full use of the original data but also has good robustness to noise. Researchers at home and abroad have developed polarimetric SAR change detection methods using deep learning theory and achieved excellent results.

[0005] Currently, polarimetric SAR change detection methods based on deep learning are mainly divided into supervised methods and unsupervised methods. In 2020, Seyed Teymoor Seydi et al. proposed a fully supervised change detection framework based on a convolutional neural network (CNN). This network consists of three parallel channels: the first and second channels extract deep features from the original dual-temporal images respectively, and the third channel focuses on extracting change deep features based on the difference and superimposed deep features. At the same time, each channel uses three types of convolutional kernels, namely 1D, 2D, and 3D dilated convolutions, for deep feature extraction. Since there are a large number of labeled datasets available, supervised deep learning methods have achieved satisfactory results in many computer vision tasks. However, for the polarimetric SAR change detection task, it is unrealistic to spend a large amount of time and effort to construct a sufficient amount of training data that reflects the true change information of ground objects. Therefore, in many cases, it is more practical to perform the polarimetric SAR change detection task in an unsupervised manner. In 2019, Liu.G et al. proposed a stacked Fisher autoencoder (SFAE) for polarimetric SAR change detection. In the SFAE framework, unsupervised hierarchical feature learning and supervised fine-tuning are jointly performed during network training. In addition, by introducing the principle of linear discriminant analysis (LDA) to constrain the unsupervised autoencoder classifier, the SFAE framework can address the challenges of the "inter-class similarity and intra-class difference" problem, thus obtaining good change detection results. In 2023, D.Xu et al. proposed a differential-guided multi-scale graph convolutional network for unsupervised change detection of PolSAR images. First, it introduced the Shannon entropy difference image to obtain an enhanced difference map. Then, an undersampling method was used to balance the number of changed and unchanged samples by reducing the number of invariant samples without introducing redundancy for the clustered sample results. Finally, a differential-guided multi-scale GCN (DGMGCN) for polarimetric SAR image change detection was designed. This network uses difference information to eliminate the adverse effects of speckle noise on change detection and fully captures the change perception features of multi-temporal polarimetric SAR images at fine and coarse scales, thereby improving feature discriminability.

[0006] However, existing polarimetric SAR image change detection methods generally have problems of large computational complexity and sample imbalance in the training stage. Among them, the sample imbalance problem is particularly prominent, seriously restricting the improvement of detection accuracy. Summary of the Invention

[0007] To solve the above problems existing in the prior art, the present invention provides a sample augmentation method and device for polarimetric SAR image change detection.

[0008] The technical problems to be solved by the present invention are realized through the following technical solutions:

[0009] In a first aspect, the present invention provides a sample augmentation method for polarimetric SAR image change detection, and the sample augmentation method includes:

[0010] Obtain a set of dual-temporal original polarimetric SAR images, and register the set of dual-temporal original polarimetric SAR images to obtain a set of dual-temporal registered polarimetric SAR images; the set of dual-temporal original polarimetric SAR images includes two original polarimetric SAR images acquired at two different times in the same area;

[0011] Generate a set of dual-temporal polarimetric total power maps based on the set of dual-temporal registered polarimetric SAR images, and generate a difference map by performing a logarithmic ratio operator operation on the set of dual-temporal polarimetric total power maps;

[0012] Generate a pseudo-label set according to the difference map, and divide the set of dual-temporal polarimetric total power maps into three types of pseudo-labeled samples according to the pseudo-label set; the three types of pseudo-labeled samples include initial change class samples, unchanged class samples, and uncertain class samples;

[0013] Use DDPM to augment the initial change class samples to obtain augmented change class samples, so as to realize polarimetric SAR image change detection based on the augmented change class samples, the unchanged class samples, and the uncertain class samples.

[0014] Optionally, generating a set of dual-temporal polarimetric total power maps based on the set of dual-temporal registered polarimetric SAR images includes:

[0015] Obtain the covariance matrix of each pixel point in each registered polarimetric SAR image in the set of dual-temporal registered polarimetric SAR images;

[0016] Generate the polarimetric total power Span value corresponding to each pixel point according to the covariance matrix of each pixel point in each registered polarimetric SAR image;

[0017] Generate the polarization total power maps corresponding to the registered polarimetric SAR images based on the Span values of the polarization total power corresponding to each pixel in each registered polarimetric SAR image, and obtain a set of dual-temporal polarization total power maps.

[0018] Optionally, use DDPM to augment the initial change class samples to obtain augmented change class samples, including:

[0019] Use DDPM to gradually add Gaussian noise to the initial change class samples until the initial change class samples are transformed into pure noise;

[0020] Use the DDPM to gradually perform data recovery operations from the pure noise to generate new samples with the same distribution as the initial change class samples;

[0021] Combine the initial change class samples and the new samples to obtain augmented change class samples.

[0022] Optionally, generate a pseudo-label set according to the difference map, including:

[0023] Use the FCM clustering algorithm to process the difference map to generate a pseudo-label set.

[0024] Optionally, generate a difference map by performing a logarithmic ratio operator operation on the set of dual-temporal polarization total power maps, including:

[0025] I DI = |log(I″ 2 / I″ 1 )|;

[0026] where I DI represents the difference map; I″ 1 represents the polarization total power map of the first phase in the set of dual-temporal polarization total power maps; I″ 2 represents the polarization total power map of the second phase in the set of dual-temporal polarization total power maps.

[0027] Optionally, the number of the augmented transformation samples is equal to the number of the invariant class samples.

[0028] In a second aspect, the present invention provides a sample augmentation device for polarimetric SAR image change detection, and the sample augmentation device includes:

[0029] A registration module, configured to obtain a set of dual-temporal original polarimetric SAR images, and register the set of dual-temporal original polarimetric SAR images to obtain a set of dual-temporal registered polarimetric SAR images; the set of dual-temporal original polarimetric SAR images includes two original polarimetric SAR images acquired at two different times in the same area;

[0030] A generation module, configured to generate a set of dual-temporal polarization total power maps based on the set of dual-temporal registered polarimetric SAR images, and generate a difference map by performing a logarithmic ratio operator operation on the set of dual-temporal polarization total power maps;

[0031] A partitioning module, configured to generate a pseudo-label set according to the difference map, and partition the set of dual-temporal polarization total power maps into three types of pseudo-labeled samples according to the pseudo-label set; the three types of pseudo-labeled samples include initial change class samples, unchanged class samples, and uncertain class samples;

[0032] An augmentation module, configured to augment the initial change class samples by using DDPM to obtain augmented change class samples, so as to implement polarimetric SAR image change detection based on the augmented change class samples, the unchanged class samples, and the uncertain class samples.

[0033] In a third aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0034] The memory is used for storing a computer program;

[0035] The processor is configured to implement the method steps of any of the above sample augmentation methods for polarimetric SAR image change detection when executing the computer program stored on the memory.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method steps of any of the above sample augmentation methods for polarimetric SAR image change detection are implemented.

[0037] The sample augmentation method for polarimetric SAR image change detection provided by the present invention reduces the feature dimension of the set of dual-temporal original polarimetric SAR images by obtaining the set of dual-temporal polarization total power maps, reduces the feature redundancy problem existing in the original data, and greatly reduces the computational amount in the subsequent processing process; uses the DDPM model to augment the initial change class samples, solves the problems of sample training bias and low training efficiency caused by the imbalance between the number of change class samples and unchanged class samples in the existing samples for polarimetric SAR image change detection, provides high-quality and sufficient training samples for the deep neural network to perform better training by augmenting the initial change class samples, and improves the efficiency of polarimetric SAR image change detection.

[0038] The following will further describe the present invention in detail with reference to the accompanying drawings. Description of the Drawings

[0039] Figure 1It is a schematic flow chart of a sample augmentation method for polarimetric SAR image change detection provided by an embodiment of the present invention;

[0040] Figure 2 It is a schematic algorithm flow chart of DDPM provided by an embodiment of the present invention;

[0041] Figure 3 It is a schematic structural diagram of a sample augmentation device for polarimetric SAR image change detection provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0043] The following further describes the present invention in detail with specific embodiments, but the embodiments of the present invention are not limited thereto.

[0044] In order to solve the technical problems of large computational complexity and sample imbalance existing in the existing polarimetric SAR image change detection methods, an embodiment of the present invention provides a sample augmentation method for polarimetric SAR image change detection. Refer to Figure 1 , Figure 1 It is a schematic flow chart of a sample augmentation method for polarimetric SAR image change detection provided by an embodiment of the present invention, which specifically includes the following steps:

[0045] Step S101: Obtain a set of dual-temporal original polarimetric SAR images, and register the set of dual-temporal original polarimetric SAR images to obtain a set of dual-temporally registered polarimetric SAR images; the set of dual-temporal original polarimetric SAR images includes two original polarimetric SAR images acquired at the same location and at two different times.

[0046] In the embodiment of the present invention, the set of dual-temporal original polarimetric SAR images includes two original polarimetric SAR images acquired at the same location and at two different times. Specifically, the set of dual-temporal original polarimetric SAR images may include the original polarimetric SAR image of the first time phase and the original polarimetric SAR image of the second time phase in the same area. Among them, the original polarimetric SAR image of the first time phase reflects the ground object scattering characteristics of the area at the first time point. The original polarimetric SAR image of the second time phase reflects the ground object scattering characteristics of the area at the second time point.

[0047] In the embodiment of the present invention, registering the set of dual-temporal original polarimetric SAR images may be using the original polarimetric SAR image I 1 of the first time phase in the same area as the reference image, and the original polarimetric SAR image I 2Register with the reference image to obtain a set of dual-temporal registration polarimetric SAR images. This set of dual-temporal registration polarimetric SAR images includes the registered polarimetric SAR image I′ of the first temporal phase 1 and the registered polarimetric SAR image I' of the second temporal phase 2 , where, I 1 and I 2 are both of size H×L.

[0048] In the embodiment of the present invention, the original polarimetric SAR image I of the first temporal phase in the same area 1 is used as the reference image, and registering the original polarimetric SAR image I of the second temporal phase 2 with the reference image can analyze the changes in this area in the time dimension. By registering these two images, it is ensured that the spatial positions of the images in the two temporal phases are completely aligned, thus providing a high-quality data basis for subsequent change detection and analysis.

[0049] Step S102, generate a set of dual-temporal polarimetric total power maps based on the set of dual-temporal registration polarimetric SAR images, and generate a difference map by performing a logarithmic ratio operator operation on the set of dual-temporal polarimetric total power maps.

[0050] In the embodiment of the present invention, the set of dual-temporal polarimetric total power maps is generated by calculating the total scattering power of each pixel point in each image in the set of dual-temporal registration polarimetric SAR images.

[0051] In one implementation, generating a set of dual-temporal polarimetric total power maps based on the set of dual-temporal registration polarimetric SAR images includes:

[0052] Obtain the covariance matrix of each pixel point in each registered polarimetric SAR image in the set of dual-temporal registration polarimetric SAR images;

[0053] Generate the polarimetric total power Span value corresponding to each pixel point according to the covariance matrix of each pixel point in each registered polarimetric SAR image;

[0054] Generate the polarimetric total power map corresponding to each registered polarimetric SAR image based on the polarimetric total power Span values corresponding to the pixel points in each registered polarimetric SAR image, to obtain a set of dual-temporal polarimetric total power maps.

[0055] In the embodiment of the present invention, the method for obtaining the covariance matrix of each pixel point in each registered polarimetric SAR image is the same, that is, the covariance matrix C of each pixel point in I′ 1 and the covariance matrix C of each pixel point in I' 1 and the covariance matrix C of each pixel point in I' 2 and the covariance matrix C of each pixel point in I 2The calculation method is the same. Below, taking the covariance matrix C as an example, the calculation method of the covariance matrix of each pixel in each registered polarimetric SAR image in the dual-temporal registration polarimetric SAR image set will be further described:

[0056]

[0057] where l = 1, 2,..., L; L represents the nominal number of times for averaging, and this nominal number can be set by technicians according to experience and is not limited here; ψ represents the complex scattering vector in the single-look complex format of the registered polarimetric SAR image; (·) H represents the Hermitian transpose operator; in C, the diagonal elements C 11 、C 22 、C 33 are real numbers, and the upper triangular non-diagonal elements C 12 、C 13 、C 23 are complex numbers. It can be understood that when C = C 1 , ψ is the complex scattering vector represented in the single-look complex format of I′ 1 ; when C = C 2 , ψ is the complex scattering vector represented in the single-look complex format of I' 2 .

[0058] Calculate the polarization total power Span value Span(C) of the covariance matrix C:

[0059] Span(C)=C 11 +C 22 +C 33 ;

[0060] where, when Span(C)=Span(C 1 ), C 11 、C 22 、C 33 are the diagonal elements in C 1 ; when Span(C)=Span(C 2 ), C 11 、C 22 、C 33 are the diagonal elements in C 2 . Among them, the magnitude of the Span value reflects the width of the particle size distribution.

[0061] After calculating the polarization total power Span value corresponding to each pixel in each registered polarimetric SAR image, a dual-temporal polarization total power map set including the polarization total power maps corresponding to each registered polarimetric SAR image is obtained. The dual-temporal polarization total power map set includes the polarization total power map I″ 1and the total polarization power map I″ of the second temporal phase 2 。

[0062] In the embodiments of the present invention, by obtaining the total polarization power map, the feature dimension and feature redundancy of the input image are reduced, and the calculation amount is reduced.

[0063] In one implementation, by performing a logarithmic ratio operator operation on the dual-temporal total polarization power map set, a difference map is generated, including:

[0064] I DI =|log(I″ 2 / I″ 1 )|;

[0065] where, I DI represents the difference map; I″ 1 represents the total polarization power map of the first temporal phase in the dual-temporal total polarization power map set; I″ 2 represents the total polarization power map of the second temporal phase in the dual-temporal total polarization power map set.

[0066] Step S103, generating a pseudo-label set according to the difference map, and dividing the dual-temporal total polarization power map set into three types of pseudo-labeled samples according to the pseudo-label set; the three types of pseudo-labeled samples include initial change class samples, unchanged class samples, and uncertain class samples.

[0067] In the embodiments of the present invention, generating a pseudo-label set according to the difference map includes:

[0068] Using the FCM (Fuzzy-c means) clustering algorithm to process the difference map to generate a pseudo-label set. Among them, FCM clustering is a clustering analysis method based on fuzzy mathematics.

[0069] Using the FCM clustering algorithm to process the difference map DI can obtain a pixel-level pseudo-label set, and dividing the dual-temporal total polarization power map set into three types of pseudo-labeled samples Ω; among them, the three types of pseudo-labeled samples include initial change class samples Ω c1 , unchanged class samples Ω uch and uncertain class samples Ω uc . Among them, the change class samples represent the samples corresponding to the regions that have changed significantly in the polarization SAR images of the two temporal phases; the unchanged class samples represent the samples corresponding to the regions that have not changed in the polarization SAR images of the two temporal phases; the uncertain class samples represent the samples corresponding to the regions that are difficult to classify clearly in the change detection, usually noise interference or weak change regions.

[0070] Step S104: Use DDPM (Denoising Diffusion Probabilistic Models) to augment the initial change-class samples to obtain augmented change-class samples, so as to implement polarimetric SAR image change detection based on the augmented change-class samples, invariant-class samples, and uncertain-class samples.

[0071] In the embodiment of the present invention, the obtained initial change-class samples Ω c1 are fed into DDPM to obtain augmented change-class samples.

[0072] In one implementation, using DDPM to augment the initial change-class samples to obtain augmented change-class samples includes:

[0073] Use DDPM to gradually add Gaussian noise to the initial change-class samples until the initial change-class samples are transformed into pure noise;

[0074] Use DDPM to gradually perform data recovery operations from the pure noise to generate new samples with the same distribution as the initial change-class samples;

[0075] Combine the initial change-class samples and the new samples to obtain augmented change-class samples.

[0076] The process of DDPM augmenting the initial change-class samples is divided into two stages. See Figure 2 , Figure 2 which is the algorithm flow diagram of DDPM provided by the embodiment of the present invention. The specific augmentation steps will be described below:

[0077] 1) Forward process:

[0078] The forward process is also called the diffusion process, which is to gradually add Gaussian noise with a variance of a fixed value β 0 (t = 0, 1, 2... T) to the original image x t , that is, the initial change-class samples, to make it become x T , that is, pure noise, so as to achieve the purpose of destroying the image. Here, T represents the total number of diffusion steps; the distribution of the noise q(x t |x t-1 ) can be set by those skilled in the art according to experience:

[0079]

[0080] where I represents the identity matrix; N(·) represents the normal distribution; x t represents the change-class samples with Gaussian noise added at any time t; x t-1 represents the change-class samples with Gaussian noise added at time t - 1.

[0081] Let α t = 1 - β t ,

[0082]

[0083] where i = 1, 2....t; α at the i-th moment i = 1 - β i , and β i represents the fixed value at the i-th moment.

[0084] In the above formula, z and z t-1 are both noise data randomly sampled from a normal distribution;

[0085] Obtain the final distribution:

[0086]

[0087] It can be seen from the above formula that only by giving the initial change class sample x 0 , the change class sample x t added with Gaussian noise at any moment t and the final pure noise x T can be calculated.

[0088] 2) Backward process

[0089] The denoising process is the inverse process of the noise addition process and is a process of recovering the original data from Gaussian noise. It can be assumed that the noise for denoising is also taken from a Gaussian distribution. Since it is impossible to directly fit the distribution step by step, in the embodiment of the present invention, an estimated parameter distribution is constructed. The inverse diffusion process is still a Markov chain process and is a distribution that needs to be learned by a neural network:

[0090]

[0091] where q(x t-1 |x t ,x 0 ) represents the distribution of the noise in the denoising process; represents the mean of this distribution; represents the variance of this distribution;

[0092]

[0093]

[0094] where, represents the mean of this distribution at time t.

[0095] The final distribution can be obtained:

[0096]

[0097] The new sample Ω can be obtained from the above formula c2 。

[0098] Merge the initial changed-class samples Ω c1 and the new sample Ω c2 to obtain the final augmented changed-class sample Ω c =Ω c1 +Ω c2 。

[0099] In one implementation, the number of augmented transformation samples is equal to the number of unchanged-class samples.

[0100] Since the changed regions in the actual ground object scene are more difficult to detect than the unchanged regions, and the changed regions are also much smaller in the actual ground object scene, the number of labeled samples in the changed regions is far less than the number of unchanged samples, which will lead to problems such as sample training bias, low training efficiency, distorted evaluation metrics, and data drift. Therefore, by augmenting the initial changed samples to make the number of augmented transformation samples equal to the number of unchanged-class samples, the problem of sample imbalance caused by the number of transformation samples being far less than the number of unchanged samples is solved.

[0101] In the embodiments of the present invention, the polarization SAR image change detection can be realized by combining the augmented changed-class samples, unchanged-class samples, and uncertain-class samples. The specific process of using the augmented total samples for polarization SAR image change detection is as follows:

[0102] Send the obtained augmented changed-class sample Ω c , unchanged-class sample Ω uch into the deep learning network for training until the network converges. At this time, the network has clearly grasped the characteristics of the changed and unchanged regions, and a classifier for change perception feature extraction can be obtained. Then, the uncertain-class sample Ω uc is sent into the classifier as a prediction sample. The classifier can accurately distinguish the changed regions and unchanged regions in the uncertain samples, and the final change detection result map can be obtained to complete the change detection of the polarization SAR image.

[0103] In the embodiment of the present invention, by obtaining the dual-temporal polarization total power map set, the feature dimension of the dual-temporal original polarization SAR image set is reduced, the feature redundancy problem existing in the original data is reduced, and the computational complexity in the subsequent processing process is greatly reduced; the DDPM model is used to expand the initial change class samples, which solves the problems such as sample training bias and low training efficiency caused by the imbalance between the number of change class samples and non-change class samples in the existing samples for polarimetric SAR image change detection. By expanding the initial change class samples, high-quality and sufficient training samples are provided for better training of the deep neural network, and the polarimetric SAR image change detection efficiency is improved.

[0104] In addition, the embodiment of the present invention can continue to be used in the future to solve the sample imbalance problem of SAR and polarimetric SAR image change detection, and can also be used in other image research directions such as optics and medicine, and can also be applied to many image sample expansions.

[0105] Based on the same inventive concept, the embodiment of the present invention also provides a sample expansion device for polarimetric SAR image change detection. Refer to Figure 3 , Figure 3 which is a schematic structural diagram of a sample expansion device for polarimetric SAR image change detection provided by the embodiment of the present invention. The sample expansion device includes:

[0106] A registration module 301, configured to obtain a dual-temporal original polarization SAR image set, and register the dual-temporal original polarization SAR image set to obtain a dual-temporal registered polarization SAR image set; the dual-temporal original polarization SAR image set includes two original polarization SAR images acquired at the same location and at two different temporal phases;

[0107] A generation module 302, configured to generate a dual-temporal polarization total power map set based on the dual-temporal registered polarization SAR image set, and generate a difference map by performing a logarithmic ratio operator operation on the dual-temporal polarization total power map set;

[0108] A division module 303, configured to generate a pseudo-label set according to the difference map, and divide the dual-temporal polarization total power map set into three types of pseudo-labeled samples according to the pseudo-label set; the three types of pseudo-labeled samples include initial change class samples, non-change class samples, and uncertain class samples;

[0109] An expansion module 304, configured to expand the initial change class samples by using DDPM to obtain expanded change class samples, so as to implement polarimetric SAR image change detection based on the expanded change class samples, the non-change class samples, and the uncertain class samples.

[0110] In the embodiment of the present invention, by obtaining the set of dual-temporal polarization total power maps, the feature dimension of the set of dual-temporal original polarization SAR images is reduced, the feature redundancy problem existing in the original data is reduced, and the computational complexity of the subsequent processing is greatly reduced; using DDPM to augment the initial change class samples solves the problems of sample training bias and low training efficiency caused by the imbalance between the number of change class samples and non-change class samples in the existing samples for polarimetric SAR image change detection. By augmenting the initial change class samples, high-quality and sufficient training samples are provided for better training of the deep neural network, improving the efficiency of polarimetric SAR image change detection.

[0111] Optionally, the generating module generates a set of dual-temporal polarization total power maps based on the set of dual-temporal registered polarimetric SAR images, including:

[0112] Obtain the covariance matrix of each pixel in each registered polarimetric SAR image in the set of dual-temporal registered polarimetric SAR images;

[0113] Generate the polarization total power Span value corresponding to each pixel according to the covariance matrix of each pixel in each registered polarimetric SAR image;

[0114] Generate the polarization total power map corresponding to each registered polarimetric SAR image based on the polarization total power Span value corresponding to each pixel in each registered polarimetric SAR image, and obtain the set of dual-temporal polarization total power maps.

[0115] Optionally, the augmenting module uses DDPM to augment the initial change class samples to obtain augmented change class samples, including:

[0116] Use DDPM to gradually add Gaussian noise to the initial change class samples until the initial change class samples are transformed into pure noise;

[0117] Use the DDPM to gradually perform data recovery operations from the pure noise to generate new samples with the same distribution as the initial change class samples;

[0118] Merge the initial change class samples and the new samples to obtain augmented change class samples.

[0119] Optionally, the partitioning module generates a pseudo-label set according to the difference map, including:

[0120] Use the FCM clustering algorithm to process the difference map to generate a pseudo-label set.

[0121] Optionally, the generating module generates a difference map by performing a logarithmic ratio operator operation on the set of dual-temporal polarization total power maps, including:

[0122]

[0123] Among them, I DI represents the difference map; I″ 1 represents the polarization total power map of the first phase in the set of dual-temporal polarization total power maps; I″ 2 represents the polarization total power map of the second phase in the set of dual-temporal polarization total power maps.

[0124] Optionally, the number of the augmented transformation samples is equal to the number of the invariant class samples.

[0125] An embodiment of the present invention further provides an electronic device, as Figure 4 shown, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404. Among them, the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,

[0126] The memory 403 is used to store a computer program;

[0127] When the processor 401 is used to execute the program stored on the memory 403, it implements the method steps of any one of the above sample augmentation methods for polarimetric SAR image change detection.

[0128] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0129] The communication interface is used for communication between the above electronic device and other devices.

[0130] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0131] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0132] The present invention also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the method steps described in any of the above sample augmentation methods for polarimetric SAR image change detection are implemented.

[0133] Optionally, the computer-readable storage medium may be a non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk memory.

[0134] Optionally, the above computer-readable storage medium may also be at least one storage device located away from the aforementioned processor.

[0135] In another embodiment of the present invention, a computer program product containing instructions is also provided. When it runs on a computer, the computer is caused to execute the method steps described in any of the above sample augmentation methods for polarimetric SAR image change detection.

[0136] It should be noted that terms such as "first", "second", etc. are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention.

[0137] In the description of this specification, the descriptions with reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0138] Although the present invention has been described in connection with various embodiments herein, however, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the accompanying drawings and the disclosure. In the description of the present invention, the term "including" does not exclude other components or steps, the term "a" or "one" does not exclude a plurality of cases, and the meaning of "a plurality" is two or more, unless otherwise specifically defined. In addition, certain measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0139] The method provided by the embodiments of the present invention can be applied to an electronic device. Specifically, the electronic device can be: a desktop computer, a portable computer, a smart mobile terminal, a server, etc. There is no limitation here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0140] For the device / electronic device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0141] It should be noted that the device, electronic device, and storage medium of the embodiments of the present invention are respectively the device, electronic device, and storage medium applying the above-mentioned method for sample augmentation for polarimetric SAR image change detection. Then all embodiments of the above-mentioned method for sample augmentation for polarimetric SAR image change detection are applicable to the device, electronic device, and storage medium, and can achieve the same or similar beneficial effects.

[0142] The above content is a further detailed description of the present invention in combination with specific preferred implementation manners, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A sample expansion method for polarimetric SAR image change detection, characterized in that: The sample expansion method comprises: Acquire a dual-phase original polarization SAR image set, and register the dual-phase original polarization SAR image set to obtain a dual-phase registered polarization SAR image set; the dual-phase original polarization SAR image set includes two original polarization SAR images acquired in the same area and at two different phases; Generating a set of dual-phase polarization total power maps based on the dual-phase registered polarization SAR image set, and generating a difference map by performing a logarithmic ratio operator operation on the dual-phase polarization total power map set; Generate a pseudo label set according to the difference map, and divide the dual-phase polarization total power map set into three types of pseudo label samples according to the pseudo label set; the three types of pseudo label samples include initial change class samples, unchanged class samples and uncertain class samples; The initial change class samples are expanded by using DDPM to obtain expanded change class samples, so as to implement polarimetric SAR image change detection based on the expanded change class samples, the invariant class samples and the uncertain class samples.

2. The sample expansion method according to claim 1, characterized in that: Generating a set of dual-phase polarization total power images based on the dual-phase registered polarization SAR image set includes: Obtaining a covariance matrix of each pixel in each registered polarimetric SAR image in the dual-time-phase registered polarimetric SAR image set; Generate the polarization total power Span value corresponding to each pixel point according to the covariance matrix of each pixel point in each registered polarization SAR image; Based on the polarization total power Span value corresponding to each pixel point in each registered polarization SAR image, a polarization total power map corresponding to each registered polarization SAR image is generated to obtain a dual-phase polarization total power map set.

3. The sample expansion method according to claim 1, characterized in that: The initial change class samples are expanded by using DDPM to obtain expanded change class samples, including: Using DDPM to gradually add Gaussian noise to the initial change class samples until the initial change class samples are converted into pure noise; Using the DDPM to gradually perform data recovery operations from the pure noise to generate new samples that are consistent with the distribution of the initial change class samples; The initial change class sample and the new sample are combined to obtain an expanded change class sample.

4. The sample expansion method according to claim 1, characterized in that: Generating a pseudo label set according to the difference map includes: The difference map is processed using the FCM clustering algorithm to generate a pseudo label set.

5. The sample expansion method according to claim 1, characterized in that: A difference map is generated by performing a logarithmic ratio operator operation on the dual-phase polarization total power map set, including: I DI =|log(I″2 / I1”)|; Among them, I DI Represents the difference diagram; I1" represents the polarization total power diagram of the first phase in the dual-phase polarization total power diagram set; I"2 represents the polarization total power diagram of the second phase in the dual-phase polarization total power diagram set.

6. The sample expansion method according to claim 1, characterized in that: The number of the expanded transformation samples is equal to the number of the invariant class samples.

7. A sample expansion device for polarimetric SAR image change detection, characterized in that: The sample expansion device comprises: A registration module is used to obtain a dual-phase original polarization SAR image set, and register the dual-phase original polarization SAR image set to obtain a dual-phase registered polarization SAR image set; the dual-phase original polarization SAR image set includes two original polarization SAR images acquired in the same area and at two different phases; A generating module, configured to generate a set of dual-phase polarization total power images based on the dual-phase registered polarization SAR image set, and generate a difference image by performing a logarithmic ratio operator operation on the dual-phase polarization total power image set; A division module, used to generate a pseudo label set according to the difference map, and divide the dual-phase polarization total power map set into three types of pseudo label samples according to the pseudo label set; the three types of pseudo label samples include initial change class samples, unchanged class samples and uncertain class samples; The expansion module is used to expand the initial change class samples by using DDPM to obtain expanded change class samples, so as to realize polarimetric SAR image change detection based on the expanded change class samples, the invariant class samples and the uncertain class samples.

8. The sample expansion device according to claim 7, characterized in that: The generating module generates a set of dual-phase polarization total power images based on the dual-phase registered polarization SAR image set, including: Obtaining a covariance matrix of each pixel in each registered polarimetric SAR image in the dual-time-phase registered polarimetric SAR image set; Generate the polarization total power Span value corresponding to each pixel point according to the covariance matrix of each pixel point in each registered polarization SAR image; Based on the polarization total power Span value corresponding to each pixel point in each registered polarization SAR image, a polarization total power map corresponding to each registered polarization SAR image is generated to obtain a dual-phase polarization total power map set.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, used to implement the sample expansion method described in any one of claims 1 to 6 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sample expansion method according to any one of claims 1 to 6 is implemented.

Images