An SAR Image Change Detection Method Based on Unsupervised Spatial-Frequency Representation Learning Fusion

Abstract

The present invention relates to a SAR image change detection method based on unsupervised spatio-frequency feature learning fusion, including: generating a difference map from the pre-change SAR image and the post-change SAR image; performing hierarchical clustering on the difference map to generate determined data and data to be classified; extracting the spatial domain features, frequency domain feature information, and pixel block information of the first pixel block; splicing the extracted first spatial fusion feature, first frequency domain feature, and first pixel information feature to obtain a first spliced feature; classifying the first spliced feature to obtain the classification result of the data to be classified; fusing the classification result of the determinable data with the classification result of the data to be classified to obtain the final classification result. This method uses an unsupervised method to avoid the problem that SAR image change detection cannot be trained due to the lack of labels, and at the same time can effectively extract key information and well focus on the main information, thereby improving the accuracy of change detection and making the detection result more reasonable and credible.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a SAR image change detection method based on unsupervised spatio-frequency representation learning fusion. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active high-resolution microwave remote sensor and one of the effective means for earth observation from space. It has the characteristics of high resolution, effective identification of camouflage, and penetration of covering objects, and has now been widely used in geological exploration, urban planning, military detection and other fields. Polarimetric Synthetic Aperture Radar is a new imaging system developed with the development of Synthetic Aperture Radar technology. It obtains various polarization information of the target by acquiring the scattering characteristics of the same resolution cell on the ground under different polarization modes, and thus can obtain richer ground object information than traditional single-polarization SAR, which greatly enhances the radar's ability to obtain target information.

[0003] Change detection is the quantitative analysis of remote sensing data at different times and the determination of the characteristics and processes of surface changes. It is the change in the spectral response of image pixels in two periods caused by the changes in surface characteristics in the instantaneous field of view of remote sensing over time, and can be used to determine and evaluate various surface phenomena over time. The general process of traditional methods is to first obtain the difference image of two images of the same location at different times, and then process the difference image to divide the pixel points into two categories: changed and unchanged. The accuracy of the results of traditional change detection methods depends greatly on the difference map, but a lot of information is lost in the process of generating the difference map, resulting in unstable detection results.

[0004] Due to the inherent characteristics of the SAR sensor, SAR images are greatly affected by speckle noise, and the characteristics of speckle noise cause significant damage to the images. Therefore, due to the existence of speckle noise, it poses a great challenge to SAR image processing. Many researchers related to change detection are looking for a filter that can reduce the destructive influence of speckle noise. For unsupervised change detection in SAR images, since many detailed parts in the images are damaged by speckle noise, the main challenge is also to reduce the influence of speckle noise. To reduce the speckle noise in SAR images, many machine learning vision methods are designed specifically for different applications, such as wavelet transform, active contour model, and deep neural networks (including deep belief network and convolutional neural network). For SAR image change detection, the main solution of machine vision lies in despeckling in different steps. Unsupervised change detection usually follows image preprocessing, difference image generation, and difference image analysis, and these steps are also for conveniently comparing SAR images obtained at different times.

[0005] However, in unsupervised change detection, there are problems such as only using spatial domain information, not being able to focus well on the main information, and redundant feature information extraction; there are also some networks that need to be trained using labeled data. Since it is difficult to obtain labels for remote sensing image change detection and a large amount of manpower and material resources are required for annotation, there is a lack of labeled data and no large-scale data for training, resulting in the network not being robust. Summary of the Invention

[0006] To solve the above problems existing in the prior art, the present invention provides a SAR image change detection method based on unsupervised spatio-frequency representation learning fusion. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0007] An embodiment of the present invention provides a SAR image change detection method based on unsupervised spatio-frequency representation learning fusion, including the steps of:

[0008] Generate a difference map from the pre-change SAR image and the post-change SAR image;

[0009] Perform hierarchical clustering on the difference map to generate an image with pseudo-labels, where the image with pseudo-labels includes determinable data and data to be classified;

[0010] Taking the data to be classified as the central pixel, extract the first pixel blocks of the first target window size from the pre-change SAR image and the post-change SAR image;

[0011] Extract the spatial domain features of the first pixel blocks to obtain the first spatial fusion features, extract the frequency domain feature information of the first pixel blocks to obtain the first frequency domain features, and extract the pixel block information of the first pixel blocks to obtain the first pixel information features;

[0012] Stitch the first spatial fusion features, the first frequency domain features, and the first pixel information features to obtain the first stitched feature;

[0013] Use the trained classifier to classify the first stitched feature to obtain the classification result of the data to be classified, where the trained classifier is obtained by training the initial classifier using the determinable data;

[0014] Fuse the classification result of the determinable data with the classification result of the data to be classified to obtain the final classification result.

[0015] In an embodiment of the present invention, generating a difference map from the pre-change SAR image and the post-change SAR image includes:

[0016] Generate the difference map using the logarithmic ratio method.

[0017] In one embodiment of the present invention, hierarchical clustering is performed on the difference map to generate an image with pseudo-labels. The image with pseudo-labels includes determinable data and data to be classified, including:

[0018] The difference map is convolved with a Gabor kernel to obtain Gabor features;

[0019] The fuzzy C-means algorithm is used to perform a first classification on the Gabor features to obtain the determined pixel point categories and the pixel point categories with a relatively high probability of determination, and the values for determining the changed categories are defined using the pixels in the determined pixel point categories;

[0020] The fuzzy C-means algorithm is used to perform a second classification on the Gabor features to obtain a first cluster, a second cluster, a third cluster, a fourth cluster, and a fifth cluster arranged in descending order of average value;

[0021] The pixels in the first cluster are assigned to the first type of data, and the pixels in the second cluster, the third cluster, the fourth cluster, and the fifth cluster are respectively assigned to the second type of data and the third type of data according to the values for determining the changed categories, to obtain the image with pseudo-labels, where the first type of data and the second type of data are the determinable data, and the third type of data is the data to be classified.

[0022] In one embodiment of the present invention, taking the data to be classified as the central pixel, first pixel blocks of a first target window size are extracted from the SAR image before change and the SAR image after change, including:

[0023] The SAR image before change and the SAR image after change are superimposed to obtain a superimposed image;

[0024] Taking the pixel points of the data to be classified as the center, pixels of the first target window size are selected from the superimposed image to obtain the first pixel blocks.

[0025] In one embodiment of the present invention, extracting the spatial domain features of the pixel blocks to obtain spatial fusion features includes:

[0026] The pixel blocks are input into a first convolutional layer to obtain first global features;

[0027] The first global features are input into a second convolutional layer to guide the second convolutional layer to pay attention to the features of the central pixel of the first global features, to obtain second global features;

[0028] The first global features are horizontally divided into three equal parts, and the pixel blocks in the middle horizontal part are input into the second convolutional layer to obtain horizontal features;

[0029] Vertically divide the first global feature into three equal parts, and input the pixel blocks in the middle vertical part into the second convolutional layer to obtain a vertical feature;

[0030] Perform feature addition and fusion on the second global feature, the horizontal feature, and the vertical feature to obtain the spatial fusion feature.

[0031] In an embodiment of the present invention, extracting the frequency domain feature information of the pixel block to obtain a frequency domain feature includes:

[0032] Resize the pixel block and convert it into a frequency domain image to obtain a one-dimensional feature;

[0033] Input the one-dimensional feature into the first linear layer and the second linear layer respectively to obtain a first feature and a second feature;

[0034] Use an activation function to calculate the first feature to obtain a third feature;

[0035] Fuse the second feature and the third feature to obtain the frequency domain feature.

[0036] In an embodiment of the present invention, extracting the pixel block information of the first pixel block to obtain a first pixel information feature includes:

[0037] Use a multi-layer perceptron to extract the pixel block information of the first pixel block to obtain a first pixel information feature.

[0038] In an embodiment of the present invention, training the initial classifier using the determinable data includes:

[0039] Taking the determinable data as the central pixel, extract the second pixel block with the second target window size from the SAR image before change and the SAR image after change;

[0040] Extract the spatial domain feature of the second pixel block to obtain a second spatial fusion feature, extract the frequency domain feature information of the second pixel block to obtain a second frequency domain feature, and extract the pixel block information of the second pixel block to obtain a second pixel information feature;

[0041] Concatenate the second spatial fusion feature, the second frequency domain feature, and the second pixel information feature to obtain a second concatenated feature;

[0042] Input the second concatenated feature into the initial classifier to obtain an output result:

[0043] The network weights of the initial classifier are updated by calculating the loss value using the output result and the determinable data, and the minimization problem of the cost function of the initial classifier is solved using the gradient descent method to obtain the trained classifier parameters, thereby obtaining the trained classifier.

[0044] In one embodiment of the present invention, the output result is:

[0045]

[0046] where w, w2, …, w k ∈R n+1 are the parameters of the model, n is the feature dimension of the input classifier, is the normalization term for the probability distribution, and x (i) is a column vector in the feature matrix.

[0047] In one embodiment of the present invention, the cost function of the initial classifier is:

[0048]

[0049] where θ represents the classifier parameters, m represents the number of training samples, k represents the number of classes, 1{·} represents the indicator function, λ is the weight decay parameter, n is the dimension of the input features, and x (i) is a column vector in the feature matrix.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] 1. The image change detection method of the present invention generates an image with pseudo-labels including determinable data and data to be classified, and uses the determinable data to train the initial classifier, and uses the trained classifier to classify the data to be classified. The whole process uses an unsupervised method, does not rely on a large amount of training data and difficult-to-label data labels, avoids the problem that the SAR image change detection cannot be trained without labels, makes the network robust. At the same time, taking the pixels in the image with pseudo-labels as the central pixels, the features in the spatial domain and frequency domain are extracted, the connection between the spatial domain and frequency domain features is strengthened in a unified framework model, and the spatial domain features, frequency domain features and pixel information features are fused, which can effectively extract key information, well focus on the main information, thereby improving the accuracy of change detection and making the detection result more reasonable and credible.

[0052] 2. The image change detection method of the present invention uses the fuzzy C-means algorithm hierarchical clustering to generate pseudo-labels to calculate the loss for updating the model weights, which can effectively make the network better fit the existing data. Brief Description of the Drawings

[0053] Figure 1 Schematic flow chart of a SAR image change detection method based on unsupervised spatio - frequency representation learning fusion provided by an embodiment of the present invention;

[0054] Figure 2 Schematic flow chart of another SAR image change detection method based on unsupervised spatio - frequency representation learning fusion provided by an embodiment of the present invention;

[0055] Figure 3 Schematic diagram of hierarchical clustering of a difference map provided by an embodiment of the present invention;

[0056] Figure 4 Schematic diagram of spatial domain feature extraction provided by an embodiment of the present invention;

[0057] Figure 5 Schematic diagram of another spatial domain feature extraction provided by an embodiment of the present invention;

[0058] Figure 6 Schematic diagram of frequency domain feature extraction provided by an embodiment of the present invention;

[0059] Figure 7 Schematic diagram of training an initial classifier provided by an embodiment of the present invention. Detailed implementation manners

[0060] The following further describes the present invention in detail with specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0061] Embodiment 1

[0062] The SAR image change detection method based on unsupervised spatio - frequency representation learning fusion in this embodiment is a method for detecting changes in polarimetric SAR images based on time - domain - frequency - domain information and extracting central pixel information. It utilizes the idea of a deep network, obtains the spatial domain features of pixels by using a method of enhancing the central region in feature extraction, simultaneously obtains the frequency domain features of pixel blocks through an improved discrete cosine transform, and obtains the global features of pixel blocks through a multi - layer perceptron MLP, and fuses the three to ensure the rationality of the final classification result. Its detection idea is as follows: First, generate a difference map for the SAR images before and after the change of training data, perform FCM hierarchical clustering on the difference map to generate pseudo - labels, respectively extract the spatial domain features, frequency domain features, and pixel information features of pixels with a window size, fuse these three features and obtain features with a lower dimension through a fully - connected layer, and finally obtain the classification result through a classifier; finally, use the new classification result to correct the initial classification result to obtain the final classification result.

[0063] Please refer to Figure 1 and Figure 2 , Figure 1Schematic flow chart of a SAR image change detection method based on unsupervised spatio - frequency representation learning fusion provided by an embodiment of the present invention Figure 2 Schematic flow chart of another SAR image change detection method based on unsupervised spatio - frequency representation learning fusion provided by an embodiment of the present invention

[0064] The SAR image change detection method based on unsupervised spatio - frequency representation learning fusion includes the steps:[

[0065] S1. Generate a difference map from the pre - change SAR image and the post - change SAR image.[

[0066] Specifically, calculate the pixels in the pre - change SAR image and the post - change SAR image using the logarithmic ratio method to generate a difference map image ID and obtain the difference map.[

[0067] S2. Perform hierarchical clustering on the difference map to generate an image with pseudo - labels, where the image with pseudo - labels includes determinable data and data to be classified.[

[0068] Please refer to Figure 3 , Figure 3 Schematic diagram of hierarchical clustering of a difference map provided by an embodiment of the present invention. This step specifically includes:[

[0069] S21. Convolve the difference map with a Gabor kernel to obtain Gabor features.[

[0070] Specifically, by convolving the ID of the difference map with a set of Gabor kernels, obtain the Gabor wavelet representation of the difference map ID, thereby obtaining Gabor features.[

[0071] In this embodiment, calculating the Gabor wavelet representation of the difference map ID can extract the features of the image texture and reduce the interference caused by illumination and position to image recognition.[

[0072] S22. Use the fuzzy C - means algorithm to perform the first classification on the Gabor features, obtain the determined pixel point categories and the pixel point categories with a relatively high probability of determination, and define the numerical values of the determined changed categories using the pixels in the determined pixel point categories.[

[0073] Specifically, perform the first classification on the Gabor features using the fuzzy C - means (FCM) algorithm, and divide the difference map into two clusters:[ and wherein,[ represents the determined pixel point categories, including determined changed and determined unchanged,[ represents the pixel point categories with a relatively high probability of determination.[ The pixels in are represented by T1 It is represented that, thus, the numerical value for determining the category change is defined as:

[0074] TT = σ·T 1

[0075] where σ is a coefficient, which can be set to 1.20 in implementation.

[0076] S23. Use the fuzzy C - means algorithm to perform a second classification on the Gabor features, obtaining the first cluster, the second cluster, the third cluster, the fourth cluster, and the fifth cluster arranged in descending order of the average value.

[0077] Specifically, perform the FCM algorithm on the Gabor features for the second classification, dividing the difference map into 5 clusters: the first cluster the second cluster the third cluster the fourth cluster and the fifth cluster These 5 clusters are arranged in descending order of the average value. A higher average value in the clustering means a higher possibility of it becoming a changed category. Define the number of pixels in the 5 clusters as

[0078] S24. Assign the pixels in the first cluster to the first - type data, and respectively assign the pixels in the second cluster, the third cluster, the fourth cluster, and the fifth cluster to the second - type data and the third - type data according to the numerical value of the category change, obtaining the image with pseudo - labels, where the first - type data and the second - type data are the determinable data, and the third - type data is the data to be classified.

[0079] First, set the parameters t = 1 and c = T1 2 , and assign the pixels in

[0080] to the first - type data t 2 , := is defined as meaning; and judge the size of c and TT. If c < TT, assign the pixels in to the third - type data Otherwise, assign the pixels in and the second - type data to the second - type data. The third - type data

[0081] After classifying all five clusters, a pre-classified change map is obtained, which is represented by an image with a label to obtain the image with pseudo-labels.

[0082] S3. Using the data to be classified as the central pixel, extract the first pixel block of the first target window size from the SAR image before change and the SAR image after change.

[0083] Specifically, first, the SAR image before change and the SAR image after change are superimposed to obtain a superimposed image. Then, with the pixel point of the data to be classified as the center, pixels of the first target window size are selected from the superimposed image to obtain the first pixel block. In a specific embodiment, the size of the superimposed image is 200×300, and the pixel block of the first target window size is centered on the pixel point of the data to be classified, with a size of 7×7.

[0084] S4. Extract the spatial domain feature of the first pixel block to obtain the first spatial fusion feature, extract the frequency domain feature information of the first pixel block to obtain the first frequency domain feature, and extract the pixel block information of the first pixel block to obtain the first pixel information feature. In this embodiment, the extraction order of the first spatial fusion feature, the first frequency domain feature, and the first pixel information feature is not limited, and the three can be performed simultaneously or sequentially.

[0085] In a specific embodiment, step S4 specifically includes the steps:

[0086] S41. Extract the spatial domain feature of the first pixel block to obtain the first spatial fusion feature. Please refer to Figure 4 and Figure 5 , Figure 4 which is a schematic diagram of spatial domain feature extraction provided by an embodiment of the present invention, Figure 5 and this is another schematic diagram of spatial domain feature extraction provided by an embodiment of the present invention.

[0087] 1) Input the pixel block into the first convolutional layer to obtain the first global feature.

[0088] Specifically, the pixel block passes through a 1*1 convolutional layer to extract the features of the window pixels, obtaining the first global feature F g .

[0089] 2) Input the first global feature into the second convolutional layer to guide the second convolutional layer to pay attention to the features of the central pixel of the first global feature, obtaining the second global feature.

[0090] Specifically, the entire pixel block is used as a global feature through a 3×3 convolutional layer to guide the convolutional neural network (CNN) to pay attention to the features of its central pixel, and finally the second global feature F′ is obtained. g 。

[0091] 3) Horizontally divide the first global feature into three parts, and input the pixel block in the middle horizontal part into the second convolutional layer to obtain a horizontal feature.

[0092] Specifically, horizontally divide the first global feature into three parts, delete the pixels in the upper and lower parts, and take the pixel block in the middle horizontal part as a horizontal feature to obtain the horizontal feature F′ through a 3×3 convolutional layer. h 。

[0093] 4) Vertically divide the first global feature into three parts, and input the pixel block in the middle vertical part into the second convolutional layer to obtain a vertical feature.

[0094] Specifically, vertically divide the first global feature into three parts, delete the pixels on the left and right sides, and take the pixel block in the middle vertical part as a vertical feature to obtain the vertical feature F′ through a 3×3 convolutional layer. v 。

[0095] 5) Add and fuse the second global feature, the horizontal feature, and the vertical feature to obtain the spatial fusion feature.

[0096] Specifically, the second global features F′ at different positions obtained through the convolutional layer above g , the horizontal feature F′ h and the vertical feature F′ v are added and fused to obtain the feature of the final central pixel point, that is, the spatial fusion feature:

[0097] F fus = F′ g + F′ h + F′ v

[0098] where F fus represents the spatial fusion feature.

[0099] Through the above steps, the central region of the pixel block can be enhanced.

[0100] S42. Extract the frequency-domain feature information of the pixel block to obtain the frequency-domain feature. Please refer to Figure 6 , Figure 6 , which is a schematic diagram of frequency-domain feature extraction provided by an embodiment of the present invention.

[0101] 1) Resize the pixel block and convert it into a frequency-domain image to obtain a one-dimensional feature.

[0102] Specifically, through bilinear interpolation, the input image patch of size 2×r×r is adjusted to 2×8×8. Then, the image patch is transformed into the frequency domain through the discrete cosine transform (DCT) to obtain one-dimensional features.

[0103] 2) Input the one-dimensional features into the first linear layer and the second linear layer respectively to obtain the first feature and the second feature.

[0104] Specifically, the one-dimensional feature F1 obtained through the transformation is passed through two linear layers to obtain two features, namely the first feature F L and the second feature F L , and the first feature and the second feature are the same.

[0105] 3) Calculate the first feature using an activation function to obtain the third feature.

[0106] Specifically, the first feature F L after passing through the linear layer is passed through the Sigmoid activation function to obtain the third feature F' L . The Sigmoid activation function is:

[0107]

[0108] where y is the output feature and x is the input feature.

[0109] 4) Fuse the second feature and the third feature to obtain the frequency domain feature.

[0110] Specifically, fuse the second feature F L obtained through another linear layer and the third feature F L ' after passing through the activation function to obtain the frequency domain feature:

[0111] F dct = F' L + F L

[0112] where F dct is the frequency domain feature.

[0113] S43. Extract the pixel block information of the first pixel block to obtain the first pixel information feature.

[0114] Specifically, use a multi-layer perceptron to extract the pixel block information of the first pixel block to obtain the first pixel information feature F mlp .

[0115] S5. Concatenate the first spatial fusion feature, the first frequency domain feature, and the first pixel information feature to obtain the first concatenated feature.

[0116] Specifically, the first spatial fusion feature F fus extracted above, the first frequency domain feature F dct and the first pixel information feature F mlp are spliced and combined and passed through a fully connected layer to obtain a feature with a lower dimension, that is, the first spliced feature F out .

[0117] S6. Use the trained classifier to classify the first spliced feature to obtain the classification result of the data to be classified, where the trained classifier is obtained by training the initial classifier using the determinable data.

[0118] S61. Use the determinable data to train the initial classifier.

[0119] Specifically, it includes the steps of:

[0120] 1) Taking the determinable data as the central pixel, extract the second pixel blocks of the second target window size from the SAR image before change and the SAR image after change.

[0121] 2) Extract the spatial domain feature of the second pixel block to obtain the second spatial fusion feature, extract the frequency domain feature information of the second pixel block to obtain the second frequency domain feature, and extract the pixel block information of the second pixel block to obtain the second pixel information feature.

[0122] 3) Splice the second spatial fusion feature, the second frequency domain feature and the second pixel information feature to obtain the second spliced feature.

[0123] Specifically, the specific implementation steps of steps 1)-3) are the same as the specific implementation steps of S3-S5, except that when extracting the second pixel block, the pixel point of the determinable data is used as the pixel center.

[0124] 4) Input the second spliced feature into the initial classifier to obtain an output result.

[0125] Specifically, in this embodiment, the softmax layer is used to calculate the probability of change or non-change to generate the output. The second spliced feature is input into the initial softmax classifier, and the output of the classifier is calculated by the following formula:

[0126]

[0127] where w, w2,..., w k ∈R n+1 are the parameters of the model, n is the feature dimension of the input classifier, is the normalization term for the probability distribution, and x (i) is a column vector in the feature matrix.

[0128] 5) Update the network weights of the initial classifier by calculating the loss value using the output result and the determinable data, and solve the minimization problem of the cost function of the initial classifier using the gradient descent method to obtain the trained classifier parameters, thereby obtaining the trained classifier. Please refer to Figure 7 , Figure 7 FIG. is a schematic diagram for training an initial classifier provided by an embodiment of the present invention.

[0129] Specifically, first, initialize the Softmax classifier parameter θ as a very small random number close to 0. For example, θ is a random number less than or equal to 0.001.

[0130] Then, input the obtained output result into the softmax classifier to obtain a preliminary classification result. Calculate the gradients of each parameter using the binary cross-entropy loss function (BCE) for the preliminary classification result and the pseudo-labels obtained by pre-classification in step S2, and update the network weights according to the gradients.

[0131] Next, calculate the Softmax classifier cost function:

[0132]

[0133] where θ represents the classifier parameter, m represents the number of training samples, k represents the number of classes, 1{·} represents the indicator function, λ is the weight decay parameter, n is the dimension of the input features, and x (i) is a column vector in the feature matrix.

[0134] After that, use the gradient descent method to solve the minimization problem of the cost function J(θ) to obtain the trained Softmax classifier parameter θ.

[0135] Through the above update of the network weights and calculation of the classifier parameters, a trained classifier is obtained.

[0136] S62. Classify the first spliced feature using the trained classifier to obtain the classification result of the data to be classified.

[0137] Specifically, after training, use the trained Softmax classifier to classify the uncertain pixels (data to be classified) in the FCM hierarchical clustering. Input the first spliced feature F out into the trained classifier. The trained classifier finally outputs the results judged by the network pixel by pixel to obtain a binary image to judge the changed part, thereby obtaining the classification result of the data to be classified.

[0138] S7. Integrate the classification result of the determinable data with the classification result of the data to be classified to obtain the final classification result.

[0139] Specifically, after hierarchical clustering, the difference map is divided into three categories. One category is the data with determined changes, one category is the data with determined non - changes, and one category is the data to be classified. The data with determined changes and the data with determined non - changes are collectively referred to as determinable data, and their classification results are initial classification results and are determined. Therefore, integrate the classification result of the determinable data with the classification result of the data to be classified to obtain the final classification result.

[0140] The method for change detection of polarimetric SAR images based on time - domain and frequency - domain information and extraction of central pixel information in this embodiment. Its key steps are to extract the spatial - domain central features using features in different directions, extract the frequency - domain features through the improved discrete cosine transform, and extract the features of pixel blocks through a multi - layer perceptron, and finally fuse the three features to obtain the classification result of each pixel point, so as to determine whether it has changed. Due to the inherent characteristics of the SAR sensor, the SAR image is greatly affected by speckle noise, and the characteristics of speckle noise cause significant damage to the image. Therefore, due to the existence of speckle noise, it poses a great challenge to SAR image processing, thus limiting the accuracy of the change detection result. Therefore, this embodiment takes this factor into account, introduces a frequency - domain feature extraction part and a multi - layer perceptron for extracting overall features, improving the accuracy of change detection. In addition, use clustering to obtain pseudo - labels for training, using an unsupervised learning method, which can obtain a relatively high accuracy without relying on a large amount of data and labels. The image change detection method in this embodiment can detect the changed parts of polarimetric SAR data at different times, and both the accuracy and credibility of the change detection are relatively high.

[0141] In summary, the image change detection method of this embodiment has the following advantages: First, by generating an image with pseudo-labels including determinable data and data to be classified, and using the determinable data to train the initial classifier, and using the trained classifier to classify the data to be classified, the unsupervised method is used throughout the process, without relying on a large amount of training data and difficult-to-label data tags, avoiding the problem that the SAR image change detection cannot be trained without labels, making the network robust. Second, taking the pixels in the image with pseudo-labels as the central pixels, extracting the features in the spatial domain and frequency domain, the compressed representation in the frequency domain can suppress the noise in the spatial domain, enrich the model for image understanding, strengthen the connection between the spatial domain and frequency domain features in a unified framework model, and fuse the spatial domain features, frequency domain features and pixel information features. Extracting the spatial domain features emphasizes the central pixel features of the pixel blocks, can effectively extract key information, and well focus on the main information, thereby improving the accuracy of change detection and making the detection results more reasonable and credible. Third, using FCM hierarchical clustering to generate pseudo-labels to calculate the loss for updating the model weights can effectively make the network better fit the existing data. Fourth, considering the feature connection in the middle of the overall pixel blocks, a multi-layer perceptron is introduced to extract the features of the overall pixel blocks, and the connection with the features in the spatial domain and frequency domain is strengthened. Finally, the accuracy of change detection can be improved, and the detection results are more reasonable and credible.

[0142] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should all be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for SAR image change detection based on the fusion of unsupervised spatio-frequency representation learning, characterized in that, Including the steps: Generating a difference map from the pre-change SAR image and the post-change SAR image; Performing hierarchical clustering on the difference map to generate an image with pseudo-labels, where the image with pseudo-labels includes determinable data and data to be classified; Taking the data to be classified as the central pixel, and extracting first pixel blocks of a first target window size from the pre-change SAR image and the post-change SAR image; Extracting the spatial domain features of the first pixel blocks to obtain first spatial fusion features, extracting the frequency domain feature information of the first pixel blocks to obtain first frequency domain features, and extracting the pixel block information of the first pixel blocks to obtain first pixel information features; Concatenating the first spatial fusion features, the first frequency domain features, and the first pixel information features to obtain first concatenated features; Using a trained classifier to classify the first concatenated features to obtain a classification result of the data to be classified, where the trained classifier is obtained by training an initial classifier using the determinable data; Fusing the classification result of the determinable data and the classification result of the data to be classified to obtain a final classification result.

2. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 1, wherein Generating a difference map from the pre-change SAR image and the post-change SAR image, including: Generating the difference map using the logarithmic ratio method.

3. The SAR image change detection method based on unsupervised spatio-frequency feature learning fusion according to claim 1, characterized in that Performing hierarchical clustering on the difference map to generate an image with pseudo-labels, where the image with pseudo-labels includes determinable data and data to be classified, including: Convolving the difference map with a Gabor kernel to obtain Gabor features; Performing a first classification on the Gabor features using the fuzzy C-means algorithm to obtain determined pixel point categories and pixel point categories with a relatively high probability of being determined, and defining a value for determining the changed category using the pixels in the determined pixel point categories; Performing a second classification on the Gabor features using the fuzzy C-means algorithm to obtain a first cluster, a second cluster, a third cluster, a fourth cluster, and a fifth cluster arranged in descending order of average value; Assigning the pixels in the first cluster to the first type of data, and respectively assigning the pixels in the second cluster, the third cluster, the fourth cluster, and the fifth cluster to the second type of data and the third type of data according to the value for determining the changed category to obtain the image with pseudo-labels, where the first type of data and the second type of data are the determinable data, and the third type of data is the data to be classified.

4. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 1, characterized in that, Taking the data to be classified as the central pixel, and extracting first pixel blocks of a first target window size from the pre-change SAR image and the post-change SAR image, including: Overlaying the pre-change SAR image and the post-change SAR image to obtain an overlaid image; Taking the pixels of the first target window size centered on the pixel point of the data to be classified from the overlaid image to obtain the first pixel blocks.

5. The SAR image change detection method based on unsupervised spatio-frequency feature learning fusion according to claim 1, wherein Extracting the spatial domain features of the pixel blocks to obtain spatial fusion features, including: Inputting the pixel blocks into a first convolutional layer to obtain first global features; Input the first global feature into a second convolutional layer to guide the second convolutional layer to pay attention to the feature of the central pixel of the first global feature, and obtain a second global feature; Horizontally divide the first global feature into three equal parts, and input the pixel block in the middle horizontal part into the second convolutional layer to obtain a horizontal feature; Vertically divide the first global feature into three equal parts, and input the pixel block in the middle vertical part into the second convolutional layer to obtain a vertical feature; Perform feature addition and fusion on the second global feature, the horizontal feature, and the vertical feature to obtain the spatial fusion feature.

6. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 1, wherein, Extract the frequency-domain feature information of the pixel block to obtain a frequency-domain feature, including: Resize the pixel block and convert it into a frequency-domain image to obtain a one-dimensional feature; Input the one-dimensional feature into a first linear layer and a second linear layer respectively to obtain a first feature and a second feature; Use an activation function to calculate the first feature to obtain a third feature; Fuse the second feature and the third feature to obtain the frequency-domain feature.

7. The SAR image change detection method based on unsupervised spatio-frequency feature learning fusion according to claim 1, characterized in that Extract the pixel block information of the first pixel block to obtain a first pixel information feature, including: Use a multi-layer perceptron to extract the pixel block information of the first pixel block to obtain a first pixel information feature.

8. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 1, characterized in that, Use the determinable data to train an initial classifier, including: Taking the determinable data as the central pixel, extract a second pixel block of a second target window size from the pre-change SAR image and the post-change SAR image; Extract the spatial-domain feature of the second pixel block to obtain a second spatial fusion feature, extract the frequency-domain feature information of the second pixel block to obtain a second frequency-domain feature, and extract the pixel block information of the second pixel block to obtain a second pixel information feature; Concatenate the second spatial fusion feature, the second frequency-domain feature, and the second pixel information feature to obtain a second concatenated feature; Input the second concatenated feature into the initial classifier to obtain an output result: Use the output result and the determinable data to calculate a loss value to update the network weights of the initial classifier, and use the gradient descent method to solve the minimization problem of the cost function of the initial classifier to obtain the trained classifier parameters, thereby obtaining the trained classifier.

9. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 8, characterized in that, The output result is: where \(w, w_2, \ldots, w\) k \(\in \mathbb{R}\) n+1 are the parameters of the model, \(n\) is the dimensionality of the features input to the classifier, is the normalization term for the probability distribution, and \(x\) (i) is a column vector in the feature matrix.

10. The SAR image change detection method based on unsupervised spatio-frequency representation learning fusion according to claim 8, characterized in that The cost function of the initial classifier is: Among them, θ represents the classifier parameter, m represents the number of training samples, k represents the number of categories, 1{·} represents the indicator function, λ is the weight decay parameter, n is the dimension of the input features, and x (i) is a column vector in the feature matrix.

Images