Superpixel-based unsupervised multi-modal change detection method for automatic encoder AEKAN

Abstract

The invention discloses an unsupervised multi-modal change detection method of an AEKAN (automatic encoder Karan) based on superpixels, and the method comprises the steps: 1, carrying out the superpixel segmentation of an image after an original event, and obtaining a superpixel list under two modals according to a segmentation index; step 2, inputting heterogeneous superpixels in the superpixel lists in two different modes into an AEKAN to learn potential common characteristics of the heterogeneous superpixels, establishing a loss function in a common characteristic learning process, and training the AEKAN in an unsupervised learning mode; and step 3, obtaining the variation amplitude by calculating the distance between the common characteristics of the heterogeneous superpixels learned by the encoder, and segmenting a variation amplitude image into a binary variation image by adopting a threshold algorithm. The method is used for effectively extracting the change characteristics in the remote sensing image.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image surface change detection, and in particular relates to an unsupervised multimodal change detection method based on a superpixel-based automatic encoder AEKAN. Background Art

[0002] In order to dynamically monitor the geographical environment, guide urban development planning, and analyze urban expansion trends, it is of great practical significance to detect urban surface cover changes in real time and efficiently. In this context, using satellite images of multiple spatiotemporal scales for change detection can obtain richer spatial and temporal information. The rapid progress of modern satellite and aerospace technology has led to an increasing frequency of satellite observations, continuous improvement of image resolution, and significant enhancement of earth observation capabilities. Surface change detection based on high-resolution remote sensing images has become an effective means of current surface change detection due to its convenience, efficiency, and accuracy. This technology extracts change features by comparing and analyzing multi-temporal remote sensing images of the same area, and has important application value in natural disaster warning, farmland protection, dynamic monitoring of land use, land and resources protection, and social development analysis.

[0003] At present, facing the massive amount of remote sensing image data, relying on visual interpretation can no longer meet the real-time requirements of large-scale image classification, and manual interpretation is inefficient and easily limited by personal experience. The effectiveness of existing multimodal change detection methods is limited by factors such as data quality and model training methods. In particular, it is difficult for the network to focus on the changed area and fully learn the features, which affects the detection accuracy and robustness. Therefore, how to effectively extract the change features in remote sensing images is still a core problem that needs to be solved in remote sensing image change detection. Summary of the invention

[0004] The purpose of the present invention is to provide an unsupervised multimodal change detection method based on superpixel autoencoder AEKAN, which is used to effectively extract change features in remote sensing images.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: an unsupervised multimodal change detection method based on superpixel autoencoder AEKAN, which is specifically implemented according to the following steps:

[0006] Step 1: perform superpixel segmentation on the original post-event image, and obtain superpixel lists under two modalities based on the segmentation index;

[0007] Step 2: Input the heterogeneous superpixels in the superpixel lists under two different modalities into AEKAN to learn their potential common features. In the process of learning the common features, a loss function is established to train AEKAN in an unsupervised learning manner.

[0008] Step 3, the change amplitude is obtained by calculating the distance between the common features of the heterogeneous superpixels learned by the encoder, and a threshold algorithm is used to segment the change amplitude image into a binary change map.

[0009] As a preferred technical solution of the present invention, in step 1, different superpixels in the superpixel list are used as input units of AEKAN for training and testing.

[0010] As a preferred technical solution of the present invention, in step 3, AEKAN mainly consists of three parts: Siamese encoder ξ(·), decoder D X (·) and decoder D Y (·), and all three parts are composed of pure KAN neural network layers.

[0011] As a preferred technical solution of the present invention, in step 1, the original post-event image is segmented into superpixels using the SLIC algorithm in the skimage package to obtain a superpixel index Ω={Ω 1 ,Ω 2 ,...Ω T}, where T is the actual number of superpixels; then, according to the segmentation index Ω, we can get the superpixel list of X and Y as X Ω ={X i |1≤i≤T} and Y Ω = {Y i |1≤i≤T}; therefore, X Ω and Y Ω Used as input units of AEKAN for both training and testing.

[0012] As a preferred technical solution of the present invention, in step 2, AEKAN is constructed with pure KAN as the underlying network;

[0013] Among them, the Siamese encoder ξ(·) is used to transform the heterogeneous superpixels X i and Y i Mapped to the shared feature space, it is expressed as follows:

[0014]

[0015] in and Respectively represent X i and Y i potential common characteristics of

[0016] Then the common features of the encoder output and Input decoder D X (·) and decoder DY (·), used to reconstruct the original heterogeneous superpixel X i and Y i ,Right now:

[0017]

[0018] in and They refer to the pseudo-reconstructed X i and the pseudo-reconstructed Y i .

[0019] As a preferred technical solution of the present invention, in step 2, the loss function includes two sub-items: reconstruction loss and layer commonality loss

[0020] As a preferred technical solution of the present invention, in step 2:

[0021] Reconstruction losses Aims to convert the original superpixel X i and Y i Reconstructed into and And meet: and Therefore, the reconstruction loss Can be marked as:

[0022]

[0023] Among them, ∥·∥ MSE Refers to the mean square error (MSE), which can measure the similarity between the original superpixel and the pseudo-reconstructed superpixel; therefore, X i and Y i can be approximately reconstructed as and

[0024] Level commonality loss The purpose of the hierarchical commonality loss is to explicitly guide the SiameseKAN encoder in the proposed AEKAN to obtain the latent commonality features between HRSIs. Its specific form can be expressed as:

[0025]

[0026] Where e is the number of KAN layers in the Siamese KAN encoder, and Refers to the kth layer X i and Y icommon characteristics of and The MSE distance between them is used to enhance the effective common features between HRSIs; therefore, the proposed total loss can be expressed as:

[0027]

[0028] As a preferred technical solution of the present invention, in step 3, when training AEKAN, the trained Siamese encoder ξ(·) and decoder D can be obtained. X (·) and decoder D Y (·); Therefore, the trained AEKAN can be used to extract the common features between HRSIs. On this basis, the change amplitude map and the binary change map can be generated. Specifically:

[0029] In AEKAN, the common features of the last layer output of the Siamese KAN encoder are used and To measure the difference between each pair of heterogeneous superpixels in HRSIs, we can obtain the final change amplitude map, namely:

[0030]

[0031] in represents the difference between each superpixel pair within HRSIs, Represents the final DI of the proposed superpixel-based AEKAN method.

[0032] The beneficial effects of the present invention are:

[0033] (1) The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN of the present invention explores the unsupervised multimodal change detection method based on superpixel autoencoder Kolmogorov-Arnold network. To the best of our knowledge, this is the first time that a pure KAN model is proposed for multimodal remote sensing change detection (MCD). The effectiveness and capability of the pure KAN model for MCD have been verified, providing a reference for future research focusing on reliable and interpretable MCD methods. It can capture the potential common features between HRSIs. This study verifies and visualizes these common features, demonstrating their practicality in directly obtaining high-quality DI and solving the challenge of incomparable features between HRSIs in MCD. A hierarchical commonality loss is introduced in the Siamese KAN encoder in our proposed AEKAN, which can promote the extraction of common features between HRSIs by minimizing the difference in features extracted from HRSIs at each layer of the Siamese encoder. Comprehensive comparison and ablation experiments on five challenging MCD datasets confirm the effectiveness and superiority of the proposed AEKAN;

[0034] (2) The unsupervised multimodal change detection method based on the superpixel autoencoder AEKAN of the present invention explores the unsupervised multimodal change detection method based on the superpixel autoencoder Kolmogorov-Arnold network. By designing the process of the change detection method, including data preprocessing, AEKAN common feature representation learning, the encoder and decoder therein are used for feature extraction, image reconstruction and other functions, and finally achieve accurate detection of the change target. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0036] Figure 1 It is the execution principle diagram of the present invention;

[0037] Figure 2 It is a comparison diagram of the effects of Example 6. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0039] Example 1

[0040] like Figure 1 As shown, the unsupervised multimodal change detection method based on superpixel autoencoder AEKAN of the present invention is implemented in the following steps:

[0041] Follow the steps below to implement it:

[0042] Step 1: Perform superpixel segmentation on the original post-event image, and obtain superpixel lists under two modalities based on the segmentation index. Different superpixels in the superpixel lists are used as input units of AEKAN for training and testing.

[0043] Step 2: Input the heterogeneous superpixels in the superpixel lists under two different modalities into AEKAN to learn their potential common features. In the process of learning the common features, a loss function is established to train AEKAN in an unsupervised learning manner. Effective potential common feature learning is achieved by minimizing these losses.

[0044] In step 3, the change amplitude is obtained by calculating the distance between the common features of the heterogeneous superpixels learned by the encoder, and a threshold algorithm (Otsu) is used to segment the change amplitude image into a binary change map.

[0045] Example 2

[0046] like Figure 1 As shown, different from Example 1, in Example 2, an unsupervised multimodal change detection method based on superpixel autoencoder AEKAN of the present invention is used. In step 1, the original post-event image is segmented into superpixels by the SLIC algorithm in the skimage package to obtain a superpixel index Ω={Ω 1 ,Ω 2 ,...Ω T}, where T is the actual number of superpixels; then, according to the segmentation index Ω, we can get the superpixel list of X and Y as X Ω ={X i |1≤i≤T} and Y Ω = {Y i |1≤i≤T}; therefore, X Ω and Y Ω Used as input units of AEKAN for both training and testing.

[0047] Example 3

[0048] like Figure 1 As shown, unlike Example 2, in Example 3, an unsupervised multimodal change detection method based on superpixel autoencoder AEKAN of the present invention is constructed with pure KAN (Kolmogorov-Arnold Network) as the underlying network in step 2;

[0049] Among them, the Siamese encoder ξ(·) is used to transform the heterogeneous superpixels X i and Yi Mapped to the shared feature space, it is expressed as follows:

[0050]

[0051] in and Respectively represent X i and Y i The potential common features of HRSIs; therefore, the Siamese encoder ξ(·) aims to map HRSIs into the same feature space to achieve more effective common feature representation;

[0052] Then the common features of the encoder output and Input decoder D X (·) and decoder D Y (·), used to reconstruct the original heterogeneous superpixel X i and Y i ,Right now:

[0053]

[0054] in and They refer to the pseudo-reconstructed X i and the pseudo-reconstructed Y i ,Therefore, the AEKAN we designed is able to mine the potential common ,features among HRSIs in an unsupervised learning manner through the ,reconstruction task.

[0055] Example 4

[0056] like Figure 1 As shown, unlike Example 3, in Example 4, the unsupervised multimodal change detection method based on superpixel autoencoder AEKAN of the present invention has a loss function including two sub-items: reconstruction loss and layer commonality loss

[0057] In step 2:

[0058] Reconstruction losses Aims to convert the original superpixel X i and Y i Reconstructed into and And meet: and Therefore, the reconstruction loss Can be marked as:

[0059]

[0060] Among them, ∥·∥ MSE Refers to the mean square error (MSE), which can measure the similarity between the original superpixel and the pseudo-reconstructed superpixel; therefore, X i and Y i can be approximately reconstructed as and

[0061] Level commonality loss The purpose of the hierarchical commonality loss is to explicitly guide the SiameseKAN encoder in the proposed AEKAN to obtain the latent commonality features between HRSIs. Its specific form can be expressed as:

[0062]

[0063] Where l is the number of KAN layers in the Siamese KAN encoder, and Refers to the kth layer X i and Y i common characteristics of and The MSE distance between them is used to enhance the effective common features between HRSIs; therefore, the proposed total loss can be expressed as:

[0064]

[0065] Therefore, we can minimize the total loss by Extract and enhance common features among HRSIs in an unsupervised manner.

[0066] Example 5

[0067] like Figure 1 As shown, different from Example 4, in Example 5 of the present invention, an unsupervised multimodal change detection method based on superpixel autoencoder AEKAN is used. In step 3, when training AEKAN, the trained Siamese encoder ξ(·) and decoder D can be obtained. X (·) and decoder D Y (·); Therefore, the trained AEKAN can be used to extract the common features between HRSIs. On this basis, the change amplitude map and the binary change map can be generated. Specifically:

[0068] In AEKAN, the common features of the last layer output of the Siamese KAN encoder are used and To measure the difference between each pair of heterogeneous superpixels in HRSIs, we can obtain the final change amplitude map, namely:

[0069]

[0070] in represents the difference between each superpixel pair within HRSIs, Represents the final DI of the proposed superpixel-based AEKAN method.

[0071] Example 6

[0072] In order to explore the unsupervised multimodal change detection based on superpixel autoencoder Kolmogorov-Arnold network, the paper compares it with the advanced methods of similar tasks, and illustrates the effect of the change detection method of the present invention.

[0073] The experiment of this embodiment selected relatively advanced unsupervised multimodal change detection methods for comparison with the method of this article. IRG-McS, GIR-MRF, SCASC, AGSCC, IST-CRF, GBF-CD, GLSS, CANet, X-Net, ACE-Net, CACD, SR-GCAE, and BAACL were used respectively. These methods cover methods based on traditional image analysis and methods based on deep learning. The experiment chose to calculate the AUR and AUP corresponding to images with different change amplitudes for quantitative comparison. The overall classification accuracy (Overall Accuracy, OA), Kappa coefficient (Kppa Coefficient, KC), and F1-score (F1) were also selected to evaluate and compare the accuracy of the binary change map. In addition to the quantitative accuracy comparison, the visual effects of the binary change results based on different methods are also compared.

[0074] Among them, the corresponding AUR is the area under the ROC curve (Area Under ROC, AUR) and AUP is the area under the PR curve (Area Under PR, AUP), which are used to evaluate the performance of the classifier. The Kappa coefficient is a statistical indicator used to evaluate the performance of a classifier, which takes into account the difference between the correct classification of the classifier and the random classification. The value range of the Kappa coefficient is [-1,1], where 1 means complete consistency, 0 means equivalent to random classification, and -1 means complete inconsistency. F1-score is an indicator that comprehensively considers the accuracy and recall of the classifier. It measures the performance of the classifier by calculating the harmonic mean of the accuracy and recall. Overall Accuracy (OA) is one of the most commonly used evaluation indicators for classification problems. It represents the proportion of correct classifications of all samples by the classifier.

[0075] The experimental results are shown in Table 1. It can be seen from the experimental results that compared with other benchmark methods, the multimodal change detection method based on common feature representation learning of the present invention can obtain quite competitive AUP and AUR on these six multimodal datasets. Moreover, the average AUP and AUR of the method of the present invention on the five datasets achieved the best performance. Table 1 shows the comparison results of AUR and AUP of the change amplitude image of different methods on five surface change datasets

[0076] Table 1

[0077]

[0078] Table 2

[0079]

[0080]

[0081] The experimental results are shown in Table 2. It can be seen from the experimental results that compared with other benchmark methods, the AEKAN of the present invention: Exploring the superpixel-based automatic encoder Kolmogorov-Arnold network for unsupervised multimodal change detection shows superior performance on the binary change map based on Otsu on the change detection dataset. Almost all the accuracy evaluations on all datasets reached the highest (red represents the highest, blue represents the second highest), and more importantly, the average is much higher than the second highest. Table 2 shows the quantitative accuracy evaluation and comparison of different methods on the binary change map of 5 surface change datasets.

[0082] The experimental results are as follows Figure 2 As shown in the figure, through observation and visual results, it can be found that the superpixel-based automatic encoder Kolmogorov-Arnold network proposed in the present invention for unsupervised multimodal change detection can achieve better visual effects on five sets of multimodal change detection data sets. This is enough to show that the method proposed in the present invention can effectively improve the accuracy of detection.

[0083] The above description shows and describes several preferred embodiments of the invention, but as mentioned above, it should be understood that the invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention concept described herein through the above teachings or the technology or knowledge of the relevant field. Changes and modifications made by those skilled in the art should be within the scope of protection of the claims attached to the invention without departing from the spirit and scope of the invention.

Claims

1. Unsupervised multimodal change detection method based on superpixel autoencoder AEKAN, characterized in that: Follow the steps below to implement it: Step 1: perform superpixel segmentation on the original post-event image, and obtain superpixel lists under two modalities based on the segmentation index; Step 2: Input the heterogeneous superpixels in the superpixel lists of two different modalities into AEKAN to learn their potential common features. In the process of learning common features, a loss function is established to train AEKAN in an unsupervised learning manner; Step 3, the change amplitude is obtained by calculating the distance between the common features of the heterogeneous superpixels learned by the encoder, and a threshold algorithm is used to segment the change amplitude image into a binary change map.

2. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 1, characterized in that: In step 1, different superpixels in the superpixel list are taken as input units of AEKAN for training and testing.

3. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 2, characterized in that: In step 3, AEKAN mainly consists of three parts: Siamese encoder ξ(·), decoder D X (·) and decoder D Y (·), and all three parts are composed of pure KAN neural network layers.

4. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 3, characterized in that: In step 1, the original post-event image is segmented into superpixels using the SLIC algorithm in the skimage package to obtain superpixel index Ω = {Ω1, Ω2, ...Ω T }, where T is the actual number of superpixels; then, according to the segmentation index Ω, we can get the superpixel list of X and Y as X Ω ={X i |1≤i≤T} and Y Ω = {Y i |1≤i≤T}; therefore, X Ω and Y Ω Used as input units of AEKAN for both training and testing.

5. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 4, characterized in that: In step 2, AEKAN is constructed with pure KAN as the underlying network; Among them, the Siamese encoder ξ(·) is used to transform the heterogeneous superpixels X i and Y i Mapped to the shared feature space, it is expressed as follows: in and Respectively represent X i and Y i potential common characteristics of Then the common features of the encoder output and Input decoder D X (·) and decoder D Y (·), used to reconstruct the original heterogeneous superpixel X i and Y i ,Right now: in and They refer to the pseudo-reconstructed X i and the pseudo-reconstructed Y i .

6. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 5, characterized in that: In step 2, the loss function includes two sub-items: reconstruction loss and layer commonality loss 7. The unsupervised multimodal change detection method based on superpixel autoencoder AEKAN according to claim 6, characterized in that: In step 2: Reconstruction losses Aims to convert the original superpixel X i and Y i Reconstructed into and And meet: and Therefore, the reconstruction loss Can be marked as: Among them, ∥·∥ MSE refers to the mean square error, which measures the similarity between the original superpixel and the pseudo-reconstructed superpixel; therefore, X i and Y i can be approximately reconstructed as and Level commonality loss The purpose of the hierarchical commonality loss is to explicitly guide the Siamese KAN encoder in the proposed AEKAN to obtain the latent commonality features between HRSIs. Its specific form can be expressed as: Where l is the number of KAN layers in the Siamese KAN encoder, and Refers to the kth layer X i and Y i common characteristics of and The MSE distance between them is used to enhance the effective common features between HRSIs; therefore, the proposed total loss can be expressed as:

8. Unsupervised multimodal change detection method based on superpixel autoencoder AEKAN, characterized in that: In step 3, when training AEKAN, the trained Siamese encoder ξ(·), decoder D X (·) and decoder D Y (·); Therefore, the trained AEKAN can be used to extract the common features between HRSIs. On this basis, the change amplitude map and the binary change map can be generated. Specifically: In AEKAN, the common features of the last layer output of the Siamese KAN encoder are used and To measure the difference between each pair of heterogeneous superpixels in HRSIs, we can obtain the final change amplitude map, namely: in represents the difference between each superpixel pair within HRSIs, Represents the final DI of the proposed superpixel-based AEKAN method.

Images