Cross-sensor remote sensing image unsupervised change detection method based on structured graph auto-encoder

Abstract

The invention discloses a cross-sensor remote sensing image unsupervised change detection method based on a structured graph auto-encoder, belongs to the field of computer vision and remote sensing image processing, and particularly relates to a cross-sensor remote sensing image unsupervised change detection method. The objective of the invention is to solve the problem of low cross-sensor remote sensing image change detection accuracy caused by difficulty in explicit modeling of semantic change and insufficient robustness to modal difference in an existing model. The method comprises the steps of 1, obtaining a source domain and a target domain; 2, obtaining a source domain graph structure and a target domain graph structure; 3, constructing a learnable structural difference tensor; 4, constructing a structured graph auto-encoder 1; 5, constructing a structured graph auto-encoder 2; 6, obtaining a trained structured graph auto-encoder 1 and a corresponding weight, offset and structural difference tensor; 7, obtaining a trained structured graph auto-encoder 2 and a corresponding weight, offset and structural difference tensor; and 8, obtaining a final difference chart.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision and remote sensing image processing, and particularly relates to a cross-sensor remote sensing image unsupervised change detection method. BACKGROUND

[0002] With the continuous improvement of remote sensing image resolution and the development of multi-source sensor systems, cross-sensor remote sensing image change detection shows important value in urban expansion monitoring, post-disaster assessment, resource investigation and other scenarios. However, there are significant differences between different sensors (such as optical sensors and SAR sensors) in imaging mechanism, spatial resolution, spectral response and other aspects. This inter-modal inconsistency, referred to as domain difference, seriously affects the accurate extraction of the changed area between images. Under the unsupervised condition, especially without registration error compensation and manual annotation support, the change detection task faces greater challenges. Current researches mostly rely on implicit modeling through image space adaptive normalization, feature alignment or latent space mapping strategies to alleviate domain difference, which cannot effectively distinguish between real changes and pseudo changes caused by sensors at the semantic level, resulting in missed detection and false alarm in the detection results. Therefore, how to construct a detection mechanism that is robust to modal difference and can explicitly model semantic changes under the unsupervised framework is a key problem that needs to be broken through in the current cross-sensor remote sensing image change detection field. SUMMARY

[0003] The purpose of the present application is to solve the problem that the existing model is difficult to explicitly model semantic changes and is not robust to modal difference, resulting in low accuracy of cross-sensor remote sensing image change detection, and to propose a cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder.

[0004] The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder specifically includes the following steps:

[0005] Step one, use sensor A to take pictures of region C at time 1 to obtain the source domain;

[0006] Use sensor B to take pictures of region C at time 2 to obtain the target domain;

[0007] Step two, divide the source domain to obtain K superpixel regions, each superpixel region is taken as a node in the graph, and the nodes are connected to obtain the source domain graph structure denoted as Gs; ;

[0008] Divide the target domain to obtain K superpixel regions, each superpixel region is taken as a node in the graph, and the nodes are connected to obtain the target domain graph structure denoted as Gt; ;​​

[0009] Step 3: Construct a learnable structural difference tensor Learnable structural difference tensors Initialize to a zero matrix;

[0010] , For the set of real numbers, This represents the total number of superpixel regions. The number of features in each superpixel region;

[0011] Step 4: Construct the structured graph autoencoder 1; the specific process is as follows:

[0012] The structured graph autoencoder 1 includes encoder 1 and decoder 1 in sequence;

[0013] Encoder 1 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0014] Decoder 1 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0015] Step 5: Construct the structured graph autoencoder 2; the specific process is as follows:

[0016] The structured graph autoencoder 2 includes an encoder 2 and a decoder 2 in sequence;

[0017] Encoder 2 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0018] Decoder 2 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0019] Step Six: Based on Learnable Structural Difference Tensors and The structured graph autoencoder 1 is trained to obtain the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. ;

[0020] Step 7: Based on learnable structural difference tensors and The structured graph autoencoder 2 is trained to obtain the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. ;

[0021] Step 8: Based on the structural difference tensor corresponding to the trained structured graph autoencoder 1 The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map .

[0022] The beneficial effects of this invention are as follows:

[0023] The method of this invention constructs a bidirectional structured graph autoencoder framework, explicitly introducing a structural difference tensor to effectively model real semantic changes in dual-temporal cross-modal remote sensing images, thereby improving the accuracy and interpretability of change detection. By combining an asymmetric architecture design of a shared graph convolutional encoder and a domain-specific graph decoder, efficient feature alignment and bidirectional semantic reconstruction between images from different sensors are achieved. Integrating a multi-component loss function and applying it to this asymmetric architecture for collaborative optimization enhances the model's robustness to modal differences. As an unsupervised method, this invention does not rely on manually labeled training samples, reducing application costs and deployment complexity.

[0024] This invention comprises three main parts: graph structure construction and cross-modal feature alignment, explicit difference modeling and bidirectional semantic reconstruction, and difference graph optimization and mapping generation. The graph structure construction and cross-modal feature alignment part establishes a graph structure with spatial adjacency by performing superpixel segmentation on dual-temporal cross-sensor remote sensing images, and uses a shared graph convolutional encoder to extract cross-modal common latent representations, achieving structural alignment between images of different modalities. The explicit difference modeling and bidirectional semantic reconstruction part introduces two learnable structural difference tensors to achieve bidirectional semantic information conversion between the source and target domains, distinguishing between real changes and sensor differences. The difference graph optimization and mapping generation part jointly optimizes the network parameters and difference tensors using a multi-component loss function, and maps the learned difference information back to the original pixel space, generating unsupervised change detection results with higher accuracy and stronger robustness.

[0025] The unsupervised change detection model for cross-sensor remote sensing images based on a structured graph autoencoder designed in this invention can effectively detect changes in cross-sensor remote sensing images in application scenarios such as urban expansion monitoring and complex terrain. It improves the accuracy and stability of difference map generation, provides a guarantee for change detection and region identification processing, meets practical needs, and is easy to implement.

[0026] This invention constructs a bidirectional structured graph autoencoder and introduces an explicit learnable difference tensor to effectively model semantic changes while achieving cross-modal feature alignment, thus ensuring the accuracy of difference graph generation and its smooth application in fields such as environmental monitoring, terrain change assessment, and disaster monitoring.

[0027] This invention innovatively proposes a structured graph autoencoder for unsupervised difference map generation of cross-sensor remote sensing images. The structured graph autoencoder takes a pair of cross-sensor remote sensing images in two time phases as input and outputs a difference map. It includes a graph construction and feature alignment module and a difference modeling module, which provides a guarantee for high-precision image analysis such as change detection, anomaly detection, and target recognition, and has certain methodological advantages.

[0028] This invention utilizes deep learning and image processing technologies to better meet the needs of change detection and monitoring in different scenarios. The cross-sensor remote sensing image unsupervised difference map generation model of structured graph autoencoder maintains a certain degree of stability against modal differences, temporal variations and noise disturbances in the input image. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of the cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder described in this invention;

[0030] Figure 2 This is a schematic diagram of a bidirectional graph autoencoder structure;

[0031] Figure 3 Flowchart for modeling and fusing learnable structural difference tensors;

[0032] Figure 4 This is a schematic diagram of multi-component loss function optimization. Detailed Implementation

[0033] Specific Implementation Method 1: The specific process of the cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoders in this implementation method is as follows:

[0034] Step 1: Use sensor A to take pictures of region C (of the same size) at time 1 to obtain the source domain;

[0035] Sensor B is used to capture images of region C at time 2 to obtain the target domain;

[0036] The source domain image and the target domain image target the same shooting area, have the same shooting area size, but different shooting times and different sensors;

[0037] Step 2: Divide the source domain to obtain... There are 10 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the source domain graph structure is obtained based on these nodes and lines, denoted as […]. ;

[0038] Divide the target domain to obtain There are 3 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the target domain graph structure is obtained based on the nodes and lines, denoted as […]. ;

[0039] , Each is composed of a set of superpixel nodes and their spatial adjacency edges of the corresponding temporal images, and is used for graph convolution extraction and propagation of graph features in the source and target domains, respectively.

[0040] Step 3: Construct a learnable structural difference tensor Learnable structural difference tensors Initialize to a zero matrix;

[0041] , For the set of real numbers, This represents the total number of superpixel regions. The number of features in each superpixel region; for OK List;

[0042] Joint optimization with network parameters during training; Used to introduce difference information in the path from the source domain to the target domain. This is used to introduce difference information in the path from the target domain to the source domain, where each row corresponds to a superpixel node, progressively capturing and representing the true semantic changes between images.

[0043] Step 4: Construct the structured graph autoencoder 1; the specific process is as follows:

[0044] The structured graph autoencoder 1 includes encoder 1 and decoder 1 in sequence;

[0045] Encoder 1 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0046] Decoder 1 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0047] Step 5: Construct the structured graph autoencoder 2; the specific process is as follows:

[0048] The structured graph autoencoder 2 includes an encoder 2 and a decoder 2 in sequence;

[0049] Encoder 2 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0050] Decoder 2 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0051] Step Six: Based on Learnable Structural Difference Tensors and The structured graph autoencoder 1 is trained to obtain the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. ;

[0052] Step 7: Based on learnable structural difference tensors and The structured graph autoencoder 2 is trained to obtain the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. ;

[0053] Step 8: Based on the structural difference tensor corresponding to the trained structured graph autoencoder 1 The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map .

[0054] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that, in step two, the source domain is divided to obtain... There are 10 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the source domain graph structure is obtained based on these nodes and lines, denoted as […]. ;

[0055] Divide the target domain to obtain There are 3 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the target domain graph structure is obtained based on the nodes and lines, denoted as […]. ;

[0056] The specific process is as follows:

[0057] The source domain is partitioned using the SLIC algorithm to obtain... Each superpixel region is a node in the graph. Get the distance to each node The nearest feature (the feature refers to the location feature of the node itself) Nodes, select nodes Distance nodes respectively Recent Connect the nodes and obtain the source domain graph structure based on the nodes and lines, denoted as . ; ;

[0058] The target domain is partitioned using the SLIC algorithm to obtain... Each superpixel region is a node in the graph. Get the distance to each node The nearest feature (the feature refers to the location feature of the node itself) Nodes, select nodes Distance nodes respectively Recent Connect the nodes and obtain the target domain graph structure based on the nodes and lines, denoted as . . .

[0059] The other steps and parameters are the same as in Specific Implementation Method 1.

[0060] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that step six is ​​based on the learnable structural difference tensor. and The structured graph autoencoder 1 is trained to obtain the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. ;

[0061] The specific process is as follows:

[0062] Step 61: Transfer the learnable structural difference tensor and By adding elements one by one, we can obtain the features. ;

[0063] Step 62, Features Input structured graph autoencoder 1, output feature matrix The specific process is as follows:

[0064] feature Encoder 1 and decoder 1 are input sequentially, and decoder 1 outputs the feature matrix. ;

[0065] Step 63: Repeat steps 61 to 62 until the total loss function converges, obtaining the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. .

[0066] Other steps and parameters are the same as in specific implementation method one or two.

[0067] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the feature in step six-two is... Encoder 1 and decoder 1 are input sequentially, and decoder 1 outputs the feature matrix. The specific process is as follows:

[0068] Encoder 1 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0069] Decoder 1 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0070] feature The inputs are sequentially: first GCNConv, first ReLU activation function layer, first Dropout layer, second GCNConv, second ReLU activation function layer, second Dropout layer, third Dropout layer, third GCNConv, third ReLU activation function layer, fourth Dropout layer, fourth GCNConv, and fourth ReLU activation function layer. The fourth ReLU activation function layer outputs the feature matrix. .

[0071] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0072] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that step seven is based on the learnable structural difference tensor. and The structured graph autoencoder 2 is trained to obtain the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. ;

[0073] The specific process is as follows:

[0074] Step 71: Transfer the learnable structural difference tensor and By adding elements one by one, we can obtain the features. ;

[0075] Step 72, Characteristics Input: Structured Graph Autoencoder 2; Output: Feature Matrix The specific process is as follows:

[0076] feature The inputs are sequentially fed into encoder 2 and decoder 2, and decoder 2 outputs the feature matrix. ;

[0077] The weights of encoder 2 are the same as those of encoder 1;

[0078] Step 73: Repeat steps 71 to 72 until the total loss function converges, obtaining the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. .

[0079] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0080] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the feature in step seven-two is... The inputs are sequentially fed into encoder 2 and decoder 2, and decoder 2 outputs the feature matrix. The specific process is as follows:

[0081] Encoder 2 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer;

[0082] Decoder 2 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence.

[0083] feature The inputs are sequentially: first GCNConv, first ReLU activation function layer, first Dropout layer, second GCNConv, second ReLU activation function layer, second Dropout layer, third Dropout layer, third GCNConv, third ReLU activation function layer, fourth Dropout layer, fourth GCNConv, and fourth ReLU activation function layer. The fourth ReLU activation function layer outputs the feature matrix. .

[0084] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0085] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the total loss function is... ;

[0086]

[0087] in, Indicates losses from cross-domain reconstruction; This represents the structure regularization loss; This indicates sparse energy loss; Indicates the two-way alignment loss; Indicates spatial smoothing loss;

[0088] express The weights; express The weights; express The weights; express The weights; express The weights; ;

[0089] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0090] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the cross-domain reconstruction loss... The expression is:

[0091]

[0092] in,

[0093] Representing the source domain graph structure The corresponding source domain node feature matrix; Represents the target domain graph structure The corresponding target domain node feature matrix;

[0094] , ; For the set of real numbers, This represents the total number of superpixel regions. The number of features in each superpixel region;

[0095] Indicates passage Characteristics of reconstruction; Indicates passage Characteristics of reconstruction;

[0096] ;

[0097] ;

[0098] Encoder 1 represents the structured graph autoencoder 1; Decoder 1 represents the structured graph autoencoder 1;

[0099] This represents the learnable structural difference tensor. Source domain graph structure Corresponding source domain node feature matrix Perform element-by-element addition, and input the result. , Output result input , Output feature matrix;

[0100] Encoder 2 represents the structured graph autoencoder 2; Decoder 2 represents the structured graph autoencoder 2;

[0101] This represents the learnable structural difference tensor. and target domain graph The corresponding target domain node feature matrix Perform element-by-element addition, and input the result. , Output result input , Output feature matrix;

[0102] This represents the Frobenius norm square operation;

[0103] The structure regularization loss The expression is:

[0104]

[0105] in,

[0106] express The graph Laplace matrix, ;

[0107] express The graph Laplace matrix, ;

[0108] This represents the learnable structural difference tensor. Source domain graph structure Corresponding source domain node feature matrix Perform element-by-element addition, and then sum to... Multiply;

[0109] This represents the learnable structural difference tensor. and target domain graph The corresponding target domain node feature matrix Perform element-by-element addition, and then sum to... Multiply;

[0110] Represents the trace operation of a matrix; superscript This indicates the transpose;

[0111] The energy sparsity loss The expression is:

[0112]

[0113] The bidirectional alignment loss The expression is:

[0114]

[0115] in,

[0116] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0117] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0118] Indicates the total number of superpixel regions;

[0119] Denotes the Euclidean norm of a vector; express The square of;

[0120] Represents the natural exponential function;

[0121] The spatial smoothing loss The expression is:

[0122]

[0123] in,

[0124] Represents the set of adjacency edges defined in the difference tensor space;

[0125] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0126] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0127] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0128] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0129] Representing vectors Norm.

[0130] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0131] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that, in step eight, the structural difference tensor corresponding to the trained structured graph autoencoder 1 is used. The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map ;

[0132] The specific process is as follows:

[0133] Step 81: Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 1. norm square , For the reason The difference map obtained by mapping , ;

[0134] For difference diagram Height and width;

[0135] This represents the Frobenius norm square operation;

[0136] Step 82: Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 2. norm square , For the reason The difference map obtained by mapping , ;

[0137] For difference diagram Height and width;

[0138] Step 83: Based on the difference map And difference diagram To obtain the final difference map ; indicates as:

[0139]

[0140] in,

[0141] This represents the mean of all elements in the difference plot matrix;

[0142] For the final difference diagram Perform automatic thresholding to extract regions of variation.

[0143] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0144] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that the automatic threshold segmentation method is the Otsu method, K-means clustering method, iterative thresholding method, or a thresholding method based on machine learning.

[0145] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0146] This invention implements an unsupervised change detection method for cross-sensor remote sensing images based on a structured graph autoencoder, belonging to the field of intelligent remote sensing image processing. Addressing the significant domain differences between heterogeneous sensor images and the insufficient accuracy caused by implicit change modeling in existing methods, this invention proposes a bidirectional graph autoencoder framework to extract difference map information from dual-temporal cross-sensor images. The framework includes: constructing a bidirectional graph autoencoder framework using a shared graph convolutional encoder and a domain-specific graph decoder structure; learning structured representations of images from different temporal phases and modalities based on the graph structure constructed from superpixels, achieving effective alignment of cross-modal features and bidirectional semantic information conversion; introducing two learnable structural difference tensors to explicitly model the real semantic changes between images, effectively distinguishing between real changes and sensor differences; designing a multi-component loss function to jointly optimize network parameters and difference tensors, comprehensively considering factors such as reconstruction consistency, graph structure regularization, energy sparsity, cross-domain alignment, and spatial smoothness to improve the model's robustness to noise, illumination perturbations, and complex backgrounds. Finally, the difference tensors are mapped back to the original pixel space to generate difference maps. This invention realizes a method for generating difference maps of dual-temporal cross-sensor remote sensing images in unsupervised scenarios.

[0147] Example 1

[0148] Combination Figures 1 to 4 As shown, this invention constructs a bidirectional graph autoencoder framework, which adopts a shared graph convolutional encoder and a domain-specific graph decoder structure. Based on the graph structure constructed by superpixels, it learns the structured representations of images in different time phases and different modalities, thereby achieving effective alignment of cross-modal features and bidirectional semantic information conversion.

[0149] By introducing two learnable structural difference tensors, we can explicitly model the real semantic changes between images, effectively distinguishing between real changes and sensor differences.

[0150] A multi-component loss function is designed to jointly optimize network parameters and the difference tensor, comprehensively considering factors such as reconstruction consistency, graph structure regularization, energy sparsity, cross-domain alignment, and spatial smoothness to improve the model's robustness to noise, illumination perturbations, and complex backgrounds. Finally, the difference tensor is mapped back to the original pixel space to generate a difference map.

[0151] In this embodiment, the dual-temporal cross-sensor image can be a cross-sensor image after preprocessing operations such as registration and radiometric correction; the dual-temporal cross-sensor image is two cross-sensor images of the same region, the same size, and different temporal phases. After being input into the cross-sensor remote sensing image unsupervised change detection architecture based on structured graph autoencoder, the final difference map is obtained in the output layer.

[0152] Combination Figure 1As shown, the unsupervised change detection process for cross-sensor remote sensing images based on a structured graph autoencoder includes four stages: superpixel segmentation and graph structure construction, bidirectional graph autoencoder modeling, difference tensor generation and optimization, and difference map generation. First, the input dual-temporal cross-sensor image is divided into several spatially consistent regions using superpixel segmentation, and a graph structure is constructed based on this. In superpixel segmentation, the number of superpixels is 5000, the superpixel compactness factor is 1, and the superpixel color space is Lab color space. Then, common feature representations in the graph structure are extracted through a shared encoder, and feature transformations are performed through paths from the source domain to the target domain and from the target domain to the source domain, respectively, explicitly modeling the semantic differences between images. Next, a multi-component loss function is used to jointly optimize the network and the difference tensor to enhance the model's robustness. Finally, the optimized difference information is mapped back to the original pixel space to generate the final change detection difference map, and the method terminates.

[0153] Furthermore, combined with Figure 2 As shown, the structure of the graph is based on superpixel segmentation of the dual-temporal remote sensing images, and the SLIC algorithm is used to perform structure-aware segmentation of the images. Each superpixel region acts as a node in the graph, and edges are established through spatial adjacency relationships to form a graph structure representation of the source and target domain graphs. The source domain graph structure is denoted as […]. The target domain graph structure is denoted as Both consist of a set of superpixel nodes and their spatial adjacency edges from corresponding temporal images, and are used for graph convolution extraction and propagation of graph features in the source and target domains, respectively.

[0154] Furthermore, combining Figure 2 As shown, the shared graph convolutional encoder and Each layer consists of multiple stacked graph convolutional layers, used to extract common latent representations in the graph structures of the source and target domains; the domain-specific graph decoder... and These correspond to the decoding processes from the source domain to the target domain and from the target domain to the source domain, respectively. They are used to reconstruct shared latent features into graph node feature spaces of different modalities, supporting bidirectional cross-domain alignment and semantic transformation.

[0155] Furthermore, combining Figure 3 As shown, the learnable structural difference tensor Initialize it as a zero matrix and optimize it jointly with the network parameters during training; Used to introduce difference information in the path from the source domain to the target domain. This is used to introduce difference information in the path from the target domain to the source domain, where each row corresponds to a superpixel node, progressively capturing and representing the true semantic changes between images.

[0156] In the reconstruction process from the source domain image to the target domain image, a difference tensor is introduced. The reconstruction process is represented as follows:

[0157]

[0158] in, This is the feature matrix of the source domain nodes. The difference tensor. The introduction of this allows the model to explicitly model semantic changes while preserving structural features.

[0159] In the symmetric path from the target domain image back to the source domain image, a difference tensor is introduced. The reconstruction process is represented as follows:

[0160]

[0161] in, This is the feature matrix of the target domain nodes. This process uses the difference tensor. By converting target domain features into source domain features, bidirectional alignment and consistent modeling are achieved, effectively improving the model's robustness to cross-modal variations.

[0162] Furthermore, combining Figure 4 As shown, the cross-domain reconstruction loss in the multi-component loss function module The formula is:

[0163]

[0164] in, This represents the Frobenius norm square operation;

[0165] Furthermore, combining Figure 4 As shown, the structure regularization loss in the multi-component loss function module The formula is:

[0166]

[0167] in, express The graph Laplace matrix, ; express The graph Laplace matrix, ; This represents the learnable structural difference tensor. Source domain graph structure Corresponding source domain node feature matrix Perform element-by-element addition, and then sum to... Multiply; This represents the learnable structural difference tensor. and target domain graph The corresponding target domain node feature matrix Perform element-by-element addition, and then sum to... Multiply; Represents the trace operation of a matrix; superscript This indicates the transpose;

[0168] Furthermore, combining Figure 4 As shown, the energy sparsity loss in the multi-component loss function module The formula is:

[0169]

[0170] Furthermore, combining Figure 4 As shown, the bidirectional alignment loss in the multi-component loss function module The formula is:

[0171]

[0172] in, Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region; Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region; Indicates the total number of superpixel regions; Denotes the Euclidean norm of a vector; express The square of; Represents the natural exponential function;

[0173] Furthermore, combining Figure 4 As shown, the spatial smoothing loss in the multi-component loss function module The formula is:

[0174]

[0175] in, Represents the set of adjacency edges defined in the difference tensor space; Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region;

[0176] Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region; Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region; Representing the difference tensor No. The row vector of a row. , For the set of real numbers, The number of features in each superpixel region; Representing vectors Norm.

[0177] Based on the structural difference tensor corresponding to the trained structured graph autoencoder 1 The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map The specific process is as follows:

[0178] Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 1 norm square , For the reason The difference map obtained by mapping , ; For difference diagram Height and width; This represents the Frobenius norm square operation;

[0179] Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 2. norm square , For the reason The difference map obtained by mapping , ; For difference diagram Height and width;

[0180] Based on difference map And difference diagram To obtain the final difference map ; indicates as:

[0181]

[0182] in, This represents the mean of all elements in the difference plot matrix;

[0183] For the final difference diagram Perform automatic thresholding to extract regions of variation.

[0184] The post-processing of the difference map involves automatically thresholding the final difference map to extract regions of change. Automatic thresholding methods include, but are not limited to, the Otsu method, K-means clustering, iterative thresholding, and machine learning-based thresholding methods.

[0185] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoders, characterized in that: The specific process of the method is as follows: Step 1: Use sensor A to take a picture of region C at time 1 to obtain the source domain; Sensor B is used to capture images of region C at time 2 to obtain the target domain; Step 2: Divide the source domain to obtain... There are 10 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the source domain graph structure is obtained based on these nodes and lines, denoted as […]. ; Divide the target domain to obtain There are 3 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the target domain graph structure is obtained based on the nodes and lines, denoted as […]. ; Step 3: Construct a learnable structural difference tensor Learnable structural difference tensors Initialize to a zero matrix; , For the set of real numbers, This represents the total number of superpixel regions. The number of features in each superpixel region; Step 4: Construct the structured graph autoencoder 1; the specific process is as follows: The structured graph autoencoder 1 includes encoder 1 and decoder 1 in sequence; Encoder 1 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer; Decoder 1 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence. Step 5: Construct the structured graph autoencoder 2; the specific process is as follows: The structured graph autoencoder 2 includes an encoder 2 and a decoder 2 in sequence; Encoder 2 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer; Decoder 2 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence. Step Six: Based on Learnable Structural Difference Tensors and The structured graph autoencoder 1 is trained to obtain the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. ; Step 7: Based on learnable structural difference tensors and The structured graph autoencoder 2 is trained to obtain the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. ; Step 8: Based on the structural difference tensor corresponding to the trained structured graph autoencoder 1 The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map .

2. The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder according to claim 1, characterized in that: In step two, the source domain is divided to obtain... There are 10 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the source domain graph structure is obtained based on these nodes and lines, denoted as […]. ; Divide the target domain to obtain There are 3 superpixel regions, each of which is a node in the graph. Connections are made between these nodes, and the target domain graph structure is obtained based on the nodes and lines, denoted as […]. ; The specific process is as follows: The source domain is partitioned using the SLIC algorithm to obtain... Each superpixel region is a node in the graph. Get the distance to each node The most recent feature Nodes, select nodes Distance nodes respectively Recent Connect the nodes and obtain the source domain graph structure based on the nodes and lines, denoted as . ; The target domain is partitioned using the SLIC algorithm to obtain... Each superpixel region is a node in the graph. Get the distance to each node The most recent feature Nodes, select nodes Distance nodes respectively Recent Connect the nodes and obtain the target domain graph structure based on the nodes and lines, denoted as . .

3. The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder according to claim 2, characterized in that: In step six, the learnable structural difference tensor is used. and The structured graph autoencoder 1 is trained to obtain the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. ; The specific process is as follows: Step 61: Transfer the learnable structural difference tensor and By adding elements one by one, we can obtain the features. ; Step 62, Features Input structured graph autoencoder 1, output feature matrix The specific process is as follows: feature Encoder 1 and decoder 1 are input sequentially, and decoder 1 outputs the feature matrix. ; Step 63: Repeat steps 61 to 62 until the total loss function converges, obtaining the trained structured graph autoencoder 1 and its corresponding weights, biases, and structural difference tensors. .

4. The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder according to claim 3, characterized in that: The features in step six-two Encoder 1 and decoder 1 are input sequentially, and decoder 1 outputs the feature matrix. The specific process is as follows: Encoder 1 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer; Decoder 1 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence. feature The inputs are sequentially: first GCNConv, first ReLU activation function layer, first Dropout layer, second GCNConv, second ReLU activation function layer, second Dropout layer, third Dropout layer, third GCNConv, third ReLU activation function layer, fourth Dropout layer, fourth GCNConv, and fourth ReLU activation function layer. The fourth ReLU activation function layer outputs the feature matrix. .

5. The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder according to claim 4, characterized in that: In step seven, the learnable structural difference tensor is used. and The structured graph autoencoder 2 is trained to obtain the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. ; The specific process is as follows: Step 71: Transfer the learnable structural difference tensor and By adding elements one by one, we can obtain the features. ; Step 72, Characteristics Input: Structured Graph Autoencoder 2; Output: Feature Matrix The specific process is as follows: feature The inputs are sequentially fed into encoder 2 and decoder 2, and decoder 2 outputs the feature matrix. ; The weights of encoder 2 are the same as those of encoder 1; Step 73: Repeat steps 71 to 72 until the total loss function converges, obtaining the trained structured graph autoencoder 2 and its corresponding weights, biases, and structural difference tensors. .

6. The method for unsupervised change detection in cross-sensor remote sensing images based on a structured graph autoencoder according to claim 5, characterized in that: The features in step seven two The inputs are sequentially fed into encoder 2 and decoder 2, and decoder 2 outputs the feature matrix. The specific process is as follows: Encoder 2 includes, in sequence, a first GCNConv, a first ReLU activation function layer, a first Dropout layer, a second GCNConv, a second ReLU activation function layer, and a second Dropout layer; Decoder 2 consists of the third Dropout layer, the third GCNConv layer, the third ReLU activation function layer, the fourth Dropout layer, the fourth GCNConv layer, and the fourth ReLU activation function layer in sequence. feature The inputs are sequentially: first GCNConv, first ReLU activation function layer, first Dropout layer, second GCNConv, second ReLU activation function layer, second Dropout layer, third Dropout layer, third GCNConv, third ReLU activation function layer, fourth Dropout layer, fourth GCNConv, and fourth ReLU activation function layer. The fourth ReLU activation function layer outputs the feature matrix. .

7. The cross-sensor remote sensing image unsupervised change detection method based on structured graph autoencoder according to claim 6, characterized in that: The total loss function is: ; in, Indicates losses from cross-domain reconstruction; This represents the structure regularization loss; This indicates sparse energy loss; Indicates the two-way alignment loss; Indicates spatial smoothing loss; express The weights; express The weights; express The weights; express The weights; express The weights; .

8. The method for unsupervised change detection in cross-sensor remote sensing images based on a structured graph autoencoder according to claim 7, characterized in that: The cross-domain reconstruction loss The expression is: in, Representing the source domain graph structure The corresponding source domain node feature matrix; Represents the target domain graph structure The corresponding target domain node feature matrix; Indicates passage Characteristics of reconstruction; Indicates passage Characteristics of reconstruction; ； ； Encoder 1 represents the structured graph autoencoder 1; Decoder 1 represents the structured graph autoencoder 1; This represents the learnable structural difference tensor. Source domain graph structure Corresponding source domain node feature matrix Perform element-by-element addition, and input the result. , Output result input , Output feature matrix; Encoder 2 represents the structured graph autoencoder 2; Decoder 2 represents the structured graph autoencoder 2; This represents the learnable structural difference tensor. and target domain graph Corresponding target domain node feature matrix Perform element-by-element addition, and input the result. , Output result input , Output feature matrix; This represents the Frobenius norm square operation; The structure regularization loss The expression is: in, express The graph Laplace matrix, ; express The graph Laplace matrix, ; This represents the learnable structural difference tensor. Source domain graph structure Corresponding source domain node feature matrix Perform element-by-element addition, and then sum to... Multiply; This represents the learnable structural difference tensor. and target domain graph Corresponding target domain node feature matrix Perform element-by-element addition, and then sum to... Multiply; Represents the trace operation of a matrix; superscript This indicates the transpose; The energy sparsity loss The expression is: The bidirectional alignment loss The expression is: in, Representing the difference tensor No. The row vector of a row; Representing the difference tensor No. The row vector of a row; Indicates the total number of superpixel regions; Denotes the Euclidean norm of a vector; express The square of; Represents the natural exponential function; The spatial smoothing loss The expression is: in, Represents the set of adjacency edges defined in the difference tensor space; Representing the difference tensor No. The row vector of a row; Representing the difference tensor No. The row vector of a row; Representing the difference tensor No. The row vector of a row; Representing the difference tensor No. The row vector of a row; Representing vectors Norm.

9. The method for unsupervised change detection in cross-sensor remote sensing images based on a structured graph autoencoder according to claim 8, characterized in that: In step eight, the structural difference tensor corresponding to the trained structured graph autoencoder 1 is used. The structural difference tensor corresponding to the trained structured graph autoencoder 2 To obtain the final difference map ; The specific process is as follows: Step 81: Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 1. norm square , For the reason The difference map obtained by mapping , ; For difference diagram Height and width; This represents the Frobenius norm square operation; Step 82: Calculate the structural difference tensor corresponding to the trained structured graph autoencoder 2. norm square , For the reason The difference map obtained by mapping , ; For difference diagram Height and width; Step 83: Based on the difference map And difference diagram To obtain the final difference map ; indicates as: in, This represents the mean of all elements in the difference plot matrix; For the final difference diagram Perform automatic thresholding to extract regions of variation.

10. The method for unsupervised change detection of cross-sensor remote sensing images based on structured graph autoencoders according to claim 9, characterized in that: The automatic threshold segmentation method is the Otsu method, K-means clustering, iterative thresholding, or a thresholding method based on machine learning.

Images