Multi-modal remote sensing image change detection method based on robust anchor graph energy model

Abstract

The application relates to a multi-modal remote sensing image change detection method based on a robust anchor graph energy model. The method comprises the following steps: preprocessing registered pre-event and post-event multi-modal remote sensing images to obtain a superpixel set and a feature matrix; obtaining a stable anchor set through candidate anchor selection and reliable anchor screening; constructing a robust anchor graph to replace a traditional complete graph; calculating a structural difference measure of each superpixel through bidirectional mapping; constructing a hybrid energy function that fuses structural consistency, spatial smoothness and change sparsity; and obtaining a change probability through convex optimization and generating a difference map and a binary change map. The method can actively suppress the interference of change nodes on the graph representation, significantly reduce the computational complexity, and realize reliable spatial consistent change reasoning by combining various prior knowledge, and has robustness, high efficiency and high precision in the multi-modal remote sensing image change detection task.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a multimodal remote sensing image change detection method based on a robust anchor point graph energy model. Background Technology

[0002] With the rapid development of remote sensing sensors and airborne imaging technologies, multimodal remote sensing images, including optical, synthetic aperture radar (SAR), and infrared images, are becoming increasingly abundant. Multimodal change detection (MCD) has become an important and active research direction in the field of remote sensing. MCD aims to identify changes in land cover from multi-temporal remote sensing images of the same geographic area acquired by different sensors. Unlike single-modal change detection (UCD), which relies on pairs of images from the same source, MCD utilizes complementary information between heterogeneous observation data, demonstrating great potential in tasks such as disaster monitoring, urban sprawl analysis, and land use / cover change mapping.

[0003] Compared to UCD, MCD has two main advantages: First, it reduces the reliance on homogeneous image pairs acquired from the same sensor, which is particularly important in scenarios such as disaster assessment and emergency rescue. In these situations, homogeneous observations are often difficult to obtain due to changes in illumination, severe weather, cloud cover, or limited sensor availability, while multimodal imagery can provide more flexible and timely observation information. Second, MCD can utilize increasingly abundant multi-source remote sensing data to improve the temporal resolution of change monitoring.

[0004] However, since the images involved in MCD originate from different sensors, they have different imaging mechanisms and often exhibit significant differences in image style and statistical characteristics. Therefore, these images are difficult to compare directly, and cross-modal incomparability becomes a core problem that needs to be addressed. To solve this problem, existing MCD methods typically focus on projecting heterogeneous images into a common comparable space to reduce cross-modal differences and achieve change detection. Based on the construction method of the comparable space, these methods can be broadly classified into three categories: The first category is classification-based comparison methods. These methods first use a classifier to project multi-temporal images into the same semantic category space, and then achieve change detection by comparing the resulting land cover classification maps, such as post-classification comparison methods. The main advantage of this type of method is its conceptual simplicity and ability to provide semantic-level change information. However, its performance is highly dependent on the accuracy of the classification results, making it prone to error propagation. Furthermore, under unsupervised conditions, it is difficult to obtain high-precision multimodal image classification maps, which further limits the accuracy and robustness of change detection.

[0005] The second category is image translation-based methods. These methods aim to first convert multi-temporal images into the same image domain before performing change detection. Typically, one modality of imagery is converted to another, allowing the application of traditional UCD methods, such as those based on GANs, image regression, and diffusion models, in the converted image domain. The main advantage of this type of method is its ability to intuitively reduce cross-modal differences at the image level. Furthermore, the converted imagery provides richer visual information for subsequent analysis. However, since change comparison is still performed in the image domain, its detection performance is susceptible to image noise, especially speckle noise in SAR images. In addition, image translation and change detection are usually considered two independent processes; therefore, translation errors in changed regions directly affect subsequent difference analysis, reducing the reliability of the final detection results.

[0006] The third category is feature transformation-based methods. These methods aim to project multi-temporal images into a common feature space, preserving change-related information while suppressing cross-modal differences. Examples include methods based on structural features, frequency domain features, and depth features. Compared to image translation-based methods, they avoid direct comparisons in the image domain, thus generally being more robust to image noise. Furthermore, because modeling is performed in a latent feature space, these methods offer greater flexibility in modeling cross-modal relationships. However, these methods place high demands on the feature transformation model: it needs to effectively reduce inter-modal differences while ensuring good separability between changed and unchanged samples in the transformed feature space. Under unsupervised conditions or with insufficient training samples, these two objectives are often difficult to achieve simultaneously. Therefore, the learned feature space may still suffer from cross-modal confusion or insufficient change separability, thus limiting the final detection performance.

[0007] As the above analysis shows, the key to MCD lies in establishing robust correlations between multimodal images. Existing research indicates that multimodal remote sensing images of the same scene still share universal structural correlations, and these structural correlations constitute the invariant information required for reliable change detection. Inspired by this, graph-based MCD methods have received increasing attention in recent years for mining the shared structural consistency between multimodal images. This consistency provides an effective basis for projecting heterogeneous images into a common comparable space and performing change analysis. These graph-based methods are highly flexible because graph models can be constructed based on different types of nodes and different connection strategies (K-nearest neighbors, spatial neighborhood, and full connectivity, etc.), forming different structures such as local graphs, global graphs, and heterogeneous graphs to represent rich topological, geometric, and spatiotemporal relationships. Furthermore, they utilize the inherent structural consistency between modalities, achieving image transformation or feature transformation without requiring large amounts of training data. They typically operate under unsupervised or semi-supervised conditions and have explicit structural or physical constraints, thus exhibiting good interpretability.

[0008] Although existing graph-based MCD methods have achieved good performance, they still have several limitations in complex real-world scenarios: First, most methods indiscriminately connect changed nodes and unchanged nodes when constructing the graph model, resulting in insufficient robustness of the graph representation. During graph-based information propagation and interaction, changed nodes may introduce unreliable connections and further interfere with the structural representation of other nodes, thus causing change confusion. Second, existing graph-based MCD methods usually rely on large-scale complete graphs covering all image elements, which leads to high computational complexity and limits their application in large-scale, time-sensitive change detection tasks such as disaster assessment. Third, spatial constraints and structural constraints are usually modeled separately or only weakly coupled, making it difficult to fully utilize their complementarity to achieve accurate change inference. Summary of the Invention

[0009] Therefore, it is necessary to provide a multimodal remote sensing image change detection method based on a robust anchor point graph energy model that can simultaneously improve robustness, computational efficiency, and detection accuracy, addressing the aforementioned technical problems.

[0010] A multimodal remote sensing image change detection method based on a robust anchor point map energy model, the method comprising: The input registered pre-event multimodal remote sensing images and post-event multimodal remote sensing images are preprocessed to obtain the superpixel set of the pre-event image, the superpixel set of the post-event image, the superpixel feature matrix of the pre-event image, and the superpixel feature matrix of the post-event image. Candidate anchor points are selected for the pre-event and post-event images based on the corresponding superpixel sets and superpixel feature matrices, resulting in a pre-event candidate anchor point set and a post-event candidate anchor point set. Based on the principle of structural consistency, the approximate change probability of each candidate anchor point in the pre-event candidate anchor point set and the post-event candidate anchor point set is calculated, and the pre-event stable anchor point set and the post-event stable anchor point set are obtained by filtering according to the approximate change probability. Based on the superpixel feature matrix and stable anchor point set corresponding to the pre-event and post-event images, robust anchor point maps are constructed for the pre-event and post-event images to obtain the pre-event robust anchor point map and the post-event robust anchor point map. The pre-event robust anchor map is mapped to the post-event image domain, and the post-event robust anchor map is mapped to the pre-event image domain. The bidirectional mapping results are fused to calculate the structural difference metric for each superpixel, resulting in a global structural difference vector. A structural consistency constraint term is constructed based on the global structural difference vector, a spatial smoothness constraint term is constructed based on the spatial adjacency relationship of superpixels, and a variation sparsity constraint term is constructed based on the prior of the variation sparsity of remote sensing images. The three constraint terms are fused to obtain the hybrid energy function. The convex optimization solution of the hybrid energy function is used to obtain the change probability of each superpixel and generate a difference map; the difference map is then binarized to obtain the final binary change map.

[0011] In the above multi-modal remote sensing image change detection method based on the robust anchor graph energy model, first, candidate anchor point selection and reliable anchor point screening based on the structural consistency principle are adopted to extract high-confidence unchanged anchor points from a large number of superpixels as stable structural references, and then a robust anchor graph in which each node is connected only to a small number of reliable anchor points is constructed. This measure actively cuts off the interference links of changed nodes to the neighborhood structure in the graph construction stage, avoids the drawbacks of traditional methods that rely on complete graph dilution or iterative optimization to passively mitigate unstable connections, thereby effectively suppressing change confusion and improving the robustness of graph representation; at the same time, the compact node-anchor sparse graph is used to replace the original fully connected graph with O(N²) scale, reducing the computational complexity to O(N×M) (where M is the number of anchor points and M<<N), which significantly reduces storage and computing overhead and solves the efficiency bottleneck in large-scale remote sensing scenarios. Secondly, the global structural difference vector is obtained by bidirectionally mapping and fusing the pre-event and post-event robust anchor graphs, and on this basis, a hybrid energy function that jointly encodes structural consistency, spatial smoothness and change sparsity is constructed. This measure unifies three complementary priors in the same convex optimization framework, realizes the collaborative reasoning of structural evidence, spatial context and sparse prior, overcomes the problem of insufficient discriminative ability caused by the separation or weak coupling of spatial and structural constraints in existing methods, thereby effectively suppressing false alarms and missed detections while maintaining the integrity of the change area, and finally obtaining more accurate and spatially consistent change detection results. In summary, the present application simultaneously solves the technical problems of poor robustness, low efficiency and limited accuracy of existing methods through the core technical path of reliable anchor graph construction combined with joint optimization of the hybrid energy model. Brief Description of the Drawings

[0012] Figure 1 is a schematic flowchart of a multi-modal remote sensing image change detection method based on a robust anchor graph energy model in an embodiment; Figure 2 is a schematic diagram of the overall framework of a robust anchor graph energy model in an embodiment. Detailed Description of Embodiments

[0013] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0014] In one embodiment, as shown in Figure 1 and Figure 2 , a multi-modal remote sensing image change detection method based on a robust anchor graph energy model is provided, comprising the following steps: Step 1: Preprocess the input registered pre-event multimodal remote sensing images and post-event multimodal remote sensing images to obtain the superpixel set of the pre-event image, the superpixel set of the post-event image, the superpixel feature matrix of the pre-event image, and the superpixel feature matrix of the post-event image.

[0015] The preprocessing stage mainly includes two steps: superpixel segmentation and feature extraction. First, a collaborative segmentation strategy is used to segment the input image... and Jointly divided into There are superpixels, denoted as _ . and This ensures that each superpixel maintains good homogeneity within its respective image and corresponds to the same land cover category as much as possible. Using superpixels has two main advantages: firstly, it significantly reduces the computational complexity of large-scale remote sensing images; secondly, it better preserves important contextual and boundary information. After superpixel segmentation, feature extraction is further performed. Unlike many existing MCD methods that typically require extracting shared or comparable features and performing direct comparisons in a unified feature space, the method proposed in this application only requires that the extracted features have good discriminative power within their respective images, without needing direct comparability between different modalities. This greatly relaxes the requirements for feature extraction. For simplicity, this application uses the band mean and three quartile statistics of each superpixel as its feature representation, thus obtaining the feature matrix. and It should be noted that this is not the only feasible feature selection method. Other features, including those extracted by pre-trained models (such as semantic segmentation networks), can also be used in the method of this application, as long as they have sufficient discriminative power within their respective images.

[0016] Step 2: Based on the corresponding superpixel set and superpixel feature matrix, candidate anchor points are selected for the pre-event image and the post-event image to obtain the pre-event candidate anchor point set and the post-event candidate anchor point set.

[0017] Existing methods based on complete graphs have two main limitations: first, they have high computational complexity; second, they indiscriminately introduce changing nodes during the neighborhood construction process, which may lead to unreliable structural relationships and confusion caused by changes. To address these issues, this application constructs robust anchor maps for each image, achieving both compactness and robustness against interference from changing nodes.

[0018] Specifically, unlike building a graph model directly on all superpixels, this application first selects a small set of anchor points from each image. ,in These anchor points It is expected to serve as a representative structural reference for the entire image.

[0019] To construct the anchor point set, each image is first divided into... Each region consists of non-overlapping grid regions. Within each region, k-means clustering is performed on the stitched superpixel features. (Number of classes), and select the superpixel closest to the cluster center as the anchor point of the first class, denoted as . Furthermore, to enhance the representation of structurally significant regions, a superpixel with the largest spatial difference from its neighborhood is selected in each region and denoted as... Meanwhile, to increase anchor point diversity and avoid excessive concentration of anchor points in the dominant mode, a superpixel is randomly sampled in each region, denoted as . Finally, by merging the three subsets and removing duplicate superpixels, the final set of anchor points is obtained. The number of its anchor points satisfies .

[0020] This candidate anchor point selection strategy has three advantages. First, the grid-based division method ensures good spatial coverage of anchor points across the entire image, avoiding over-concentration of anchor points in local areas. Second, The anchor points selected based on cluster centers can preserve the dominant structural patterns in each region, providing a representative reference for graph construction. Third, and The anchor points in the graph incorporate structurally significant nodes and randomly sampled nodes, thereby further enhancing the diversity of the anchor point set and improving the robustness and representativeness of the anchor point graph.

[0021] Step 3: Calculate the approximate change probability of each candidate anchor point in the pre-event candidate anchor point set and the post-event candidate anchor point set based on the structural consistency principle, and obtain the pre-event stable anchor point set and the post-event stable anchor point set based on the approximate change probability.

[0022] In obtaining candidate anchor points The next step is to further screen for reliable anchor points, that is, anchor points that are more likely to remain unchanged, for use in the subsequent construction of a robust anchor point graph.

[0023] because Therefore, existing unsupervised MCD methods based on structural consistency can be directly applied to the candidate anchor point set to perform a coarse probability estimation of change for each candidate anchor point through complete graph modeling. This application employs a change detection method based on structure rule-induced change (RIEM). Considering that the spatial relationships between candidate anchor points are not explicit, the spatial neighborhood constraints used in RIEM are removed. The superpixels of candidate anchor points are used as the basis for this method. As input, the method constructs K-nearest neighbor graphs and K-farthest neighbor graphs respectively, and infers the change probability of each candidate anchor point based on a set of structural association rules. Its computational complexity is... Since the set of candidate anchor points is much smaller than the original set of superpixels, this computational overhead is acceptable.

[0024] After obtaining the rough change probability vector of the candidate anchor points Subsequently, only anchor points with high confidence levels that remained unchanged were retained as stable anchor points. Specifically, this application employs a threshold... Anchor point screening is performed, among which From the This was obtained by performing Otsu threshold segmentation, and Then set to The 0.5 quantile (considering that the variation is usually sparsity). Finally, it satisfies... The anchor point is selected as the stable anchor point, denoted as . The number of stable anchor points satisfies This data is then used to construct a robust anchor point graph.

[0025] Step 4: Construct robust anchor maps for the pre-event and post-event images based on the superpixel feature matrices and stable anchor set corresponding to the pre-event and post-event images, thus obtaining the pre-event robust anchor map and the post-event robust anchor map.

[0026] Robust anchor graph construction is one of the core innovations of this invention. It achieves a compact and robust graph representation by connecting each node to only a small number of reliable anchor points. The specific process of this step is as follows: After obtaining a set of stable anchor points... Subsequently, robust anchor maps (RAGs) were constructed on multi-temporal images. and Specifically, for each superpixel Only its relationship with the set of stable anchor points is considered. The association relationship between the K nearest neighbor anchor points, where In this way, the original complete graph... It has been replaced with a compact "node-anchor" representation. The specific definition is as follows: ; in, Indicates that it is determined based on the feature distance. In stable anchor point superpixel set The K nearest neighbors; The weight matrix represents the node-anchor association, where the connection weights are determined by the feature distance and normalized to satisfy the constraint that the row sum is 1. Similarly, a corresponding robust anchor graph can be constructed for post-event images. : ; in, express In stable anchor point superpixel set The K nearest neighbors in the array.

[0027] In the above way, each superpixel All can be represented as a small number of reliable anchor points. The weighted combination of anchor points enables the encoding of the intrinsic structural relationships of an image within a compact anchor-point domain. Compared to traditional full graph construction methods, anchor-point-based representation in RAG not only reduces the graph size from... Significantly reduced to Furthermore, by restricting graph interactions between stable structural references, the robustness of the model is improved.

[0028] Step 5: Map the pre-event robust anchor map to the post-event image domain, and simultaneously map the post-event robust anchor map to the pre-event image domain. Fuse the bidirectional mapping results to calculate the structural difference metric for each superpixel, and obtain the global structural difference vector.

[0029] The physical meaning of structural difference measurement lies in assessing whether the structural relationships defined in one image (i.e., the connections between superpixels and anchor points) can be preserved in another image. When a superpixel remains unchanged, its structural difference measurement should be small; conversely, if the superpixel changes, its structural difference measurement should be relatively large. A fusion method that averages after bidirectional mapping can make the change measurement more balanced and reliable. Specifically: set up Let each element be a variable to be inferred. Indicates the first The probability of change of each superpixel. Therefore, the unsupervised MCD problem proposed in this application can be expressed as the following energy minimization problem: ; in, This represents an energy function used to measure changes in variables. Under given constraints, with two multimodal images and The degree of matching. A smaller energy value means that the corresponding configuration variation can better meet the desired prior and objective. Therefore, by minimizing This will allow you to obtain the optimal configuration for the changes.

[0030] The energy term based on structural consistency is built upon the constructed robust anchor graph. Since each superpixel is connected only to its K nearest reliable anchors, and these anchors are assumed to be unchanged, the RAG graph, under the condition of invariance, [is optimized / enhanced]. and In theory, the connection nodes should maintain cross-modal structural consistency.

[0031] Specifically, in the diagram For example, for the edge ,because yes K nearest neighbors, therefore and They are considered to belong to the same land cover category; at the same time, due to ,therefore It is considered to remain unchanged. If If nothing changes, then the corresponding node in the image after the event... and (in A high degree of similarity should still be maintained to obtain a smaller distance. Conversely, if If changes occur, then and They are more likely to belong to different land cover categories, resulting in greater distances. Therefore, for the first The change metric for a superpixel can be expressed as: .

[0032] This process can be understood as transforming the graph Mapping to the post-event image domain and performing structural difference measurements within the same domain. Similarly, another set of change measures can be obtained: .

[0033] Furthermore, the two are combined to obtain the final measure of change. : .

[0034] thus, Used to measure the Can the structural relationships of the nodes be preserved between the two images? If each superpixel remains unchanged, then its It should be relatively small; conversely, if the superpixel changes, then its... It should be relatively large.

[0035] Step 6: Construct a structural consistency constraint term based on the global structural difference vector, a spatial smoothness constraint term based on the spatial adjacency relationship of superpixels, and a variation sparsity constraint term based on the prior of the variation sparsity of remote sensing images. Then, fuse the three constraint terms to obtain the hybrid energy function.

[0036] An energy model is an optimization framework that describes a target problem by minimizing an objective function composed of multiple energy terms, each encoding specific prior knowledge, constraints, or data consistency requirements. This step unifies three complementary priors into a single optimization framework, enabling more reliable change-based reasoning.

[0037] Based on the above analysis, the structural consistency term is defined as: .

[0038] This energy term is used to penalize nodes that have significant structural inconsistencies but are given unchanged labels.

[0039] According to the first law of geography, spatially adjacent regions are more likely to have similar properties than regions that are farther apart. In change detection tasks, this means that adjacent superpixels usually have similar states of change. Therefore, the change probability between spatially adjacent superpixels should remain smooth unless there is clear evidence that they are located on the boundary between changed and unchanged regions.

[0040] Based on the above observations, this application introduces a spatial smoothing term to encourage neighboring superpixels with high similarity to have similar change probabilities. Let... Represents the set of spatially adjacent superpixel pairs. Indicates the adjacent number With the The spatial affinity between superpixels is defined as follows: ; in ; and ,in, and Let represent the average neighborhood feature distance in multi-temporal images. The energy term based on spatial smoothness is defined as: ; Penalty weight The feature relationship between adjacent superpixels in two images is adaptively determined. When the distance between the two features is large (i.e., and When this happens, the superpixel pair is more likely to be located at the target boundary between the two images, and therefore is given a moderate penalty weight. .otherwise, Adjustments will be made based on the consistency of cross-modal pairwise relationships: when two superpixels are more likely to have the same changing state (i.e., and ), assigning greater weight ( ); and when they are more likely to have different states of change, they are given smaller weights.

[0041] Furthermore, If we consider it as the weight matrix of the spatial neighborhood graph, then the above energy terms It can be further written as: ; in, Indicates by and The Laplace matrix of the induced spatial neighborhood graph. This form indicates that the changing variables... Expected by The encoded spatial neighborhood map remains smooth. Since only spatially adjacent superpixel pairs are considered, therefore... and Both exhibit high sparsity, thus enabling the proposed spatial smoothing term to achieve high computational efficiency.

[0042] In real-world remote sensing scenarios, changed areas typically represent only a small proportion of the entire scene, while most areas remain unchanged. This sparsity prior is particularly pronounced in large-scale change detection tasks because changes are often localized and spatially finite. Therefore, the estimated change variables... It should satisfy the sparsity constraint.

[0043] Based on the above analysis, under constraints Under the given conditions, the following sparsity constraint energy term is introduced: ; By introducing this energy term, the proposed model can explicitly favor sparse configurations, thereby effectively suppressing false alarms in unchanged regions.

[0044] By combining the above energy terms based on structural consistency Energy terms based on spatial smoothness And energy terms based on the sparsity of change The hybrid energy function constructed in this application can be expressed as: ; in, These are parameters used to balance the contributions of each energy term.

[0045] In the mixed energy function In the middle, the first item Nodes with significant structural differences tend to be assigned variational solutions, i.e. ; Second item The tendency is to obtain spatially smooth solutions, meaning that adjacent nodes tend to have similar probabilities of change. ,in It is a constant; the third term Due to the prior of sparsity of change, it tends to obtain the unchanged solution, i.e. These energy terms are complementary in nature. Through joint optimization, structural evidence, spatial consistency, and sparsity of change can be considered simultaneously, thereby achieving more balanced and reliable inferences about changing variables.

[0046] To reduce the difficulty of parameter adjustment, this application addresses the parameter... An adaptive setting strategy is adopted. Specifically, through joint... and The corresponding energy can be written as: ; Regarding the above energy To find the minimum value, we can obtain the following solution: .

[0047] This result indicates that the equilibrium parameter Essentially, it acts as a threshold, used to measure change. The parts are separated into changed and unchanged parts. Inspired by this, this application will... Set to an adaptive Otsu threshold so that it can be based on The optimal segmentation point is automatically determined by the distribution. In this way, the proposed model avoids manual parameter adjustment and improves its adaptability to different scenarios and changing events.

[0048] Step 7: Perform convex optimization on the hybrid energy function to obtain the change probability corresponding to each superpixel and generate a difference map; perform binarization processing on the difference map to obtain the final binary change map.

[0049] Because the proposed energy function is simple in form and convex, it can be efficiently optimized using gradient descent. This application uses initial values... Set as the normalized structural difference vector After optimization, the result will be... The backmap is applied to the corresponding superpixel to generate a difference map.

[0050] To obtain a binary transformation diagram, two strategies can be used. The first strategy is to directly... Threshold segmentation or cluster segmentation is performed. This application uses the Otsu threshold segmentation method to obtain the final binary change label map. .

[0051] The second strategy is to use the probability vector in the mixed energy function. Replace with binary label vectors ,in Let represent the unchanged and changed states, respectively. Then, the energy function can be written as: .

[0052] This problem is essentially a classic graph cut model, therefore it can be solved efficiently using the minimum cut / maximum flow algorithm, thus obtaining the optimal binary solution. .

[0053] In one embodiment, candidate anchor points are selected for the pre-event image and the post-event image based on the corresponding superpixel set and superpixel feature matrix, resulting in a pre-event candidate anchor point set and a post-event candidate anchor point set, including: Divide the image into S Each grid region is a non-overlapping grid region; within each grid region, the superpixel features within that region are processed. k-means Clustering is performed, and the superpixel closest to the cluster center is selected as the cluster center anchor point. Within each grid region, the superpixel with the largest feature difference from the superpixels in the spatial neighborhood is selected as the structurally significant anchor point. A superpixel is randomly sampled within each grid region as the random sampling anchor point. The cluster center anchor point, the structurally significant anchor point, and the random sampling anchor point are merged and duplicate superpixels are removed to obtain the candidate anchor point set.

[0054] Specifically, this candidate anchor point selection strategy has three advantages: First, the grid-based partitioning method ensures good spatial coverage of anchor points across the entire image, avoiding excessive concentration of anchor points in local areas; second, anchor points selected based on cluster centers can preserve the dominant structural patterns in each region, providing a representative reference for graph construction; finally, structurally significant anchor points and randomly sampled anchor points introduce structurally significant nodes and randomly sampled nodes, respectively, further enhancing the diversity of the anchor point set and improving the robustness and representativeness of the anchor point graph.

[0055] In one embodiment, based on the principle of structural consistency, a rough probability of change for each candidate anchor point in the pre-event candidate anchor point set and the post-event candidate anchor point set is calculated. The pre-event stable anchor point set and the post-event stable anchor point set are then obtained by filtering based on the rough probability of change, including: The change detection method RIEM based on structural rules is adopted. A K nearest neighbor graph and a K farthest neighbor graph are constructed on the candidate anchor point set. A coarse change probability vector of each candidate anchor point is obtained by reasoning based on structural association rules. The first threshold is obtained by performing Otsu thresholding on the coarse change probability vector, and the second threshold is taken as the 0.5 quantile of the coarse change probability vector. Candidate anchor points whose coarse change probability is lower than both the first and second thresholds are retained as a set of stable anchor points.

[0056] Specifically, when constructing the K-nearest neighbor and K-farthest neighbor graphs on the candidate anchor set using the RIEM method, considering that the spatial relationships between candidate anchors are not explicit, the spatial neighborhood constraints used in RIEM can be removed. Since the candidate anchor set is much smaller than the original superpixel set, this computational cost is acceptable. A dual-threshold screening strategy—combining the Otsu threshold with a 0.5 quantile threshold—can retain anchors with high confidence and no change. Furthermore, considering that changes are typically sparse, the 0.5 quantile setting further ensures the reliability of the screening results.

[0057] In one embodiment, a robust anchor map is constructed for the pre-event and post-event images based on the superpixel feature matrices and stable anchor set corresponding to the pre-event and post-event images, including: For each superpixel, calculate its feature distance to all anchor points in the set of stable anchor points, and select the K anchor points with the smallest feature distance as the connection anchor points of the superpixel. The weights of each connection are calculated based on the feature distance, and the weights are normalized so that the sum of the connection weights for each superpixel is 1; a robust anchor map is constructed from all superpixels, connection anchors, and the normalized connection weights.

[0058] Specifically, by restricting the connectivity of each superpixel node to its K nearest-neighbor reliable anchors, the original complete graph is replaced with a compact node-anchor representation. The connection weights are determined by feature distances and normalized to satisfy the constraint that the row sum is 1. This allows each superpixel node to be represented as a weighted combination of a small number of reliable anchors, thus enabling the encoding of the image's intrinsic structural relationships within a compact anchor domain. Compared to traditional complete graph construction methods, robust anchor graphs not only significantly reduce graph size but also improve model robustness by restricting graph interactions to stable structural references.

[0059] In one embodiment, a pre-event robust anchor map is mapped to a post-event image domain, and a post-event robust anchor map is mapped to a pre-event image domain. The bidirectional mapping results are fused to calculate a structural difference metric for each superpixel, including: The pre-event robust anchor map is mapped to the post-event image domain, and the post-event robust anchor map is simultaneously mapped to the pre-event image domain. The fused bidirectional mapping results are used to calculate the structural difference metric for each superpixel. ; ; ; in, This indicates the structural difference in the mapping from the post-event anchor map to the pre-event image domain. This indicates the structural difference in the mapping from the pre-event anchor map to the post-event image domain. This represents the result of mapping the pre-event robust anchor map to the post-event image domain. i Structural differences of each superpixel This represents the set of edges in the robust anchor graph of the pre-event image. In the robust anchor map of the pre-event image, superpixels i With anchor point j The connection weights, Indicates superpixels in the image domain after the event. i With anchor point j Feature distance, In the robust anchor point map representing the post-event image, superpixels i With anchor point j The connection weights, Indicates superpixels in the image domain before the event. i With anchor point j Feature distance, This represents the set of edges in the robust anchor graph of the post-event image. Indicates the first i The difference in mapping from the post-event anchor map to the pre-event domain for each superpixel.

[0060] Specifically, this calculation process can be understood as mapping the pre-event robust anchor map to the post-event image domain and performing structural difference measurement within the same domain. Simultaneously, the post-event robust anchor map is mapped back to the pre-event image domain for symmetric measurement. Finally, the bidirectional mapping results are averaged to obtain the final difference measurement. This bidirectional mapping fusion method effectively avoids the biases that may arise from unidirectional mapping, making the change measurement more balanced and reliable.

[0061] In one embodiment, the spatial smoothness constraint is: ; ;

[0062] in, For spatial smoothness constraints, It is a set of spatially adjacent superpixel pairs. For the first i The and the first j Adaptive penalty weights between adjacent superpixels , The first i The, the j The probability of change of each superpixel As an intermediate feature variable, Indicates superpixels in the image domain before the event. i With anchor point j Feature distance, Indicates superpixels in the image domain after the event. i With anchor point j Feature distance, This is the statistical mean of the feature distances between all adjacent superpixel pairs in the pre-event image. It is the statistical mean of the feature distances between all adjacent superpixel pairs in the post-event image.

[0063] Specifically, this spatial smoothness constraint term is implemented through adaptive penalty weights. This is used to adjust the strength of the smoothing constraint between adjacent superpixel pairs. When the feature distance between adjacent superpixel pairs is large in both time phases, it indicates that they are likely located on both sides of the true change boundary, and a moderate smoothing penalty is applied. When the feature distance is small in at least one time phase, it indicates that they are more likely to belong to the same land cover, and the penalty weight is adaptively adjusted according to the degree of difference in feature distance. Since only spatially adjacent superpixel pairs are considered, this constraint term has high sparsity and computational efficiency.

[0064] In one embodiment, the hybrid energy function is: ; in, For mixed energy functions, Indicates by and The Laplace matrix of the induced spatial neighborhood graph, The equilibrium parameters for the spatial smoothness constraint term, The balance parameter for the varying sparse terms, Represents the probability vector of change. This represents a structural difference vector, where 1 indicates a vector of all 1s, and the superscript... T This indicates the transpose operation.

[0065] Specifically, the hybrid energy function includes a structural consistency constraint term that penalizes nodes with significant structural inconsistencies that are assigned unchanged labels; a spatial smoothness constraint term that penalizes drastic differences in change probabilities between adjacent superpixels; and a variation sparsity constraint term that, due to the properties of the L1 norm, induces sparse solutions, aligning with the prior knowledge that changed regions typically constitute only a small proportion in real-world remote sensing scenarios. These three energy terms are complementary in nature, and joint optimization can simultaneously consider structural evidence, spatial consistency, and variation sparsity.

[0066] In one embodiment, the balance parameter is set adaptively, specifically by setting the balance parameter in an adaptive manner. Set as the adaptive Otsu threshold for the global structural difference vector.

[0067] Specifically, by combining the structural consistency term and the variation sparsity term, the corresponding energy can be written as... Finding the minimum value yields a solution whose form indicates the equilibrium parameters. Essentially, it acts as a threshold, separating structural difference measures into changed and unchanged components. The adaptive Otsu threshold is set so that it can automatically determine the optimal segmentation point based on the distribution of the structural difference vector, thereby avoiding manual parameter adjustment and improving the model's adaptability to different scenarios and changing events.

[0068] In one embodiment, a convex optimization solution is performed on the hybrid energy function to obtain the change probability corresponding to each superpixel, generating a difference map; the difference map is then binarized to obtain the final binary change map, including: The normalized global structural difference vector is used as the initial value of the change probability; the gradient descent method is used to iteratively optimize the mixing energy function to obtain the converged change probability vector; the change probability vector is back-mapped to the corresponding superpixel to generate the difference map. The difference map is binarized using the Otsu threshold segmentation method to obtain a binary change map; or the change probability vector in the mixed energy function is replaced with a binary label vector, and the binary change map is obtained by solving the minimum cut / maximum flow algorithm.

[0069] Specifically, since the constructed energy function is convex, gradient descent can efficiently obtain the global optimum. Using the normalized structural difference vector as the initial value of the change probability can accelerate convergence. For obtaining the binary change map, the difference map can be directly segmented using the Otsu threshold, which is simple and efficient; alternatively, the probability vector in the energy model can be replaced with a binary label vector, transforming the problem into a classic graph cut model, where the minimum cut / maximum flow algorithm can be used to obtain the globally optimal binary solution. Both strategies can yield reliable binary change detection results.

[0070] In one embodiment, the preprocessing process includes: A collaborative segmentation strategy is adopted to jointly segment the pre-event image and the post-event image into N superpixels, so that each superpixel maintains homogeneity within its own image. The band mean and three quartile statistics of each superpixel are extracted to form the feature vector of the superpixel. Arrange the feature vectors of all superpixels in order to obtain the superpixel feature matrix of the image before the event and the superpixel feature matrix of the image after the event.

[0071] Specifically, the collaborative segmentation strategy ensures that each superpixel maintains good homogeneity within its respective image and corresponds to the same land cover category as much as possible. Feature extraction uses the band mean and three quartile statistics of each superpixel as feature representations. This method is simple, effective, and does not require comparability between modalities. It should be noted that this is not the only feasible feature selection method; other features, as long as they have sufficient discriminative power within their respective images, can also be used in this method.

[0072] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this application, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0073] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multimodal remote sensing image change detection method based on a robust anchor point graph energy model, characterized in that, The method includes: The input registered pre-event multimodal remote sensing images and post-event multimodal remote sensing images are preprocessed to obtain the superpixel set of the pre-event image, the superpixel set of the post-event image, the superpixel feature matrix of the pre-event image, and the superpixel feature matrix of the post-event image. Candidate anchor points are selected for the pre-event and post-event images based on the corresponding superpixel sets and superpixel feature matrices, resulting in a pre-event candidate anchor point set and a post-event candidate anchor point set. Based on the principle of structural consistency, the approximate change probability of each candidate anchor point in the pre-event candidate anchor point set and the post-event candidate anchor point set is calculated, and the pre-event stable anchor point set and the post-event stable anchor point set are obtained by filtering according to the approximate change probability. Based on the superpixel feature matrix and stable anchor point set corresponding to the pre-event and post-event images, robust anchor point maps are constructed for the pre-event and post-event images to obtain the pre-event robust anchor point map and the post-event robust anchor point map. The pre-event robust anchor map is mapped to the post-event image domain, and the post-event robust anchor map is mapped to the pre-event image domain. The bidirectional mapping results are fused to calculate the structural difference metric for each superpixel, resulting in a global structural difference vector. Based on the global structural difference vector, a structural consistency constraint term is constructed, a spatial smoothness constraint term is constructed based on the spatial adjacency relationship of superpixels, and a variation sparsity constraint term is constructed based on the prior of the variation sparsity of remote sensing images. The three constraint terms are fused to obtain a hybrid energy function. The hybrid energy function is solved by convex optimization to obtain the change probability corresponding to each superpixel, and a difference map is generated; the difference map is then binarized to obtain the final binary change map.

2. The method according to claim 1, characterized in that, Based on the corresponding superpixel set and superpixel feature matrix, candidate anchor points are selected for the pre-event and post-event images, resulting in a pre-event candidate anchor point set and a post-event candidate anchor point set, including: Divide the image into S Each grid region is a non-overlapping grid region; within each grid region, the superpixel features within that region are processed. k-means Clustering is performed, and the superpixel closest to the cluster center is selected as the cluster center anchor point. Within each grid region, the superpixel with the largest feature difference from the superpixels in the spatial neighborhood is selected as the structurally significant anchor point. A superpixel is randomly sampled within each grid region as the random sampling anchor point. The cluster center anchor point, the structurally significant anchor point, and the random sampling anchor point are merged and duplicate superpixels are removed to obtain the candidate anchor point set.

3. The method according to claim 1, characterized in that, Based on the principle of structural consistency, the approximate change probability of each candidate anchor point in the pre-event and post-event candidate anchor point sets is calculated. Based on these approximate change probabilities, the pre-event stable anchor point set and the post-event stable anchor point set are obtained, including: The change detection method RIEM based on structural rules is adopted. A K nearest neighbor graph and a K farthest neighbor graph are constructed on the candidate anchor point set. A coarse change probability vector of each candidate anchor point is obtained by reasoning based on structural association rules. The first threshold is obtained by performing Otsu threshold segmentation on the coarse change probability vector, and the 0.5 quantile of the coarse change probability vector is taken as the second threshold. Candidate anchor points whose coarse change probability is lower than both the first and second thresholds are retained as a set of stable anchor points.

4. The method according to claim 1, characterized in that, Robust anchor maps are constructed for the pre-event and post-event images based on the superpixel feature matrices and stable anchor point sets corresponding to the pre-event and post-event images, including: For each superpixel, calculate its feature distance to all anchor points in the set of stable anchor points, and select the K anchor points with the smallest feature distance as the connection anchor points of the superpixel. The weights of each connection are calculated based on the feature distance, and the weights are normalized so that the sum of the connection weights for each superpixel is 1; a robust anchor map is constructed from all superpixels, connection anchors, and the normalized connection weights.

5. The method according to claim 1, characterized in that, The pre-event robust anchor map is mapped to the post-event image domain, and the post-event robust anchor map is mapped to the pre-event image domain. The bidirectional mapping results are fused to calculate the structural difference metric for each superpixel, including: The pre-event robust anchor map is mapped to the post-event image domain, and the post-event robust anchor map is simultaneously mapped to the pre-event image domain. The fused bidirectional mapping results are used to calculate the structural difference metric for each superpixel. in, This indicates the structural difference in the mapping from the post-event anchor map to the pre-event image domain. This indicates the structural difference in the mapping from the pre-event anchor map to the post-event image domain. This represents the result of mapping the pre-event robust anchor map to the post-event image domain. i Structural differences of each superpixel This represents the set of edges in the robust anchor graph of the pre-event image. In the robust anchor map of the pre-event image, superpixels i With anchor point j The connection weights, Indicates superpixels in the image domain after the event. i With anchor point j Feature distance, In the robust anchor point map representing the post-event image, superpixels i With anchor point j The connection weights, Indicates superpixels in the image domain before the event. i With anchor point j Feature distance, This represents the set of edges in the robust anchor graph of the post-event image. Indicates the first i The difference in mapping from the post-event anchor map to the pre-event domain for each superpixel.

6. The method according to claim 5, characterized in that, The spatial smoothness constraint term is: in, For spatial smoothness constraints, It is a set of spatially adjacent superpixel pairs. For the first i The and the first j Adaptive penalty weights between adjacent superpixels , The first i The, the j The probability of change of each superpixel As an intermediate feature variable, Indicates superpixels in the image domain before the event. i With anchor point j Feature distance, Indicates superpixels in the image domain after the event. i With anchor point j Feature distance, This is the statistical mean of the feature distances between all adjacent superpixel pairs in the pre-event image. It is the statistical mean of the feature distances between all adjacent superpixel pairs in the post-event image.

7. The method according to claim 6, characterized in that, The hybrid energy function is: in, For mixed energy functions, Indicates by and The Laplace matrix of the induced spatial neighborhood graph, These are the equilibrium parameters for the spatial smoothness constraint term. The balance parameter for the varying sparse terms, Represents the probability vector of change. This represents a structural difference vector, where 1 indicates a vector of all 1s, and the superscript... T This indicates the transpose operation.

8. The method according to claim 7, characterized in that, The balance parameters are set adaptively, specifically by setting the following parameters: Set as the adaptive Otsu threshold for the global structural difference vector.

9. The method according to claim 1, characterized in that, The hybrid energy function is solved by convex optimization to obtain the change probability corresponding to each superpixel, generating a difference map; the difference map is then binarized to obtain the final binary change map, including: The normalized global structural difference vector is used as the initial value of the change probability; the gradient descent method is used to iteratively optimize the mixing energy function to obtain the converged change probability vector; the change probability vector is back-mapped to the corresponding superpixel to generate the difference map. The difference map is binarized using the Otsu threshold segmentation method to obtain a binary change map; or the change probability vector in the mixed energy function is replaced with a binary label vector, and the binary change map is obtained by solving the minimum cut / maximum flow algorithm.

10. The method according to claim 1, characterized in that, The preprocessing process includes: A collaborative segmentation strategy is adopted to jointly segment the pre-event image and the post-event image into N superpixels, so that each superpixel maintains homogeneity within its own image. The band mean and three quartile statistics of each superpixel are extracted to form the feature vector of the superpixel. Arrange the feature vectors of all superpixels in order to obtain the superpixel feature matrix of the image before the event and the superpixel feature matrix of the image after the event.

Images