Bayesian fusion method of latent variables for optical-sar isometric change image segmentation

Abstract

The application discloses an optical-SAR isomer change image segmentation latent variable Bayesian fusion method, and belongs to the technical field of remote sensing image processing and multi-modal information fusion. The method comprises the following steps: acquiring double-time-phase remote sensing images and preprocessing; estimating the latent variable posterior distribution of the real ground state by using an optical latent variable encoder and a SAR latent variable encoder; calculating comprehensive uncertainty and modal reliability parameters; obtaining the fusion latent variable posterior distribution of two time phases by Bayesian fusion weighted by the precision matrix and the modal reliability parameters; calculating the probability distance and the statistical difference to infer the change posterior probability; and outputting the change image segmentation graph, the uncertainty graph and the modal reliability graph by using spatial prior optimization. The application can significantly reduce false changes, improve the accuracy and robustness of change image segmentation, and is widely applicable to urban construction monitoring, flood inundation extraction, forest felling identification and other scenes.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and multimodal information fusion technology, specifically to a latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation. Background Technology

[0002] Remote sensing image change segmentation refers to the analysis of remote sensing images of the same geographic area acquired at different times to identify areas where land cover, land cover structure, or target status has changed. With the widespread application of optical remote sensing satellites and synthetic aperture radar (SAR) satellites, optical-SAR heterogeneous change image segmentation has become a crucial technical component in urban construction monitoring, flood inundation extraction, deforestation identification, disaster damage assessment, agricultural monitoring, and land resource supervision. Optical remote sensing images possess rich spectral, textural, and semantic information, facilitating the identification of land cover categories, boundary morphology, and visual textures. SAR remote sensing images offer all-weather, all-day imaging capabilities, are insensitive to cloud and fog obstruction, and can reflect physical characteristics such as surface roughness, structural scattering, and specular reflection from water bodies. While optical and SAR remote sensing images are significantly complementary, their imaging mechanisms differ greatly, making direct comparison extremely difficult.

[0003] Prior art related to this invention includes image segmentation methods based on image difference or feature difference, heterogeneous image segmentation methods based on deep feature fusion, image segmentation methods based on domain adaptation or cross-region generalization, probabilistic or Bayesian image segmentation methods, and multimodal fusion methods assisted by uncertainty or reliability.

[0004] Image segmentation methods based on image difference or feature difference first perform geometric registration, radiometric normalization, and filtering on dual-temporal remote sensing images. Then, they calculate pixel-level differences, ratios, logarithmic ratios, change vectors, or depth feature differences. Finally, they output a change map through thresholding, clustering, classifiers, or change image segmentation networks. These methods are simple in structure and suitable for segmenting same-modality optical-optical or SAR-SAR change images. In optical-SAR heterogeneous scenarios, it is usually necessary to first map different modalities to a comparable feature space before performing difference discrimination.

[0005] Heterogeneous change image segmentation methods based on deep feature fusion typically employ optical encoders and SAR encoders to extract features from optical and SAR remote sensing images, respectively. The fused features are then obtained through techniques such as stitching, subtraction, attention, cross-attention, gating fusion, Siamese networks, or non-shared weight networks, and the change image segmentation head outputs a change probability map. To mitigate the modal differences between optical and SAR remote sensing images, existing methods often introduce common latent space learning, feature consistency loss, contrastive learning loss, reconstruction loss, or cross-modal projection modules to make different modalities semantically or distributionally similar.

[0006] Variation image segmentation methods based on domain adaptation or cross-regional generalization address the performance degradation problem when there are differences in season, sensor, resolution, land cover type, lighting conditions, noise intensity, and dominant variation type between the training and target regions. A common approach is to add a domain discriminator, adversarial training module, pseudo-label filtering module, contrastive learning module, or optimal transport distribution alignment module after the feature extraction network to reduce the distributional differences between the source and target domains. The main goal of these methods is to learn stable variation representations across regions, seasons, or sensors.

[0007] Probabilistic or Bayesian change-based image segmentation methods not only output hard labels but also estimate the probability of changes occurring, or introduce spatial priors, temporal priors, or probabilistic inference mechanisms through Markov random fields, conditional random fields, Bayesian nonparametric models, Bayesian online change point detection, etc. Existing methods can handle problems involving the number of unknown categories, spatial correlations, temporal change points, and multi-source data fusion, but they still have shortcomings in optical-SAR deep latent variable fusion, pixel-level modal reliability modeling, posterior uncertainty output, and engineering system deployment.

[0008] Uncertainty- or reliability-assisted multimodal fusion methods typically generate confidence maps through dual decoder consistency, model ensemble, Monte Carlo Dropout, remote sensing image quality assessment, cloud masking, SAR speckle noise estimation, or out-of-domain detection scores, and then use the confidence scores for pseudo-label screening, loss weighting, attention weighting, or fusion weight allocation. The level of confidence varies across different regions and modalities, but most existing methods treat uncertainty as a training aid or post-processing weight, failing to embed uncertainty into the posterior inference of real surface state latent variables and the main Bayesian fusion process.

[0009] Existing optical-SAR heterogeneous change image segmentation techniques are prone to misinterpreting modal differences as real changes. Optical remote sensing images reflect solar reflectance spectra and ground texture, while SAR remote sensing images reflect radar backscattering intensity and coherence structure. These two types of remote sensing images differ significantly in numerical distribution, spatial texture, edge morphology, and noise characteristics. Directly differentiating optical and SAR features, or simply stitching and fusing them using ordinary attention modules, may misinterpret differences in brightness, scattering, shadowing, geometric overlay, and speckle noise caused by imaging mechanisms as surface changes. The impact is particularly pronounced on building edges, water body boundaries, mountain shadows, and areas of phenological change in farmland.

[0010] Most existing fusion methods lack explicit representations of latent variables related to the true surface state. Conventional deep learning methods typically map optical and SAR remote sensing images directly to discriminative features, and then use a segmentation head to learn the classification boundaries of change or no change, without clearly distinguishing between the true surface state, sensor observations, observation noise, and modal style. When training data is insufficient, the distribution of the target region changes, or the quality of any modality deteriorates, the model struggles to explain the source of errors and adjust the fusion logic based on observation reliability.

[0011] Existing uncertainty estimation methods typically remain at an auxiliary level, failing to generate calibrable posterior probabilities of change. Some methods use bi-headed consistency or multiple forward inference to estimate confidence levels, but such confidence levels usually reflect differences in the model's internal output and are not equivalent to the uncertainty of the posterior distribution of the true surface state. Even when two predictors agree on the same erroneous conclusion, the model may still give a high-confidence prediction. Applications such as disaster emergency response, land enforcement, and major engineering monitoring require not only binary change image segmentation maps but also the identification of areas with high change probabilities, uncertain areas, and areas requiring manual verification.

[0012] Existing methods lack adaptive modeling of modal reliability for different spatial locations. Optical remote sensing images exhibit decreased reliability in areas with clouds, thin clouds, shadows, low solar altitude angles, atmospheric interference, or seasonal differences; SAR remote sensing images show decreased reliability in areas with strong speckle noise, overlay, shadows, abnormal incident angles, topographic relief, or strong scattering saturation. Fixed fusion weights, channel attention, or global gating are insufficient to achieve dynamic reliability allocation at the pixel, patch, or object levels, and localized quality degradation scenarios can easily lead to single-modal errors affecting the fusion results.

[0013] Existing methods are not sufficiently adaptable to missing modes or asynchronous observations. In practical remote sensing operations, high-quality optical and SAR remote sensing images may not be available simultaneously for two time phases. During post-disaster cloud and rain weather, only SAR remote sensing images may be available, and historical time phases may only have optical remote sensing image archives. Some areas may have missing or invalid remote sensing image stripes. The fixed four-channel input structure limits the scope of engineering application, while filling missing modes with zero or mean values ​​introduces false information.

[0014] Existing image segmentation methods for changes lack sufficient spatial prior constraints. Remote sensing changes typically appear as objects such as land parcels, buildings, roads, water bodies, landslides, and burned areas, exhibiting spatial continuity and boundary structures. Simple pixel-level discrimination is prone to producing salt-and-pepper noise, isolated change points, fragmented boundaries, and voids within objects. Some methods use conditional random fields or morphological post-processing, but spatial constraints are usually treated as external post-processing and are not jointly modeled with the posterior probability of change, latent variable uncertainty, and modal reliability.

[0015] Existing training objectives often focus on F1, IoU, or cross-entropy, lacking confidence calibration and risk representation. Change image segmentation results require tiered processing in operational applications, including direct storage of high-confidence changes, manual verification of medium-confidence changes, and waiting for new remote sensing imagery for high-uncertainty areas. When models cannot output prediction uncertainty maps and modal reliability maps, even with high pixel accuracy, it is difficult to support traceable, interpretable, and auditable application processes. Summary of the Invention

[0016] The purpose of this invention is to provide a latent variable Bayesian fusion method for optical-SAR heterogeneous change image segmentation, which solves the technical problems of existing optical-SAR heterogeneous change image segmentation technology, which cannot effectively distinguish between differences in imaging mechanisms, observation noise and real surface changes, resulting in spurious changes, poor adaptability and single output information when local remote sensing image quality is degraded or modality is missing.

[0017] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: Latent variable Bayesian fusion methods for optical-SAR heterogeneous variation image segmentation include: Acquire remote sensing image data of the target area in the first and second time phases, wherein the remote sensing image data includes at least one of optical remote sensing images and SAR remote sensing images; The remote sensing image data is preprocessed to form image blocks containing quality masks; the optical remote sensing image and SAR remote sensing image are used as observations of latent variables of the real surface state under different imaging mechanisms, and the posterior distribution parameters of the latent variables are obtained by optical latent variable encoder and SAR latent variable encoder respectively. The comprehensive uncertainty and modal reliability parameters are calculated based on the latent variable posterior distribution parameters, the quality mask, and the out-of-domain anomaly information. Within each time phase, Bayesian fusion is performed based on the prior latent variables, the posterior latent variable distributions of each available mode, the precision matrix, and the mode reliability parameters to obtain the fused posterior latent variable distributions for the two time phases. The posterior probability of change is calculated based on the probability distance, mean difference, covariance difference, fusion uncertainty, and spatial context between the posterior distributions of the fused latent variables of the two time phases. Based on spatial adjacency relationships, the spatial prior consistency optimization is performed on the posterior probability of the change, and the output includes a change probability map, a binary change image segmentation map, a prediction uncertainty map, an optical mode reliability map, and a SAR mode reliability map, as well as optional change object boundary maps and confidence level maps.

[0018] As a preferred technical solution of the present invention, the preprocessing includes one or more of the following: coordinate system one, geometric fine registration, resolution resampling, clipping and segmentation, boundary overlap, radiometric normalization, SAR radiometric calibration, terrain correction, logarithmic transformation, speckle filtering, optical atmospheric correction, cloud and fog detection, shadow detection, and invalid value mask generation; the quality mask is used to represent optical clouds and fog, shadows, SAR anomalous noise, invalid values, or edge-filled values.

[0019] As a preferred embodiment of the present invention, the optical latent variable encoder and the SAR latent variable encoder adopt a non-shared weight structure, or share some parameters in the high-level semantic part; the optical latent variable encoder and the SAR latent variable encoder respectively output the latent variable mean and the latent variable covariance, the latent variable posterior distribution is Gaussian distribution, the latent variable covariance is diagonal covariance, the latent variable mean represents the estimate of the corresponding mode on the real surface state, and the latent variable covariance represents the uncertainty of the corresponding mode estimate.

[0020] As a preferred embodiment of the present invention, the comprehensive uncertainty of the m-th mode at position u in phase t is calculated according to the following formula:

[0021] in, This represents the overall uncertainty of the m-th mode at position u in phase t; This represents posterior uncertainty, obtained from the covariance trace of the latent variables; This represents the uncertainty in quality, obtained from a quality mask. This indicates uncertainty regarding out-of-domain anomalies, derived from the out-of-domain detection score. , and represents the preset or learnable coefficients; t represents the phase number; m represents the mode number; u represents the spatial position; the modal reliability parameters are calculated according to the following formula:

[0022] in, This represents the modal reliability parameter of the m-th mode at position u in phase t. This represents an exponential function; 'a' represents temperature or scaling parameter. This represents the overall uncertainty of the m-th mode at position u in phase t; This represents the set of available modes at time t; k represents the mode index in the set of available modes. The expression represents the comprehensive uncertainty of the k-th mode at position u in phase t; t represents the phase number; m represents the mode number; and u represents the spatial position.

[0023] As a preferred embodiment of the present invention, the posterior distribution of the fusion latent variable in each time phase is calculated according to the following formula:

[0024] in, Indicates the posterior distribution of the fused latent variables; Represents the latent variable at time t; This represents the optical remote sensing image at time t. This represents the SAR remote sensing image at time t. Indicates a direct proportional relationship; This represents the prior distribution of the latent variable at time t; The set of available modes for phase t is represented by m; m represents the mode number. Denotes the posterior distribution of the latent variable in the m-th mode; This represents the remote sensing image of the m-th mode at time t. This represents the modal reliability parameter of the m-th mode in phase t; t represents the phase number; the covariance of the fusion latent variables is calculated according to the following formula:

[0025] in, This represents the covariance of the fusion latent variables at time t. The superscript -1 indicates the prior covariance of the latent variable; The set of available modes for phase t is represented by m; m represents the mode number. The modal reliability parameter represents the m-th mode in phase t. The latent variable covariance of the m-th mode in phase t represents the latent variable covariance; t represents the phase number; the mean of the fused latent variable is calculated according to the following formula:

[0026] in, This represents the mean of the fusion latent variables at time t; This represents the covariance of the fusion latent variables at time t. This represents the prior covariance of latent variables; This represents the prior mean of the latent variable; The set of available modes for phase t is represented by m; m represents the mode number. The modal reliability parameter represents the m-th mode in phase t. This represents the latent variable covariance of the m-th mode at time t. This represents the mean of the latent variables in the m-th mode at time t; t represents the time phase number.

[0027] As a preferred embodiment of the present invention, the symmetric KL divergence of the posterior distribution of the two temporal fusion latent variables is calculated according to the following formula:

[0028] in, This represents the symmetric KL divergence at position u; KL represents the Kullback-Leibler divergence. This represents the posterior distribution of the fusion latent variables at the first time phase position u; This represents the posterior distribution of the fusion latent variables at the second time phase position u; Indicates the KL divergence direction separator; This represents the latent variable at the first phase position u; This represents the latent variable at the second time phase position u; u represents the spatial position; the Mahalanobis distance is calculated according to the following formula:

[0029] in, Indicates the Mahalanobis distance at position u; This represents the mean of the fusion latent variables at position u in the first time phase; This represents the mean of the fusion latent variables at position u in the second time phase; the superscript T indicates matrix transpose. This represents the covariance of the fusion latent variables at the first time phase position u; Let u represent the covariance of the fusion latent variable at the second time phase position u; the superscript -1 indicates matrix inversion; u represents the spatial position; the posterior probability of the change is calculated according to the following formula:

[0030] in, Represents a given set of available remote sensing images The posterior probability of the change at position u; This represents the variable that changes at position u; The mapping function represents the probability of output change; g represents the change discrimination module; This represents the probability distance feature at position u; This represents the difference vector of the means of the fusion latent variables at position u. ; This represents the diagonal element difference vector of the covariance of the fusion latent variables at position u. ; This represents the fusion uncertainty at position u. ; The local spatial context feature vector at position u is obtained by concatenating the channel vectors of the bottom feature maps of the optical latent variable encoder and the SAR latent variable encoder; u represents the spatial position.

[0031] As a preferred technical solution of the present invention, the spatial prior consistency optimization includes constructing a pixel map, superpixel map, plot map or object map, using spatial location or object as nodes and spatial adjacency as edges, determining edge weights based on spatial distance, optical spectral similarity, SAR scattering similarity, latent variable similarity, edge intensity or plot boundary relationship, and using Markov random field, conditional random field, graph cut, mean field inference, total variation regularization, morphological consistency constraint or graph neural network to obtain spatially consistent variation image segmentation results, and using the maximum value of the posterior target of the variation label as the output condition of spatial prior consistency optimization.

[0032] As a preferred embodiment of the present invention, the training samples include dual-temporal multimodal remote sensing images and change labels. The training objectives include one or more of the following: change image segmentation supervision loss, KL constraint between latent variable posterior and latent variable prior, consistency constraint between latent variables of different modalities in the same temporal phase, spatial smoothing or object consistency loss, and confidence calibration loss. During the training process, the optical latent variable encoder, SAR latent variable encoder, change discrimination module, and out-of-domain detector are updated according to the training objectives. When the modal reliability calculation module and the spatial prior optimization module are set to have learnable parameter structures, the corresponding module parameters are updated synchronously.

[0033] As a preferred technical solution of the present invention, when any time phase is missing an optical mode or SAR mode, the latent variable prior of the corresponding time phase, the latent variable posterior distribution of the available modes, and the mode reliability parameters of the available modes are used to obtain the fused latent variable posterior distribution; when setting a cross-mode completion branch, the latent variable posterior distribution parameters of the missing modes are predicted by the available modes, and then fused together with the latent variable prior and the latent variable posterior distribution of the available modes.

[0034] As a preferred technical solution of the present invention, the change probability map, binary change image segmentation map, prediction uncertainty map, optical mode reliability map and SAR mode reliability map are output as geographic raster files, vector speckles, tile services, raster probability maps, confidence level maps or manual verification task lists, for use in urban construction monitoring, flood inundation extraction, deforestation identification, disaster damage assessment, agricultural monitoring or land and resources supervision.

[0035] In addition, the present invention also discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and executed by the processor, wherein the processor executes the computer program to implement the latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation as described above.

[0036] In addition, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation as described above.

[0037] Furthermore, this invention also discloses an optical-SAR heterogeneous variation image segmentation latent variable Bayesian fusion system, comprising a data acquisition module, a preprocessing module, an optical latent variable encoding module, a SAR latent variable encoding module, an uncertainty estimation module, a modal reliability calculation module, a Bayesian fusion module, a variation posterior probability calculation module, a spatial prior optimization module, and a result output module. The data acquisition module transmits dual-temporal remote sensing image data, quality information, and geographic coordinate information to the preprocessing module. The preprocessing module transmits preprocessed corresponding modal remote sensing images to the optical and SAR latent variable encoding modules and transmits a quality mask to the uncertainty estimation module. The optical and SAR latent variable encoding modules transmit latent variable means and latent variable covariances to the uncertainty estimation and Bayesian fusion modules. The uncertainty estimation module transmits posterior uncertainty, quality uncertainty, and out-of-domain anomaly uncertainty to the modal reliability calculation module. The modal reliability calculation module transmits modal reliability parameters to the Bayesian fusion module. The Bayesian fusion module transmits the posterior distribution of the fused latent variables from the two temporal phases to the variation posterior probability calculation module. The posterior probability calculation module transmits the posterior probability of change and the prediction uncertainty to the spatial prior optimization module; the spatial prior optimization module transmits the spatially consistent change image segmentation result and optional change object boundary map and confidence level map to the result output module; the modal reliability calculation module transmits optical modal reliability parameters and SAR modal reliability parameters to the result output module; the result output module outputs the change probability map, the binary change image segmentation map, the prediction uncertainty map, the optical modal reliability map and the SAR modal reliability map, as well as the optional change object boundary map and confidence level map.

[0038] Compared with the prior art, the present invention has the following beneficial effects: This invention differs from existing technologies that directly compare or simply fuse optical and SAR remote sensing images based on their features. Instead, this method treats optical and SAR images as different observations of the same latent variable representing the true surface state. It obtains the posterior distribution parameters of the latent variable representing the true surface state using both optical and SAR latent variable encoders. This technique shifts change discrimination from the original image space with significant modal differences to a unified latent variable probability space. This fundamentally reduces spurious changes caused by differences in the imaging mechanisms of optical and SAR sensors, illumination variations, cloud and fog shadows, SAR speckle noise, and geometric overlay, significantly improving the accuracy of optical-SAR heterogeneous change image segmentation.

[0039] Building upon this foundation, this invention further calculates a comprehensive uncertainty encompassing posterior uncertainty, quality uncertainty, and out-of-domain anomaly uncertainty, and generates modal reliability parameters accordingly, thus realizing a Bayesian fusion mechanism that adapts to modal quality and observation conditions. By weighted fusion based on latent variable priors, the accuracy matrix of each available modality, and the modal reliability parameters within each temporal phase, high-quality, low-uncertainty modes contribute more to posterior inference, while the contribution of modes affected by cloud cover or strong speckle noise automatically decreases. This dynamic, pixel-level reliability assessment and fusion strategy not only avoids the influence of single-modal error information on the fusion result but also endows the system with robustness against local remote sensing image quality degradation.

[0040] To address potential modality loss in real-world remote sensing operations, this method employs a single-modal fusion path based on available modalities and latent variable priors. This avoids the spurious information introduced by filling missing modalities with meaningless values ​​such as zero or mean, significantly expanding its applicability across various scenarios including urban, disaster, and agricultural environments. Furthermore, by utilizing multi-dimensional probabilistic distance metrics such as symmetric KL divergence and Mahalanobis distance, and fusing mean differences, covariance differences, and local spatial context, this method can more accurately capture the statistical differences between the posterior distributions of two temporally fused latent variables. This allows for the calculation of calibrable posterior probabilities of change, simultaneously outputting a prediction uncertainty map.

[0041] This invention combines posterior probability of change with spatial prior consistency optimization, utilizing spatial adjacency relationships and edge weights to suppress isolated noise points while maintaining the boundaries and connectivity of real changed objects. This effectively reduces problems such as salt-and-pepper noise, boundary fragmentation, and internal voids within objects. The final output—a change probability map, a binary change image segmentation map, a prediction uncertainty map, and reliability maps for optical and SAR modes—constitutes a complete information chain from change probability and uncertainty to the reliable source of modes. This not only provides high-precision change identification results but also offers rich, traceable, and interpretable evidence for subsequent manual review, risk classification, and decision support. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0044] Figure 2 This is a flowchart of the uncertainty estimation and modal reliability calculation of the present invention.

[0045] Figure 3 This is a flowchart of the sub-processes for calculating the posterior probability of change and optimizing the consistency of spatial priors in this invention. Detailed Implementation

[0046] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0047] The following is in conjunction with the appendix Figures 1-3 The embodiments of the present invention will be described in detail below.

[0048] This invention proposes a latent variable Bayesian fusion method for segmenting heterogeneous change images from optical to SAR. The core idea is to avoid directly treating optical and SAR remote sensing images as subtractable homogeneous images, and to avoid mapping them merely to empirical common features. Instead, it represents the actual surface state as a latent variable, and the optical and SAR images as different observations of this latent variable. Each modal encoder estimates the posterior distribution of the latent variable, and then Bayesian fusion is performed based on uncertainty and modal reliability. Change image segmentation is transformed into an inference problem of probability differences between the posterior distributions of two temporally fused latent variables, thereby distinguishing between actual surface changes, modal differences, and observation noise.

[0049] Example 1: This example discloses a latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation, as detailed below: Let the target area be Ω and its spatial location be u. Remote sensing image data of the target area are acquired at the first time phase t=1 and the second time phase t=2. In this embodiment, the remote sensing image data includes:

[0050] in, Represents the optical remote sensing image at time t; This represents the SAR remote sensing image at time t; t represents the time phase number, which can be 1 or 2; the superscript O indicates the optical mode; the superscript S indicates the SAR mode.

[0051] The aforementioned remote sensing images undergo preprocessing. Preprocessing includes one or more of the following: coordinate system one, geometric fine registration, resolution resampling, cropping and partitioning, boundary overlap, radiometric normalization, SAR radiometric calibration, terrain correction, logarithmic transformation, speckle filtering, optical atmospheric correction, cloud and fog detection, shadow detection, and invalid value mask generation. Preprocessing aims to reduce significant geometric misalignments and scale inconsistencies to a level manageable by the model and provides quality information for subsequent reliability estimation.

[0052] Image blocks are formed after preprocessing:

[0053] in, This represents the i-th image block; i represents the image block number. This represents the i-th optical remote sensing image block in the first time phase; This represents the i-th SAR remote sensing image block in the first time phase; This represents the i-th optical remote sensing image block in the second time phase; This represents the i-th SAR remote sensing image block in the second time phase; This represents the quality mask for the i-th optical remote sensing image block; This represents the quality mask for the i-th SAR remote sensing image patch. The quality mask is a continuous value between [0,1], where 1 indicates excellent pixel quality and 0 indicates an invalid or completely obscured pixel. The quality mask can be obtained by methods such as cloud and fog detection, shadow detection, and SAR speckle noise intensity estimation.

[0054] In practice, geometric registration can be achieved through traditional control point matching, mutual information registration, depth feature matching, or joint registration networks; SAR speckle suppression can employ Lee filtering, Frost filtering, nonlocal mean filtering, depth denoising networks, or edge-preserving statistical filtering; optical cloud and fog detection can utilize threshold rules, cloud mask products, or depth cloud detection networks. Geographic coordinate information is used to maintain the spatial correspondence between two temporal remote sensing image patches and to write spatial reference information into the final exported geographic raster file or vector speckle.

[0055] The true surface state at time t and location u is determined by latent variables. Representation. Latent variables. It is not a raw observation from any single sensor, but a shared representation of land cover categories, structures, textures, scattering properties, and semantic states. Optical remote sensing images and SAR remote sensing images represent observations of this latent variable under different imaging mechanisms, respectively.

[0056] in, This represents the optical observation at position u in the t-th phase. Represents the probability distribution of optical observations; X represents the observation variable; This represents the latent variable at position u in phase t; Represents the probability distribution parameters of optical observations;

[0057] in, This represents the SAR observation at position u in time phase t; The SAR observation probability distribution is represented by X; X represents the observation variable. This represents the latent variable at position u in phase t; This represents the SAR observation probability distribution parameter.

[0058] The variable of change is defined as:

[0059] in, This represents the variable that changes at position u; Indicates an indicator function; This represents the latent variable at the first phase position u; This represents the latent variable at the second time phase position u. The inequality between latent variables indicates that the actual surface state represented by the latent variables has undergone substantial changes, while the values ​​of non-continuous latent variables are only due to arbitrary small differences caused by observation noise.

[0060] The objective of this invention is to infer the following posterior probability:

[0061] Where P represents probability; This indicates a change at position u; This represents the first phase of optical remote sensing imagery; This represents the first-phase SAR remote sensing image; This represents the second-phase optical remote sensing image; This represents the second-phase SAR remote sensing image.

[0062] The preprocessed optical remote sensing image is input into the optical latent variable encoder. The preprocessed SAR remote sensing image is input into the SAR latent variable encoder. Optical latent variable encoder and SAR latent variable encoder A non-shared weight structure is employed to adapt to different imaging mechanisms; alternatively, some parameters are shared in the high-level semantic part to enhance the consistency of the common latent space. The encoder outputs the posterior distribution parameters of the latent variables, preferably Gaussian distribution parameters:

[0063] in, This represents the posterior distribution of the latent variables of the optical modes; Represents the latent variable at time t; This represents the optical remote sensing image at time t. Indicates a Gaussian distribution; This represents the mean value of the latent variables of the optical modes at time t. This represents the latent variable covariance of the optical modes at time t.

[0064] in, Indicates the posterior distribution of SAR modal latent variables; Represents the latent variable at time t; This represents the SAR remote sensing image at time t. Indicates a Gaussian distribution; This represents the mean value of the latent variables of the SAR mode at time t. Let represent the covariance of the latent variables of the SAR mode at time t.

[0065] The latent variable mean represents the estimate of the actual surface state by the corresponding mode, and the latent variable covariance represents the uncertainty of the corresponding mode estimate. To reduce computational cost, diagonal covariance is used:

[0066] in, Let represent the latent variable covariance of the m-th mode at time t; m represents the optical mode or SAR mode. Represents the construction of diagonal matrices; to represents the standard deviation of each latent variable dimension in the m-th mode of phase t; d represents the latent variable dimension, and in this embodiment, the latent variable dimension is set to d=64.

[0067] Optical latent variable encoder and SAR latent variable encoder Both use the U-Net encoder structure, with an input image block spatial size of 256×256 and an output tensor of size 256×256×d. Each spatial location u corresponds to a d-dimensional mean vector. A d-dimensional variance vector, with the variances forming a diagonal covariance. .

[0068] The preferred latent variable posterior distribution is a Gaussian distribution with diagonal covariance, but low-rank covariance, block diagonal covariance, Gaussian mixture distribution, Student-t distribution, Laplace distribution, normalized flow, or diffusion-type latent variable representation can also be used. Gaussian distribution and diagonal covariance are used to form directly computable latent variable means, covariances, and accuracy matrices; other latent variable distributions are used to express heavy-tailed noise, multimodal states, or nonlinear posteriors.

[0069] This invention categorizes uncertainties into observational uncertainty, model uncertainty, and modal quality uncertainty. Observational uncertainty mainly stems from optical fog, shadows, atmospheric conditions, SAR speckle noise, incident angle differences, and geometric distortion; model uncertainty mainly arises from insufficient training samples, cross-regional distribution drift, and model parameter uncertainty; modal quality uncertainty primarily originates from remote sensing image quality scores, quality masks, reconstruction errors, or out-of-domain anomaly detection scores.

[0070] The quantities used to calculate overall uncertainty include posterior uncertainty, quality uncertainty, and out-of-domain anomaly uncertainty. Posterior uncertainty is obtained from the latent variable covariance trace; quality uncertainty is obtained from the quality mask; and out-of-domain anomaly uncertainty is obtained from the out-of-domain detection score.

[0071] Specifically, in this embodiment, each component is calculated as follows: Posterior uncertainty is defined as the trace of the latent variable covariance matrix:

[0072] in, This represents the posterior uncertainty of the m-th mode at position u in phase t; Represents the trace of a matrix; This represents the latent variable covariance of the m-th mode at position u in phase t; It represents the standard deviation of the m-th mode at position u in the j-th latent variable dimension at time t.

[0073] Quality uncertainty is defined as:

[0074] in, This represents the mass uncertainty of the m-th mode at position u in phase t; This represents the mass mask value of the m-th mode at position u in phase t.

[0075] The out-of-domain anomaly uncertainty is addressed by a lightweight out-of-domain detector, which is a binary classification convolutional neural network. Its input is the intermediate feature map of the encoder, and its output is the probability score of the current pixel belonging to the training distribution. During training, the feature maps of all samples in the training set are labeled as positive samples, while feature maps of remote sensing images with artificially added noise or from different regions and sensors are labeled as negative samples. Binary cross-entropy loss is used for training. After training, the parameters of the out-of-domain detector are fixed and not updated during the inference phase. The uncertainty of out-of-domain anomalies is defined as:

[0076] in, This indicates the out-of-domain anomalous uncertainty of the m-th mode at position u in phase t; This represents the out-of-domain detection score of the m-th mode at position u in phase t.

[0077] Define the comprehensive uncertainty of the m-th mode at position u in phase t as:

[0078] in, This represents the overall uncertainty of the m-th mode at position u in phase t; , and This represents a preset coefficient, which is taken in this embodiment. , , .

[0079] Further calculation of modal reliability parameters:

[0080] in, This represents the modal reliability parameter of the m-th mode at position u in phase t. represents an exponential function; 'a' represents the temperature parameter, which is taken as 1.0 in this embodiment. This represents the overall uncertainty of the m-th mode at position u in phase t; This represents the set of available modes at time t; k represents the mode index in the set of available modes. This represents the overall uncertainty of the k-th mode at position u in phase t.

[0081] Modes with higher modal reliability parameters contribute more to subsequent fusion; modes with lower modal reliability parameters participate in posterior fusion with lower accuracy. Modal reliability calculation can also be achieved using normalized inverse variance, learnable gating, Bayesian model averaging, evidence fusion, or a hybrid approach of rules and learning.

[0082] Clouds, shadows, or invalid values ​​in optical remote sensing images increase quality uncertainty, as do speckle noise, overlay, shadows, anomalies in incident angles, or strong scattering saturation in SAR remote sensing images. When the posterior distribution of latent variables deviates from the training data distribution, out-of-domain anomaly uncertainty increases. This overall uncertainty is transmitted to the latent variable Bayesian fusion process through modal reliability parameters, causing the modal contributions at different locations to vary with the remote sensing image quality.

[0083] Latent variable Bayesian fusion is performed within each time phase. In this embodiment, the prior distribution of the latent variables is set to a standard Gaussian distribution:

[0084] in, This represents the prior distribution of the latent variable at time t; represents the prior mean of the latent variable, and is a zero vector; Let represent the prior covariance of the latent variables, which is the identity matrix.

[0085] The posterior distributions of latent variables from different modalities are fused using a product-expert approach:

[0086] in, Indicates the posterior distribution of the fused latent variables; Represents the latent variable at time t; This represents the optical remote sensing image at time t. This represents the SAR remote sensing image at time t. Indicates a direct proportional relationship; This represents the prior distribution of the latent variable at time t; The set of modes available at time t; Denotes the posterior distribution of the latent variable in the m-th mode; This represents the remote sensing image of the m-th mode at time t. This represents the modal reliability parameter of the m-th mode in phase t.

[0087] When the posterior and prior of each modality are both Gaussian distributions, the posterior distribution of the fused latent variables is represented by a Gaussian distribution:

[0088] in, This represents the posterior distribution of the fusion latent variable at time t; Indicates a Gaussian distribution; This represents the mean of the fusion latent variables at time t. Let represent the covariance of the fusion latent variables at time t.

[0089] The covariance of the fused latent variables is:

[0090] in, This represents the covariance of the fusion latent variables at time t. The superscript -1 indicates the prior covariance of the latent variable; The set of modes available at time t; The modal reliability parameter represents the m-th mode in phase t. Let represent the latent variable covariance of the m-th mode at time t.

[0091] The mean of the fusion latent variables is:

[0092] in, This represents the mean of the fusion latent variables at time t; This represents the covariance of the fusion latent variables at time t. This represents the prior covariance of latent variables; This represents the prior mean of the latent variable; The set of modes available at time t; The modal reliability parameter represents the m-th mode in phase t. This represents the latent variable covariance of the m-th mode at time t. Let represent the mean of the latent variables in the m-th mode at time t.

[0093] The fusion weights are jointly determined by the accuracy matrix formed by the inverse covariance matrix of the latent variables and the modal reliability parameter. When the optical remote sensing image area is obscured by clouds, the posterior covariance of the optical modal latent variables increases or the optical modal reliability parameter decreases, reducing the contribution of the optical mode in the fusion process. When the SAR remote sensing image area is affected by speckle noise or overlay, the fusion process reduces the contribution of the SAR mode. When both the optical and SAR modes are reliable, the posterior distribution of the fused latent variables has relatively small uncertainty.

[0094] Bayesian fusion preferably employs product expert and precision matrix weighted fusion, but hybrid expert, Bayesian model averaging, evidence theory fusion, covariance cross fusion, variational inference fusion, or attention-based approximate Bayesian fusion can also be used. The fusion process utilizes the posterior uncertainty or modality reliability parameters of each mode to adjust the contribution and outputs the posterior distribution of the fused latent variables.

[0095] Obtain the posterior distribution of the first phase fusion latent variables Second phase fusion latent variable posterior distribution Then, the probabilistic distance, mean difference, and uncertainty difference between the posterior distributions of the two temporal fusion latent variables are calculated. The probabilistic distance includes symmetric KL divergence, Wasserstein distance, Mahalanobis distance, Bhattacharyya distance, or a combination of probabilistic distances.

[0096] The symmetric KL divergence is:

[0097] in, This represents the symmetric KL divergence at position u; KL represents the Kullback-Leibler divergence. This represents the posterior distribution of the fusion latent variables at the first time phase position u; This represents the posterior distribution of the fusion latent variables at the second time phase position u; The KL divergence direction separator is used to indicate the distance between two probability distributions.

[0098] For two d-dimensional diagonal Gaussian distributions , The explicit calculation of its KL divergence is as follows:

[0099] in, and Let represent the mean vectors of the two Gaussian distributions respectively; and Let represent the covariance matrices of the two Gaussian distributions, respectively. The matrix trace is indicated by the superscript T; the matrix transpose is indicated by the superscript -1; and the latent variable dimension is indicated by d. Represents the determinant of a matrix; It represents the natural logarithm.

[0100] The Mahalanobis distance is:

[0101] in, Indicates the Mahalanobis distance at position u; This represents the mean of the fusion latent variables at position u in the first time phase; This represents the mean of the fusion latent variables at position u in the second time phase; the superscript T indicates matrix transpose. This represents the covariance of the fusion latent variables at the first time phase position u; The value represents the covariance of the fusion latent variable at the second time phase position u; the superscript -1 indicates matrix inversion.

[0102] Furthermore, the input features used for change detection are defined as follows: Mean difference:

[0103] in, This represents the difference vector of the mean of the fusion latent variables at position u.

[0104] Covariance differences:

[0105] in, This represents the diagonal element difference vector of the covariance of the fusion latent variables at position u; This means taking the diagonal elements of a matrix to form a vector.

[0106] Integration uncertainty:

[0107] in, This represents the fusion uncertainty at position u.

[0108] The local spatial context feature H(u) is generated by the optical latent variable encoder. and SAR latent variable encoder The lowest layer feature map (resolution 256×256) is formed by concatenating the channel vectors at position u, which is used to provide local texture and edge information.

[0109] Combine the above features with probability distance features Input change detection module together:

[0110] in, Represents a given set of available remote sensing images The posterior probability of the change at position u; The mapping function representing the probability of output change, i.e. ;g represents the change detection module; The probability distance feature vector at position u is composed of the symmetric KL divergence and the Mahalanobis distance. This represents the difference vector of the means of the fusion latent variables at position u; This represents the diagonal difference vector of the covariance of the fusion latent variables at position u; H(u) represents the fusion uncertainty scalar at position u; H(u) represents the local spatial context feature vector at position u.

[0111] The change discrimination module g is implemented using a three-layer multilayer perceptron (MLP) with the number of neurons in each layer [256, 128, 1]. The first two layers use the ReLU activation function, and the last layer is a linear layer, outputting a scalar logit, which is then converted into a posterior probability of change using a sigmoid function. The change discrimination module g can also be implemented using a convolutional network, a Transformer module, a graph neural network, or a combination of these modules. The change probability is determined by the statistical difference in the posterior distribution of the latent variables of the true surface state at two different time points; when both time points have high uncertainty, the change discrimination module outputs a high prediction uncertainty.

[0112] Let the logit before the sigmoid output of the change discrimination module be... posterior probability of change Prediction uncertainty can be calculated using binary classification entropy:

[0113] in, This indicates the prediction uncertainty at position u; It represents the natural logarithm. Used to determine spatial smoothing intensity and generate prediction uncertainty maps; also applicable to... As an alternative measure of forecast uncertainty.

[0114] Change posterior probability Used for spatial prior consistency optimization, predicting uncertainty. Used to determine the intensity of spatial smoothing and to generate a prediction uncertainty map.

[0115] Transformed image segmentation results typically exhibit spatial continuity. This invention constructs a pixel map, using each spatial location as a node:

[0116] Where G represents the spatial relationship graph; V represents the set of nodes, each node corresponding to a spatial location u; and E represents the set of spatial adjacency edges, connecting each cell to its 8 neighboring cells.

[0117] Define the spatial prior of the change label C:

[0118] in, Represents the spatial prior probability of the changed label C; Indicates a direct proportional relationship; Represents an exponential function; Represents spatial prior coefficients, which are taken in this embodiment. ; E represents the set of spatial adjacent edges; This represents the edge weight between position u and position v; Indicates an indicator function; Indicates the change label at position u; Indicates the change label at position v.

[0119] edge weight The value is determined by the RGB color similarity and spatial distance of the optical remote sensing images. The specific calculation formula is as follows:

[0120] in, This represents the color vector of the optical remote sensing image at position u; Represents the L2 norm; and For bandwidth parameters, in this embodiment, we take... , Pixels. If optical remote sensing imagery is unavailable, the backscattering intensity of SAR remote sensing imagery is used instead of the color vector.

[0121] The final change result was obtained through maximum a posteriori estimation, and approximate optimization was performed using the mean field inference algorithm:

[0122] in, This represents the change label results obtained from the maximum a posteriori estimation; This indicates the operation that maximizes the objective function on the change label C; u represents the spatial location; Represents a given set of available remote sensing images The posterior probability of the label changing at position u; Represents a logarithmic function; This represents the spatial prior probability of the changed label.

[0123] Spatial priors are implemented using conditional random fields, Markov random fields, graph cuts, mean field inference, total variation regularization, morphological consistency constraints, or graph neural networks. Spatial constraints utilize the change posterior probability and latent variable uncertainty to jointly determine the smoothing strength: maintaining sensitivity to low-uncertainty, high-confidence change boundaries; and strengthening suppression of high-uncertainty, isolated noise points. Spatial prior consistency optimization uses the maximization of the change label posterior objective as the output condition to obtain spatially consistent change image segmentation results.

[0124] Spatial priors can also employ level sets, morphological opening and closing operations, superpixel consistency, plot vector constraints, road or building outline constraints, graph neural network message passing, or object-level post-processing. Building change processing maintains straight boundaries and object connectivity; flood change processing maintains water body connectivity; and deforestation change processing maintains patch integrity.

[0125] The final output includes a change probability map, a binary change image segmentation map, a prediction uncertainty map, an optical modal reliability map, a SAR modal reliability map, and optional change object boundary maps and confidence level maps. The change image segmentation results form a comprehensive representation of change probability, change category, confidence level, and modal reliability source. The change object boundary map is formed from the boundaries of continuous change objects in the spatially consistent change image segmentation results, and the confidence level map is formed by the posterior probability of change and the prediction uncertainty. The change probability map is formed from the posterior probability of change at each spatial location, the binary change image segmentation map is formed from the spatially consistent change image segmentation results, and the prediction uncertainty map is formed by... Geographic coordinate information is used to reconstruct the output results into the spatial coordinate system of the target area.

[0126] The model in this embodiment adopts the following training strategy: The training samples include dual-temporal multimodal remote sensing images and pixel-level change labels Y. The total training loss function is:

[0127] in, Indicates the total training loss; The variable image segmentation supervision loss is composed of binary cross-entropy loss. This represents the KL constraint between the latent variable posterior and the latent variable prior, specifically calculated for each phase fusion posterior. with prior The sum of KL divergences; This represents the consistency constraint between latent variables of different modes in the same time phase. The consistency loss for that time phase is calculated only if both optical and SAR modes exist simultaneously. It is defined as follows: ,in A time phase set is defined as one that simultaneously possesses both optical and SAR modes. If a time phase has only a single mode, then that time phase is not included. Loss calculation; The spatial smoothing loss is represented by the total variation loss using a probability plot of change. This represents the confidence level calibration loss, which is calculated using negative log-likelihood. The weights for each loss term are set as follows: , , , .

[0128] Training was performed using the Adam optimizer with an initial learning rate of 1×10⁻⁶. -4 The batch size is 16, and the training run is 100 epochs. If the validation set loss does not decrease for five consecutive epochs after each training epoch, the learning rate is reduced to 0.5 times its original value. During training, the optical latent variable encoder is updated according to the stated training objective. SAR latent variable encoder The module includes a change discrimination module g and an out-of-domain detector; when the modal reliability calculation module and the spatial prior optimization module are set to a structure with learnable parameters, the corresponding module parameters are updated synchronously.

[0129] Example 2: The difference between this example and Example 1 is that any time phase has only optical remote sensing images or only SAR remote sensing images.

[0130] The data acquisition module reads available modal remote sensing images, geographic coordinate information, and corresponding quality information. The preprocessing module performs the same preprocessing as in Example 1 on the available modal remote sensing images and forms image patches. The available modal remote sensing images are input into the corresponding latent variable encoder to obtain the posterior distribution parameters of the available modal latent variables. Based on the posterior distribution parameters of the available modal latent variables, the quality mask, and out-of-domain anomaly information, the comprehensive uncertainty and modal reliability parameters are calculated.

[0131] The temporal phase of optical remote sensing imagery alone yields the posterior distribution of fused latent variables based on prior latent variables, posterior distributions of optical modal latent variables, and optical modal reliability parameters:

[0132] in, Indicates the posterior distribution of the fused latent variables; This represents the prior distribution of the latent variable at time t; This represents the posterior distribution of the latent variables of the optical modes; The modal reliability parameter represents the optical mode at time t. It indicates a direct proportional relationship.

[0133] The temporal phase of SAR remote sensing images alone yields the fused latent variable posterior distribution based on the prior latent variables, the posterior distribution of SAR modal latent variables, and the SAR modal reliability parameters:

[0134] in, Indicates the posterior distribution of the fused latent variables; This represents the prior distribution of the latent variable at time t; Indicates the posterior distribution of SAR modal latent variables; The modal reliability parameter represents the SAR mode at time t. It indicates a direct proportional relationship.

[0135] When setting up cross-modal completion branches, the posterior distribution parameters of the latent variables of the missing modes are predicted from the available modes, and then fused together with the prior latent variables and the posterior distributions of the latent variables of the available modes.

[0136] After obtaining the posterior distributions of the two temporal fusion latent variables, the change probability map, binary change image segmentation map, prediction uncertainty map, and available mode reliability map are output according to the change posterior probability calculation and spatial prior consistency optimization process in Example 1. Missing modes are not filled with zero or mean values, nor are they included in the available mode set of the corresponding temporal phase.

[0137] In practical implementation, during flood inundation extraction scenarios, post-disaster optical remote sensing images may be obscured by clouds and rain. Optical quality masks increase the quality uncertainty of corresponding areas and reduce optical modal reliability parameters. SAR remote sensing images can reflect changes in water body specular reflection and scattering. The posterior distribution of SAR modal latent variables participates in fusion according to the corresponding accuracy matrix and SAR modal reliability parameters. The posterior distribution of the fusion latent variables from the two time phases is used to calculate the posterior probability of change. Spatial prior optimization maintains the connectivity of water body change areas. The result output module outputs a change probability map, a prediction uncertainty map, and a modal reliability map to distinguish between high-confidence flooded areas and areas requiring supplementary remote sensing images.

[0138] In urban construction monitoring scenarios, optical remote sensing imagery provides building texture, boundary morphology, and semantic information, while SAR remote sensing imagery provides scattering information about building structures. Geometric overlay or shadows near building edges cause changes in local modal uncertainty, and modal reliability parameters and accuracy matrices jointly adjust the fusion contribution. Spatial prior optimization utilizes building object adjacency relationships, edge intensity, or plot boundary relationships to maintain the continuity of changing objects, and the results output module outputs vector patches or a list of tasks for manual verification.

[0139] In scenarios involving deforestation identification, agricultural monitoring, and disaster damage assessment, there may be seasonal differences, phenological differences, sensor differences, or missing remote sensing images between two time phases. Posterior inference of latent variables for the true land surface state is used to separate the differences in the observed layer from the changes in the true land surface state. Missing modal processing uses the posterior distribution of available modal latent variables and the prior latent variables to form a fused posterior distribution of latent variables. Prediction uncertainty maps and modal reliability maps are used to determine the scope of manual verification.

[0140] In some preferred embodiments, in addition to simultaneously inputting four remote sensing images, the input of remote sensing images can also include any available combination of optical or SAR modes. For example, the first time phase may be an optical remote sensing image and the second time phase may be a SAR remote sensing image; the first time phase may contain both optical and SAR remote sensing images, while the second time phase may contain only SAR remote sensing images; or, in a multi-time phase scenario, a sequence of optical or SAR remote sensing images from three or more time phases may be input. For multi-time phase sequences, the bi-time phase posterior comparison is extended to the detection of temporal latent variable change points.

[0141] In terms of preprocessing, geometric registration can be achieved using traditional control point matching, mutual information registration, deep feature matching, or joint registration networks; SAR speckle suppression can employ Lee filtering, Frost filtering, nonlocal mean filtering, deep denoising networks, or edge-preserving statistical filtering; optical cloud and fog detection can utilize threshold rules, cloud mask products, or deep cloud detection networks. Changing the preprocessing method does not alter the subsequent latent variable Bayesian fusion data stream.

[0142] Regarding latent variable encoders, optical and SAR latent variable encoders can employ convolutional neural networks, U-Net encoders, ResNet, DenseNet, Swing Transformer, Vision Transformer, multi-scale feature pyramids, lightweight MobileNet structures, hybrid CNN-Transformer structures, or remote sensing base model adapters. The two encoders can be completely non-shared, partially shared, or share a projection head at a higher level. Lightweight encoders can be used for edge deployment scenarios, while encoders with larger parameter scales can be used for high-precision offline processing.

[0143] Regarding the distribution of latent variables, Gaussian distribution and diagonal covariance are preferred, but low-rank covariance, block diagonal covariance, Gaussian mixture distribution, Student-t distribution, Laplace distribution, normalized flow, or diffuse latent variable representation can also be used. When SAR noise exhibits heavy-tailed characteristics, the Student-t distribution is used to express the heavy-tailed posterior; when land cover categories have a multi-peak structure, the Gaussian mixture distribution is used to express multiple latent states; when a nonlinear posterior representation is required, the normalized flow is used to transform from a simple distribution to a complex posterior.

[0144] For uncertainty estimation, latent variable covariance, prediction entropy, mutual information, model ensemble, Monte Carlo Dropout, deep evidence learning, test-time augmentation, multi-head prediction discrepancy, temperature calibration, reconstruction error, out-of-domain detection score, cloud mask, shadow mask, SAR noise index, or a combination of the above methods can be used. Modal reliability functions can also be implemented using normalized inverse variance, learnable gating, Bayesian model averaging, evidence fusion, or a hybrid approach of rules and learning.

[0145] For Bayesian fusion, product expert and precision matrix weighted fusion are preferred, but hybrid expert, Bayesian model averaging, evidence theory fusion, covariance cross fusion, variational inference fusion, or attention-based approximate Bayesian fusion can also be used. The fusion process adjusts the contribution using the posterior uncertainty or modal reliability parameter of each modality's latent variable and outputs the posterior distribution of the fused latent variable.

[0146] For calculating the probability of change, symmetric KL divergence, Wasserstein distance, Mahalanobis distance, Bhattacharyya distance, Hellinger distance, energy distance, cosine difference, mean difference, covariance difference, or a combination of the above methods can be used. The change discrimination module can be a threshold function, logistic regression, multilayer perceptron, convolutional decoder, Transformer decoder, graph neural network, or multi-scale context network.

[0147] For spatial priors, Markov random fields, conditional random fields, graph cuts, mean field inference, total variation regularization, level sets, morphological opening and closing operations, superpixel consistency, plot vector constraints, road or building outline constraints, graph neural network message passing, or object-level post-processing can be employed. Building change processing maintains straight-line boundaries and object connectivity; flood change processing maintains water body connectivity; and deforestation processing maintains patch integrity.

[0148] Regarding training methods, fully supervised training, semi-supervised training, weakly supervised training, unsupervised training, active learning, transfer learning, or federated learning can be employed. With labeled data, training samples include dual-temporal multimodal remote sensing images and change labels Y. Training objectives include one or more of the following: change image segmentation supervision loss, KL constraint between latent variable posterior and prior, consistency constraint between latent variables of different modalities in the same temporal phase, spatial smoothing or object consistency loss, and confidence calibration loss.

[0149] The output can be GeoTIFF, vector spectra, tile services, raster probability maps, confidence level maps, or a list of manual verification tasks. The system is deployed on cloud-based remote sensing processing platforms, local natural resource monitoring platforms, emergency disaster assessment platforms, edge computing devices, or mobile patrol systems.

[0150] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0151] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation, characterized by: include: Acquire remote sensing image data of the target area in the first and second time phases, wherein the remote sensing image data includes at least one of optical remote sensing images and SAR remote sensing images; The remote sensing image data is preprocessed to form image blocks containing quality masks; the optical remote sensing image and SAR remote sensing image are used as observations of latent variables of the real surface state under different imaging mechanisms, and the posterior distribution parameters of the latent variables are obtained by optical latent variable encoder and SAR latent variable encoder respectively. The comprehensive uncertainty and modal reliability parameters are calculated based on the latent variable posterior distribution parameters, the quality mask, and the out-of-domain anomaly information. Within each time phase, Bayesian fusion is performed based on the prior latent variables, the posterior latent variable distributions of each available mode, the precision matrix, and the mode reliability parameters to obtain the fused posterior latent variable distributions for the two time phases. The posterior probability of change is calculated based on the probability distance, mean difference, covariance difference, fusion uncertainty, and spatial context between the posterior distributions of the fused latent variables of the two time phases. Based on spatial adjacency relationships, the spatial prior consistency optimization is performed on the posterior probability of the change, and the output includes a change probability map, a binary change image segmentation map, a prediction uncertainty map, an optical mode reliability map, and a SAR mode reliability map, as well as optional change object boundary maps and confidence level maps.

2. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 1, characterized in that: The preprocessing includes one or more of the following: coordinate system one, geometric fine registration, resolution resampling, cropping and segmentation, boundary overlap, radiometric normalization, SAR radiometric calibration, terrain correction, logarithmic transformation, speckle filtering, optical atmospheric correction, cloud and fog detection, shadow detection, and invalid value mask generation; the quality mask is used to represent optical clouds and fog, shadows, SAR anomalous noise, invalid values, or edge-filled values.

3. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 2, characterized in that: The optical latent variable encoder and the SAR latent variable encoder adopt a non-shared weight structure, or share some parameters in the high-level semantic part; the optical latent variable encoder and the SAR latent variable encoder output the latent variable mean and the latent variable covariance, respectively. The posterior distribution of the latent variable is a Gaussian distribution, and the latent variable covariance is a diagonal covariance. The latent variable mean represents the estimate of the actual surface state of the corresponding mode, and the latent variable covariance represents the uncertainty of the estimate of the corresponding mode.

4. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 3, characterized in that: The combined uncertainty of the m-th mode at position u in phase t is calculated according to the following formula: in, This represents the overall uncertainty of the m-th mode at position u in phase t; This represents posterior uncertainty, obtained from the covariance trace of the latent variables; This represents the uncertainty in quality, obtained from a quality mask. This indicates uncertainty regarding out-of-domain anomalies, derived from the out-of-domain detection score. , and represents the preset or learnable coefficients; t represents the phase number; m represents the mode number; u represents the spatial position; the modal reliability parameters are calculated according to the following formula: in, This represents the modal reliability parameter of the m-th mode at position u in phase t. This represents an exponential function; 'a' represents temperature or scaling parameter. This represents the overall uncertainty of the m-th mode at position u in phase t; This represents the set of available modes at time t; k represents the mode index in the set of available modes. The expression represents the comprehensive uncertainty of the k-th mode at position u in phase t; t represents the phase number; m represents the mode number; and u represents the spatial position.

5. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 4, characterized in that: The posterior distribution of the fusion latent variables within each time phase is calculated according to the following formula: in, Indicates the posterior distribution of the fused latent variables; Represents the latent variable at time t; This represents the optical remote sensing image at time t. This represents the SAR remote sensing image at time t. Indicates a direct proportional relationship; This represents the prior distribution of the latent variable at time t; The set of available modes for time phase t is represented by m, where m represents the mode number. Denotes the posterior distribution of the latent variable in the m-th mode; This represents the remote sensing image of the m-th mode at time t. This represents the modal reliability parameter of the m-th mode in phase t; t represents the phase number; the covariance of the fusion latent variables is calculated according to the following formula: in, This represents the covariance of the fusion latent variables at time t. The superscript -1 indicates the prior covariance of the latent variable; The set of available modes for time phase t is represented by m, where m represents the mode number. The modal reliability parameter represents the m-th mode in phase t. The latent variable covariance of the m-th mode in phase t represents the latent variable covariance; t represents the phase number; the mean of the fused latent variable is calculated according to the following formula: in, This represents the mean of the fusion latent variables at time t; This represents the covariance of the fusion latent variables at time t. This represents the prior covariance of latent variables; This represents the prior mean of the latent variable; The set of available modes for time phase t is represented by m, where m represents the mode number. The modal reliability parameter represents the m-th mode in phase t. This represents the latent variable covariance of the m-th mode at time t. This represents the mean of the latent variables in the m-th mode at time t; t represents the time phase number.

6. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 5, characterized in that: The symmetric KL divergence of the posterior distributions of the two temporally merged latent variables is calculated according to the following formula: in, This represents the symmetric KL divergence at position u; KL represents the Kullback-Leibler divergence. This represents the posterior distribution of the fusion latent variables at the first time phase position u; This represents the posterior distribution of the fusion latent variables at the second time phase position u; Indicates the KL divergence direction separator; This represents the latent variable at the first phase position u; This represents the latent variable at the second time phase position u; u represents the spatial position; the Mahalanobis distance is calculated according to the following formula: in, Indicates the Mahalanobis distance at position u; This represents the mean of the fusion latent variables at position u in the first time phase; This represents the mean of the fusion latent variables at position u in the second time phase; the superscript T indicates matrix transpose. This represents the covariance of the fusion latent variables at the first time phase position u; Let u represent the covariance of the fusion latent variable at the second time phase position u; the superscript -1 indicates matrix inversion; u represents the spatial position; the posterior probability of the change is calculated according to the following formula: in, Represents a given set of available remote sensing images The posterior probability of the change at position u; This represents the variable that changes at position u; The mapping function represents the probability of output change; g represents the change discrimination module; This represents the probability distance feature at position u; This represents the difference vector of the means of the fusion latent variables at position u. ; This represents the diagonal element difference vector of the covariance of the fusion latent variables at position u. ; This represents the fusion uncertainty at position u. ; The local spatial context feature vector at position u is obtained by concatenating the channel vectors of the bottom feature maps of the optical latent variable encoder and the SAR latent variable encoder; u represents the spatial position.

7. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 6, characterized in that: The spatial prior consistency optimization includes constructing a pixel map, superpixel map, plot map, or object map, using spatial location or object as nodes and spatial adjacency as edges. Edge weights are determined based on spatial distance, optical spectral similarity, SAR scattering similarity, latent variable similarity, edge intensity, or plot boundary relationship. Markov random fields, conditional random fields, graph cuts, mean field inference, total variation regularization, morphological consistency constraints, or graph neural networks are used to obtain spatially consistent variation image segmentation results. The maximum value of the posterior target of the variation label is used as the output condition for spatial prior consistency optimization.

8. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 7, characterized in that: The training samples include dual-temporal multimodal remote sensing images and change labels. The training objectives include one or more of the following: change image segmentation supervision loss, KL constraint between latent variable posterior and latent variable prior, consistency constraint between latent variables of different modalities in the same temporal phase, spatial smoothing or object consistency loss, and confidence calibration loss. During the training process, the optical latent variable encoder, SAR latent variable encoder, change discrimination module, and out-of-domain detector are updated according to the training objectives. When the modal reliability calculation module and spatial prior optimization module are set to have learnable parameter structures, the corresponding module parameters are updated synchronously.

9. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 5, characterized in that: When any time phase lacks an optical mode or SAR mode, the fused latent variable posterior distribution is obtained using the latent variable prior of the corresponding time phase, the latent variable posterior distribution of the available modes, and the mode reliability parameters of the available modes. When setting a cross-mode completion branch, the latent variable posterior distribution parameters of the missing modes are predicted by the available modes and then fused together with the latent variable prior and the latent variable posterior distribution of the available modes.

10. The latent variable Bayesian fusion method for optical-SAR heterogeneous variation image segmentation according to claim 7, characterized in that: The change probability map, binary change image segmentation map, prediction uncertainty map, optical modal reliability map, and SAR modal reliability map are output as geographic raster files, vector speckles, tile services, raster probability maps, confidence level maps, or manual verification task lists for use in urban construction monitoring, flood inundation extraction, deforestation identification, disaster damage assessment, agricultural monitoring, or land and resources supervision.

Images