Dual temporal remote sensing image change detection method based on split everything model

Abstract

The application is a two-phase remote sensing image change detection method based on a split-all model, comprising: performing radiation correction, geometric registration, blocking and scale normalization preprocessing on the two-phase remote sensing image; constructing a double-branch feature extraction architecture, wherein the first branch extracts multi-scale general semantic features by freezing the SAM model parameters, and the second branch extracts temporal difference features; inserting a learnable adapter layer after each Transformer layer of the SAM to realize task adaptation; a aggregation module unifies the feature scale, fuses heterogeneous features, and inputs a nested U-Net decoder to generate an initial change detection map; combining the SAM object to realize result refinement; and solving the technical problems of insufficient generalization ability, high small change missing detection rate and weak semantic discrimination ability in unsupervised scenes by averaging and fusing the blocking results in the overlapping area, which has high precision, low label dependence and strong cross-scene adaptability, and has practical application value.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and computer vision technology, and particularly relates to a method for detecting changes in dual-temporal remote sensing images based on the Segmentation All Model (SAM). Background Technology

[0002] Dual-temporal remote sensing image change detection is a core remote sensing information extraction technology that analyzes remote sensing images acquired at different times in the same geographic area to identify changes in land cover, land cover types, or spatial states. It is a key supporting link for applications such as dynamic land use monitoring, geospatial database updates, natural disaster assessment, ecological environment change analysis, and urban expansion supervision.

[0003] Traditional change detection methods mainly rely on a combination of manually designed features (such as NDVI and GLCM texture statistics) and shallow machine learning models (such as support vector machines and random forests) to achieve change recognition. In recent years, deep neural networks, represented by U-Net, ResNet, and Swin Transformer, have been widely used for end-to-end feature learning and change map generation, significantly improving detection accuracy and robustness. Meanwhile, the Segmentation All Model (SAM) has demonstrated strong semantic modeling potential for complex ground structures due to its zero-shot segmentation capability obtained through pre-training on ultra-large-scale image data. Multimodal large models such as Lava have achieved high-fidelity semantic alignment between image regions and natural language descriptions, providing a new interpretable path for change discrimination.

[0004] However, existing technologies still have the following problems: the semantic features extracted by the general visual large model SAM are weakly correlated with the target of remote sensing change detection tasks, and the transfer and adaptation capabilities are insufficient; the general semantic features extracted by the two branches have significant heterogeneity with the temporal difference features, resulting in low efficiency of multi-scale fusion and easy suppression of key change cues; in supervised mode, the boundaries of small-area change regions are blurred and the false detection rate is high; in unsupervised mode, there is a lack of semantic-level discrimination mechanism between real changes and false changes (such as pixel-level perturbations caused by illumination differences and imaging angle shifts), resulting in a high false detection rate.

[0005] The aforementioned problems collectively lead to unstable generalization performance of existing methods across sensors, seasons, and regions, making it difficult to meet the actual business needs of high precision, low labeling dependence, and strong adaptability. Therefore, improvement and innovation are imperative. Summary of the Invention

[0006] This invention provides a method for detecting changes in dual-temporal remote sensing images based on the Segmentation All Model (SAM) to solve at least one of the above-mentioned technical problems.

[0007] In a first aspect, the present invention provides a method for detecting changes in dual-temporal remote sensing images based on the Segmentation All Model (SAM), comprising the following steps: (1) Preprocessing: Radiometric correction, geometric registration, block division and scale normalization are performed on the dual-temporal remote sensing images in sequence to ensure the consistency of input data; (2) Dual-branch feature extraction: A dual-branch architecture is adopted. The first branch freezes all parameters of the SAM model and extracts multi-scale general semantic features; the second branch uses Swin Transformer V2 as the backbone network to extract the temporal difference features of dual-temporal images. (3) Feature adaptation: A learnable adapter layer is inserted after each Transformer layer of the SAM model to adapt the general semantic features extracted by SAM to the task, thereby realizing the transfer of general features to change detection-specific features; (4) Feature fusion: The adapted SAM features and temporal difference features are unified at the same scale through the multi-scale feature aggregation module, and the heterogeneous feature fusion is achieved by using the channel attention weighted fusion strategy; (5) Initial detection: Input the fused features into the nested U-Net decoder head to generate an initial change detection binary map; (6) Result refinement: In supervised mode, bounding box prompts are generated through 8-neighbor connected component analysis and input into SAM for fine-tuning; in unsupervised mode, SAM is used for object-level segmentation, combined with multimodal large model to generate text descriptions and calculate semantic similarity, and then fused with the initial detection results for refinement. (7) Output of results: The block detection results are spliced ​​together to output the final high-precision change detection results.

[0008] This invention presents a scientifically sound and easy-to-operate method that performs radiometric correction, geometric registration, block segmentation, and scale normalization preprocessing on dual-temporal remote sensing images. A dual-branch feature extraction architecture is constructed, where the first branch freezes SAM model parameters to extract multi-scale general semantic features, and the second branch uses Swing Transformer V2 to extract temporal difference features. Learnable adapter layers are inserted after each Transformer layer of the SAM to achieve task adaptation. A multi-scale feature aggregation module unifies the feature scale, and a channel attention-weighted fusion strategy is used to fuse heterogeneous features. The fused features are input into a nested U-Net decoder to generate an initial change detection map. In supervised mode, bounding box prompts are generated based on connected component analysis and SAM is used for refinement. In unsupervised mode, the results are refined by combining SAM object segmentation with Lava-13B / MPNet semantic similarity calculation. Finally, the block results are stitched together using an overlapping region averaging fusion strategy. This effectively solves technical problems such as insufficient generalization ability, high false negative rate for small changes, and weak semantic discrimination ability in unsupervised scenes. It combines high accuracy, low annotation dependence, and strong cross-scene adaptability, making it valuable for practical development and application. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart illustrating a method for detecting changes in dual-temporal remote sensing images based on the Segmentation All Model (SAM) provided by the present invention. Detailed Implementation

[0010] The present invention will now be described in further detail with reference to embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit the invention.

[0011] Various embodiments of this disclosure will be described more fully in the following detailed description.

[0012] It should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all modifications, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of this disclosure.

[0013] The following is a brief introduction to some of the terms and techniques involved in the embodiments of the present invention: This embodiment provides a method for detecting changes in dual-temporal remote sensing images based on the Segmentation All Model (SAM). The method includes the following steps: Step 1: Preprocessing: Perform radiometric correction, geometric registration, block segmentation, and scale normalization on the dual-temporal remote sensing images in sequence to ensure the consistency of the input data; Among them, radiometric correction refers to eliminating radiation distortion caused by factors such as atmospheric scattering, absorption, and nonlinear sensor response, which can be achieved using the FLAASH algorithm; geometric registration refers to aligning two images in a spatial coordinate system so that the corresponding ground object pixels are in the same position, which can be achieved using the SIFT algorithm, with a registration error of no more than 0.5 pixels; tiling refers to dividing the entire image into non-overlapping sub-regions of a fixed size, which facilitates batch processing of models and memory management, and the tiling size can be 512×512 pixels; scale normalization refers to mapping pixel values ​​to a uniform numerical range to eliminate the influence of sensor response differences and lighting conditions, and can be normalized to the range [0,1].

[0014] This embodiment can perform radiometric correction on the image using the FLAASH algorithm to eliminate radiometric bias introduced by atmospheric effects; it can also perform geometric registration using SIFT feature point matching and affine transformation models to ensure strict alignment of the spatial coordinates of the two-temporal images; further, this embodiment can divide the registered image into 512×512 pixel blocks and perform min-max normalization on each block to ensure that the pixel values ​​fall within the [0,1] interval. This embodiment obtains a sequence of two-temporal image blocks that is radiometrically and geometrically consistent, scaled uniformly, and meets the input requirements of the adaptation model based on any of the above methods.

[0015] For example, this embodiment can take a pair of pre-flood / post-flood remote sensing images (0.8-meter resolution) acquired by Gaofen-2 satellite as input. First, the FLAASH module is used to correct the influence of atmospheric radiation. Then, SIFT feature points are extracted and the optimal affine transformation parameters are calculated. The image of the later time phase is strictly registered to the coordinate system of the previous time phase, and the root mean square error of the registration residual is 0.43 pixels. Then, the registered image is divided into several blocks along the row and column directions with a step size of 512 pixels. Min-max normalization is performed on each block, and floating-point image blocks in the range of [0,1] are output as the unified input for subsequent steps.

[0016] Optionally, in the preprocessing step, radiometric correction adopts the FLAASH algorithm, geometric registration adopts the SIFT algorithm with a registration error ≤ 0.5 pixels, the image block size is 512×512 pixels, and the scale is normalized to the range [0,1].

[0017] FLAASH (Fast Line-of-sight Atmospheric Analysis of Spectral Hypercubes) is a physical-driven radiometric correction algorithm based on the MODTRAN radiative transfer model. FLAASH uses atmospheric parameters (such as visibility, water vapor content, and ozone column concentration) to construct band-by-band atmospheric transmittance and path radiative functions, and obtains surface reflectance through iterative inversion. In this embodiment, FLAASH converts the original DN value image into a physically meaningful surface reflectance image, making the two-temporal images comparable under the same illumination and atmospheric conditions, and providing a radiometrically consistent input basis for subsequent multi-scale feature extraction.

[0018] SIFT (Scale-Invariant Feature Transform) is a keypoint extraction and matching algorithm based on Difference of Gaussian (DoG) extremum detection and orientation histogram description. SIFT constructs an image pyramid, detects scale space extrema, assigns principal orientations, generates 128-dimensional descriptors, and performs initial matching based on the Euclidean distance ratio criterion. In this embodiment, SIFT registration ensures that the same geographic entity in dual-temporal images has a deviation of no more than 0.5 pixels in the pixel coordinate system, thereby ensuring the accuracy of the spatial correspondence in subsequent feature-level difference calculations and avoiding the introduction of spurious change responses due to misalignment.

[0019] The 512×512 pixel block strategy balances GPU memory usage and receptive field coverage. This ensures that each block can accommodate sufficient contextual information to support the multi-scale modeling capabilities of SAM and Swin Transformer V2, while avoiding exceeding the GPU memory capacity and causing training or inference interruptions. In this embodiment, the block size directly adapts to the input resolution constraints of the SAM-ViT-L image encoder (maximum support 1024×1024, 512×512 recommended), and forms an integer multiple relationship with the hierarchical shrinkage / expansion step size of the nested U-Net decoder, ensuring unambiguous alignment of feature map spatial dimensions.

[0020] The scaling normalization to the [0,1] range is a standardized preprocessing operation that eliminates the differences in dynamic range between different sensors, unifies the numerical dimensions of each channel, and improves the stability of gradient propagation. In this embodiment, the normalization operation is applied to the segmented image to ensure that the input data distribution of the SAM and Swin Transformer V2 backbone networks is within the numerical range adapted by the model weight initialization, thus avoiding feature extraction distortion due to input overflow or saturation.

[0021] This embodiment completes radiometric correction based on the FLAASH atmospheric parameter modeling and surface reflectance inversion process; it completes geometric alignment based on the SIFT keypoint detection, descriptor generation, and RANSAC robust registration process; further, it completes data normalization based on a block-based and scale-mapping process jointly executed by fixed-size cropping and global extremum normalization. This embodiment obtains a bi-temporal image block sequence with high radiometric consistency, sub-pixel spatial alignment accuracy, uniform input format, and compatibility with the model architecture based on any of the above methods.

[0022] For example, this embodiment uses WorldView-3 dual-temporal images (0.5-meter panchromatic + 2-meter multispectral fusion) acquired in June 2022 and August 2023 of a certain region as input. First, the built-in FLAASH module of ENVI 5.6 is called, and the measured atmospheric parameters (visibility 23 km, water vapor content 1.8 g / cm², ozone 0.3 cm-atm) are input to complete radiometric correction and obtain two surface reflectance images. Then, the SIFT interface in OpenCV 4.8.0 is used to extract and match key points. After fitting the affine transformation matrix with RANSAC, the mean residual value of the control points is verified to be 0.37 pixels, which meets the requirement of ≤0.5 pixels. Then, the images are cut into 512×512 pixel non-overlapping segments, and the edges are padded with zeros. Then, each image block is normalized to [0,1] according to the channel global extremum. Finally, 64 pairs of dual-temporal image blocks are output for subsequent dual-branch feature extraction module processing.

[0023] Step 2: Dual-branch feature extraction: A dual-branch architecture is adopted. The first branch freezes all parameters of the SAM model and extracts multi-scale general semantic features; the second branch uses Swing Transformer V2 as the backbone network to extract temporal difference features of dual-temporal images. SAM (Segment Anything Model) is a fundamental visual model whose core capability is to perform zero-shot image segmentation with arbitrary cues (points, boxes, masks), and can extract multi-scale general features covering texture, structure, and semantics. Parameter freezing refers to not updating the SAM model weights during training, using it only as a fixed feature extractor to preserve its strong generalization ability obtained through pre-training on large-scale natural images. Swin Transformer V2 is a hierarchical visual Transformer backbone network with a sliding window attention mechanism and scaling-aware design, capable of modeling long-range dependencies and temporally sensitive patterns. Temporal difference features refer to latent space representations that characterize pixel-level or region-level difference responses between two temporal images, and they have the ability to suppress pseudo-changes such as illumination changes, cloud occlusion, and viewpoint shifts.

[0024] This embodiment can extract three-level feature maps (C1 (shallow texture), C2 (mid-level structure), and C3 (deep semantics) from the preprocessed dual-temporal images using the SAM-ViT-L model with frozen parameters. The spatial resolution of each layer is 1 / 4, 1 / 8, and 1 / 16 of the original image, respectively. Alternatively, this embodiment can stack the dual-temporal images into a 2-channel input using the Swin Transformer V2 backbone network, and output a 1 / 4-scale temporal difference feature map after four-stage downsampling. Further, this embodiment can input the two temporal phases of the same image block into a shared-weight Swin Transformer V2 branch, and extract explicit temporal response features through layer-by-layer differencing operations. This embodiment obtains a dual-path heterogeneous feature set with both general semantic representation capabilities and change-sensitive discrimination capabilities based on any of the above methods.

[0025] For example, in this embodiment, a 512×512 image block output above can be input into the SAM-ViT-L model with frozen parameters to obtain the outputs of the ViT encoder at layers 4, 8, and 12, corresponding to three sets of feature tensors: C1 (128×128×768), C2 (64×64×768), and C3 (32×32×768). At the same time, the two temporal images of the same image block are stitched together along the channel dimension to form a 512×512×2 tensor, which is input into Swin Transformer V2 (window size=8, depths=[2,2,18,2]). After four stages of downsampling, a 128×128×1024 temporal difference feature map is output as a contrast feature source that is aligned with C1 in both spatial and channel dimensions.

[0026] Step 3: Feature Adaptation: A learnable adapter layer is inserted after each Transformer layer of the SAM model to adapt the general semantic features extracted by SAM to the task, thereby transferring the general features to features specific to change detection. The learnable adapter layer is a lightweight plug-in module used to inject task-specific prior knowledge without disrupting the backbone network structure. Its bottleneck structure can refer to a compact design that includes three sub-modules: dimensionality reduction, nonlinear activation, and dimensionality increase, which can significantly reduce the number of trainable parameters. The residual connection can refer to adding the adapter output to the original SAM output to ensure gradient pass-through and feature fidelity.

[0027] This embodiment can compress the output features of each Transformer layer of SAM from 768 dimensions to 256 dimensions using a 1×1 convolutional dimensionality reduction layer, activate them with ReLU, and then increase the dimensionality back to 768 dimensions using a 1×1 convolution, adding the residual connection to the original output. Alternatively, this embodiment can initialize the adapter layer weights according to a He normal distribution, training only the adapter layer and the nested U-Net decoder head, while keeping the other parameters frozen. Further, this embodiment can deploy the adapter layer after each Transformer layer in any version of SAM-ViT-B, SAM-ViT-L, or SAM-ViT-H, with the adapter input / output dimensions strictly consistent with the corresponding layer feature dimensions. This embodiment obtains a dedicated semantic feature representation that is highly suitable for change detection tasks and has a controllable number of parameters (≤5% of the total SAM parameters) based on any of the above methods.

[0028] For example, in this embodiment, after the 12 Transformer layers of SAM-ViT-L, adapter modules are inserted layer by layer. The first-layer adapter receives the output of the first ViT layer (512×512×768), and after undergoing Conv1×1(768→256)→ReLU→Conv1×1(256→768) transformation, it is added to the original output, and the output dimension is still 512×512×768. Subsequent layers follow the same pattern, and finally output three sets of adaptation features: C1_adapt (128×128×768), C2_adapt (64×64×768), and C3_adapt (32×32×768), which are used for subsequent multi-scale aggregation.

[0029] Optionally, the learnable adapter layer adopts a bottleneck structure design, including... 1 × 1 Convolutional dimensionality reduction layer, ReLU activation layer, 1 × 1The convolutional upscaling layer, and the output of the corresponding Transformer layer of SAM is added through residual connections, can train no more than 5% of the total parameters of SAM.

[0030] The learnable adapter layer (bottleneck adapter layer) is a lightweight task adaptation module used to introduce fine-tuning signals specific to the change detection task without compromising the semantic representation capabilities of the SAM pre-trained model. This technical feature, within its technical field, can refer to an embedded parameter-efficient fine-tuning structure, whose core lies in achieving controllable correction of intermediate features of a large model with extremely low parameter overhead. In this embodiment, the role of this technical feature is to receive the multi-scale general semantic features (e.g., C1 / C2 / C3) output from each Transformer layer of the SAM and map them to a dedicated feature representation for the change detection task of dual-temporal remote sensing images, while ensuring that the original feature information is not covered or distorted, providing semantically consistent and task-aligned input for subsequent feature fusion.

[0031] This embodiment can be based on 1 × 1 The convolutional dimensionality reduction layer compresses the channel dimension of the SAM Transformer layer output, then introduces nonlinear modeling capability through ReLU activation, and finally... 1 × 1 The convolutional upscaling layer restores the original number of channels, thus completing a lightweight learning of the task prior. Alternatively, this embodiment can construct a nonlinear mapping function based on a three-level cascaded structure of dimensionality reduction-activation-upscaling, ensuring that the adaptation increment only applies to the feature subspace strongly correlated with the change-sensitive region. Furthermore, this embodiment can add the adapter output and the original SAM output through a residual connection, making the final feature representation a general semantic basis plus task-specific perturbations, ensuring both zero-shot segmentation capability and task accuracy are preserved. This embodiment obtains the adapted features based on any of the above methods, maintaining the inherent cross-domain generalization capability of SAM while significantly improving its semantic discrimination accuracy in remote sensing change detection tasks.

[0032] For example, this embodiment can take SAM-ViT-L as an example, where each Transformer layer outputs a feature dimension of 1024, and the adapter layer first passes through... 1 × 1 Convolution reduces the number of channels from 1024 to 256, followed by ReLU activation, and then another... 1 × 1Convolution increases the number of channels back to 1024; the total number of trainable parameters introduced in this process is 1024×256+256×1024=524288, while the total number of parameters in SAM-ViT-L is about 900M, accounting for 0.058%, which meets the constraint of not exceeding 5%; the output of this adapter is added to the original output of SAM and then used as the final feature of this layer to be fed into the next layer or feature fusion module.

[0033] Step 4: Feature Fusion: The adapted SAM features and temporal difference features are unified at the same scale through the multi-scale feature aggregation module, and the heterogeneous feature fusion is achieved by using a channel attention weighted fusion strategy. Among them, the multi-scale feature aggregation module can refer to the operation unit that upsamples, aligns and stitches feature maps with different spatial resolutions; the channel-wise attention weighted fusion strategy can refer to dynamically allocating fusion weights based on the contribution of each channel to the change discrimination, thereby improving the response intensity of key channels.

[0034] This embodiment can be implemented by upsampling C1_adapt, C2_adapt, and C3_adapt to a spatial resolution of 128×128 using bilinear interpolation, and then concatenating them along the channel dimension with the 128×128 temporal difference features to form a 128×128×(768×3+1024) pre-fusion feature tensor. Alternatively, this embodiment can improve the structure using the SE module, performing global average pooling to compress the spatial dimension of the concatenated features, then generating a 1280-dimensional channel weight vector through two fully connected layers, and finally weighting each channel after Sigmoid activation. Further, this embodiment can also add the weighted SAM-adapted features and the weighted temporal difference features proportionally to obtain the final fusion feature F_fusion = W × F_SAM_adapt + (1−W) × F_temp. This embodiment obtains a unified feature representation that balances multi-scale context and change sensitivity based on any of the above methods.

[0035] For example, in this embodiment, the outputs C1_adapt (128×128×768) and C2_adapt (64×64×768) can be upsampled to 128×128×768 via bilinear interpolation, and C3_adapt (32×32×768) can be upsampled to 128×128×768. These three are then concatenated with the 128×128×1024 temporal difference features output from the Swin branch to obtain a 128×128×3328 tensor. Global average pooling is then performed on this tensor to obtain a 3328-dimensional vector. Channel weights W are generated by FC(3328→1280)→ReLU→FC(1280→3328)→Sigmoid. The first 2304 dimensions of W are applied to the SAM adaptation features, and the last 1024 dimensions are applied to the temporal difference features. The 128×128×3328 fusion features are then output according to the weighted sum formula.

[0036] Optionally, the multi-scale feature aggregation module upsamples the C1 shallow texture features, C2 mid-level structural features, C3 deep semantic features, and temporal difference features output by SAM to 1 / 4 scale of the original image through bilinear interpolation and then stitches them along the channel dimension.

[0037] Among them, C1 shallow texture feature (C1 feature, shallow spatial texture response feature), C2 mid-layer structure feature (C2 feature, mid-layer geometric structure response feature), and C3 deep semantic feature (C3 feature, deep high-layer semantic response feature) are multi-scale feature maps output by the SAM model at different Transformer layers, with original spatial resolutions of 1 / 4, 1 / 8, and 1 / 16 of the original image, respectively; the temporal difference feature is a feature map extracted by the SwinTransformer V2 backbone network under dual-temporal image input, representing the pixel-level difference response between two images, with a spatial resolution of 1 / 4 of the original image.

[0038] The three types of features, C1, C2, and C3, are upsampled using bilinear interpolation to uniformly adjust to the same spatial resolution as the temporal difference features, i.e., 1 / 4 scale of the original image. This upsampling operation does not introduce learnable parameters; it only achieves spatial coordinate mapping and pixel value resampling through the interpolation kernel, maintaining the spatial alignment of each feature map.

[0039] After scale alignment is completed, the upsampled C1, C2, and C3 feature maps are concatenated with the temporal difference feature map along the channel dimension to form a fused feature tensor. This concatenation operation does not change the spatial structure of each feature map, but only expands the number of channels, providing heterogeneous feature input with complete hierarchical semantic coverage for subsequent channel attention weighted fusion.

[0040] This embodiment can achieve spatial scale unification of C1 / C2 / C3 features using bilinear interpolation upsampling; it can also achieve spatial alignment between C1 / C2 / C3 features and temporal difference features using bilinear interpolation upsampling; further, it can achieve semantic granularity matching of multi-scale features at a 1 / 4 scale using bilinear interpolation upsampling. This embodiment obtains a multi-scale heterogeneous feature set with a unified spatial resolution based on any of the above methods, providing a structurally consistent and semantically complementary fusion foundation for the channel attention mechanism.

[0041] For example, this embodiment can be based on the original image of 512×512 pixels. The initial size of the C1 feature map is 128×128 (i.e., 1 / 4 scale), which is directly retained; the initial size of the C2 feature map is 64×64 (i.e., 1 / 8 scale), which is upsampled to 128×128 through bilinear interpolation; the initial size of the C3 feature map is 32×32 (i.e., 1 / 16 scale), which is upsampled to 128×128 through bilinear interpolation; the temporal difference feature map output by the Swin backbone network is also 128×128; the four are stitched together along the channel dimension on a 128×128 spatial grid to form a single tensor, which is the output of the multi-scale feature aggregation module.

[0042] Optionally, the channel attention weighted fusion strategy is based on an improvement of the SE module. It compresses spatial information through global average pooling, generates channel weight vectors through two fully connected layers, and adaptively allocates fusion weights.

[0043] Among them, (1) Channel-wise Attention-based WeightedFusion Strategy can refer to a technical mechanism that dynamically adjusts the fusion weights of each feature channel based on its contribution to the change detection task. Functionally, this strategy can enhance channels that are strongly correlated with the change region in heterogeneous features and suppress channels that are strongly correlated with the background or noise. In this embodiment, the strategy is applied to the spliced ​​features output by the multi-scale feature aggregation module to generate weighted fusion features, which are used as inputs to the nested U-Net decoder head, thereby improving the sensitivity of the initial detection results to the real change region and the accuracy of boundary localization.

[0044] (2) The SE module (Squeeze-and-Excitation Module) is a general channel attention modeling paradigm in the field of computer vision. Its core idea is to recalibrate the channel feature response by explicitly modeling the dependencies between channels. The improvement of this module in this embodiment is that the global average pooling + two fully connected layers + Sigmoid process in the original SE structure is directly embedded into the channel dimension processing stage after multi-scale feature aggregation, and its input is limited to the four-way feature tensor obtained by upsampling and channel concatenation (i.e., the C1, C2, C3 adapted features and the temporal difference feature F_temp concatenated along the channel dimension), rather than the single backbone network output feature. This improvement enables the channel attention mechanism to simultaneously perceive the synergistic / conflict relationship between general semantic features and task-specific temporal difference features, avoiding weight bias caused by heterogeneous feature sources.

[0045] (3) Global Average Pooling (GAP) is a spatial dimension compression operation that performs channel-wise averaging on the input feature map in the H×W spatial dimension, and outputs a 1D statistical vector with a length equal to the number of channels. In this embodiment, GAP is applied to the concatenated feature map, with an input size of C_total×H×W (where C_total=C_c1+C_c2+C_c3+C_temp) and an output size of C_total×H×W. 1 × 1 Vector; This operation can functionally eliminate local spatial noise interference, preserve the overall response intensity of each channel across the entire image block, and provide a stable and robust statistical basis for subsequent channel weight generation.

[0046] (4) The two fully connected layers can refer to linear transformation layers connected in sequence. The first layer maps the GAP output vector to the intermediate dimension (e.g., dimensionality reduction to C_total / 16), and the second layer maps it back to the original number of channels C_total. A ReLU nonlinear activation function is inserted between the two layers. In this embodiment, the structure is used to model the high-order nonlinear dependencies between channels and identify which channel combinations are more likely to characterize the essential features of changes such as spectral jumps, texture breaks, or structural displacements across time. Its function does not depend on a specific activation function type or the value of the intermediate dimension, but it satisfies the following: the first layer has the ability to reduce dimensions to introduce bottleneck constraints, and the second layer has the ability to restore dimensions to ensure the integrity of channel granularity.

[0047] (5) Adaptive allocation of fusion weights can refer to the fact that after the final generated channel weight vector is normalized by the Sigmoid function, each element takes a value in the (0,1) interval, and the weight of each channel is independently adjustable. In this embodiment, the weight vector is multiplied by the original stitched feature map channel by channel to achieve differentiated weighting of feature channels from different sources and different abstraction levels. Its technical function is to enable the channel representing edge sharpness in the shallow layer C1 of SAM, the channel representing building outline in the middle layer C2, the channel representing land cover category in the deep layer C3, and the channel representing temporal spectral difference in the Swin branch to obtain corresponding weights according to their actual discriminative power in judging the changes of the current image block during the fusion process, thereby implicitly completing task-oriented feature selection and enhancement without introducing additional supervision signals.

[0048] This embodiment can achieve channel attention weighted fusion by compressing spatial information using global average pooling and then generating channel weight vectors through two fully connected layers. Alternatively, it can achieve channel attention weighted fusion by first performing spatial dimension statistical aggregation on the concatenated feature maps and then modeling channel dependencies using a two-layer nonlinear mapping. Furthermore, this embodiment can also achieve channel attention weighted fusion by learning joint response patterns between channels and generating normalized weight vectors. Based on any of the above methods, this embodiment obtains adaptive control capability for channel-level responses of heterogeneous features, supporting high-precision localization and low false detection rate output of changing regions in subsequent initial detection steps.

[0049] For example, this embodiment can take the concatenated features (including C1, C2, C3 adaptation features and time series difference features) as input, first perform a global average pooling operation on them to obtain a channel statistical vector of length C_total; then input the vector into the first fully connected layer (weight matrix size is C_total × (C_total / 16)), after ReLU activation, it is sent to the second fully connected layer (weight matrix size is (C_total / 16) × C_total), and the unnormalized channel weights are output; finally, it is mapped by the Sigmoid function to a weight vector with values ​​in the interval [0,1]; the vector is scaled channel by channel with the original concatenated feature map to obtain the weighted fusion feature F_fusion, which is used as the input of the nested U-Net decoder head.

[0050] Step 5: Initial Detection: Input the fused features into the nested U-Net decoder head to generate an initial change detection binary map; Among them, the nested U-Net decoder head can refer to a decoding structure that includes multi-level hop connections and nested hop paths, which has stronger detail recovery capabilities and boundary positioning accuracy; the initial binary change map can refer to a binary output that identifies changes or no changes in units of pixels, and its resolution is the original resolution of the input image block.

[0051] This embodiment can perform four-stage upsampling and skip connection fusion on the fused features output by the nested U-Net decoder head, gradually restoring the spatial resolution to 512×512, and outputting a single-channel probability map, which is then binarized with a threshold of 0.5 after Sigmoid activation. Alternatively, this embodiment can apply auxiliary losses to the intermediate outputs of each stage of the decoder based on a multi-scale supervision mechanism to enhance the responsiveness of shallow features to regions with small changes. Furthermore, this embodiment can store the binarization result in TIFF format as input to the subsequent refining module. This embodiment obtains coarse-grained detection results covering the complete image patch and possessing preliminary change localization capabilities based on any of the above methods.

[0052] For example, in this embodiment, the 128×128×3328 fused features output above can be input into a nested U-Net decoder. After two stages of upsampling and skip connections in Stage 1 (128→256) and Stage 2 (256→512), a 512×512×1 probability map is output. Sigmoid activation is performed on the map and it is truncated by 0.5 to generate a 512×512 binary matrix, where pixels with a value of 1 are marked as changed and pixels with a value of 0 are marked as unchanged. This matrix is ​​saved as change_init.tif.

[0053] Step Six: Result Refinement: In supervised mode, bounding box prompts are generated through 8-neighbor connected component analysis and input into SAM for fine-tuning; in unsupervised mode, SAM is used for object-level segmentation, combined with a multimodal large model to generate text descriptions and calculate semantic similarity, and then fused with the initial detection results for refinement. Among them, 8-connected component analysis can refer to region labeling and statistics of binary images based on the connectivity of eight neighborhoods of pixels, used to identify independently changing regions; bounding box prompt can refer to spatial location guidance information provided to SAM in the form of rectangular boxes, replacing traditional point prompts and improving the modeling ability of small targets and irregular boundaries; object-level segmentation can refer to dividing images into semantically coherent ground objects (such as buildings, roads, and water bodies), rather than pixel-level segmentation; semantic similarity can refer to measuring the semantic consistency of objects in two temporal phases through the cosine distance between text embedding vectors, used to determine whether substantial changes have occurred.

[0054] This embodiment can extract connected components from the initial binary image output using the 8-neighbor labeling algorithm, calculate the minimum bounding rectangle of each connected component, filter out tiny connected components with an area smaller than 5×5 pixels, and dilate the retained bounding boxes by 5 pixels to generate SAM-acceptable cue input. Alternatively, this embodiment can perform 32×32 density grid cue point segmentation on the dual-temporal images using SAM to obtain an object-level mask set. The Lava-13B model is then used to generate text descriptions of each object using the cue phrase "Please describe this remote sensing image," which are then embedded into 768-dimensional vectors using MPNet-base-v2. The cosine similarity of corresponding objects in both temporal phases is calculated, and the OTU algorithm adaptively determines the similarity threshold; objects below the threshold are considered changed. Furthermore, this embodiment can also perform a logical OR operation between the SAM refinement result in supervised mode and the initial detection result, and a logical AND operation between the semantic discrimination result in unsupervised mode and the initial detection result, achieving complementary refinement. This embodiment obtains high-confidence change detection results with clearer boundaries, more complete small targets, and fewer false changes based on any of the above methods.

[0055] For example, this embodiment may perform 8-neighborhood connected component analysis on change_init.tif, detecting a total of 7 connected components. Three small regions with areas of 3, 4, and 7 pixels are removed, leaving 4 valid connected components. The minimum bounding rectangle (x_min, y_min, x_max, y_max) of each region is extracted, and each set of coordinates is extended outward by 5 pixels to generate 4 sets of bounding box prompts. The prompts are input into the frozen SAM-ViT-L, and fine segmentation is performed on the corresponding regions to output 4 high-precision masks. The 4 masks are then subjected to pixel-level logical OR operation with change_init.tif to generate the refined binary image change_refined_sup.tif. In the unsupervised path, SAM object segmentation is performed on the same image block to obtain 12 land cover objects. Lava-13B generates corresponding text, and MPNet embedding is used to calculate 12 sets of cosine similarity. The OTU algorithm determines the threshold to be 0.632, among which 5 sets of similarity < 0.632 are marked as changes, and an object-level change mask is generated. This mask is logically ANDed with change_init.tif to output change_refined_unsup.tif.

[0056] Optionally, the process of generating the bounding box prompt is as follows: perform 8-neighborhood connected component analysis on the initial change detection binary map, extract the minimum bounding rectangle of each connected component, and filter out areas smaller than 8. 5 × 5 The tiny connected regions of the pixels are calculated, and the bounding box is dilated by 5 pixels.

[0057] Among them, 8-connected component analysis refers to a region labeling method in a binary image that takes the current pixel as the center and considers it and its eight adjacent pixels (top, bottom, left, right, and four diagonals) as the basis for connectivity judgment. This analysis can be a connected region identification process based on the scan line algorithm or the recursive seed filling method, used to cluster all pixels with a value of 1 in the initial binary image into several sets of unconnected pixels. In this embodiment, this analysis is used to organize the discrete activated pixels in the initial change detection results into change region entities with spatial continuity, providing an input basis for the subsequent generation of structured prompts.

[0058] The minimum bounding rectangle (MBR) can be defined as the rectangle with the smallest area that completely covers all pixels within a connected region and whose edges are parallel to the image coordinate axes. It is determined by the quadruple (x_min, y_min, x_max, y_max). This rectangle is a compact geometric representation of the connected region in the image space, and its size reflects the spatial extensibility of the changing region. In this embodiment, this rectangle is directly input as a bounding box prompt acceptable to the SAM model, which is used to constrain the segmentation range and guide the model to focus on the real changing region, avoiding computational redundancy and semantic drift caused by indiscriminate segmentation of the entire image.

[0059] Area smaller 5 × 5 A tiny connected region of a pixel can refer to a connected region with a total number of pixels of less than 25; the area threshold is a noise suppression standard set according to the physical scale (2.5 square meters) corresponding to the typical spatial resolution (e.g., 0.5 meters) of the remote sensing image; in this embodiment, the filtering operation is used to remove isolated noise points caused by threshold segmentation error, salt-and-pepper noise or edge oscillation, to prevent them from generating invalid prompts that interfere with the stability of SAM segmentation, thereby improving the prompt signal-to-noise ratio.

[0060] The 5-pixel dilation process refers to expanding each side of the smallest bounding rectangle outward by 5 pixels to form a new rectangle (x_min-5, y_min-5, x_max+5, y_max+5). This dilation is a geometric expansion operation based on the translation of the rectangle's vertices, without changing the original rectangle's orientation or aspect ratio. In this embodiment, this process is used to compensate for the inherent uncertainties in the boundary localization of the SAM model (such as blurred mask edges and delayed prompt response), ensuring that the changed area falls completely within the prompt box, avoiding segmentation and truncation caused by overly tight prompts, and enhancing the coverage integrity and boundary continuity of small targets.

[0061] This embodiment can identify the set of all spatially connected changing pixels in the initial binary image using 8-neighbor connected component analysis; it can also traverse the binary image row by row using a scanline marking algorithm, merging overlapping intervals between adjacent rows to construct a connected component index table; further, it can use a recursive seed filling strategy, starting from each unvisited active pixel and spreading the marking along the 8-neighbor direction until no new pixels are added. This embodiment obtains accurate connected component partitioning results based on any of the above methods, serving as reliable input for subsequent minimum bounding rectangle extraction.

[0062] This embodiment can construct a bounding rectangle by taking the minimum and maximum values ​​of the x and y coordinates of all pixels within each connected component using the extreme value statistical method of the minimum bounding rectangle coordinates. Alternatively, it can improve the compactness of the rectangle by using a projection method after convex hull approximation, while maintaining computational efficiency. Furthermore, this embodiment can quickly obtain the projection span of the connected component in the X / Y directions using an integral image-accelerated boundary localization method. This embodiment obtains a compact and clearly defined bounding box cue based on any of the above methods.

[0063] This embodiment can expand the four sides of the rectangle at equal intervals based on a fixed pixel value (5) to generate an expanded bounding box; this embodiment can also adaptively adjust the expansion coefficient according to the image resolution to keep the physical scale expansion consistent (2.5 meters); further, this embodiment can also dynamically scale the expansion amount according to the area of ​​the connected components to enhance coverage in small areas and suppress excessive expansion in large areas. This embodiment obtains a cue boundary that meets the robustness requirements of SAM segmentation based on any of the above methods.

[0064] For example, this embodiment can take a pair of 0.5-meter resolution dual-temporal images from the LEVIR-CD dataset as an example. After the initial change binary map is output by the nested U-Net decoder, an 8-neighborhood connected component analysis is performed on the map, and a total of 137 connected components are detected. 92 tiny connected components with an area of ​​less than 25 pixels are removed, leaving 45 effective change regions. The minimum bounding rectangle is calculated for each region and uniformly dilated by 5 pixels to generate 45 sets of bounding box cueing. All cueing is batch input into the SAM-ViT-L model with frozen parameters to drive it to perform fine object-level segmentation on the regions within each cueing box, and finally outputs a refined change mask with continuous boundaries and complete small targets.

[0065] Optionally, the semantic similarity calculation steps in the unsupervised mode are as follows: using the Llava-13B model to generate text descriptions of dual-temporal object regions, converting the text descriptions into 768-dimensional semantic vectors using the MPNet model, using cosine similarity to measure feature differences, and adaptively determining the similarity threshold using the OTU algorithm.

[0066] Llava-13B (Large Language and Vision Assistant) is an open-source multimodal large model that can generate natural language descriptions with geographic semantic consistency based on image region masks and fixed prompts. In this embodiment, pixel-level mask regions obtained by performing object-level segmentation on dual-temporal images using SAM can be input into Llava-13B with the unified prompt "Please describe this remote sensing image," generating corresponding text descriptions for each region. Alternatively, Llava-13B can generate text descriptions covering land cover types, structural morphology, and spatial relationships based on the segmented regions guided by the 32×32 density prompts output by SAM, combined with image spatial context information. Furthermore, Llava-13B can input object masks within the same geographic coordinate range extracted by SAM from dual-temporal images into Llava-13B, generating two comparable sets of text descriptions while maintaining consistent prompts. This embodiment obtains semantic text representations of the same geographic object in dual-temporal images at different temporal phases based on any of the above methods, providing an input basis for subsequent semantic vector mapping.

[0067] For example, this embodiment can generate a text description of a large rectangular plot of cultivated farmland with visible furrows and uniform green vegetation cover for a certain farmland area obtained by SAM segmentation (at time t1), and generate a description of a rectangular plot of bare soil with no vegetation, showing recent plowing marks for the segmented area at the corresponding position at time t2. The two descriptions show significant differences at the semantic level, which can support subsequent quantitative discrimination.

[0068] MPNet (Masked and Permuted Pre-training MPNet-base-v2 is a pre-trained text encoding model based on the Transformer architecture. Its base-v2 version outputs sentence vectors with a fixed dimension of 768. In this embodiment, each text description generated above can be used as an independent input sequence. After being encoded into a token sequence by the WordPiece of MPNet-base-v2, it is fed into the backbone of the model, and the final hidden layer output corresponding to the [CLS] tag is taken as the 768-dimensional semantic vector of the text. In this embodiment, the text description generated above can also be pre-processed by standardization (removing extra spaces, unifying punctuation format, and converting to lowercase) before being input into MPNet-base-v2 to ensure the consistency of vector representation. Furthermore, in this embodiment, two text descriptions of the same object at times t1 and t2 can be concatenated into a single sentence (connected by a special delimiter [SEP]), input into MPNet-base-v2, and their respective [CLS] vectors can be extracted to avoid cross-sentence interference. This embodiment obtains a dense vector representation of bi-temporal text descriptions in a unified semantic space based on any of the above methods, making semantic differences computable.

[0069] For example, in this embodiment, the description of a large rectangular plot of cultivated farmland at time t1 and the description of a rectangular plot of bare soil at time t2 can be encoded as vectors v_1∈R^768 and v_2∈R^768, respectively. The two vectors have a large angle in the semantic space, which reflects the substantial change in the land cover category.

[0070] Cosine similarity is a dimensionless metric that measures the directional consistency between two non-zero vectors, with values ​​ranging from [...]. -1,1 The closer the value is to 1, the more semantically similar the data is; OTU (Otsu's method) is an automatic thresholding segmentation algorithm based on maximizing inter-class variance, suitable for one-dimensional data with unimodal or bimodal distributions; this embodiment can calculate the cosine similarity of the vector pair [v1,v2] obtained above:

[0071] The similarity values ​​of all object region pairs are used to construct a one-dimensional histogram, and the OTU algorithm is applied to solve for the optimal threshold τ that maximizes the inter-class variance. Alternatively, the cosine similarity set of all object pairs can be regarded as a grayscale image histogram, the threshold can be initialized as the histogram mean, and iterative updates can be performed until the inter-class variance converges, and the final threshold τ can be output. Furthermore, this embodiment can also perform Gaussian mixture modeling on the similarity distribution, fit the double Gaussian components with the EM algorithm, take the intersection of the two components as the initial threshold, and then obtain the final τ through OTU local optimization. This embodiment obtains a dynamic discrimination threshold adapted to the current image scene distribution based on any of the above methods, which is used to map semantic similarity to binary change judgment results.

[0072] For example, in a certain LEVIR-CD test, the similarity of all object pairs is distributed in the range of 0.25–0.93, showing an approximately bimodal shape. The OTU algorithm determines the optimal threshold τ=0.58. When sim(v1,v2)<0.58, it is determined that the object region has undergone a real change; otherwise, it is considered as no change or a pseudo change.

[0073] Step 7: Result Output: The segmented detection results are stitched together to output the final high-precision change detection result; Among them, stitching can refer to the process of recombining the detection results of multiple overlapping or non-overlapping image blocks into a whole image according to their original spatial positions; overlapping region averaging fusion can refer to the weighted averaging of pixel values ​​in the overlapping regions of adjacent blocks according to their distance from the center, thereby mitigating the block boundary effect.

[0074] This embodiment can be implemented by filling the refined results of each block into a full-size output buffer according to their spatial location based on the original image block coordinate index, and by using a Gaussian weighted average to calculate the pixel values ​​of overlapping areas, with the weight decreasing as the distance from the block center increases. Alternatively, this embodiment can use a linear gradient weighting function to set a weighting coefficient for the 128-pixel edge band of the 512×512 block, linearly decreasing from 1.0 to 0.2. Furthermore, this embodiment can convert the stitched result to GeoTIFF format, embed the original image's geographic coordinates and projection information, and output a final change detection layer conforming to GIS system standards. This embodiment obtains spatially continuous, smooth-boundary, and geographically accurate whole-scene change detection results based on any of the above methods.

[0075] For example, in this embodiment, a 1024×1024 image can be divided into four blocks (top left, top right, bottom left, and bottom right) at 512×512 resolution. After each block is processed by the output change_refined_sup.tif, the original coordinates are filled into a 2048×2048 output buffer (with reserved edges). For the pixels in the overlapping areas of the four blocks (such as the 512×512 overlap zone), Gaussian weights are calculated based on the Euclidean distance from the center of each block. After weighted averaging, a seamless 1024×1024 result is generated and saved as change_final.tif. The pixel value 1 indicates a change and 0 indicates no change. The georeferenced information is strictly consistent with the input image.

[0076] Optionally, the result stitching adopts an overlapping area average fusion strategy, which performs a weighted average of the pixel values ​​of the overlapping areas of adjacent image blocks, with the weight decreasing as the distance from the center increases.

[0077] The Overlap-Average Fusion Strategy is a post-processing technique used to eliminate spatial discontinuities introduced by block processing. This strategy involves setting an overlap region with a fixed pixel width (e.g., 64 pixels) between adjacent blocks during image block inference, so that the output of each block covers its original area and the surrounding overlap band. The effect of this strategy is to assign differential fusion weights to multiple pixel values ​​obtained by repeated detection of the same geographical location in multiple adjacent blocks, based on the Euclidean distance from the coordinates within the block to the center of the image of that block. This ensures that the high confidence response at the center of the block dominates the final output, while the low confidence response at the edge is smoothly attenuated, thereby suppressing stitching artifacts.

[0078] The core mathematical characteristic of the weight function in this fusion strategy is that the weight decreases as the distance from the center increases. This characteristic can be defined as follows: defining a two-dimensional coordinate system with the center of the current image block as the origin, for any pixel point within the overlapping region (… x,y ), calculate its distance to the block center. Based on this distance, a monotonically decreasing non-negative weight function is constructed. w ( d )∈[ 0,1 The specific function of this feature is to give higher fusion weights to pixels that are closer to the center of the block, and lower weights to pixels that are farther away, thereby forming a natural transition zone in the overlapping area and avoiding intensity steps or jagged edges caused by differences in prediction between blocks.

[0079] This embodiment constructs the weight function based on the Gaussian kernel function form, that is... ,in s This is a scale parameter, and its value is 1 / 4 of the half-width of the image block (e.g., for a 512×512 block). s=64), making the weight close to 1 near the center of the block and close to 0.3 to 0.5 at the edge of the overlapping area; this embodiment constructs the weight function based on the inverse square function, that is ,in r The normalization radius is set to the width of the overlapping region (e.g., 64 pixels), making the weight 1 at the center and approximately 0.5 at the boundary of the overlapping region. Furthermore, this embodiment constructs a weight function based on a piecewise linear function, where the weight linearly decreases from 1 to 0.8 within the range of 0–32 pixels from the center, and linearly decreases from 0.8 to 0.2 within the range of 32–64 pixels. Beyond 64 pixels, the weight is reset to zero. This embodiment obtains a smooth weighted fusion result of the pixel values ​​of the overlapping region of adjacent image blocks based on any of the above methods, ensuring that the intensity of the entire scene change detection map is continuous in the spatial dimension, the boundaries are natural, and there are no visible stitching seams.

[0080] For example, in this embodiment: using 512×512 pixels as the basic block size and setting the overlap width to 64 pixels, for a pair of adjacent horizontal blocks A and B, their overlapping area is 64 columns to the right of A and 64 columns to the left of B; for any column j∈[1,64] within the overlapping area, all rows of pixels in that column of block A are assigned weights. (Based on column A, x=256), the corresponding columns in block B are assigned weights. (Based on column B, center column x=256), the final merged pixel value for this column is... The process is executed column by column to complete the fusion of the entire overlapping zone; the vertical direction and the four corner overlapping areas are processed according to the same logic.

[0081] Optionally, the SAM model is a version of SAM-ViT-B, SAM-ViT-L, or SAM-ViT-H, the multimodal large model uses Lava-13B, and the text feature embedding uses the MPNet-base-v2 model.

[0082] Among them, SAM (Segment Anything Model) is a general image segmentation basic model proposed by Meta. SAM-ViT-B (Vision Transformer-Base), SAM-ViT-L (Vision Transformer-Large), and SAM-ViT-H (Vision Transformer-Large) are also mentioned. Transformer-Huge is one of three versions of this model with increasing parameter count and computational complexity under the ViT backbone network structure, corresponding to approximately 91M, 308M, and 636M trainable parameters, respectively. In this embodiment, SAM-ViT-B is suitable for edge deployment scenarios (such as airborne embedded platforms), providing basic segmentation capabilities while maintaining an inference latency of ≤0.4 seconds / 512×512 image blocks. SAM-ViT-L is suitable for B-end server environments, achieving a balance between accuracy and efficiency, with a mask IoU of 78.3% for initial changed regions on the LEVIR-CD validation set. SAM-ViT-H is suitable for high-precision tasks on cloud platforms, improving the recall rate for small changed regions (area <25 pixels) to 85.6%. All three support freezing all parameters and training only the adapter layer, satisfying the constraint of freezing all parameters of the SAM model in the first branch of this embodiment.

[0083] Llava (Large Language and Vision Assistant) is an open-source, multimodal large model. Llava-13B refers to its 13 billion parameter version, fine-tuned jointly based on the LLaMA-2-13B language model and the ViT-L / 14 visual encoder. In this embodiment, Llava-13B receives object-level image regions obtained from SAM segmentation (mask cropping maps generated with a 32×32 cue point density) and generates natural language text descriptions based on the preset cue phrase "Please describe this remote sensing image." In remote sensing image understanding tasks, compared to models such as BLIP-2-17B and Kosmos-2, it achieves a 12.7% higher semantic integrity score on the WHU-CD dataset, and its inference time is consistently 1.8–2.1 seconds per object region (GPU: RTX 8000), meeting the real-time requirements of the unsupervised refinement stage. The text descriptions output by this model serve as the input source for subsequent semantic vector embedding, without introducing additional annotation dependencies or manual intervention.

[0084] MPNet (Masked and Permuted Pre-training Network) is a pre-trained language model specifically designed for semantic similarity modeling, with MPNet-base-v2 being its second-generation base version. In this embodiment, MPNet-base-v2 maps the bi-temporal object region text descriptions generated by Llava-13B to fixed-dimensional 768-dimensional semantic vectors. This model achieves a Spearman correlation coefficient of 89.2% on the STS-B (Semantic Textual Similarity Benchmark), significantly outperforming BERT-base (85.1%) and RoBERTa-base (86.7%), ensuring that the cosine similarity calculation results have high discriminative power for real semantic differences. Its output vector directly participates in the cosine similarity measurement feature difference and the OTU algorithm's adaptive determination of the similarity threshold in this embodiment, constituting a numerical basis for change judgment in the unsupervised mode.

[0085] This embodiment can extract general semantic features based on the SAM-ViT-L model and combine Lava-13B and MPNet-base-v2 to complete semantic-driven refinement. Alternatively, it can perform lightweight object segmentation on an edge device using the SAM-ViT-B model, then compress the description using Lava-13B before embedding and comparison using MPNet-base-v2. Further, this embodiment can use a high-fidelity mask output from the SAM-ViT-H model to drive Lava-13B to generate finer-grained text descriptions, which are then extracted by MPNet-base-v2 to perform more discriminative 768-dimensional vectors for similarity discrimination. This embodiment achieves stable, reproducible, and cross-hardware platform compatible unsupervised change detection capabilities based on any of the above model combinations, ensuring that the semantic similarity discrimination module achieves an F1 score of no less than 68.08% under different deployment conditions.

[0086] For example, this embodiment can be implemented on the WHU-CD dataset (0.2-meter resolution), using SAM-ViT-L to extract C1 / C2 / C3 multi-scale features from dual-temporal images; using the bounding boxes obtained from 8-neighbor connected component analysis as prompts, SAM-ViT-L is called to perform object-level segmentation on suspected change areas, obtaining a total of 137 object masks; each mask cropping image is input into Llava-13B to generate text descriptions such as "a newly constructed concrete road segment with clear lanemarkings, surrounded by bare soil" and "a dirt track with no pavement, bordered by vegetation"; the two texts are respectively fed into MPNet-base-v2 to obtain 768-dimensional vectors v1 and v2; cosine(v1, v2) = 0.421 is calculated; the optimal threshold for the current batch is determined to be 0.537 by the OTU algorithm; since 0.421 < 0.537, the object region is determined to be a real change; finally, the intersection with the initial detection results is taken to output the refined change map.

[0087] In summary, this invention offers the following advantages: by freezing the SAM backbone and introducing a lightweight, learnable adapter layer, it achieves accurate feature transfer for remote sensing change detection tasks while retaining its large-scale visual prior knowledge; through a dual-branch collaborative modeling mechanism, it captures the inherent semantic structure of ground objects and cross-temporal sensitive differences, improving the completeness and discriminativeness of feature representation; through a channel attention-driven heterogeneous feature fusion strategy, it adaptively enhances the response of change-related channels and suppresses background interference; through a bidirectional cue-based interactive refinement mechanism, it significantly improves the boundary accuracy and recall rate of small-change regions in supervised mode, and achieves reliable differentiation between real and pseudo-changes in unsupervised mode by relying on multimodal semantic alignment capabilities; ultimately, it achieves highly robust, high-precision, and interpretable change detection performance across scenarios while significantly reducing annotation dependence and computational overhead. It effectively solves technical problems such as insufficient generalization ability, high false negative rate for small changes, and weak semantic discriminative ability in unsupervised scenarios, combining high precision, low annotation dependence, and strong cross-scenario adaptability. This represents a major innovation in remote sensing image processing and computer vision technology and has practical development and application value.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting changes in dual-temporal remote sensing images based on a segmentation model, characterized in that, Includes the following steps: (1) Preprocessing: Radiometric correction, geometric registration, block division and scale normalization are performed on the dual-temporal remote sensing images in sequence to ensure the consistency of input data; (2) Dual-branch feature extraction: A dual-branch architecture is adopted. The first branch freezes all parameters of the SAM model and extracts multi-scale general semantic features; the second branch uses Swin Transformer V2 as the backbone network to extract the temporal difference features of dual-temporal images. (3) Feature adaptation: A learnable adapter layer is inserted after each Transformer layer of the SAM model to adapt the general semantic features extracted by SAM to the task, thereby realizing the transfer of general features to change detection-specific features; (4) Feature fusion: The adapted SAM features and temporal difference features are unified at the same scale through the multi-scale feature aggregation module, and the heterogeneous feature fusion is achieved by using the channel attention weighted fusion strategy; (5) Initial detection: Input the fused features into the nested U-Net decoder head to generate an initial change detection binary map; (6) Result Refinement: Training mode selection: In supervised mode, bounding box prompts are generated by analyzing the 8-neighbor connected components and input into SAM for fine-tuning; In unsupervised mode, SAM is used for object-level segmentation, combined with a multimodal large model to generate text descriptions and calculate semantic similarity, and then fused with the initial detection results for refinement. (7) Output of results: The block detection results are spliced ​​together to output the final high-precision change detection results.

2. The method according to claim 1, characterized in that, The learnable adapter layer adopts a bottleneck structure design, including 1 × 1 Convolutional dimensionality reduction layer, ReLU activation layer, 1 × 1 The convolutional upscaling layer, and the output of the corresponding Transformer layer of SAM, are added through residual connections, so that the number of trainable parameters does not exceed 5% of the total parameters of SAM.

3. The method according to claim 1, characterized in that, In the preprocessing step, radiometric correction uses the FLAASH algorithm, geometric registration uses the SIFT algorithm with a registration error ≤ 0.5 pixels, and the image block size is [missing information]. 512 × 512 Pixels, scaled to the range [0,1].

4. The method according to claim 1, characterized in that, The multi-scale feature aggregation module upsamples the C1 shallow texture features, C2 mid-level structural features, C3 deep semantic features, and temporal difference features output by SAM to 1 / 4 scale of the original image through bilinear interpolation and then stitches them along the channel dimension.

5. The method according to claim 1, characterized in that, The channel attention weighted fusion strategy is an improvement on the SE module. It compresses spatial information through global average pooling, generates channel weight vectors through two fully connected layers, and adaptively allocates fusion weights.

6. The method according to claim 1, characterized in that, The process for generating the bounding box hint is as follows: Perform 8-neighborhood connected component analysis on the initial change detection binary map, extract the minimum bounding rectangle of each connected component, and filter out areas smaller than... 5 × 5 The tiny connected regions of the pixels are calculated, and the bounding box is dilated by 5 pixels.

7. The method according to claim 1, characterized in that, The semantic similarity calculation steps in the unsupervised mode are as follows: the Lava-13B model is used to generate text descriptions of dual-temporal object regions, the text descriptions are converted into 768-dimensional semantic vectors through the MPNet model, cosine similarity is used to measure feature differences, and the similarity threshold is adaptively determined through the OTU algorithm.

8. The method according to claim 1, characterized in that, The stitching of the results adopts an overlapping area average fusion strategy, which performs a weighted average of the pixel values ​​of the overlapping areas of adjacent image blocks, with the weight decreasing as the distance from the center increases.

9. The method according to claim 1, characterized in that, The SAM model is a version of SAM-ViT-B, SAM-ViT-L, or SAM-ViT-H. The multimodal large model uses Lava-13B, and the text feature embedding uses the MPNet-base-v2 model.

Images