A method and system for detecting semantic changes in remote sensing images

Abstract

This invention discloses a method and system for detecting semantic changes in remote sensing images. The method involves using a pre-trained semantic anchoring and fine-grained optimization network to detect semantic changes in input images from two different time phases: an encoder is used to extract corresponding multi-scale feature maps from the input images of each time phase; a semantic segmentation decoder is used to decode the images to obtain features at different resolutions; the features at the highest resolution of both time phases are used for semantic segmentation by a semantic segmentation head to obtain corresponding semantic segmentation maps; a binary change map is obtained and multiplied pixel-by-pixel with the semantic segmentation maps of both time phases to obtain a semantic change detection map. This invention aims to improve the accuracy of semantic change detection in remote sensing images by suppressing semantic drift under limited labeled sample conditions through efficient parameter fine-tuning based on semantic anchoring and fine-grained optimization, while maintaining the consistency of globally homogeneous regions and accurately depicting local heterogeneous boundaries.

Description

Technical Field

[0001] This invention relates to the field of semantic change detection technology for remote sensing images, and specifically to a method and system for detecting semantic changes in remote sensing images. Background Technology

[0002] Remote sensing image change detection aims to identify land cover changes from multi-temporal satellite imagery and is one of the fundamental methods for spatiotemporal observation of the Earth's surface. Based on different semantic interpretation levels, remote sensing image change detection can be divided into two categories: binary change detection and semantic change detection. Binary change detection only distinguishes between changed and unchanged pixels, answering the question "has a change occurred?", but it cannot provide information on land cover categories before and after the change. This semantic blind spot limits its application value in scenarios requiring clear land cover transformation information, such as urban expansion monitoring and vegetation phenology analysis. In contrast, remote sensing image semantic change detection not only locates the changed area but also identifies the land cover category in both temporal phases before and after the change, intuitively demonstrating the semantic category transformation process of land cover. Therefore, it has stronger applicability in semantically sensitive scenarios such as post-disaster damage assessment, urban renewal monitoring, and precision agriculture.

[0003] Early methods for semantic change detection in remote sensing images often employed post-classification comparison strategies, combined with traditional machine learning classifiers such as support vector machines or random forests, to represent change information by constructing multi-channel feature inputs. While these methods achieved some success on low- and medium-resolution images, their ability to capture deep semantic associations across spatiotemporal dimensions was significantly insufficient when dealing with very high-resolution remote sensing scenes. With the development of deep learning technology, end-to-end trainable network architectures have gradually become the mainstream paradigm for semantic change detection, with representative backbone networks including convolutional neural networks (CNNs) and Vision Transformers (ViT). CNNs, with their translational equivariance and local receptive fields, excel at capturing fine textures and spatial details, which is beneficial for accurately depicting change boundaries in high-resolution images. Early work directly fed convolutional neural networks with stitched bi-temporal images along channels for multi-class semantic segmentation, achieving a significant improvement over traditional post-classification processing methods. However, this simple stacking strategy has limited ability to explicitly model cross-temporal target correspondences. Subsequent research introduced Siamese network architectures, employing weight sharing or contrastive learning to extract temporally consistent features to enhance the discriminative ability of change regions. Furthermore, some methods introduce attention-like mechanisms or feature pyramid designs within the convolutional framework to enhance multi-scale representation capabilities. However, the inherent locality of convolutional operations fundamentally limits the receptive field, making it difficult for such models to perceive large-scale contextual changes and long-range semantic dependencies across the entire scene. ViT utilizes a global self-attention mechanism to capture long-distance spatial dependencies, effectively perceiving large-scale contextual changes that are difficult for convolutional encoders to capture. To address this, existing research has introduced the ViT architecture in various ways to enhance global semantic modeling, such as using it as a backbone network, parallel fusion with convolutional networks, or serial cascading. While such hybrid designs alleviate the problem of limited receptive field in pure convolutional models to some extent, they also bring two significant drawbacks: first, the structural splicing or parallelization of heterogeneous backbone networks significantly increases computational overhead, reducing the efficiency of large-scale training and inference; second, the coupling between local and global branches introduces complex optimization trade-offs—under the condition of scarce high-quality semantic change detection labeled samples, the model tends to rely on the convolutional branch with faster convergence, resulting in insufficient gradient updates of transformer branches, inadequate utilization of global contextual information, and poor performance in modeling cross-temporal changes. The aforementioned problems highlight the urgent need for a network architecture that balances local accuracy with global perception capabilities. The Swin Transformer, based on hierarchical window self-attention, provides a good foundation for this. However, when applied to downstream semantic change detection tasks, its full-parameter training method severely perturbs the visual priors established during pre-training, leading to irreversible degradation of general representation capabilities. This degradation directly weakens the model's ability to extract deep semantically invariant features, causing the discrimination process to overly rely on shallow, volatile visual representation features.Under this mechanism, the inherent spectral differences within land cover categories and the fuzzy similarities between categories are easily misjudged as real land cover changes. This results in the model being unable to accurately characterize fine-grained heterogeneous boundaries within changed regions, and also struggling to maintain global homogeneity and smoothness in unchanged regions. This semantic drift problem, caused by full-parameter training under conditions of scarce high-quality labeled samples, continues to constrain the reliability and generalization ability of semantic change detection methods. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method and system for detecting semantic changes in remote sensing images, addressing the aforementioned problems in the prior art. This invention aims to improve the accuracy of semantic change detection in remote sensing images by using efficient parameter fine-tuning based on semantic anchoring and fine-grained optimization, suppressing semantic drift under limited labeled sample conditions, and balancing the maintenance of consistency in globally homogeneous regions with the accurate characterization of local heterogeneous boundaries.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for detecting semantic changes in remote sensing images includes using a pre-trained semantic anchoring and fine-grained optimization network to detect semantic changes in input images from two different time phases. The method comprises: S1, extracting corresponding multi-scale feature maps from the input images from the two time phases using an encoder; S2, decoding the multi-scale feature maps from the two time phases using a semantic segmentation decoder to obtain features of different resolutions; S3, performing semantic segmentation on the features of the highest resolution from the two time phases using a semantic segmentation head to obtain corresponding semantic segmentation maps; decoding feature pairs composed of features of different resolutions from the two time phases using a change detection decoder, and then detecting them using a change detection head to obtain binary change maps; S4, multiplying the binary change maps pixel-by-pixel with the semantic segmentation maps from the two time phases to obtain semantic change detection maps for the two time phases.

[0006] Optionally, the encoder includes multiple cascaded coding layers, each coding layer including stacked basic coding blocks, each basic coding block consisting of sequentially connected Swin Transformer modules and multi-scale fine-grained adapters (MA), wherein the multi-scale fine-grained adapters (MA) are adapted for Swin... The processing of the feature map x output by the Transformer module includes: multiplying feature map x by a learnable parameter s1 to obtain the first feature map x1; ​​normalizing feature map x by an LN layer and then multiplying it by a learnable parameter s2 to obtain the second feature map x2, where the learnable parameters s1 and s2 are used to balance the contributions of the original features and the normalized features; adding the first feature map x1 and the second feature map x2, and then passing it through a layer normalization and a fully connected network to obtain the third feature map x3; using multi-scale convolutions with kernel sizes of 1×1, 3×3, and 5×5 in parallel to capture semantic context information under different receptive fields, adding the three captured semantic context information and the original third feature map x3 to obtain the fourth feature map x4; concatenating feature map x and the fourth feature map x4 along the channel dimension and using the sigmoid function to generate screening weights, subtracting the screening weights from 1 to obtain the remaining weights, multiplying the fourth feature map x4 by the remaining weights, adding it to the first feature map x1, and then passing it through a fully connected network to obtain the output feature map f of the basic coding block. N .

[0007] Optionally, the semantic segmentation decoder includes n-1 semantic segmentation decoding layers, where n is the number of scales of the multi-scale feature maps; each semantic segmentation decoding layer includes a CBG module, an upsampling module, and a summing module. The feature map of the corresponding scale in the multi-scale feature map is processed by the CBG module and the upsampling module, and then added to the feature map of the next scale in the multi-scale feature map by the summing module to obtain the feature of the corresponding resolution. The feature of the corresponding resolution is also used as the input of the next semantic segmentation decoding layer or the input of the semantic segmentation head. The CBG module is composed of a convolution module, a batch normalization layer, and a GELU activation function.

[0008] Optionally, the semantic segmentation head is a 1×1 2D convolutional block.

[0009] Optionally, the change detection decoder includes n-1 fine-grained spatiotemporal modules (FGSMs), where n is the number of scales in the multi-scale feature map; the function expression for the fine-grained spatiotemporal module (FGSM) to decode feature pairs composed of features of different resolutions from two temporal phases using the change detection decoder is: ; ; ; ; in, and These represent the features of the j-th layer resolution for two different time phases. For the spatial difference features of the j-th layer resolution, It is a 3D depth-separable convolutional module. For timing modules, For the original temporal difference features of the j-th layer resolution, For channel-dimensional splicing, and These are average pooling and max pooling operations, respectively. For the temporal difference features of the j-th layer resolution, For residual networks, These are the features obtained from decoding.

[0010] Optionally, the processing of the j-th resolution features of the two time phases by the time-series module includes: stacking the j-th resolution features of the two time phases along the time dimension and recombining them into a five-dimensional tensor of shape B×C×T×H×W as input to the pseudo-video sequence of the three-dimensional depth-separable convolution module, where B is the number of images in the batch, C is the number of image channels, T is the time series length, H is the image height, and W is the image width; the processing of the input pseudo-video sequence by the three-dimensional depth-separable convolution module includes: processing the pseudo-video sequence through three-dimensional depth-separable convolution branches, each of which includes a three-dimensional depth-separable convolution and a spatial enhancement module, the kernel sizes of the three-dimensional depth-separable convolutions of the three three-dimensional depth-separable convolution branches are different, and the output features of the three three-dimensional depth-separable convolution branches are concatenated along the channel dimension and added to the original pseudo-video sequence to obtain the time difference feature of the j-th resolution. .

[0011] Optionally, the change detection head is a 1×1 2D convolutional block.

[0012] The present invention also provides a remote sensing image semantic change detection system, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the remote sensing image semantic change detection method.

[0013] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the remote sensing image semantic change detection method by a processor.

[0014] The present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the remote sensing image semantic change detection method via a processor.

[0015] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: The method of the present invention includes detecting semantic changes in remote sensing images by using a pre-trained semantic anchoring and fine-grained optimization network to input images from two different time phases: The input images from the two time phases are used to extract corresponding multi-scale feature maps using an encoder; the input images from the two time phases are used to decode the corresponding features at different resolutions using a semantic segmentation decoder; the features at the maximum resolution of the two time phases are used to perform semantic segmentation using a semantic segmentation head to obtain the corresponding semantic segmentation map; a binary change map is obtained and multiplied pixel-by-pixel with the semantic segmentation maps of the two time phases to obtain a semantic change detection map. The present invention can suppress semantic drift under limited labeled sample conditions by efficiently fine-tuning parameters based on semantic anchoring and fine-grained optimization, while maintaining the consistency of globally homogeneous regions and accurately depicting local heterogeneous boundaries, thereby improving the accuracy of semantic change detection in remote sensing images. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the overall network structure of the semantic anchoring and fine-grained optimization network in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the encoder structure in an embodiment of the present invention, wherein (a) is a schematic diagram of the network structure of the basic coding block, and (b) is a schematic diagram of the network structure of the multi-scale fine-grained adapter MA.

[0019] Figure 4 This is a schematic diagram of the network structure of the semantic segmentation decoder in an embodiment of the present invention.

[0020] Figure 5 This is a schematic diagram of the network structure of the change detection decoder in an embodiment of the present invention.

[0021] Figure 6 This is a schematic diagram illustrating the working principle of the change detection decoder in an embodiment of the present invention.

[0022] Figure 7 This is a schematic diagram of the network structure of the timing module in an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0024] like Figure 1 and Figure 2As shown, the remote sensing image semantic change detection method in this embodiment includes detecting semantic changes in remote sensing images from two different time phases using a pre-trained semantic anchoring and fine-grained optimization network (SRNet). The method includes: S1, extracting corresponding multi-scale feature maps from the input images of the two different time phases using an encoder; S2, decoding the multi-scale feature maps of the two different time phases using a semantic segmentation decoder to obtain features of different resolutions; S3, performing semantic segmentation on the features of the highest resolution in both time phases using a semantic segmentation head to obtain corresponding semantic segmentation maps; decoding the feature pairs composed of features of different resolutions from the two time phases using a change detection decoder, and then detecting them using a change detection head to obtain binary change maps; S4, multiplying the binary change maps pixel-by-pixel with the semantic segmentation maps of the two different time phases to obtain semantic change detection maps for the two different time phases.

[0025] In step S1, when the encoder is used to extract the corresponding multi-scale feature maps of the two temporal phases of the input images, the two temporal phases of the input images are denoted as follows: Where H and W are the height and width of the image, respectively. The encoder is used to extract the corresponding multi-scale feature maps from the input images of the two time phases, which can be denoted as... .

[0026] In this embodiment, the encoder includes multiple cascaded coding layers, each coding layer including stacked basic coding blocks, such as... Figure 3 As shown in (a), each basic coding block consists of sequentially connected Swing Transformer modules and multi-scale fine-grained adapters (MA). For example, in this embodiment, the encoder includes four cascaded coding layers, with each of the four coding layers stacking {2, 2, 18, 2} basic coding blocks.

[0027] Existing fine-tuning methods for extracting semantic information from images mostly follow the fully connected adapter structure in natural language processing. This design struggles to effectively preserve spatial location information, resulting in limited performance improvements in spatially sensitive tasks such as semantic segmentation and change detection. To address this issue, this embodiment uses a multi-scale fine-grained adapter (MA) that combines frozen window self-attention (the core attention module of the Swin Transformer module) with a multi-scale convolutional adapter (a group of convolutional kernels of 1×1, 3×3, and 5×5) to extract rich semantic features from local to global perspectives. Figure 3 As shown in (b), the processing of the feature map x output by the multi-scale fine-grained adapter MA for the Swin Transformer module includes: The feature map x is multiplied by a learnable parameter s1 to obtain the first feature map x1; ​​the feature map x is normalized by an LN layer and then multiplied by a learnable parameter s2 to obtain the second feature map x2, where the learnable parameters s1 and s2 are used to balance the contributions of the original features and the normalized features; the first feature map x1 and the second feature map x2 are added together, and then normalized by layers and passed through a fully connected network to obtain the third feature map x3; the third feature map is subjected to attention filtering, including: the third feature map x3 is processed in parallel using convolutional kernels with kernel sizes of 1×1, 3×3 and 5×1 respectively. Multi-scale convolutions in layer 5 capture semantic context information under different receptive fields. The captured three types of semantic context information are added to the original third feature map x3 to obtain the fourth feature map x4. The fourth feature map x4 is then processed using the sigmoid function to generate screening weights. The screening weights are subtracted from 1 to obtain the remaining weights. The fourth feature map x4 is multiplied by the remaining weights, and the first feature map x1 is multiplied by the screening weights. The sum of the two results is the screening attention processing result. The screening attention processing result is then passed through a fully connected network to obtain the output feature map f of the basic coding block. N To adapt to different task requirements, a task-aware gating mechanism is introduced within the module to dynamically recalibrate feature channels, thereby enhancing the backbone network's ability to model semantic changes. An adaptive fusion strategy is employed at the input, using learnable parameters s1 and s2 to balance the contributions of original and normalized features, improving the stability and expressive power of the representation. Furthermore, by using multi-scale convolutions with kernel sizes of 1×1, 3×3, and 5×5 in parallel, semantic contextual information under different receptive fields is further captured.

[0028] like Figure 4 As shown, in this embodiment, the semantic segmentation decoder includes n-1 semantic segmentation decoding layers, where n is the number of scales of the multi-scale feature maps. Each semantic segmentation decoding layer includes a CBG module, an upsampling module, and a summing module. The feature map of the corresponding scale in the multi-scale feature map is processed by the CBG module and the upsampling module, and then added to the feature map of the next scale in the multi-scale feature map by the summing module to obtain the feature map of the corresponding resolution. This feature map of the corresponding resolution is also used as the input of the next semantic segmentation decoding layer or the input of the semantic segmentation head. The CBG module is composed of a convolution module, a batch normalization layer, and a GELU activation function. For example, feature map E4 is processed by the CBG module and the upsampling module, and then added to feature map E3 by the summing module to obtain the feature map of the corresponding resolution. f 3. Feature Map f 3. After passing through the CBG module and the upsampling module, the feature map is obtained by adding it to the feature map E2 in the summation module. f 2. Feature Map f 2. After passing through the CBG module and the upsampling module, the feature map is obtained by adding it to the feature map E1 in the summation module, thus obtaining the feature map at the corresponding resolution. f 1. Feature Mapf 1. The final output is sent to the semantic segmentation head. In this embodiment, the semantic segmentation head is a 1×1 2D convolutional block, but other segmentation head structures can also be used.

[0029] Semantic change detection requires models to not only capture fine-grained feature differences among various land features in a single temporal image in the spatial dimension, but also accurately perceive the semantic evolution of land features at the same location over time in the temporal dimension. However, existing methods still have shortcomings in the joint modeling of spatiotemporal dependencies. To address this issue, this embodiment proposes a novel feature fusion strategy when decoding feature pairs composed of features from two temporal phases at different resolutions using a change detection decoder. This strategy employs multi-scale 3D convolution to jointly model the spatial structure and temporal evolution of land features, effectively compensating for the limitations of traditional spatial difference features, suppressing false change interference, and improving the localization accuracy of changed areas. Figure 5 As shown, in this embodiment, the change detection decoder includes n-1 fine-grained spatiotemporal modules (FGSMs), where n is the number of scales in the multi-scale feature map. The function expression for the fine-grained spatiotemporal module (FGSM) to decode the feature pairs composed of features from two different time phases using the change detection decoder is as follows: ; ; ; ; in, and These represent the features of the j-th layer resolution for two different time phases. For the spatial difference features of the j-th layer resolution, It is a 3D depth-separable convolutional module. For timing modules, For the original temporal difference features of the j-th layer resolution, For channel-dimensional splicing, and These are average pooling and max pooling operations, respectively. For the temporal difference features of the j-th layer resolution, For residual networks, The features obtained from decoding. The number of scales n in the multi-scale feature map can be chosen as needed. For example, when the number of scales n in the multi-scale feature map is 4, the change detection decoder includes 3 fine-grained spatiotemporal modules (FGSMs), such as... Figure 6 As shown, and The decoded features are obtained by passing the first fine-grained spatiotemporal module, FGSM. , and The decoded features are obtained through a second fine-grained spatiotemporal module, FGSM. , and The decoded features are obtained through a third fine-grained spatiotemporal module, FGSM. .

[0030] To enhance the model's ability to model temporal variation patterns, a time series module is introduced in this embodiment. Specifically, timing module The bi-temporal features are stacked along the time dimension and recombined into a five-dimensional tensor of shape B×C×T×H×W, thereby constructing a pseudo-video sequence input. This tensor is then fed into a multi-scale convolutional kernel. 3D depthwise separable convolutional modules ∈{1, 3, 5} To extract time-aware features, and to obtain a globally consistent temporal semantic representation, this invention further employs a temporal module, using average pooling and max pooling to refine the temporal features. Aggregation is performed, followed by splicing and fusion. After extracting multi-scale difference features and global temporal features, this embodiment fuses them along the channel dimension to fully integrate information on both spatial variation and temporal evolution. The fused features are then fed into the residual module to uncover deeper semantic differences, ultimately generating change detection results. Figure 7 As shown, in this embodiment, the processing of the j-th resolution features of the two time phases by the time-series module includes: stacking the j-th resolution features of the two time phases along the time dimension and recombining them into a five-dimensional tensor of shape B×C×T×H×W as input to the pseudo-video sequence of the three-dimensional depth-separable convolution module, where B is the number of batch images, C is the number of image channels, T is the time series length, H is the image height, and W is the image width; the processing of the input pseudo-video sequence by the three-dimensional depth-separable convolution module includes: processing the pseudo-video sequence through three-dimensional depth-separable convolution branches, each of which includes a three-dimensional depth-separable convolution and a spatial enhancement module. The convolution kernel sizes of the three-dimensional depth-separable convolutions of the three three-dimensional depth-separable convolution branches are different. The output features of the three three-dimensional depth-separable convolution branches are concatenated along the channel dimension and added to the original pseudo-video sequence to obtain the time difference feature of the j-th resolution. In this embodiment, the change detection head is a 1×1 2D convolutional block; other detection head structures can also be used.

[0031] In this embodiment of the remote sensing image semantic change detection method, SRNet is trained on the SECOND dataset, and the training hyperparameters are set as follows: batch size is 4, and the initial learning rate is 3e. -4 The weight decays to 3e -4 The training run consisted of 50 epochs. The optimizer used was AdamW, coupled with a cosine annealing learning rate scheduling strategy. For data augmentation, random flipping and rotation operations were applied online during training to improve the model's generalization ability.

[0032] To verify the effectiveness of SRNet in the remote sensing image semantic change detection method of this embodiment, the SECOND dataset, HRSCD dataset, SJH-SCD dataset, and Hi-UCD-mini dataset were used to test the remote sensing image semantic change detection method of this embodiment. These datasets cover the following six object classes: water bodies, ground, low vegetation, trees, buildings, and sports fields. The mean intersection-over-union ratio (mIoU), sensitivity SeK, and anomaly contour detection F-value F_acd were used as performance indicators. The units for mean intersection-over-union ratio (mIoU), sensitivity SeK, and anomaly contour detection F-value F_acd are all percentages. Existing methods such as HRSCD4, SSCDL, BiSRNet, DFINet, TBFNet, BTSCD, SCanNet, SCNet, and SRNet-tiny were selected for comparison. The final comparative experimental results are shown in Table 1.

[0033] Table 1: Comparative Experiment Results

[0034] As shown in Table 1, compared with existing methods such as HRSCD4, SSCDL, BiSRNet, DFINet, TBFNet, BTSCD, SCANNet, SCNet, and SRNet-tiny, the SRNet method for detecting semantic changes in remote sensing images in this embodiment achieves better results on the SECOND dataset, HRSCD dataset, SJH-SCD dataset, and Hi-UCD-mini dataset, as well as on the three metrics of mean intersection-union ratio (mIoU), sensitivity SeK, and anomaly contour detection F-value (F_acd).

[0035] To verify the effectiveness of each module in SRNet in the remote sensing image semantic change detection method of this embodiment, the SECOND dataset, HRSCD dataset, SJH-SCD dataset, and Hi-UCD-mini dataset were used to test the remote sensing image semantic change detection method of this embodiment. Sensitivity SeK, salient contour detection F-value Fscd, and mean intersection-union ratio mIoU were used as performance indicators. The units of sensitivity SeK, salient contour detection F-value Fscd, and mean intersection-union ratio mIoU are all percentages. Three model variants, "w / o MA", "w / o FGSM", and "basic", were used for comparison. "w / o MA" is the model without using the multi-scale fine-grained adapter MA, "w / o FGSM" is the model without using the fine-grained spatiotemporal module FGSM, and "basic" is the model without using both the multi-scale fine-grained adapter MA and the fine-grained spatiotemporal module FGSM. The final ablation experiment results are shown in Table 2.

[0036] Table 2: Ablation Experiment Results

[0037] As shown in Table 2, the multi-scale fine-grained adapter (MA) and the fine-grained spatiotemporal module (FGSM) of SRNet in the remote sensing image semantic change detection method of this embodiment have a positive effect on the performance of SRNet on the SECOND dataset, HRSCD dataset, SJH-SCD dataset, and Hi-UCD-mini dataset, as well as on the three performance indicators of sensitivity SeK, salient contour detection F-value Fscd, and mean intersection-union ratio mIoU. This proves the effectiveness of the multi-scale fine-grained adapter (MA) and the fine-grained spatiotemporal module (FGSM).

[0038] Those skilled in the art will understand that the technical solutions provided by this invention can take the form of a method, system, or computer program product. Therefore, this invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. For example, this invention can provide a remote sensing image semantic change detection system, including a microprocessor and a memory interconnected, the microprocessor being programmed or configured to execute the remote sensing image semantic change detection method. This invention can provide a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the remote sensing image semantic change detection method via a processor. This invention can provide a computer program product including a computer program or instructions programmed or configured to execute the remote sensing image semantic change detection method via a processor. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0039] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting semantic changes in remote sensing images, characterized in that, The method involves using a pre-trained semantic anchoring and fine-grained optimization network to detect semantic changes in remote sensing images from two different time phases. The steps include: S1, extracting corresponding multi-scale feature maps from the input images of the two time phases using an encoder; S2, decoding the multi-scale feature maps of the two time phases using a semantic segmentation decoder to obtain features of different resolutions; S3, performing semantic segmentation on the features of the highest resolution from both time phases using a semantic segmentation head to obtain corresponding semantic segmentation maps; decoding feature pairs composed of features of different resolutions from the two time phases using a change detection decoder, and then detecting them using a change detection head to obtain binary change maps; and S4, multiplying the binary change maps pixel-by-pixel with the semantic segmentation maps of the two time phases to obtain semantic change detection maps for both time phases.

2. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The encoder comprises multiple cascaded coding layers, each coding layer comprising stacked basic coding blocks. Each basic coding block consists of sequentially connected Swin Transformer modules and multi-scale fine-grained adapters (MA). The multi-scale fine-grained adapters (MA) are designed for Swin... The processing of the feature map x output by the Transformer module includes: multiplying feature map x by a learnable parameter s1 to obtain the first feature map x1; ​​normalizing feature map x by an LN layer and then multiplying it by a learnable parameter s2 to obtain the second feature map x2, where the learnable parameters s1 and s2 are used to balance the contributions of the original features and the normalized features; adding the first feature map x1 and the second feature map x2, and then passing it through a layer normalization and a fully connected network to obtain the third feature map x3; using multi-scale convolutions with kernel sizes of 1×1, 3×3, and 5×5 in parallel to capture semantic context information under different receptive fields, adding the three captured semantic context information and the original third feature map x3 to obtain the fourth feature map x4; concatenating feature map x and the fourth feature map x4 along the channel dimension and using the sigmoid function to generate screening weights, subtracting the screening weights from 1 to obtain the remaining weights, multiplying the fourth feature map x4 by the remaining weights, adding it to the first feature map x1, and then passing it through a fully connected network to obtain the output feature map f of the basic coding block. N .

3. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The semantic segmentation decoder includes n-1 semantic segmentation decoding layers, where n is the number of scales of the multi-scale feature maps. Each semantic segmentation decoding layer includes a CBG module, an upsampling module, and a summing module. The feature map of the corresponding scale in the multi-scale feature map is processed by the CBG module and the upsampling module, and then added to the feature map of the next scale in the multi-scale feature map by the summing module to obtain the feature of the corresponding resolution. The feature of the corresponding resolution is also used as the input of the next semantic segmentation decoding layer or the input of the semantic segmentation head. The CBG module is composed of a convolution module, a batch normalization layer, and a GELU activation function.

4. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The semantic segmentation head is a 1×1 2D convolutional block.

5. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The change detection decoder includes n-1 fine-grained spatiotemporal modules (FGSMs), where n is the number of scales in the multi-scale feature map. The function expression for the FGSM to decode feature pairs composed of features from two different temporal phases using the change detection decoder is as follows: ； ； ； ； in, and These represent the features of the j-th layer resolution for two different time phases. For the spatial difference features of the j-th layer resolution, It is a 3D depth-separable convolutional module. For timing modules, For the original temporal difference features of the j-th layer resolution, For channel-dimensional splicing, and These are average pooling and max pooling operations, respectively. For the temporal difference features of the j-th layer resolution, For residual networks, These are the features obtained from decoding.

6. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The temporal module's processing of features at the j-th resolution for two temporal phases includes: stacking the features at the j-th resolution of the two temporal phases along the time dimension and recombining them into a five-dimensional tensor of shape B×C×T×H×W as input to the pseudo-video sequence of the three-dimensional depthwise separable convolution module, where B is the number of images in the batch, C is the number of image channels, T is the time series length, H is the image height, and W is the image width; the processing of the input pseudo-video sequence by the three-dimensional depthwise separable convolution module includes: processing the pseudo-video sequence separately through three-dimensional depthwise separable convolution branches, each of which includes a three-dimensional depthwise separable convolution and a spatial enhancement module. The convolution kernel sizes of the three three-dimensional depthwise separable convolution branches are different. The output features of the three three-dimensional depthwise separable convolution branches are concatenated along the channel dimension and added to the original pseudo-video sequence to obtain the temporal difference feature at the j-th resolution. .

7. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The change detection head is a 1×1 2D convolutional block.

8. A remote sensing image semantic change detection system, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the remote sensing image semantic change detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the remote sensing image semantic change detection method according to any one of claims 1 to 7 via a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the remote sensing image semantic change detection method according to any one of claims 1 to 7 via a processor.

Images