Remote sensing image semantic change detection method and device

Abstract

The invention discloses a remote sensing image semantic change detection method and device, and belongs to the technical field of detection result generation modules, and the method comprises the steps: obtaining a front-time-phase remote sensing image and a rear-time-phase remote sensing image of a first region; the method comprises the following steps: performing semantic extraction on a remote sensing image to form a multi-scale semantic feature map of a front time phase and a rear time phase, and performing weight sharing feature extraction on the multi-scale semantic feature map to obtain a shared feature map of the front time phase and the rear time phase; performing absolute difference on the multi-scale semantic feature maps of the front time phase and the rear time phase and the shared feature maps of the front time phase and the rear time phase to obtain two groups of difference feature maps; performing difference enhancement on the two groups of difference feature maps to obtain an enhanced difference feature map; and on the basis of the enhanced difference feature map, performing irrelevant information filtering on the shared feature map of the front time phase and the rear time phase to obtain a change detection result of the first region. Therefore, by implementing the method, the problem of low accuracy of remote sensing image change detection in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of detection result generation module technology, and in particular to a method and apparatus for detecting semantic changes in remote sensing images. Background Technology

[0002] Remote sensing images, due to their high spatiotemporal resolution and multispectral characteristics, are widely used in land use monitoring, ecological environment assessment, and disaster emergency response. In these applications, the Semantic Change Detection (SCD) task is particularly crucial, aiming to simultaneously identify changes in land cover categories and their semantic levels from multi-temporal remote sensing images. This is an important step in achieving intelligent understanding and refined analysis of remote sensing images. Land cover categories in remote sensing images are complexly distributed with blurred boundaries, and are influenced by multiple factors such as time span, sensor differences, and climatic conditions between different imaging phases, resulting in varying scales and diverse morphologies of changed areas.

[0003] In recent years, end-to-end deep learning-based methods have been used to achieve semantic change detection in remote sensing images by introducing attention mechanisms and multi-task structures. However, these methods still suffer from two core bottlenecks: first, the semantic segmentation and change detection tasks are highly coupled, leading to mutual interference and unstable optimization processes; second, the methods for extracting change information are limited, making it difficult to fully extract the differential information of changed regions, resulting in low accuracy in change region detection. Summary of the Invention

[0004] This invention provides a method and apparatus for detecting semantic changes in remote sensing images, which can solve the problem of low accuracy in detecting changes in remote sensing images in the prior art.

[0005] To address the aforementioned technical problems, this invention provides a method for detecting semantic changes in remote sensing images, comprising:

[0006] Acquire the pre-temporal and post-temporal remote sensing images of the first region;

[0007] Multi-scale semantic features are extracted from the preceding and subsequent time-phase remote sensing images to form a preceding time-phase multi-scale semantic feature map and a subsequent time-phase multi-scale semantic feature map.

[0008] Weighted shared features are extracted from the preceding phase multi-scale semantic feature map and the following phase multi-scale semantic feature map to obtain the preceding phase shared feature map and the following phase shared feature map.

[0009] The first difference feature map is obtained by performing absolute difference on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map.

[0010] The absolute difference between the pre-phase shared feature map and the post-phase shared feature map is obtained to produce a second difference feature map.

[0011] Based on the first difference feature map and the second difference feature map, difference enhancement processing is performed to obtain an enhanced difference feature map;

[0012] Based on the enhanced difference feature map, irrelevant information is filtered from the pre-phase shared feature map and the post-phase shared feature map to obtain the change detection result of the first region.

[0013] As a preferred embodiment, the step of extracting multi-scale semantic features from the preceding and following time-phase remote sensing images to form a preceding-time-phase multi-scale semantic feature map and a following-time-phase multi-scale semantic feature map includes:

[0014] Multi-scale semantic feature extraction is performed on the preceding and following remote sensing images to form several preceding semantic feature maps and several following semantic feature maps.

[0015] Several preceding temporal semantic feature maps are fused to form a preceding temporal multi-scale semantic feature map;

[0016] Several post-temporal semantic feature maps are fused to form a post-temporal multi-scale semantic feature map.

[0017] As a preferred embodiment, the step of performing difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map includes:

[0018] Spatial focusing processing is performed on the second difference feature map to obtain the first spatial focusing feature map;

[0019] The first difference feature map and the first spatial focus feature map are multiplied element by element to obtain the third difference feature map;

[0020] The third difference feature map is upsampled to obtain the fourth difference feature map;

[0021] The third and fourth difference feature maps are added element by element to obtain the fifth difference feature map;

[0022] Cross-layer attention integration is performed on the fifth difference feature map to obtain an enhanced difference feature map.

[0023] As a preferred embodiment, the step of filtering irrelevant information from the pre-temporal shared feature map and the post-temporal shared feature map based on the enhanced difference feature map to obtain the change detection result of the first region includes:

[0024] The enhanced difference feature map is subjected to spatial focusing processing to obtain a second spatial focusing feature map;

[0025] The second spatial focusing feature map is multiplied element-wise with the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature map to obtain the pre-temporal spatial attention feature map, the post-temporal spatial attention feature map, and the difference spatial attention feature map.

[0026] Based on the difference spatial attention feature map, irrelevant information is filtered from the previous temporal spatial attention feature map and the subsequent temporal spatial attention feature map respectively to obtain the change detection result of the first region.

[0027] As a preferred embodiment, the step of filtering irrelevant information from the preceding temporal spatial attention feature map and the following temporal spatial attention feature map based on the difference spatial attention feature map to obtain the change detection result of the first region includes:

[0028] A nonlinear mapping is performed on the difference space attention feature map to obtain a change probability map;

[0029] The change probability map is binarized to obtain the binarized result;

[0030] Based on the binarization result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are masked to obtain the change detection result of the first region.

[0031] As a preferred embodiment, based on the binarization processing result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are respectively masked to obtain the change detection result of the first region, including:

[0032] On several channels of the preceding temporal spatial attention feature map, the maximum value index of each pixel position in the preceding temporal spatial attention feature map is calculated respectively to form the preceding temporal index feature map;

[0033] On several channels of the post-temporal spatial attention feature map, the maximum value index of each pixel position in the post-temporal spatial attention feature map is calculated to form a post-temporal index feature map;

[0034] Based on the binarization result, the preceding time-phase index feature map and the following time-phase index feature map are masked respectively to obtain the change detection result of the first region.

[0035] As a preferred embodiment, the step of calculating the maximum value index of each pixel position in the preceding temporal spatial attention feature map on several channels of the preceding temporal spatial attention feature map to form a preceding temporal index feature map includes:

[0036] For each pixel position, calculate the value of the pixel position in each channel of the preceding temporal spatial attention feature map;

[0037] The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position;

[0038] For the aforementioned pre-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position order to form the pre-temporal index feature map.

[0039] As a preferred embodiment, the step of calculating the maximum value index of each pixel position in the post-temporal spatial attention feature map on several channels of the post-temporal spatial attention feature map to form a post-temporal index feature map includes:

[0040] For each pixel position, calculate the value of the pixel position in each channel of the post-temporal spatial attention feature map;

[0041] The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position;

[0042] For the post-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position to form the post-temporal index feature map.

[0043] As a preferred embodiment, based on the binarization processing result, masking processing is performed on the previous time-phase index feature map and the subsequent time-phase index feature map respectively to obtain the change detection result of the first region, including:

[0044] Based on the binarization result, the previous temporal index feature map is masked to obtain the previous land cover category of the first region.

[0045] Based on the binarization result, the post-temporal index feature map is masked to obtain the changed land cover category of the first region;

[0046] Based on the land cover categories before and after the change, the changed areas and land cover changes in the first region are determined.

[0047] Accordingly, the present invention provides a remote sensing image semantic change detection device, comprising: an image acquisition module, a semantic extraction module, a shared feature extraction module, a first absolute difference module, a second absolute difference module, a difference enhancement module, and a detection result generation module;

[0048] The image acquisition module is used to acquire the previous and subsequent time-phase remote sensing images of the first region;

[0049] The semantic extraction module is used to extract multi-scale semantic features from the preceding and subsequent time-phase remote sensing images respectively, forming a preceding time-phase multi-scale semantic feature map and a subsequent time-phase multi-scale semantic feature map.

[0050] The shared feature extraction module is used to perform weighted shared feature extraction on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map to obtain the pre-temporal shared feature map and the post-temporal shared feature map.

[0051] The first absolute difference module is used to perform absolute difference on the previous phase multi-scale semantic feature map and the subsequent phase multi-scale semantic feature map to obtain the first difference feature map;

[0052] The second absolute difference module is used to perform absolute difference on the pre-phase shared feature map and the post-phase shared feature map to obtain a second difference feature map;

[0053] The difference enhancement module is used to perform difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map;

[0054] The detection result generation module is used to filter irrelevant information from the pre-phase shared feature map and the post-phase shared feature map based on the enhanced difference feature map, and obtain the change detection result of the first region.

[0055] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0056] This invention provides a method for detecting semantic changes in remote sensing images. The method acquires a first-phase and a second-phase remote sensing image of a first region. Multi-scale semantic feature extraction is performed on both the first-phase and second-phase images to form a first-phase multi-scale semantic feature map and a second-phase multi-scale semantic feature map. This invention performs multi-scale semantic extraction on the image before extracting change information after acquisition, decoupling semantic segmentation and change detection tasks and improving the stability of semantic extraction. Weighted shared features are extracted from the first-phase and second-phase multi-scale semantic feature maps to obtain a first-phase shared feature map and a second-phase shared feature map. Absolute difference is performed between the first-phase and second-phase multi-scale semantic feature maps to obtain a first difference feature map. Absolute difference is also performed between the first-phase and second-phase shared feature maps to obtain a second difference feature map. Difference enhancement processing is then performed based on the first and second difference feature maps to obtain an enhanced difference feature map. This invention generates two sets of distinct difference feature maps by performing absolute difference analysis on two different types of feature maps. The extracted difference feature maps are then enhanced to fully extract the difference information from the changed regions. Based on the enhanced difference feature maps, irrelevant information is filtered from the shared feature maps of the preceding and following time phases to obtain the change detection result for the first region. This invention improves the accuracy of the extracted features by decoupling semantic extraction and change detection tasks and by performing multi-scale difference extraction. By using the feature maps obtained from semantic extraction and difference information extraction to perform change detection on the first region, the accuracy of change detection in remote sensing images can be effectively improved. Attached Figure Description

[0057] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0058] Figure 1 A flowchart illustrating an embodiment of the remote sensing image semantic change detection method provided by the present invention;

[0059] Figure 2 A flowchart illustrating an embodiment of the feature fusion method provided by the present invention;

[0060] Figure 3 A flowchart illustrating an embodiment of the differential information extraction method provided by the present invention;

[0061] Figure 4 A flowchart illustrating an embodiment of the spatial feature enhancement method provided by the present invention;

[0062] Figure 5 This is a flowchart illustrating another embodiment of the remote sensing image semantic change detection method provided by the present invention;

[0063] Figure 6 This is a schematic diagram of one embodiment of the remote sensing image semantic change detection device provided by the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0066] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0069] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0070] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0071] See Figure 1 To address the problem of low accuracy in remote sensing image change detection in existing technologies, an embodiment of the present invention provides a method for detecting semantic changes in remote sensing images. This method includes steps 101 to 107, each step of which is detailed below:

[0072] Step 101: Obtain the pre-temporal and post-temporal remote sensing images of the first region.

[0073] In this embodiment of the invention, the first region is the area where changes need to be detected and difference information extracted. By performing semantic difference recognition on remote sensing images of the first region at two different time points, regions with semantic differences can be identified within the first region. The previous and subsequent remote sensing images of the first region are remote sensing images taken at two different time points, both covering the same geographical area. The previous and subsequent remote sensing images can be derived from remote sensing satellite data. To avoid spurious changes caused by shooting angle, lighting conditions, or geometric distortion, image registration processing is required when acquiring the two remote sensing images at two different time points to align them with pixel-level precision, ensuring high spatial consistency between the previous and subsequent remote sensing images.

[0074] Step 102: Extract multi-scale semantic features from the preceding and following remote sensing images to form a preceding multi-scale semantic feature map and a following multi-scale semantic feature map.

[0075] In practical applications, semantic changes not only involve changes in spatial location, but also include shifts in category hierarchy. Therefore, traditional methods are difficult to accurately capture this kind of deep semantic information, and are prone to false detections and false negatives in small target changes or boundary areas.

[0076] In this embodiment of the invention, by performing multi-scale semantic feature extraction on the previous and subsequent time-phase remote sensing images respectively, the semantic features of each remote sensing image can be extracted from multiple scales, thereby improving the accuracy of subsequent change area analysis.

[0077] As a preferred embodiment, multi-scale semantic feature extraction is performed on the preceding and following remote sensing images to form a preceding multi-scale semantic feature map and a following multi-scale semantic feature map, including:

[0078] Multi-scale semantic feature extraction is performed on the preceding and following remote sensing images to form several preceding semantic feature maps and several following semantic feature maps.

[0079] Several preceding temporal semantic feature maps are fused to form a preceding temporal multi-scale semantic feature map;

[0080] Several post-temporal semantic feature maps are fused to form a post-temporal multi-scale semantic feature map.

[0081] In this embodiment of the invention, multi-scale semantic feature extraction of remote sensing images to form multi-scale semantic feature maps mainly includes two main steps. The first main step is to extract semantic features of remote sensing images at different scales to form multiple feature maps of different resolutions. The second main step is to fuse the features of multiple feature maps of different resolutions to form a multi-scale semantic feature map.

[0082] As an example of an embodiment of the present invention, the Swin Transformer, through its hierarchical window attention mechanism, possesses powerful global modeling capabilities and local detail preservation capabilities, making it suitable for modeling complex land cover changes in multi-source remote sensing images. Therefore, the Swin Transformer can be selected as the backbone network for multi-scale semantic feature extraction. The Swin Transformer consists of four stages, each outputting a set of feature maps for subsequent feature fusion and decoding. To extract semantic features using the Swin Transformer, the input remote sensing image is first divided into 4×4 patches, and each patch is linearly projected to obtain initial embedded features. This design can compress redundant information while preserving spatial structure. Each stage of the Swin Transformer stacks multiple Swin Transformer Blocks, alternately using conventional window attention (W-MSA) and shifted window attention (SW-MSA) for local modeling and global receptive field expansion, with possible numbers of 2, 4, 12, and 4 blocks respectively. Each block includes: a multi-head self-attention module to model long-range dependencies within the image; an MLP feedforward network to enhance nonlinear expressive power; and residual connections + layer normalization to stabilize training and improve generalization performance. Each stage of the Swin Transformer extracts semantic features from the input remote sensing image and outputs a set of feature maps at different resolutions, used to model low-level texture information and high-level semantic information respectively, meeting the needs of multi-scale difference detection. The feature dimensions and spatial resolution of the feature maps (C1–C4) in each stage can be: C1: 128×H / 4×W / 4, C2: 256×H / 8×W / 8, C3: 512×H / 16×W / 16, C4: 1024×H / 32×W / 32. Here, H and W represent the height and width of the input remote sensing image, respectively. As the network depth increases, the number of feature channels gradually increases, while the spatial resolution gradually decreases.

[0083] In this embodiment of the invention, for a remote sensing image, multiple sets of semantic feature maps are obtained by multi-scale semantic feature extraction. These semantic feature maps are then fused to form a multi-scale semantic feature map. This fully leverages the advantages of shallow features in detail perception and the semantic understanding capabilities of deep features, thereby fully exploring the correlation between features at different scales, achieving dynamic weighted fusion, and improving the expressive power of multi-scale features and the accuracy of identifying changing regions.

[0084] In this embodiment of the invention, feature fusion includes several steps: scale alignment, cross-scale cross-attention mechanism processing, and channel fusion. Specifically, scale alignment involves upsampling the lower-resolution deep semantic feature map step by step using bilinear interpolation to ensure its spatial resolution is consistent with that of the adjacent shallower semantic feature maps. Assuming that in the semantic feature maps C1 to C4, semantic feature map C4 is at the deepest layer and semantic feature map C1 is at the shallowest layer, then during scale alignment, semantic feature map C4 is first upsampled to the same H / 16×W / 16 as semantic feature map C3. The upsampled semantic feature map C4 and semantic feature map C3 are then used as a pair of inputs to the cross-attention module for feature interaction fusion. The fused semantic feature map is then upsampled to the resolution of semantic feature map C2, and fusion continues. Finally, the fused result is upsampled to the resolution of semantic feature map C1.

[0085] In this embodiment of the invention, it is assumed that the two sets of semantic feature maps obtained after scale alignment are as follows: and Among them, C a For feature map F a The channel dimension size, C b For feature map F b The channel dimension size. Feature interaction fusion is achieved between the two sets of semantic feature maps based on a cross-attention mechanism:

[0086] The query is composed of shallower features F a Through linear transformation, we obtain:

[0087] Q = W Q ·F a

[0088] The key and value are obtained from the deeper feature F_b through a linear transformation:

[0089] K = W k ·F b V = W v ·F b

[0090] Calculate the attention weight matrix A:

[0091]

[0092] We obtain the weighted fusion features:

[0093] F = A × V × F a

[0094] Where Q represents the query, K represents the key, V represents the value, and W represents the value. Q W k W vLet d be a learnable linear projection matrix. k is the dimensional scaling factor for the key. Through this mechanism, shallow features can adaptively "query" useful information from deep features, achieving a dynamically weighted fusion representation.

[0095] In this embodiment of the invention, the cross-attention fusion process iteratively processes semantic feature map C4 with semantic feature map C3, the fusion result of semantic feature map C4 and semantic feature map C3 with semantic feature map C2, and the fusion result of the previous step with semantic feature map C1, finally outputting a fused feature map with the same resolution as semantic feature map C1. After obtaining the fused feature map, a 1×1 convolutional layer is used to compress the channels of the fused feature map to reduce the computational burden. At the same time, batch normalization and ReLU activation function are combined to improve the non-linear representation capability of the features. Finally, a fused feature map F with the same resolution as semantic feature map C1 and compressed channel dimensions is obtained, which is used for subsequent change information extraction and decoding tasks.

[0096] In this embodiment of the invention, by performing feature fusion on the pre-temporal semantic feature map through the above steps, a pre-temporal multi-scale semantic feature map FA with unified spatial resolution and channel dimension can be obtained; by performing feature fusion on the post-temporal semantic feature map through the above steps, a post-temporal multi-scale semantic feature map FB with unified spatial resolution and channel dimension can be obtained.

[0097] As an example of an embodiment of the present invention, see Figure 2This is a flowchart illustrating an embodiment of the feature fusion method provided by the present invention. It is assumed that the semantic feature map C4 has 1024 channels and a resolution of H / 32×W / 32; the semantic feature map C3 has 512 channels and a resolution of H / 16×W / 16; the semantic feature map C2 has 256 channels and a resolution of H / 8×W / 8; and the semantic feature map C1 has 128 channels and a resolution of H / 4×W / 4. During feature fusion, semantic feature map C4 is first upsampled to the same resolution as semantic feature map C3, resulting in semantic feature map C4_up. At this point, semantic feature map C4_up has 1024 channels and a resolution of H / 16×W / 16. Semantic feature map C4_up and semantic feature map C3 are then used as a pair of inputs to the cross-attention module for feature interaction fusion, resulting in semantic feature map CAF1. At this point, semantic feature map CAF1 has 1024 channels and a resolution of H / 16×W / 16. Semantic feature map CAF1 is then upsampled to the resolution of semantic feature map C2, resulting in semantic feature map CAF1_up. At this point, semantic feature map CAF1_up has 1024 channels and a resolution of H / 8×W / 16. W / 8; Semantic feature map CAF1_up and semantic feature map C2 are input as a pair into the cross-attention module for feature interaction fusion to obtain semantic feature map CAF2. At this time, CAF2 has 1024 channels and a resolution of H / 8×W / 8; Semantic feature map CAF2 is upsampled to the resolution of semantic feature map C1 to obtain semantic feature map CAF2_up. At this time, CAF2_up has 1024 channels and a resolution of H / 4×W / 4; Semantic feature map CAF2_up and semantic feature map C1 are input as a pair into the cross-attention module for feature interaction fusion to obtain semantic feature map CAF3. At this time, semantic feature map CAF3 has 128 channels and a resolution of H / 4×W / 4.

[0098] Step 103: Perform weighted shared feature extraction on the preceding phase multi-scale semantic feature map and the following phase multi-scale semantic feature map to obtain the preceding phase shared feature map and the following phase shared feature map.

[0099] In current practical applications, the methods for extracting difference information from feature maps are limited, making it difficult to fully extract the difference information in changing regions. This invention extracts difference information through two-level differential features and enhances the calculated difference information, effectively improving the extraction rate of change information in changing regions, thereby improving the accuracy of changing region identification.

[0100] In this embodiment of the invention, differential features can be extracted by performing absolute difference between two sets of feature maps. Therefore, to obtain two sets of differential features, further processing of the pre-phase multi-scale semantic feature map and the post-phase multi-scale semantic feature map is required to obtain two more sets of feature maps. This involves performing absolute difference between the two different types of feature maps, resulting in two different sets of differential features. Therefore, by performing weighted shared feature extraction on the pre-phase multi-scale semantic feature map and the post-phase multi-scale semantic feature map, pre-phase shared feature maps and post-phase shared feature maps can be obtained for extracting another set of differential features.

[0101] In this embodiment of the invention, a weighted convolutional module (Seg_head) is used to extract weighted shared features from the front-phase multi-scale semantic feature map FA and the back-phase multi-scale semantic feature map FB, which can yield the front-phase shared feature map SegA and the back-phase shared feature map SegB.

[0102] In deep learning, shared weights are an important parameter optimization technique. When the same convolutional module (Seg_head) is used for feature maps FA and FB, it means that the two processing steps use the same convolutional kernel parameters. The feature maps SegA and SegB extracted by the convolutional module (Seg_head) with shared weights have the same metric in the semantic space, which facilitates subsequent difference calculation or feature fusion, reduces the number of model parameters, lowers computational cost, effectively prevents overfitting, and makes the model translation invariant.

[0103] Step 104: Perform absolute difference analysis on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map to obtain the first difference feature map.

[0104] Step 105: Perform absolute difference analysis on the pre-phase shared feature map and the post-phase shared feature map to obtain a second difference feature map.

[0105] In this embodiment of the invention, absolute differencing of feature maps refers to taking the absolute value of the difference between corresponding elements of two feature maps to generate a new difference feature map. When performing absolute differencing, it is necessary to ensure that the two feature maps have the same height, width, and number of channels; this can be achieved by first aligning their sizes through operations such as interpolation and cropping. The difference feature map obtained after absolute differencing reflects the feature differences between the feature maps. Therefore, by performing absolute differencing on the preceding and following time-scale semantic feature maps, a first difference feature map reflecting the difference information between them can be obtained; similarly, by performing absolute differencing on the preceding and following time-shared feature maps, a second difference feature map reflecting the difference information between them can be obtained. Both the first and second difference feature maps reflect the difference information between the preceding and following time-scale remote sensing images; therefore, combining the first and second difference feature maps enhances the difference information, thereby improving the accuracy of subsequent analysis of changed areas.

[0106] Step 106: Perform difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map.

[0107] As a preferred embodiment, a difference enhancement process is performed based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map, including:

[0108] Spatial focusing processing is performed on the second difference feature map to obtain the first spatial focusing feature map;

[0109] The first difference feature map and the first spatial focus feature map are multiplied element by element to obtain the third difference feature map;

[0110] The third difference feature map is upsampled to obtain the fourth difference feature map;

[0111] The third and fourth difference feature maps are added element by element to obtain the fifth difference feature map;

[0112] Cross-layer attention integration is performed on the fifth difference feature map to obtain an enhanced difference feature map.

[0113] In this embodiment of the invention, a spatial attention module (SAM) is used to perform spatial focusing processing on the second difference feature map to obtain a first spatially focused feature map. Specifically:

[0114] S1=Sigmoid(Conv_(1×1)(ReLU(BN(Conv_(3×3)(Conv_(1×1)(diff2))))))

[0115] In the formula, S1 is the first spatial focusing feature map; diff2 is the second difference feature map; Sigmoid is the activation function; Conv is the convolution; ReLU is the linear rectified activation function; and BN is the normalization function.

[0116] The Spatial Attention Module (SAM) is a module in convolutional neural networks used for spatial attention. It generates spatial attention weights through operations such as pooling and convolution, and weights different spatial locations of the feature map to focus on key regions, suppress irrelevant backgrounds, and filter out important differential features from the spatial dimension.

[0117] In this embodiment of the invention, after obtaining a first spatially focused feature map by spatial focusing processing of the second difference feature map, the first difference feature map is multiplied by the first spatially focused feature map to obtain a new difference feature, forming a third difference feature map. This makes the resulting third car cover feature map more focused on the target area and suppresses background noise. Next, the second difference feature map is upsampled using a convolution module to obtain a fourth difference feature map, mapping high-level semantic features back to the original input space to achieve pixel-level accurate prediction. Then, based on the idea of ​​a residual network, the third and fourth difference feature maps are added to form a fifth difference feature map. Adding the third and fourth difference feature maps element-wise is essentially cross-path information fusion. If the responses of the third and fourth difference feature maps are consistent in certain regions, the signals in these regions will be strengthened after addition. If the responses of the two maps differ significantly in certain regions, they may cancel each other out after addition, thus filtering out unstable noise. Therefore, adding the third and fourth difference feature maps effectively enhances common features, suppresses redundant information, and improves the reliability of feature data. Finally, the fifth differential feature map is decomposed into features at multiple scales and dynamically sparsely computed using the sparse attention pyramid module. Combined with dynamic sparse attention to focus on key regions, a high-quality enhanced differential feature map with both detail and semantics is generated.

[0118] As an example of an embodiment of the present invention, see Figure 3This is a flowchart illustrating an embodiment of the difference information extraction method provided by the present invention. A shared feature map SegA is extracted from the preceding temporal multi-scale semantic feature map FA using a weighted convolutional module (Seg_head); a shared feature map SegB is extracted from the following temporal multi-scale semantic feature map FB using the same weighted convolutional module (Seg_head); weighted shared feature extraction is performed on FA and the following temporal multi-scale semantic feature map FB to obtain the following and following temporal shared feature maps SegB respectively. Absolute difference is performed between the preceding temporal multi-scale semantic feature map FA and the following temporal multi-scale semantic feature map FB to obtain a first difference feature map diff1; absolute difference is performed between the preceding temporal shared feature map SegA and the following temporal shared feature map SegB to obtain a second difference feature map diff2; the second difference feature map diff2 is processed by a spatial attention module (SAM) to obtain a first spatial focus feature map S1; the first difference feature map diff1 and the first spatial focus feature map S1 are multiplied by a dot to obtain a new third difference feature map diff3. The second difference feature map, diff2, is upsampled using a convolutional module to obtain the fourth difference feature map, diff4. Based on the idea of ​​residual networks, the third difference feature map, diff3, is added to the fourth difference feature map, diff4, and finally, the enhanced difference feature map, diff5, is obtained through a sparse attention pyramid module. This invention, through difference information extraction, can derive the intermediate representations required for the semantic segmentation (SS) task: the pre-temporal shared feature map SegA and the post-temporal shared feature map SegB; and the enhanced difference feature map, diff5, required for the change detection (BCD) task.

[0119] Step 107: Based on the enhanced difference feature map, perform irrelevant information filtering on the previous phase shared feature map and the subsequent phase shared feature map to obtain the change detection result of the first region.

[0120] As a preferred embodiment, based on the enhanced difference feature map, irrelevant information is filtered from the pre-temporal shared feature map and the post-temporal shared feature map to obtain the change detection result of the first region, including:

[0121] The enhanced difference feature map is subjected to spatial focusing processing to obtain a second spatial focusing feature map;

[0122] The second spatial focusing feature map is multiplied element-wise with the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature map to obtain the pre-temporal spatial attention feature map, the post-temporal spatial attention feature map, and the difference spatial attention feature map.

[0123] Based on the difference spatial attention feature map, irrelevant information is filtered from the previous temporal spatial attention feature map and the subsequent temporal spatial attention feature map respectively to obtain the change detection result of the first region.

[0124] In this embodiment of the invention, after obtaining the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature, spatial feature enhancement is performed on the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature to improve the accuracy of subsequent change region identification. Specifically, the enhanced difference feature map is first spatially focused using a spatial attention module to obtain a second spatially focused feature map, thereby increasing the pixel values ​​of the change region and decreasing the pixel values ​​of the unchanged region. Then, the second spatially focused feature map is multiplied element-wise with the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature map to obtain the pre-temporal spatial attention feature map, the post-temporal spatial attention feature map, and the difference spatial attention feature map, thereby enhancing the focus on information about the change region.

[0125] As an example of an embodiment of the present invention, see Figure 4 This is a flowchart illustrating an embodiment of the spatial feature enhancement method provided by the present invention. First, a second spatial focus feature map (SAF) is generated from the enhanced difference feature map (diff5) using a spatial attention module (SAM). Then, the second spatial focus feature map (SAF) is element-wise multiplied with the preceding temporal shared feature map (SegA), the following temporal shared feature map (SegB), and the enhanced difference feature map (diff5). The multiplied features are further enhanced using 3x3 and 1x1 convolutional modules (Conv), and the spatial dimensions are restored using bilinear upsampling (Upsample). This yields the final preceding temporal spatial attention feature map (Fssa) and the following following temporal spatial attention feature map (Fssb), with dimensions n×H×W, where n is the number of semantic categories; and the difference spatial attention feature map (Fcd), with dimensions 1×H×W.

[0126] As a preferred embodiment, based on the difference spatial attention feature map, irrelevant information is filtered from the preceding temporal spatial attention feature map and the following temporal spatial attention feature map respectively to obtain the change detection result of the first region, including:

[0127] A nonlinear mapping is performed on the difference space attention feature map to obtain a change probability map;

[0128] The change probability map is binarized to obtain the binarized result;

[0129] Based on the binarization result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are masked to obtain the change detection result of the first region.

[0130] In this embodiment of the invention, based on the differential spatial attention feature map obtained after spatial feature enhancement, irrelevant information is filtered from both the pre-temporal and post-temporal spatial attention feature maps to identify the changing regions. The irrelevant information refers to the same feature information in both the pre-temporal and post-temporal spatial attention feature maps. By filtering out the irrelevant information from both maps, the changing regions within the first region can be identified.

[0131] In this embodiment of the invention, to identify the changing region, firstly, a change probability map CM is obtained by nonlinearly mapping the difference spatial attention feature map using the Sigmoid function, with dimensions of 1×H×W. The change probability map is then binarized. In the change probability map CM, if the pixel change probability exceeds a fixed threshold T (default 0.5), it is determined to be a changing pixel; otherwise, it is considered an invariant pixel, thus yielding the binarization result. The binarization result records whether each pixel position on the difference spatial attention feature map corresponds to a changing pixel. Therefore, based on the binarization result, masking is performed on the preceding and following temporal spatial attention feature maps respectively, which reveals the region where the changing pixels are located, thereby identifying the change detection result of the first region.

[0132] As a preferred embodiment, based on the binarization processing result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are respectively masked to obtain the change detection result of the first region, including:

[0133] On several channels of the preceding temporal spatial attention feature map, the maximum value index of each pixel position in the preceding temporal spatial attention feature map is calculated respectively to form the preceding temporal index feature map;

[0134] On several channels of the post-temporal spatial attention feature map, the maximum value index of each pixel position in the post-temporal spatial attention feature map is calculated to form a post-temporal index feature map;

[0135] Based on the binarization result, the preceding time-phase index feature map and the following time-phase index feature map are masked respectively to obtain the change detection result of the first region.

[0136] In this embodiment of the invention, when masking the preceding and following temporal spatial attention feature maps and identifying the change detection result in the first region, the maximum value index of each pixel position in the channel dimension on both the preceding and following temporal spatial attention feature maps is first calculated. Then, a preceding temporal index feature map and a following temporal index feature map are formed based on the maximum value index of each pixel position. Finally, masking is performed on the preceding and following temporal index feature maps to obtain the change detection result in the first region.

[0137] As a preferred embodiment, on several channels of the pre-temporal spatial attention feature map, the maximum value index of each pixel position in the pre-temporal spatial attention feature map is calculated to form a pre-temporal index feature map, including:

[0138] For each pixel position, calculate the value of the pixel position in each channel of the preceding temporal spatial attention feature map;

[0139] The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position;

[0140] For the aforementioned pre-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position order to form the pre-temporal index feature map.

[0141] As a preferred embodiment, on several channels of the post-temporal spatial attention feature map, the maximum value index of each pixel position in the post-temporal spatial attention feature map is calculated to form a post-temporal index feature map, including:

[0142] For each pixel position, calculate the value of the pixel position in each channel of the post-temporal spatial attention feature map;

[0143] The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position;

[0144] For the post-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position to form the post-temporal index feature map.

[0145] In this embodiment of the invention, the method for determining the maximum value index of each pixel position is as follows: For each pixel position, determine its value in each channel of the feature map. Each channel has its own index. Compare the values ​​in each channel, and determine the index of the channel corresponding to the maximum value as the maximum value index. For example, assuming the feature map has 3 channels, for the pixel position (0, 0), its value is 5 in channel 1, 3 in channel 2, and 8 in channel 3, then 2 is the maximum value index of the pixel position (0, 0).

[0146] In this embodiment of the invention, for each feature map, after determining the maximum value index corresponding to all pixel positions, since the pixel position records the spatial position of the pixel on the feature map, by collecting each pixel position and its corresponding maximum value index, the maximum value index of each pixel can be organized according to the spatial position order to form an index feature map.

[0147] As a preferred embodiment, based on the binarization processing result, masking processing is performed on the previous time-phase index feature map and the subsequent time-phase index feature map respectively to obtain the change detection result of the first region, including:

[0148] Based on the binarization result, the previous temporal index feature map is masked to obtain the previous temporal anomaly region of the first region and the previous land cover category of the previous temporal anomaly region.

[0149] Based on the binarization result, the post-temporal index feature map is masked to obtain the post-temporal anomaly region of the first region and the changed land cover category of the post-temporal anomaly region.

[0150] The change region of the first region is determined based on the anomaly region of the preceding time phase and the anomaly region of the following time phase.

[0151] Based on the land cover categories before and after the change, the land cover change results of the changed area are determined.

[0152] In this embodiment of the invention, by masking the preceding and following time-phase index feature maps, identical features between them can be filtered out, leaving only the difference information. Based on the position of the difference information on the preceding time-phase index feature map, the preceding time-phase anomalous area and the pre-change land cover category of the preceding time-phase anomalous area can be determined. Based on the position of the difference information on the following time-phase index feature map, the following time-phase anomalous area and the post-change land cover category of the following time-phase anomalous area can be determined. Based on the detected preceding and following time-phase anomalous areas, the changed area of ​​the first region can be determined; for example, the intersection of the preceding and following time-phase anomalous areas can be taken as the changed area of ​​the first region. The land cover change result of the changed area is obtained based on the pre-change land cover category of the preceding time-phase anomalous area and the post-change land cover category of the following time-phase anomalous area.

[0153] See Figure 5This is a flowchart illustrating another embodiment of the remote sensing image semantic change detection method provided by the present invention. Semantic detection is performed on a first region. First, remote sensing images corresponding to the preceding and following time phases of the first region are acquired. Main features are extracted from both sets of remote sensing images to obtain multiple sets of semantic features corresponding to each set. Multi-scale feature aggregation of these multiple sets of semantic features yields multi-scale semantic feature maps corresponding to the preceding and following time phases. By extracting change features from the multi-scale semantic feature maps of the preceding and following time phases, a shared feature map segA for the preceding time phase, a shared feature map segB for the following time phase, and a difference feature map diff are obtained. Based on the extracted preceding and following time phase features… Spatial feature enhancement is performed using the phase-shared feature map segA, the post-temporal shared feature map segB, and the difference feature map diff. This yields the output of the semantic segmentation task (ssA) – the pre-temporal spatial attention feature map Fssa, the output of the semantic segmentation task (ssB) – the post-temporal spatial attention feature map Fssb, and the output of the change detection task (bcd) – the difference spatial attention feature map Fcd. Change detection is then performed on the first region based on the pre-temporal spatial attention feature map Fssa, the post-temporal spatial attention feature map Fssb, and the difference spatial attention feature map Fcd.

[0154] In this embodiment of the invention, semantic segmentation tasks (ssA) and (ssB) can be unified into a single semantic segmentation task (ss). To achieve collaborative optimization between the semantic segmentation task (ss) and the change detection task (bcd), a multi-task loss function design method is proposed, aiming to balance the differences and correlations between the objectives of the two sub-tasks and improve overall detection performance. This loss function consists of three parts: semantic segmentation loss (ss Loss), change detection loss (bcd Loss), and consistency constraint loss for changed regions (Consistency Loss), which are weighted and fused to construct the final total loss function.

[0155] The semantic segmentation loss (ss Loss) is as follows:

[0156] The semantic segmentation task aims to predict the semantic category of each pixel. In this invention, the semantic prediction results (i.e., Fssa and Fssb) of the preceding and following temporal semantic feature maps are subjected to a multi-class cross loss function and supervised training with their corresponding ground truth semantic labels Ya and Yb, as defined below:

[0157]

[0158] In the formula, L ssis the semantic segmentation loss; Lce is the multi-class cross-entropy loss; Fssa is the pre-temporal spatial attention feature map; Ya is the true semantic label of the pre-temporal semantic feature map; Fssb is the post-temporal spatial attention feature map; Yb is the true semantic label of the post-temporal semantic feature map.

[0159] The change detection loss (bcd Loss) is:

[0160] The change detection subtask is a binary classification task used to determine whether a pixel at the same location has changed between two time phases. Considering the common problem of imbalanced positive and negative samples in change detection tasks, a joint design strategy of "binary cross-entropy loss + Dice coefficient loss" is adopted to enhance the model's ability to learn small targets or sparse changes. The loss function is defined as follows:

[0161] L bcd =L BCE (Fc,Yc)+L Dice (Fc,Yc)

[0162] In the formula, L bcd Loss due to change detection; L BCE For binary cross-entropy loss; L Dice is the Dice coefficient loss; Fc is the model-predicted probability map of change; Yc is the true binary change label map.

[0163] The consistency loss for the changed region is:

[0164] Considering that most pixels remain semantically invariant across two temporal phases, a consistency constraint mechanism based on a change mask is designed to enhance the prediction consistency of unchanged regions in the semantic segmentation task ss, avoiding misjudgments due to noise or semantic drift. Specifically, let the preceding temporal index feature map and the following temporal index feature map be the semantic prediction results obtained after applying softmax + argmax to the preceding temporal spatial attention feature map Fssa and the following temporal spatial attention feature map Fssb, respectively. Then, the unchanged region U is selected using Yc, and the semantic pixel semantic consistency loss is defined in this region:

[0165]

[0166] In the formula, L consisctency For consistency constraint loss in the changing region; This is a feature map of the previous time phase index; For the previous phase index feature map; U={(i,j)∣Yc i,j =0} represents the set of pixels in the unchanged region.

[0167] The total loss function is:

[0168] By balancing the contributions of different subtasks to the total loss according to certain weights, the total loss function is defined as follows:

[0169] L 总 =λ ss L ss +λ bcd L bcd +λ cons L consisctency

[0170] In the formula, λ ss For semantic segmentation loss L ss The corresponding loss weight coefficient; λ bcd For change detection loss L bcd The corresponding loss weight coefficient; λ cons For the consistency constraint loss L in the changing region consisctency The corresponding loss weight coefficient.

[0171] In this embodiment of the invention, to ensure efficient convergence and stable optimization of the multi-task change detection model, the following training strategy can be used to learn the model parameters:

[0172] During the training phase, the AdamW optimizer was chosen. Compared to the traditional Adam optimizer, AdamW is more suitable for training Transformer-type structures and can effectively alleviate the coupling problem between weight decay and gradient update. The baseline learning rate was set to 6*10⁻⁴, and the batch size was set to 16. To improve stability in the early stages of training, a joint scheduling strategy of Warm-up + Poly was adopted for the learning rate.

[0173] (1) Warm-up phase: In the first 30 epochs of training, a linear Warm-up strategy is used to increase the learning rate from the initial 0 to the set baseline learning rate of 6*10-4, so as to avoid unstable learning of the model due to gradient oscillation in the initial stage.

[0174] (2) Poly Learning Rate Strategy: After the Warm-up phase (i.e., starting from the 30th epoch), a polynomial decay strategy (Poly LR Schedule) is adopted, gradually reducing the learning rate as the training epochs increase. Throughout the training process, the loss functions of the two sub-tasks, semantic segmentation (SS) and change detection (BCD), are jointly supervised by weighted summation to guide the model to simultaneously learn the ability to determine change regions and extract semantic information, thereby improving the overall detection performance.

[0175] After the model is trained, its performance is evaluated from two aspects: semantic segmentation (SS) and change detection (BCD). Then, the model’s performance in terms of task synergy and overall effectiveness is measured in a unified manner to ensure the objectivity and credibility of the evaluation results.

[0176] For the change detection task (BCD), the focus is on the model's ability to detect changed regions, and the following standard metrics are used for evaluation:

[0177] (1) Precision: The proportion of the detected change area that is actually a change.

[0178]

[0179] In the formula, Precision is the accuracy; TP is the number of true positives; and FP is the number of false positives.

[0180] (2) Recall: The proportion of regions with real changes that are successfully detected.

[0181]

[0182] In the formula, Recall is the recall rate; TP is the number of true negatives; and FN is the number of false negatives.

[0183] (3) F1 score: The harmonic mean of precision and recall, used to comprehensively measure detection performance.

[0184]

[0185] In the formula, F1 is the F1 score; Precision is the precision rate; Recall is the recall rate.

[0186] For the semantic segmentation task (ss), the focus is on the semantic segmentation quality of the model before and after the change, mainly using the mean intersection over union (mIoU) to represent the mean IoU of all semantic categories:

[0187]

[0188] In the formula, mIoU is the average intersection-union ratio; C is the number of semantic categories; TP i Let be the number of true instances in class i; FP i FN is the number of false positives in the i-th class. i Let be the number of false negatives in class i.

[0189] To comprehensively quantify the overall performance of semantic change detection models on semantic segmentation (SS) and change detection (BCD) tasks, a semantic change detection comprehensive score (SCD Score) is designed to uniformly measure the model's performance in task synergy and overall effectiveness. This score combines change detection accuracy and semantic segmentation accuracy, and is defined as follows:

[0190] SCD Score =α·F1 change +β·mIoU semantic

[0191] In the formula, SCD Score F1 is a comprehensive scoring index for semantic change detection. change F1 score for the change detection task; mIoU semantic α represents the average intersection-union ratio (IUU) for semantic segmentation tasks; α and β are weight coefficients, which sum to 1.

[0192] Implementing the above embodiments has the following effects:

[0193] This invention provides a method for detecting semantic changes in remote sensing images. The method acquires a first-phase and a second-phase remote sensing image of a first region. Multi-scale semantic feature extraction is performed on both the first-phase and second-phase images to form a first-phase multi-scale semantic feature map and a second-phase multi-scale semantic feature map. This invention performs multi-scale semantic extraction on the image before extracting change information after acquisition, decoupling semantic segmentation and change detection tasks and improving the stability of semantic extraction. Weighted shared features are extracted from the first-phase and second-phase multi-scale semantic feature maps to obtain a first-phase shared feature map and a second-phase shared feature map. Absolute difference is performed between the first-phase and second-phase multi-scale semantic feature maps to obtain a first difference feature map. Absolute difference is also performed between the first-phase and second-phase shared feature maps to obtain a second difference feature map. Difference enhancement processing is then performed based on the first and second difference feature maps to obtain an enhanced difference feature map. This invention generates two sets of distinct difference feature maps by performing absolute difference analysis on two different types of feature maps. The extracted difference feature maps are then enhanced to fully extract the difference information from the changed regions. Based on the enhanced difference feature maps, irrelevant information is filtered from the shared feature maps of the preceding and following time phases to obtain the change detection result for the first region. This invention improves the accuracy of the extracted features by decoupling semantic extraction and change detection tasks and by performing multi-scale difference extraction. By using the feature maps obtained from semantic extraction and difference information extraction to perform change detection on the first region, the accuracy of change detection in remote sensing images can be effectively improved.

[0194] like Figure 6As shown, based on the above method embodiments, corresponding apparatus embodiments are provided;

[0195] An embodiment of the present invention provides a remote sensing image semantic change detection device, comprising: an image acquisition module, a semantic extraction module, a shared feature extraction module, a first absolute difference module, a second absolute difference module, a difference enhancement module, and a detection result generation module;

[0196] The image acquisition module is used to acquire the previous and subsequent time-phase remote sensing images of the first region;

[0197] The semantic extraction module is used to extract multi-scale semantic features from the preceding and subsequent time-phase remote sensing images respectively, forming a preceding time-phase multi-scale semantic feature map and a subsequent time-phase multi-scale semantic feature map.

[0198] The shared feature extraction module is used to perform weighted shared feature extraction on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map to obtain the pre-temporal shared feature map and the post-temporal shared feature map.

[0199] The first absolute difference module is used to perform absolute difference on the previous phase multi-scale semantic feature map and the subsequent phase multi-scale semantic feature map to obtain the first difference feature map;

[0200] The second absolute difference module is used to perform absolute difference on the pre-phase shared feature map and the post-phase shared feature map to obtain a second difference feature map;

[0201] The difference enhancement module is used to perform difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map;

[0202] The detection result generation module is used to filter irrelevant information from the pre-phase shared feature map and the post-phase shared feature map based on the enhanced difference feature map, and obtain the change detection result of the first region.

[0203] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the remote sensing image semantic change detection method provided by any of the above-described method embodiments of the present invention.

[0204] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0205] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for detecting semantic changes in remote sensing images, characterized in that, include: Acquire the pre-temporal and post-temporal remote sensing images of the first region; Multi-scale semantic features are extracted from the preceding and subsequent time-phase remote sensing images to form a preceding time-phase multi-scale semantic feature map and a subsequent time-phase multi-scale semantic feature map. Weighted shared features are extracted from the preceding and following phases of the multi-scale semantic feature map to obtain the preceding and following phases of the shared feature map. The first difference feature map is obtained by performing absolute difference on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map. The absolute difference between the pre-phase shared feature map and the post-phase shared feature map is obtained to produce a second difference feature map. Based on the first difference feature map and the second difference feature map, difference enhancement processing is performed to obtain an enhanced difference feature map; Based on the enhanced difference feature map, irrelevant information is filtered from the pre-phase shared feature map and the post-phase shared feature map to obtain the change detection result of the first region.

2. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The step of extracting multi-scale semantic features from the preceding and following time-phase remote sensing images to form preceding and following time-phase multi-scale semantic feature maps includes: Multi-scale semantic feature extraction is performed on the preceding and following remote sensing images to form several preceding semantic feature maps and several following semantic feature maps. Several preceding temporal semantic feature maps are fused to form a preceding temporal multi-scale semantic feature map; Several post-temporal semantic feature maps are fused to form a post-temporal multi-scale semantic feature map.

3. The method for detecting semantic changes in remote sensing images according to claim 1, characterized in that, The step of performing difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map includes: Spatial focusing processing is performed on the second difference feature map to obtain the first spatial focusing feature map; The first difference feature map and the first spatial focus feature map are multiplied element by element to obtain the third difference feature map; The third difference feature map is upsampled to obtain the fourth difference feature map; The third and fourth difference feature maps are added element by element to obtain the fifth difference feature map; Cross-layer attention integration is performed on the fifth difference feature map to obtain an enhanced difference feature map.

4. The method for detecting semantic changes in remote sensing images according to claim 3, characterized in that, The step of filtering irrelevant information from the pre-temporal shared feature map and the post-temporal shared feature map based on the enhanced difference feature map to obtain the change detection result of the first region includes: The enhanced difference feature map is subjected to spatial focusing processing to obtain a second spatial focusing feature map; The second spatial focusing feature map is multiplied element-wise with the pre-temporal shared feature map, the post-temporal shared feature map, and the enhanced difference feature map to obtain the pre-temporal spatial attention feature map, the post-temporal spatial attention feature map, and the difference spatial attention feature map. Based on the difference spatial attention feature map, irrelevant information is filtered from the previous temporal spatial attention feature map and the subsequent temporal spatial attention feature map respectively to obtain the change detection result of the first region.

5. The method for detecting semantic changes in remote sensing images according to claim 4, characterized in that, The step of filtering irrelevant information from the preceding and following temporal spatial attention feature maps based on the difference spatial attention feature map, and obtaining the change detection result of the first region, includes: A nonlinear mapping is performed on the difference space attention feature map to obtain a change probability map; The change probability map is binarized to obtain the binarized result; Based on the binarization result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are masked to obtain the change detection result of the first region.

6. The method for detecting semantic changes in remote sensing images according to claim 5, characterized in that, Based on the binarization result, the preceding temporal spatial attention feature map and the following temporal spatial attention feature map are respectively masked to obtain the change detection result of the first region, including: On several channels of the preceding temporal spatial attention feature map, the maximum value index of each pixel position in the preceding temporal spatial attention feature map is calculated respectively to form the preceding temporal index feature map; On several channels of the post-temporal spatial attention feature map, the maximum value index of each pixel position in the post-temporal spatial attention feature map is calculated to form a post-temporal index feature map; Based on the binarization result, the preceding time-phase index feature map and the following time-phase index feature map are masked respectively to obtain the change detection result of the first region.

7. The method for detecting semantic changes in remote sensing images according to claim 6, characterized in that, The step of calculating the maximum value index of each pixel position in the preceding temporal spatial attention feature map on several channels of the preceding temporal spatial attention feature map to form a preceding temporal index feature map includes: For each pixel position, calculate the value of the pixel position in each channel of the preceding temporal spatial attention feature map; The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position; For the aforementioned pre-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position order to form the pre-temporal index feature map.

8. The method for detecting semantic changes in remote sensing images according to claim 6, characterized in that, The step of calculating the maximum value index of each pixel position in the post-temporal spatial attention feature map on several channels of the post-temporal spatial attention feature map to form a post-temporal index feature map includes: For each pixel position, calculate the value of the pixel position in each channel of the post-temporal spatial attention feature map; The channel number corresponding to the channel with the largest value is determined as the maximum value index of the pixel position; For the post-temporal spatial attention feature map, the maximum value indices of all pixels are organized into a two-dimensional matrix according to their spatial position to form the post-temporal index feature map.

9. The method for detecting semantic changes in remote sensing images according to claim 6, characterized in that, Based on the binarization result, the preceding temporal index feature map and the following temporal index feature map are respectively masked to obtain the change detection result of the first region, including: Based on the binarization result, the previous temporal index feature map is masked to obtain the previous land cover category of the first region. Based on the binarization result, the post-temporal index feature map is masked to obtain the changed land cover category of the first region; Based on the land cover categories before and after the change, the changed areas and land cover changes in the first region are determined.

10. A device for detecting semantic changes in remote sensing images, characterized in that, include: The system includes an image acquisition module, a semantic extraction module, a shared feature extraction module, a first absolute difference module, a second absolute difference module, a difference enhancement module, and a detection result generation module. The image acquisition module is used to acquire the previous and subsequent time-phase remote sensing images of the first region; The semantic extraction module is used to extract multi-scale semantic features from the preceding and subsequent time-phase remote sensing images respectively, forming a preceding time-phase multi-scale semantic feature map and a subsequent time-phase multi-scale semantic feature map. The shared feature extraction module is used to perform weighted shared feature extraction on the pre-temporal multi-scale semantic feature map and the post-temporal multi-scale semantic feature map to obtain the pre-temporal shared feature map and the post-temporal shared feature map. The first absolute difference module is used to perform absolute difference on the previous phase multi-scale semantic feature map and the subsequent phase multi-scale semantic feature map to obtain the first difference feature map; The second absolute difference module is used to perform absolute difference on the pre-phase shared feature map and the post-phase shared feature map to obtain a second difference feature map; The difference enhancement module is used to perform difference enhancement processing based on the first difference feature map and the second difference feature map to obtain an enhanced difference feature map; The detection result generation module is used to filter irrelevant information from the pre-phase shared feature map and the post-phase shared feature map based on the enhanced difference feature map, and obtain the change detection result of the first region.

Images