Cross-scale feature fusion remote sensing image change detection method and system, medium, equipment and product
By employing a cross-scale feature fusion method, combined with FastSAM with frozen weights and an attention mechanism, the problems of missed detection of small-area changes and interference from false changes in remote sensing image change detection are solved, achieving efficient identification and detection of change regions.
Patent Information
- Application Number
- CN202510977391.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-16
- Publication Date
- 2025-12-19
AI Technical Summary
Existing remote sensing image change detection methods are prone to missing small-area changes and are susceptible to spurious changes, making it difficult to simultaneously guarantee recall and accuracy. This is especially true in high-resolution remote sensing scenarios where real-time deployment is challenging.
A cross-scale feature fusion method is adopted, which combines FastSAM with frozen weights, row and column attention and coordinate attention mechanisms. Through feature extraction, fusion, differential feature extraction and channel expansion optimization modules, the model’s ability to perceive local changes and global dependencies is improved, and the change detection mask map is output by the decoder.
It effectively improves the detection performance of fine-grained local change regions, reduces false difference misjudgments, improves the recognition accuracy and detection capability of change regions, and reduces the computational resource overhead.
Smart Images
Figure CN121170614A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, system, medium, device, and product for detecting changes in remote sensing images through cross-scale feature fusion. Background Technology
[0002] Remote sensing satellite platforms are capable of observing the Earth on a large scale and continuously acquiring massive amounts of image data, making comprehensive, all-weather, and multi-dimensional surface monitoring possible. In recent years, high-resolution remote sensing imagery (sub-meter to centimeter level) has become increasingly widespread, and its rich texture, geometric, and spectral information provides refined data support for applications such as land use classification, disaster assessment, and urban planning. Among these, change detection tasks, which identify changes in land features in the same area by comparing two-temporal images, are an important technical approach to maintaining the timeliness and integrity of geographic information databases and to promptly grasp surface dynamics.
[0003] However, high-resolution remote sensing images themselves are characterized by multi-scale target mixing, significant illumination differences, and frequent interference from clouds, fog, and shadows. Traditional change detection methods struggle to simultaneously achieve accuracy, robustness, and real-time performance, mainly in the following aspects: Classical methods based on pixel difference and statistical features (such as image difference, vector change analysis, and principal component analysis) rely on simple algebraic or statistical operations. These algorithms are easily affected by factors such as sensor noise, lighting, and seasonal vegetation changes, resulting in a large number of pseudo-changes. At the same time, they lack spatial context modeling capabilities and are insufficient in identifying structural changes in complex scenes.
[0004] End-to-end methods based on convolutional neural networks (CNNs), such as UNet and Siamese-CNN, have significantly improved detection accuracy on multiple public datasets by extracting hierarchical features through deep convolutions. However, CNNs have limited receptive fields, making it difficult to take into account both fine-grained small targets and large-scale macroscopic changes, and they are still vulnerable to pseudo-changes such as shadows and road reflections.
[0005] Hybrid or pure Transformer methods incorporating Vision Transformer (ViT), such as VisionTransformer and Swing Transformer, improve multi-scale contextual representation capabilities by explicitly modeling long-range dependencies through self-attention mechanisms. However, pure Transformers lack local inductive biases, which can easily lead to blurred edges; at the same time, their parameter count, computational cost, and memory overhead are significantly increased, limiting their real-time deployment in ultra-high resolution remote sensing scenarios.
[0006] Furthermore, recent advancements such as Segment Anything Model (SAM) and FastSAM have achieved open-world segmentation capabilities in the field of natural images, providing a new paradigm for downstream vision tasks. However, the pre-training data for these models mainly comes from natural scenes, resulting in a significant "domain bias" problem when applied to remote sensing images—the shooting angle, structural scale, and spectral distribution of ground objects differ significantly from those in natural images. Direct transfer and application often leads to problems such as feature mismatch, low inference efficiency, and missed detection of small targets.
[0007] In real-world tasks, the area of changed regions is often much smaller than that of unchanged regions (the ratio of positive to negative samples can reach 1:20 or more). Existing loss functions or sampling strategies often struggle to simultaneously guarantee recall and precision, leading to missed detections of small-area changes or false detections of large-area spurious changes; shadows, fog, and seasonal vegetation changes further exacerbate the difficulty of identifying changed regions. Summary of the Invention
[0008] The purpose of this invention is to address the problem of missed detection of small-area changes in remote sensing image change detection in existing technologies, and to propose a cross-scale feature fusion remote sensing image change detection method, comprising the following steps: S1. Acquire two temporal remote sensing images of the same area at two different times, and preprocess the two temporal remote sensing images. S2. Construct a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module; The feature extraction module was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images. The feature fusion module performs multi-scale feature fusion on the multi-scale feature maps of the two temporal remote sensing images to obtain multi-scale fused feature maps of the two temporal images. The multi-scale fused feature maps of the two time phases are added at the corresponding scales, and the difference feature map is obtained by the difference feature extraction module. The differential feature map is input into the dilation channel optimization module. The differential feature map is initially optimized by separable convolution with different dilation rates, and then the initially optimized differential feature map is weighted and optimized by combining the channel attention mechanism. The weighted and optimized difference feature map is input into the decoder module and gradually upsampled to the original image size to obtain the final change detection mask map; S3. Train the model and use the trained model for remote sensing image change detection.
[0009] Furthermore, the pre-trained FastSAM with frozen weights was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images.
[0010] Furthermore, the differential feature extraction module includes a row and column attention mechanism branch and a coordinate attention mechanism branch; The output features of row and column attention obtained from the two branches and the output features of coordinate attention are multiplied pixel by pixel to obtain the joint attention output features:
[0011] in, This represents the joint attention output features of the two time phases. The output features representing row and column attention in two time phases. The coordinate attention output features represent the two temporal phases; Joint attention output features Two series-connected Extracting nonlinear relationships in convolution operations:
[0012] in, This represents the characteristic representation under two time phases. , Indicates two consecutive Convolutional layers, where ReLU represents a non-linear activation function; Will and By adding each pixel individually, we obtain the difference feature map:
[0013] in, This represents a difference feature map.
[0014] Furthermore, the multi-scale feature map of the dual-temporal remote sensing image has four scales, and the fused features of the four different scales are as follows:
[0015]
[0016]
[0017]
[0018] in This indicates a convolution operation used to adjust the feature dimensions; This indicates a bilinear upsampling operation; This indicates a feature concatenation operation performed on two features; Indicates feature fusion, Two layers Convolutional structures are used to unify and integrate multi-scale information; This indicates a max-pooling downsampling operation; This indicates that a feature concatenation operation is performed on three features. , These are feature maps at different scales for two different time phases. This represents the feature map before the i-th scale change. This represents the feature map after the i-th scale change. , and These are fused feature maps at different scales from two different time phases.
[0019] Furthermore, the differential feature map is initially optimized through separable convolutions with different dilation rates, specifically as follows: The difference feature map is divided into low-level difference features based on the number of channels and size. Differences between high-level and high-level characteristics Transform the high-level features:
[0020] in, This represents the transformed high-level features. Indicates an upsampling operation. express Convolution operation, Represents the ReLU activation function. This indicates a batch normalization operation; The transformed high-level features are added element-wise to the low-level features to obtain preliminary fused features. :
[0021] Preliminary fusion characteristics The channel dimension is divided into 4 subsets, which are then fed into convolutional branches with different dilation rates for processing. Let the first subset be... The subset is The corresponding void ratio is Each branch uses depthwise separable convolution for feature extraction, represented as:
[0022] in, Indicates the void ratio of Dilated convolution operation, This represents the output of the k-th branch; The preliminary optimized difference feature map is F:
[0023] in, This indicates that the outputs of the four branches are concatenated.
[0024] Furthermore, the weighted optimization of the initially optimized differential feature map is performed using a channel attention mechanism, specifically as follows:
[0025]
[0026] in, Represents the weight vector. This indicates the average pooling operation. This indicates a max pooling operation. This represents the convolution operation. express Activation function This represents the Sigmoid function. This represents the difference feature map after weighted optimization.
[0027] This invention also proposes a cross-scale feature fusion remote sensing image change detection system, comprising: The data processing module is used to acquire two-temporal remote sensing images of the same area at two different times and to preprocess the two-temporal remote sensing images. The model building module is used to build a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module; The feature extraction module was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images. The feature fusion module performs multi-scale feature fusion on the corresponding multi-scale feature maps of the two-temporal remote sensing images to obtain multi-scale fused feature maps of the two temporal phases. The multi-scale fused feature maps of the two time phases are added at the corresponding scales, and the difference feature map is obtained by the difference feature extraction module. The differential feature map is input into the dilation channel optimization module. The differential feature map is initially optimized by separable convolution with different dilation rates, and then the initially optimized differential feature map is weighted and optimized by combining the channel attention mechanism. The weighted and optimized difference feature map is input into the decoder module and gradually upsampled to the original image size to obtain the final change detection mask map; The remote sensing image change detection module is used to train the model, and then use the trained model for remote sensing image change detection.
[0028] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described cross-scale feature fusion remote sensing image change detection method.
[0029] The present invention also proposes an electronic device, including a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to execute the above-described cross-scale feature fusion remote sensing image change detection method.
[0030] The present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described cross-scale feature fusion remote sensing image change detection method.
[0031] The beneficial effects of the technical solution provided by this invention are: This invention employs FastSAM with frozen weights to extract multi-scale features from dual-temporal images. By fusing features at different scales and combining row / column attention and coordinate attention mechanisms, the model's ability to perceive local changes and global dependencies is enhanced. This allows the feature map to retain channel representation capabilities while also possessing spatial orientation awareness. This mechanism helps the model accurately identify the spatial location and changes of targets, effectively improving the detection performance for fine-grained local change regions. An expanded channel optimization module is designed, utilizing multi-scale dilated convolution and channel attention mechanisms to optimize differential features, strengthening contextual information representation and effectively improving the model's ability to detect complex change patterns and small target changes, while reducing false positives caused by spurious differences. Finally, the change detection results are output through a decoder. Attached Figure Description
[0032] Figure 1 This is a flowchart of a cross-scale feature fusion remote sensing image change detection method according to an embodiment of the present invention; Figure 2 This is a block diagram of an electronic device according to an exemplary embodiment of Embodiment 1 of the present invention; Figure 3 This is a qualitative comparison of the method of this invention with other models. S1-S4 are four different dual-temporal remote sensing images and the results of remote sensing image change detection. (a) and (b) are dual-temporal remote sensing images. (c)-(k) are the results of remote sensing image change detection obtained by FC-FF, FC-Siamconc, FC-Siamdiff, SNU-Net48, STANet, HANet, BIT, Change-Former and the method of this invention, respectively. (i) is the actual remote sensing image change result. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0034] The flowchart of the cross-scale feature fusion remote sensing image change detection method according to an embodiment of the present invention is as follows: Figure 1 Specifically, it includes the following steps: S1. Acquire two temporal remote sensing images of the same area at two different times, and preprocess the two temporal remote sensing images.
[0035] Bi-temporal remote sensing imagery comprises both pre- and post-change images. Image sources can include aerial imagery, satellite imagery, or remote sensing data acquired by UAVs. The imaging time interval can be set to several hours, days, or months depending on application requirements. Preprocessing includes: precise geometric registration of the bi-temporal remote sensing images to ensure pixel-level correspondence; and normalization and histogram matching operations on the bi-temporal images to improve the model's sensitivity to real-world changes.
[0036] S2. Construct a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module.
[0037] (1) Feature extraction module In a preferred embodiment of the present invention, a pre-trained FastSAM with frozen weight parameters is used to extract multi-scale feature maps. During model training, the weight parameters of FastSAM are frozen, and only the parameters of other modules are updated. The purpose of freezing its weights is to utilize its robust image understanding capabilities as the basis for feature extraction and avoid feature degradation caused by downstream task training. Dual-temporal remote sensing images are input into FastSAM, which outputs feature maps at two different scales from the two temporal phases. and These represent multiple layers of information, from local details to global semantics, including feature layers at 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size. This represents the feature map before the i-th scale change. This represents the feature map after the i-th scale change. .
[0038] (2) Feature fusion module The feature fusion module fuses the multi-scale feature maps of corresponding time phases of the dual-temporal remote sensing images at different scales to obtain fused features at different scales of the two time phases.
[0039] In this embodiment of the invention, feature maps at two different time scales are used. After multi-scale fusion, fused feature maps of two different temporal scales are obtained. and Each path achieves spatial size alignment through convolution, upsampling, or downsampling operations, and utilizes a feature fusion module to extract multi-layer information to enhance expressive power.
[0040] Fusion features j=a,b is a pair j=a,b and The result of feature fusion for j=a,b is shown in the following formula:
[0041]
[0042]
[0043] Fusion features Yes , and The result of feature fusion is shown in the following formula:
[0044]
[0045]
[0046] Fusion features Yes , and The result of feature fusion is shown in the following formula:
[0047]
[0048]
[0049] Fusion features Yes and The result of feature fusion. The formula is as follows:
[0050]
[0051]
[0052] in This indicates a convolution operation used to adjust the feature dimensions; This indicates a bilinear upsampling operation. Will Upsampling to Same spatial dimensions; This indicates a feature concatenation operation performed on two features; Indicates feature fusion, Two layers Convolutional structures are used to unify and integrate multi-scale information; This indicates a max-pooling downsampling operation; This indicates that a feature concatenation operation is performed on three features.
[0053] (3) Differential Feature Extraction Module To further enhance the model's ability to model spatial structure changes, fused features from two different time phases at different scales were obtained. Subsequently, the fused features from different scales of the two time phases are input into the row-column attention mechanism and the coordinate attention mechanism, respectively. The row-column attention mechanism effectively improves the global perception capability of features by modeling the long-range dependencies between features in both row and column directions.
[0054] Fusion features of row and column attention mechanism input or use It means that, among them These represent batch size, number of channels, image height, and width, respectively. The enhanced features with the same dimensions as the input are processed in subsequent steps using a row and column attention mechanism to operate on the aggregated feature map at each scale.
[0055] The specific steps are as follows: ① Row attention mechanisms model the dependencies between positions within each row of the feature map. First, through... Convolution generates query, key, and value vectors. , and :
[0056] Where X represents the input feature map of the row and column attention mechanism, This represents a convolution operation on the query vector. This represents a convolution operation on the key vector. This represents a convolution operation on a value vector.
[0057] ② Rearrange the tensors in the channel and height dimensions to enable self-attention computation along the row direction (width):
[0058]
[0059] in, and These represent the query vector and key vector after rearrangement along the channel and height dimensions, respectively. This indicates a shape reshaping operation.
[0060] ③ Calculate the attention matrix And it acts on the value vector The line attention output is restored to its original shape. , to obtain line attention :
[0061]
[0062] Similarly, by modeling the dependencies between positions within each column (column-direction attention), we obtain column attention. .
[0063] ④ Output characteristics of row and column attention mechanism The following is obtained by weighted fusion of row and column attention results and input feature maps:
[0064] in, It is a learnable scalar parameter used to adjust the balance between attention output and original features.
[0065] To further enhance the spatial modeling capability of feature maps, for input features A spatial awareness enhancement branch, namely the coordinate attention mechanism, is introduced. This mechanism enhances the model's ability to capture spatial location information by introducing explicit positional information, while preserving the channel feature representation capability, thus helping to accurately locate fine-grained variation regions. The specific steps are as follows: ① Input feature map is ,in These represent batch size, number of channels, image height, and width, respectively. The coordinate attention mechanism first performs orientation-aware global average pooling in the spatial dimension, extracting contextual features in the horizontal and vertical directions respectively. and :
[0066]
[0067] in, This means fixing the i-th row in the height dimension, that is, selecting the feature data of all columns in the i-th row of the feature map, thereby obtaining complete horizontal information of all channels at that row position. This indicates that the j-th column is fixed in the width dimension, meaning that feature data from all rows in the j-th column of the feature map are selected to obtain complete vertical information for that column across all channels. The encoding of the two directions forms the key inputs used in the coordinate attention mechanism to model spatial location dependencies.
[0068] ② Contextual features in the horizontal and vertical directions and After splicing, send it to the shared [system / platform]. Convolution compresses the channel dimension:
[0069] in, This represents an activation function, such as ReLU or Swish. This indicates splicing along spatial dimensions.
[0070] ③ Split the output f into two attention branches, each corresponding to a horizontal attention weight. With vertical attention weights :
[0071]
[0072] in, This is the Sigmoid function.
[0073] ④ Coordinate attention is applied to the input feature X in an element-wise weighted manner to obtain the coordinate attention output feature:
[0074] The above operations explicitly model long-term dependencies in spatial directions, effectively improving the spatial expressiveness of the model.
[0075] To further enhance the fusion expressive power between attention-enhanced features and original features, after obtaining the initially optimized fusion features... and Subsequently, this invention further performs a pixel-by-pixel multiplication operation on the two, that is, constructs a joint attention output feature:
[0076] in, This represents the Hadamard multiplication per pixel, used to jointly model the spatial structure and location information extracted by row and column attention and coordinate attention.
[0077] Subsequently, the joint attention output features will be... Two series-connected Convolution operations are used to further extract the nonlinear relationships between fused features and improve the model's ability to model spatially varying structures. The specific operations are as follows:
[0078] in: , Indicates two consecutive Convolutional layers, where ReLU stands for non-linear activation function, are used to enhance feature representation.
[0079] Final output This represents the feature representations generated in two time phases. In the change detection task, the model employs a dual-branch structure to process the images before and after the change, outputting the corresponding features for each of the two time phases. and Subsequently, the two are added pixel by pixel to obtain a difference feature map that explicitly represents temporal changes:
[0080] This multi-scale difference feature map It will be sent to the subsequent decoder module, where the change detection mask will be restored by gradually upsampling, so as to achieve accurate identification and positioning of the changed area.
[0081] (4) Expansion Channel Optimization Module The differential feature maps are input into the channel expansion optimization module. Separable convolutions with different dilation rates enhance the model's sensitivity to changing regions, and a channel attention mechanism is used to perform weighted optimization of the feature maps. The channel expansion optimization module adaptively fuses differential feature maps from different scales, using high-level contextual information to guide low-level detail recovery. On the other hand, multi-scale convolution extraction is introduced to expand the spatial receptive field of features and optimize channel representation, thereby better detecting changing regions at different scales and suppressing spurious change interference.
[0082] The specific steps are as follows: ① Difference feature map Based on the number of channels and size, it is divided into low-level differential characteristics. Differences between high-level and high-level characteristics Let the input be low-level differential features and high-level differential features. First, use bilinear interpolation to... Upsampling to Same space size, and through Convolution aligns its channel dimensions to obtain the transformed high-level features. :
[0083] in, Indicates an upsampling operation. express Convolution operation, Represents the ReLU activation function. This indicates batch normalization operation.
[0084] ② The processed high-level features are added element-wise to the low-level features to obtain preliminary fused features:
[0085] ③ To improve the receptive field while maintaining computational efficiency, the channel expansion optimization module optimizes the fused features. The system is divided into four subsets along the channel dimension, and each subset is fed into a convolutional branch with a different dilation rate for processing. Let the first subset be... The subset is The corresponding void ratio is Each branch uses depthwise separable convolution for feature extraction, represented as:
[0086] in Indicates a void ratio of Depthwise convolution operation, followed by concatenation Convolution is used for channel integration.
[0087] ④ Concatenate the outputs of the four branches to obtain multi-scale features:
[0088] in, This indicates a concatenation operation between the four branch outputs. Finally, use... Pointwise convolution pairs Channel fusion and nonlinear mapping are performed to obtain a preliminary optimized differential feature map F:
[0089] The expansion channel optimization module realizes the deep fusion of features at different semantic levels and the adaptive integration of multi-scale contexts, effectively improving the model's ability to express fine-grained changes and complex interferences.
[0090] ⑤ To further improve the discrimination ability, a channel attention mechanism is introduced. The channel descriptions of the initially optimized differential feature map F are extracted by max pooling and average pooling, respectively, and a weight vector is generated. Used to guide the weighted fusion of high- and low-level features:
[0091]
[0092] in, Represents the weight vector. This indicates the average pooling operation. This indicates a max pooling operation. This represents the convolution operation. express Activation function This represents the Sigmoid function. This represents the difference feature map after weighted optimization.
[0093] Final output fusion features It retains both low-level detailed information and high-level semantic information, and achieves adaptive information selection through channel dimension weighting, effectively improving the model's ability to perceive and express changing regions.
[0094] (5) Decoder module To obtain the final change detection result, the optimized feature map is input into the decoder and progressively upsampled to the original image size to obtain the final change detection mask. In remote sensing change detection tasks, class imbalance is often a serious problem, meaning that changed regions (positive samples) are much smaller than unchanged regions (negative samples). This imbalance can cause the network to favor negative samples during training, thus ignoring important changed regions. To alleviate this problem, this invention employs a combination of binary cross-entropy loss. With dice loss The hybrid loss function is used to comprehensively optimize pixel classification accuracy and region overlap.
[0095] Hybrid loss function The expression is as follows:
[0096] in, The pixel-wise error between the predicted probability and the true label is defined as:
[0097] in, Indicates the total number of pixels. For the first The real label of each pixel for The predicted probability.
[0098] This measure assesses the degree of overlap between the predicted results and the labels, and is more robust in handling class imbalance problems. Its definition is as follows:
[0099] in, To avoid the smoothing term introduced by a denominator of zero, a small constant is usually chosen (e.g., ...). ).
[0100] This hybrid loss function utilizes BCE to improve overall classification performance and combines it with Dice to enhance sensitivity to the minority class (variable regions), thereby achieving balanced modeling of positive and negative samples during training and effectively improving the model's change detection capability.
[0101] This invention also proposes a cross-scale feature fusion remote sensing image change detection system, comprising: The data processing module is used to acquire two-temporal remote sensing images of the same area at two different times and to preprocess the two-temporal remote sensing images. The model building module is used to build a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module; The feature extraction module was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images. The feature fusion module performs multi-scale feature fusion on the corresponding multi-scale feature maps of the two-temporal remote sensing images to obtain multi-scale fused feature maps of the two temporal phases. The multi-scale fused feature maps of the two time phases are added at the corresponding scales, and the difference feature map is obtained by the difference feature extraction module. The differential feature map is input into the dilation channel optimization module. The differential feature map is initially optimized by separable convolution with different dilation rates, and then the initially optimized differential feature map is weighted and optimized by combining the channel attention mechanism. The weighted and optimized difference feature map is input into the decoder module and gradually upsampled to the original image size to obtain the final change detection mask map; The remote sensing image change detection module is used to train the model, and then use the trained model for remote sensing image change detection.
[0102] In one exemplary embodiment, a computer-readable storage medium is included, which stores a computer program that, when executed by a processor, implements the aforementioned cross-scale feature fusion remote sensing image change detection method.
[0103] Please see Figure 2 In one exemplary embodiment, the device further includes an electronic device including at least one processor, at least one memory, and at least one communication bus.
[0104] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the aforementioned cross-scale feature fusion remote sensing image change detection method.
[0105] In one exemplary embodiment, a computer program product is proposed, including a computer program / instructions that, when executed by a processor, implement the steps of the cross-scale feature fusion remote sensing image change detection method described above.
[0106] To verify the effectiveness of the method of the present invention, the method of the present invention is compared with eight algorithms: FC-FF, FC-Siamconc, FC-Siamdiff, SNU-Net48, STANet, HANet, BIT, and Change-Former. Among them, FC-FF (Fully Convolutional Siamese Networks for Change Detection) adopts an early fusion strategy, with a network architecture based on UNet. The original bitemporal images are concatenated as network input and processed by a single-stream convolutional network to detect changes. FC-Siamconc (Fully Convolutional Siamese Networks for Change Detection) adopts a post-fusion strategy based on the FC-EF network, extracting multi-scale features from the bitemporal images' biconvolutional networks and using algebraic operations to obtain disparity features for change detection. FC-Siamdiff (Fully Convolutional Siamese Networks for Change Detection) employs a multi-scale feature-level concatenation method. This method differs from FC-Siam-Diff in that the multi-scale features extracted from the biconvolutional networks are concatenated in the channel dimension for change detection. SNU-Net48 (Change Detection in Remote Sensing Images Based on Convolutional Neural Network and Multiscale Conditional Random) The Field (Field) maintains high resolution and fine-grained representation through dense skip connections between the encoder and decoder, as well as between decoders. In addition, it proposes an integrated channel attention module (ECAM) to refine the most representative features at different levels for final classification. The core architecture of STANet (Spatial-Temporal Attention-Based Method for Remote Sensing Image ChangeDetection) is built on the basis of ResNet18 and introduces a self-attention mechanism to calculate the spatial and temporal correlation between two-temporal remote sensing images. This design aims to better capture subtle differences in two-temporal images, thereby more accurately detecting changed regions.HANet (HANet: Hybrid Attention Network for Change Detection in Remote Sensing Images) is a hierarchical attention network for change detection in high-resolution remote sensing images. Through a progressive foreground balancing sampling strategy and a lightweight self-attention mechanism, the HAN module effectively addresses the sample imbalance problem and improves detection performance. BIT (Remote Sensing Image Change Detection with Bi-Temporal Image Transformer) uses a late-stage fusion strategy, adding a transformer as an encoder at the top of the convolutional network to model spatial features to better simulate context. The learned context-rich tokens are fed back into the pixel space to detect changes. Change-Former (ChangeFormer: A Transformer-Based Model for Change Detection in VHR Remote Sensing Images) is a purely transformer-based Siamese network. Its backbone consists entirely of a Transformer Encoder, without using a CNN. It employs an MLP as the decoder, unifying the architecture within the CD transformer framework.
[0107] The complexity of each model was measured by floating-point operations (FLOPs) and the number of parameters. Based on the LEVIR-CD remote sensing building change detection dataset, the models were compared using several evaluation metrics, including overall accuracy (OA), intersection over union (IoU), F1 score, precision (pre), and recall (Rec). Table 1 shows the quantitative comparison results of the proposed method with other models. As can be seen from Table 1, the proposed method achieved optimal performance on the LEVIR-CD dataset across multiple metrics, with an overall accuracy (OA) of 99.03%, an IoU of 82.39%, an F1 score of 90.34%, and a precision of 91.80%, all outperforming existing mainstream methods such as ChangeFormer, BIT, and HANet. Simultaneously, it maintained a high recall rate (88.94%), differing from the current best value by only 0.42%. In terms of computational complexity, the method of this invention has a FLOPs of 6.03 G and a parameter count of 14.50 M, which is at a moderate level. The FLOPs of this invention (6.03 G) are significantly lower than those of BIT (15.17 G) and HANet (14.07 G), significantly reducing computational resource consumption while maintaining accuracy advantages, demonstrating strong practicality and deployment advantages. In summary, the method of this invention achieves comprehensive leadership in change detection accuracy while maintaining low computational cost, demonstrating the superiority and practical application potential of the proposed method in remote sensing change detection tasks.
[0108] Table 1
[0109] Qualitative comparison results of the method of this invention with other models (see figure) Figure 3S1-S4 are four different dual-temporal remote sensing images and remote sensing image change detection results. (a) and (b) are dual-temporal remote sensing images. (c)-(k) are remote sensing image change detection results obtained by FC-FF, FC-Siamconc, FC-Siamdiff, SNU-Net48, STANet, HANet, BIT, Change-Former and the method of this invention, respectively. (l) is the actual remote sensing image change result. TP (True Positive) indicates that the classifier predicts a positive sample and it is also a positive sample, that is, the number of positive samples correctly identified; FP (False Positive) indicates that the classifier predicts a positive sample and it is actually a negative sample, that is, the number of falsely reported negative samples; TN (True Negative) indicates that the classifier predicts a negative sample and it is actually a negative sample, that is, the number of negative samples correctly identified; FN (False Negative) indicates that the classifier predicts a negative sample and it is actually a positive sample, that is, the number of missed positive samples.
[0110] from Figure 3 It can be seen that traditional methods (such as FC-EF and FC-Siam series) exhibit significant false positives (red areas) and false negatives (green areas), especially at building edges and small target areas, where change detection results are rather blurry and lack good boundary preservation capabilities. Moderately complex models such as SNU-Net48 and STANet improve the false positive problem to some extent, but still suffer from fragmented change regions and discontinuous edges. High-performance models such as HANet, BIT, and Change-Former have strong change perception capabilities in large-scale regions and high overall prediction quality, but may still exhibit boundary misjudgment or false negatives for small targets in complex backgrounds. In contrast, the method of this invention can accurately locate change regions in multiple scenarios (S1–S4), significantly reducing false positives and false negatives, and the detection results are highly consistent with the actual change results, demonstrating superior performance in maintaining edge continuity and detail integrity.
[0111] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting changes in remote sensing images across scales using feature fusion, characterized in that, Includes the following steps: S1. Acquire two temporal remote sensing images of the same area at two different times, and preprocess the two temporal remote sensing images. S2. Construct a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module; The feature extraction module was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images. The feature fusion module performs multi-scale feature fusion on the corresponding multi-scale feature maps of the two-temporal remote sensing images to obtain multi-scale fused feature maps of the two temporal phases. The multi-scale fused feature maps of the two time phases are added at the corresponding scales, and the difference feature map is obtained by the difference feature extraction module. The differential feature map is input into the dilation channel optimization module. The differential feature map is initially optimized by separable convolution with different dilation rates, and then the initially optimized differential feature map is weighted and optimized by combining the channel attention mechanism. The weighted and optimized difference feature map is input into the decoder module and gradually upsampled to the original image size to obtain the final change detection mask map; S3. Train the model and use the trained model for remote sensing image change detection.
2. The method for detecting changes in remote sensing images across scales by feature fusion according to claim 1, characterized in that, Pre-trained FastSAM with frozen weights was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images.
3. The method for detecting changes in remote sensing images across scales by feature fusion according to claim 1, characterized in that, The differential feature extraction module includes a row and column attention mechanism branch and a coordinate attention mechanism branch; The output features of row and column attention obtained from the two branches and the output features of coordinate attention are multiplied pixel by pixel to obtain the joint attention output features: in, This represents the joint attention output features of the two time phases. The output features representing row and column attention in two time phases. The coordinate attention output features represent the two temporal phases; Joint attention output features Two series-connected Extracting nonlinear relationships in convolution operations: in, This represents the characteristic representation under two time phases. , Indicates two consecutive Convolutional layers, where ReLU represents a non-linear activation function; Will and By adding each pixel individually, we obtain the difference feature map: in, This represents a difference feature map.
4. The method for detecting changes in remote sensing images across scales by feature fusion according to claim 3, characterized in that, The multi-scale feature map of the dual-temporal remote sensing image has four scales, and the fused features of the four different scales are as follows: in This indicates a convolution operation used to adjust the feature dimensions; This indicates a bilinear upsampling operation; This indicates a feature concatenation operation performed on two features; Indicates feature fusion, Two layers Convolutional structures are used to unify and integrate multi-scale information; This indicates a max-pooling downsampling operation; This indicates that a feature concatenation operation is performed on three features. , These are feature maps at different scales for two different time phases. This represents the feature map before the i-th scale change. This represents the feature map after the i-th scale change. , and These are fused feature maps at different scales from two different time phases.
5. The method for detecting changes in remote sensing images across scales by feature fusion according to claim 1, characterized in that, The differential feature map is initially optimized using separable convolutions with different dilation rates, specifically as follows: The difference feature map is divided into low-level difference features based on the number of channels and size. Differences between high-level and high-level characteristics Transform the high-level features: in, This represents the transformed high-level features. Indicates an upsampling operation. express Convolution operation, Represents the ReLU activation function. This indicates a batch normalization operation; The transformed high-level features are added element-wise to the low-level features to obtain preliminary fused features. : Preliminary fusion characteristics The channel dimension is divided into 4 subsets, which are then fed into convolutional branches with different dilation rates for processing. Let the first subset be... The subset is The corresponding void ratio is Each branch uses depthwise separable convolution for feature extraction, represented as: in, Indicates the void ratio of Dilated convolution operation, This represents the output of the k-th branch; The preliminary optimized difference feature map is F: in, This indicates that the outputs of the four branches are concatenated.
6. The method for detecting changes in remote sensing images across scales by feature fusion according to claim 5, characterized in that, The weighted optimization of the initially optimized differential feature map using the channel attention mechanism is as follows: in, Represents the weight vector. This indicates the average pooling operation. This indicates a max pooling operation. This represents the convolution operation. express Activation function This represents the Sigmoid function. This represents the difference feature map after weighted optimization.
7. The cross-scale feature fusion remote sensing image change detection system according to claim 1, characterized in that, include: The data processing module is used to acquire two-temporal remote sensing images of the same area at two different times and to preprocess the two-temporal remote sensing images. The model building module is used to build a remote sensing image change detection model, including a feature extraction module, a feature fusion module, a differential feature extraction module, an extended channel optimization module, and a decoder module; The feature extraction module was used to extract multi-scale feature maps from the preprocessed dual-temporal remote sensing images. The feature fusion module performs multi-scale feature fusion on the corresponding multi-scale feature maps of the two-temporal remote sensing images to obtain multi-scale fused feature maps of the two temporal phases. The multi-scale fused feature maps of the two time phases are added at the corresponding scales, and the difference feature map is obtained by the difference feature extraction module. The differential feature map is input into the dilation channel optimization module. The differential feature map is initially optimized by separable convolution with different dilation rates, and then the initially optimized differential feature map is weighted and optimized by combining the channel attention mechanism. The weighted and optimized difference feature map is input into the decoder module and gradually upsampled to the original image size to obtain the final change detection mask map; The remote sensing image change detection module is used to train the model, and then use the trained model for remote sensing image change detection.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.
9. An electronic device, characterized in that, The device includes a processor and a memory, the processor being interconnected with the memory, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to perform the method as described in any one of claims 1-6.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-6.