A remote sensing image change detection method of an ultra-lightweight twin multi-scale network

Abstract

The application discloses a kind of super-light twin multiscale network remote sensing image change detection methods, comprising: obtaining two remote sensing images of same area different time phases;Two images are input into feature extraction network based on twin architecture, the network includes two weight-sharing encoder branches, and obtains multi-level two-phase feature map by multi-level extraction;Difference is generated in each level of encoder to two-phase feature map, and generates multi-scale difference feature map;Decoder is constructed, and the difference map of the corresponding level of encoder is cascaded decoding by feature fusion module to last-stage decoding feature, and the resolution is recovered to obtain decoding feature gradually;Change probability map is generated based on decoding feature and output result.The application enhances feature expression by multi-scale hollow convolution and spatial-spectral feature correlation mechanism, realizes super-light by combining channel scaling factor, reduces parameter quantity and amount of calculation while ensuring high detection accuracy, and is suitable for edge device.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, and in particular relates to a method for detecting changes in remote sensing images using an ultra-lightweight twin multiscale network. Background Technology

[0002] Remote sensing image change detection is a technique that identifies changes in land cover or land cover by comparing and analyzing remote sensing images of the same area acquired at different times. This technology has significant applications in areas such as urban sprawl monitoring, land use change analysis, disaster assessment, ecological environment monitoring, and military reconnaissance.

[0003] Traditional remote sensing image change detection methods often rely on manually designed feature and threshold discrimination strategies, such as image differencing, ratio methods, change vector analysis, and statistical model-based methods. These methods are typically sensitive to changes in illumination, noise interference, and image registration errors, and have limited feature representation capabilities, making it difficult to obtain stable and accurate change detection results in complex scenes.

[0004] With the development of deep learning technology, change detection methods based on convolutional neural networks have gradually become a research hotspot. Existing methods typically employ dual-branch or Siamese network structures to extract features from remote sensing images at different time phases, and then fuse them at the feature layer or decision layer to achieve automatic detection of changed regions. These methods have significant advantages over traditional methods in terms of feature representation capability and detection accuracy.

[0005] However, existing deep learning-based change detection methods still have several shortcomings. First, in order to improve detection accuracy, some methods introduce complex network structures, such as multi-branch feature fusion modules, attention mechanisms, or self-attention Transformer structures, which significantly increase the number of model parameters and computational complexity, requiring high computing and storage resources, making them difficult to deploy on edge computing devices or in resource-constrained real-world application scenarios.

[0006] Secondly, some existing methods rely on large-scale convolutional kernels or deep network structures to expand the receptive field during feature extraction. While this improves the context modeling capability to some extent, it also increases computational overhead and easily introduces redundant features, which is not conducive to lightweight model design.

[0007] Furthermore, in terms of multi-scale feature fusion and change feature modeling, some existing methods employ complex feature interaction or attention calculation methods. Although these methods can enhance the expressive power of change regions, the implementation process is relatively complex, the inference efficiency is low, and there are still problems such as blurred change boundaries or missed detection in scenarios with small target changes or fine-grained changes.

[0008] Therefore, how to reduce the number of model parameters and computational complexity while ensuring the accuracy of change detection, and to construct a lightweight remote sensing image change detection method that is simple in structure, computationally efficient, and easy to deploy, remains a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes a remote sensing image change detection method using an ultra-lightweight twin multiscale network, thereby resolving the issues present in the prior art.

[0010] To achieve the above objectives, this invention provides a method for detecting changes in remote sensing images using an ultralight twin multiscale network, comprising: Acquire first-phase and second-phase remote sensing images of the same geographic area at different time points; The first and second temporal remote sensing images are respectively input into a feature extraction network constructed based on a Siamese network architecture. The feature extraction network contains two encoder branches with identical structures and shared weights. The first and second temporal feature maps at each level are obtained through multi-level feature extraction. At multiple levels of the encoder branch, the first temporal feature map and the second temporal feature map are respectively subjected to step-by-step differential calculation to generate multi-scale differential feature maps; A decoder is constructed, wherein the decoder performs concatenated decoding of the previous level decoded features and the multi-scale differential feature map of the corresponding level of the encoder through a feature fusion module, thereby restoring the spatial resolution of the feature map level by level to obtain the decoded features; A change probability map is generated based on the decoded features, and the change detection result is output based on the change probability map.

[0011] Optionally, during the mid-to-high-level feature extraction process, the encoder branch uses a multi-scale dilated convolution module to replace the standard convolutional layer for feature extraction, and introduces a channel scaling factor to globally constrain the number of feature channels in each layer, thereby reducing the number of parameters and computational cost of the network model.

[0012] Optionally, the processing procedure of the multi-scale dilated convolution module includes: Convolutional layers are used to reduce the number of channels in the input feature tensor to obtain dimensionality-reduced features; The dimensionality reduction features are respectively input into multiple convolutional branches set in parallel. The multiple convolutional branches contain dilated convolutions with different dilation rates, and respectively extract feature information under different receptive fields. The features output by the multiple convolutional branches are concatenated along the channel dimension to obtain multi-scale aggregated features. By utilizing the spatial-spectral feature correlation mechanism, the features are adaptively weighted and recombined based on the multi-scale aggregated features to obtain recombined features; The recombined features and the dimensionality reduction features are then fused together and output.

[0013] Optionally, the plurality of convolutional branches include at least a convolutional branch with an expansion rate of 1, a convolutional branch with an expansion rate of 3, and a convolutional branch with an expansion rate of 6.

[0014] Optionally, the adaptive weighted recombination of features using the spatial-spectral feature correlation mechanism includes: The multi-scale aggregated features are subjected to convolution transformation to generate a weight tensor; the weight tensor is then subjected to matrix multiplication or element-wise multiplication with the features to be weighted.

[0015] Optionally, the encoder branch employs a standard double convolution structure during shallow feature extraction, extracting the basic spatial structure features of the image through continuous convolution, batch normalization, and nonlinear activation operations.

[0016] Optionally, the step-by-step differential calculation process includes: at the end of each downsampling stage of the encoder, obtaining the first and second phase feature maps of that level; and calculating the differential feature map of that level by subtracting elements one by one or by differing the absolute values ​​of elements one by one.

[0017] Optionally, the processing procedure of the feature fusion module includes: The output features of the previous level decoder are upsampled to obtain upsampled features; the upsampled features are concatenated with the multi-scale difference feature map of the corresponding level of the encoder in the channel dimension to obtain concatenated features; the concatenated features are then convolved and output.

[0018] Optionally, the network model can be trained using a hybrid loss function, which is a weighted sum of weighted binary cross-entropy loss and Dice loss.

[0019] Compared with the prior art, the present invention has the following advantages and technical effects: (1) Strong feature consistency and high detection stability; By adopting a twin network structure with fully shared parameters, the pseudo-change noise introduced by different feature extractor parameters is fundamentally eliminated, ensuring strict alignment of features of dual-temporal images in the same feature space, and effectively improving the robustness of the change detection algorithm.

[0020] (2) The model is lightweight, efficient, and easy to deploy; This method creatively introduces a lightweight multi-scale convolutional structure combined with a channel scaling strategy. While maintaining a large receptive field to handle large-scale terrain variations, it significantly reduces the number of model parameters and floating-point operations (FLOPs). This makes the method particularly suitable for edge computing scenarios with limited computing resources, such as UAV embedded platforms and mobile terminals.

[0021] (3) Low computational complexity and fast reasoning speed; A variation feature modeling approach based on absolute value difference is adopted, replacing complex feature correlation calculations or attention mechanisms. This approach has a simple structure and good genotyping, and while ensuring the significance of differential features, it significantly reduces the computational burden and improves the real-time performance of image processing.

[0022] (4) Multi-scale fusion, accurate boundary positioning; By employing a step-by-step decoding and multi-scale variation feature fusion strategy, this approach effectively addresses the problem of the separation between high-level semantic information and low-level detailed information in traditional methods. This strategy not only improves the detection rate of targets changing at different scales but also significantly enhances the integrity and geometric accuracy of the boundaries of changed regions (such as building outlines and road edges). Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the overall architecture of USMSNet according to an embodiment of the present invention; Figure 2 This is a schematic diagram of multiscale decoupled convolution (MSDConv) according to an embodiment of the present invention; Figure 3 This is a comparison chart of change detection results for dual-temporal remote sensing images under different networks according to an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0026] Example 1 This embodiment provides a remote sensing image change detection method based on an ultra-lightweight Siamese multi-scale network. The method employs a Siamese network architecture and includes an encoding stage, a decoding stage, and a change feature modeling and output stage. The specific technical solution is as follows: (a) Input data and overall architecture; The input data consists of first-temporal and second-temporal remote sensing images of the same geographic area acquired at different times. Both input images have undergone spatial registration before being input into the network, ensuring they have consistent spatial resolution and image size.

[0027] The overall network is constructed based on a parameter-sharing Siamese architecture, comprising two structurally identical and weight-sharing feature extraction branches, used for parallel feature extraction of the first and second temporal remote sensing images, respectively. Through a weight-sharing mechanism, it is ensured that the two temporal images follow consistent transformation rules during feature space mapping, thereby guaranteeing the strict comparability of the extracted high-dimensional features and laying the foundation for subsequent calculation of variation differences.

[0028] (II) Encoding stage: Multi-level feature extraction; The encoding stage aims to extract abstract semantic features from the image, employing a multi-level convolutional feature extraction structure that progresses from shallow to deep: 1. Shallow feature extraction (texture and detail capture); To leverage the rich texture information and local geometric details in remote sensing images, a standard double convolutional structure is employed in the shallow layers of the network. Let the input image be... The shallow feature extraction process can be formally expressed as: ; in, Represents a two-dimensional convolution operation; and These represent the weight parameters of the first and second convolutional kernels, respectively. This indicates the batch normalization operation, used to accelerate network convergence; This represents a non-linear activation function (such as ReLU); This is the output shallow feature map.

[0029] This dual-convolutional structure effectively enhances nonlinear expressive power and extracts the basic spatial structural features of images by performing continuous convolution, normalization, and activation operations while maintaining low computational complexity.

[0030] 2. Mid-to-high level feature extraction (lightweight multi-scale perception); In the mid-to-high-level parts of the network, a lightweight multi-scale dilated convolution module is introduced to expand the receptive field while controlling the growth of parameters. Let the... The input features of the layer are The multi-scale feature extraction process is represented as follows: ; Or if it is a single-channel dilated convolution: ; In the above formula, This indicates a cascading and stitching operation of multiple multi-scale feature maps along the channel dimension; to These represent the kernel weights of the first to the kth parallel dilated convolution branches, respectively. to Then, they respectively refer to the dilation rate. to Similarly, the dilated convolution operation, Indicates the expansion rate Single-path dilated convolution operation, The corresponding convolution kernel parameters are given. By setting different predetermined dilation rates, this structure can capture contextual information over a larger spatial range without changing the convolution kernel size or reducing the feature map resolution, significantly enhancing the network's ability to perceive large-scale changes in land features (such as large buildings and farmland plots).

[0031] Furthermore, by introducing a unified channel scaling factor, a global constraint is imposed on the number of feature channels in each layer, allowing the overall size of the network model (FLOPs and number of parameters) to be flexibly adjusted according to actual hardware resource requirements, achieving an optimal balance between detection accuracy and inference speed.

[0032] 3. Downsampling operation; The spatial resolution of the feature maps is progressively reduced through downsampling operations at each encoding level to extract more abstract semantic features. The downsampling process is represented as follows: ; in, This indicates the Max Pooling operation.

[0033] (III) Decoding stage and feature fusion; The decoding stage aims to restore the spatial resolution of the feature map step by step, and to fuse the multi-scale feature information extracted in the encoding stage through skip connections to compensate for the loss of spatial details caused by downsampling.

[0034] 1. Upsampling and feature recovery; First, the high-level features in the decoding stage are upsampled. To avoid sign conflicts with the feature variables in the encoding stage, let the high-level features of the l-th layer in the decoding stage be... The upsampling results are as follows: in, This indicates a bilinear interpolation or deconvolution operation.

[0035] 2. Skip connections and feature fusion; Upsampled features Corresponding features at the same level as the encoding stage The process of fusion is as follows: ; in, This indicates a feature fusion operation (such as channel concatenation or element-wise addition), which is typically followed by a convolutional layer for feature integration. This step-by-step decoding structure enables the network to gradually refine the localization accuracy of edges in changing regions while restoring spatial resolution.

[0036] (iv) Modeling of change characteristics and multi-scale fusion; This invention abandons the complex attention mechanism and adopts an efficient differential feature modeling strategy to capture temporal change information.

[0037] 1. Absolute value difference modeling; In each decoding level, element-wise differencing is performed on the features of the same level from the first and second phase branches. Let the first phase feature at the same level be... The second phase is characterized by Characteristics of change The calculation is as follows: ; in This represents element-wise absolute value operation. This strategy utilizes distance metrics in the feature space to directly highlight the difference regions between two temporal images, while effectively suppressing redundant responses in the background (unchanged areas), resulting in extremely low computational overhead.

[0038] 2. Fusion of multi-scale variation features; To accommodate targets with varying sizes, a top-down, hierarchical feature fusion structure is adopted. The fusion process is represented as follows: ; in, This indicates a cross-scale fusion operation. This strategy combines high-level semantic change information with low-level detail change information, significantly improving the detection performance of small-scale changing targets (such as vehicles and small buildings) and changing boundary regions.

[0039] (v) Output of change results; The final change feature map is obtained after multi-scale fusion. This information is then input into the change prediction module. The prediction process is represented as follows: ; in, The convolution kernel parameters of the output layer (usually ) convolution), For the generated single-channel variation probability map, Sigmoid() is the Sigmoid activation function, which maps the output value to... Interval.

[0040] Finally, for the probability graph By performing threshold segmentation (such as Otsu or a fixed threshold of 0.5), a binary change mask can be obtained.

[0041] Example 2 This embodiment provides a method for detecting changes in remote sensing images using an ultralight twin multiscale network. The following detailed description uses the detection of changes in dual-temporal high-resolution remote sensing images as an example, in conjunction with the accompanying drawings and specific embodiments.

[0042] I. Implementation Environment and Data Preparation; The present invention can be implemented on a computer device equipped with a high-performance computing unit.

[0043] 1. Hardware environment: The processor is an Intel Xeon Gold or equivalent CPU, the graphics processor (GPU) is an NVIDIA Tesla V100 or RTX 3090 (with more than 24GB of video memory), and the memory is more than 64GB.

[0044] 2. Software environment: The operating system is Ubuntu 20.04 LTS, the deep learning framework is PyTorch 1.10 or above, the programming language is Python 3.8, and the parallel computing library is CUDA 11.3.

[0045] 3. Data preprocessing: Acquire two remote sensing images of the same geographic area taken at different times (referred to as the previous time phase image). and subsequent phase images ).

[0046] (1) Image cropping: Cropping large-format remote sensing images into fixed-size image patches using a sliding window, for example or Pixel.

[0047] (2) Data augmentation: In order to prevent overfitting, random flipping (horizontal, vertical), random rotation (90°, 180°, 270°), color jitter and other operations are performed on the training set.

[0048] (3) Normalization: Normalize the image pixel values ​​to 0. The interval is then standardized (subtract the mean and divide by the standard deviation).

[0049] II. Overall Network Structure Construction; like Figure 1 As shown, this invention proposes a change detection network for Siamese feature extraction and multi-level differential fusion. This network mainly consists of a Siamese encoder, a difference calculation module, a multi-scale decoder, and the core module of this invention, MSDConv (Multi-scale Dilated Convolution). MSDConv is as follows... Figure 2 As shown.

[0050] 1. Twin encoder branch; The network contains two identical encoder branches with shared weights, each responsible for processing... and Each encoder contains four downsampling stages, each consisting of a convolutional layer, a normalization layer, an activation function layer, and a max pooling layer.

[0051] set up The input image.

[0052] The encoder in the layer( The extracted feature maps are denoted as follows: and As the number of layers increases, the spatial resolution of the feature maps gradually decreases (respectively...). ), while the number of channels gradually increases.

[0053] 2. Differential and fusion mechanisms; This invention employs a unique "step-by-step difference and skip connection" architecture: (1) Deep difference: In the deepest layer of the encoder (layer 4), the features of the two branches are compared. and Perform element-wise subtraction. Figure 1 In ), to obtain deep differential features .

[0054] ; Alternatively, you can use direct subtraction: ; (2) Shallow interaction: such as Figure 1 As shown, not only in the deepest layers, but also in the intermediate layers, the network introduces encoder features into the decoder through skip connections. It is worth noting that, in the decoding stage, the input to each layer comes from the fusion of the upsampling result from the previous layer and the encoder features of the current layer.

[0055] 3. Decoder and Feature Fusion Module (F Module); The decoder is responsible for progressively restoring the low-resolution variations to the original image size. The decoder contains multiple feature fusion modules (...). Figure 1 (The circle marked F in the middle).

[0056] The specific structure of the F module (Fusion Block): like Figure 1 As shown in the legend in the lower right corner, module F contains three steps: (1) Upsample: The output feature map of the previous level decoder is bilinearly interpolated or deconvolved to increase its size by 2 times.

[0057] (2) Concatenation: The upsampled features are concatenated with the differential features or original features from the corresponding level of the encoder along the channel dimension.

[0058] ; In the above formula, This indicates an upsampling operation (such as bilinear interpolation or transposed convolution), which modifies the feature map of layer i+1. The spatial size is enlarged to keep its resolution consistent with the feature map of the i-th layer; This represents the corresponding features (difference features or original extracted features) from the i-th layer of the encoder that are directly introduced through skip connections. This means that the upsampled deep features are concatenated and stitched together with the corresponding layer features passed from the skip connections along the channel dimension. This operation can effectively integrate contextual information at different scales, providing richer details for subsequent feature recovery and prediction.

[0059] (3) Convolution: Perform convolution operation on the concatenated features to fuse information and adjust the number of channels.

[0060] 4. Core module: MSDConv; To address the problem that traditional convolutional receptive fields are fixed and it is difficult to capture targets with multi-scale changes, this invention embeds the MSDConv (Multi-Scale Dilated Convolution) module at key locations in network feature extraction (such as inside each block of the encoder or at the connection between the encoder and the decoder).

[0061] III. Specific implementation details of the MSDConv module; like Figure 2 As shown, the MSDConv module aims to enhance feature representation through Cyclic Multi-scale Convolution (CMC) and Spatial-Spectral Feature Correlation (SSFC) mechanisms. Assume the input feature tensor of this module is... ,in For the number of channels, This refers to the spatial dimensions.

[0062] The specific implementation steps of MSDConv are as follows: In the first stage, pointwise convolution is used to decouple and reduce the channel dimension of the input features, eliminating computational redundancy in the spatial dimension and generating intermediate features that only contain channel correlations. : ; in, Indicates the kernel size as Standard convolution, This represents batch normalization, and ReLU() represents the ReLU activation function. This step compresses the number of feature channels, reducing the computational load for subsequent multi-scale processing.

[0063] In the second stage, a recurrent multi-scale strategy is introduced to expand spatial correlation. Unlike parallel use of multiple convolution kernels, MSDConv performs spatial correlation expansion within the same layer. Perform grouped recurrent atrous convolution. Set a set of recurrent dilation rates. (For example, in this embodiment, the following is used) These correspond to capturing local details, medium-scale context, and large-scale semantic information, respectively. Features at each scale The calculation is as follows: ; in, Indicates the expansion rate of Depthwise convolution. This operation flexibly expands the effective receptive field by adjusting the dilation rate while keeping the feature map resolution constant.

[0064] Multiple convolutional branches must include at least three paths: The first branch uses a standard 3×3 convolution with a dilation rate of 1 to extract local detail features; The second branch uses a 3×3 dilated convolution with a dilation rate of 3 to extract mesoscale features; The third branch employs a 3×3 dilated convolution with a dilation rate of 6 to extract a wide range of semantic context features.

[0065] In the third stage, spatial features at different scales are stitched together and fused to restore channel information and output the final multi-scale features. : ; Through the above decoupling and recombination process, MSDConv simultaneously completes feature dimensionality reduction, multi-scale context capture, and feature aggregation within a single module. This avoids the parameter redundancy of standard large kernel convolution and solves the problem of poor adaptability of single-scale convolution to complex terrain features.

[0066] Phase 4: Spatial-Spectral Feature Correlation Modeling (SSFC); Figure 2 The right side shows the SSFC (Spatial-Spectral Feature Correlation) section. The purpose of this section is to utilize multi-scale features. As a guide, the original input features (or the transformed features) are weighted or enhanced.

[0067] The correlation modeling steps specifically include: performing convolution transformation on the multi-scale aggregated features to generate a weight tensor; and performing matrix multiplication or element-wise multiplication on the weight tensor and the features to be weighted to achieve feature recalibration in the spatial or channel dimensions.

[0068] (1) Feature input: SSFC receives two inputs: one from the output of "recurrent multi-scale convolution". Another bypass path originates from the module's initial input ( Figure 2 The arrow above usually indicates that... or Another part).

[0069] (2) Correlation calculation and matrix multiplication: like Figure 2 As shown, SSFC utilizes Generate spatial or channel attention maps. Specifically, extract key features through a convolution operation, and then perform matrix multiplication with features from another channel. ).

[0070] Let the other feature input to SSFC be... .

[0071] SSFC first Perform the transformation to generate the weight tensor. Then perform matrix multiplication: ; Detailed derivation Figure 2 Matrix multiplication logic in: The image shows a small cube (representing a convolution kernel or local feature block) from... The extracted features are applied to the feature stack on the right. This effectively achieves an adaptive feature recalibration.

[0072] Mathematically, this can be represented as a variant of channel attention or spatial attention. (Based on the illustration...) The symbol represents a Hadamard product (element-wise multiplication) or matrix multiplication. If interpreted as spatial attention: ; ; in This indicates element-wise multiplication.

[0073] Fifth stage: Final feature fusion; The final output of the MSDConv module is to fuse the features processed by SSFC with the original features (or multi-scale features). For example... Figure 2 As shown on the far right, the output is restored to... aisle.

[0074] This is usually achieved through concatenation: ; Or through Convolution adjusts the number of channels back . Figure 2 The right parenthesis indicates that the final output dimension is .

[0075] IV. Detailed Explanation of Change Detection Process (Logical Steps); Based on the above network structure, the complete change detection process of this invention is as follows: Step 1: Feature extraction; Will and Input the twin encoders separately.

[0076] In each layer of the encoder Input features After processing by the MSDConv module (which replaces ordinary convolution), the features are obtained. .

[0077] ; in, This describes the composite feature extraction workflow for this level. Specifically, the input features first pass through the MSDConv module (replacing standard convolution in the mid-to-high levels) to capture multi-scale spatial context information; then, they are sequentially processed through batch normalization to accelerate convergence, and non-linear activation functions (such as ReLU) to enhance non-linear representation. Finally, at the end of each downsampling stage, downsampling is performed through max pooling or stride convolution, thereby further expanding the receptive field while reducing the spatial resolution of the feature map.

[0078] Step 2: Calculation of multi-level differential features; Unlike differential mapping only in the last layer, this embodiment calculates differential feature maps at the end of each stage of the encoder in order to capture the details of changes at different resolutions.

[0079] For the l layer: ; The stepwise difference calculation is as follows: At the end of each downsampling stage of the encoder, the first temporal feature map of that level is obtained. Second phase feature map The difference feature map of this level is calculated by subtracting elements one by one or by differing the absolute values ​​of elements one by one. The calculation formula is: .

[0080] Step 3: Decoding and fusion (Cascade Decoding); The decoder starts working from the deepest layer (layer 4).

[0081] (1) Initial input: take As the initial input to the decoder.

[0082] (2) Iterative fusion: For the decoder's... Layer (corresponding to the encoder's first layer) layer): Output of the previous decoding layer Perform upsampling.

[0083] Using module F ( Figure 1 As shown in the figure, the upsampling result is compared with the difference features of the current layer. Then, the parts are assembled.

[0084] Figure 1 The flow direction shown in the image is from deep to shallow layers, with the middle layer... The resulting difference map is fed into the corresponding decoding layer via skip connections.

[0085] Step 4: Generate a change graph; The size of the last layer output feature map of the decoder is the same as that of the original. Figure 1 To (or 1 / 2, 1 / 4 of the original image), after a Convolutional layers compress the number of channels to 1 (for binary classification problems, change / no change).

[0086] ; Finally, the output is mapped to a Sigmoid activation function. The interval is used to obtain the probability map P of the change.

[0087] ; By setting a threshold (e.g., 0.5), Binarization yields the final change mask.

[0088] V. Loss Function and Model Training; To train the network, this invention employs a hybrid loss function, aiming to address the common problem of extreme imbalance between positive and negative samples (changing pixels and unchanging pixels) in change detection.

[0089] Total loss function Defined as the weighted sum of the weighted binary cross-entropy loss (BCE Loss) and the Dice loss (Dice Loss): ; 1. Weighted binary cross-entropy loss ( ): ; in, For real labels, To predict probabilities, These are the weights for positive samples, used to penalize missed detections.

[0090] 2. Dice loss ( ): Dice loss focuses on the degree of overlap between the predicted and real regions and is robust to imbalanced samples.

[0091] ; in, For smoothing terms (e.g.) ( ), to prevent the denominator from being zero.

[0092] 3. Optimization strategy: Use the AdamW optimizer; initial learning rate set to... The learning rate is dynamically adjusted using a cosine annealing strategy; the number of training epochs is set to 100-200.

[0093] This invention constructs a remote sensing image change detection framework based on a twin structure, enabling alignment and comparison of images from different time phases within the same feature space, reducing feature shifts caused by temporal differences. By introducing lightweight multi-scale feature extraction structures at different network levels, the receptive field of features is expanded without significantly increasing computational overhead, thereby enhancing the ability to represent complex ground feature changes. A simple and computationally efficient change feature modeling method is adopted to perform differential modeling of features from two time phases, avoiding the additional computational burden caused by complex attention mechanisms. Through a step-by-step decoding and multi-scale change feature fusion strategy, the detection accuracy of changed areas at different spatial scales is improved, especially enhancing the ability to identify small-scale changed targets and boundary regions. Overall, a lightweight change detection model with adjustable parameters, clear structure, and high inference efficiency is realized to meet the practical needs of remote sensing change detection in terms of real-time performance and engineering deployment.

[0094] VI. Experimental Results and Analysis; (I) Comparative Experiment; To verify the effectiveness of the proposed ultra-lightweight twin multi-scale network (USMSNet) in remote sensing image change detection, this embodiment conducts a quantitative comparative experiment with several mainstream change detection methods. The comparison methods include early methods based on fully convolution (FC-EF, FC-Siam-Conc, FC-Siam-Diff), the deep feature interaction-based method DSIFN / IFN, the Transformer-based method BiT, and the equally lightweight model USSFCNet.

[0095] From an overall performance perspective, the proposed USMSNet achieves optimal results in both the F1 score and IoU, key performance indicators, reaching 91.01% and 83.50% respectively, significantly outperforming other comparative methods. Specifically, compared to the BiT method based on the Transformer architecture, USMSNet improves the IoU by 2.82% while maintaining the advantages of a lightweight convolutional architecture. This data demonstrates that the invention, through its carefully designed multi-scale convolutional modules, can capture local details and global semantics in remote sensing images more efficiently than the parameter-intensive Transformer structure. Furthermore, compared to the high-precision DSFN method, although DSFN has a high precision (94.02%), its recall is only 82.93%, exhibiting a significant false negative rate. USMSNet, however, significantly improves the recall to 90.40%, greatly reducing the false negative rate while maintaining high precision, indicating that the model is more sensitive and complete in capturing changing regions.

[0096] Of particular note is the in-depth comparative analysis between this invention and the second-best performing USSFCNet. Although both employ feature correlation modeling, USMSNet achieves a significant performance leap, with an IoU score 2.91% higher than USSFCNet (83.50% vs. 80.59%) and an F1 score improvement of 1.76%. This substantial difference is primarily attributed to the MSDConv module introduced in this invention. While USSFCNet utilizes feature interactions, its receptive field is limited by a single scale, making it less adaptable to targets with vastly different sizes (such as large areas of farmland coexisting with narrow roads). In contrast, USMSNet explicitly constructs a multi-scale perception mechanism through multiple dilated convolutions with varying dilation rates. This allows it to cover the overall structure of large-scale features while preserving the edge details of small targets, resulting in a higher geometric overlap between the generated detection results and the ground truth.

[0097] Furthermore, from the balance analysis of precision and recall, all metrics of USSFCNet hovered around 89%, while USMSNet achieved improvements in both directions. This model not only improved the recall to 90.40%, but also maintained a high precision of 91.62%. This indicates that the SSFC spatial-spectral feature interaction structure integrated in this invention can more effectively utilize contextual information to suppress background noise (improving precision), while, combined with the Siamese difference fusion strategy, it can better recover weak change signals (improving recall). In summary, the experimental data fully demonstrates that USMSNet, by fusing lightweight multi-scale convolution and feature interaction mechanisms, effectively overcomes the limitations of existing methods in multi-scale object detection, significantly improving the integrity of target edges while ensuring high detection accuracy. A comparative analysis of different change detection models is shown in Table 1.

[0098] Table 1 (II) Model Efficiency Analysis; Table 2 Table 2 presents the quantitative evaluation results of the proposed method and the comparative methods in terms of model parameter count (params) and computational complexity (FLOPs), with the input image size uniformly set to 256×256. From the perspective of parameter size, USMSNet exhibits a remarkably significant lightweight advantage, with only 0.87M model parameters, making it the only model among all the compared methods with fewer than 1M parameters. Compared to the computationally intensive DSIFN (50.44M), the parameter count of this model is reduced by nearly two orders of magnitude; even compared to the similar lightweight network USSFCNet (1.52M), this invention, thanks to its efficient parameter sharing mechanism and channel scaling strategy, successfully reduces the parameter count by approximately 42.8%. This extreme compression rate greatly saves model storage space, making it highly suitable for deployment on edge devices with limited storage resources, such as drones and handheld terminals.

[0099] In terms of computational complexity, this invention also maintains a strong competitive advantage. USMSNet's floating-point operations (FLOPs) are 6.29 G, slightly higher than the simplest fully convolutional baseline model (FC-EF) and USSFCNet, but significantly lower than the Transformer-based BiT method (8.75 G) and the heavyweight network DSIFN (82.26 G). It should be noted that, combined with the aforementioned accuracy evaluation results, USMSNet improves the IoU metric by 2.91% compared to USSFCNet, at the cost of only a slight increase in computational cost. This indicates that the increased computational overhead through the dual decoder model yields a very high performance benefit. Overall, USMSNet achieves the best balance between detection accuracy and computational efficiency while maintaining an extremely low parameter scale, fully validating its engineering practical value as an "ultra-lightweight" network.

[0100] (III) Visual Analysis; Figure 3 This paper visually compares the qualitative detection performance of the proposed method (USMSNet) with various mainstream models in typical remote sensing scenarios, where white, red, and green indicate correctly detected, falsely detected, and missed detection areas, respectively. When processing dense and fragmented targets shown in the first and second rows, this invention significantly eliminates the green missed detection phenomenon commonly found in the FC series and USSFCNet, effectively restoring the fine structure of ground features. For large-scale building targets in the fourth row, USMSNet, with its multi-scale perception capabilities, generates white areas with clear outlines and complete internal filling, overcoming the structural fragmentation problems generated by other models at the target center and edges. In the complex terrain environment of the fifth row, compared to the comparative method with scattered red false detection noise, the background of this invention is cleaner, demonstrating the superior performance of the feature interaction mechanism in suppressing pseudo-change interference. This visually confirms the core advantages of this invention in improving detection accuracy and boundary integrity.

[0101] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting changes in remote sensing images using an ultralightweight twin multiscale network, characterized in that, Includes the following steps: Acquire first-phase and second-phase remote sensing images of the same geographic area at different time points; The first and second temporal remote sensing images are respectively input into a feature extraction network constructed based on a Siamese network architecture. The feature extraction network contains two encoder branches with identical structures and shared weights. The first and second temporal feature maps at each level are obtained through multi-level feature extraction. At multiple levels of the encoder branch, the first temporal feature map and the second temporal feature map are respectively subjected to step-by-step differential calculation to generate multi-scale differential feature maps; A decoder is constructed, wherein the decoder performs concatenated decoding of the previous level decoded features and the multi-scale differential feature map of the corresponding level of the encoder through a feature fusion module, thereby restoring the spatial resolution of the feature map level by level to obtain the decoded features; A change probability map is generated based on the decoded features, and the change detection result is output based on the change probability map.

2. The remote sensing image change detection method using ultralightweight twin multiscale networks according to claim 1, characterized in that, In the process of mid-to-high-level feature extraction, the encoder branch uses a multi-scale dilated convolution module to replace the standard convolutional layer for feature extraction, and introduces a channel scaling factor to globally constrain the number of feature channels in each layer, thereby reducing the number of parameters and computational cost of the network model.

3. The remote sensing image change detection method using ultralight twin multiscale networks according to claim 2, characterized in that, The processing steps of the multi-scale dilated convolution module include: Convolutional layers are used to reduce the number of channels in the input feature tensor to obtain dimensionality-reduced features; The dimensionality reduction features are respectively input into multiple convolutional branches set in parallel. The multiple convolutional branches contain dilated convolutions with different dilation rates, and respectively extract feature information under different receptive fields. The features output by the multiple convolutional branches are concatenated along the channel dimension to obtain multi-scale aggregated features. By utilizing the spatial-spectral feature correlation mechanism, the features are adaptively weighted and recombined based on the multi-scale aggregated features to obtain recombined features; The recombined features and the dimensionality reduction features are then fused together and output.

4. The remote sensing image change detection method using an ultralight twin multiscale network according to claim 3, characterized in that, The plurality of convolutional branches include at least a convolutional branch with an expansion rate of 1, a convolutional branch with an expansion rate of 3, and a convolutional branch with an expansion rate of 6.

5. The remote sensing image change detection method using ultralight twin multiscale networks according to claim 3, characterized in that, The adaptive weighted recombination of features using a spatial-spectral feature correlation mechanism includes: The multi-scale aggregated features are subjected to convolution transformation to generate a weight tensor; the weight tensor is then subjected to matrix multiplication or element-wise multiplication with the features to be weighted.

6. The remote sensing image change detection method using an ultralight twin multiscale network according to claim 3, characterized in that, The encoder branch employs a standard double convolution structure during shallow feature extraction, extracting the basic spatial structure features of the image through continuous convolution, batch normalization, and nonlinear activation operations.

7. The remote sensing image change detection method using an ultralight twin multiscale network according to claim 1, characterized in that, The step-by-step differential calculation process includes: at the end of each downsampling stage of the encoder, obtaining the first and second phase feature maps of that level; and calculating the differential feature map of that level by subtracting elements one by one or by differing the absolute values ​​of elements one by one.

8. The remote sensing image change detection method using an ultralight twin multiscale network according to claim 1, characterized in that, The processing procedure of the feature fusion module includes: The output features of the previous level decoder are upsampled to obtain upsampled features; the upsampled features are concatenated with the multi-scale difference feature map of the corresponding level of the encoder in the channel dimension to obtain concatenated features; the concatenated features are then convolved and output.

9. The remote sensing image change detection method using an ultralight twin multiscale network according to claim 1, characterized in that, The network model is trained using a hybrid loss function, which is a weighted sum of weighted binary cross-entropy loss and Dice loss.

Images